Announcement

Collapse
No announcement yet.

Automatic import from a file

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Automatic import from a file

    Good afternoon, dear participants of the forum. I need to configure automatic import of contacts from a file. I followed this instruction :


    But there were no changes to ESPOCRM. Please help me set up automatic import from a file .

  • #2
    Maybe you want to paste your code and someone might be able to spot the mistake in the code.

    This of course assume you didn't fail these task (1) enable CRON is running and (2) file is in correct location.

    Comment


    • Vok
      Vok commented
      Editing a comment
      Thank you very much for your response. I understand how to connect CRON. I don't understand how to set up automatic import from a file. I would be very grateful if you would give me an example of the code and where to insert it. I'm a newbie, so I'm sorry for the stupid questions.

  • #3
    Hi Vok, unfortunately that is outside of my skill level. You said, "I followed this instruction" so I thought you have a working (draft) code that does the importing. All I'm trying to say is that if you copy/paste your code, someone might be able to see a mistake or provide tips/hints.

    Comment


    • #4
      Hello,
      "automatic import of contacts from a file"

      what kind of file is ? where is the file ? how is data in ?

      i do somethink like this..
      a first custom job :
      check if file exist on ftp server every hours
      if file exist .. download file and put to a custom folder

      a second custom job :
      check if file exist in custom folder.
      if file exist ... read file (csv) .. and "import"

      so i do the import in 2 jobs
      every month .. more than 150000 record are created so.

      You need, check duplicate and so ..

      Comment


      • #5
        Originally posted by item View Post
        Hello,
        "automatic import of contacts from a file"

        what kind of file is ? where is the file ? how is data in ?

        i do somethink like this..
        a first custom job :
        check if file exist on ftp server every hours
        if file exist .. download file and put to a custom folder

        a second custom job :
        check if file exist in custom folder.
        if file exist ... read file (csv) .. and "import"

        so i do the import in 2 jobs
        every month .. more than 150000 record are created so.

        You need, check duplicate and so ..
        Thank you very much for your response. You've written all the right things for me. But I can't figure out what to write after "public function run ()." in YourJobName.php

        How to specify that you need to import this particular file . I would be very grateful if you could share a sample or help me write the code

        Comment


        • #6
          Here is my code in the task file :
          <?php

          namespace Espo\Custom\Jobs;

          class YourJobName extends \Espo\Core\Jobs\Base
          {

          public function run()
          {
          // all the logic needs to be defined in the method run
          $job = $this->getEntityManager()->getEntity('Job');
          $job->set(array(
          'serviceName' => $serviceName,
          'methodName' => $methodName,
          'data' => array('someVar' => 'someValue'),
          'executeTime' => date('Y-m-d H:i:s'),
          ));
          $this->getEntityManager()->saveEntity($job);
          }
          }

          But this code is created when I create a manual import :
          {
          "entityType": "Opportunity",
          "params": {
          "headerRow": true,
          "delimiter": ";",
          "textQualifier": """,
          "dateFormat": "YYYY-MM-DD",
          "timeFormat": "HH:mm",
          "timezone": "UTC",
          "personNameFormat": "f l",
          "decimalMark": ".",
          "currency": "RUB",
          "defaultValues": {},
          "action": "createAndUpdate",
          "skipDuplicateChecking": false,
          "idleMode": false,
          "silentMode": true,
          "updateBy": [
          8
          ]
          },
          "attachmentId": "5f2d1c98eccd50af8",
          "importAttributeList": [
          "dataT",
          "accountId",
          "accountName",
          "name",
          "kol",
          "amountCurrency",
          "amount",
          "stage",
          "accountFiscalNumber"
          ],
          "importId": "5f2d1c992e125ab06",
          "userId": "1"
          }

          Comment


          • #7
            We don't use Jobs in our application, but perhaps looking at a built in job script can help https://github.com/espocrm/espocrm/b...ifications.php

            Comment


            • #8
              Hello,

              here my sample .. i have don't write all because i don't know your file :


              PHP Code:
              <?php

              namespace Espo\Custom\Jobs;
              use 
              Espo\ORM\Entity;

              class 
              CareJobs extends \Espo\Core\Jobs\Base
              {
              public function 
              run()
              {
              ini_set('memory_limit','16M');
              $fileName 'data/xxxx/care.csv';
              if(!
              file_exists($fileName) || !is_readable($fileName))
              {
              $GLOBALS['log']->info"CareJobs Nothing to process");
              return 
              false;
              }
              $GLOBALS['log']->warning"Begin Jobs CareJobs");
              if ((
              $handle fopen$fileName"r")) !== false) {
              $entityManager $this->getEntityManager();
              $time_start microtime(true);


              $header fgetcsv($handle);
              while ((
              $data fgetcsv($handle2000",")) !== false) {
              $practriceArray explode("_"$data[0] );
              $practrice $practriceArray[1];
              ...
              this jobs import more than 150000 record each month
              as you can see.. another jobs check ftp for file.csv...if file exist download and put in folder of espocrm

              Comment


              • #9
                Originally posted by item View Post
                Hello,

                here my sample .. i have don't write all because i don't know your file :


                PHP Code:
                <?php

                namespace Espo\Custom\Jobs;
                use 
                Espo\ORM\Entity;

                class 
                CareJobs extends \Espo\Core\Jobs\Base
                {
                public function 
                run()
                {
                ini_set('memory_limit','16M');
                $fileName 'data/xxxx/care.csv';
                if(!
                file_exists($fileName) || !is_readable($fileName))
                {
                $GLOBALS['log']->info"CareJobs Nothing to process");
                return 
                false;
                }
                $GLOBALS['log']->warning"Begin Jobs CareJobs");
                if ((
                $handle fopen$fileName"r")) !== false) {
                $entityManager $this->getEntityManager();
                $time_start microtime(true);


                $header fgetcsv($handle);
                while ((
                $data fgetcsv($handle2000",")) !== false) {
                $practriceArray explode("_"$data[0] );
                $practrice $practriceArray[1];
                ...
                this jobs import more than 150000 record each month
                as you can see.. another jobs check ftp for file.csv...if file exist download and put in folder of espocrm
                Thank you so much for taking an interest in my question. I am very grateful to you!!!

                Comment


                • #10
                  Hello,

                  this jobs download the csv file.. put in folder .. and create the other jobs who inject csv to espoCRM

                  PHP Code:
                  <?php

                  namespace Espo\Custom\Jobs;
                  use 
                  Espo\ORM\Entity;
                  use 
                  phpseclib\Net\SFTP;

                  class 
                  SftpJobs extends \Espo\Core\Jobs\Base
                  {

                  public function 
                  run()
                  {
                  $GLOBALS['log']->warning"Start SftpJobs : " );

                  $path "/home/path/CSV/";
                  $fileDate date('Y-n'strtotime(date('Y-m')." -1 month"));
                  $fileName "xxx-" .$fileDate .".csv";

                  $file $path .$fileName;

                  $sftp = new SFTP('10.10.10.10'2121 );
                  $sftp_login $sftp->login('login''password');
                  if(
                  $sftp_login) {

                  if( 
                  $sftp->file_exists$file)) {
                  $GLOBALS['log']->warning'SftpJobs : File Exist' );
                  $sftp->get($file'data/path/care.csv');
                  $sftp->rename($file'/home/path/CSV/Backup/' .$fileName);
                  $GLOBALS['log']->warning'SftpJobs : File Downloaded & Moved To Backup : ' .$fileName );
                  $entityManager $this->getEntityManager();
                  $job $entityManager->getEntity('Job');
                  $job->set([
                  'name' => 'Care Jobs',
                  'status' => 'Pending',
                  'serviceName' => 'CareJobs',
                  'methodName' => 'run',
                  'data' => NULL// array('' => ''),
                  'executeTime' => date('Y-m-d H:i:s',strtotime(date("Y-m-d H:i:s")." +5 minutes")),
                  'scheduledJobId' => '5c7e592209b492e20',
                  ]);
                  $entityManager->saveEntity($job);
                  }else{
                  $GLOBALS['log']->warning'SftpJobs : File Dont Exist');
                  }
                  }else{
                  $GLOBALS['log']->warning'SftpJobs : Cannot login into your server !');
                  }

                  $GLOBALS['log']->warning"End SftpJobs : " );
                  return 
                  true;

                  }
                  }

                  Comment


                  • #11
                    Originally posted by item View Post
                    Hello,

                    this jobs download the csv file.. put in folder .. and create the other jobs who inject csv to espoCRM

                    PHP Code:
                    <?php

                    namespace Espo\Custom\Jobs;
                    use 
                    Espo\ORM\Entity;
                    use 
                    phpseclib\Net\SFTP;

                    class 
                    SftpJobs extends \Espo\Core\Jobs\Base
                    {

                    public function 
                    run()
                    {
                    $GLOBALS['log']->warning"Start SftpJobs : " );

                    $path "/home/path/CSV/";
                    $fileDate date('Y-n'strtotime(date('Y-m')." -1 month"));
                    $fileName "xxx-" .$fileDate .".csv";

                    $file $path .$fileName;

                    $sftp = new SFTP('10.10.10.10'2121 );
                    $sftp_login $sftp->login('login''password');
                    if(
                    $sftp_login) {

                    if( 
                    $sftp->file_exists$file)) {
                    $GLOBALS['log']->warning'SftpJobs : File Exist' );
                    $sftp->get($file'data/path/care.csv');
                    $sftp->rename($file'/home/path/CSV/Backup/' .$fileName);
                    $GLOBALS['log']->warning'SftpJobs : File Downloaded & Moved To Backup : ' .$fileName );
                    $entityManager $this->getEntityManager();
                    $job $entityManager->getEntity('Job');
                    $job->set([
                    'name' => 'Care Jobs',
                    'status' => 'Pending',
                    'serviceName' => 'CareJobs',
                    'methodName' => 'run',
                    'data' => NULL// array('' => ''),
                    'executeTime' => date('Y-m-d H:i:s',strtotime(date("Y-m-d H:i:s")." +5 minutes")),
                    'scheduledJobId' => '5c7e592209b492e20',
                    ]);
                    $entityManager->saveEntity($job);
                    }else{
                    $GLOBALS['log']->warning'SftpJobs : File Dont Exist');
                    }
                    }else{
                    $GLOBALS['log']->warning'SftpJobs : Cannot login into your server !');
                    }

                    $GLOBALS['log']->warning"End SftpJobs : " );
                    return 
                    true;

                    }
                    }
                    Thanks a lot for your code and interest in my problem !! There is one last question left. Where do I need to indicate that the column from the CSV file corresponds to the column in ESPOCRM /
                    Example:
                    in the CSV file, the third column is the client's name. It is necessary that when importing, ESPOCRM understands that the third column is clients.

                    "importAttributeList": [
                    "dataT",
                    "accountId",
                    "accountName",
                    "name",
                    "kol",
                    "amountCurrency",
                    "amount",
                    "stage",
                    "accountFiscalNumber"
                    ],

                    Comment


                    • #12
                      Hello,

                      i use a csv from another provider .. who put in ftp ... i have give my job who check existence of the file .. if exist.. download to espocrm folder.. then make a new job

                      the job is just read csv file .. and make the "inject"..
                      yes i need to match first column to .. .second column to ...

                      all depend how are you csv

                      PHP Code:
                      $header fgetcsv($handle);
                      while ((
                      $data fgetcsv($handle2000",")) !== false) {
                      //$data = mb_convert_encoding($data, 'ISO-8859-1','ISO-8859-1');
                      $practriceArray explode("_"$data[0] );
                      $practrice $practriceArray[1];

                      $inamiTo substr($inamiNumberTo08);
                      // ACCOUNT
                      $account $entityManager->getRepository('Account')->where([ 'practrice' => $practrice ])->findOne();
                      if (!
                      $account){
                      $account $entityManager->getEntity('Account');
                      $account->set([
                      'name' => $practrice,
                      'practrice' => $practrice,
                      'teamsIds' => $teamsIds,
                      'assignedUserId' => '1'
                      ]);
                      $entityManager->saveEntity($account);
                      }
                      //else{
                      // foreach ($account->get('teams') as $team) {
                      // $teamsIds[] = $team->id;
                      // }
                      //}
                      // CONTACT
                      $contact $entityManager->getRepository('Contact')->where([ 'inami' => $inami ])->findOne();
                      if (!
                      $contact){
                      $contact $entityManager->getEntity('Contact');
                      $contact->set([
                      'firstName' => ucwords(mb_strtolower($data[16])),
                      'lastName' => mb_strtoupper$data[15]) ,
                      'inamiNumber' => $inamiNumber,
                      'practrice' => $practrice,
                      'dateFirstCare' => $dt->format('Y-m-d'),
                      'teamsIds' => $teamsIds,
                      'assignedUserId' => '1'
                      ]);
                      $entityManager->saveEntity($contact);
                      }

                      // PATIENT
                      $patient $entityManager->getRepository('Patient')->where([ 'nationalNumber' => $nationalNumber ])->findOne();
                      if (!
                      $patient){
                      $patient $entityManager->getEntity('Patient');
                      $patient->set([
                      'firstName' => ucwords(mb_strtolower($data[2] )),
                      'lastName' => mb_strtoupper($data[1] ),
                      'addressStreet' => $addressStreet,
                      'addressPostalCode' => $addressPostalCode,
                      'addressCity' => $addressCity,
                      'nationalNumber' => $nationalNumber,
                      'cg1' => $cg1,
                      'cg2' => $cg2,
                      'patHealth' => $patHealth,
                      'score' => $score,
                      'dateFirstCare' => $dt->format('Y-m-d'),
                      'teamsIds' => $teamsIds,
                      'assignedUserId' => '1'
                      ]);
                      $entityManager->saveEntity($patient);
                      }
                      // CARE
                      $care $entityManager->getEntity('Care');
                      $care->set(array(
                      'name' => $patient->get('name'),
                      'practrice' => $practrice,
                      'dateTime' => $dateTime,
                      ));
                      $entityManager->saveEntity($care);

                      $convention $entityManager->getRepository('Convention')->where([ 'name' => $year .$month ])->findOne();
                      if (!
                      $convention){
                      $convention $entityManager->getEntity('Convention');
                      $convention->set([
                      'name' => $year .$month,
                      'dateStart' => date('Y-m-01'strtotime($care->get('dateTime'))),
                      'dateEnd' => date('Y-m-t'strtotime($care->get('dateTime'))),
                      'teamsIds' => $teamsIds,
                      'assignedUserId' => '1',
                      ]);
                      $entityManager->saveEntity($convention);
                      }

                      if (!
                      $entityManager->getRepository('Convention')->isRelated($convention'contacts'$contact)){
                      $entityManager->getRepository('Convention')->relate($convention'contacts'$contact);
                      }


                      }
                      fclose($handle);

                      Comment


                      • #13
                        Thanks again for the code! There will be something to keep yourself busy on the day

                        Comment

                        Working...
                        X