Slow Import

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Ashif Malayil
    Senior Member
    • Dec 2023
    • 176

    Slow Import

    I have done a import around 2700+ data was there on the sheet with 97 columns, the file size was 5mb. But it took around 4 hour to complete the import. The import executed in idle mod. Skip duplicates and silent mode has not checked. Is there any way to fix this issue?
    Last edited by Ashif Malayil; 05-10-2024, 09:59 AM.
  • eymen-elkum
    Active Community Member
    • Nov 2014
    • 472

    #2
    Enabling the skip duplicate will make it faster, silent mode also make it faster, especially if the table has large number of records
    CEO of Eblasoft
    EspoCRM Expert since 2014
    Full Stack Web Developer since 2008
    Creator of Numerous Successful Extensions & Projects​

    Comment

    • victor
      Active Community Member
      • Aug 2022
      • 729

      #3
      Originally posted by Ashif Malayil
      I have done a import around 2700+ data was there on the sheet with 97 columns, the file size was 5mb. But it took around 4 hour to complete the import. The import executed in idle mod. Skip duplicates and silent mode has not checked. Is there any way to fix this issue?
      For example, an import file of 18,000 lines on a very powerful server without the Skip searching for duplicates option but wiht Silent mode option enabled takes about 10 minutes (screenshot 1).​

      So, most likely, your server is simply too weak. You can enable Skip searching for duplicates and Silent mode and try to import through the UI, or better through the CLI.​
      Attached Files
      Last edited by victor; 05-10-2024, 10:37 AM.

      Comment

      • yuri
        Member
        • Mar 2014
        • 8454

        #4
        Adding necessary indexes may held if search for duplicates performs long. Also maybe consider upgrading server's hardware.
        If you find EspoCRM good, we would greatly appreciate if you could give the project a star on GitHub. We believe our work truly deserves more recognition. Thanks.

        Comment

        • Ashif Malayil
          Senior Member
          • Dec 2023
          • 176

          #5
          4 Core 16 GB RAM 250 GB SSD server? is this two weak?

          Comment

          • yuri
            Member
            • Mar 2014
            • 8454

            #6
            Server is not weak. The text is not boldenough.
            If you find EspoCRM good, we would greatly appreciate if you could give the project a star on GitHub. We believe our work truly deserves more recognition. Thanks.

            Comment

            • yuri
              Member
              • Mar 2014
              • 8454

              #7
              Try to figure out what exactly is slow. Is it fast if you skip duplicates. Is it fast in silent mode.

              How many records are already in the database table.

              Database tweaking is also may be reasonable if you have big tables.
              If you find EspoCRM good, we would greatly appreciate if you could give the project a star on GitHub. We believe our work truly deserves more recognition. Thanks.

              Comment

              Working...