Big data troubles

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • de1eted
    Junior Member
    • Dec 2016
    • 10

    Big data troubles


    Import 500,000 leads makes work impossible.
  • yuri
    Member
    • Mar 2014
    • 8440

    #2
    What server configuration do you have?
    Last edited by yuri; 12-26-2016, 09:25 AM.
    If you find EspoCRM good, we would greatly appreciate if you could give the project a star on GitHub. We believe our work truly deserves more recognition. Thanks.

    Comment

    • de1eted
      Junior Member
      • Dec 2016
      • 10

      #3
      VM fedora23 4CPU 8Gb

      Comment

      • yuri
        Member
        • Mar 2014
        • 8440

        #4
        With such amount of records you might need some additional db configuration. Need more details what exactly runs slowly, what sql queries. We tried big db and it worked. Of course it's more slowly than empy database.
        If you find EspoCRM good, we would greatly appreciate if you could give the project a star on GitHub. We believe our work truly deserves more recognition. Thanks.

        Comment

        • de1eted
          Junior Member
          • Dec 2016
          • 10

          #5
          New problem. Import by cron make duplicates (4 times per record). CSV - 49Mb, 40000 records. File:https://drive.google.com/file/d/0B30...ew?usp=sharing

          Comment

          • de1eted
            Junior Member
            • Dec 2016
            • 10

            #6
            Import was launched yesterday (GMT+3)
            Last edited by de1eted; 12-29-2016, 09:25 AM.

            Comment

            • de1eted
              Junior Member
              • Dec 2016
              • 10

              #7
              I guess cron task started 4 times. How to prevent the simultaneous import?

              Comment

              • de1eted
                Junior Member
                • Dec 2016
                • 10

                #8
                Limit number of CPUs to 1 solved problem.

                Comment

                Working...