Big data troubles

[de1eted]

Import 500,000 leads makes work impossible.

[yuri]

What server configuration do you have?

[de1eted]

VM fedora23 4CPU 8Gb

[yuri]

With such amount of records you might need some additional db configuration. Need more details what exactly runs slowly, what sql queries. We tried big db and it worked. Of course it's more slowly than empy database.

[de1eted]

New problem. Import by cron make duplicates (4 times per record). CSV - 49Mb, 40000 records. File:https://drive.google.com/file/d/0B30...ew?usp=sharing

[de1eted]

Import was launched yesterday (GMT+3)

[de1eted]

I guess cron task started 4 times. How to prevent the simultaneous import?

[de1eted]

Limit number of CPUs to 1 solved problem.