PHP memory exhausted on a big mass update

[goingUp]

I did a mass update over thousands of records (adding Team) and process just silently stopped processing only 4k records. I found this error in log:
ERROR: (0) Uncaught Exception Spatie\Async\Output\ParallelError: "PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 16384 bytes) in /var/www/html/application/Espo/ORM/Executor/DefaultSqlExecutor.php on line 77 " at /var/www/html/vendor/spatie/async/src/Output/ParallelError.php line 11 :: /var/www/html/vendor/spatie/async/src/Output/ParallelError.php(11)

Espo Version 9.1.8, from official docker image.

I know that this error is hard to fix, so I'm filing it as a feature request to be ready for a huge datasets . Perhaps ultimate solution can be unbuffered query, if framework allows it.

[yuri]

The mass update is not supposed to cause the memory limit issue. If it does in some cases, it needs to be fixed.

[yuri]

I think I've found the culprit of the memory leak. It's in a relatively new feature in ORM. Need to incorporate a WeakReference usage, but more testing needed.

[yuri]

This will help https://github.com/espocrm/espocrm/c...a85b2730c7b308.

But needs more testing.

[goingUp]

yuri I'll test it, just need some time to set up Espocrm locally, can't update prod to master branch. Thank you!

[yuri]

No need to test.

[goingUp]

yuri I finally set up Espocrm locally, I need it anyways, and tested on master. Now everything works fine, successfully updated 100k records! Thank you.

[goingUp]

yuri another test, exporting 12k contacts (9.1.8 has "memory exhausted" issue with it too). On master: csv works, xls lite works, normal xls - out of memory (I guess that's why we have "lite" option). So this change fixes export too.

[yuri]

The fix was applied in the master, not available in the hotfix release.

The XLS memory issue is expected.