6.1 upgrade error 'Data truncated'

[murugappan]

Hi,

I just tried the migration from 6.0.1 to 6.1.2. The intermediate migration to 6.0.10 executed well. But the next to 6.1.2 failed. I checked the log which says that there is some permission error on a permission.php file. I checked the file and the permission is set to 644. Not sure what is happening.. I have attached the log file and also the upgrade SSH screencap 1. Please note that the database (one table has 25,000 rows and another has 9,000 rows) and the application folder are quite large.

I tested on six different instances and all the same results. The odd part of this, the upgrade went through for one particular instance. On investigating, i found the log for the instance had the same permission errors as others. The only difference between this instance and the others is that this instance the database is small.

Need help on this.

[murugappan]

yuri Can i get some help on my issues in post #21

[murugappan]

Hi Yuri, I uploaded the custom folder but the forum does not allow me to upload the sql dump. The size of the dump file is 870kb. How do i upload the dump? Additional information, the permission error continues to be logged even in the instances where the upgrade succeeded.

[yuri]

I think you need to zip it before uploading.

[murugappan]

I zipped the file and the 870kb is the size of the zipped file.

[Maximus]

Hi murugappan,
I've tried to reproduce it a few times and always it went good. I tested it on PHP v.7.3.25 and MariaDB v.10.2.36.
What environment do you have?
Could you run rebuild and see what will happen? Will rebuild fail because of the same error?

[murugappan]

Maximus

My environment is PHP v7.3.26 and Maria 10.3.27. After performing the upgrade to 6.0.10, i did a rebuild. Then i did an upgrade. The upgrade took a long time(roughly 3 minutes) and then died. The rebuild failed with the same error and no others. There hundreds of lines.

Worse, after the upgrade failed the instance remained at v 6.0.10 but keeps logging the same error in the log non-stop.

[murugappan]

Maximus I got you private message asking me to contact you support team for database. I could not do that because your support only allows paid support. How do i send the db dump?

[esforim]

I guess upload it somewhere (your own file host) or public place/private cloud sharing site and send them the link.

Hopefully we can figure this out.

[murugappan]

This is really a serious issue. If this does not get resolved, it will surely become a "show-stopper". So much so that subsequent release problems may all point the finger to this unresolved issued.

I commented out the 2 areas in the code and yet the same error pops into the log. The error messages are so cryptic that i may need a decipher machine. I dug into the code and it points to 2 variables - $user and $path. The odd part is that the error message should print these 2 variables and it may provide an insight into the entities causing the problem. The code seems to changing the ownership and group for some entity but it is unable to.

[murugappan]

Hi espcrm, I considered your option but the problem is who to share it wit and send the link to. The problem is definitely in version 6.1.2. I have tested by upgrading step by step. All worked but failed only at 6.1.2

[murugappan]

Maximus
esforim

I have one more card to play. One peculiar thing about failed instance is that it has this Advance Pack (outdated) installed. I am going to test the upgrade on a fresh 6.0.10 version with the advance pack installed. Will keep you posted.

[murugappan]

Maximus
esforim

Hi,

I conducted a test to verify if the problem is due to the Advance Pack. These are my finding:

(1) Installed a fresh version 6.0.8 and installed Advance Pack 2.5.10 (old)
(2) Did a rebuild with no errors.
(3) Upgraded to 6.1.2 and successful without any errors.
(4) Restored our production version of 6.0.8.
(5) Did a rebuild with no errors
(6) Upgraded first to 6.0.10, looked good as there were no errors in the log.
(7) Performed a rebuild after the upgrade and BANG!! The log was full of permission errors but the application still works.
(8) Upgraded to 6.1.2 and ... wait for it... IT CRASHED

It appears that problem has been in existence with upgrade to 6.0.10. The rebuild in 6.0.10 seems to be the culprit here.

[esforim]

Hi muru, I don't use Advance Pack so your issue might be two separated isolated issue. I just read Yuri post here before coming to this thread: https://forum.espocrm.com/forum/anno...age2#post67358

Hopefully fixing this Collation thing get the performance back to normal.

It be good if these this solution help you with your upgrade but reading your post it doesn't look like it is, perhaps it is related to MariaDB in some way?

[murugappan]

Hi Espcrm, it resolved the performance issue. I have update my comment on the link. The older version of espocrm created the IDX as such and carries forward this setting even after the upgrade and migration to MariaDB..

[murugappan]

Hi,
Great news from Maximus, He just advised me that the problem i was facing has been fixed and the will be released in version 6.1.3.

[murugappan]

I am reposting my post here too to keep both tracks updated.

This is what I did:

(1) Restored my instance to version 6.0.8 - no problems
(2) Performed an upgrade and it rolled up to 6.0.10.
(3) Checked the log and whole bunch of errors shows up, similar to the previous error which was posted under https://forum.espocrm.com/forum/inst...data-truncated . (refer to screencap)
(4) Then i upgraded again which rolled-up to version 6.1.3 after a very long time.
(5) The same errors were logged in but the upgrade was successful (i think!)
(6) Delete the logs and loaded the instance. No errors logged.
(7) Did a rebuild and voila! the same error appears again.

Its odd, as i can remember that my problem surfaced when i upgraded 6.0.10 previously. Unfortunately, the upgrade from 6.0.8 steps through version 6.0.10 and seem to be creating the same issue. Thereafter any further upgrade appears to carry forward the problems. Espocrm developers need to investigate this.

I have halted at version 6.0.9. That's it for the time being because i am worried about the integrity of the database once i cross into any further upgrade which manifests the same errors in the log. I feel Espocrm need to perform the upgrade straight through from 6.0.8 or 6.0.9 to version 6.1.x and avoid the 6.0.10 step up.

[murugappan]

Great New folks! My problem has resolved and the upgrades are fast and smooth without any errors. The problem was the permissions parameter in the config.php. All my instances, not sure why or how, show the following:

user => 1003,
groups = 1005

After a fantastic help from Maximus who debugged the problem, i installed a fresh version 5.9.4 and found that the permissions were:

user => 1000,
groups => 1002

Based on this, i changed the permissions and now all the upgraded completed fast and smooth without any errors.

MY HEARTIEST THANKS TO Maximus FOR HIS PATIENCE WITH ME AND ALL THE HELP.

[murugappan]

Repeated from another Post

A word of reservation. After more testing, i found that this may only work if the instance has not been upgraded to version 6.0.10. Once you sink into versions > 6.0.10, the problem continues to exists. I have raised this with Maximus again. I do have 3 instances (test, staging, and production) relating to one app with this problem.

In one instance, after smooth upgrading to 6.1.4, there was a problem/error with the Scheduled Jobs on the Group Email Accounts. I found that a Group Email Account and the scheduler has some problem. All i did was to remove the "faulty" Group Email Account and created a new one in its place. Everything went on fine and cron is fetching the email correctly now. Phew!

[esforim]

Looking forward to your issue resolve as well, I'm stuck on 6.0.9 at the moment and too scare to try again until 6.2 is release.

[murugappan]

Hi all,

With a lot of tireless help from Maximus, we managed to resolve the problem. The problem was related to a mismatched owner and group id in the Espocrm instance config .php file and linux file ownnerships. This is what we did to resolve the problem:

(1) cd public_html/ ===> Web root
(2) stat tigascrm =====> tigascrm is the instance root folder
(3) I get "Uid: ( 1001/tigasdispensing) Gid: ( 1003/tigasdispensing)" as output
(4) edit the tigascrm->data->config.php
(5) locate the code that says 'defaultPermissions'
(6) Change the value next to "user' to 1001
(7) Change the value next to 'group' to 1003

Yipeee! All issues and errors gone. Log is clean now. A lot of thanks to the great help from Maximus.