Upgrading from 4.3.1 to 4.4.1: member function get() on null error

[michib]

Hi to all,

I'm facing some hurdles updating an old Espo deployment in 4.2.5. I'd like to upgrade to current 5.0.
Problems start when I update from 4.3.1 to 4.4.1:

php upgrade.php ../EspoCRM-upgrade-4.3.1-to-4.4.1.zip
Current version is 4.3.1
Start upgrade process...
PHP Fatal error: Call to a member function get() on null in [...]/application/Espo/Core/ORM/Repositories/RDB.php on line 283

Line 283 of RDB.php tries to do

if ($entity->hasAttribute('modifiedById')) {
$entity->set('modifiedById', $this->getEntityManager()->getUser()->id);
$entity->set('modifiedByName', $this->getEntityManager()->getUser()->get('name')); [--line 283]
}

So as far as I can understand the code why does getUser() return null at line 283 when at the previous line it was valid and ->id did not crash code?

Tried with 5.6.25 and even 7.1: php -v
PHP 5.6.25 (cli) (built: Oct 21 2016 18:00:07)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies

Any ideas?
Thanks in advance,

Michele

[yuri]

Hi Michele,

Do you have a system user in the database? In the user table the record with id = 'user'.

[yuri]

Check whether the 'system' user is fetched at this line https://github.com/espocrm/espocrm/b...pgrade.php#L63.

echo $user->id;
die;

[michib]

Check whether the 'system' user is fetched at this line https://github.com/espocrm/espocrm/b...pgrade.php#L63.

echo $user->id;
die;

Hi Yuri,

Thanks for your prompt reply. Adding your debug check in upgrade.php return 'system', so it seems that the system user is fetched.

It appears that I have two "special" user ids in my db:

| system | 0 | 0 | system | NULL | NULL | | System | 1 | 0 | 0 | NULL | | 2016-09-18 20:26:03 | NULL | NULL | NULL |

| 1 | 0 | 1 | admin | Iku.bXPcbEZeVeseP7ic1sxoodxAOKrSxR/nZjtCbICDMnhfozSzXacRVYGw9wvFZw7KHM/CnzxVSQZW1/m3N/ | | | Admin

I have "system" and also "1" as Admin id. Is it normal?

thanks, regards
Michele

[michib]

I put some logging at line 283 in RDB.php and I was able to figure that at line 282
$entity->set('modifiedById', $this->getEntityManager()->getUser() is null when code enters the block...

I was able to figure out $entity->id which is:
string(17) "57def9b21ec28579a"

Then I grep-ed it into a sql dump and I found it's id of an email_account.... of... Admin!

Any ideas?

thanks,
Michele

[yuri]

> I have "system" and also "1" as Admin id. Is it normal?

Yes it's normal.

Weird things. Never run into such an issue before. Do you have any customizations made manually? Maybe some modifications in metadata.

[michib]

> I have "system" and also "1" as Admin id. Is it normal?

Yes it's normal.

Weird things. Never run into such an issue before. Do you have any customizations made manually? Maybe some modifications in metadata.

Hi Yuri,

Absolutely not.
It's something broken inside email settings thou. If I remove all email settings for all users then upgrade is successfull.

Here are two examples of sql metadata in email account. Maybe you can see if something looks "strange" to you at first glance? I suspect it is not able to create the entire entity from db...

INSERT INTO `email_account` VALUES ('57def9b21ec28579a','Amministratore',0,'crm2test@ XXXXXX','Active','ms1.XXXXXX','993',1,'crm2test@XX XXXX','LYf4dPjhxxK6OwdbZp3h9W2MjM0Mna1KpCTSBnaFfpU =','INBOX',NULL,0,0,'2016-09-18','{"lastUID":{"INBOX":"260"},"lastDate":{"INBOX ":"2016-11-07 22:37:40"}}','2016-09-18 20:31:46','2016-09-18 20:31:46',0,NULL,25,0,NULL,NULL,NULL,NULL,'1','1', '1'),

('5857eb6e7d2b76e19','bbbbb@news.XXXXXX',0,'bertol otto@news.XXXXXX','Active','nl1.XXXXXX','993',1,'b bbbb@news.XXXXXX','EaNCY7aW8oJc8x33sBQ5Lmc1eKvNSsb fgMf50kwbKvQ=','INBOX',NULL,0,0,'2016-12-19','{"lastUID":{"INBOX":"5442"},"lastDate":{"INBO X":"2017-12-01 23:45:39"}}','2016-12-19 14:15:10','2016-12-19 14:15:29',1,'nl1.XXXXXX',587,1,'TLS','bbbbb@news.X XXXXX','EaNCY7aW8oJc8x33sBQ5Lmc1eKvNSsbfgMf50kwbKv Q=',NULL,'57fd0042a853e9c80','1','1');


thanks,
Michele

[yuri]

Seems it's related to the after-upgrade script of the upgrade package. Email accounts are modified there.

class AfterUpgrade
{
    public function run($container)
    {
        $entityManager = $container->get('entityManager');
        $config = $container->get('config');

        function mcryptDecryptOld($encryptedString, $key) {
            $encryptedString = base64_decode($encryptedString);
            $string = substr($encryptedString, 0, strlen($encryptedString) - 16);
            $iv = substr($encryptedString, -16);

            return trim(mcrypt_decrypt(MCRYPT_RIJNDAEL_128, $key, $string, MCRYPT_MODE_CBC, $iv));
        }

        function mcryptEncrypt($string, $key, $iv) {
            $block = mcrypt_get_block_size(MCRYPT_RIJNDAEL_128, 'cbc');
            $pad = $block - (strlen($string) % $block);
            $string .= str_repeat(chr($pad), $pad);
            return base64_encode(mcrypt_encrypt(MCRYPT_RIJNDAEL_128, $key, $string, MCRYPT_MODE_CBC, $iv) . $iv);
        }

        function opensslDecrypt($encryptedString, $key) {
            $encryptedString = base64_decode($encryptedString);
            $string = substr($encryptedString, 0, strlen($encryptedString) - 16);
            $iv = substr($encryptedString, -16);

            return trim(openssl_decrypt($string, 'aes-256-cbc', $key, OPENSSL_RAW_DATA, $iv));
        }

        function mcryptDecrypt($encryptedString, $key) {
            $encryptedString = base64_decode($encryptedString);
            $string = substr($encryptedString, 0, strlen($encryptedString) - 16);
            $iv = substr($encryptedString, -16);

            $string = mcrypt_decrypt(MCRYPT_RIJNDAEL_128, $key, $string, MCRYPT_MODE_CBC, $iv);

            $block = mcrypt_get_block_size(MCRYPT_RIJNDAEL_128, 'cbc');
            $len = strlen($string);
            $pad = ord($string[$len - 1]);
            return substr($string, 0, strlen($string) - $pad);
        }

        $iv = mcrypt_create_iv(16, MCRYPT_RAND);

        $cryptKey = $config->get('cryptKey');

        if (!$cryptKey) {
            $newCryptKey = md5(uniqid());
            $config->set('cryptKey', $newCryptKey);
            $config->save();
        } else {
            $newCryptKey = $cryptKey;
        }

        $key = hash('sha256', $cryptKey, true);
        $newKey = hash('sha256', $newCryptKey, true);

        foreach ($entityManager->getRepository('EmailAccount')->find() as $entity) {
            foreach (['password', 'smtpPassword'] as $attribute) {
                $value = $entity->get($attribute);
                if (empty($value)) continue;
                $valueDecrypted = mcryptDecryptOld($value, $key);
                $valueEncrypted = mcryptEncrypt($valueDecrypted, $newKey, $iv);

                $entity->set($attribute, $valueEncrypted);
            }
            $entityManager->saveEntity($entity);
        }

        foreach ($entityManager->getRepository('InboundEmail')->find() as $entity) {
            foreach (['password'] as $attribute) {
                $value = $entity->get($attribute);
                if (empty($value)) continue;
                $valueDecrypted = mcryptDecryptOld($value, $key);
                $valueEncrypted = mcryptEncrypt($valueDecrypted, $newKey, $iv);

                $entity->set($attribute, $valueEncrypted);
            }
            $entityManager->saveEntity($entity);
        }

        foreach ($entityManager->getRepository('User')->find() as $user) {
            $entity = $entityManager->getEntity('Preferences', $user->id);
            if (!$entity) continue;
            foreach (['smtpPassword'] as $attribute) {
                $value = $entity->get($attribute);
                if (empty($value)) continue;
                $valueDecrypted = mcryptDecryptOld($value, $key);
                $valueEncrypted = mcryptEncrypt($valueDecrypted, $newKey, $iv);

                $entity->set($attribute, $valueEncrypted);
            }
            $entityManager->saveEntity($entity);
        }

    }
} 

[michib]

Seems it's related to the after-upgrade script of the upgrade package. Email accounts are modified there.

foreach ($entityManager->getRepository('EmailAccount')->find() as $entity) {
foreach (['password', 'smtpPassword'] as $attribute) {
$value = $entity->get($attribute);
if (empty($value)) continue;
$valueDecrypted = mcryptDecryptOld($value, $key);
$valueEncrypted = mcryptEncrypt($valueDecrypted, $newKey, $iv);

$entity->set($attribute, $valueEncrypted);
}
$entityManager->saveEntity($entity);
} 

Hi Yuri,
I worked on that piece of code you wrote and I was able to isolate that it breaks inside the first foreach... but there's nothing really strange there, only some smtpPassword value which is null. I tried to set it to something to see if it was related to that but no luck.

I debugged application/Espo/Core/ORM/Repositories/RDB.php and in saveEntity()

$this->getEntityManager()->getUser()

returns NULL ... I think it's something - as you pointed out - related to metadata because getUser simply returns a property of EntityManager. I'd like to dig and find what has happened to this Espo installation. Can you give me more help? Is that a serialization problem from db? Where can I start to look at it?

thanks, regards.
M.

[tanya]

Hello
I have php 7.2, I can't even install this old version, so I can't help you to fix it.
But, if this upgrade is crashed on step afterUpgrade script, it means upgrade is done, but only passwords could be wrong.
If you can login, go to About and check the version. If it is 4.4.1, check your Email Accounts. Fix wrong passwords.
Rebuild EspoCRM to check if everything is OK.
Try to upgrade to a newer version.

Regards,
Tanya

[michib]

Hello
I have php 7.2, I can't even install this old version, so I can't help you to fix it.
But, if this upgrade is crashed on step afterUpgrade script, it means upgrade is done, but only passwords could be wrong.
If you can login, go to About and check the version. If it is 4.4.1, check your Email Accounts. Fix wrong passwords.
Rebuild EspoCRM to check if everything is OK.
Try to upgrade to a newer version.

Regards,
Tanya

Hi Tanya, thanks for your kind reply.
My point is that:

- I have to delete every Email Account to make it go [UPDATED: as you said, even if it crashes I can see all email account also after "upgrade"]

- It does not fail exactly inside afterUpgrade() because it actually is able to change password values correctly there... What I see is that it crashes invoking saveEntity() when searching for a getUser() info related to an entity. This make me think of what Yuri said days ago, something like a metadata corruption. So I'd like to investigate further just to be sure that there isn't something bad going on. But I need some hint here...

thanks, regards,
M.

[michib]

Hi Tanya, thanks for your kind reply.
My point is that:

- I have to delete every Email Account to make it go [UPDATED: as you said, even if it crashes I can see all email account also after "upgrade"]

- It does not fail exactly inside afterUpgrade() because it actually is able to change password values correctly there... What I see is that it crashes invoking saveEntity() when searching for a getUser() info related to an entity. This make me think of what Yuri said days ago, something like a metadata corruption. So I'd like to investigate further just to be sure that there isn't something bad going on. But I need some hint here...

thanks, regards,
M.

Hi, maybe the solution is really trivial: does EntityManager provides via getUser() a valid userId even if lauched via CLI? Isn't it provided only inside web UI?
If I load the upgrade via web instead of CLI it never report any error!!

thanks, regards,
Michele

[michib]

$app = new \Espo\Core\Application();
$config = $app->getContainer()->get('config');
$entityManager = $app->getContainer()->get('entityManager');
$user = $entityManager->getEntity('User', 'system');

$entityManager->setUser($user);
echo "User name is " . $user->get('name') . "\n";

$app->getContainer()->setUser($user);
$upgradeManager = new \Espo\Core\UpgradeManager($app->getContainer()); 

If I forcefully call setUser explicitly for entityManager then it works... Can this have any sense to you?

thanks,
Michele