Unable to manage email with Malformed UTF-8 characters

[item]

Hi,
for information :
what's is version? :
php
espocrm

[mdmen]

[2023-03-10 22:51:12] ERROR: Slim Application Error Type: ParseError Code: 0 Message: syntax error, unexpected token "namespace" File: /home/mdmen/pk.mdmen.org/application/Espo/Entities/Email.php Line: 1 Trace: #0 /home/mdmen/pk.mdmen.org/vendor/composer/ClassLoader.php(344): Composer\Autoload\includeFile('/home/mdmen/pk....') EspoCRM Forum /home/mdmen/pk.mdmen.org/application/Espo/Services/Email.php(1167): Composer\Autoload\ClassLoader->loadClass('Espo\\Entities\\E...') Forum /home/mdmen/pk.mdmen.org/application/Espo/Controllers/Email.php(287): Espo\Services\Email->getFoldersNotReadCounts() #3 /home/mdmen/pk.mdmen.org/application/Espo/Core/Api/ActionProcessor.php(123): Espo\Controllers\Email->getActionGetFoldersNotReadCounts(Array, '', Object(Espo\Core\Api\RequestWrapper), Object(Espo\Core\Api\ResponseWrapper)) #4 /home/mdmen/pk.mdmen.org/application/Espo/Core/Api/RequestProcessor.php(141): Espo\Core\Api\ActionProcessor->process('Email', 'getFoldersNotRe...', Object(Espo\Core\Api\RequestWrapper), Object(Espo\Core\Api\ResponseWrapper)) Groups /home/mdmen/pk.mdmen.org/application/Espo/Core/Api/RequestProcessor.php(112): Espo\Core\Api\RequestProcessor->proceed(Object(Espo\Core\Api\RequestWrapper), Object(Espo\Core\Api\ResponseWrapper)) Uncategorized Groups /home/mdmen/pk.mdmen.org/application/Espo/Core/Api/RequestProcessor.php(83): Espo\Core\Api\RequestProcessor->processInternal(Object(Espo\Core\Api\Route), Object(Espo\Core\Api\RequestWrapper), Object(Espo\Core\Api\ResponseWrapper)) Special /home/mdmen/pk.mdmen.org/application/Espo/Core/Api/Starter.php(103): Espo\Core\Api\RequestProcessor->process(Object(Espo\Core\Api\Route), Object(Espo\Core\Api\RequestWrapper), Object(Espo\Core\Api\ResponseWrapper)) Visitor Messages /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/Handlers/Strategies/RequestResponse.php(43): Espo\Core\Api\Starter->Espo\Core\Api\{closure}(Object(Slim\Psr7\Request) , Object(Slim\Psr7\Response), Array) Private Messages /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/Routing/Route.php(384): Slim\Handlers\Strategies\RequestResponse->__invoke(Object(Closure), Object(Slim\Psr7\Request), Object(Slim\Psr7\Response), Array) Albums /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/MiddlewareDispatcher.php(81): Slim\Routing\Route->handle(Object(Slim\Psr7\Request)) #11 /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/MiddlewareDispatcher.php(81): Slim\MiddlewareDispatcher->handle(Object(Slim\Psr7\Request)) #12 /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/Routing/Route.php(341): Slim\MiddlewareDispatcher->handle(Object(Slim\Psr7\Request)) CSS Examples /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/Routing/RouteRunner.php(84): Slim\Routing\Route->run(Object(Slim\Psr7\Request)) #14 /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/Middleware/RoutingMiddleware.php(59): Slim\Routing\RouteRunner->handle(Object(Slim\Psr7\Request)) General Discussion /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/MiddlewareDispatcher.php(147): Slim\Middleware\RoutingMiddleware->process(Object(Slim\Psr7\Request), Object(Slim\Routing\RouteRunner)) Announcements /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(107): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(Slim\Psr7\Request)) Feature Requests /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/MiddlewareDispatcher.php(147): Slim\Middleware\ErrorMiddleware->process(Object(Slim\Psr7\Request), Object(Psr\Http\Server\RequestHandlerInterface@ano nymous)) Installation and Upgrade Help /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/MiddlewareDispatcher.php(81): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(Slim\Psr7\Request)) Extensions /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/App.php(215): Slim\MiddlewareDispatcher->handle(Object(Slim\Psr7\Request)) Developer Help /home/mdmen/pk.mdmen.org/vendor/slim/slim/Slim/App.php(199): Slim\App->handle(Object(Slim\Psr7\Request)) Bug Reports /home/mdmen/pk.mdmen.org/application/Espo/Core/Api/Starter.php(79): Slim\App->run() Unminify assets or how to recreate t...o.min.js file? /home/mdmen/pk.mdmen.org/application/Espo/Core/ApplicationRunners/Api.php(49): Espo\Core\Api\Starter->start() /home/mdmen/pk.mdmen.org/application/Espo/Core/Application/RunnerRunner.php(85): Espo\Core\ApplicationRunners\Api->run() #24 /home/mdmen/pk.mdmen.org/application/Espo/Core/Application.php(81): Espo\Core\Application\RunnerRunner->run('Espo\\Core\\Appli...', NULL) /home/mdmen/pk.mdmen.org/public/api/v1/index.php(37): Espo\Core\Application->run('Espo\\Core\\Appli...') how to upgrade ? {main} Tips: To display error details in HTTP response set "displayErrorDetails" to true in the ErrorHandler constructor. [] []

[yuri]

Note that fix will work only new fetched emails.

[yuri]

It's likely you did something wrong.

[yuri]

What is the log?

[mdmen]

Thanks but that does not work. I cannot retrieve any email at all now (500).

[yuri]

Here's the fix: https://github.com/espocrm/espocrm/c...c1f5ab18f736ed

[mdmen]

Any progress with this issue? I run into it pretty often - seems that there are certain senders than regularly send emails with this error - and I can't read them.

One of them is attached.

Diib alert virusys.com may have an issue with homepage canonicalization. This could negatively impact rankings and visitor expe.zip

BTW - can read it fine from RoundCube webmail

[chr1st0ph9]

Attached the email (eml format)

[showard]

A quick workaround solution I found in dealing with a similar situation recently (the first and only time I have seen this in 4+ years) was to use the Formula Sandbox as admin. I could read the email, but got the same error when trying to move to trash or delete. Using the sandbox, I selected "Email" as target type, and chose the specific problem email as target. Then entered "deleted = 1" in the formula sandbox. That got rid of the problem email.

[yuri]

Could you provide such an email so we try to reproduce?

[yuri]

What collation is set for the `body` and `subject` columns in the `email` table? It should be utf8mb4_unicode_ci.

[chr1st0ph9]

Thanks for your help :-)

[item]

Hi, Salut

https://github.com/espocrm/espocrm/tree/master/application/Espo/Modules/Crm/Jobs

take one job.. and modify,

add just (choose $entityType > maybe email .. ) and ID :

$entityManager->getRDBRepository($entityType)->deleteFromDb($id);

then run Jobs

php command.php run-job JobName

[chr1st0ph9]

Bonjour,
I received an email which is impossible to read and even delete (but moving to trash os ok) , If I try I get the error message:

Erreur 500
Malformed UTF-8 characters, possibly incorrectly encoded

Thanks for any feedbacks/ideas/... how to remove this kind of emails.
---
meilleures salutations,
Christophe