Email Fetching Duplicate Messages

[joy11]

I'm not the op, but I've been having a similar issue. I have duplicate emails in EspoCRM with the same message ids and after checking the Gmail account there is only 1 email in the inbox. It seems that it's not just the other email client. It happens in both personal emails (where I could see possibly the "sent" folder being the issue) and also group email (where the "sent" is not monitored) which generates Cases and then generates multiple cases with the email being in the stream of both and attached in the activities of the last one. To me, it seems like the job is possibly taking too long and starts to fetch again. I personally haven't seen more than 3 duplicates which is the same number of times a job will try before failing, correct? Is there something I could try to increase the time between retry on the email fetching job?

I also have seen the issue where duplicate emails have been sent out of the system. There is a workflow to send out support case closed notifications and about 15 minutes ago I had the "same" notification go out twice. The content of the email was the same and same time etc. In this circumstance, the message_id was different, however.

Do you think it could it be a conflict issue with workflow generated emails and the email fetch process?

[yuri]

Do not treat EspoCRM as an ordinary mail client. It's a CRM with a different implementation than any email client.

[yuri]

This bug was reported to Microsoft multiple times and they don't want to fix.

[yuri]

How can I explain. It's a bug in the mail client that stored those email copies with different message ids.

If you have such a situation then don't fetch "Sent" folder that contains email duplicates with different message id.

[cardmaverick]

I also have hundreds of sent messages that are doubled up.

[cardmaverick]

I have one message repeated 6 times. This is a real bug.

[joy11]

I've had the same thing happening. Duplicate and sometimes triplicate. I've had several this morning that are identical except for id field. I'm still running 4.8.4 with adv pack 1.28. We use gmail. This is my error log for today:

[2018-01-23 13:30:19] Espo.ERROR: API [GET]:.*, Params:Array ( ) , InputData: - [] []
[2018-01-23 13:30:19] Espo.ERROR: Display Error: , Code: 400 URL: /?entryPoint=attachment&a%3Clink%20href= [] []
[2018-01-23 13:30:19] Espo.ERROR: Uncaught Exception Slim\Exception\Stop: "" at /home/NAME/public_html/vendor/slim/slim/Slim/Slim.php line 1022 {"exception":"[object] (Slim\\Exception\\Stop(code: 0): at /home/NAME/public_html/vendor/slim/slim/Slim/Slim.php:1022)"} []
[2018-01-23 13:53:57] Espo.ERROR: API [GET]:/:controller/:id, Params:Array ( [controller] => Case [id] => 5a5e2a5ca7908b735 ) , InputData: - [] []
[2018-01-23 13:53:57] Espo.ERROR: Display Error: , Code: 404 URL: /api/v1/Case/5a5e2a5ca7908b735 [] []


The only true email error that I get sometimes is like this one:
[]
[2018-01-22 20:42:05] Espo.ERROR: EmailAccount 58ac52b87766cc88f (Import Message w/ MailMimeParser): [23000] SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '5a664c76d8bd02868-586e8f0f44e8f7036' for key 'UNIQ_12A5F6CCA832C1C9A76ED395' []

[yuri]

There is the issue in Outlook (maybe some other clients too) that stored 'Sent' emails are stored with a different message id than original email. Maybe this is that case.

[yuri]

In EspoCRM there is a check whether any email with the same message id exists. If exists then new record won't be created.

We don't have any ability to check whether emails are real duplicates if you have emails with different message ids on your IMAP.

[cardmaverick]

I just found other messages with message_id field not empty that are also duplicates.

[cardmaverick]

message_id is NULL - a quick search of the real inbox shows this message only appears once. They are real duplicates.

[yuri]

It's very unlikely. Can you check what message_id is stored in database for those emails. If they are different then email can't be treated as duplicates.

[cardmaverick]

The email fetcher is fetching the same messages over and over. I have several copies of some messages.