Announcement

Collapse
No announcement yet.

Email Fetching Duplicate Messages

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Email Fetching Duplicate Messages

    The email fetcher is fetching the same messages over and over. I have several copies of some messages.

  • #2
    It's very unlikely. Can you check what message_id is stored in database for those emails. If they are different then email can't be treated as duplicates.

    Comment


    • #3
      message_id is NULL - a quick search of the real inbox shows this message only appears once. They are real duplicates.

      Comment


      • #4
        I just found other messages with message_id field not empty that are also duplicates.

        Comment


        • #5
          In EspoCRM there is a check whether any email with the same message id exists. If exists then new record won't be created.

          We don't have any ability to check whether emails are real duplicates if you have emails with different message ids on your IMAP.

          Comment


          • #6
            There is the issue in Outlook (maybe some other clients too) that stored 'Sent' emails are stored with a different message id than original email. Maybe this is that case.

            Comment


            • #7
              I've had the same thing happening. Duplicate and sometimes triplicate. I've had several this morning that are identical except for id field. I'm still running 4.8.4 with adv pack 1.28. We use gmail. This is my error log for today:

              [2018-01-23 13:30:19] Espo.ERROR: API [GET]:.*, Params:Array ( ) , InputData: - [] []
              [2018-01-23 13:30:19] Espo.ERROR: Display Error: , Code: 400 URL: /?entryPoint=attachment&a%3Clink%20href= [] []
              [2018-01-23 13:30:19] Espo.ERROR: Uncaught Exception Slim\Exception\Stop: "" at /home/NAME/public_html/vendor/slim/slim/Slim/Slim.php line 1022 {"exception":"[object] (Slim\\Exception\\Stop(code: 0): at /home/NAME/public_html/vendor/slim/slim/Slim/Slim.php:1022)"} []
              [2018-01-23 13:53:57] Espo.ERROR: API [GET]:/:controller/:id, Params:Array ( [controller] => Case [id] => 5a5e2a5ca7908b735 ) , InputData: - [] []
              [2018-01-23 13:53:57] Espo.ERROR: Display Error: , Code: 404 URL: /api/v1/Case/5a5e2a5ca7908b735 [] []


              The only true email error that I get sometimes is like this one:
              []
              [2018-01-22 20:42:05] Espo.ERROR: EmailAccount 58ac52b87766cc88f (Import Message w/ MailMimeParser): [23000] SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '5a664c76d8bd02868-586e8f0f44e8f7036' for key 'UNIQ_12A5F6CCA832C1C9A76ED395' []
              Last edited by joy11; 01-23-2018, 02:01 PM.

              Comment


              • #8
                I have one message repeated 6 times. This is a real bug.

                Comment


                • #9
                  I also have hundreds of sent messages that are doubled up.

                  Comment


                  • #10
                    How can I explain. It's a bug in the mail client that stored those email copies with different message ids.

                    If you have such a situation then don't fetch "Sent" folder that contains email duplicates with different message id.
                    Last edited by yuri; 01-23-2018, 02:40 PM.

                    Comment


                    • joy11
                      joy11 commented
                      Editing a comment
                      I'm not the op, but I've been having a similar issue. I have duplicate emails in EspoCRM with the same message ids and after checking the Gmail account there is only 1 email in the inbox. It seems that it's not just the other email client. It happens in both personal emails (where I could see possibly the "sent" folder being the issue) and also group email (where the "sent" is not monitored) which generates Cases and then generates multiple cases with the email being in the stream of both and attached in the activities of the last one. To me, it seems like the job is possibly taking too long and starts to fetch again. I personally haven't seen more than 3 duplicates which is the same number of times a job will try before failing, correct? Is there something I could try to increase the time between retry on the email fetching job?

                      I also have seen the issue where duplicate emails have been sent out of the system. There is a workflow to send out support case closed notifications and about 15 minutes ago I had the "same" notification go out twice. The content of the email was the same and same time etc. In this circumstance, the message_id was different, however.

                      Do you think it could it be a conflict issue with workflow generated emails and the email fetch process?
                      Last edited by joy11; 01-23-2018, 04:00 PM.

                    • yuri
                      yuri commented
                      Editing a comment
                      When an email sent twice, it could be the situation when email has been sent but SMTP server responded with an error. Then EspoCRM tries to re-send it (up to 3 times).

                      Do you have the latest EspoCRM version?

                    • joy11
                      joy11 commented
                      Editing a comment
                      I'm in the process of updating to the newest version. I'm on 4.8.4 on the live server and running 5.0.3 on my test but having to fix some custom things that no longer work before upgrading live.

                  • #11
                    This bug was reported to Microsoft multiple times and they don't want to fix.

                    Comment


                    • #12
                      Do not treat EspoCRM as an ordinary mail client. It's a CRM with a different implementation than any email client.

                      Comment


                      • #13
                        Originally posted by yurikuzn View Post
                        How can I explain. It's a bug in the mail client that stored those email copies with different message ids.

                        If you have such a situation then don't fetch "Sent" folder that contains email duplicates with different message id.
                        That's not what's happening. The message ids are identical. I'm getting duplicates of both sent messages as well as duplicates of incoming messages.

                        Comment


                        • #14
                          So you have emails with identical message ids in espocrm database?

                          Comment


                          • #15
                            Originally posted by yurikuzn View Post
                            So you have emails with identical message ids in espocrm database?
                            Yes. This is a huge problem. I have literally hundreds of these. Email dates are wrong as well - I'm getting messages marked today that were received years ago. The whole thing seems to not work correctly.

                            Comment

                            Working...
                            X