Announcement

Collapse
No announcement yet.

Email slow loading or timeout. v8.0.3

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Email slow loading or timeout. v8.0.3

    I updated to version 8.0.3 and working with email in CRM became impossible. The email list loads a little slower than usual, but not critical. And when I try to open an email, I experience very slow loading or timeout. The problem appeared after updating to 8.0.3

    PHP 8.2.11
    MySQL 5.7.43
    There are no errors in the logs or on the server, the browser returns an error "timeout".

    I made a new installation EspoCRM v8.0.3 and linked it to the current database, the problem remained.​
    After a long wait and a possible timeout, the email opens again quickly. As it should be, but the next day the problem repeats. There are 45,000 emails in the system, which is not that much.
    Hard rebuild doesn't help.

    Please share your ideas on how to fix the problem.​

  • #2
    I don't know of any change in v8.0 that could affect emails performance. From which version did you upgrade?

    Comment


    • alexisc
      alexisc commented
      Editing a comment
      updated from Version 7.5.6
      I don't think this is related to CRM, but the interface is slow in getting information from the database

  • #3
    I recommend to try to tweak your database. Maybe your have not enough disk space on your server?

    Comment


    • #4
      free 22 GB out of 32 GB
      the rebuild command from the administration menu also ends with a timeout

      I'll try to set up the database, I don't understand why there were no problems with the previous version...​

      Comment


      • #5
        Check active processes on the server. Maybe Mysql takes up all free resources.

        Comment


        • #6
          free 22 GB out of 32 GB
          the rebuild command from the administration menu also ends with a timeout

          I'll try to set up the database, I don't understand why there were no problems with the previous version...​

          Comment


          • #7
            there are no problems with server loading, but there are a large number of slow requests related to email
            Sample type Possible keys Key Link Strings Additional info
            SIMPLE : email
            ALL
            32436 Using where; Using temporary; Using filesort
            each email request scans all 45091 records and returns 32436​ Strings
            Last edited by alexisc; 10-13-2023, 12:29 PM.

            Comment


            • #8
              Just for information. On the instance with 250,000 emails the list view and detail view load instantaneously. v8.0.3, MariaDB 11.

              Comment


              • alexisc
                alexisc commented
                Editing a comment
                you are right, the problem is not related to the update or the number of records

            • #9
              Could you find out which exactly SQL queries causes the issue?

              Comment


              • #10
                PHP Code:
                SELECT
                email
                .id AS `id`,
                email.name AS `name`,
                email.deleted AS `deleted`,
                email.from_string AS `fromString`,
                email.reply_to_string AS `replyToString`,
                email.address_name_map AS `addressNameMap`,
                email.is_replied AS `isReplied`,
                email.message_id AS `messageId`,
                email.message_id_internal AS `messageIdInternal`,
                email.body_plain AS `bodyPlain`,
                email.body AS `body`,
                email.is_html AS `isHtml`,
                email.status AS `status`,
                email.has_attachment AS `hasAttachment`,
                email.date_sent AS `dateSent`,
                email.delivery_date AS `deliveryDate`,
                email.created_at AS `createdAt`,
                email.modified_at AS `modifiedAt`,
                email.is_system AS `isSystem`,
                email.ics_contents AS `icsContents`,
                email.ics_event_uid AS `icsEventUid`,
                email.from_email_address_id AS `fromEmailAddressId`,
                fromEmailAddress.name AS `fromEmailAddressName`,
                email.parent_id AS `parentId`,
                email.parent_type AS `parentType`,
                email.created_by_id AS `createdById`,
                NULLIF(
                TRIM(
                CONCAT(
                IFNULL(createdBy.first_name''),
                ' ',
                IFNULL(createdBy.last_name'')
                )
                ),
                ''
                ) AS `createdByName`,
                email.sent_by_id AS `sentById`,
                NULLIF(
                TRIM(
                CONCAT(
                IFNULL(sentBy.first_name''),
                ' ',
                IFNULL(sentBy.last_name'')
                )
                ),
                ''
                ) AS `sentByName`,
                email.modified_by_id AS `modifiedById`,
                NULLIF(
                TRIM(
                CONCAT(
                IFNULL(modifiedBy.first_name''),
                ' ',
                IFNULL(modifiedBy.last_name'')
                )
                ),
                ''
                ) AS `modifiedByName`,
                email.assigned_user_id AS `assignedUserId`,
                NULLIF(
                TRIM(
                CONCAT(
                IFNULL(assignedUser.first_name''),
                ' ',
                IFNULL(assignedUser.last_name'')
                )
                ),
                ''
                ) AS `assignedUserName`,
                email.replied_id AS `repliedId`,
                replied.name AS `repliedName`,
                email.created_event_id AS `createdEventId`,
                email.created_event_type AS `createdEventType`,
                email.group_folder_id AS `groupFolderId`,
                groupFolder.name AS `groupFolderName`,
                email.account_id AS `accountId`,
                account.name AS `accountName`,
                emailUser.is_read AS `isRead`,
                emailUser.is_important AS `isImportant`,
                emailUser.in_trash AS `inTrash`,
                emailUser.folder_id AS `folderId`
                FROM
                `email` AS `email` USE INDEX (`IDX_DATE_SENT`)
                LEFT JOIN `account` AS `accountON email.account_id account.id
                LEFT JOIN 
                `group_email_folder` AS `groupFolderON email.group_folder_id groupFolder.id
                LEFT JOIN 
                `email_address` AS `fromEmailAddressON email.from_email_address_id fromEmailAddress.id
                LEFT JOIN 
                `email` AS `repliedON email.replied_id replied.id
                LEFT JOIN 
                `user` AS `sentByON email.sent_by_id sentBy.id
                LEFT JOIN 
                `user` AS `assignedUserON email.assigned_user_id assignedUser.id
                LEFT JOIN 
                `user` AS `modifiedByON email.modified_by_id modifiedBy.id
                LEFT JOIN 
                `user` AS `createdByON email.created_by_id createdBy.id
                LEFT JOIN 
                `email_user` AS `emailUserON emailUser.email_id email.id
                AND emailUser.deleted 0
                AND emailUser.user_id '1'
                WHERE
                email
                .id '65211b495413eb143'
                AND email.deleted 0
                ORDER BY
                email
                .date_sent DESC,
                email.id DESC
                LIMIT
                0
                1​ 
                such a request runs on average from 73 to 122 seconds and ends with a timeout
                Last edited by alexisc; 10-16-2023, 08:16 AM.

                Comment


                • jflores
                  jflores commented
                  Editing a comment
                  alexisc - where did you grab this code from? I saw the raw SQL in my server (I log slow queries) and it looks very similar.

              • #11
                If you would like just to fix the problem but not to figure out what is the cause, I suggest migrating to MariaDB. Anyway, if it won't fix the problem, Espo works better on MariaDB.

                Comment


                • jflores
                  jflores commented
                  Editing a comment
                  alexisc - I took yuri advice and switched to Mariadb. The performance got much much better. We still can't send emails since upgrading, but that's a separate issue, I assume. My initial hypothesis was that the server was timing out while executing some sql query and preventing emails from going out. Looks not to be the case. So, we aren't able to send, but at least we can see inbound again - without waiting 30 seconds, timing out or crashing server.

              • #12
                As a side note: MySQL 5.7 support seems to end October 2023.

                This is another incentive to change or upgrade the DB sooner or later. I'm just not sure if MySQL 8.x works with EspoCRM (docs state 5.7 or later) or would improve the situation for you.

                Comment


                • #13
                  Thank you!
                  I'll try to upgrade to MySQL 8.0
                  according to the result, I will write here​

                  Comment


                  • #14
                    Always make a backup first, in case it does not work. Or try first with a test server.

                    Comment


                    • #15
                      I had been using Espo for a long time and was delaying upgrading since version 6 because there were so many changes.

                      I ultimately decided to make the move this past weekend and I'm running espocrm through docker, connected to my MySQL database, which is on the same server, but outside the container.

                      My operating system is Ubuntu 22 and my MySQL version is 8 on a digital ocean server with a dedicated CPU, 8GB RAM, and 50GB storage (with all files stored externally in a digital ocean storage block)

                      I'm also experiencing painfully slow email performance. I'm not getting a timeout error, but it's taking several seconds for emails to load. And I just discovered inbound emails aren't coming in at all (all connection tests are ok).

                      yuri - When you say 'tweak the database' - do you have something specific in mind? Would you suggest I migrate everything to mariadb? I'm not seeing any errors in the logs that suggest anything is wrong with email - especially the inbound email not making it in. Where would you suggest I dig first?

                      ( alexisc - FYI - I'm on mysql 8)

                      In case it helps, this is the performance graph -> it seems to be waiting for server response. Other Entities are performing just fine, though.
                      Attached Files
                      Last edited by jflores; 10-17-2023, 01:02 AM.

                      Comment


                      • jflores
                        jflores commented
                        Editing a comment
                        Sending emails using the default editor is also very slow and ultimately times out. System-generated autoresponders and the 'test' emails from the SMTP screens are sending fine. It's just the outbound emails from the client that aren't sending.
                    Working...
                    X