emails marked as read

[rmorgan]

Can anybody help with my issue?


As a business we use Outlook 2007 for emails.

I have connected ESPOCRM and we link the email to issues, tasks etc and all works well but with one annoying issue which may mean that ultimately we cannot use the system (we do not use the CRM email for daily use a we are sticking with Outlook) -

If you monitor just the inbox it works fine. When an email comes in you can see it in Outlook and also see it in ESPOCRM. In Outlook the unread messages are in BOLD with a number next to them for the unread messages as is normal for Outlook.

The issue is that some people use Outlook rules so that emails from certain customers get put into certain folders. To simulate this, in Outlook setup a rule so that on receipt of an email from a specific email address, Outlook will automatically put the email into a new folder. When an email is received from that email address I can see that Outlook puts the email into the new folder and it is highlighted in bold along with the unread messages number next to it which is incremented. Then I wait for ESPOCRM to poll the Outlook server (I have it setup in scheduled jobs for */1 * * * *). When the Outlook server is polled, the message is copied into ESPOCRM BUT the original copy in Outlook is then marked as read. Therefore it is almost certain that the user will miss the fact that he\she has received an email.

Note: I have "Keep Fetched Emails Unread" ticked.

The issue with this is that people are now missing emails in Outlook as they rely on seeing the unread markers. It only seems to happen when you use sub folders - the main Inbox does not have this issue.

I am using version 5.5.6

[Maximus]

Hello Morgan.

I have tested this with fetching Emails from the Gmail. Everything works well while the 'Keep Fetched Emails Unread' box is checked. This means that this issue exists on the Outlook side.

[rmorgan]

Hello Maximus,

Did you try using a separate folder rather than just the inbox? I dont have an issue with inbox - this seems ok, it is only when emails are automatically placed in different monitored folders using outlook rules.

I am now looking through the code to see if I can debug.

[Maximus]

Yep, on the Gmail side, I have created a new one folder with the filtering rule to put inbound emails into it.

[rmorgan]

Hi Maximus,

I am getting cannot set flags, have you tried to set the recent flag or special chars? in the error log

I have done some more digging....in the file \apps\espocrm\htdocs\vendor\zendframework\zend-mail\src\protocol\imap.php the following produces a boolen response when just the inbox is used

$tag = null; // define $tag variable before first use
$this->sendRequest($command, $tokens, $tag);
$response = $this->readResponse($tag, $dontParse);

$Response is now = true, all working as expected.

If sub-folders are used instead of the inbox, this returns a null, which is an error according to the function header for readResponse: * @return null|bool|array tokens if success, false if error, null if bad request

When using just the inbox the data for sendRequest($command, $tokens, $tag); is as follows-
$command = STORE
$tokens = 213,FLAGS.SILENT,()

when using sub-folders (which fails), it is as follows -
$command = STORE
$tokens = 25,FLAGS.SILENT,(\Recent)

Now according to IMAP RFC here https://tools.ietf.org/html/rfc3501#section-2.3.2 in section 2.3.2

Note: The \Recent system flag is a special case of a session flag. \Recent can not be used as an argument in a STORE or APPEND command, and thus can not be changed at all. Yet it seems that \Recent is being used with a STORE command. So I am wondering if Exchange Server 2013 rejects the \Recent tag, but gmail is more forgiving?

I don't believe that there are any exchange settings that can effect this - here they are -

RunspaceId : f0638def-813a-4d66-9119-962b83e42607
ProtocolName : IMAP4
Name : 1
MaxCommandSize : 10240
ShowHiddenFoldersEnabled : False
UnencryptedOrTLSBindings : {[::]:143, 0.0.0.0:143}
SSLBindings : {[::]:993, 0.0.0.0:993}
InternalConnectionSettings : {XXXXMAIL02.XXXXLtd.local:993:SSL, XXXXMAIL02.XXXXLtd.local:143:TLS}
ExternalConnectionSettings : {mail.XXXXltd.co.uk:143:TLS, mail.XXXXltd.co.uk:993:SSL}
X509CertificateName : mail.XXXXltd.co.uk
Banner : The Microsoft Exchange IMAP4 service is ready.
LoginType : SecureLogin
AuthenticatedConnectionTimeout : 00:30:00
PreAuthenticatedConnectionTimeout : 00:01:00
MaxConnections : 2147483647
MaxConnectionFromSingleIP : 2147483647
MaxConnectionsPerUser : 16
MessageRetrievalMimeFormat : BestBodyFormat
ProxyTargetPort : 9933
CalendarItemRetrievalOption : iCalendar
OwaServerUrl :
EnableExactRFC822Size : False
LiveIdBasicAuthReplacement : False
SuppressReadReceipt : False
ProtocolLogEnabled : False
EnforceCertificateErrors : False
LogFileLocation : E:\Microsoft\Exchange Server\V15\Logging\Imap4
LogFileRollOverSettings : Daily
LogPerFileSizeQuota : 0 B (0 bytes)
ExtendedProtectionPolicy : None
EnableGSSAPIAndNTLMAuth : True
Server : XXXXMAIL02
AdminDisplayName :
ExchangeVersion : 0.10 (14.0.100.0)
DistinguishedName : CN=1,CN=IMAP4,CN=Protocols,CN=XXXXMAIL02,CN=Server s,CN=Exchange Administrative
Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=XXXX Ltd,CN=Microsoft
Exchange,CN=Services,CN=Configuration,DC=XXXXLtd,D C=local
Identity : XXXXMAIL02\1
Guid : 04a300a9-a68c-4059-900d-307637b1aa2b
ObjectCategory : XXXXLtd.local/Configuration/Schema/ms-Exch-Protocol-Cfg-IMAP-Server
ObjectClass : {top, protocolCfg, protocolCfgIMAP, protocolCfgIMAPServer}
WhenChanged : 22/03/2019 15:49:32
WhenCreated : 15/07/2015 13:40:08
WhenChangedUTC : 22/03/2019 15:49:32
WhenCreatedUTC : 15/07/2015 12:40:08
OrganizationId :
OriginatingServer : XXXXFILE01.XXXXLtd.local
IsValid : True
ObjectState : Unchanged

Any thoughts on this? I will try to do some more digging tomorrow to understand the IMAP protocol as I have no previous knowledge of this.

Thanks

[rmorgan]

So MS Exchange server 2013 does not like the IMAP message

STORE
26
FLAGS.SILENT
(\Recent)

When I connect to GMAIL it seems to get a slightly different message

STORE
array
5
FLAGS.SILENT
()

I dont know why with Exchange it adds the \Recent flag!

I did a little 'hack' in function requestAndResponse() located in ..\src\protocol\Imap.php and if FLAGS.SILENT and (\Recent) are detected, I then replace with -FLAGS and (\Seen).

The system now seems to work properly -

public function requestAndResponse($command, $tokens = [], $dontParse = false)
{
if ($tokens[2]=='(\Recent)' && $tokens[1]=='FLAGS.SILENT')
{
$tokens[1]='-FLAGS'; //CANOT USE FLAGS.SILENT for exchange 2013
$tokens[2]='(\Seen)'; //CANOT USE FLAGS.SILENT for exchange 2013
}

......

I tried using outlook webmail but it does not appear to support SSL (only TLS) so I could not test using an online MS server.

Anyway...my hack has resolved my problem!