Email Correlation in Dynamics 365
We recently discovered some very specific requirements around matching incoming emails to the correct record in Dynamics 365. This customer had an issue with ongoing email conversations where the senders were not tracking their emails in Dynamics 365, but rather relying on the Email tracking and Smart Matching features within Dynamics 365 to handle the job for them. Manual tracking can give ideal results in some instances, but the amount of labor and time involved would not work with in this case.
Our focus during this project was to optimize performance, and saving time was a key factor. We made several observations on how Dynamics 365 handles incoming emails, and throughout this post, I will explain some of these key factors.
When an email is sent to an incoming queue, Dynamics 365 analyzes it to see if it is a part of an ongoing email conversation, so it can set the ‘Regarding’ field to the correct record.
Dynamics 365 figures out which record the email belongs to by looking at the:
In Reply To tag
Knowing how these different matching options work will help you understand where your emails end up, and what you need to change if they don’t end up in the right place.
This is likely the most reliable way of tracking email in Dynamics 365. The process works by adding a token like “CRM:0001323” in the subject line of outgoing emails. When a reply is received, Dynamics 365 uses the tracking token to match the email to the correct conversation and record.
There are some configuration options available for what the tracking token looks like in the System Settings and Email tab of Dynamics 365. For example; rather than the generic “CRM” prefix, you could have something more meaningful like “eLogic:”
While you do have some flexibility in the syntax of the tracking token, it is often considered a seemingly intrusive way of tracking emails. There’s nothing like an 11-digit tracking number to say, “I’M FOLLOWING YOU!!!” In service scenarios, the tracking codes can be mixed up with the case number and lead to some confusion when customers call for help and refer to the tracking code with a service representative rather than the actual case number.
That said, the tracking token is very effective, and emails don’t necessarily have to be a part of the same email string to be tracked on the right record. You can manually add an existing tracking token to a subject line, and the email will track on the same record as where that token originated.
In Reply To
If a tracking token is not available, the reply information in the email can be helpful to track if it is a part of an ongoing conversation. Each email has a unique message ID. If one of those emails is replied to, the original email ID is captured in the ‘InReplyTo’ field in Dynamics 365 and used for matching against the correct conversation.
A conversation index string is generated for each email conversation, and is used to match the conversation and where in the conversation this email took place.
The initial string is 44 characters long. For each reply in the conversation, another 10 characters gets added to the conversation index. Dynamics 365 also has a field called ‘Base Conversation Hash’. This field is a ten-digit number of the starting conversation index.
This means you can use the ‘Base Conversation Hash’ easily to find all emails tracked as a part of the same conversation.
Smart Matching is enabled and configured in the System Settings Email tab, the same place where the tracking token is configured.
The smart matching feature analyzes the subject key words and the recipients to identify potential matches. The smart matching feature is not enabled by default. A system administrator will need to enable it in the settings.
The subject keywords used are filtered on the RegEx expression in the “Filter subject keywords” setting. ^[\s]*([\w]+\s?:[\s]*)+ can look confusing, but it just means that any word or character combination immediately followed by a colon is not included in the analysis for smart match.
For example, in a subject line “Re: Fw: Re: Questions on service availability,” the “Re: Fw: Re:” part will be excluded when performing the smart match.
While this handles most forward and reply email combinations, there is a catch. If someone uses the subject line “Question: What about service availability?” In this situation, the word “Question” is not included in the smart match because a colon follows it.
Note that if a colon is present later in the subject’s sentence, then the words are included.
e.g. Re: Fw: Re: Question on service availability: Resolved”, then the “:Resolved” is included because it appears later in the sentence.
To test your subject lines, go to an online tool like www.regextester.com and add ^[\s]*([\w]+\s?:[\s]*)+ to the Regular Expression and try out some subject lines.
The RegEx expression in the settings can be modified to fit your specific requirements, if there are certain words you always need to filter out.
The “Maximum difference allowed between subject keywords” setting also allows you to define how precise the match should be and how many words the subjects in two different emails can be off by, but still match appropriately. The default is “0” meaning an exact match. You can make it more lenient by increasing the number, but this also increases the risk of false positives.
The recipients are also used in the smart match feature and at least two matching recipients are needed. You can increase this number if you only want this feature to run on emails that contain a larger number of contacts and thereby exclude conversation between only a couple of people.
The smart match feature can also match two unrelated emails if they happen to have the same subject and recipients. For example, a user sends an email to support that creates a case. The subject line is “Please help”. An hour later, that same user sends another email to the same recipients, also with “Please help” in the subject line. This email may have different content, but smart match could match it regarding the already existing case, because it matched parameters on subject and recipients from the Smart Match settings.
Worth noting that the smart match feature is not space or case sensitive, so “PLEASE HELP”, and “please help” will be matched with each other.
The smart match feature relies on analyzing the email subject and recipients. This is an area that varies from organization to organization and depends a lot on established email processes and culture. You can work on continually optimizing the settings by analyzing the performance of the email matching features.
Tracking Token, InReplyTo, Conversation Index, and Smart Matching are the different ways that Dynamics 365 can match an email to the relevant conversation or record automatically. To the user, the email just seems to magically appear on the correct record. It is possible that you are running into some issues where emails are incorrectly tracked or not tracked on a record at all. When that happens, we can analyze the email records and see exactly how each email got tracked and why it ended where it did in Dynamics 365. All the necessary fields are not available in Advanced Find, so you may need to use a tool like Power BI to get the data.
With the data in Power BI, we can then start adding visuals, drill down on the data, and create visuals around email correlation.