Understanding SIEM Logs

Learn about the data returned by the /api/audit/get-siem-logs endpoint.

Mimecast offers an Enhanced Logging feature allowing you to programatically download log file data from your Mimecast service.

The following data types are available:

Email logs

  • Inbound - logs for messages from external senders to internal recipients
  • Outbound - logs for messages from internal senders to external recipients
  • Internal - logs for messages between internal domains

These logs are enabled in the Enhanced Logging section of the Administration | Account | Account Settings menu in the Administration Console. Once enabled the logs are then available using the /api/audit/get-siem-logs function.

The source application of these log files is the Mimecast MTA. The following list describes some of the characteristics of the MTA that impact logging:

  • The MTA runs on many servers in our infrastructure, consequently log files are written on each server that processes an email for your account.
  • As the first point of entry and last point of exit for your organization's email traffic there are 3 stages that each email will go through:
    • Receipt - where the MTA receives a new connection for an email, either from your organization's email infrastructure or the outside world.
    • Process - where Mimecast policies are applied to the email.
    • Delivery - where the MTA delivers the email to it's intended recipient, either to your organization's email infrastructure for inbound messages, or to another mail server for outbound messages.
  • As a result of this, for each email there can be up to 3 or more log lines, 1 for each stage. The MTA holds different pieces of information at each stage, the tables below describe the available information or fields you can expect for each line.
    • For messages where the first delivery attempt fails you can additionally expect another line for each future delivery attempt.
  • These log lines are split by Mimecast when we make log files available for download. Each file you download will contain either receipt, delivery, or process lines.
  • For customers subscribing to our Targeted Threat Protection URL Protect and / or Attachment Protect features, additional log lines are available for malicious activity detected by Mimecast.
    • For URL Protect, a log line is written each time a user clicks a link that has been rewritten by Mimecast in an email and has been found to be malicious.
    • For Attachment Protect, a log line is written for each file processed by the sandbox and found to be malicious.
    • The fields available for each event are documented in the tables below.
  • For each email that passes through the MTA, we maintain a unique ID (aCode) to help correlate log events through each stage of the email's journey.
  • Log data is rolled up and made available for download every 30 minutes throughout the day. As logs are written to all Mimecast MTA servers it is worthwhile checking for new data more frequently, for example every 10 minutes.
  • Log data is stored by Mimecast for 7 days only, however once downloaded you can keep the data for as long as you require. This, combined with our token based system allows for up to 7 days of downtime in your SIEM or data analytics platform.

Field Descriptions

Receipt

Field NameDescription
acc The Mimecast account code for your account.
aCode The unique ID used to track the email through the different log types.
Act The action taken at the receipt stage.
Cphr The TLS Cipher used if the email was received using TLS.
datetime The date and time that the email was received by the Mimecast MTA.
Dir The direction of the email based on the sending and receiving domains.
Error Information about any errors that occurred during receipt.
IP The source IP of the sending mail server.
MsgId The internet message id of the email.
Rcpt The recipient of the email.
headerFrom The sender address found in the from header of the email.
RejCode The rejection code issued if the email was rejected at the receipt stage.
RejInfo The rejection information if the email was rejected at the receipt stage.
RejType The rejection type if the email was rejected at the receipt stage.
Sender The sender of the email.
SpamInfo Information from Mimecast Spam scanners for messages found to be Spam.
SpamLimit The Spam limit defined for the given sender and recipient.
SpamScore The Spam score the email was given.
Subject The subject of the email, limited to 50 characters.
TlsVer The TLS version used if the email was received using TLS.
Virus The name of the virus found on the email, if applicable.
Sample log lines:

Message received successfully:

datetime=2017-05-26T16:47:41+0100|aCode=7O7I7MvGP1mj8plHRDuHEA|acc=C0A0|SpamLimit=0|IP=123.123.123.123|Dir=Internal|MsgId=<messageId@messageId>|Subject=\message subject\|headerFrom=from@mimecast.com|Sender=from@mimecast.com|Rcpt=auser@mimecast.com|SpamInfo=[]|Act=Acc|TlsVer=TLSv1|Cphr=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA|SpamScore=1

Message rejected

datetime=2017-05-26T17:01:36+0100|aCode=cx9u0J0pOJGscX_KPpilkg|acc=C0A0|IP=123.123.123.123|RejType=\Invalid Recipient Address\|Error=\Failed Known address verification\|RejCode=550|Dir=Inbound|headerFrom=|Sender=from@domain.com|Rcpt=auser@mimecast.com|Act=Rej|RejInfo=\Invalid Recipient\|TlsVer=TLSv1|Cphr=TLS_DHE_RSA_WITH_AES_256_CBC_SHA

Process

Field NameDescription
acc The Mimecast account code for your account.
aCode The unique ID used to track the email through the different log types.
Act The action taken at the process stage.
AttCnt The number of attachments on the email.
AttSize The total size of all attachments on the email.
AttNames The filenames of all attachments on the email
datetime The date and time that processing of the email occurred.
Hld The reason the email was held for review (quarantined), if applicable.
IPInternalName For emails subject to Targeted Threat Protection: Impersonation Protect, if the email was detected to be from an internal user name.
IPNewDomain For emails subject to Targeted Threat Protection: Impersonation Protect, if the email was detected to be from a new domain.
IPReplyMismatch For emails subject to Targeted Threat Protection: Impersonation Protect, if the email was detetced to have a mismatch in the reply to address.
IPSimilarDomain For emails subject to Targeted Threat Protection: Impersonation Protect, if the email was detetced to be from a similar domain to any domain you have registered as an Internal Domain.
IPThreadDict For emails subject to Targeted Threat Protection: Impersonation Protect, if the content of the email was detected to contain words in the Mimecast threat dictionary.
MsgId The internet message id of the email.
MsgSize The total size of the email.
Sample Log Lines

Message processed successfully with attachments:

datetime=2017-05-26T19:36:48+0100|aCode=BY81J52RPjSmp7MrubnlZg|acc=C0A0|AttSize=1267|Act=Acc|AttCnt=2|AttNames=\”filename.docx”, “filename2.xlsx”\|MsgSize=2116|MsgId=messageId@mssageId

Message processed successfully without attachments

datetime=2017-05-26T19:36:48+0100|aCode=BY81J52RPjSmp7MrubnlZg|acc=C0A0|AttSize=0|Act=Acc|AttCnt=0|AttNames=|MsgSize=2116|MsgId=messageId@mssageId

Message held for review

datetime=2017-05-26T19:24:18+0100|aCode=015vTYvNN-Wn30v7M5MzNw|acc=C0A0|Hld=Spm|AttSize=0|Act=Hld|IPNewDomain=false|IPReplyMismatch=false|AttCnt=0|IPInternalName=false|AttNames=|MsgSize=56442|MsgId=messageId@mssageId|IPThreadDict=false|IPSimilarDomain=false

Delivery

Field NameDescription
acc The Mimecast account code for your account.
aCode The unique ID used to track the email through the different log types.
AttCnt The number of attachments delivered.
Attempt The count of attempts that the Mimecast MTA has made to deliver the email.
AttSize The total size of attachments delivered.
Cphr The TLS Cipher used on delivery, if the email was sent using TLS.
datetime The date and time delivery was attempted.
Delivered If the email was delivered successfully or not.
Dir The direction of the email based on the sending and receiving domains.
Err Information about any errors that occurred on the delivery attempt.
IP The destination IP address for the delivery attempt.
Latency The time in milliseconds that the delivery attempt took.
MsgId The internet message id of the email.
Rcpt The recipient of the email.
ReceiptAck The receipt acknowledgment message received by Mimecast from the receiving mail server.
RejCode The rejection code, for messages rejected by the receiving mail server.
RejInfo The rejection information, for messages rejected by the receiving mail server.
RejType The rejection type, for messages rejected by the receiving mail server.
Route The Mimecast delivery route used.
Sender The sender of the email.
Subject The subject of the email, limited to 50 characters.
Snt The amount of data in bytes that were delivered.
TlsVer The TLS version used if the email was delivered using TLS.
UseTls If the message was delivered using TLS or not.
Sample Log Lines

Message delivered successfully

datetime=2017-05-26T19:40:33+0100|aCode=9q_HeIHHPYejZTBsnipWmQ|acc=C0A0|Delivered=true|IP=123.123.123.123|AttCnt=0|Dir=Inbound|ReceiptAck=\250 2.6.0 messageId@mssageId [InternalId=25473608] Queued mail for delivery\|MsgId=messageId@mssageId|Subject=\Auto Reply\|Latency=5618|Sender=from@domain.com|Rcpt=auser@mimecast.com|AttSize=0|Attempt=1|TlsVer=TLSv1|Cphr=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA|Snt=28237|UseTls=Yes|Route=\Mimecast Exchange Route

Message delivery failed

datetime=2017-05-26T19:40:06+0100|aCode=ClBDLlnTPH6-T-3KJayNew|acc=C0A0|Delivered=false|Err=\Connection timed out\|RejType=\Recipient server unavailable or busy\|AttCnt=0|Dir=Outbound|ReceiptAck=null|MsgId=messageId@mssageId|Subject=\message subject\|Latency=34848442|Sender=<>|Rcpt=auser@mimecast.com|AttSize=0|Attempt=14|Snt=0|UseTls=No

Targeted Threat Protection URL Protect

Field NameDescription
acc The Mimecast account code that the event has been detected for.
datetime The date and time the click was detected.
reason The reason that the click was blocked.
recipient The recipient of the original message that the link was clicked from.
route The route of the original message that the link was clicked from.
sender The sender of the original message that the link was clicked from.
senderDomain The sender domain of the original message that the link was clicked from.
sourceIp The source IP of the original message that the link was clicked from.
url The URL clicked.
urlCategory The category of the URL that was clicked.
Sample Log Lines

A user has clicked on link that is potentially malicious

datetime=2017-05-26T19:22:37+0100|acc=C0A0|reason=malicious|url=http://bgmtechnology.com.au|route=inbound|sourceIp=123.123.123.123|sender=from@domain.com|recipient=auser@mimecast.com|urlCategory=Blocked|senderDomain=domain.com

Targeted Threat Protection Attachment Protect

Field NameDescription
acc The Mimecast account code that the event has been detected for.
datetime The date and time that the file was detected as malicious.
fileExt The file extension of the malicious file.
fileMime The detected MIME type of the malicious file.
fileName The file name of the malicious file.
IP The source IP of the original message that contained the malicious file.
md5 The md5 hash of the malicious file.
Recipient The recipient of the original message that contained the malicious file.
Route The route of the original message that contained the malicious file.
Sender The sender of the original message that contained the malicious file.
SenderDomain The sender domain of the original message that contained the malicious file.
sha1 The sha1 hash of the malicious file.
sha256 The sha256 hash of the malicious file.
Size The size (in bytes) of the malicious file.
Sample Log Lines

The Mimecast sandbox has detected a potentially malicious file:

datetime=2017-05-23T21:45:21+0100|acc=C1A1|fileName=1XCOLUMN.PVC|sha256=8746bb4b31ab6f03eb0a3b2c62ab7497658f0f85c8e7e82f042f9af0bb876d83|Size=378368|IP=123.123.123.123|Recipient=auser@mimecast.com|SenderDomain=domain.com|fileExt=doc|sha1=a27850da9e7adfc8e1a94dabf2509fc9d65ee7e2|Sender=from@domain.com|fileMime=application/vnd.ms-office|Route=Inbound|md5=7b52770644da336a9a59141c80807f37

Understanding the Logs API

The API endpoint used to download logs is /api/audit/get-siem-logs. This function is designed to ensure that you can:

  • Easily download log data by type.
    • Logs are returned in application/octet-stream which many http clients understand and can easily convert back to text for human or machine consumption.
    • The type and date of the log downloaded is indicated in the Content-Disposition response header.
  • Only download new data each time you make a request.
    • Each time you make a request to the API, a page token is provided in the "mc-siem-token" response header.
    • This value of this field should be used in "token" request body parameter the next time you send a request to this endpoint to ensure that you only get logs written after the last file you downloaded.
  • Easily know when there are no more logs of the given type to download.