Email Application Response Time (EmailART) Monitoring

Description

The email application response time, or EmailART, check monitors the availability and performance of your email applications. The following statistics are collected by the EmailART check:

• Total Response Time
• Total Send Time
• Total Receive Time
• Send Handshake Time
• Receive Handshake Time

It includes a built-in availability service check (passive) and a default threshold check to actively monitor the Total Response Time statistic. Additional threshold checks may be added to monitor other statistics. Multiple mail servers may be monitored from within the same EmailART check.

Details

Netreo's EmailART check makes use of synthetic checks to send test emails and measure the response performance of your mail application. Each synthetic check can monitor a different email application and uses two mail servers: The internal server to check (the receive server) and a (preferably) external server off of which to reflect email (the send server). The send server can be any working email account to which Netreo can connect using SMTP (you may wish to set up a separate email account just for this purpose). By using two servers in this way, Netreo is additionally able to passively verify that a network is able to connect to the internet.

Much like a WebART check, the EmailART check isn't actually a single check itself but rather a collection of related checks designed to monitor the performance of a single target email application. It is the second most complex of the check types (after the WebART check) and, in some ways, is actually more similar to a managed device than a type of check, in that it is a self-contained, discreet entity that gets checked for availability and is polled for multiple types of statistical data. Since the email application check is not associated with any specific host, it doesn't appear in device-based dashboards such as the Tactical Overview. Instead, it gets its own dedicated dashboard.

Although it looks very similar to a WebART check, the email application check is configured and activated very differently. A brief explanation of these differences will be helpful in understanding the application of the email check.

Although visually similar, the underlying principles of the two checks are very different. The implementation of WebART checks assumes that multiple and varied web-based applications are being used and monitored by an organization. Thus, Netreo allows you to create as many WebART checks as required to monitor all of your different web-based applications. Conversely, most organizations typically tend to rely on a single email application to service the entire organization. Thus, Netreo allows you to create only one email application check—as more than one is virtually never necessary.

Another difference is in their use of synthetic checks. Where WebART checks utilize multiple synthetic checks to simulate a complex path of application use, the email application check rarely uses more than a single synthetic check, which forms a core part of the check's configuration. This synthetic check is where the application and server data for connecting to the email service are configured.

Since the email application check monitors an email system by sending test emails in one direction, a second synthetic check may occasionally be added to test the service using the reverse path.

Monitoring

The EmailART check and the WebART check both share the same dashboard functionality for the purposes of monitoring the checks. See the Monitoring section of Web Application Response Time (WebART) Monitoring for information about the dashboards where the EmailART check can be seen.

Configuration and Management

Create an EmailART Check

Creating an email application response time (EmailART) check requires configuration of several distinct parts:

1. The base check (does nothing by itself - just a container to hold the synthetic checks).
2. One synthetic check for each email application you wish to monitor (this part controls the creation and management of the passive availability service check that monitors the availability of the application and also allows Netreo to collect response performance information).
3. (Optional) Any desired threshold checks to monitor the performance of the application (this part allows Netreo to warn about performance issues).

To create a new WebART check in Netreo, follow the steps below.

Create the base check

These steps create the base email application response time check. Only one of these checks may be created in any given Netreo deployment.

1. Log in to Netreo as a user with the SuperAdmin access level.
2. Go to the main menu and select Administration > Modules > Email to navigate to the Email Application Overview page.
3. Select Add Application.
   1. Enter a name for the EmailART check. The value you enter here is used only to identify the check in the edit screen and does not appear anywhere else in Netreo.
   2. Select Create.
4. You are taken to the Mail - Edit page.

By itself, the base check does nothing. You must now add a synthetic check for each email application you would like to monitor using the steps below.

Add a synthetic check

You must first have created the base EmailART check using the steps above.

1. If not already open, navigate to the Mail - Edit page by going to the main menu and selecting Administration > Modules > Email.
2. In the Synthetic Checks section, select Add Custom Email Check.
3. On the page that appears:
   1. In the Check Description field, enter a name for the synthetic check. This is how the check will be identified in reports and dashboard displays.
   2. In the Send section:
      1. In the SMTP SERVER field, enter the domain name of the external mail server that will be used to send the test email.
      2. In the DESTINATION EMAIL field, enter the email address that will be tested.
      3. In the FROM EMAIL field, enter the email address that will be used to send the test email.
      4. Enter the account username and password for the "from email" address above if required.
      5. Check SMTP AUTHENTICATION if required.
         • Most external mail servers require authentication to send mail.
      6. Use OAuth 2.0 - Select the checkbox to use modern OAuth 2.0 authentication  with your send server. (Microsoft is phasing out the use of basic authentication for Office 365, so this option is highly recommended.)
         • If you selected to use OAuth 2.0, enter your OAuth 2.0 credentials in the fields provided.
         • You may use the Test button in the OAuth 2.0 field to test your credentials before saving the check.
   3. In the Receive section:
      1. In the RECEIVE SERVER field, enter the domain name of the internal mail server where the "destination email" resides. This is the email server that you want to check.
      2. Select the RECEIVE SERVER TYPE for that server. (OAuth 2.0  versions are available for POP and IMAP.)
         • If you use Microsoft Office 365 Exchange services for your email, it is highly recommended to use OAuth 2.0, as Microsoft is phasing out the use of basic authentication for Exchange.
      3. If a specific port on the receive server must be used, specify it in the RECEIVE SERVER PORT field.
      4. Enter the account username and password of the "destination email."
      5. Check USE SSL if required by your email server. (This option is selected and disabled automatically if an OAuth 2.0 version is selected for the RECEIVE SERVER TYPE.)
      6. If you selected an OAuth 2.0 version for your RECEIVE SERVER TYPE, enter your OAuth 2.0 credentials in the fields provided.
         • You may use the Test button in the OAuth 2.0 field to test your credentials before saving the check.
   4. Select Save Synthetic Check.
4. Repeat for any additional email applications you wish to monitor.

Your new synthetic check appears in the Synthetic Checks table with an APPLICATION STATUS status of PENDING.

These synthetic checks create and manage the passive service checks used for availability monitoring of the specified email application.

Configure the availability service check

Your new synthetic check must now be left to run for a short time before you may configure any of its settings.

Once the synthetic check's APPLICATION STATUS shows as ACTIVE, you may configure it.

The synthetic check(s) created above are for availability monitoring only. If you wish to monitor the performance of your email application(s) you will need to add threshold checks to each synthetic check by following the steps below.

1. If not already open, navigate to the Mail - Edit page by going to the main menu and selecting Administration > Modules > Email.
2. In the Synthetic Checks section, locate the synthetic check you wish to configure and ensure that its APPLICATION STATUS is ACTIVE.
3. Select the edit icon in the synthetic check's ACTIONS column.
4. In the Custom Application Check panel, configure the settings for the passive availability service check.
   1. In the DESCRIPTION field enter a description for the synthetic check. This field is referenced in the alert notification sent when the check fails.
   2. In the ACTION GROUP field, select the action group(s) to receive alert notifications when the check enters the CRITICAL state, and at each renotification interval.
   3. In the ESCALATION GROUP field, select the action group(s) to receive alert notifications at each renotification interval after escalation.
   4. In the RENOTIFICATION INTERVAL  field, enter the number of minutes for Netreo to wait before sending another alert notification if the incident  is not acknowledged by a user.
      • The default value of 1440 minutes (24 hours) is recommended to minimize alert noise.
      • Setting a value of 0 (zero) will disable renotifications (meaning alerts are sent only once).
   5. In the ESCALATION DELAY field enter the number of alert notifications after the first for Netreo to wait before sending alert notifications to the action groups in the ESCALATION GROUP field as well as to the groups in the ACTION GROUP field.
      • The default value of 1 means that a total of 2 alerts must be sent before escalation groups start receiving them.
   6. In the ALARM HOURS field select a time frame during which alerts may be sent.
   7. In the ALERT AFTER field select the number of failures the check is allowed to experience before sending an alert notification (default is 3).
   8. In the NOTES field, enter any notes that you would like included in an alert notification about this check.
   9. Select Save Synthetic Check.

Add a threshold check to the synthetic check

You must first have added at least one synthetic check to the EmailART base check before you are allowed to add any threshold checks to monitor its performance.

1. If not already open, navigate to the Mail - Edit page by going to the main menu and selecting Administration > Modules > Email.
2. Select the Thresholds tab at the top of the page.
3. In the Add Thresholds panel, select the statistic and synthetic check whose performance you would like to monitor from the pull-down menu.
4. Select Add Threshold.
5. On the Mail- Add Threshold page:
   1. In the ACTION GROUP field, select the action group(s) to receive alert notifications  before escalation.
   2. In the ESCALATION GROUP field, select the action group(s) to receive alert notifications after escalation.
   3. In the RENOTIFICATION INTERVAL  field, enter the number of minutes for Netreo to wait before sending another alert notification if the incident  is not acknowledged by a user.
      • The default value of 1440 minutes (24 hours) is recommended to minimize alert noise.
      • Setting a value of 0 (zero) will disable renotifications.
   4. In the ESCALATE AT field, enter the number of alert notifications after the first for Netreo to wait before sending alert notifications to the action groups in the ESCALATION GROUP field, as well as to the groups in the ACTION GROUP field.
      • The default value of 1 means that a total of 2 alerts must be sent before escalation groups start receiving them.
   5. In the STATISTICAL GROUP field, select the type that has the greatest relevance to the check. This field determines which statistical calculations this check contributes to for reports.
   6. (Optional) If you would like to configure static threshold monitoring:
      1. In the HIGH warning field (yellow), enter the exact value at which the check should enter the WARNING state for high values.
         • Next to the value type, select the multiplier prefix.
      2. In the HIGH critical field (red), enter the exact value at which the check should enter the CRITICAL state for high values.
         • Next to the value type, select the multiplier prefix.
      3. In the LOW warning field (yellow), enter the exact value at which the check should enter the WARNING state for low values.
         • Next to the value type, select the multiplier prefix.
      4. In the LOW critical field (red), enter the exact value at which the check should enter the CRITICAL state for low values.
         • Next to the value type, select the multiplier prefix.
      5. Repeat these steps for each variable if two variables are present.
   7. In the TIME PERIOD field, select the time period over which data values will be sampled for the calculated average.
   8. (Optional) If you would like to configure anomaly detection:
      1. In the Boundary field, select whether to check for upper boundary anomalies, lower boundary anomalies, or both.
      2. In the Sensitivity warning field (yellow), select the desired sensitivity. (This should always be at least one setting higher than the critical sensitivity field so that the warning state occurs first.)
      3. In the Sensitivity critical field (red), select the desired sensitivity. (This should always be at least one setting lower than the warning sensitivity field so that the warning state occurs first.)
      4. In the Season field select the desired season for the data samples.
      5. Repeat these steps for each variable if two variables are present.
      6. (Optional) In the Min Value field set the minimum value that a polled value must be to qualify for anomaly detection.
         • The value entered in this field should be specified in the same base unit displayed in the static threshold configuration without the prefix (for example, bytes, not megabytes; seconds, not milliseconds).Note: For bandwidth monitoring (only), the value must be specified in bits per second and not as a percentage.
   9. Select Create Thresholds.
6. Repeat for any additional threshold checks you would like to add to this synthetic check.

Disable an EmailART Threshold Check

Threshold checks cannot be removed once added; they can only be disabled. To disable a threshold check in your EmailART check, follow the steps below.

1. If not already open, navigate to the Mail - Edit page by going to the main menu and selecting Administration > Modules > Email.
2. Select the Thresholds tab at the top of the page.
3. In the "Web Application Threshold Configuration" section, locate the threshold check you would like to disable.
4. Select its disable switch in the ACTIONS column. Select again to re-enable the check.

Disabling a threshold check only halts performance monitoring of the collected statistics. The statistic values continue to be collected and displayed in the appropriate locations in Netreo.

See Threshold Check to learn about threshold checks and their uses.

EmailART Best Practices

Email Addresses Used for Monitoring

Netreo strongly recommends that you create two dedicated email addresses (one internal and one external) used exclusively for the purpose of this check. These addresses should not be shared or used for any other purpose, as it may interfere with the performance of the check.