Autopilot (AIOps Module)

Description

(AIOps modules are accessible only to users with the SuperAdmin access level.)

The Autopilot module for AIOps automatically manages administrative configuration tasks within Netreo. Many operational and administrative issues that arise from suboptimal or broken configuration settings can be automatically fixed (or avoided altogether) through the use of this module.

This module monitors and manages the following administrative configuration areas:

• Solution setup - Looks for system-level problems such as incorrect DNS, NTP or mail settings. (On-premise deployments only, not available in SaaS deployments.)
• Environmental integration - Looks for bad credentials within the monitored environment.
• Best practices - Looks for issues in device organization and configuration consistency.
• Data collection - Checks that the correct data is being collected from each type of device.
• Threshold check baselining – Looks for threshold check static values that appear to be configured poorly (potentially resulting in excessive alerts).

Details

The Autopilot module scans for issues at approximately midnight every night. Scan times will vary depending on the number of devices being managed by Netreo.

Autopilot uses a series of plugins to scan for each particular type of administrative configuration issue. These plugins can be switched on or off, as desired.

Once an issue is found by Autopilot, the action taken depends on the fixing behavior configured for the plugin that found the problem. All actions are recorded into the Autopilot activity log.

Autopilot Plugins

Each plugin used by Autopilot is described here, organized by category as they appear in the module's settings.

Baselines

• Threshold Baseline Check
  Checks for devices with static threshold check values that appear to be configured poorly. Tunes the respective threshold values to better represent the actual consistent values for the statistic each check is monitoring.

Best Practices

• Auto-configuration Rule Mismatch
  Checks for devices whose Netreo monitoring configuration doesn't conform to the autoconfiguration rulesets. Runs the autoconfiguration rules against the respective devices.
• Device Templates Status
  Checks for devices that have device template usage switched off. Switches the device template usage setting on for the respective devices.
• Host Alert Contacts Present
  Checks for missing host alert contacts on each device. If no host alert contacts are found for a device, applicable device templates are applied to the respective devices.
  
• Missing Device Organization Information
  Checks for devices that are not assigned to category, site and strategic groups. Runs the autoconfiguration rules against the respective devices.
  

Data Collectors

• All Missing Basic Polling Data
  (Available for Netreo on-premise deployments only.)
  Checks that all performance data specified in a device type is being collected from all managed devices of that type. The results card displays the number of statistics missing for a failed device. To determine which statistics are missing for that device, compare the instances listed on the Performance tab of its Device Dashboard with the pollers listed in its device type.
  
• Device Type Check
  (Available for Netreo on-premise deployments only.)
  Checks that the device type assigned to a monitored device by Netreo matches the device type as reported by SNMP when querying the device. Changes the assigned device type for mismatched devices.
  
• Interface Speed Check
  (Available for Netreo on-premise deployments only.)
  Checks device interfaces for missing in/out bandwidth speeds. If five or more interfaces on a device are missing in or out bandwidth speed data, an exception is output to the Autopilot activity log. Netreo will then attempt to determine and set the correct speeds using a combination of SNMP interrogation of the device and its own database information.
• Missing Basic Polling Data
  (Available for Netreo on-premise deployments only.)
  Checks that appropriate performance data is being collected from all monitored server devices. If the collected data for a device is missing any expected data from the previous 24-hour period, the plugin checks all credentials stored in Netreo and identifies any working credentials that are found. The user may then manually update their device templates or the respective device's monitoring configuration.
• NetFlow Validity Check
  Checks that appropriate traffic flow data is being collected from all monitored devices set as traffic flow exporters. If no data has been collected for a device in the previous 24-hour period, the plugin will schedule the device for an immediate repoll to attempt to remedy the problem.
  
• Selected Polling Data Missing
  (Available for Netreo on-premise deployments only.)
  (No description available.)
• Syslog or SNMP Trap Validity Check
  Checks that appropriate log data is being collected from all monitored devices set as log exporters. If no data has been collected for a device in the previous 24-hour period, the plugin will schedule the device for an immediate repoll to attempt to remedy the problem.
  
• Unlocked Template Thresholds Detected
  Looks for threshold checks assigned to a device by a device template that have any of its configuration settings unlocked (i.e., editable within the device configuration, overriding the settings of the device template that assigned it) and reports all unlocked thresholds found.
  

Environmental Integration

• Device Credential Check
  Looks for devices experiencing an Authentication service check failure (part of Config Manager). Then parses through all device templates to find working credentials for that device. If working credentials are found, the name device template is output to the Autopilot activity log along with the name of the failing device so that an administrator may update the device configuration to include the working credentials. If no working are found, an exception is output to the Autopilot activity log along with the device name and a message that no working credentials could be found.

Solution Setup

• DNS Server Incorrect
  Checks to see if the DNS server is valid. If not, will attempt to find a valid DNS server and configure Netreo to use it. If no valid servers can be found, an exception is output to the Autopilot activity log.
• NTP Server Incorrect
  Checks to see if the NTP server is valid. If not, will attempt to find a valid NTP server and configure Netreo to use it. If no valid servers can be found, an exception is output to the Autopilot activity log.
  
• SMTP Mail Delivery Problem
  Looks for bounce errors in the mail log for the previous 24-hour period and reports them in the Autopilot activity log.

Using Autopilot

Viewing the Summary

To open the Autopilot dashboard go to the main menu and select Administration > AIOps: Autopilot.

The Summary tab of the Autopilot dashboard visualizes the results of the Autopilot scan using three gauge-style indicators. These indicators are Observe, Analyze and Act, and they summarize the recent activity of Autopilot.

• Observe - Shows how many Netreo configuration items were scanned and how many items that require fixing (called findings) were discovered.
• Analyze - Shows how many findings were not automatically fixed by Autopilot, and separates them into items that can be automatically fixed, but weren't, and items that require user intervention.
• Act - Shows how many items were fixed by Autopilot in total, separated into how many were automatically fixed and how many were automatically fixed after manual user authorization.

The Autopilot plugins that detect and fix issues are grouped into clusters of related plugins. Below the indicator displays, the findings are broken down by plugin group. For each group, the number of currently active (unresolved) findings are displayed next to the number of findings that have been remedied (fixed).

Reviewing Active Findings

Autopilot can automatically fix a wide variety of configuration issues in Netreo, but some findings require user intervention. When Autopilot comes across something that it either can't fix or has been instructed not to fix, it flags those findings for review. These are called active findings, and require a user to either approve and apply Autopilot's recommended remedy or fix the issue manually outside of Autopilot.

To view the current list of active findings in Autopilot select the Findings tab and make sure that Finding Sources is selected.

Each active finding is displayed in its own card, which shows the name of the item affected (typically a device), its current host availability status, whether or not Autopilot can fix it, the nature of the issue, and the name of the module and plugin that discovered it.

If a finding can be automatically fixed, but Autopilot has been configured not to, the card will say "Can Be Addressed." If the finding cannot be fixed by Autopilot, the card will say "Needs Review."

To select a finding click on its card. Doing so extends a side panel that displays more detail about the finding and the device involved.

Multiple findings may be selected and deselected simply by clicking on their cards. If multiple findings are selected, the side panel will instead show a summary of the selected findings.

The findings may be filtered in several ways to focus in on certain types of findings.

• Module
  To filter the findings by module use the Modules pull-down menu. By default, findings from all modules are displayed. Click a module in the pull-down menu to select it and deselect the other modules. This will add a filter to show only findings from the selected module. Then, if desired, click on additional modules in the list to add them to the filtered selection.
• Issue Status
  To filter the findings by issue status, click one of the status displays below the Modules pull-down menu. By default, results from all statuses are displayed. Click a status to select it and deselect the other statuses. This will add a filter to show only findings with the selected status. Then, if desired, click on additional statuses to add them to the filtered selection.
• Specific Term
  To filter the findings by a specific term, enter a term in the filter box at the top right of the results display. A realtime filter is applied to the findings and updated as you type. Only a single search term (with no spaces) may be entered.

All filtering of the findings happens in realtime, and any combination of filters may be used together. The currently applied filters are displayed above the findings. To remove all filtering options and display the complete list of findings again, click clear filter.

Fixing Unremedied Findings

Active findings are fixed from the Findings tab (see Reviewing Active Findings above).

Only active findings labeled as "Can Be Addressed" can be fixed from within Autopilot. Findings labeled as "Needs Review" cannot be fixed from within Autopilot and must be addressed appropriately by the Netreo administrator.

To fix active findings:

1. Click on one or more active findings labeled "Can Be Addressed" on the Findings tab to select them.
2. In the side panel select Fix.
3. Autopilot will display the remedy it intends to apply.
   • Selecting the Fix future occurrences automatically option will update Autopilot's plugin configuration to fix these type of findings automatically in the future.
4. Select Submit.

If findings labeled "Needs Review" were selected along with fixable findings, they will be ignored.

If you choose not to fix a finding labeled "Can Be Addressed" it will remain as an active finding and continue to display on the Findings tab. However, after a set period of time passes (configured in Autopilot settings below), it will be automatically fixed even if the plugin fixing behavior is set to Manual.

Configuring Autopilot

To configure Autopilot select the Settings tab.

The Preferences section contains the setting for how long Autopilot will allow a fixable finding to remain active before automatically applying the recommended remedy. Findings will always be automatically fixed after they have been active for this period of time, regardless of what the fixing behavior of the associated plugin is set to (see below).

The Modules section contains the plugins that make up Autopilot, grouped into their respective modules. Settings can be configured for a whole module, or set individually on each plugin.

• If you wish to change a setting for an entire module simultaneously, select the desired setting in the row of the module you wish to change.
• If you wish to change a setting for an individual plugin, click the + next to a module name to expand its plugin list and select the desired setting in the row of the plugin you wish to change.

Changes to settings must be saved to take effect. Click Save Changes to do this.

To set whether a module or plugin will participate in the Autopilot scan: In the ENABLED column, click the switch to select the desired state. ON to enable, OFF to disable.

To exclude devices from the Autopilot scan by functional group: Create one or more functional groups (strategic groups) to exclude and add the desired devices to them. On the Autopilot configuration tab, click the icon in the EXCLUSIONS column of the module or plugin from which you would like to exclude the functional group devices, select the desired functional group(s) in the pop-up dialog and then click Save Changes.

To specify the behavior of Autopilot when a module or plugin reports its findings: In the FIXING column, select the desired behavior from the drop-down selector. The following behaviors are available:

• Automatic - (Default behavior for all modules and plugins.) When the plugin discovers a finding, it will run its prescribed remedy automatically to try and fix the problem. No user action is required and the issue will not be displayed on the Findings tab. (Although, it will still be viewable in the activity log.)
• Manual - When the plugin discovers a finding, it will not run its prescribed remedy automatically. But, will instead flag the finding for review and require the user to apply the remedy manually on the Findings tab using the Fix button. Additionally, selecting Fix Automatically when applying the remedy will reset this value to Automatic for the plugin.
• Disabled - When the plugin discovers a finding, it will not run its prescribed remedy automatically and will not provide the option to apply the remedy manually from within the Findings tab. However, the findings will still be flagged for review. If you do not wish to see findings from this plugin at all, disable the plugin in the ENABLED column.

Setting the behavior of a module automatically sets all plugins in the module to the same behavior. Conversely, setting the behavior of a plugin differently from its module will cause the module behavior to show as Plugin-specific.

Viewing the Autopilot Activity Log

Autopilot maintains a complete log of its actions. To view this log, select the Findings tab and then select Activity Log.