Config Manager

Config Manager

Netreo provides a degree of device config setting management for managed devices through its built-in configuration manager and configuration management rulesets.

To open the Config Manager, go to the main menu and select Quick Views > Dashboards > Config Manager.

The tools available include:

• A config settings monitoring and archiving tool (configuration check) for monitoring and alerting on device config setting changes, and archiving and storing of previous configs.
• A config settings push tool for scheduling config change commands to be executed on groups of managed devices of the same device type.
• Custom config management rules (managed through device templates) for enforcing config settings on specific device types.

Prerequisites for Configuration Management

Device config management is active for all managed devices by default. But, it can only manage devices that use text-style configs (such as most routers, switches, load balancers, and firewalls). Devices that use other forms of config are ignored. Additionally, a device will not be considered eligible for configuration management unless it meets all of the following criteria:

1. The SCHEDULED CONFIG CHECKS setting for that device must be set to ON in its "Advanced" device administration options. (By default, this setting is ON for all managed devices.) Manually switching this setting to OFF will exempt the device from all configuration management.
2. The Netreo device type assigned to the managed device must contain a configuration map that is capable of executing configuration management. (This is not something that can be seen by the user. However, most device types in Netreo that would benefit from configuration management have this mapping included. Contact Netreo support if you have any questions.)
3. The device must have authentication credentials configured in its "Authentication" device administration options.
   • This last criterion can be met by having any password or username and password combination configured. If either of these is present, Netreo will attempt to use them. However, credentials with full administrative privileges for the device are required for the config manager to work properly.

If a managed device meets all of these criteria, its config settings are managed. Otherwise, the config manager ignores it.

The Configuration Check (and Archiving Tool)

The config manager automatically tracks changes to device config settings for all of your eligible managed devices using its configuration check. There is only one configuration check in Netreo. It is built into the config manager (as opposed to the other types of monitoring checks), and it manages all devices at the same time. It does this by downloading a device's current config settings and comparing them to any archived versions Netreo already has stored in its database.

Every night, at 1 A.M., the config manager's configuration check automatically retrieves the device config settings from each eligible managed device and compares them to that device's most recently archived versions. If a change is detected within the most recently retrieved configuration for a device, Netreo will perform several actions:

• The current retrieved config settings are archived.
• Any configuration management rulesets associated with the device are run to force compliance of any incorrect configuration settings.
• An incident is opened and immediately closed. (The incident is only necessary for the purposes of creating a historical record.)
• The event is recorded and displayed in the Config Manager dashboard.
• A custom alert notification containing contextual information about that change is sent out to contacts in the “Default Email Alerts” action group. A different action group can be selected if desired (see below).

If no change in a device's config settings is detected, the retrieved config settings are discarded, and no further action is taken. If config settings are being downloaded from a managed device for the first time, Netreo will save that config in a zip archive, set it as the baseline config for that device, and take no further action until the next configuration check.

By default, the action group used for config manager alert notifications is the “Default Email Alerts” action group. Different action groups can be selected on the Incident Criteria Administration page under the rule “Configuration Change Alerts”. The alert rule itself can also be edited or deleted if desired. However, if deleted, there is no reset! Even though it is a default rule—if it's deleted, it will have to be recreated manually.

Optional Service Checks for Config Manager

While the config manager's configuration check is a singular built-in check with no settings, there are also two optional service checks associated with the config manager: the "Configuration Manager" passive service check and the "Cisco Configuration Save Alert" active service check, which may be added to devices to enhance configuration management.

As with any monitoring check in Netreo, it is recommended to add these checks to managed devices by using device templates.

Configuration Manager Service Check

The Configuration Manager service check is a passive service check added to every managed device by the “Default” device template. This particular service check is only updated by the config manager and is used to alert on a failure of the device's authentication credentials. Any failure by the config manager to retrieve a device's config (scheduled, manual, or triggered) due to failed authentication will cause a WARNING alarm state for this service check (resulting in an alert notification).

It is important to remember that this check is only updated by the config manager. If the config management criteria mentioned above are not all met for a respective device (resulting in the device being ignored by the config manager), this check will always remain in an OK state for that given device (since it's a passive service check). This means that even if the device does have bad credentials, if the config manager is not managing the config settings for it, then this check will never alert you about those bad credentials.

Although this check is directly tied to the config manager, the incident opened by this check because of an authentication failure alarm is completely separate and unrelated to the incident opened by the config manager itself due to a detected configuration change. These two events generally shouldn't happen together for a single device anyway, but the distinction is useful to make for troubleshooting purposes.

By default, this service check uses the “blackhole” action group as the only assigned action group in its alarm configuration. This means that no alert notifications are sent when an authentication failure alarm causes an incident to be opened. However, the alarm is displayed in the "Services" column of any Tactical Overview dashboard widgets, as well as in the Config Manager dashboard (both of which are also represented on the Consolidated Dashboard). Administrators may add or change the action group(s) assigned to this passive service check in the "Default" device template if they wish to receive alert notifications about an authentication failure of the config manager.

Cisco Configuration Save Alert Service Check

(For Cisco devices only.) A "Cisco Configuration Save Alert" active service check can be added to a Cisco device to trigger a config manager configuration check for that device outside of the normal schedule if the "last configuration change" timestamp on the device changes. The use of this service check on a Cisco device provides a closer to realtime response to any configuration changes that might occur, since the "last configuration change" timestamp in a Cisco device is updated anytime a user enters configuration mode while logged in to the device (even if no changes were actually made). However, when the triggered configuration check is run; if no changes are detected in the retrieved config, Netreo will still ignore the event for configuration management purposes. Like all active service checks, this check typically runs every three minutes (although this schedule is adjustable within the check).

Config Manager Page

The Config Manager page is where you manage device configs for the managed devices on your network. Here, you can view configuration management errors, archived versions of device configs, and even compare different versions of a device's saved configs.

The "CalView" tab of this page is also available as the Config Manager widget for custom dashboards.

TimeView Tab

This tab is opened by default when the Config Manager page is opened from the main menu. It shows a list of all device config change events clustered by hour for the selected day.

At the top and bottom of the hour list is a date display. Click this display to select the date you would like to view. On either side of the date display are the previous ( < ) and next ( > ) arrow buttons. Click these buttons to navigate the dates one day at a time.

If any config change events have happened for the selected day, an entry will be present in the hour during which the event occurred. A badge on the left side of the entry displays the number of devices on which config change events occurred.

If only one config change event occurred in that hour, the entry will display the exact event time, along with the device name and the number of changes made to that device. If more than one device experienced config changes that hour, the entry will display the number of changes that occurred and the number of devices affected.

To see the actual changes made to the device configs, select details on the right side of the entry. A pop-up window will appear, showing the exact entries added/removed from the device configuration settings.

CalView Tab

This tab is opened by default when the Config Manager page is opened from its widget on a custom dashboard. By default, it shows device config change events by day of the month.

Options at the top right allow you to display the calendar in month, week, or day mode (month by default). Previous ( < ) and next ( > ) arrow buttons at the top left cycle through the months, weeks, or days, depending on the mode selected. The different modes are similar in appearance to a business appointment calendar and fairly easy to navigate. Selecting a day in month or week mode switches to day mode and shows the selected day. Selecting Today focuses the display on the current day for whatever mode is selected.

Config change events appear as orange entries. To see the actual changes made to the device configs, select an individual entry. A pop-up window appears showing the exact entries added/removed from the device configuration settings.

If there are too many events to display for a day while in month mode, a message indicating “+x more” will be displayed at the bottom of the day. Select that message to open a pop-up window with the list of additional events. To see the actual changes made to the device configs, select details on the right side of the entry in the pop-up window.

ListView Tab

This tab is where you manage archived device configs. It displays a list of config manager events and includes a tool for downloading archived device configs from Netreo's database.

To immediately perform a configuration check for a device, select a device name to open the Configuration Management Report for that device in its Device Dashboard and select the Retrieve Current Configuration button at the top of the page. (Note: Once you’ve opened the Configuration Management Report in a device’s Device Dashboard, all available config manager tabs and features relate to that device only. You will need to navigate back to the main Config Manager page to again view configs for other devices.)

In the ListView tab of the Configuration Management Report, all archived configs for that device are displayed in a table, sorted by date. To view the settings contained in a particular device config, select View in its ACTIONS column. To compare the settings in two different configs, select Compare in its ACTIONS column. You may then flag a specific config as the base and compare another to it by selecting the relevant radio buttons and then selecting Compare Configuration (only two configs may be compared at a time). Any archived version of a device config can be compared with any other, and any changes between versions will be highlighted.

Download Configuration Archives

You can download archived device configs by selecting the Download Configuration Archives button. On the Config Download page that opens, you can select which devices to download config archives for and whether to download all archived configs for the selected devices or just the newest. The configs are provided in a compressed archive (.zip) with the collected configs for each device in separate directories.

Advanced Configuration Push

The Advanced Configuration Push Tool allows you to schedule a one-time push of configuration command sets to any device managed by the Netreo config manager.

Select the Advanced Configuration Push button to open the Configuration Push page.

The table displays scheduled command set pushes along with their current state. The list can be filtered by description, user, or status. Multiple pushes scheduled at the same time are executed sequentially.

The STATUS column of the table displays the current state of the respective configuration command set.

• created - The configuration command set job has been created, but the matching CSV file has not yet been uploaded.
• configured - The config push job has been scheduled and is awaiting execution.
• queued - The config push job is currently trying to execute, but another job is ahead of it in line.
• executed - The config push job has successfully executed all tasks in the set.
• failed - The config push job has begun execution, but at least one task in the job set did not complete before the TIMEOUT period expired. Check output for details (see below).
• missedexecution - The config push job has been scheduled, but more than 5 minutes have passed since the scheduled execution time. This can happen if there is too large a backlog of config push jobs trying to run simultaneously.

Output
To see the output of an executed command set (whether successful or failed), select the upload CSV icon in the command set's ACTIONS column. At the bottom of the Configuration Upload page is a list of devices. Select the view icon in the ACTIONS column of the device for which you would like to see the output to open a pop-up dialog containing the configuration commands and their respective outputs.

Schedule a Configuration Push

Follow the steps below to schedule the one-time execution of a set of configuration commands on a chosen set of devices of a specific type. Once a command set is scheduled, it will run once and remain in the scheduled list for reference until deleted manually.

1. Log in to Netreo as a user with the Admin access level or higher.
2. From the main menu, select Quick Views > Config Manager to navigate to the Config Manager page.
   
3. Select Advanced Configuration Push to open the Configuration Push page.
   
4. On the Configuration Push page, select New Configuration to open the Download Configuration CSV page.
5. On the Download Configuration CSV page:
   
   1. In the DEVICE TYPE field, use the drop-down selector to choose the type of device to which to apply this command set.
   2. In the DEVICE field that appears, select the specific device or devices to which you would like the command set to be applied (multiple selection is allowed).
   3. In the DESCRIPTION field, enter a descriptive name for this command set.
      • This name will be displayed on the command set table on the Configuration Push page.
   4. In the COMMAND field, create a commandlet by entering the set of commands that you want to be executed on the configuration of the above devices.
      • Connect and disconnect commands are not required in the commandlet. Netreo will handle these on its own.
      • Begin the commandlet with a command to enter configuration mode for the device. End the commandlet with a command to exit configuration mode. For each deeper level of context you go (for example, accessing a specific interface), a matching exit command must also be included.
      • To create a variable in the entered commandlet that will be automatically set to different values for each device, use the format $variableName$ in each location where the variable name should be replaced by its appropriate value. For each unique variable name used in the commandlet, a column will be added to the CSV file you will shortly download, allowing you to enter whatever value is required in the row for each device.
        • Even if you didn't create any new variables, you must still download and then upload the resulting CSV file.
      • No error checking is done on the configuration commands used. Therefore, it is imperative that you double-check the commands you enter to avoid doing something destructive.
      • If a command fails, the commandlet process will halt and return the output up to that point in the device list of the Configuration Upload page (see below).
   5. Select Download CSV.
   6. Select a location to save the file and select Save.
   7. You are then redirected back to the Configuration Push page, where your command set is created and added to the schedule table with a status of "created".
      • If you created any variables in your commandlet, you must now edit the downloaded CSV file and add the necessary values for each device. Simply open the file in your spreadsheet program of choice, fill in the required values for each device, and save the file. Remember to save or export as a CSV.
6. Once your CSV file has been properly edited (if necessary), it's time to upload the file back into Netreo.
   1. On the Configuration Push page, select the upload CSV icon in the ACTIONS column of your created config command set to open the Configuration Upload page.
      
   2. Select Browse next to the CSV UPLOAD field, locate your edited CSV file, and select Open.
      • Your CSV file will be uploaded after you've finished configuring the push schedule.
   3. In the TIMEOUT field, enter the number of minutes Netreo will wait for a command to finish executing before the push fails.
   4. Using the SCHEDULE TIME fields, schedule a time for your config command set to be pushed to its respective devices.
      • Click in the left field to select a date, and then click in the right field to select a time.
   5. Select Upload CSV to upload your CSV file and schedule your config command set to be pushed.
   6. You are then redirected back to the Configuration Push page where your config command set is scheduled and is updated with a status of "configured."

When the scheduled time occurs, your config command set will be pushed to each device configuration, and the command set's status will be updated to "executed."

The command set will then remain in the list for reference but can be deleted if you want.

Config Manager Failures Report

If the configuration manager service check (see above) for any given device has failed, you can view these events by selecting the Config Manager Failures Report button. (Remember that this service check fails on any config manager failures—not just authentication.) Click the button to see a list of config manager alarms for the past 24 hours. If there are any failures, we recommend that you check the authentication credentials configured for the respective devices (verifying that there are no access lists or firewalls preventing the Netreo appliance from reaching the device via Telnet or SSH) and then attempt to download a configuration file manually (by using the Download Configuration Archives tool) to view any errors that are occurring.

Search Latest Configurations

The search tool allows you to search through the most recently downloaded device configs managed by Netreo. Select the Search Latest Configurations button to open the search page.

This tool was written primarily for Cisco IOS and may not work properly with non-Cisco configs. Note that the search tool searches all of the most recent device configs during each search.

The Config Search tool has two fields, explained below.

• CONTEXT
  Enter a regular expression to specify the context under which to search. In a Cisco device configuration file, a "context" is basically a subsection of the file containing related configuration parameters (for example, "interface fastethernet 1"). When searching for a context nested within another context, only specify the immediate context level you want to search. It is not necessary to include any higher-level contexts. Leave the default ".*"  to not match a context and search the entire file.
• LINE
  Enter a regular expression to specify the configuration parameter to look for within the context specified above (for example, "ip flow ingress"). In a Cisco device configuration file, configuration parameters within a context are indented to identify that they belong to that context.

Additional filters can be added to the search parameters by selecting Add Filter. By doing this, you can search for lines in more than one context at a time.

Config Management Rulesets

To add a configuration management ruleset to a device template, follow the steps below.

1. Log in to Netreo as a user with the SuperAdmin access level.
2. Go to the main menu and select Administration > Templates to open the Device Templates Administration page.
3. Locate the device template to which you would like to add a logging rule and select its edit icon in the ACTIONS column.
4. In the Template Components panel, locate the Configuration Management Ruleset section.
5. Select the plus (+) button to configure a new ruleset.
   
6. On the page that follows:
   
   1. In the TITLE field, enter a name for your ruleset. All configuration ruleset names must be unique.
   2. In the CONTEXT field, enter the section of the device configuration that should be evaluated by the ruleset (you can use a regular expression here to affect multiple sections of the configuration. if necessary). To match the top-level context (i.e. no context) enter ^$. This field is used in the commandlet below.
   3. In the RULE(S) section, add conditions that represent the desired state of the device config (regex is allowed in the value field). Click the plus button on the right to add additional conditions (additional conditions use a logical AND, so be careful when constructing your conditions).
      • "Must Have" means that if the device config does not contain the value to the right of the condition, execute the ACTION.
      • "Must Not Have" means that if the device config does contain the value to the right of the condition, execute the ACTION.
      • In the example above, if any device configs affected by this template do not contain the value "keepalive 10" under the specified context "interface GigabitEthernet1\/0\/2", the action is executed on that device config. In this case, the action adds that value to that context.
   4. (Optional) In the ACTION  field, enter the set of commands (commandlet) that you want to be executed on the device configuration exactly as you would type them on the command line.
      • Connect and disconnect commands are not required; Netreo handles these on its own.
      • Begin the commandlet with a command to enter configuration mode for the device. End the commandlet with a command to exit configuration mode. For each deeper level of context you go (for example, accessing a specific interface) a matching exit command must also be included.
      • When specifying the context in the commandlet, enter $CONTEXT$. Netreo then replaces this with the value entered in the CONTEXT field (see example in image above).
      • No error checking is done on the configuration commands used. Therefore, it is imperative that you double-check the commands you enter to avoid doing something destructive.
7. Click the Add Rule button.
   • You can add as many config management rulesets to your device template as necessary. When finished, you do not need to click the Update button on the template's edit page. This is only required when editing authentication credentials.
8. Now follow the instructions in Re-Apply Device Templates After Editing.

If no commands are entered in the ACTION field of the ruleset, any violations of the ruleset are still picked up by the configuration check and processed as if a configuration change had occurred—even though no changes have actually been made. This means that the violation event is displayed in the config manager dashboard, and an alert notification is sent out to contacts in the “Default Email Alerts” action group or the action group specified in the Configuration Change Alerts incident management rule. It is then up to an administrator to rectify the configuration of the device manually, if necessary.

If you've added config management rulesets to a device template that is already applied to any devices, navigate back to the Device Templates Administration page using the arrow icon at the top left of the page and reapply your device templates.