Managing Data

Monitoring systems such as AppPerfect Agentless Monitor generate vast quantities of data. Effective data management is crucial to ensure a properly functioning monitoring system. This chapter describes the basics of how AppPerfect Agentless Monitor manages its data.

Lets start by first understanding how data is generated. When a device is monitored, at a regular time interval, the AppPerfect Agentless Monitor requests the device for values of the device properties that it is monitoring. This data is processed and stored by the AppPerfect Agentless Monitor in a database. As the number of monitors defined in the system increase, so does the amount of data. If you have several dozen or hundred monitors, the data can accumulate very quickly.

To ensure smooth functioning of the system, AppPerfect Agentless Monitor creates two databases: One for active (latest) data and one for archive (historical) data. Active data is usually for the past 2 days. Archive data can be for as long as user defines. We recommend keeping data no older than 1 month.

AppPerfect Agentless Monitor data can be managed in the following ways:

• While defining monitors, you can set the archiving/purging policies as well as set the frequency with which data should be collected from the monitored devices.
• When updating a monitor, you can edit the archiving/purging policies as well as edit the frequency with which data should be collected from the monitored devices.

Data Archiving Options

Archiving data is a process by which data is moved from an active data store to a secondary data store. To be able to maintain a system that is responsive and stable, it is necessary to archive data that is collected from the monitored devices.

The following fields are required to be filled to set Data Archiving.

1. Archive Data Older than: You can specify how long you want data to reside in your active database, after which time it is automatically archived to the archive database. You can choose from:
   • One hour
   • Eight Hours
   • One day
   • Two days
   • One week

   Once you select the age of the data, the Monitor server's data will automatically get archived after this duration regularly. The default value is: One Day.

2. Consolidate One Record For: This indicates the duration for which you wish to consolidate the data. Consolidation reduces granularity but greatly improves disk space utilization working under the assumption that as data gets older, the lower the need to analyze the data in fine granularity. For example, if you are looking at a historical report for the past month, it is unlikely you would like to see data at a minute granularity. Additionally, lesser the amount of data to store in the archive database, better the performance of the database. With these factors in mind, AppPerfect Monitor implements a concept called data consolidation. As data is being archived, rather than transfer every value from the active to the archive database, a rolling average of data is taken and then transferred from the active to the archive database. For any set of data where a rule violation occurred, the entire data set is given the value of the offending data value. Note despite consolidation, all alerts generated in the system are nevertheless saved. You can select the consolidation time from:
   • One Minute
   • Fifteen Minutes
   • One Hour
   • Two Hours
3. Start Time (in hours): Here you can select the time at which you want data archiving to start at the interval you have specified in the first field. You can select a time where the server is relatively free so that other activities are not affected as data archiving is generally a resource-hungry process.

Data Purging Options

Most organizations have mandates to keep historical data for some period of time. After such a period expires, data is of no use. Such data should be permanently deleted, or purged. AppPerfect Monitor supports data purging based on time interval you specify.

1. Enabled: Select this check box to enable Data Purging and clear it to disable it. By default Purging is enabled.
2. Purge Data Older than: Select the age of the archive data after which you wish to purge it. You can choose from:
   • One Day
   • Two Days
   • One Week
   • Fifteen Days
   • Thirty Days
3. Start Time (in hours): Select the time at which purging should start.