Operations DB – General
A dedicated maintenance workflow is
executed at Midnight Daily controlled by the RMS and performs the following
actions in this order:-
1.
2.
Grooming is controlled by the Grooming
Settings UI in the Administration console
![](3D"OperationsManagerDataRetentionGrooming_files/image002.jpg")
Unlike MOM 2005 there is no groom block=
ing,
meaning data is groomed as per settings and does not have to be moved to DW
first.
The Table InternalJobHistory shows the grooming history.
Any updates to the Grooming settings ar=
e applied
immediately.
In RTM there were no Index Optimization=
or
Statistics update, in SP1 these workflows now exist and are executed daily =
at
2am relative to the RMS.
For Operations DB installed on SQL
Enterprise Edition, we perform an online re-index, for all other versions of
SQL an index re-organize is performed.
Automatic alert resolution is controlle=
d by
the Alert UI in the administration console.
![](3D"OperationsManagerDataRetentionGrooming_files/image004.jpg")
DW Maintenance – General
DW Maintenance settings e= xist in the table MaintenanceSetting. = The DW contains 1 list of settings per DW instance even if DW contains multiple MG= s.
A dedicated maintenance w= orkflow is executed every minute within the DW to perform the following actions in = this order
- A=
llocate
storage
- I=
ndex
optimize
- S=
tats
updates
- G=
rooming
- A=
ggregation
In SP1 these tasks have a= 30 second threshold. If Data aggregation takes 40 seconds to complete steps 3 and 4 will not execute unt= il the next maintenance run.
Data Aggregation Informat= ion and Grooming settings are stored in the StandardDataSetAggregation Table. Updates to this table take effect immediately.
When grooming, if tables = need to be dropped they are dropped one per Grooming Frequency Period. Therefore if grooming frequency is every 4 hours and 4 tables need to be dropped the grooming will be complete after 16 hours.
Grooming looks at the StandardDataSetTableMap to determ= ine what data exists in each table and will groom accordingly.
SP1 ResKit has a tool cal= led DWDATARP.exe that allows you to view/set the data retention policies for all configured datasets.
Aside from the inbuilt maintenance we also recommend as part of your normal maintenance windows performing a DB Check Integrity and DB Backup.
Data Retention & Grooming
There are two places in t= he DW where we store data retention-related settings.
Config & Instance Data
For "config space&qu= ot; (your management packs, rules they contain, overrides you've created, etc) = and "instance space" (objects discovered, their properties and relationships, etc.) we store setting inside the MaintenanceSetting table. = Here are the columns of interest and their default values:
Instance space settings:<= /p>
1. LastInstanceGroomingDa= teTime - the last time grooming operations were performed;
2. InstanceGroomingFreque= ncyMinutes - frequency of the grooming process start (default: 480)
3. (most important) InstanceMaxAgeDays - maximum age (since the day instance was deleted) for the instance space objects (default: 400)
4. InstanceMaxRowsToGroom= - maximum number of objects to delete in one run (default: 5000).
Config space settings:
1. LastConfigGroomingDate= Time - the last time grooming operations were performed;
2. ConfigGroomingFrequenc= yMinutes - frequency of the grooming process start (default: 60)
3. ManagementPackMaxAgeDa= ys - maximum age for the management pack (since the day MP was uninstalled) (default: 400)
4. NonSealedManagementPackMaxVersionCount - maximum # of non-sealed MP version= s to preserve (independent of the age) (default: 3)
Based on these settings f= or config space, sealed MP will be removed 400 days after it was uninstalled f= rom all management groups that are members of the DW. Non-sealed MPs play by the same rules, but in addition we keep up to 3 old versions of non-sealed MP m= aximum.
Data
Each data type is stored = in a separate structure called "dataset". There is Performance dataset= for perf data, state dataset for monitor state transitions, event dataset for events, etc. etc. MPs may introduce new datasets as well. All datasets in e= xistance known today are so-called "standard datasets". For those, we have= a set of tables that hold description of the dataset including data retention policies. Non-standard datasets may be introduced (we do not know of one to= day though) and they don't have to follow the same rules - data retention setti= ngs for non-standard datasets are dataset specific.
For standard dataset data retention is set at the "aggregation" level. Such that performance "raw" data (samples themselves) stored certain number of days whi= ch may be different from the number of days daily aggregates of performance counters are stored. These settings are stored in the StandardDatasetAggregation table. Here are the columns of interest. Note th= at "primary key" of the table is composite consisting of dataset id = (you can lookup which dataset is which id in the Dataset table and AggregationTy= peId which can be looked up at the AggregationType table). The defaults vary by dataset / aggregation type:
1. MaxDataAgeDays - maxim= um number of days to store data;
2. GroomingIntervalMinute= s - grooming process frequency;
3. MaxRowsToGroom - max n= umber of rows to delete per transaction (see note below);
4. LastGroomingDateTime -= last time grooming process run.
One important not = here is that we do not always groom data row-by row. If data inflow is high (which = is usually the case in medium-to-large organizations for performance and event data) we create additional tables to store data. For example, we store first 10M performance samples in the first table. Once we get more data we leave = the first table there, create second table and start inserting into it. At the = same time we calculate min and max date for the data in the first table (and sto= re it separately in the StandardDatasetTableMap table). Then the grooming proc= ess works like that (for certain dataset/aggregation type comnbination): Check = to see if we have only one table. If one - delete records row-by-row using DEL= ETE TOP and using MaxRowsToGroom parameter. If there is more then one table, fi= nd the table with the oldest "max date" for data in it. If the "max date" is older then retention period - drop entire table if not, leave everything there. So, we do not necessarily "up to date" on grooming all the time. If you have a table which spans one month, we will k= eep some records one month longer then relly needed, but performance gains of dropping whole table vs. row deletes is so huge that we think it is way bet= ter to store a bit more data for a bit longer then to pay the penalty.