INTRODUCTION
The Linux Resource Monitor Facility provides a consistent programmatic access to statistics and create threshold, watermark, and leaky bucket monitors for any instrumented subsystem in the Linux operating system.
The architecture of the Resource Monitor defines five primary components:
// *************************************************************************** // CplusplusExample.cpp - description // This example shows a C++ style setup of a bidirectional threshold monitor // required compile options: -I/usr/include/ResourceMonitor // required link options: -lResourceMonitor -luuid // *************************************************************************** #include <ResourceMonitor.h> #include <RM_ProcessMonitor.h> ResourceMonitor::RMmonitor *CreateVirtualMemoryMonitor(rmValue threshold) { RMuid ssid = RMuid(RM_PROCESS_UUID_STRING); rmMonitorControl moncontrol; rmMonitorConfiguration monconfig; monconfig.statisticKey.SubsystemId = ssid.id; monconfig.statisticKey.ResourceId = getpid(); monconfig.statisticKey.StatisticId = STAT_vsize_ID; monconfig.monitorType = rmThresholding; monconfig.statisticTransform = rmNone; monconfig.RMThresholdType = rmThreshold; monconfig.RMThresholdValue.rmValue32 = threshold.rmValue32; monconfig.RMThresholdCondition = rmValueIsAtOrAbove; monconfig.RMThresholdPrecondition = rmNoPrecondition; monconfig.RMThresholdEventSeverity = LOG_WARNING; monconfig.RMThresholdCancelEventSeverity = LOG_NOTICE; moncontrol.monitorType = rmThresholding; clearMonitorUID(moncontrol); clearDataCapture(moncontrol); moncontrol.location = rmDaemon; moncontrol.monitorId = 0; // unused for daemon monitors moncontrol.monitoringInterval = 0; // monitor until stopped moncontrol.monitoringRate = 5; // check every 5 seconds moncontrol.microMonitoringRate = 0; // check every 5 seconds moncontrol.RMThresholdTolerance.rmValueU32 = 0; // qualifies what "At" or "equal to" is moncontrol.RMThresholdSamples = 2; // log event after 2 consecutive readings above the threshold moncontrol.RMThresholdLoggingRate = 60; // log events no more than once a minute try { ResourceMonitor::RMmonitor *mp = new ResourceMonitor::RMmonitor(monconfig, &moncontrol); mp->StartMonitor(); return mp; } catch(...) { cerr << "failed to create CreateVirtualMemoryMonitor" << endl; return 0; } }
// *************************************************************************** // Cexample.c - description // This example shows a C-style setup of a bidirectional threshold monitor // required compile options: -I/usr/include/ResourceMonitor // required link options: -lResourceMonitor -luuid // *************************************************************************** #include <ResourceMonitor.h> #include <RM_ProcessMonitor.h> #include <stdio.h> rmHandle CreateVirtualMemoryMonitor(rmValue threshold) { rmHandle handle; rmMonitorControl moncontrol; rmMonitorConfiguration monconfig; uuid_parse(RM_PROCESS_UUID_STRING, monconfig.statisticKey.SubsystemId); monconfig.statisticKey.ResourceId = getpid(); monconfig.statisticKey.StatisticId = STAT_vsize_ID; monconfig.monitorType = rmThresholding; monconfig.statisticTransform = rmNone; monconfig.RMThresholdType = rmThreshold; monconfig.RMThresholdValue.rmValue32 = threshold.rmValueU32; monconfig.RMThresholdCondition = rmValueIsAtOrAbove; monconfig.RMThresholdPrecondition = rmNoPrecondition; monconfig.RMThresholdEventSeverity = LOG_WARNING; monconfig.RMThresholdCancelEventSeverity = LOG_NOTICE; moncontrol.monitorType = rmThresholding; clearMonitorUID(moncontrol); clearDataCapture(moncontrol); moncontrol.location = rmDaemon; moncontrol.monitoringInterval = 0; // monitor until stopped moncontrol.monitoringRate = 5; // check every 5 seconds moncontrol.microMonitoringRate = 0; // check every 5 seconds moncontrol.RMThresholdTolerance.rmValue32 = 0; // qualifies what "At" or "NotAt" is moncontrol.RMThresholdSamples = 2; // log event after 2 consecutive readings above the threshold moncontrol.RMThresholdLoggingRate = 60; // log events no more than once a minute handle = rmCreateMonitor( &monconfig, &moncontrol); if( handle == 0 ) perror("rmCreateMonitor"); else rmStartMonitor(handle); return handle; }
// *************************************************************************** // EventingExample.c - description // required compile options: -I/usr/include/ResourceMonitor // required link options: -lResourceMonitor -luuid -levl -lnsl -lpthread // *************************************************************************** #include <ResourceMonitor.h> #include <RM_ProcessMonitor.h> #include <stdio.h> #include <stdlib.h> #define SMALLBUFSIZE 1024 #include <string.h> int eventReceived = 0; rmHandle CreateVirtualMemoryMonitor(rmValue threshold) { rmHandle handle; rmMonitorControl moncontrol; rmMonitorConfiguration monconfig; uuid_parse(RM_PROCESS_UUID_STRING, monconfig.statisticKey.SubsystemId); monconfig.statisticKey.ResourceId = getpid(); monconfig.statisticKey.StatisticId = STAT_vsize_ID; monconfig.monitorType = rmThresholding; monconfig.statisticTransform = rmNone; monconfig.RMThresholdType = rmThreshold; monconfig.RMThresholdValue.rmValueU32 = threshold.rmValueU32; monconfig.RMThresholdCondition = rmValueIsAtOrAbove; monconfig.RMThresholdPrecondition = rmNoPrecondition; monconfig.RMThresholdEventSeverity = LOG_WARNING; monconfig.RMThresholdCancelEventSeverity = LOG_NOTICE; moncontrol.monitorType = rmThresholding; clearMonitorUID(moncontrol); clearDataCapture(moncontrol); moncontrol.location = rmDaemon; moncontrol.monitoringInterval = 0; // monitor until stopped moncontrol.monitoringRate = 5; // check every 5 seconds moncontrol.microMonitoringRate = 0; // check every 5 seconds moncontrol.RMThresholdTolerance.rmValueU32 = 0; // qualifies what "At" or "NotAt" is moncontrol.RMThresholdSamples = 2; // log event after 2 consecutive readings above the threshold moncontrol.RMThresholdLoggingRate = 60; // log events no more than once a minute handle = rmCreateMonitor( &monconfig, &moncontrol); if( handle == 0 ) perror("rmCreateMonitor"); else rmStartMonitor(handle); return handle; } // // The call to posix_log_notify_add registered a query // string and this function as the callback. If the query // evaluates to true for any incoming event, then this // function will be called. Normal application execution // continues after this function returns. // void catchNotificationEvent(int signo, siginfo_t *info, void *ignored) { posix_log_recid_t recid; rmHandle handle = (rmHandle)info->si_int; rmMonitorState state; rmGetMonitorState( handle, &state); // Get the recid of the event that caused this function call. // IBM : we change our posix API - see update posix_log_siginfo_getrecid(info, ignored, &recid); printf(" signal=%d from event: handle=%d, Record ID=%d\n", info->si_signo, info->si_int, recid); rmStopMonitor(handle); rmDeleteMonitor(handle); eventReceived = 1; } int TestVirtualMemoryEvent() { char query_string[SMALLBUFSIZE], error_string[SMALLBUFSIZE]; int once = 0; posix_log_notify_t notifyThresholdHandle; // Remember the monitor's query_handle and its query posix_log_query_t thresholdQuery; // so we can stuff them into the Monitorset to use later. struct sigaction SigRTAction; // IBM's way of defining the signal handler... struct sigevent mynotification; char monitor_string[MAX_GUID_STRLEN]; rmHandle handle = CreateVirtualMemoryMonitor( (rmValue)(u_int32_t)1, 0); rmMonitorControl control; if( handle == 0 ) return 1; // Initialize thresholdNotification with the local function name which is to be the callback function // and set the desired POSIX signal value. If your application only needs one callback registration // then use SIGRTMIN+1. For each additional callback function, you should use different signal values // between SIGRTMIN to SIGRTMAX (a range of 32 values). mynotification.sigev_value.sival_int = (int)handle; // pass monitor handle to event handler mynotification.sigev_notify = SIGEV_SIGNAL; mynotification.sigev_signo = SIGRTMIN + 1; // needs to match the sigaction number below (void) memset(&SigRTAction, 0, sizeof(SigRTAction)); SigRTAction.sa_flags = SA_SIGINFO | SA_RESTART; SigRTAction.sa_handler = (void (*) (int)) catchNotificationEvent; // to use catchNotificationEvent() for this Monitor's events. if (sigaction( SIGRTMIN + 1, // signo_offset can typically default to 0, if &SigRTAction, NULL) < 0) { // you are only using a single listener. perror("sigaction failed for new SIGRTMIN."); return 1; } // Initialize m_notification with the local function name which is // to be the callback function and set the desired POSIX signal // value. If your application only needs one callback registration // then use SIGRTMIN+1. For each additional callback function, you // should use different signal values between SIGRTMIN to SIGRTMAX // (a range of 32 values). // A query string will typically consist of a monitor's RID // e.g. "data contains \"Monitor_ID=cc63f3bb-d74b-41a0-ab46-5d81579c9b11\"" rmGetMonitorControl(handle, &control); uuid_unparse(control.uid, monitor_string); snprintf(query_string,SMALLBUFSIZE,"data contains \"Monitor_ID=%s\" && severity=WARNING",monitor_string); // Register the query string and check it for basic syntax req. if(posix_log_query_create( query_string, POSIX_LOG_PRPS_NOTIFY, &thresholdQuery, error_string, SMALLBUFSIZE) != 0) { perror("could not create threshold query!"); return 1; } if(posix_log_notify_add(&thresholdQuery, &mynotification, once, ¬ifyThresholdHandle)!=0) { perror("Threshold posix_log_notify_add() failed"); posix_log_query_destroy(&thresholdQuery); return 1; } printf("Registered Threshold Notification with POSIX Event logger (handle=%d)\n", notifyThresholdHandle); while(eventReceived==0) { sleep(1); if( ++once > 10 ) return 1; } return 0; } int main(int argc, char *argv[]) { if( TestVirtualMemoryEvent() == 0 ) exit( EXIT_SUCCESS ); exit(1); }
SYNOPSIS
resourcemonitord [ -t rmtab file ] [ -p persist file ] [ -s socket ] [ -r Registry Refresh Rate ] [ -m Self Monitoring Rate ] [ -c daemon CPU limit ] [ -l daemon resident memory limit ] [ -d ]
DESCRIPTION
resourcemonitord provides resource monitoring capability for system statistics made available through subsystem libraries. Use of Unix domain sockets limits monitoring to local resources, i.e. the same node.
OPTIONS
Use /etc/sysconfig/resourcemonitor to set resourcemonitord options. -t resource subsystem file
Subsystem plug-ins are registered with the daemon in this file. Each entry in the file contains the subsystem's universally unique identifier(uid) and the path to the shared library of the plug-in. The default location of this file is /etc/opt/resourcemon/rmtab. -p persistant monitor file
Upon restart, the daemon allows monitors to be recreated automatically from a persistent store. The file rmMonitors contains information necessary to recreate monitors. The default location of this file is /var/opt/resourcemon. -s socket
This option is unsupported on this release. The default socket is /var/run/.RM-socket. -r Registry Refresh Rate
The daemon periodically checks for registration changes in the subsystems and for changes in the rmtab file which contains the library locations for each subsystem. The daemon dynamically loads and unloads subsystem libraries as changes are made to this file. The default location of rmtab is /etc/opt/resourcemon. This refresh rate is frequency in seconds that the daemon checks the file for changes. The default refresh rate is 60 seconds. -m Self Monitoring Rate
The daemon throttles itself when its CPU utilization exceeds the configured limit. When in the throttled state,EBUSY is returned for requests to create new monitors or to start existing monitors. The Self Monitoring Rate is the frequency the CPU and memory utilization is checked. The default value is 15 seconds. -c daemon CPU limit
This is the daemon's percent CPU utilization limit. 74,000 daemon based monitor on 27 statistics use about 8% on a dual CPU 933 MZ Pentium III system. The default value is 5 percent. -l daemon resident memory limit
This is the daemon's resident memory limit in megabytes. 74,000 daemon based monitors use about 50 MB. The malloc/free library tends to fragment memory, so deleteing monitors may not return memory to the system. Therfore, the daemon may remain throttled once this limit is reached. The default value is 25 MB. -d
This option enables debugging of the daemon. With this option, resourcemonitord does not make itself a daemon or close its standard out and error file descriptors.
SIGNALS
resourcemonitord reacts to a set of signals. You may easily send a signal to resourcemonitord using the following:
kill -SIGNAL `cat /var/run/resourcemonitord.pid`
DEBUGGING
When debugging is turned on using -d option then resourcemonitord will be very verbose, writing much of what it does on stdout.
COLLABORATORS
andrew.r.cress@intel.com, todd.c.davis@intel.com, bo.z.li@intel.com, perry.g.vincent@intel.com, and sonic.zhang@intel.com