When activating SNMP on an IBM IMM module do not panic when SNMP is not working right after the restart. Go get yourself some coffee. And while your at it get something to eat. Or clean
the coffeemaker or do something else for about 15 to 20 minutes. Then try queering the module via SNMP again.
If can access your Icinga / Nagios installation from the internet and you have an android phone you should take a look a aNag, a really nice App to interact with one or more Icinga / Nagios installations. You get alerts and acknowledge them, schedule downtimes, …
I’ll be giving a three hour Wireshark tutorial and a 1hour presentation about check_mk (and some other Icinga / Nagios related tools) at this years FrOSCon on August 25th and 26th. Both are in German. The tutorial is at a beginner level and hands on. So bring your own notebook. I’ll also explain some other network troubleshooting techniques.
Most of my work is done on the command line. My mail (Gnus) , IRC (irssi) and jabber client (irssi + bitlebee) are all running inside a screen session. I now added
icli to another screen window to keep track of my monitoring
Just calling icli will list all checks, I usualy just filter for service that are either warning or critical (or not okay) using the option -z \!o.
root@mon:~# icli -z \!o
file.example.com
APT-Status WARNING APT WARNING: 13 packages available for upgrade (0 critical updates).
web.example.net
SSH CRITICAL Server answer:
dev.example.com
APT-Status WARNING APT WARNING: 31 packages available for upgrade (0 critical updates).
web1.example.net
HTTP WARNING HTTP WARNING: HTTP/1.0 404 Not Found - 869 bytes in 0.123 second response time
blog.example.net
WordPress example.net CRITICAL Your core is out of date! 3.4 -> 3.4.1
Now it’s time to add some host, service and contact groups to our check_mk config. You can configure those either within main.mk or using the normal Nagios / Icinga configuration.
I’ll use the configuration in main.mk for the following examples. To create groups you can use tags so mark
a specific host as member of a specific group. You can either manually create the groups or let check_mk take care of defining the groups.
define_hostgroups = True
define_servicegroups = True
define_contactgroups = True
host_groups = [
( 'NET', [ 'NET' ], ALL_HOSTS ),
( 'WEB', [ 'WEB' ], ALL_HOSTS ),
( 'INTERNAL', [ 'INTERNAL' ], ALL_HOSTS ),
]
host_contactgroups = [
( 'NET', [ 'NET' ], ALL_HOSTS ),
( 'WEB', [ 'WEB' ], ALL_HOSTS ),
( 'INTERNAL', [ 'INTERNAL' ], ALL_HOSTS ),
]
service_contactgroups = [
( 'NET', [ 'NET' ], ALL_HOSTS ),
( 'WEB', [ 'WEB' ], ALL_HOSTS ),
( 'INTERNAL', [ 'INTERNAL' ], ALL_HOSTS ),
]
A rather short post on check_mk today. Sometimes it is necessary to adjust warning and critical levels or disable automatically discovered checks.
The following example shows how to adjust warning and critcal levels for one of our servers:
check_parameters += [
( { "levels" : (90, 95)}, [ "www.example.com" ], [ "fs_/" ] ),
]
This example shows how to disable a check completely:
ignored_services = [
( [ "www.example.com" ], [ "Vmalloc address space" ] )
]
After a short introduction too check_mk we’ll now continue configuring our monitoring system. I’ll only use Icinga for this example, but everything should work with Nagios as well.
In the previous part of this tutorial we defined three hosts:
* router.example.com
* www.example.com
* www.example.net
As mentioned in a previous last post I’m currently moving my monitoring setup from Nagios to Icinga and check_mk. This is the first of a series of articles about check_mk. I assume you are already have a running Nagios / Icinga setup.
Check_mk generates a Nagios / Icinga configuration for a given set of hosts and makes configuring large Setups quite easy. There is also a check_mk agent which allows it to automatically discover informations on the host where the agent is installed. Last but not least check_mk makes it quite easy to check host via SNMP.
After installing check_mk on your Nagios / Icinga Host you’ll add your host to the check_mk configuration file called main.mk.
On Debian (Squeeze) you can install via apt-get. I assume that you’ll want to monitor the monitoring server as well (This might lead to some problems if you haven’t installed Nagios / Icinga via apt-get.):
apt-get install check-mk-server check-mk-client
After the installation you can edit the main configuration file /etc/check_mk/main.mk Here is a small example:
all_hosts = [
'router.example.com|snmp|net',
'www.example.com|linux',
'www.example.net|PING',
]
There is a group all_hosts which contains the host which might be followed by one or more tags. Tags can be used to deiced how a host is queried (either PING, SNMP or using the check-mk agent. The check_mk client is the default method). Tags can also be used to create host and service groups, define legacy check (read classic Nagios checks) and so on.
So router.example.com will be quiried via SNMP, www.example.com via the check_mk agent and www.example.net will only be pinged.
After editing main.mk you can use check_mk -II to discover services on all the hosts specified in main.mk. After that check_mk -O generates the Nagios / Icinga configuration and restarts Nagios.
The agent is available for different operating system, including different Linux distributions, Solaris, Windows. The agent needs xinetd which should be installed automatically when you install the check_mk_agent with your distributions package management system. At least on Debian check_mk is disabled in /etc/xinetd.d/check_mk . Change it (one of the last lines) and restart xinetd.
So now all you have to do is install agents on different host and add those hosts to your main.mk.
Next up:
Part II: SNMP configuration and service checks, check_mk Options
Part III: Adjusting warning and critical levels and disabling checks
Part IV: Host and service groups and contacts
Part V: check_mk and monitoring of dual stacked hosts
Part VI: check_mk livestatus and the check_mk GUI
