OK so using naemon or nagios, you start to have alot of files, and a reliance on the GUI. But we have a package called pynag. It’s well written, does alot, and here, I’m just recapping it’s features.
You can explore yourself (see https://pynag.readthedocs.io/en/latest/introduction.html ):
pynag --help
pynag --examplesIt is SQL like:
pynag list WHERE host_name__contains=webarc and object_type=hostobject_type shortname filename
--------------------------------------------------------------------------------
host webarc01 /etc/naemon/conf.d/uoh/servers/webarc01.cfg
host webarc02 /etc/naemon/conf.d/uoh/servers/webarc02.cfg
----------2 objects matches search condition------------------------------------
Example: Listing on hostname
pynag list WHERE host_name__contains=-lb-
pynag list WHERE host_name__startswith=vcl-lb
pynag list WHERE host_name__notcontains=devExamples: targeting a specific service:
pynag list shortname filename register WHERE host_name=hostexample.domain and object_type=service and shortname__contains=https-webminshortname filename register
--------------------------------------------------------------------------------
hostexample.domain/https-webmin /etc/naemon/conf.d/uoh/servers/hostexample.cfg 1
----------1 objects matches search condition------------------------------------
Examples: Disable a service :
pynag update set register=0 WHERE host_name=hostexample.domain and object_type=service and shortname__contains=https-webminobject_type shortname filename
--------------------------------------------------------------------------------
service hostexample.domain/https-webmin /etc/naemon/conf.d/uoh/servers/hostexample.cfg
----------1 objects matches search condition------------------------------------
Update these 1 objects ? (y/N) Examples: hosts where with Address attribute:
pynag list host_name address WHERE address__exists=TrueExamples: What checks do we have unregistered:
pynag list WHERE register=0 Examples: How many hosts are monitored:
<strong> pynag list where object_type=host and register=1 |tail -1
</strong>----------390 objects matches search condition----------------------------------Examples: How many service checks are there:
pynag list where object_type=service and register=1 |tail -1 ----------2606 objects matches search condition---------------------------------
Examples: What is monitored on a specific host:
pynag list shortname where host_name__contains=example1.domainExecute a specific check on a host:
pynag execute example3.domain diskspace# /usr/lib64/naemon/plugins/check_nrpe -H example3.domain -t 120 -c check_disk
DISK OK - free space: / 2110 MiB (31.75% inode=95%); /tmp 2005 MiB (98.40% inode=100%); /var 10781 MiB (82.79% inode=100%); /boot 1657 MiB (87.37% inode=100%); /home 3745 MiB (98.42% inode=100%);| /=4535MiB;5316;5981;0;6646 /tmp=32MiB;1630;1834;0;2038 /var=2240MiB;10417;11719;0;13022 /boot=239MiB;1517;1707;0;1897 /home=60MiB;3044;3425;0;3806
# service check exited with exit code 0What is the actual commands behind these checks:
pynag list host_name service_description effective_command_line WHERE host_name=example3.domainhost_name service_description effective_command_line
--------------------------------------------------------------------------------
example3.domain null /usr/lib64/naemon/plugins/check_ping -H example3.domain -w 3000.0,80% -c 5000.0,100% -p 5
example3.domain / /usr/lib64/naemon/plugins/check_http -I example3.domain -f follow -S -s 'Campus'
...
----------13 objects matches search condition-----------------------------------Examples: Delete a service
Actually, this does remove the service, but in my case, as it was a file per service, the empty file was left around.
pynag list shortname WHERE shortname__contains='nagios-checker/certdb/example1.com'
pynag delete WHERE shortname='nagios-checker/certdb/example1.com'Using a dummy template to create
So, if I prepare a dummy configuration template, and have the hosts all disabled with register=0 and the hostname is HOSTNAME:
pynag list WHERE host_name=HOSTNAMEobject_type shortname filename
--------------------------------------------------------------------------------
host HOSTNAME /etc/naemon/conf.d/uoh/servers/template-newhost.cfg
service HOSTNAME/Current Load /etc/naemon/conf.d/uoh/servers/template-newhost.cfg
...
----------8 objects matches search condition------------------------------------Then I can copy this set and change the hostname:
pynag copy SET host_name=newhostname WHERE host_name=HOSTNAME filename=/etc/naemon/conf.d/uoh/servers/newhostname.cfg
pynag SET register=1 WHERE host_name=newhostnamePynag config
This interrogates your naemon.cfg or nagios.cfg file.
pynag config --get log_file
/var/log/naemon/naemon.logPynag livestatus
OK, well, an obvious thing, how to see what is alerting. So my broker_module setting is
pynag config --get broker_module
/usr/lib64/naemon/naemon-livestatus/livestatus.so /var/cache/naemon/liveSo I have to specify the socket as for some reason it’s not finding it.
Examples: what hosts are down
<strong> pynag livestatus --socket=/var/cache/naemon/live --get hosts --columns "state address name" --filter "state = 1"</strong>state address name
--------------------------------------------------------------------------------
1 student1 student1
1 student3 student3
----------2 objects matches search condition------------------------------------Examples: what services are alerting
pynag livestatus --socket=/var/cache/naemon/live --get services --columns "state host_name description plugin_output" --filter "state = 1"
state host_name description plugin_output
--------------------------------------------------------------------------------
1 nagios-checker certcheck//example.com SSL WARNING - Certificate 'example.com' expires in 10 day(s) (2021-03-27 17:14 +0000/GMT).
1 example1.com diskspace DISK WARNING - free space: / 2061 MiB (18.31% inode=97%): /tmp 2005 MiB (98.41% inode=100%): /var 21587 MiB (81.97% inode=99%): /boot 1573 MiB (90.74% inode=100%): /home 3770 MiB (99.06% inode=100%):
1 example2.com Current Load WARNING - load average: 4.22, 5.47, 5.53
----------3 objects matches search condition------------------------------------OK a great tool. I was wary at first but it’s been dependable. I can see how potentially we can use the python module for some funky flask application.