Pynag

OK so using naemon or nagios, you start to have alot of files, and a reliance on the GUI. But we have a package called pynag. It’s well written, does alot, and here, I’m just recapping it’s features.

You can explore yourself (see https://pynag.readthedocs.io/en/latest/introduction.html ):

pynag --help
pynag --examples

It is SQL like:

pynag list WHERE host_name__contains=webarc and object_type=host
object_type          shortname            filename            
--------------------------------------------------------------------------------
host                 webarc01      /etc/naemon/conf.d/uoh/servers/webarc01.cfg
host                 webarc02      /etc/naemon/conf.d/uoh/servers/webarc02.cfg
----------2 objects matches search condition------------------------------------

Example: Listing on hostname

pynag list WHERE host_name__contains=-lb-
pynag list WHERE host_name__startswith=vcl-lb
pynag list WHERE host_name__notcontains=dev

Examples: targeting a specific service:

pynag list  shortname  filename  register WHERE host_name=hostexample.domain   and object_type=service and shortname__contains=https-webmin
shortname            filename             register            
--------------------------------------------------------------------------------
hostexample.domain/https-webmin /etc/naemon/conf.d/uoh/servers/hostexample.cfg 1                   
----------1 objects matches search condition------------------------------------

Examples: Disable a service :

pynag  update set   register=0  WHERE host_name=hostexample.domain and object_type=service and shortname__contains=https-webmin
object_type          shortname            filename            
--------------------------------------------------------------------------------
service              hostexample.domain/https-webmin /etc/naemon/conf.d/uoh/servers/hostexample.cfg
----------1 objects matches search condition------------------------------------
Update these 1 objects ? (y/N) 

Examples: hosts where with Address attribute:

pynag list host_name address WHERE address__exists=True

Examples: What checks do we have unregistered:

pynag list  WHERE   register=0 

Examples: How many hosts are monitored:

<strong> pynag list where object_type=host and register=1 |tail -1
</strong>----------390 objects matches search condition----------------------------------

Examples: How many service checks are there:

pynag list where object_type=service and register=1 |tail -1
----------2606 objects matches search condition---------------------------------

Examples: What is monitored on a specific host:

pynag list shortname where  host_name__contains=example1.domain

Execute a specific check on a host:

pynag execute example3.domain diskspace
# /usr/lib64/naemon/plugins/check_nrpe -H example3.domain  -t 120  -c check_disk
DISK OK - free space: / 2110 MiB (31.75% inode=95%); /tmp 2005 MiB (98.40% inode=100%); /var 10781 MiB (82.79% inode=100%); /boot 1657 MiB (87.37% inode=100%); /home 3745 MiB (98.42% inode=100%);| /=4535MiB;5316;5981;0;6646 /tmp=32MiB;1630;1834;0;2038 /var=2240MiB;10417;11719;0;13022 /boot=239MiB;1517;1707;0;1897 /home=60MiB;3044;3425;0;3806

# service check exited with exit code 0

What is the actual commands behind these checks:

pynag list host_name service_description effective_command_line WHERE host_name=example3.domain
host_name            service_description  effective_command_line
--------------------------------------------------------------------------------
example3.domain null                 /usr/lib64/naemon/plugins/check_ping -H example3.domain -w 3000.0,80% -c 5000.0,100% -p 5
example3.domain /                    /usr/lib64/naemon/plugins/check_http  -I example3.domain   -f follow -S -s 'Campus'
...
----------13 objects matches search condition-----------------------------------

Examples: Delete a service

Actually, this does remove the service, but in my case, as it was a file per service, the empty file was left around.

pynag list  shortname  WHERE shortname__contains='nagios-checker/certdb/example1.com'
pynag delete  WHERE shortname='nagios-checker/certdb/example1.com'

Using a dummy template to create

So, if I prepare a dummy configuration template, and have the hosts all disabled with register=0 and the hostname is HOSTNAME:

pynag list  WHERE host_name=HOSTNAME
object_type          shortname            filename            
--------------------------------------------------------------------------------
host                 HOSTNAME             /etc/naemon/conf.d/uoh/servers/template-newhost.cfg
service              HOSTNAME/Current Load /etc/naemon/conf.d/uoh/servers/template-newhost.cfg
...
----------8 objects matches search condition------------------------------------

Then I can copy this set and change the hostname:

pynag copy SET host_name=newhostname WHERE host_name=HOSTNAME filename=/etc/naemon/conf.d/uoh/servers/newhostname.cfg
pynag SET register=1 WHERE host_name=newhostname

Pynag config

This interrogates your naemon.cfg or nagios.cfg file.

pynag config --get log_file
/var/log/naemon/naemon.log

Pynag livestatus

OK, well, an obvious thing, how to see what is alerting. So my broker_module setting is

pynag config --get broker_module
/usr/lib64/naemon/naemon-livestatus/livestatus.so /var/cache/naemon/live

So I have to specify the socket as for some reason it’s not finding it.

Examples: what hosts are down

<strong> pynag livestatus --socket=/var/cache/naemon/live --get hosts --columns "state address name" --filter "state = 1"</strong>
state                address              name                
--------------------------------------------------------------------------------
1                    student1      student1     
1                    student3      student3     
----------2 objects matches search condition------------------------------------

Examples: what services are alerting

pynag livestatus --socket=/var/cache/naemon/live --get services  --columns "state host_name description plugin_output" --filter "state = 1"
state                host_name            description          plugin_output       
--------------------------------------------------------------------------------
1                    nagios-checker       certcheck//example.com SSL WARNING - Certificate 'example.com' expires in 10 day(s) (2021-03-27 17:14 +0000/GMT).
1                    example1.com      diskspace            DISK WARNING - free space: / 2061 MiB (18.31% inode=97%): /tmp 2005 MiB (98.41% inode=100%): /var 21587 MiB (81.97% inode=99%): /boot 1573 MiB (90.74% inode=100%): /home 3770 MiB (99.06% inode=100%):
1                    example2.com        Current Load         WARNING - load average: 4.22, 5.47, 5.53
----------3 objects matches search condition------------------------------------

OK a great tool. I was wary at first but it’s been dependable. I can see how potentially we can use the python module for some funky flask application.