How to Integrate monit with yum

Monit is a utility for managing and monitoring processes, programs, files, directories and filesystems on a Unix system. It is often used to restart daemons on failures (e.g., a daemon crashed) or if an abnormal situation occurs (e.g., a daemon consumes too much CPU or RAM). Sometimes it is also used as a “poor man’s IDS” to detect changes into critical files and / or daemon binaries.

However, this functionality does not play well with package managers.

Consider a situation: monit is configured to watch MySQL and restart when it crashes:

check process mysqld with pidfile '/var/run/mysqld.pid'
    start program = "/usr/bin/systemctl start mysql"
    stop program = "/usr/bin/systemctl stop mysql"
    if failed host 127.0.0.1 port 3306 protocol mysql with timeout 10 seconds then restart
    if 5 restarts within 5 cycles then timeout
    depends on mysql_bin

check file mysql_bin with path /usr/sbin/mysqld
    if failed checksum       then unmonitor
    if failed permission 755 then unmonitor
    if failed uid root       then unmonitor
    if failed gid root       then unmonitor

Then, when you run yum update, and there is an update for mysql available, yum will stop mysql daemon, replace its binary, and then start mysql again.

What can go wrong here?

1. monit activates in the middle of the process, detects that mysql is no longer running, and tries to start it.
2. monit detects that /usr/sbin/mysqld‘s checksum is now different, and “unmonitors” MySQL (which means it will not be automatically restarted on crashes).

The solution is obvious: stop monit before yum is run and start it afterwards. If you are OK with stopping and starting monit manually, then everything is great (but note that yum can be called by some external software such as a server control panel).

With apt, the solution is pretty easy: we can use DPkg::Pre-Invoke and DPkg::Post-Invoke hooks to stop and start monit. Unfortunately, I am not aware of similar configuration options for yum. However, yum supports plugins, and we can write a custom plugin to stop and start monit automatically.

import os

import yum
from yum.plugins import TYPE_CORE

requires_api_version = '2.1'
plugin_type = (TYPE_CORE,)

def pretrans_hook(conduit):
    conduit.info(2, 'Stopping monit')
    command = '/usr/bin/systemctl stop monit.service'
    os.system(command)

def posttrans_hook(conduit):
    conduit.info(2, 'Starting monit')
    command = '/usr/bin/systemctl start monit.service'
    os.system(command)

This plugin (/usr/lib/yum-plugins/monit.py) uses two hooks: pretrans (called before yum begins the RPM update transaction), and posttrans (called just after yum has finished the RPM update transaction). The first one is used to stop monit, the second one starts it, it is that simple.

The only things that is left is to enable this plugin. To do that, we create a file called /etc/yum/pluginconf.d/monit.conf with this content:

[main]
enabled=1

Done!

I have also created a GitHub repository with all the code.