MONIT HOW TO INSTALL AND CONFIGURE ON CENTOS 6+ 64 BIT SYSTEMS

How to install and configure Monit on our CentOS 6+ servers

The following tutorial is just a basic install and configuration for Monit on our specific servers with our specific configuration which is currently CentOS 6.7 with WHM/cPanel.


For those wanting more in depth information regarding Monit one will want to visit their official web site https://mmonit.com/monit/
Our deployment of monit is in an effort to combat the "nobody" Apache attacks on our servers.

Now lets get started ...

By default the Monit tool is not available to be installed from the OS base repositories. You will need to add and enable third party EPEL repository (if not already available) to install the monit package on your RHEL/CentOS systems

Starting with the installation of the EPEL repsoitory, the first thing to do is to get an ssh session going so that one can enter the commands (CLI) for installation.

Next we want to go ahead and install the EPEL repository if not already installed by entering:

# wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

# rpm -ivh epel-release-6-8.noarch.rpm

Now that the EPEL repository is installed now we want to install the Monit script itself which is easily done using YUM:

# yum install monit

Once the Monit script is installed the next thing one want to do is to setup the configuration BEFORE starting Monit.

The configuaration file for Monit is located in the /etc/monit.conf and below we have included the basic configuration setup meant to fight Apache nobody attacks ~or~ more accurately when an attack is on spawning many "nobody" processes in which once they cause a certain load we want to bump them all off by killing the processes they spawned. 

This type of type of attack appears to be a DDoS type of attack and Monit can handle the job automatically. Most of the settings below are default and the important one here other then making a user name and password ( at the end of the Global section just before the services section ) is the execution every minute of the apachenobody.sh script when the load reaches greater then 4:

###############################################################################
## Monit control file
###############################################################################
##
## Comments begin with a '#' and extend through the end of the line. Keywords
## are case insensitive. All path's MUST BE FULLY QUALIFIED, starting with '/'.
##
## Below you will find examples of some frequently used statements. For 
## information about the control file and a complete list of statements and 
## options, please have a look in the Monit manual.
##
##
###############################################################################
## Global section
###############################################################################
##
## Start Monit in the background (run as a daemon):
#
set daemon  60              # check services at 60 seconds intervals
#   with start delay 240    # optional: delay the first check by 4-minutes (by 
#                           # default Monit check immediately after Monit start)
#
#
## Set syslog logging. If you want to log to a standalone log file instead,
## specify the full path to the log file
#
set logfile syslog

#
#
## Set the location of the Monit lock file which stores the process id of the
## running Monit instance. By default this file is stored in $HOME/.monit.pid
#
# set pidfile /var/run/monit.pid
#
## Set the location of the Monit id file which stores the unique id for the
## Monit instance. The id is generated and stored on first Monit start. By 
## default the file is placed in $HOME/.monit.id.
#
# set idfile /var/.monit.id
#
## Set the location of the Monit state file which saves monitoring states
## on each cycle. By default the file is placed in $HOME/.monit.state. If
## the state file is stored on a persistent filesystem, Monit will recover
## the monitoring state across reboots. If it is on temporary filesystem, the
## state will be lost on reboot which may be convenient in some situations.
#
# set statefile /var/.monit.state
#
## Set the list of mail servers for alert delivery. Multiple servers may be 
## specified using a comma separator. If the first mail server fails, Monit 
# will use the second mail server in the list and so on. By default Monit uses 
# port 25 - it is possible to override this with the PORT option.
#
# set mailserver mail.bar.baz,               # primary mailserver
#                backup.bar.baz port 10025,  # backup mailserver on port 10025
#                localhost                   # fallback relay
#
#
## By default Monit will drop alert events if no mail servers are available. 
## If you want to keep the alerts for later delivery retry, you can use the 
## EVENTQUEUE statement. The base directory where undelivered alerts will be 
## stored is specified by the BASEDIR option. You can limit the queue size 
## by using the SLOTS option (if omitted, the queue is limited by space
## available in the back end filesystem).
#
# set eventqueue
#     basedir /var/monit  # set the base directory where events will be stored
#     slots 100           # optionally limit the queue size
#
#
## Send status and events to M/Monit (for more informations about M/Monit 
## see http://mmonit.com/). By default Monit registers credentials with 
## M/Monit so M/Monit can smoothly communicate back to Monit and you don't
## have to register Monit credentials manually in M/Monit. It is possible to
## disable credential registration using the commented out option below. 
## Though, if safety is a concern we recommend instead using https when
## communicating with M/Monit and send credentials encrypted.
#
# set mmonit http://monit:monit@192.168.1.10:8080/collector
#     # and register without credentials     # Don't register credentials
#
#
## Monit by default uses the following format for alerts if the the mail-format
## statement is missing::
## --8<--
## set mail-format {
##      from: monit@$HOST
##   subject: monit alert --  $EVENT $SERVICE
##   message: $EVENT Service $SERVICE
##                 Date:        $DATE
##                 Action:      $ACTION
##                 Host:        $HOST
##                 Description: $DESCRIPTION
##
##            Your faithful employee,
##            Monit
## }
## --8<--
##
## You can override this message format or parts of it, such as subject
## or sender using the MAIL-FORMAT statement. Macros such as $DATE, etc.
## are expanded at runtime. For example, to override the sender, use:
#
# set mail-format { from: monit@foo.bar }
#
#
## You can set alert recipients whom will receive alerts if/when a 
## service defined in this file has errors. Alerts may be restricted on 
## events by using a filter as in the second example below.
#
# set alert sysadm@foo.bar                       # receive all alerts
#
## Do not alert when Monit starts, stops or performs a user initiated action.
## This filter is recommended to avoid getting alerts for trivial cases.
#
# set alert your-name@your.domain not on { instance, action }
#
#
## Monit has an embedded HTTP interface which can be used to view status of 
## services monitored and manage services from a web interface. The HTTP 
## interface is also required if you want to issue Monit commands from the
## command line, such as 'monit status' or 'monit restart service' The reason
## for this is that the Monit client uses the HTTP interface to send these
## commands to a running Monit daemon. See the Monit Wiki if you want to 
## enable SSL for the HTTP interface. 
#
set httpd port 2812 and
#    use address localhost  # only accept connection from localhost
#    allow localhost        # allow localhost to connect to the server and
    allow admin:monit      # require user 'admin' with password 'monit'

###############################################################################
## Services
###############################################################################
##
## Check general system resources such as load average, cpu and memory
## usage. Each test specifies a resource, conditions and the action to be
## performed should a test fail.
#
check system $HOST
    if loadavg (1min) > 4 then exec "/etc/apachenobody.sh"
#    if loadavg (5min) > 2 then alert
#    if cpu usage > 95% for 10 cycles then alert
#    if memory usage > 75% then alert
#    if swap usage > 25% then alert
#
#    
## Check if a file exists, checksum, permissions, uid and gid. In addition
## to alert recipients in the global section, customized alert can be sent to 
## additional recipients by specifying a local alert handler. The service may 
## be grouped using the GROUP option. More than one group can be specified by
## repeating the 'group name' statement.
#    
#  check file apache_bin with path /usr/local/apache/bin/httpd
#    if failed checksum and 
#       expect the sum 8f7f419955cefa0b33a2ba316cba3659 then unmonitor
#    if failed permission 755 then unmonitor
#    if failed uid root then unmonitor
#    if failed gid root then unmonitor
#    alert security@foo.bar on {
#           checksum, permission, uid, gid, unmonitor
#        } with the mail-format { subject: Alarm! }
#    group server
#
#    
## Check that a process is running, in this case Apache, and that it respond
## to HTTP and HTTPS requests. Check its resource usage such as cpu and memory,
## and number of children. If the process is not running, Monit will restart 
## it by default. In case the service is restarted very often and the 
## problem remains, it is possible to disable monitoring using the TIMEOUT
## statement. This service depends on another service (apache_bin) which
## is defined above.
#    
#  check process apache with pidfile /usr/local/apache/logs/httpd.pid
#    start program = "/etc/init.d/httpd start" with timeout 60 seconds
#    stop program  = "/etc/init.d/httpd stop"
#    if cpu > 60% for 2 cycles then alert
#    if cpu > 80% for 5 cycles then restart
#    if totalmem > 200.0 MB for 5 cycles then restart
#    if children > 250 then restart
#    if loadavg(5min) greater than 10 for 8 cycles then stop
#    if failed host www.tildeslash.com port 80 protocol http 
#       and request "/somefile.html"
#    then restart
#    if failed port 443 type tcpssl protocol http
#       with timeout 15 seconds
#    then restart
#    if 3 restarts within 5 cycles then unmonitor
#    depends on apache_bin
#    group server
#    
#    
## Check filesystem permissions, uid, gid, space and inode usage. Other services,
## such as databases, may depend on this resource and an automatically graceful
## stop may be cascaded to them before the filesystem will become full and data
## lost.
#
#  check filesystem datafs with path /dev/sdb1
#    start program  = "/bin/mount /data"
#    stop program  = "/bin/umount /data"
#    if failed permission 660 then unmonitor
#    if failed uid root then unmonitor
#    if failed gid disk then unmonitor
#    if space usage > 80% for 5 times within 15 cycles then alert
#    if space usage > 99% then stop
#    if inode usage > 30000 then alert
#    if inode usage > 99% then stop
#    group server
#
#
## Check a file's timestamp. In this example, we test if a file is older 
## than 15 minutes and assume something is wrong if its not updated. Also,
## if the file size exceed a given limit, execute a script
#
#  check file database with path /data/mydatabase.db
#    if failed permission 700 then alert
#    if failed uid data then alert
#    if failed gid data then alert
#    if timestamp > 15 minutes then alert
#    if size > 100 MB then exec "/my/cleanup/script" as uid dba and gid dba
#
#
## Check directory permission, uid and gid.  An event is triggered if the 
## directory does not belong to the user with uid 0 and gid 0.  In addition, 
## the permissions have to match the octal description of 755 (see chmod(1)).
#
#  check directory bin with path /bin
#    if failed permission 755 then unmonitor
#    if failed uid 0 then unmonitor
#    if failed gid 0 then unmonitor
#
#
## Check a remote host availability by issuing a ping test and check the 
## content of a response from a web server. Up to three pings are sent and 
## connection to a port and an application level network check is performed.
#
#  check host myserver with address 192.168.1.1
#    if failed ping then alert
#    if failed port 3306 protocol mysql with timeout 15 seconds then alert
#    if failed port 80 protocol http
#       and request /some/path with content = "a string"
#    then alert
#
#
## Check a network link status (up/down), link capacity changes, saturation
## and bandwidth usage.
#
#  check network public with interface eth0
#    if failed link then alert
#    if changed link then alert
#    if saturation > 90% then alert
#    if download > 10 MB/s then alert
#    if total upload > 1 GB in last hour then alert
#
#
## Check custom program status output.
#
#  check program myscript with path /usr/local/bin/myscript.sh
#    if status != 0 then alert
#
#
###############################################################################
## Includes
###############################################################################
##
## It is possible to include additional configuration parts from other files or
## directories.
#
#  include /etc/monit.d/*
#

# set daemon mode timeout to 1 minute
set daemon 60
# Include all files from /etc/monit.d/
include /etc/monit.d/*

********************************* End of Script - Don NOT include this line with asterisks ******************************************

Note the important configuration parts are just above and below the "## Services section".

One of the most important parts is right above the "## Services" section where one decides which user name and password to use to access the Monit application.

The next item which is the magic that executes the Apache nobody attack script is approximately the eigth line down from "## Services" line is "if loadavg (1min) > 4 then exec "/etc/apachenobody.sh"

Breaking that line down ... it reads that if the load equals 4 and above to execute the Apache nobody script - in which the script is in the /etc folder. 

The Apache nobody killer script should be uploaded as root to the /etc folder - right in the root of the folder - and permissioned 755 so that Monit will be able to execute/run the script. 

Monit of course has many useful features not limited to the above purpose ... but our goal is to put a stop to the Apache nobody attacks that can disable a server with what appears to be a DDoS attack. This puts those would be crashes and Apache nobody attackers to be basically a thing of the past and the server administrators life much easier.

To start monit automatically after a reboot in the /etc/rc.d folder/directory modify the rc.local file adding the following line at the bottom of the file: /usr/bin/monit


Stop Apache Nobody attacks script
  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

WHAT WE MANAGE AND FIREWALL FALSE POSITIVES

Relevant to dedicated servers and answering an inquiry from a dedicated server client:Certainly...

ADDING MEMCACHED TO CENTOS 6.5+ AND CENTOS 7+

Step 1: SSH into your server and fire this command yum install memcached.x86_64 Step 2: Next...

APACHE NOBODY ATTACK SCRIPT

The Apache Nobody attack script is useful in stopping those that are victims of Apache Nobody...

BOOT PARTITION FILLING UP

The boot partition can fill up with excess kernels causing the system to crash if the boot...

CLOUDFLARE RAILGUN SERVER INSTALLATION

Installing CloudFlares RailgunThe following tutorial considers that one is already a CloudFlare...

Powered by WHMCompleteSolution