Monitoring SMTP server (using mailgraph, qshape and postqueue

Monitoring SMTP mails never been easier! You can check the number of sent emails, bounced emails, rejected emails and etc (you can see demo at http://www.stat.ee.ethz.ch/mailgraph.cgi).

Let’s get the source:

wget http://mailgraph.schweikert.ch/pub/mailgraph-1.14.tar.gz
tar xf mailgraph-1.14.tar.gz
mv mailgraph-1.14 /var/mailgraph

and install dependencies:

yum install perl-rrdtool
yum install perl-File-Tail

and run the mailgraph once to fetch the previous logs:

./mailgraph.pl -l /var/log/maillog -c -v

then run it as service:

cp mailgraph.pl /usr/local/bin/
./mailgraph-init start

Now the cgi file is needed to be executed by web server so once we installed the web server we will configure CGI configs. The main changes are AddHandler, Add ExecCGI to “/var/mailgraph” directory and add mailgraph.cgi as an index file like index.php.

yum install httpd
cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.original
vi /etc/httpd/conf/httpd.conf (appendix 01)
service http start
chkconfig httpd on

Now you should be able to browse the web server and see the stats. I have to note that the stats are not getting updated frequently. I coolant figure out how frequent this is getting updated but sometimes I had to run the mailgraph script manually to get latest stats. At this moment we are done with mailgraph.

Now we just take a look at shape and other CLI tools for monitoring SMTP. qshape and post queue are two useful scripts found in postfix additional scripts. We need to install it first:

yum install postfix-perl-scripts

And here are some example of the tool to get the stat of different queues:

qshape hold
qshape deferred
qshape active

postqueue is another tool:

postqueue -p
postqueue -p | egrep -c "^[0-9A-F]{10}[*]"
postqueue -p | egrep -c "^[0-9A-F]{10}[^*]"

we also can use the maillot directly with assistance of grep:

grep -c "postfix/smtp.*status=sent" /var/log/maillog 
grep -c "postfix/smtp.*status=bounced" /var/log/maillog

Continue reading

Advertisements

Adding custom metric to Amazon Cloud Watch

The default metrics in AWS cloud watch does not include Memory usage or Storage usage data. In my opinion cloud watch has what it takes to monitor a server (and more). So what we are going to do is adding additional metrics to cloud watch.

At first we need to create a user and grant necessary permissions for (we will use the access-key and secret-key of the user):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "cloudwatch:PutMetricData",
        "ec2:DescribeTags"
      ],
      "Effect": "Allow",
      "Resource": [
        "*"
      ]
    }
  ]
}

Then we need to install the dependencies, download the script and extract it:

sudo yum install -y perl-Switch perl-Sys-Syslog perl-LWP-Protocol-https
wget http://ec2-downloads.s3.amazonaws.com/cloudwatch-samples/CloudWatchMonitoringScripts-v1.1.0.zip
unzip CloudWatchMonitoringScripts-v1.1.0.zip
cd aws-scripts-mon

That is it. Done!

To make sure everything is set we do a little test but without sending data to cloud watch:

mon-put-instance-data.pl --mem-util --disk-space-util --disk-path=/ --verbose --aws-access-key-id=youraccesskey --aws-secret-key=yoursecretkey

Now we should add a cron job to send metric in any interval (we used 5 minutes). Enter the “crontab -e” command and add the following:

*/5 * * * * /home/user/aws-scripts-mon/mon-put-instance-data.pl --aws-access-key-id=youraccesskey --aws-secret-key=yoursecretkey --mem-util --disk-space-util --disk-path=/ --from-cron

And restart the crond service

sudo /etc/init.d/crond restart

*It is notable that you can add other metrics too like what we did: –mem-util is for sending memory usage and –disk-space-util is for sending the storage usage.