A Munin plugin to track http response times.
After two years of abandonment, I've finally managed to give some love to the server running this site. After all the trouble, I wanted to have a good monitoring setup, so I can evaluate the changes and how they're performing.
Munin has been running for while, but I had never tweaked it. Now, after running nginx and memcahed, I wanted to monitor those as well.
Turns out that Munin's plugin system is easy to extend. But it takes a while to understand how it all comes together: the various jobs, rdd integration, updates, permissions. Specially the autocongif and wildcards plugins, while a neat idea do make things more magical and harder to track.
I wanted to monitor is the total performance of the application. Not how apache, nginx or mysql are doing. Nor only disk or the bandwidth but the final response times, to measure how the entire stack is doing.
External tools, such as Pingdom and Monitor.us can handle it, but then you're also dealing with the internet connection, network conditions interfere with the measurements.
So I've hacked a quick plugin for Munin, that will measure response times for the urls you wish to monitor. This is a how it looks: 
.
In this particular case, I am monitoring the home url and the same one with a query string. That way, I can keep tabs on how the site is performing with memcached on and off.
Munin plugins are ran through the shell: they can be anything a perl script, c, python, anything else really. This plugin is written in python. It does require httplib2 though. I've tested this in python 2.5 and 2.4 (thanks for the patch Hermann Kase ) on linux, but it should really run on other setups without issues.
I am putting this up on github, as future updates will be easier to track. Installing is pretty easy, just copy the httpresonsetime file to where plugins are located (/etc/munin/plugins on ubuntu), chmod it appropriately and add the urls you need to monitor on the config file:
1
2
|
[httpresponsetime]
env.urls http://www.example.com,http://www.othersite.com/somewhere
|
Restart the munin-node daemon, and you're all set.
Of course, just as I finished writing this, I found a similar plugin at the munin exchange.It takes a different approach though, as it allows for one url only to be monitored, and that url is set on the plugin file itself. The solution above seems cleaner.
If anyone else finds this useful, I'll put it up on munin exchange.
getting a third opinion
Hi,
If the URL contains a '-' character, it needs to be replaced with a '_' too, to make the munin plugin configuration valid:
parsed = urlparse.urlsplit (url)
-
return ("%s_%s_%s" % (parsed[1] , parsed[2], parsed[3])).replace("/", "_").replace(".", "_")
-
return ("%s_%s_%s" % (parsed[1] , parsed[2], parsed[3])).replace("/", "_").replace(".", "_").replace("-", "_")
Otherwise, nice work, thank you.
Hi Arthur,
Very usefull script. I do have a question though, would you know how to resolve this error which I am encountering?
$ ./httpresponsetime
Traceback (most recent call last):
File "./httpresponsetime", line 104, in ? import httplib2
File "/usr/lib/python2.4/site-packages/httplib2/init.py", line 58
from .iri2uri import iri2uri
^
SyntaxError: invalid syntax
--
I have the latest httpresonsetime script (Oct 30) as well as httplib2 (httplib2-python3-0.5.0). This was run on a CentOS 5 host.
Thanks in advance
Hi Alex.
Looking from your traceback, it seems your're running python 3.0. (you can check by running $python --version .
Python 3.0 is definitely not supported at this point (too different, will require two code bases), and it's not worth it. Check your system for python2.6 and set that on the shebang, should work.
Regards
Hi Arthur,
Thanks for your help!
Yep, I had inadvertently installed httplib2-python3-0.5.0 originally, swapped that with httplib2-0.5.0 instead and now it works.
$ ./httpresponsetime You needto specify which URLS to monitor by setting the 'urls' env variable in your munin-conf.
i'll go ahead and update munin-node with our urls.
Thanks again
Great Article!
Was looking for exactly this and you made my life easy.
I also love that you did it in Python. I was assuming that most of the plugins were written in Perl, since that's the platform's language, but after seeing this in Python I'm thinking about writing some of my own!
Thanks!
Hi,
I can't seem to get this to work.
I've followed your instructions, but if I execute the script with the config param it prints - "You needto specify which URLS to monitor by setting the 'urls' env variable in your munin-conf."
I have specified the env.urls in my /etc/munin/plugin-conf.d/munin-node file (actually I just copied your example and pasted it in).
Also your error message says the urls should be defined in munin.conf but this is the conf file for the munin master - I'm assuming this is a typo.
Thanks,
Will
I just had hard time debuging this one, it seems that in munin 1.4.5 config must be returned as fast as possible.
Waiting for computed value (such as checking about a second for a http response) turned out to be a reason for not showing plugin for 'list' command in telnet.
Stupid one-line-hint: comment last line in get_config function : )
Nice plugin, it works really well. When clicking through to the graphs I get:
"NOTE! This service is in CRITICAL state because one of the values reported is outside the allowed range. Please see further down for information about the ranges and the graph for the values. Field"
which is obviously not the case
Have the last word