Personal Workflow Blog

To content | To menu | To search

Tag - munin

Entries feed

Thursday, 10 June 2010

Waiting for Munin 2.0 - Performance - Architecture

A little intro/refresh on munin's architecture on the master

Munin has a very simple architecture on the master : munin-cron is launched via cron every 5 minutes. Its only job is to launch in order munin-update, munin-graph, munin-html & munin-limits.

The various processes

munin-update

This process retrieves the values from the various nodes and to update the rrd files. This one should never take more than 5 minutes to run, otherwise there will be gaps since the next update will not be launched (lockfile-protected runs).

This process stresses the I/O on the master, and depends on the plugins execution time on the various nodes. On 1.4 the retrieval is multi-threaded[1], so an slow node doesn't impact too much the whole process.

2.0 proposes asynchronous updates and vectorized updates.

munin-graph

This process generates all the image files from the rrd files.

It is usually a process that is quite CPU-bound, it generates also a fair load of I/O. Since 1.4 there might also be a parallel graphing generation in order to take advantage of multiple CPU / multiple I/O paths.

A simple optimization is to generate only needed graphs instead of all of them each time. This leads to CGI-generation of graphs. 1.2 & 1.4 took a first step in this direction, but it's quite a hack since it's only a very basic script that calls munin-update with the correct parameters.

A FastCGI port of the wrapper (munin-cgi-graph) removes the overhead of starting the wrapper for each call, but in 1.4 the code is quite experimental and has some serious bugs that would need extensive patching to be fixed.

2.0 completes the integration of CGI graphing with removing the overhead of calling munin-graph and does this extensive patching for bugs fixing

munin-html

This process generates all the html files from the rrd files. This one is quite fast for now.

munin-limits

This process checks the limits to see if there is a warning/alert to send via mail or nagios. This one is also quite fast for now.

Notes

[1] more multi-process actually

Tuesday, 8 June 2010

Waiting for Munin 2.0 - Introduction

This is the first article of a series about the coming version 2.0 of Munin.

The idea came from the series Waiting from 8.5 about PostgreSQL.

The ironic part is that their 8.5 release has become a 9.0, just like our 1.5 will be a 2.0.

I'll post several small articles about new or enhanced-enough features. They will all be tagged munin20.

Planned summary :

  1. Performance - Architecture context
  2. Performance - FastCGI
  3. Performance - Asynchronous updates
  4. Performance - Misc
  5. Native SSH transport
  6. Custom data retention plans (keep more data)
  7. Dynamic zooming

Thursday, 26 November 2009

Native SSH transport for Munin

Update: This page is obsolete, since it has been merged in the 2.0 version of Munin. The syntax is a little bit different, but the idea is the same.

I blogged about the munin monitoring system a while ago.

The fact the Munin team did quite a remarkable job in cleaning up the 1.2 code for the 1.4 release enabled me to add a native SSH transport for Munin, and made be able to get rid of all SSH tunnels.

Continue reading...

Tuesday, 4 November 2008

A Poor Man's Munin Node to Monitor "Hostile" UNIX Servers

Munin is a nice monitoring system. Simple but quite effective. It's main selling point is the UNIX-esque simplicity of the architecture. You can just create a new plugin in a matter of minutes to monitor whatever you can imagine.

There is even a comprehensive collection of plugins ready to use (admittedly of various quality).

Various platform are supported

Usually the main issue is the MuninNode, an agent (daemon) that runs on the server to be monitored, since it is responsible of translating the request of the munin server (the one with the graphs) to the various plugins, build-ins or external. The ease of installing this agent depends on the OS and the access you have on the server :

Windows

For Windows you can install munin-node-win32.

Unix (with root access)

For Unix when you are root, usually there is a package ready to install in your distribution, or from the source.

Others (Hostile servers)

On hostile servers, you don't usually have a root access and no easy acces to a compiler.

I wrote pmmn (Poor Man's Munin Node), a little vanilla Perl script that emulates the core functionality of the real munin-node script, but without having to install many Perl CPAN modules.

It has also a nice functionality : it is possible to communicate via stdin/stdout instead of a TCP port. This way it is very easy to monitor hosts that are behind a firewall without opening (and monitoring) many tunnels.

Installation of pmmn

Suppose you have access to the server via a supervision user (let's say supusr). Installation of pmmn is quite easy : just copy the files somewhere on the disk where you have access, for example (/home/supusr).

TCP installation

Just launch the server with -p 4949 and declare it in the munin.conf file on your munin-server. This solution is quite equivalent to a regular munin-node installation.

Tunnel installation

Same as the TCP, but you have to create a TCP tunnel via SSH to be able to reach the munin-node.

Inetd+SSH installation

It is a mix between port forwarding via inetd and the Tunnel-based previously discussed.

You first have to established a key-based SSH authentication without passphrase (you will not be there to type it) from supusr on the inet server (usually the munin-node one) to the user supusr on the server to be monitored.

For example, to monitor server1 and server2, in the file /etc/inetd.conf, you have to add lines :

7001   stream  tcp     nowait  supusr /usr/bin/ssh -- supusr@server1 /home/suprusr/pmmn/pmmn.pl
7002   stream  tcp     nowait  supusr /usr/bin/ssh -- supusr@server2 /home/suprusr/pmmn/pmmn.pl

Then, in the munin.conf file of the MuninServer, you just have to declare the new nodes :

[server1]
    address localhost
    port 7001
[server2]
    address localhost
    port 7002

The MuninServer will now set up a stdin/stdout SSH tunnel transparently and launch the pmmn server when needed. You are now free to write plugins like if a real munin-node where installed. The only restriction is that y ou don't have a root access, so you are limited in the information you may collect.

page 3 of 3 -