Puppet: Installing collectd and graphite

Follow

Issue

Monitoring performance and utilization of switches running Cumulus Linux is imperative in an operational data center environment. graphite provides scalable real time graphing. collectd is a powerful tool for gathering metrics using its wide range of plugins, such as CPU, disk, load, memory, and interface statistics. collectd can be used to place statistics in a database and collectd-graphite is a plugin that runs inside the collectd process and sends collectd data to a graphite instance for graphing. There are previously released puppet modules for collectd and graphite that require simple modifications so they install easily on Debian hosts and on Cumulus Linux switches.

This article shows you how to configure puppet to install:

  • collectd and graphite on a Debian host
  • collectd on a Cumulus Linux switch

Solution

As part of this solution you need to install the following packages.

1. On the Debian host, install:

  • puppetmaster
  • sudo
  • vim
apt-get update
apt-get upgrade
apt-get install -y puppetmaster sudo vim

 

2. On the switches running Cumulus Linux, install:

  • puppet
  • sudo
apt-get update
apt-get upgrade
apt-get install -y puppet sudo

 

3. On the puppetmaster server, install the following Puppet modules:

  • dwerder-graphite
  • footballradar-python
  • pdxcat-collectd
  • puppetlabs-apache

 

puppet module install dwerder-graphite
puppet module install footballradar-python
puppet module install pdxcat-collectd
puppet module install puppetlabs-apache

 

The puppet module list contains:

user@server:/home/user# puppet module list
/etc/puppet/modules
├── dwerder-graphite (v3.0.1)
├── footballradar-python (v0.1.0)
├── pdxcat-collectd (v1.0.0)
├── puppetlabs-apache (v0.9.0)
├── puppetlabs-concat (v1.0.0)
└── puppetlabs-stdlib (v4.1.0)

 

4. Here is a simple puppet manifest to include the collectd and graphite modules. Place the following text in a file called /etc/puppet/manifest/site.pp:

case $::operatingsystem {
    'Debian': {
        include collectd
        include graphite
    }
    'CumulusLinux': {
        include collectd
    }
}

Modifying the puppet collectd Module

Define the collectd server IP address in the collectd plugin manifest. Replace <<<$IPADDRESS>>> with the IP address of the server and replace <<<$INTERFACE>>> with the proper interface that connects to the management Ethernet of the switches.  

1. Go to line 111 of /etc/puppet/modules/collectd/manifests/init.pp. Modify the lines for the network plugin for the collectd server:

vim +111 /etc/puppet/modules/collectd/manifests/init.pp

class { 'collectd::plugin::network':
         listen               => ‘<<<$IPADDRESS>>>',
         listen_interface     => '<<<$INTERFACE>>>',
         listen_securitylevel => 'None',
         reportstats          => 'true',
}

  

2. Go to line 134 of /etc/puppet/modules/collectd/manifests/init.pp. Add the IP address and interface to the network plugin for the collectd server:

vim +134 /etc/puppet/modules/collectd/manifests/init.pp

class { 'collectd::plugin::network':
          server               => '10.99.0.1',
          server_interface     => 'eth0',
          server_securitylevel => 'None',
          reportstats          => 'true',
          forward              => 'true',
          maxpacketsize        => 1024,
}

 

The fully modified configuration file should look something like this:

user@server:/etc/puppet/modules/collectd/manifests# cat init.pp
class collectd(
  $fqdnlookup   = true,
  $interval     = 10,
  $threads      = 5,
  $timeout      = 2,
  $purge        = true,
  $recurse      = true,
  $purge_config = true,
) {
  include collectd::params
 
  $plugin_conf_dir = $collectd::params::plugin_conf_dir
 
  package { 'collectd':
             ensure   => installed,
             name     => $collectd::params::package,
             provider => $collectd::params::provider,
             before   => File['collectd.conf', 'collectd.d'],
  }
 
  file { 'collectd.d':
          ensure  => directory,
          name    => $collectd::params::plugin_conf_dir,
          mode    => '0644',
          owner   => 'user',
          group   => 'user',
          purge   => $purge,
          recurse => $recurse,
  }
 
  $conf_content = $purge_config ? {
      true    => template('collectd/collectd.conf.erb'),
      default => undef,
  }
 
  file { 'collectd.conf':
          path    => $collectd::params::config_file,
          content => $conf_content,
          notify  => Service['collectd'],
  }
 
  if $purge_config != true {
    # Include conf_d directory
    file_line { 'include_conf_d':
                 line    => "Include \"${collectd::params::plugin_conf_dir}/\"",
                 path    => $collectd::params::config_file,
                 notify  => Service['collectd'],
    }
  }
 
  collectd::plugin { 'cpu': }
  class { 'collectd::plugin::df':
               fstypes          => 'userfs',
               ignoreselected => 'true',
  }
  collectd::plugin { 'disk': }
  collectd::plugin { 'entropy': }
  collectd::plugin { 'irq': }
  collectd::plugin { 'load': }
  collectd::plugin { 'memory': }
  collectd::plugin { 'processes': }
  collectd::plugin { 'swap': }
  collectd::plugin { 'users': }
 
  class { 'collectd::plugin::syslog':
           log_level     => 'info',
  }
 
  #server
  case $::operatingsystem {
    'Debian': {
        file { "/opt/collectd-plugins":
                ensure => "directory",
        }
 
       class { 'collectd::plugin::rrdtool':
                datadir       => "/var/lib/collectd/rrd",
       }
 
        file { '/etc/collectd/conf.d/filters.conf':
                ensure  => 'present',
                content => template('collectd/filters.conf.erb'),
                mode    => '0644',
                owner   => 'user',
                group   => 'user',
                purge   => $purge,
                recurse => $recurse,
         }
 
        file { '/etc/collectd/conf.d/thresholds.conf':
                ensure  => 'present',
                content => template('collectd/thresholds.conf.erb'),
                mode    => '0644',
                owner   => 'user',
                group   => 'user',
                purge   => $purge,
                recurse => $recurse,
         }
 
        file { '/opt/collectd-plugins/carbon_writer.py':
                ensure  => 'present',
                content => template('collectd/carbon_writer.py.erb'),
                mode    => '0644',
                owner   => 'user',
                group   => 'user',
                purge   => $purge,
                recurse => $recurse,
         }
 
         class { 'collectd::plugin::network':
                  listen               => '10.99.0.1',
                  listen_interface     => 'eth1',
                  listen_securitylevel => 'None',
                  reportstats          => 'true',
         }
 
         class { 'collectd::plugin::python':
                  globals                 => 'true',
                  modulepath              => '/opt/collectd-plugins/',
                  module                  => 'carbon_writer',
                  carbon_receiverhost     => 'localhost',
                  carbon_receiverport     => '2003',
                  carbon_receiverprotocol => 'tcp',
                  carbon_differentiate    => 'true',
                  carbon_lowercase        => 'true',
                  carbon_typesdb          => '/usr/share/collectd/types.db'
         }
      }
 
      'CumulusLinux': {
         class { 'collectd::plugin::interface':
                  interfaces   => [ 'eth0', 'swp1', ],
                  ignoreselected => 'true',
         }
         class { 'collectd::plugin::network':
                  server               => '10.99.0.1',
                  server_interface     => 'eth0',
                  server_securitylevel => 'None',
                  reportstats          => 'true',
                  forward              => 'true',
                  maxpacketsize        => 1024,
        }
     }
  }
 
  service { 'collectd':
             ensure    => running,
             name      => $collectd::params::service_name,
             enable    => true,
             require   => Package['collectd'],
  }
}

 

3.  Add a caveat for Cumulus Linux to the collectd params.pp file as follows:

 class collectd::params {
 
  case $::operatingsystem {
    'Debian': {
      $package           = 'collectd'
      $provider          = 'apt'
      $collectd_dir      = '/etc/collectd'
      $plugin_conf_dir   = "${collectd_dir}/conf.d"
      $service_name      = 'collectd'
      $config_file       = "${collectd_dir}/collectd.conf"
    }
    'CumulusLinux': {
      $package           = 'collectd-core'
      $provider          = 'apt'
      $collectd_dir      = '/etc/collectd'
      $plugin_conf_dir   = "${collectd_dir}/conf.d"
      $service_name      = 'collectd'
      $config_file       = "${collectd_dir}/collectd.conf"
    }
    'Solaris': {
      $package           = 'CSWcollectd'
      $provider          = 'pkgutil'
      $collectd_dir      = '/etc/opt/csw'
      $plugin_conf_dir   = "${collectd_dir}/conf.d"
      $service_name      = 'collectd'
      $config_file       = "${collectd_dir}/collectd.conf"
    }
    'Redhat': {
      $package           = "collectd.${::architecture}"
      $provider          = 'yum'
      $collectd_dir      = '/etc/collectd.d'
      $plugin_conf_dir   = $collectd_dir
      $service_name      = 'collectd'
      $config_file       = "/etc/collectd.conf"
    }
    default: {
      fail("${::osfamily} is not supported.")
    }
  }
 
}

 

Modifying the puppet graphite Module

Make the following modifications after installing the puppet graphite module.

1. Go to line 116  in init.pp. Add a secretKey password and comment out the default secretKey:

vim +116 /etc/puppet/modules/graphite/manifests/init.pp
 
#  $secretKey = 'UNSAFE_DEFAULT'
  $secretKey = 'blah'

 

2. In /etc/puppet/modules/graphite/manifests/config.pp, change permissions to the Apache user and Apache group for specific files and directories:

vim +135 /etc/puppet/modules/graphite/manifests/config.pp


        file { '/etc/apache2/mods-available':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/etc/apache2/mods-enabled':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/etc/apache2/sites-available':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/etc/apache2/sites-enabled':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/etc/apache2/conf.d':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/var/log/apache2':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/var/log/apache2/access.log':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/var/log/apache2/error.log':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }
        file { '/var/log/apache2/other_vhosts_access.log':
            owner   => $::graphite::params::web_user,
            group   => $::graphite::params::web_user,
        }

 

3. Go to line 171 in config.pp. Add the following lines to enable the Apache headers module:

vim +171 /etc/puppet/modules/graphite/manifests/config.pp


    exec { 'Enable apache headers module':
            command => 'a2enmod headers',
            notify  => Service["${::graphite::params::apache_service_name}"];
    }

 

The final /etc/puppet/modules/graphite/manifests/config.pp file needs to look like the following:

# == Class: graphite::config
#
# This class configures graphite/carbon/whisper and SHOULD NOT be called directly.
#
# === Parameters
#
# None.
#
class graphite::config inherits graphite::params {
 
            anchor { 'graphite::config::begin': }
            anchor { 'graphite::config::end': }
 
            Exec { path => '/bin:/usr/bin:/usr/sbin' }
 
            # for full functionality we need this packages:
            # madatory: python-cairo, python-django, python-twisted, python-django-tagging, python-simplejson
            # optinal: python-ldap, python-memcache, memcached, python-sqlite
 
            # we need an apache with python support
 
            package {
                        "${::graphite::params::apache_pkg}":
                             ensure => installed,
                             before => Exec['Chown graphite for apache'],
                             notify => Exec['Chown graphite for apache'];
            }
 
            package {
                        "${::graphite::params::apache_wsgi_pkg}":
                             ensure  => installed,
                             require => Package["${::graphite::params::apache_pkg}"]
            }
 
            case $::osfamily {
                        debian: {
                                    exec { 'Disable default apache site':
                                            command => 'a2dissite default',
                                            onlyif  => 'test -f /etc/apache2/sites-enabled/000-default',
                                            require => Package["${::graphite::params::apache_wsgi_pkg}"],
                                            notify  => Service["${::graphite::params::apache_service_name}"];
                                    }
                        }
                        redhat: {
                                    file { "${::graphite::params::apacheconf_dir}/welcome.conf":
                                                ensure  => absent,
                                                require => Package["${::graphite::params::apache_wsgi_pkg}"],
                                                notify  => Service["${::graphite::params::apache_service_name}"];
                                    }
                        }
                        default: {
                                    fail("Module graphite is not supported on ${::operatingsystem}")
                        }
            }
 
            service { "${::graphite::params::apache_service_name}":
                           ensure     => running,
                           enable     => true,
                           hasrestart => true,
                           hasstatus  => true,
                           require    => Exec['Chown graphite for apache'];
            }
 
            # first init of user db for graphite
 
            exec { 'Initial django db creation':
                    command     => 'python manage.py syncdb --noinput',
                    cwd         => '/opt/graphite/webapp/graphite',
                    refreshonly => true,
                    notify      => Exec['Chown graphite for apache'],
                    subscribe   => Exec["Install webapp ${::graphite::params::graphiteVersion}"],
                    before      => Exec['Chown graphite for apache'],
                    require     => File['/opt/graphite/webapp/graphite/local_settings.py'];
            }
 
            # change access permissions for apache
 
            exec { 'Chown graphite for apache':
                    command     => "chown -R ${::graphite::params::web_user}:${::graphite::params::web_user} /opt/graphite/storage/",
                    cwd         => '/opt/graphite/',
                    refreshonly => true,
                    require     => [
                                    Anchor['graphite::install::end'],
                                    Package["${::graphite::params::apache_pkg}"]
                        ]
            }
 
            # Deploy configfiles
 
            file {
                        '/opt/graphite/webapp/graphite/local_settings.py':
                                    ensure  => file,
                                    owner   => $::graphite::params::web_user,
                                    group   => $::graphite::params::web_user,
                                    mode    => '0644',
                                    content => template("graphite/opt/graphite/webapp/graphite/local_settings.py.erb"),
                                    require => Package["${::graphite::params::apache_pkg}"];
                        '/opt/graphite/conf/graphite.wsgi':
                                    ensure  => file,
                                    owner   => $::graphite::params::web_user,
                                    group   => $::graphite::params::web_user,
                                    mode    => '0644',
                                    content => template("graphite/opt/graphite/conf/graphite.wsgi.erb"),
                                    require => Package["${::graphite::params::apache_pkg}"];
                        "${::graphite::params::apache_dir}/ports.conf":
                                    ensure  => file,
                                    owner   => $::graphite::params::web_user,
                                    group   => $::graphite::params::web_user,
                                    mode    => '0644',
                                    content => template('graphite/etc/apache2/ports.conf.erb'),
                                    require => [
                                                Package["${::graphite::params::apache_wsgi_pkg}"],
                                                Exec['Initial django db creation'],
                                                Exec['Chown graphite for apache']
                                    ];
                        "${::graphite::params::apacheconf_dir}/graphite.conf":
                                    ensure  => file,
                                    owner   => $::graphite::params::web_user,
                                    group   => $::graphite::params::web_user,
                                    mode    => '0644',
                                    content => template('graphite/etc/apache2/sites-available/graphite.conf.erb'),
                                    require => [
                                                File["${::graphite::params::apache_dir}/ports.conf"],
                                    ];
            }
 
            case $::osfamily {
                        debian: {
                                    file { '/etc/apache2/sites-enabled/graphite.conf':
                                            ensure  => link,
                                            target  => "${::graphite::params::apacheconf_dir}/graphite.conf",
                                            require => File['/etc/apache2/sites-available/graphite.conf'],
                                            notify  => Service["${::graphite::params::apache_service_name}"];
                                    }
                                    file { '/etc/apache2/mods-available':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/etc/apache2/mods-enabled':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/etc/apache2/sites-available':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/etc/apache2/sites-enabled':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/etc/apache2/conf.d':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/var/log/apache2':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/var/log/apache2/access.log':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/var/log/apache2/error.log':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                                    file { '/var/log/apache2/other_vhosts_access.log':
                                            owner   => $::graphite::params::web_user,
                                            group   => $::graphite::params::web_user,
                        }
                        exec { 'Enable apache headers module':
                                command => 'a2enmod headers',
                                notify  => Service["${::graphite::params::apache_service_name}"];
                                }
                        }
                        default: {}
            }
 
            # configure carbon engine
 
            file {
                        '/opt/graphite/conf/storage-schemas.conf':
                         mode    => '0644',
                         content => template('graphite/opt/graphite/conf/storage-schemas.conf.erb'),
                         require => Anchor['graphite::install::end'],
                         notify  => Service['carbon-cache'];
                        '/opt/graphite/conf/carbon.conf':
                         mode    => '0644',
                         content => template('graphite/opt/graphite/conf/carbon.conf.erb'),
                         require => Anchor['graphite::install::end'],
                         notify  => Service['carbon-cache'];
            }
 
 
            # configure logrotate script for carbon
 
            file { '/opt/graphite/bin/carbon-logrotate.sh':
                    ensure  => file,
                    mode    => '0544',
                    content => template('graphite/opt/graphite/bin/carbon-logrotate.sh.erb'),
                    require => Anchor['graphite::install::end'];
            }
 
            cron { 'Rotate carbon logs':
                    command => '/opt/graphite/bin/carbon-logrotate.sh',
                    user    => root,
                    hour    => 1,
                    minute  => 15,
                    require => File['/opt/graphite/bin/carbon-logrotate.sh'];
            }
 
            # startup carbon engine
 
            service { 'carbon-cache':
                       ensure     => running,
                       enable     => true,
                       hasstatus  => true,
                       hasrestart => true,
                       before     => Anchor['graphite::config::end'],
                       require    => File['/etc/init.d/carbon-cache'];
            }
 
            file { '/etc/init.d/carbon-cache':
                    ensure  => file,
                    mode    => '0750',
                    content => template('graphite/etc/init.d/carbon-cache.erb'),
                    require => File['/opt/graphite/conf/carbon.conf'];
            }
}

 

Related Links

 

 

Have more questions? Submit a request

Comments

Powered by Zendesk