Groesbeek, view of the 'National Liberation Museum 1944-1945' in Groesbeek. © Ton Kersten
Fork me on GitHub

Puppet environments

2014-05-26 (141) by Ton Kersten, tagged as puppet

For my job I do a lot of Puppet and I thought it was about time to write some tips and tricks down.

First part of this post is about my environment setup. In my test setup I use a lot of environments. They are not at all useful, but that’s not the point. It’s my lab environment so things need to break once in a while. But with multiple environments Puppetlabs says that you should switch to directory environments (PuppetDoc) but some way or another I cannot get that to work in a good way with my PE version (3.4.3 (Puppet Enterprise 3.2.3)). So I started implementing dynamic environments, which is a simple way of specifying the directories for your environments.

Part of my puppet.conf looks like

[master]
    environment = production
    manifest    = $confdir/environments/$environment/manifests/site.pp
    manifestdir = $confdir/environments/$environment/manifests
    modulepath  = $confdir/environments/$environment/modules:/usr/share/puppet/modules
    templatedir = $confdir/environments/$environment/templates

So, my default environment is production and a client can specify another environment to be in. The command

puppet agent --environment=test

would place this node in the test environment. A simple module places a new puppet.conf file on the client stating this new environment. Couldn’t be more simple.

Well, that’s what you think. But what if you need to deploy 10.000+ hosts of which there are about a third in environment test and about a 1000 in environment development? It would take a lot of time to ssh into all these servers and run Puppet with the correct environment.

There has to be a way around that. And, of course, there is. In Puppet version 3 and up Hiera is integrated into Puppet and we already use that a lot. Why not integrate the environment in Hiera? Well, our hiera.yaml is now:

---
:hierarchy:
    - "%{environment}/hiera/%{::fqdn}"
    - "%{environment}/hiera/%{::hostname}"
    - "%{environment}/hiera/%{::domainname}"
    - "%{environment}/hiera/%{::systemtype}"
    - "%{environment}/hiera/%{::osfamily}"
    - "%{environment}/hiera/common"

:backends:
    - yaml

:yaml:
    :datadir:
        /etc/puppetlabs/puppet/environments

This challenges me with a chicken and egg problem. To get the environment I need to know the environment. But what if I make Hiera into an ENC and let this one deliver the environment? Can this be done? Yes, it can.

This is how I did it:

First create a part of the Hiera structure that’s not in the current environment, for example like this:

---
:hierarchy:
    - "hiera/%{::fqdn}"
    - "hiera/default"
    - "%{environment}/hiera/%{::fqdn}"
    - "%{environment}/hiera/%{::hostname}"
    - "%{environment}/hiera/%{::domainname}"
    - "%{environment}/hiera/%{::systemtype}"
    - "%{environment}/hiera/%{::osfamily}"
    - "%{environment}/hiera/common"

:backends:
    - yaml

:yaml:
    :datadir:
        /etc/puppetlabs/puppet/environments

And in the directory /etc/puppetlabs/puppet/environments/hiera I place a very small file, called default.yaml, which contains:

---
environment: 'production'

This makes sure that any node without a specific file, will get the production environment. This is the default for Puppet as well, so nothing changes for that.

To test this, run:

hiera environment ::fqdn=$(hostname -f)

This will give you something like environment: production. For every host in another environment as the production one, create a small file named the FQDN of the host with the contents stating the wanted environment.

(Watch for the :: in front of the fqdn. This means that the fqdn variable is a top scope variable, as all facter variables are.

Now integrate this into Puppet. First create a little script that executes the command above and returns the wanted output.

My script is called getenv and placed in /etc/puppetlabs/puppet/bin

#!/bin/bash

penv="$(/opt/puppet/bin/hiera                       \
            -c /etc/puppetlabs/puppet/hiera.yaml    \
            environment ::fqdn="${1}")"

echo "environment: ${penv}"

This returns a string like environment: production.

And last, but not least, place this settings in the [master] of your puppet.conf

node_terminus  = exec
external_nodes = /etc/puppetlabs/puppet/bin/getenv

It took some work to get things started, but a small shell thingy read the file with all 10.000+ hosts and required environments, that created all the Hiera files for all nodes that are not in the production environment.

Just one thing to do: When I have a lot of host-files in a single directory, this could become slow. I could place all definitions in a simple database, but things would get complicated again, and that’s not what I want. I also could split things up per letter, but I’m not sure yet if I really want that.

When I have resolved this, this entry will be continued.

Docker panics

2014-04-14 (140) by Ton Kersten, tagged as sysadm

This morning I was messing around with Docker and I wanted to build me a nice, clean container with Ubuntu in it, to test Ansible thingies. I’ve done that before and everything worked as a charm. Until today.

I have this Dockerfile (I’ve stripped it to the bare bones that still fail):

FROM ubuntu:latest
MAINTAINER Ton_Kersten
RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
RUN apt-get -y update
RUN apt-get -y upgrade
RUN apt-get -y install git git-flow
RUN apt-add-repository -y ppa:mozillateam/firefox-next
RUN apt-get install -y firefox

and when I run

docker build .

I end up with a beautiful kernel panic. Whatever I try, panic Nothing in any logfile

I’m running kernel version Linux lynx 3.2.0-60-generic #91-Ubuntu SMP Wed Feb 19 03:54:44 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux which had no problems before.

The Docker version is Docker version 0.10.0, build dc9c28f

Is there anybody out there that knows what’s happening?

Thanks.

Ansible @ Loadays

2014-04-05 (139) by Ton Kersten, tagged as ansible, sysadm

Last Saturday I attended Loadays in Antwerp, Belgium.

After listening to Jan Piet Mens’s talk about Ansible, I was up for it.

At 11:30 sharp, I started my own presentation for an almost packed room. It’s called “Ansible, why and how I use it” and you can find it on SpeackerDeck.

It was a lovely talk, with a very knowledgeable crowd.

Please, have a look at it and if you have any questions, let me know.

Thanks to the crew for organizing such a lovely event, every year.

Photos of the event where taken by Robert Keerse and you can see them at his Google Plus page. Do enjoy!!

For those of you with a strong stomach, the complete presentation is on Youtube as well. Have a look at the Youtube stream

Ansible @ CfgMgmtCamp

2014-02-05 (138) by Ton Kersten, tagged as ansible

Last couple of days I attended Configuration Managememt Camp in Ghent, Belgium. On Monday morning we started of with presentations of Mark Burgess (CFEngine), Luke Kanies (Puppet) and Adam Jacob (Chef). Good talks about the future of things.

After lunch it got nerdy ans I joined the Ansible room, to see how things went and at 17:00 I started my own presentation for a completely packed room. It’s called ‘Ansible, why and how I use it’ and you can find it on SpeackerDeck.

Later that night we joined the social event and I must say that I was really glad that Toshaan ran out of beer-tickets. Four Triple Karmeliet is more than enough for one evening smiley

The Tuesday was a complete Puppet-day, starting of with a Q-and-A with Luke Kanies. Things started a little slow, but as usual we ran out of time because of the amount of questions.

After stocking up with Belgian Beer the two days ended. Lots of new things learned and to look into. Two days well spent.

LPI Certification

2013-08-15 (137) by Ton Kersten, tagged as lpi

It’s been a while since the last post, but I’ve been very, very busy.

And in the meantime I also found some time to take the LPI-102 exam. This resulted in a Pass and now I’m officially LPI1 certified.

Well, let’s see what’s next. Puppet exam, Ansible training, LPI2, Python ………

So much to learn, so little time.

Puppet Facter Fact

2013-07-08 (135) by Ton Kersten, tagged as puppet

Look at me, I made a Puppet Facter Fact!!!

With a lot of thanks to Andrew Beresford who started the initial code. I just tweaked it.

What it does is rather simple, it finds the expiration date of the SSL certificate of this host and returns the expiration date and time when there are less than 30 days left. Otherwise it just returns a --sign. In the Puppet manifest I check if it’s this --sign and if not I generate a warning.

This is it:

#
# Set the Facter-Fact "certificate_expiry" to the SSL certificate
# expiration date and time.
#
# Usage example:
# --------
#   if "${::certificate_expiry}" != "-" {
#       notify { 'CertExp' :
#           message  => "Certificate expire date for ${::fqdn}: ${::certificate_expiry}",
#           withpath => false,
#       }
#   }

#
# $Id$%
# $URL$
#
Facter.add("certificate_expiry") do
    setcode do
        warndays = 30
        time = Puppet::SSL::Host.localhost.certificate.expiration
        warn = time - ( warndays * 60 * 24 )

        if ( warn - Time.now ) < 0
            time = time.strftime("%Y-%m-%d %H:%M:%S")
        else
            time = "-"
        end
        time
    end
end

Me proud, I am smiley

Ansible Day in Antwerp

2013-06-29 (136) by Ton Kersten, tagged as ansible

Today I’m attending the first full day Ansible configuration meeting. This meeting is in Antwerp, Belgium, a drive of almost 2 hours. Thanks to Multi Mho (Maurice Verheesen) I didn’t need to drive, he wanted to try out his nice, new car. It drives perfectly and we arrived about 30 minutes early.

For a first meeting of a new tool there where a lot of attendants, amongst others (and I don’t want to forget anybody, so I won’t even try to give a complete list), but below are the people I think that where there.

  • Ton Kersten
  • Maurice Verheesen
  • Christopher Ranschaert
  • Colin Petrie
  • Dag Wieers
  • Inigo Ortiz de Urbina Cazenave
  • Jan Piet Mens
  • Jeroen Hoekx
  • Jochen Moes
  • Joost Ringoot
  • Kevin Clymans
  • Kristof Wevers
  • Lee Van Steerthem
  • Mattias Gees
  • Nic De Muyer
  • Serge van Ginderachter
  • Toshaan Bharvani
  • Vincent Van der Kussen

All very knowledgeable people and nice company to be around.

After Jan Piet talked about Ansible Fest in Boston he supplied us with all the goodies he brought home. We all got…. drum-roll…. 1 sticker each. The T-shirts didn’t show up at the Ansible Fest. Not that bad, because I already have one. Thanks to Jan Piet for the time and effort to get some goodies, even though he couldn’t get them.

After this we started discussing and talking about things to improve Ansible. We all agreed things are great and will be greater by time.

All in all a good day to be at and I would like to thank everybody who attended and who helped organize this day. And last but not least Michael deHaan for creating Ansible.

PS: Also thanks to the guys that brought the beer. The Vicaris Triple is a fantastic one.

Puppet User Group

2013-04-07 (134) by Ton Kersten, tagged as puppet

Yesterday I attended the first meet up of the Dutch Puppet User Group and I gave a talk about how to start with Puppet.

It was called: “Puppet deployment, an introduction” and the PDF slideshow can be viewed or downloaded from speakerdeck.

If you have any comment, please send me an email.

We-Blog with clouds

2013-04-06 (133) by Ton Kersten, tagged as blog

Today I received an email from Chris Hackenschmidt with a patch for a category cloud for my We-Blog program.

Of course I have implemented that and you can see it in action on the right.

So, a new version of We-Blog is out, currently still 0.9, with a big Thank You to Chris.

Download it and enjoy.

Installing Pandoc

2013-01-26 (132) by Ton Kersten, tagged as linux, pandoc

John Macfarlane released a new version of Pandoc that has a lot of new enhancements. A lot of things have changed in the Markdown input types and it’s now compatible with PHP Markdown. This is very nice, because a lot of implementations use the extensions defined by PHP Markdown.

I downloaded the dmg file to install it on my MacBook Pro and it works like a charm.

So, I decided to install it on my new CentOS 6 server to build documents there. Well, I was in for a nice surprise.

After cloning the source from github I started as documented in the INSTALL file

make prep

and now I get

Resolving dependencies...
cabal: cannot configure directory-1.1.0.0. It requires old-time ==1.0.*
For the dependency on old-time ==1.0.* there are these packages:
old-time-1.0.0.0, old-time-1.0.0.2, old-time-1.0.0.3, old-time-1.0.0.4,
old-time-1.0.0.5, old-time-1.0.0.6 and old-time-1.0.0.7.
However none of them are available.
old-time-1.0.0.0 was excluded because pandoc-1.10.0.5 requires old-time ==1.1.*
old-time-1.0.0.7 was excluded because pandoc-1.10.0.5 requires old-time ==1.1.*

etc.

I kept tweaking the makefile and tried almost everything possible, but I was caught in a cyclic dependency loop. So, time to rethink and use Google.

I had already figured out it had to do with a cabal and ghc version that are to old, so it’s time to update.

I found newer Cabal versions at JustHub.

I started removing every RPM with haskel or ghc in the name and removed ~/.cabal and ~/.ghc directories to start of fresh. Then:

sudo rpm -Uhv
http://sherkin.justhub.org/el6/RPMS/x86_64/justhub-release-2.0-4.0.el6.x86_64.rpm
export PATH=/usr/hs/bin:~/.cabal/bin:$PATH
cabal install cabal-dev
cd ~/pandoc
make prep
.
.

Now it compiles but it breaks with:

make install
cabal-0.14.0: Error: some packages failed to install:
pandoc-1.10.0.5 failed during the tests phase. The exception was:
ExitFailure 1
cabal --config-file=/home/tonk/qq/pandoc/cabal-dev/cabal.config:
/usr/hs/tools/cabal-0.14.0 failure (return code=1)
make: *** [install] Error 1

Some tests fail, but I just removed --enable-tests from the Makefile and I got a working executable. Simply copy it to the ~/.cabal/bin directory, install the man-page and everything is fine.

I think it should be simpler, but for now I’m a happy camper.