Groesbeek, view of the 'National Liberation Museum 1944-1945' in Groesbeek. © Ton Kersten
Fork me on GitHub
Posts tagged as sysadm

sed tips and tricks

2012-06-22 (116) by Ton Kersten, tagged as code, linux, sysadm

I'm creating a Puppet Starter Kit with some standard manifests included and a complete set of documentation. All documentation should be written in Markdown and will be served by Markdoc. But I want to generate all Markdown files from the Puppet manifests, so I only need to document the manifest file. Generating the Markdown is not that difficult, except that I kept ending up with empty lines at the top of the manifest code and I wanted to get rid of those. Of course this should be done with sed, because the whole generation process is written in bash. When playing around with sed I found

sed '/./,$!d' filename

which, I think, is genius in it's simplicity. After you find something, do not remove. Life in UNIX and Linux is nice!

Read more »

Puppet updates

2012-06-18 (115) by Ton Kersten, tagged as code, puppet, sysadm

When working with Puppet and a VCS (like git and SVN) it's nice to have a simple way of updating the Puppet tree.

My tree is always in /etc/puppet and owned by user and group puppet. User puppet is allowed to checkout the complete tree from git or subversion.

I have created two one-liners to update the complete tree and make sure all rights are still correct.


# update_svn
su - puppet -c 'cd /etc/puppet; svn up; cd doc; ../bin/gendoc'


# update_git
su - puppet -c 'cd /etc/puppet; git pull; cd doc; ../bin/gendoc'

But, of course, it's not handy to type update_git today and update_svn tomorrow. And I also don't want a path to /etc/puppet/bin.

The solution is a very simple one, as always:

cd /usr/local/bin
ln -s /etc/puppet/bin/update_git pupdate

and now I only have to type pupdate and things work out.

Fix a lot of rights

2012-06-18 (114) by Ton Kersten, tagged as linux, sysadm

A customer called and wanted help with an error they made.

The error was simple, they typed:

chmod -R 660 /

and now things broke. Of course things broke. If they would not brake that would be very weird.

Luckily they had a second server and a simple one-liner stole all the rights from this second server and and we could put these on the broken one.

The oneliner

find / -depth -printf 'chmod %m\t\t-- "%p"\nchown %u:%g\t-- "%p"\n' >

produces output like this

chmod 644       -- "/etc/sysconfig/kdump"
chown root:root -- "/etc/sysconfig/kdump"
chmod 644       -- "/etc/sysconfig/rhn/sources"
chown root:root -- "/etc/sysconfig/rhn/sources"
chmod 644       -- "/etc/sysconfig/rhn/sources.rpmforge.txt"
chown root:root -- "/etc/sysconfig/rhn/sources.rpmforge.txt"

Running this script on the broken server left us with something that was working.


dpkg -x <package name>

on all cached packages in /var/cache and after that a

dpkg --reconfigure -a

Solved the rest.

Pheeeuuwww, we were lucky smiley

Updated Pygments

2012-05-16 (112) by Ton Kersten, tagged as blog, puppet, pygments, sysadm

I'm using Pygments for quite some time now and I just noticed there was a new version available (1.5). I installed that and I was wondering if there would be a lexer included for Puppet. Well, it wasn't, but a short Google action directed me to the Pygments lexer for the Puppet DSL.

Of course my old CentOS 5 system with Python 2.6 doesn't want to install this, so I hacked the Puppet lexer into Pygments.

Here's an example of the result:

class generic::ssh {
    $ssh_service       = hiera("ssh_service")
    $ssh_packages      = hiera("ssh_packages")
    $ssh_debug         = hiera("ssh_debug",         "undef")
    $permit_root_login = hiera("permit_root_login", "no")
    $ssh_users         = hiera_array("ssh_users",   "undef")
    $ssh_groups        = hiera_array("ssh_groups",  "undef")

    package { $ssh_packages:
        ensure => present,
        before => File["/etc/ssh/sshd_config"],

    file { "/etc/ssh/sshd_config":
        ensure  => present,
        content => template("generic/sshd_config.erb"),
        notify  => Service["${ssh_service}"],

    service { $ssh_service:
        ensure     => running,
        enable     => true,
        hasrestart => true,
        hasstatus  => true,

and an example of the Hiera Yaml file:


# SSH Settings
permit_root_login    : 'no'
ssh_service          :
                       - 'sshd'
ssh_users            :
                       - 'root'
                       - 'tonk'
ssh_groups           :
                       - 'wheel'
ssh_packages         :
                       - 'openssh'
                       - 'openssh-clients'
                       - 'openssh-server'

Nice smiley

Switching to tmux

2011-07-24 (104) by Ton Kersten, tagged as sysadm

Some time ago I was fighting my .screenrc again. I wanted to change the status line, but it was hardly possible to read and understand what I typed hardly half a year ago. The screen config file is not exactly poetry.

While searching the web to find how to change the status line I ran into tmux and I thought: "Let's give it a try". And after the very simple compile I started configuring it. What a surprise I was in for. This config file could be read, understood and changed. Man, this is good.

One of the first things I changed was the default Prefix key Ctrl-B. I changed it to Alt-A, so I can still use all the control keys with Vim and in the terminal.

Read more »

FreeBSD PXE boot Part 2

2011-06-09 (96) by Ton Kersten, tagged as freebsd, puppet, pxe, sysadm

Some posts ago I wrote that I was busy to find out how a FreeBSD machine can be PXE-ed from a Linux server. Well, I found that some time ago, but I didn't have the time to type it here, yet. Well, as always, once you know how it's done, it's quite simple. But because a lot of the FreeBSD documentation is very old (talking about FreeBSD 4, 5 and 6) it takes some time to find it all.

Read more »

Why does Puppet keep breaking?????

2011-05-03 (95) by Ton Kersten, tagged as freebsd, puppet, pxe, sysadm

In my previous post I stipulated that I was PXE booting FreeBSD. Well this works and I will come back on that. But for the configuration I want to run Puppet. Nice and easy config management.

On my server I run Puppet from source. This because the server is a CentOS box with a very old Ruby and Puppet. So I decided to run the Puppet client from source as well. Getting the git repo is easy enough and installing Puppet should not be to hard.

Well, well, how wrong have I been. Every time I update the Puppet client or server something breaks. And I do mean every time.

First it started with not parsing templates correctly. A couple of hours of debugging solved that, but then Puppet started crying with " Error 400 on SERVER: No support for http method POST". W.T.F. does it mean. This somehow got solved, but then the templates broke again. The Puppetlabs site stated (as always) to update to the newest version. So I did. And the template error was back again, but now it was a different one: "Failed to parse template issue/issue.erb: undefined method 'first' for "/etc/puppet/modules/issue/templates/issue.erb:19:in". Tinkering around for 4 (yes four!) hours solved this one (I can hardly remember what I tried, but I can assure you that I have seen all sites about Puppet that exist on the Inernet. Including the ones about handpuppets). And then I got the 400 error again. Running in debug mode doesn't help either, so I'm rather stuck. Man, do I hate this type of behaviour. Be stable or go away! I now completely had it with diving into Puppet sources to find the culpritt. If they are still seeking for a miracle for Pope John Paul II, maybe a stable Puppet client would be a good idea.

I'm getting rather fed up with this stuff. Ths way I'll never be able to update a server and be sure it will work. Maybe CFEngine3 is a better option!

umask per directory

2010-12-08 (83) by Ton Kersten, tagged as linux, puppet, sysadm

Some users insist on using bash. This is a good shell, but not as good as zsh. But, I do want them to be able to use the per directory umask as well as all the zsh users.

So I started digging, as the bash shell does not support a chpwd hook.

This is what I came up with:

{   # Set the initial umask
    case "${PWD}/"
            umask 007
            [[ x"${um}" != x"" ]] && umask ${um}
function cd()
    builtin cd "${@}"

Now, when I change to the directory /etc/puppet I do get a umask of 007 and when I cd somewhere else, I do get the original umask.

I do redefine the intercal cd command to run the chpwd hook. There must be a more elegant way to do this, but this does the job.

umask per directory

2010-12-06 (82) by Ton Kersten, tagged as puppet, sysadm

I've been working with Puppet some time now, and we are configuring our way through a lot of hosts, with 6 persons, all working in the same Puppet master directory.

This should work fine with all UNIX/Linux groups and setgid directories. But simple problem arose with the git version control stuff.

Once in a while the complete git repo was destroyed and quite a lot of searching revealed the reason why.

We are all working as non-root and we are all members of the Puppet group. But: When I edit a file and commit it, the corresponding files in the git repo are made by me and the rights are set according to my umask. When someone else tries to edit the same file or something else which results in the same hash files, writing is not permitted, because of my ownership. A chown in a script will not work, as a chown is not honored as a non-root user.

This problem can simply be solved by setting the umask to something like 007 (or u=gwx,g=gwx,o=). But when I do edit stuff in my home-directory I do not want an open umask like that. So what to do, as ext[234] do not support per directory umasks.

I use zsh as a shell and I found a nice function in the man-page. There is a standard function, called chpwd() that gets executed every time a directory change is made. So I only had to fill in the blanks.

This is what I came up with:

    case "${PWD}/"
            [[ ${UMSAVE} = 0 ]] &&
            {   um=$(umask)
            umask 007
            [[ x"${um}" != x"" ]] && umask ${um}

Now, when I change to the directory /etc/puppet I do get a umask of 007 and when I cd somewhere else, I do get the original umask.

How much fun can it be smiley

Compiling OpenSSL and OpenSSH

2010-08-12 (63) by Ton Kersten, tagged as sysadm

My server at home runs CentOS 5 and this has OpenSSH version 4.3. Running updates doesn't update this version, because RedHat keeps the version number stable.

But I wanted a newer OpenSSH because of some nice new features. But when I do compile a new version I'm still stuck with old OpenSSL, and that's not what I want.

Well, you can guess it by now, this is what I did.

Read more »