Last Sunday, the day after BarCampNashville, I had lunch with Luke Kanies, the owner of Reductive Labs
and the author of Puppet. It became a working lunch due to all my
questions about his product. I first met Luke at OSCON 2007 during his
Puppet session. I hadn’t planned to attend the Puppet session because I
was really trying to focus on monitoring and availability at the time
since I had stepped out of the configuration management space.
Luke gave a great presentation, and I became really interested in Puppet. Back in the day, I used to work with all the Tivoli
products and used to be as knowledgeable of the Tivoli configuration
management tools as I was of monitoring and event correlation.
Somewhere along the way, however, the Tivoli products became way too
complicated for one person to be an expert with all their ESM
offerings. In fact, my litmus test for a product is that, if I can’t be
the smartest person in a classroom with a particular product, then I
won’t teach that product. If I stop teaching the product, then I have
ceased to be the smartest person with it in the room. Luke
says that we should rename ESM and that he hates software. In fact, he
calls himself a professional software hater (’wareHater). In describing this distaste for software, he coined the term Infrastructure 2.0. He basically claims that he created Puppet because he was embarrassed by the current state of computer administration.The current state of configuration management products can be separated into two categories:.
Large Commercial Configuration Management Tools, which include
IBM Tivoli Configuration Manager (ITCM)
BMC Configuration Management (formerly Marimba)
Microsoft Systems Management Server
HP Opsware
Bladelogic
And Open Source Configuration Management Tools, which include
Cfengine
Puppet
+Note: I did not
include IBM Tivoli Provisioning Manager and Qlusters in these lists
because they are used much more as provisioning products than as
configuration management tools.+The deeper that I probed with my questions during our working lunch, the more I started to really like his product. In
my view, Puppet differentiates itself from almost all the current
products in the configuration management space in two key ways. One,
Puppet tries to define all its resources in human terms (i.e., what
Luke calls a cross-platform semantic abstraction).
Tivoli, to a certain extent, actually tried
this approach with one of its earlier products called Tivoli User
Administration (TUA). It introduced a cross platform tool for managing OS configurations. Puppet succeeds where
Tivoli fell short because Puppet not only
performs the cross-platform management (i.e., Solaris, SuSe, RedHat,
Debian, Centos, OpenBSD, Oracle Linux, and Ubuntu) but also allows
users to define all their resources in understandable human terms. For
example, Puppet allows a user to define a resource such as a file
system as “filesystem” without having to know the gory details of all
the file system commands and configuration files on different systems.
While Puppet provides this abstraction layer for OS configurations, it also allows a user to define configurations for applications using the same abstraction layer, thereby managing
the distribution and configurations of applications such as MySQL,
Apache, and PostgreSQL. Imagine being able to define all of your
complex software configurations and relationships in simple human
terminology.
Puppet’s ability to define complex relationships
in human terms brings me to the second way in which Puppet
differentiates itself from other products in configuration management
space: One of the Puppet project’s goals is to define a CPAN-like
repository for human resource descriptions (called “recipes”) for all
software products.
Here is an example of a recipe for installing and maintaining MySQL on different machines:
class mysql-server {
$password = “insert_password_here”
package { “MySQL-client”: ensure => installed }
package { “MySQL-server”: ensure => installed }
package { “MySQL-shared”: ensure => installed }
exec { “Set MySQL server root password”:
subscribe => [ Package[”MySQL-server”], Package[”MySQL-client”], Package[”MySQL-shared”]],
refreshonly => true,
unless => “mysqladmin -uroot -p$password status”,
path => “/bin:/usr/bin”,
command => “mysqladmin -uroot password $password”,
}
}
Currently, the fast-growing Puppet community has
contributed about 50 recipes. Puppet supports most of the configuration
management processes through “Providers,” and it comes with around 10
default providers for managing system configurations. One package
provider supports most of the common Unix/Linux installers, including,
but not limited to, dpkg, gem, rpm, and pkgadd on Solaris. Puppet also
allows users to define other packagers by letting them create their own
packages. For example, Puppet can run on AIX, but it doesn’t have
“installp” support. This missing functionality could be fixed by
creating a new provider. Puppet can also shell out on the agent,
allowing it to install virtually any software product. In fact, I have
been working with Tivoli software distribution
customers for over 10 years, and I can tell you that over 80 percent of
all the Tivoli software packages that I have seen customers develop
have been executions of the native installs.
Architecture
The Puppet server runs as a daemon that commutates using XMLRPC of
HTTPS. Puppet uses standard SSL certificates and includes its own CA.
Here is a simple example of the Puppet architecture:
[ Reductive Labs|http://old.johnmwillis.com/wp-content/uploads/2007/08/puppet12.png|puppet12.png]
The Puppet agent wakes up every 30 minutes (this
time can be configured) and contacts the server to get its current
configuration. Then, the agent topologically sorts the configuration
and checks the status of all the defined resources for that server. If
nothing has changed for a configured resource, Puppet will take no
action; it acts only when the state of the resource has changed. This
feature allows Puppet to do life-cycle management for a server by doing
the initial install, on-going upgrades, and de-commission. The Puppet
architecture allows machines to be kept up to date and in sync with an
organization’s business policies.
Puppet is written in Ruby and also provides its
own language for defining resources (a language called, conveniently
enough, Puppet). The Puppet language seems extremely flexible and easy
to define because it comprises mostly configuration-type statements.
Though it comes with preset defined functions, users can create their
own Ruby-based functions as well. See the Puppet Wiki for tons of
information about getting started.
Business Model
Reductive Labs, Luke’s company, provides community support for
Puppet through its Wiki, mailing lists, and IRC. Though Puppet is open
source GPL V2, Reductuve Labs, like any good open source vendor,
provides support and services around Puppet. It currently
has 5 paying customers at an average of $15k per year. It also provides
a bootstrap service of on-site installation, training, and
implementation. A customer can typically get the one-week implementation, training, and the first-year subscription support for around $25k. I
usually try to stay way from proprietary vs. OSS price comparisons, but
this cost is very insignificant compared to some of the proprietary
vendors listed above. Currently, Reductive Labs is working with about
12 serious organizations that are now using Puppet among a whole slew
of community users.[https://webmail5.isis.unc.edu/mail/services/go.php?url=http%3A%2F%2Fusers.iLike. com]
<span class="MsoHyperlink">[iLike.com|http://ilike.com/]
Uncomfortable with his recent celebrity at
conferences, Luke told me that he has difficulty measuring his
successes because he has his head so deep in the development and
services of Puppet. One of his better success stories is
with iLike.com, a website that allows users to download and share
music. When iLike created one of the first Facebook applications, it
grew from about ½ million users to over 6 million in a week. Luke,
being the entrepreneur that he is, asked how iLike planned to manage
that growth. He discovered that a services company in Seattle was managing iLike.com’s infrastructure build out using Puppet. In
fact, one of the owners of that company told Luke that he makes a
healthy living installing Puppet. Luke admitted that he felt feel
pretty good to know that other people can make a living from his
product.
My Cheesy Attempt at Being a Blog Interviewer
Near the end of the conversation, I felt that I
needed to ask some obligatory “Blogger” type questions. First, I asked,
“Why GPL?” Luke said that it’s not so much the license but what the
license has afforded him to do. For example, within the last year he
has been invited to give Puppet presentations (at not cost to himself)
in five countries. He also feels that he could never have
afforded the type of growth that he has achieved without his open
source community. He also thinks that, if he had created a proprietary
company, he would not have achieved even a small portion of what he has
accomplished in a very short period. Also, his open
source community has a lot of early adapters that thoroughly test out
new functions and features, which can then be delivered earlier in the
development cycle.
I then followed up with my personal “baptism of
fire” question. My theory is that most successful people can pinpoint
one or two “baptism of fire” moments in their career. Though at first
he didn’t recall one, he eventually pointed to his time spent compiling
Apache in 1998. Back when he started as a systems
administrator, one had to know how to compile everything, which got him
thinking about an easier way to do ESM. I also asked him about his
experiences coding, and he said that he actually majored in chemistry
and that the first program that he ever wrote was a 900-line shell
program. Some people, I guess, are born to be coders.
I also asked him what he thought might be some future opportunities. Luke
said that he saw managed services and possibly building and somehow
leveraging the recipe repository. If he really succeeds in creating a
sort of CPAN for configuration management, he should be able to make
money on that somehow.
Summary
In all honesty, even though my focus has been on
availability and event management in the OSS space, I find myself
really intrigued by what Luke is doing with Puppet. In the early days of Tivoli, all the products were bound by the Tivoli Framework. At the core of the Tivoli Framework was the configuration management infrastructure. The
Framework’s distribution manager made all the products, such as
monitoring, inventory, user administration, and event management,
viable in a large shop that had to manage many servers. Even
today, as IBM/Tivoli is putting the final nails into the Framework
coffin, thousands of Tivoli customers are scrambling to manage their
infrastructures with all the new IBM products. As I look
at all the extremely interesting OSS ESM tools out there, I struggle to
understand how they will all integrate. Maybe in an “Infrastructure
2.0″ sort of way, the framework might be created by Puppet.
