Today at <a href="barcampesm.org">BarCampESM</a> Erik Dahl led as discussion on "The need for an open agent". After this discussion tlockney and I announced the GA of the new Open Management Consortium website, and via a "show of hands" consensus, set up a new sub=project in the monitoring community for the "OMC Open Agent" project. Erik will be leading the effort and building out the community over the next weeks. Please join in and help him out. Now, go visit the new <a href="http://openmanagement.org/community/monitoring/omc_open_agent">OMC Open Agent community</a> to learn more about this exciting project.

Moments ago ahonor opened a discussion entitled Design patterns for software operations? which caught our eye. Here's specifically what cuaght my eye:

In addition to linking to his original blog post ahonor suggested that we start a workgroup within the OMC:

Well, as you all know this is exactly how we want the OMC to operate; community lead. So we have created a new workspace under the "Open Standards" section of the website called "OMC Design Patterns". Thanks to ahonor for the idea and for volunteering to kick things off and help manage the workspace. You can link directly to the workspace (from your blog or other sites) using the following URL:

http://beta.openmanagement.org/community/open_standards/omc_design_patterns

It will be very interesting to see how much adoption this idea picks up. I for one will be participating heavily in the workspace as ahonor has a great idea/perspective that I hope others join in support of.

We here at the OMC are happy to announce a new feature to the site: Chuck Talk has agreed to lead a series of interviews focusing on the concerns of the OMC community. We’re happy to kick off the series with Luke Kanies and this insightful look at Puppet.

This week, I am pleased to get some time with Luke Kanies, the developer of Puppet, an open source, automated, systems management language and toolset. Luke was kind enough to take some time out of his busy schedule to let us in on the secrets of his vision for Puppet.

Chuck Talk: Hi Luke, before we begin, I was wondering if you could tell us a little bit about your background, and how you came to the decision to develop Puppet?

Luke Kanies: I spent a long time as a Unix sysadmin, gradually adopting and writing more and more automation. Eventually, I ended up as a consultant focused on cfengine, and I found that I couldn’t really build a business around cfengine because it was too difficult to get new functionality into the product and it was not maintained with a focus on stability. After trying a few other options, I decided to create a product out of the prototype I’d written a while back.

That prototype was based heavily on the modules I’d written for cfengine and ISconf, and the ideas behind it had been kicking around for a long time. There are some tricks in the language I’d also been thinking about, and I got some ideas from the configuration management workshops at LISA for the compiler that would make configurations easier to maintain.

Chuck Talk: Thanks for that Luke, it’s nice to see what leads to the decision to make an idea a reality. I think many people fail to understand that open source is really all about making useful tools to solve problems. Speaking of which, I note that you have stated that “Puppet is an open-source next-generation server automation tool. It is composed of a declarative language for expressing system configuration, a client and server for distributing it, and a library for realizing the configuration.”

Given that statement, I suppose I would not be the first person to ask this, but what language or skill set is required for a user to deploy and utilize Puppet?

Luke Kanies: It all depends on your goals. Architect-level sysadmins have built large-scale Puppet installations with lots of process, developers with almost no sysadmin experience have used Puppet to build and maintain basic server farms, and relative newbies have used Puppet to take their first steps into using automation to get more done with less effort.

Chuck Talk: I note that the Puppet site mentions the use of Ruby for providers, yet there is also the statement that Ruby provides a bit too much functionality. For a person who might want to utilize Puppet, that might seem a little confusing. In fact, the glossary is helpful in understanding Puppet, but how do you explain it to those that will use the toolset?

Luke Kanies: Puppet’s language is very simple. This simplicity comes at somewhat of a cost, because you can’t do as much as you might like, but one of the big benefits is that people are very comfortable jumping in, because they know they won’t have to face a lot of complexity to get work done. It also makes people more comfortable sharing code – downloaded modules are easy to understand, so there’s not as much fear applying them on every machine on your network.

I’ve been pleasantly surprised, though, at how many people approach ruby with some trepidation but then announce that they’ve picked it up easily and are enjoying it. Ruby is a feature, not something to be concerned about, as the web development world has learned with the rapid adoption of Ruby on Rails.

Chuck Talk: How do you compare cfengine and Puppet? I know that you have stated you worked for years to try to improve cfengine, but do you see Puppet as being a more feature-complete toolset, or a more robust autonomous management interface?

Luke Kanies: Cfengine has been around for more than a decade, and it has changed very little in that time. It was a great tool when it first came out, but its development process is so closed that it’s very difficult to get new functionality into it. Puppet was designed based on deep experience with cfengine plus a lot of experience with other tools, and cfengine’s author, Mark Burgess, didn’t have the luxury of prior art, so it makes sense that I was able to make Puppet better based on that experience.

That being said, I work very hard in Puppet to avoid the code generation and code duplication that is essentially required to use cfengine, cfengine has a slightly different syntax for almost every resource type while Puppet has a single syntax valid for everything, and Puppet makes it very easy to write your own custom types that can be used just like any native type while adding a new type to cfengine requires modifying the whole stack from the parser on down.

It’s experience with cfengine and others that allowed me to avoid those mistakes, though, so I have to give Mark credit for his initial innovations.

Chuck Talk: Given that OpsWare has been challenged by PUBPAT over its patent on computer management, do you see that patent as more of a challenge to Open Source developers, or perhaps an opportunity to deliver better software - to the extent that PUBPAT is successful in its challenge?

Luke Kanies: Given what interaction I’ve had with the commercial vendors, they’re at war with each other and barely even know that the open source tools exist, so I don’t think of it as much of a threat. It’s clearly a ridiculous patent, but OpsWare considers their user guide to be proprietary and confidential, so they’re a pretty ridiculous company and it wouldn’t be surprising if OpsWare decided to start messing with open source projects just because they have lawyers and they know we don’t.

Either way, the patent has nothing to do with producing better software, other than the possibility that OpsWare is hoping to compete with the patent instead of having to compete in the marketplace by making better software. The patent system was developed as a way to encourage people to publish otherwise-secret technology, and this patent is totally obvious, so it’s not going to help them or anyone else make better software.

Chuck Talk: Do you see Puppet and Reductive Labs operating in the sweet spot of the market tier, the SMB marketplace? That is traditionally a much wider and more open space than the Enterprise market where commercial vendors are undercutting each other for marquee accounts.

Luke Kanies: Yeah, I largely focus on somewhat smaller organizations. I like working with large companies, and I have, but I don’t usually like working with “Enterprise” companies because there’s usually more politics than technology. There are some really great, really big companies that understand technology and know how to efficiently apply it to their problems, and I really like working with them. I’m not at all interested in writing a 50 page response to an RFP in competition with CA or Tivoli, and that’s unfortunately what too many of these companies require.

Chuck Talk: I am curious about this statement on the Puppet FAQ: “Trying to express a complex network configuration entirely through a GUI is an exercise in frustration that no one should suffer, but expressing the abstraction necessary to share those GUI configurations goes beyond frustrating.”

I know that the cross-platform capabilities of Puppet probably drive that statement, but does that mean that a browser UI for Puppet is simply not in plan at any point?

Luke Kanies: I wouldn’t say that, I’d just say that a GUI tool is bound to be much less functional than a text-based tool. Sacrificing that functionality might have its place, but it doesn’t make sense to start there.

Chuck Talk: I have noted the fact that Puppet will work on “completely different operating systems.”

Given that Puppet has both a client and a server, is the software largely meant to be run on UNIX, Linux, OS/X and variants, or will the software allow for Windows clients as well?

Luke Kanies: The framework could run on Windows, but I have essentially no Windows expertise, and I’m not in a position to provide that support. Also, most Unix-like operating systems model most things pretty similarly (e.g., users, packages, hosts), but Windows models many of them entirely differently so it would be difficult or even impossible to port some things over to Windows. Conversely, managing the registry in Windows is critical, but just doesn’t even show up in the Unix world.

Chuck Talk: Where do you see Puppet in the IT stack? Is this meant as a centralized management tool, a satellite management toolset, a NOC toolset - where do you see it working the best, and what makes an ideal environment for Puppet to be implemented?

Luke Kanies: I mostly talk about Puppet as a single tool, but the truth is that it’s lots of pieces packaged as a single tool. My real goal is to build multiple stacks communicating as part of an ecosystem of more advanced tools, where your configuration management tool talks to your monitoring tool which in turn talks to decision engines which in turn change the running configurations. Puppet is a first step towards that ecosystem, but I had to build a single product that could stand on its own, and towards that end I’ve developed Puppet as a centralized management tool, where you perform all of your work on the central server and it propagates out to clients from there.

It’s hard to pick a single ideal deployment environment for Puppet, but it is particularly good at handling variety, in terms of operating systems, applications, locations, or just about anything else. Complexity derives almost entirely from variety, so I’ve done what I can to make variety itself easier to handle directly.

Chuck Talk: Where is Reductive Labs located, and how many employees are actively engaged in supporting Puppet?

Luke Kanies: I live in Nashville, Tennessee, in the USA, and I have one employee who lives in Florida, also in the USA.

Chuck Talk: What do you think is the biggest hurdle for Reductive Labs in terms of continued development and success?

Luke Kanies: Finding a business model based on open source software that will let us hire enough people to develop all of the great products I want to build.

Chuck Talk: What one thing would you like to share with others that you are working on now (if you can tell us)?

Luke Kanies: Heh, I talk pretty openly about everything that I’m doing these days. I tried for years to give my ideas away and no one was interested, which is what led me to write Puppet in the first place.

Probably the most interesting thing we’re working on right now is integration with a Rails-based web site that will allow one host to change the configuration of another host, so that, for instance, your web server could set up the monitoring server to monitor the web service. It involves storing all of the compiled configurations in a database, which also gives you a lot of opportunity for inspecting configurations and querying what resources are deployed where.

Chuck Talk: How did you come to join the Open Management Consortium?

Luke Kanies: I’m pretty excited by the opportunity to work with other open source companies trying to really change the way people think about managing computers, and they’re all in the OMC so it really makes sense.

Chuck Talk: Speaking of OMC, how do you see this consortium improving the Systems Management space?

Luke Kanies: Nearly all progress is made through a fine balance of communication and competition, and OMC is helping all of us become more aware of the products and ideas that are out there while also giving us a forum to work together, so it really helps to encourage both sides of this balance.

Chuck Talk: Is there anything that you would like to see happen under the OMC framework?

Luke Kanies: I can certainly say that I’d like to see lots of discussion, lots of competition, and lots of teamwork. I don’t really know what the result will be from that, else I’d be writing it right now, but I really want to see the OMC drive all of us to make better, interoperable products, and hopefully products that really rely on each other to provide more functionality.

Chuck Talk: This may seem an oddball question, but what do you predict for the Systems Management market over the next few years? Do you see it growing, becoming a more vibrant marketplace with truly quality software, or do you see it becoming more consolidated and commoditized?

Luke Kanies: My biggest prediction is that we’ll start to see organizations that adopt these better tools competing more effectively against organizations that don’t. Everyone has IT, but too many organizations view it as a cost center rather than an execution engine, and those who stay near the cutting edge in management technology will easily outcompete their competitors who are fearful of adopting better, more advanced ways of managing their technology.

Chuck Talk: As always, I like to give those whose time I have taken, a chance to talk about anything I might have missed> Is there anything else you would like to cover that I haven’t mentioned?

Luke Kanies: This has been my common refrain for years: The current state of IT is far behind the current needs. Even stand-alone great products aren’t sufficient – Puppet could be the best configuration management tool in the world and it wouldn’t matter, because we’d need 10 other great tools, and even that wouldn’t suffice, because those tools would all need to talk.

Sysadmins and IT managers need to demand a lot more from their software providers, including me. Expect your software to talk to related software, and complain when it doesn’t. Expect your software to be manageable, and complain when it doesn’t. We’ve been too willing for too long to let developers ignore manageability, but we’ve also been too willing to accept “good enough” in the management space.

Chuck Talk: Thank you Luke, it has been a pleasure to talk with you. I hope that everyone finds this article as refreshing as I have found your answers and candor.

Luke Kanies: Thanks Chuck, I hope my answers have been informative and useful. Cheers.

By Greg Wallace, Emu Software

Taxonomy: (from Greek taxis meaning arrangement or division and nomos meaning law) is the science of classification according to a pre-determined system, with the resulting catalog used to provide a conceptual framework for discussion, analysis, or information retrieval. In theory, the development of a good taxonomy takes into account the importance of separating elements of a group (taxon) into subgroups (taxa) that are mutually exclusive, unambiguous, and taken together, include all possibilities. In practice, a good taxonomy should be simple, easy to remember, and easy to use.

Why do I think a new taxonomy is needed for systems management? Type “systems management” (with parens) into Google and you get more than 58 million pages to sift through, with multiple pages of sponsored links from such companies/products as Sybase, OpManager, ConfigureSoft, MySQL, BMC, Perle, groundworkopensource, heroix, handysoft, Raritan and a slew of aggregation sites offering to help you make sense of all the different solutions. And though less numerous, the search results are no less varied when one uses any number of other more specific terms, like Configuration Management or Provisioning.

People in the industry know that the varied solutions offered by the above-listed companies perform different functions, that they do them in different ways, they act on different aspects of IT (devices, applications, etc), they are variously broad or narrow in scope and are designed to appeal to different audiences. Sure, all of them can be accurately described as Systems Management, but to someone trying to find a solution to a specific problem, this is about as useful as if I were to challenge you to name the living creature I’m thinking of and only give you the clue that it is a member of the kingdom animalia.

So, the sheer diversity of the search results and sponsored links highlights, I think, the need for a tighter framework to describe systems management functions and benefits so that users and solution providers (VARs, SIs, etc) can more easily identify the right solution to their and their customer’s needs. Let me say up front that I am clearly not the first person who has recognized the need for a more precise and standardized way of describing different types of systems management products (see, for example this Network world article). But I do think that the birth of OMC provides a new and great opportunity to actually achieve this objective. Why do I think this? Because I believe that one of the fundamental tenets of open source is transparency. As the only industry body explicitly focused on promoting open source systems management, one way the OMC can do this is by driving transparency around the way various types of systems management products are described.

If we think about why this problem is so bad, perhaps it comes down to the closed nature of traditional systems management solutions, and the effect this closed nature has on marketing habits. Like other proprietary IT solutions, traditional systems management solutions compete, in part, on comprehensiveness. The more the products do, and the more products offered (products that are frequently integrated together in some closed way) the more can be sold to installed base customers, and the less likely these customers will be to turn to another vendor. This is one mechanism of lock-in, and just as it is present in the operating system market, though less so now than say 5 years ago, so too is it present in systems management. And so it is understandable why proprietary systems management solutions may have a bias towards general, if accurate, descriptions – call it “fly in the web” marketing – the broader the web, the more flies one catches.

With open source projects, in contrast, our stuff can be downloaded and used by anyone and everyone, and so we must assume, regardless of the ambiguity of the terms we use to describe them, that the true nature of our product’s functionality, architecture, scope and benefits will become known in great detail, and rather quickly. Another salient difference is that we in the commercial open source community typically price on a subscription basis, which means that each year or two, our customers can vote to keep us, or kick us off their island, so we had better deliver what we promise. For these reasons, I would argue that OMC members ought to have a strong bias to be unambiguous in the description of our projects, using terms that place them in mutually exclusive sub-categories of the broader Systems Management main category. At the end of the day, it may well be, in fact it probably will be, the case that some projects are broader than others, but until we come up with a standard way of classifying systems management solutions, we have no transparent way to measure and express a product’s scope.

Step 1:

For starters, I’d love to get general agreement that the OMC will strive to set some standards for how member projects describe themselves. I, for one, like the way Webmin does it:

Webmin is a web-based interface for system administration for Unix. Using any browser that supports tables and forms (and Java for the File Manager module), you can setup user accounts, Apache, DNS, file sharing and so on.

Webmin consists of a simple web server, and a number of CGI programs which directly update system files like /etc/inetd.conf and /etc/passwd. The web server and all CGI programs are written in Perl version 5, and use no non-standard Perl modules.

This description has almost all the essential ingredients (missing, perhaps, is a quick statement of benefits). In these three sentences, it clearly states what Webmin is (a Web-based interface for system administration of Unix), what it does (you can setup user accounts, Apache, DNS, file sharing and so on. ), what it’s scope is (system administration for Unix), and how it works (Webmin consists of a simple web server, and a number of CGI programs which directly update system files like /etc/inetd.conf and /etc/passwd. The web server and all CGI programs are written in Perl version 5, and use no non-standard Perl modules.) I think this is a good formula, and could serve as a star