Great Architectures, Stacks & DevOps at Webscale

By Chris Ueland

How Ansible Scaled their Opensource Community


In this post I will highlight the devops automation platform Ansible and the ways it has become one of the most active GitHub communities.

What makes Ansible distinguishable is that it has taken off in an open source world with plenty of good options for configuration management and remote access tools. Ansible is really unique because there is no need for a client (just ssh & python) on the remote node you’re managing.

I first met Greg DeKoenigsberg (the Ansible Community Guy) at an Ansible-focused meetup at the MaxCDN offices in Los Angeles. Greg then introduced me to the former CTO and Project Leader of Ansible, Michael DeHaan. Mike has been kind enough to share his insight and experience from building Ansible into one of the more popular projects on Github with ScaleScale so that you can get some ideas for how to successfully scale your community.

–Chris / ScaleScale / MaxCDN

Michael DeHaan

Michael DeHaan is the Former CTO and Project Leader of Ansible

How big is the Ansible community?

It’s pretty big, but the thing about open source projects is that we don’t know for sure. We currently get about 60k people reading the doc site every month, and there have been over 935 contributors (in code) to the project so far.

Did you know or intend when you started Ansible it would be one of the top projects on GitHub?

Definitely not… I was super excited when we had 50 forks on GitHub or 50 users on IRC in early 2012. I thought that was pretty nuts, and in many cases, am still surprised that things have gotten as popular as they have. Ansible does a lot of things that are common sense, and in many ways it feels too basic to have that huge level of adoption to me, but I think a lot of that is really in that it IS actually that hard of a project to get off the ground, because of all the things it touches. There was a lot of work that went into it, even though it’s common sense, and when you can get 1000 community members to collaborate on building something great, people are very likely to want to reuse those results. It was not so much the core of Ansible that was revolutionary – it was VERY evolutionary – but the idea that you have so many people working together to keep all the modules working and up to date, and the chance of your bug or problem being dealt with already is so very high. It’s sort of a shared community toolbox.

Ansible GitHub Activity Chart

Who were key players in starting Ansible and the community?

I started it in Febuary of 2012, and it grew pretty quick because of knowing a lot of people through previous projects at Red Hat (like Cobbler and Func). It would be hard to name names without leaving somebody out, but I’d like to particularly thank Daniel Hozac, Seth Vidal, Steven Fromm, and Brian Coca from the early days for being very very involved. Since then we’ve also hired Brian Coca to work for Ansible, and I had James Cammarata join me from Cobbler before that, as well as Toshio Kuratomi from the Fedora Project, and they are doing a very large portion fo the work on everything.

How does it go interacting from commercial to community?

We drew the line at management software that sat on top of Ansible, but it was very important that the commercial software was something that was optional. We didn’t want to hold back Ansible, but also recognized that building commercial tools versus a very modular base was something that would be harder for the user base. For instance, it’s hard to design a database schema with 1000 of your closest friends — and some things take several hour meetings to plow through. In all, I think everybody has been very accepting of it, and we haven’t really had to deny any improvements to Ansible.

What’s the biggest installation you know of using Ansible?

I’m not exactly sure, but probably Rackspace, who is managing all of their production OpenStack datacenters. There are some others that may rival it, but I don’t have hard numbers. In particular, I don’t have hard numbers on things like HP or Twitter, but I know they have some large installations. So we are talking 20k nodes plus, configuring maybe 5k nodes at a time from one machine.

Do companies contribute code? Do you have any examples?

Companies definitely contribute code a lot. In many cases, if a company has something a module talks to, they’ll get involved in the development – as has been the case with folks from Google, HP, Rackspace, and others. In many cases, contributors are also end users of Ansible or customers. I think probably 95% of open source contribution to projects comes from people who are contributing in the course of their jobs.

How is the product roadmap for open source defined?

It’s a lot more of a set of values about what we want, how we prioritize things, and a culture of what we test and how documented things should be at this point. It’s definitely true that you can’t really assign work to contributors, so as such we don’t have a public roadmap per se. The best idea can come from absolutely nowhere, tomorrow, so we know what kind of tool we want to create and we have basically time-based releases. There are some major features — usually language features that we do spend time on to make sure they get done, and keeping various modules up to date is one of the bigger things. For instance, the various cloud providers are always changing, as are the Linux distributions.

How do you maintain what get’s contributed to Ansible?

We basically do a lot of work to maintain the things we are not getting pull requests for, which is all over the place — but a lot of it is about things that are harder for some community members to dive into. For instance, language features, parsing, test infrastructure, are often things that we’ll have to devote a significant amount of time to build, versus something we can expect a pull request on. But the community is really excellent about adding new module features, and often new modules, and various things in the plugins.

What are some URLs where things get contributed?

Basically everything happens around github, but I strongly recommend starting reading at our Community Docs for how everything works, which also includes some info about the mailing lists.

Do you have an App Store? What’s your experience with that so far?

Ansible does’t have an App Store per se but it does have for sharing a lot of community content for free. So it’s not a store, but I think it serves that purpose. I think my experience so far is everyone expects you to have a site like this, but in general, I wanted the tool to be so easy to understand you didn’t need to have to borrow other people’s content. So Galaxy is often a good starting point, but it’s often content people may want to customize.

What do you guys have planned coming up?

Right now (Jan 2015) a lot of work is going towards refactoring the main Ansible tree to make it more unit testable, but also more able to accept some new language features we want to include, which may include things like try/except blocks, more flexible iterators, and other things. We’re hugely grateful for a giant amount of contribution to the project, so a lot of this is about keeping it very easy to contribute to the project, and reorganizing some things where it has grown a bit too organically as a result of that contribution. Also, while we’re doing that, we’re merging all sorts of pull requests from everyone, for new features, keeping up with other APIs, clouds, and distros, and so on.

Can you please give some advice for someone wanting to start a project like this?

First, be sure you really want it, as having a huge community does affect you being able to do the things you want to do all the time. You begin to serve that thing instead of your original idea — and that’s ok. That being said, you’ll learn a LOT from them. For me, I think documentation was the biggest enabler, so a big shout out to Tim Bielawa for starting the Sphinx effort and J.P. Mens for making the module docs code-generated. Another big part of that was making things pluggable and easy to extend. There’s also this thing I’d like to call “Project Marketing” which nothing but the project authors can really do well – but telling your story, putting together a good website, and telling people why they should care about your project is important. There are a lot of GREAT projects on GitHub with bare READMEs that don’t explain themselves. Figure out who should care about your project, spend some time recruiting users, and explaining it, and that can go a long long way. Enjoy writing. I like giving talks too but I didn’t give an Ansible talk for a whole year — mainly because things spread through the internet these days. Anybody can start something at any time, but they have to know how to sell the idea. For instance, I think Rails isn’t as widely deployed as people may think, but Ruby guys are amazing at writing and pick really good fonts. (I’m serious about that). Presentation matters.

Popular search terms:

  • flyq7z
  • good morning funny fb status hindi

Chris Ueland

Wanting to call out all the good stuff when it comes to scaling, Chris Ueland created this blog, ScaleScale.