In-Depth
An Agile Primer
Agile development is growing in popularity for a reason: It results in faster time to market and, in most cases, better software.
Do you remember who the popular kids were in your high school? If you're like me, it was fairly easy to identify who was a part of the "in crowd" during your younger years. It's hard to describe what separated them from others, but in a subtle way it was obvious. The popular kids stood out. Others gravitated toward them, wanting to be included in the "inner circle." They were different.
In many ways, Agile practices are the "popular kids" of the software industry today. Most development shops have either tried them, are in the process of trying them or are planning to try them. Some shops are finding success; others are struggling to see what's different. Regardless, the popularity of Agile practices can no longer be ignored. They've arrived on the scene, and are becoming more and more mainstream.
If you're new to Agile software development, you'll want to start by reading and understanding the "Agile Manifesto."
In short, 10 years ago in Snowbird, Utah, a group of software professionals met to search for answers as to why some software projects are successful, and some are not. The result of that meeting was the creation of the Agile Manifesto: four value statements and 12 principles that define and guide the approach known today as Agile software development. The four pillars:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
The manifesto states up front that while there's value to each item on the right side of a statement (for instance, processes and tools), there's more value in the items on the left side of each statement (for instance, individuals and interactions). I won't comment more on the history of the manifesto, but will point out that the creation of the Agile Manifesto set the stage for the growing movement to adopt Agile practices. That movement has taken root and is now a mainstream part of the software development world.
Last summer, Dave West of Forrester Research Inc. published an article touting the rising popularity of Agile in today's organizations ("Water-Scrum-Fall Is the Reality of Agile for Most Organizations Today," July 26, 2011). In that article, West states: "Forrester has observed Agile changing from an approach practiced only by Agilists to one that many organizations employ. We continue to see Agile methods gaining popularity; in a Q3 2010 survey fielded by Forrester Research and Dr. Dobb's Journal, about 39 percent of 1,023 IT professionals said that they follow an Agile method." (See Figure 1 for a graphical breakdown of the survey results.)
[Click on image for larger view.] |
| Figure 1. Agile development is the most popular methodology today, according to Forrester Research Inc. |
Evolving Environments
Let's first take a look at why Agile is catching on. What's driving the popularity? What's different? The place to start is by looking at the environments in which software is used, purchased and developed.
I clearly remember my software buying habits back in the late '90s and early 2000s: I'd drive to one of the "big box" stores and peruse the shelves, flipping over product boxes to stare at the screenshots on the back. I'd then glance at the "requirements" section to make sure my PC at home had enough RAM and hard disk space to keep up. After making a decision, I'd buy the box, drive home, and spend the next few hours installing and configuring the software for use. This was normal. This was how we made software decisions. A lot has changed.
I honestly don't remember the last time I bought a piece of software that came in a box -- or even on a piece of physical media, for that matter. It's just not the way things operate in today's world. Instead we download, we "try" for free, or we buy from our phones or tablets with the touch of a finger. What used to require a car trip and a cardboard box now requires simply an Internet connection. If I don't like the result, I try something else. Switching is easy. The barriers and investment that used to exist are gone. Again, a lot has changed.
Because of this change, anyone building software has been forced to look for new ways to operate. There's an increasing need to make better decisions and eliminate waste. Diego Lo Giudice and Dave West from Forrester Research said it best in their article, "Transforming Application Delivery": "Firms today experience a much higher velocity of business change. Market opportunities appear or dissolve in months or weeks instead of years. This increased business velocity demands much greater agility from business systems."
A need has developed for a new approach to building software. Enter Agile processes and methods. They're new, they're lighter-weight and they provide the perfect environment for building software in today's world. In the remainder of this article, I'll outline a few key factors I feel are absolutely necessary to be successful with Agile. I won't spend any time focusing on specific practices (daily stand-up, pairing and so on) or methodologies like scrum and extreme programming. Instead, I want to examine the environment an Agile team works in and lay out a few concrete steps necessary for an Agile transition.
Building an Environment
I live in the Pacific Northwest and have for all of my life. Yes, the rumors you've heard are true -- it does rain a lot. Despite the rain, we get a reasonable amount of sun each summer and generally have a very pleasant climate. A few years ago my wife and I decided to experiment with a vegetable garden. We had some extra space in our yard and it seemed like a fun experiment to see if we could grow something that we would actually eat. This experiment led me to build a series of raised gardening beds in our yard where our bountiful harvest would thrive. At least, that was the plan. I actually ended up building the beds twice after learning that treated lumber has high amounts of arsenic in it. Arsenic and edible vegetables are not a good mix. Lesson learned.
The point is that to successfully grow vegetables, it was necessary to create a healthy environment for the vegetables to grow. This included picking a spot in our yard that received the right amount of sun. It included creating a place for the garden and constructing a place for the plants to live. It also included adding rich soil to the garden and eliminating weeds that would choke out the plants. In the end, we were quite successful. We had more tomatoes than we knew what to do with, zucchini coming out of our ears, and enough lettuce and carrots to keep us going all summer. The garden was a success, and the environment we created was critical to that success.
I have no doubt that if we had taken a different and more cavalier approach, our harvest would have been significantly less. Imagine if I'd simply wandered into my backyard and scattered seeds in random places, "hoping" they'd grow. We might have ended up with a zucchini or two, but I'm sure the harvest would have looked much different. Moving to Agile is no different. You can't just say, "We're going to be Agile," and expect good things to happen. You've got to go out and create an environment where new practices and new thinking will work.
3 Motivations
So, where do you start in creating a healthy environment? I believe the No. 1 place is motivation. Agile is about the team: the people building the software. To build a great environment for teams, you need to understand what motivates them. This brings me to a book I read a few years ago by Daniel Pink: "Drive: The Surprising Truth About What Motivates Us" (Riverhead Hardcover, 2009). It was a book I couldn't put down; it captivated me and opened my eyes to some truths about motivation.
In "Drive," Pink contrasts traditional motivation techniques (rewards and punishment) with three new essentials critical to motivation in today's environment: autonomy, mastery and purpose. These three essentials lead to what Pink describes as "intrinsic motivation." The idea is that we all crave autonomy, mastery and purpose in everything we do. When individuals and teams find these things, good things happen. Let's define them.
- Autonomy. The first essential presented by Pink is autonomy: The ability for one to make his own decisions. For a software development team, this is critical. A team needs to feel empowered, and a team needs to be empowered. Why? Because Agile teaches us that change is inevitable during the software development process. Learning about the product, the architecture and the customers will occur after the team has started executing. The team must feel empowered to react to change and communicate what it's learning. This doesn't mean the team should have the authority to do whatever it wants; rather, it gives the team freedom to do what's right. A team lacking autonomy is more likely to blindly follow a plan handed down from above while ignoring critical learning.
- Mastery. The concept of mastery is fairly simple: People want to get better at what they do. We find this in all areas of life. Part of our basic DNA involves the desire to improve. Toddlers work tirelessly to master the art of walking. They usually need very little encouragement. By nature, they have a deep-rooted desire to master walking, and they work at it until it becomes second nature. This doesn't change as we get older. I love to play golf and I have a desire to be a better golfer year after year. Improvement is part of my love for the game. I'm certainly not a master (my career on the PGA Tour hasn't yet materialized), but I continue to strive to improve hole after hole, round after round, year after year. The process and the results are exhilarating.
Software teams are no different. Every software team has a basic desire to find newer and better ways to accomplish the goal set before it. Members of the team love to build software and love to solve difficult problems. If they didn't, they wouldn't be in a difficult industry. The challenge and art of building an elegant yet functional solution drive them. Too often, however, we treat teams and the people on them as if they're pieces on a game board that can be moved, shifted and changed without consequences. We shuffle resources, change goals and randomize their priorities. By doing this we remove their ability to become masters. It's a pattern I see repeated again and again in the industry.
- Purpose. The final essential laid out by Pink is purpose. Teams have a deep-rooted desire to understand how their work fits into the big picture. As an engineer, I want to understand why the software I'm building matters. We need a reason, and a purpose. As a leader in your organization, find ways to ensure that your team understands how the value it produces is contributing to the business. Communicate the importance of new features delivered in the last sprint. Show them delighted customers benefiting from their solutions. Too often we disconnect our development teams from the business and from customers. We do this because we think they don't care, or we're worried that we'll distract them. However, more often than not we end up alienating them and destroying their sense of purpose.
Less Is More
The second key to creating a healthy environment for Agile is to eliminate waste and unnecessary steps in your process. I'm talking about those things you've done forever, but nobody is really sure why. I often refer to them as taxes, because more often than not, doing these things feels like paying taxes. They are things that "must" be done (at least according to some handbook or list of rules somewhere), and yet they are things the team rarely understands, likely cares little about and receives very little value from. To the team, these tasks feel like a black hole. High taxes will kill an Agile team.
If you're part of the leadership team in your organization, this is where you take a long look in the mirror. What taxes are your teams subjected to? Why are they necessary? If you're having a hard time coming up with a list, I can guarantee that if you walk up to any member of your team and ask for a few suggestions, you'll receive a lengthy list. Most lists like this are filled with steps designed to reduce risk. Don't get me wrong: I'm all for managing and minimizing risk. It's absolutely necessary to ship high-quality software on time. However, risk-management techniques can quickly lead to a high-tax environment.
I've found in all the organizations I've worked for that the process the organization follows tends to "snowball" over time. You pick up a new guideline or rule in each release, and before you know it your teams are spending so much time checking all the boxes on the release checklist that they end up with little time to create value for customers. The goal should be to create an environment where your team members can maximize the value they produce. The process should be as absolutely simple and streamlined as possible. Try eliminating a few of the taxes on your teams and see what happens. If they prove necessary, add them back; or better yet, engage the team on how to solve the problem long-term. Whatever the case, ensure that you're not burdening your teams with unnecessary or outdated taxes. A simple process is always a better process.
Steps to Becoming Agile
Now that we've established a few key principles to building an environment for Agile to succeed, I want to outline a few clear and concrete steps you can take in your own Agile transformation. With the popularity of Agile increasing, there has been a corollary increase in the amount written on the topic. The fact that you're reading this article proves that point. However, among all that "guidance" it's often difficult to understand where to start. How do you actually put any of this into practice? The statements from the Agile Manifesto sound great, but what steps can you take to make them a reality? In the sections that follow I hope to lay out some practical guidelines you can use in your own shift toward an Agile approach to software development.