The technology industry is disrupting everything, including itself, and has been since the inception of Agile in the early 2000’s. Agile was born out of a need to embrace the rate of change by the very industry that created the disruption.

Things continue to change so rapidly that it's very difficult to formulate thoughtful long-term plans that are unchanging. “Today, the requirement of an organization is to be adaptable, flexible, and responsive to what's going on in the marketplace while it's going on,” notes Richard Sheridan, CEO of Menlo Innovations. Menlo is a software design and development company that has a unique approach to working on products. Its mission is to eliminate the human suffering of project managers, end users, and project sponsors, and its methodologies are Agile-based.

Allen Holub, a noted Agile transformation educator and consultant, describes the Agile environment, “Agility is being able to adapt quickly to the market in order to increase revenue. Often, people aren't even thinking along those lines. The business side people are still trying to do quarterly plans and all the normal stuff that they're used to without thinking that they have to be planning in a dynamic way. They have to learn how to leverage feedback based on very fast releases in order to make business decisions that are changing constantly based on what the market is doing at any given moment.”

Holub makes no bones about where he stands when it comes to measuring Agile ROI. “What interests me is trying to find something that is going to have so much value to my customers that if I do a good job building it, I can sell it. What I want to focus on is not some magical number that comes out of guesswork, but instead acknowledge I've got a specific burn rate and decide the best way to spend that money in order to maximize my return. And the best way to maximize return is by spending the money on the thing that has the most value because that's what's going to sell for the highest price. So, trying to figure out ROI is just ridiculous.” He paints an unflattering picture of the process, “ All those business plans with projected incomes and stuff, that's just made up garbage, and everybody knows it. The VCs know it, the people that are writing the plans know it, but you have to go through the song and dance. There's not a VC on the planet that makes an investment based on those numbers.”

Sheridan is equally skeptical and has some interesting responses when people start talking about Agile ROI. He believes that when an executive team starts asking about ROI, they’ve already lost the argument because he’s never seen an executive ask about ROI for something they truly believe in. “I will sometimes cynically ask people, absolutely, how do you track it today? Let's compare ROI of Agile versus the ROI you're measuring of everything you do today. And of course, you often get a blank stare at that point because they're not tracking ROI on the things that they're already doing today. So now the question is, even if you start tracking it now, what are you going to compare it to, to understand what you're doing better and different?”

All flippancy aside, Sheridan emphasizes that Menlo talks about joy a lot in every context of its business, which is unusual. He points out, “The questions become, for every organization, who do you serve, and what would delight look like for them, and how would you measure that delight in the world?” Adding, “Ultimately, at its core, what Agile is about is making sure we don't make big mistakes by releasing something ‘inappropriate' to the world, and I don't mean inappropriate like, oh I'm embarrassed.” Organizations using value streams to build products provide a continuous flow of value to a customer—avoiding the let-down of a waterfall-based project that can't be tested (or validated for market fit) until the team is done with the entire project.  According to Dean Leffingwell creator of the Scaled Agile Framework (SAFe®), "Value streams deliver faster learning cycles, shorter time-to-market, higher quality/productivity, and leaner budgeting mechanisms."

The conundrum when measuring an Agile transformation is that while you have to understand where people are at the beginning of the transformation, where they are in the middle and where they are at the end, anyone who's gone through one will, if they're honest, humbly declare they're never done. There's never a point where a team can say okay, we finally made it. When you're Agile and you're finished, you measure again.

Sheridan recommends, “I would indicate some of the things leaders should be looking at, whether they can tangibly be measured in a spreadsheet or not, are the teams’ human energy, perhaps via engagement surveys. Do people feel better about working at your company?” He says he’s watched decision-making getting pushed down into the organization, closer to the workers, and that the result is the workers themselves feel better. They're not waiting for somebody to make a decision above them before they can make their next move. Happy workers increase talent retention and build an experienced team. He adds, “The other thing to measure is the quality of the conversations between the business side of the house and the technology side of the house.”

Holub advises leaders to focus on what their budget is, how many teams can they run on that budget, and then look at defining a product that will solve, or find a system that will solve, a real user's real problems. He recommends figuring out what's most valuable to customers and then working on that. If the return is not adequate, then work on something else. It's a very lean way of looking at things. The lean feeling on productivity is that if the focus is on process improvement, productivity will take care of itself. Holub advises,  “Agile is a very different way of looking at the world, and trying to apply accounting thinking and governance that was designed for a different way of doing things is asking for dysfunction. When you change what you're doing in a radical way, you need appropriate support systems for the process, whatever that process is. So support systems that are appropriate for waterfall are not appropriate for Agile.”


»Read next on Emerge: 

In 10 years, Agile becomes business management framework

Agile and TBM: transforming the business of IT for the better