Traditional vs. Agile: The Future is Hybrid

Author: Adriana Beal

Even though the Agile Manifesto recently completed 10 years, there are still some lingering misconceptions about the agile approach within the BA community. In this post I’d like to address two points that are often overlooked by business analysts when they discuss agile methods:

There are multiple paths to project success

1. Waterfall is not the only alternative to agile

This is a point that should be obvious to everyone, but, based on many comments I see in blog posts and discussion forums, still seems to require clarification for many BAs who insist that either you are using an agile approach, or you are adopting waterfall.

As I wrote in one of the many “Agile vs. Waterfall” discussions in a LinkedIn group,

Plenty of BAs I know (including myself) never, or rarely, worked in waterfall projects. Long before the Agile Manifesto was created, many project teams were using evolutionary and “spiral” (iterative) development methods. (For a quick history of such methods, starting in the 1950s, please check the book Agile and Iterative Development: A Manager’s Guide).

It’s not exactly useful to compare the agile approach to a method that has so many obvious flaws that many organizations that don’t use agile would never dream of adopting either. In my opinion, a much more productive discussion would be when to use agile and when not to use it, rather than compare it to a method that never made much sense to start with.

Consultant Enterprise Architect Louise McDonagh added:

Adriana, I agree, one often feels one is banging one’s head against a brick wall when trying to explain to ‘business analysts’ that there are other approaches, not just ‘Waterfall’ and ‘Agile’. In fact, I will go so far as to say that those who are only able to get their minds around these two approaches are very limited BAs. A truly well-rounded BA knows that it is not the ‘approach’ that is important, but the ability to deliver what the business needs – whatever that deliverable is and whatever the environment.

Sure, the waterfall model had many proponents and practitioners over the past decades, but both before and after agile development methods became mainstream, successful projects continued to be completed using iterative models of software development that cannot be classified as “agile” (e.g., no self-organizing teams, no frequent release of code that you can run to see part of the system working). In fact, during the past ten years, out of the 2-3 projects a year for which I produced software requirements, only one followed the waterfall method, and none used a “pure agile” approach (at least not after I joined the team). With a few exceptions of projects that were cancelled due to changes in business strategy or insufficient budget, all other projects were successfully delivered and considered by their stakeholders to be closely aligned with the business and user needs.

2. “Pure agile” is not appropriate for all projects

I’ve seen many agile business analysts who have experienced success with one or a few agile projects develop the notion that Agile is always better than the alternative approaches to software development. Even when admitting the approach does not work in all cases, these BAs blame the organization, never a disconnect between agile principles/practices and business needs (“Of course agile won’t work for some companies! If your organization insists in extensive documentation, and is against embracing change and incorporating user feedback during development, then agile is not right for you”).

To me, this is like saying, “of course some kids don’t like books — if their parents don’t show them the value of reading or make reading a fun activity, they will never become good readers.”  The same way parents who are avid readers may raise a child who loves to read and another who never finishes a book (maybe because s/he is an auditory learner), companies will have projects that will thrive using an agile approach, and projects that won’t (maybe because the characteristics of the project simply do not mesh well with the agile model).

Making it look like it’s the company’s fault (because allegedly they don’t value getting things right early on, or prefer to stick to an ironclad plan created before all of the factors could be considered) is not beneficial to the conversation that needs to happen each time a project is about to start, about the best approaches to use to address that particular problem or opportunity.

Agile projects share certain characteristics (a backlog of work, self-organizing teams, rapid, incremental delivery of new functionality to users) that aren’t necessarily value-adding for all projects and under all circumstances. Just because a company decides not to use agile in some or all of their projects, it doesn’t mean the organization is inflexible, or backward, or closed-minded. There are many valid reasons to consider alternatives to agile, and adopting a non-agile method does not necessarily mean you are losing speed, collaboration or adaptation, or even getting near the extreme of single-pass, document-driven, waterfall development.

Can we move on to more important discussions now?

Many organizations are successfully using hybrid models that incorporate elements of agile and non-agile techniques with the goal of quickly introducing new features while ensuring proper system documentation and utilizing traditional business analysis techniques to help keep a project on track and reduce risks, uncertainty, and requirements churn.

As Senior Consultant Bhuvan Unhelkar points out (in Agile Product & Project Management, Vol. 11, No. 1 — 2010, Cutter Consortium),

My understanding is that, in practice, successful agilists tend to bring together a number of activities, tasks, and deliverables that are from beyond the boundaries of what may be called “pure agile”. This mixing and matching of software process elements from agile and non-agile (more formal) approaches is a much more practical way of using these methods.

We business analysts should be leading the discussion of what is effective for different types of software development projects, and when agile methods and practices will work and when they won’t, rather than perpetuating the worthless Agile vs. Waterfall debate that sadly is still going strong in so many BA circles.

>> Learn More About Agile Requirements

Check out the Agile Track of Crafting Better Requirements, a virtual, instructor-led course that will also earn you 21 PDs or CDUs. You’ll learn about how to create a product backlog, turn stakeholder needs and wants into user stories, and create acceptance criteria to support testing.

Click here to learn more about Crafting Better Requirements – Agile Track

Free Training - Quick Start to Success

(Stop the frustration and earn the respect
you deserve as a business analyst.)

Click here to learn more

By signing up, you agree to our Privacy Policy.

Comments

  1. Adrian, thank you for adding your comment.

    You know, many agile proponents would argue that the most of the complains you have about agile are in fact problems associated with traditional methods: they are confusing, they are too complex, they disregard the implementation of time critical requirements. I suppose these issues could be overcome by a talented agile team (in the rare situation you are able to find team members with the right skill set), the same way a talented team using a traditional software development model would be able to use the right practices to complete a project successfully.

    However, your point #1 is indeed considered by some researchers one of the weaknesses of agile methods. Many agilists say that it’s possible to perform enough planning upfront to avoid a flawed architecture, but I’m yet to see an agile project for a complex software that didn’t get in trouble with significant rework required to fix problems resulting from too little time spent understanding the big picture before jumping into piecemeal development. For this reason, I’m OK with using agile in small projects (say, a mobile app with very specific features), but not for large projects like the one you mentioned involving multiple organizations and stakeholders.

  2. Here are a few issues I have with Agile:

    1. Instead of defining a comprehensive solution architecture it defines a piece-meal/stove-piped architecture that has to be refactored. An SDLC which plans to deliver a flawed architecture is flawed. Does Honda refactor its cars? Does Poulty refactor its houses? I don’t get it.

    2. It doesn’t scale to large projects with multiple organizations/stakeholders. For example, I’m working on project today involving several federal agencies that don’t have the bandwidth to participate in sprints/scrums. Instead, these organizations rely on documentation (e.g., requiremement specs) and key milestone reviews (e.g., solution requirment reviews) which don’t exist within the Agile SDLC.

    3. It disregards the implementation of time critical requirments. For example, I worked on a project which was suppose to implement time critical legislative requirements. The Agile developers would continually move requirements from the development sprint to the product backlog. As you can imagine, the final sprint was a nightmare.

    4. It’s confusing. People think linearly and Agile is non-linear. Agile also introduces a whole new vocabulary which is confusing to many stakeholders. Simple is better. Many projects fail (i.e., don’t deliver the agreed to scope on time and on budget) as a result of an improperly executed SDLC so it’s important for the SDLC to be as simple as possible.

  3. Hi, Daniel!

    Thanks for leaving your comment.

    “Tailored approaches assume lack of rigidity on practices, roles and processes. That “is” being Agile. Doing the minimum needed for a project to succeed isn’t hybrid, it’s Agile, or Lean if you prefer. Agile by definition is the lack of formality and hard methodology, so that teams can shift, adapt and do what’s best, instead of what the Methodology prescribes.”

    I know a couple of people who share your views, but we may need to agree to disagree in this respect. Looks like our divergence is rooted in the fact that to me (and to many agilists reputable authors I exchange ideas with), to be called “agile”, an approach/method/framework needs to follow the 12 agile principles behind the Agile Manifesto. Based on this understanding, you could “lack formality and hard methodology”, and even adapt from lessons learned, and still not be able to call your approach agile — or even a hybrid between traditional and agile.

    “Finally, I beg to differ on delivering a legacy system replacements in one go. It is actually better to strangle the old system with small incremental deliveries. It reduces risks and avoids unwanted maintenance in 2 places. Big replacements fail more often than not.”

    I’m truly curious how you propose we achieve this goal! Obviously I don’t dispute the fact that incremental deliveries are better, when they can perform a complete enough function to be useful to the business. But how do you suggest users complete their work at the point where 30% of what they need to do is already available in the new system, but the remaining 70% can only be performed in the old one? I don’t see it being possible even with maintenance in 2 places, let alone avoiding this duplicate work :-).

  4. Kent,

    “Agile is best suited for High uncertainty, low complexity projects and some aspects of High Uncertainty, High Complexity projects. (Look up Context Leadership Model for more information).”

    Agreed — I don’t know the model you mentioned, but have my own matrix that provides the same conclusion :-).

    “That said, organizations do sometimes get in their own way with respect to positioning teams to effectively complete projects.”

    That’s very true, thank you for adding that observation.

    Many times we see a company declaring they are “going agile” while refusing to abandon their plan-driven, budget-scope-schedule approach, obviously with disappointing results. Blaming agile for their poor results in such cases is definitely not fair.

  5. Adrianna, I will agree that not all projects are best suited for agile. You can look at the uncertainty and complexity of a project as a guide to determining what approach is appropriate. Agile is best suited for High uncertainty, low complexity projects and some aspects of High Uncertainty, High Complexity projects. (Look up Context Leadership Model for more information).

    I also think that in some cases, agile apologists are too quick to blame poor agile implementations for project failures that were more a result to a poor fit in approach.

    That said, organizations do sometimes get in their own way with respect to positioning teams to effectively complete projects. As an example, organizations that set up measurement and reward structures that encourage people to only do their narrowly defined roles are building in limits to collaboration. That is unfortunate regardless of what approach the organization uses.

  6. For anyone interested, Here’s an article by Johanna Rothman, published right after this post, on the same subject:

    http://www.jrothman.com/blog/mpd/2011/09/agile-is-not-a-silver-bullet.html

    “Agile is not the only approach. Yes, integrating testing with development makes a ton of sense, and you get that with incremental lifecycles. Yes, iterations make a ton of sense, and you get that iterative lifecycles. You can even combine iterations with increments, and still not have an agile lifecycle. Release trains, for example, are an iterative and incremental lifecycle that are not agile, because they do not have a product owner, and do not incorporate fast feedback. But, they are better than waterfall. They are even better—for some people—than straight staged delivery.”

  7. @Duane: lol, thanks for the public display of admiration.

    I think we are not even disagreeing here. Debates about which BA and software development practices and techniques work, and under which circumstances, are always useful and capable of teaching us new things.

    The debate I say is worthless is the one that starts from the assumption that there are only two models: agile and waterfall. There are many other approaches that have been in use for many years. Some of which are much more appropriate for certain types of project than agile. By reducing the conversation to how agile is better than waterfall, BAs lose a valuable opportunity to have a much more meaningful conversation about requirements and software development practices, and when they add value or are detrimental to a project.

    I’ll give you an example: a mission- and life-critical system will hardly benefit from a “pure agile” approach (among other reasons, because evolutionary requirements could create an unacceptable risk of loss of system reliability). Does it mean the only other option then is to use waterfall? Of course not!

    This type of project would require a more lengthy and explicit requirements phase at its early stages than agile approaches propose, but it doesn’t mean it would not benefit from agile practices such as constant customer involvement and frequent user feedback.

    When you treat the problem as a matter of choice between two clearly delineated, opposing models (in one extreme, waterfall; in another, agile), you miss a huge opportunity to use integrative thinking to “constructively face the tensions of opposing models, and instead of choosing one at the expense of the other, generating a creative resolution of the tension in the form of a new model that contains elements of the individual models, but is superior to each” (to quote from Roger Martin).

    I don’t think we are talking about the same types of debate here — if you were learning from the discussions, they are most likely not the ones I am complaining about ;-).

  8. Duane Banks says:

    Adriana,

    You know I’m your #1 fan. But I disagree with your “rather than perpetuating the worthless Agile vs. Waterfall debate” comment.

    I learned a lot from some of those debates. Perhaps others have (and will) too.

    And if it hadn’t been for those “worthless” debates, perhaps you wouldn’t have consolidated your thoughts on the matter with this article :).

  9. Akarsh,

    Indeed, focusing on what adds value for each individual project is definitely the best strategy. Even when a project may benefit from an approach more to the “agile side of the spectrum”, not all agile practices may be suitable for the project, and the ability to identify what practices to adopt or discard based on the value added to the project is a skill that all IT teams should aim to develop.

    For example, frequent delivery of working software may be irrelevant if you are replacing a legacy system with many points of integration, that would need to be delivered as a whole rather than incrementally in order to provide value. In this case, the emphasis on short development cycles may not be adding any business value. That doesn’t mean you wouldn’t want to take advantage of other agile practices, such as frequent feedback from users (which can be obtained via alternatives other than working software, such as prototyping or visual requirements). Tailored approaches always win!

    • Tailored approaches assume lack of rigidity on practices, roles and processes. That “is” being Agile. Doing the minimum needed for a project to succeed isn’t hybrid, it’s Agile, or Lean if you prefer.
      Agile by definition is the lack of formality and hard methodology, so that teams can shift, adapt and do what’s best, instead of what the Methodology prescribes.
      Finally, I beg to differ on delivering a legacy system replacements in one go. It is actually better to strangle the old system with small incremental deliveries. It reduces risks and avoids unwanted maintenance in 2 places. Big replacements fail more often than not.

  10. @Adriana – I would have to say i have been the one among them who always use to jump in to this conversation saying Agile is the future.

    But after interacting with key stakeholders for last couple of years most of them always were never bothered about what approach i follow it, they were always keen to know what value we will be adding to their business every year with new ideas.

    Approaches we follow should be only limited for internal verifications and this should help the teams to focus more on adding values for customers and striving for growth.

  11. Akarsh,

    Thanks for joining the conversation. I completely agree that Louise hit the nail on the head with her comment.

    I’ll now start pointing to this article any time I see again the conversation “agile is much better than waterfall”. So what? Many IT teams never worked or recommended waterfall anyway. We need to have deeper conversations — debating, for example, when short iterations, or frequent releases, or onsite customers provide true business value, and when they don’t. It’s naive to expect “pure agile” to work in all contexts — the sheer volume of research attempting to rebuild and tailor these metods to different environments indicate this is not as simple as it seems.

  12. Great article; you have highlighted some important points with some good examples!
    As you pointed out most of the LinkedIn BA groups the discussion on this topic is going on for a long time.

    I really like the comments which you had added by Consultant Enterprise Architect Louise McDonagh; all it matters is “ability to deliver what the business needs” irrespective of the approach we follow.