In my previous post I described my experience as a business analyst on an agile project.
One of the key artifacts I produced on the project was the functional specification (FS). In this post I’m going to get right under the covers of the FS and explain exactly what it was and how it worked. My intention is to provide enough detail so that others could actually try it out on their own projects. So you’ll have to excuse me if I go on a bit – I’m sharing some love here. I’m not saying it’s perfect, but it worked for me!
To help this article make more sense, you might want to download a copy of the Functional Specification.
Can a Functional Specification really be Agile?Is there any such thing as an agile functional specification?
Given that the agile movement values working software over comprehensive documentation, you might well ask whether there is any place for a functional specification on an agile project. In Extreme Programming, the requirements are conveyed verbally, directly to the developer, with just a few notes scribbled on an index card as an aide memoire. The only documentation is the code itself plus the accompanying suite of automated acceptance tests.
The debate as to how well this approach works in practice is ongoing. Do good developers make good analysts? Is no documentation enough? Does the analyst role help or hinder the overall process? I’m not even sure which side of the fence I sit on myself yet!
Either way, a “no analyst, no documentation” approach wasn’t appropriate for this particular project – primarily for geographical reasons. The business team was located in Leeds (UK) and the development team was 200 miles away in Exeter, with no real option to re-locate. Maintaining a rich verbal dialogue directly between the two groups was always going to be tricky, and it made more sense to have a dedicated business analyst (i.e. me) bridging the gap.
And besides, I would have written myself out of a job! But I guess I don’t have to work too hard to justify the existence of a BA on this particular blog .
<Start Editor’s Interruption>: Tony, no you don’t! But we often need help justifying our existence to others, especially on agile projects! While I’ve already interrupted you, I do want to let any new readers know that if you are looking to save time on your documentation by not starting for scratch, but want some very practical and short templates to start from, I’ve made mine available as part of our Business Analyst Template Toolkit.</End Editor’s Interruption>
However, this was not like any functional specification I had ever written before. I was keen to avoid all the excess baggage that normally comes with an FS and try to keep things as pared down as possible. Let’s have a look at some of its key features, and hopefully you will agree that it is possible to have an agile functional specification.
Purpose and Objectives
It’s worth stating this explicitly, because different people seem to have different views on this…
The purpose of my functional specification was to specify the externally-visible behaviour of the system, in enough detail to allow the development team to build the system.
It existed alongside an (HTML) prototype which provided guidance on the user interface for the system.
It was in my view a design document (a functional design, not a technical design). Some people would call it a requirements document, but to me that’s just down to terminology, and perhaps mindset. But that’s another article.
Format and Authoring Tool
The first and most notable feature of my functional specification is that it was a spreadsheet. In the past I have always used a word processor for capturing requirements, so this was a bit of a change for me.
Why did I decide to do it this way?
Well, I knew I wanted to capture the system requirements as user stories, complete with acceptance criteria. And I also knew I wanted to create a living document that could easily be updated throughout the project – in particular the Product Backlog (the master list of user stories). I was initially considering using an agile CASE tool such as Mingle, but in the end I decided to go for something more low tech and see how it went. So I chose MS Excel. You don’t get much more lo-fi than that, although I suppose if you were a masochist you could always do the whole thing in Notepad!
Some of the benefits of using a spreadsheet will become apparent as I explain the other features of the FS – the ability to add columns for various purposes being particularly useful. But one of the key benefits came as quite a surprise, and deserves a special mention.
I’ve always found then when using a word processor to author a document (like now, for example), I spend a lot of time on the format, structure and grammar. I think that’s because, in my mind, poor presentation reflects poor content. What happens then is the document becomes an end in itself, rather than a means to the real end (working software).
But with a spreadsheet it was different. I felt no compunction to perfect the structure and grammar of each sentence, nor was there any real need to format the document very much. I was free to bang out the content and let the ideas flow. It was extremely liberating! And it saved a whole load of time too.
In summary, the spreadsheet allowed me to focus on content over form.
The ‘Summary’ tab within the FS contains a list of all the user stories (in a Scrum project this would be called the Product Backlog). This is where new stories got added as I worked through the project’s Feature List (which was really a kind of wish list) and elaborated the features into something more concrete (more on the process later).
There are quite a few columns on the Summary tab. Most of them are self-explanatory, and some of them also have comments against the header to help out. Here are a few useful notes:
- Story ID – I gave each story a unique ID, using the format ‘Snnn’ e.g. S001, S002 etc. The ‘S’ prefix turned out to be very handy because stories could be referred to by their IDs alone without having to bother with qualifiers like ‘story number’, as in, “Hey Joe, I’ve just been looking at S078 and I have a question…”
- Story Name – A simple name for the story, rather like a Use Case name – which was quicker and easier to digest than the “As A…I Can…” summary.
- Status - indicated where the story was in the delivery lifecycle – with colour coding – so we could all easily see which stories were ready for development, test, fixing etc. and which were withdrawn.
- Priority – I gave each story a MoSCoW priority, which sort of helped to work out what order to deliver the stories in, but in reality deciding the order was much more complicated (and subtle than that) – we needed to take into account inter-dependencies and also which stories we most wanted to show to the users sooner.
- Dev Increment – which increment the story had been allocated to. Of course, stories were only allocated to increments at the start of each increment once we had done the estimates and worked out what would fit.
The ‘Details’ tab contains the meat of the specification. Each story from the ‘Summary’ tab is expanded out into its full form.
I spent quite some time trying to find this full form. User stories are typically defined in terms of acceptance criteria – which state what it means for the story to be “done”. But badly-written acceptance criteria are as bad as badly-written requirements, and there are two pitfalls that I have found:
- Ambiguity – not giving enough detail
- Lack of context – not saying when or how the acceptance criterion applies
So rather than having an unstructured list of acceptance criteria, I decided to work them into a tried-and-tested format: use cases. And in particular, Alistair Cockburn-style use cases. I used a two column format – the “When…” column states the actor’s (user’s) actions and the “Then…” column states the system’s responses (which also double up as the acceptance criteria). If I had multiple acceptance criteria against a single action, I put them all on separate rows, so that I could track them individually.
Here’s an example of a “Log In” story:
|Story ID||As a…||I can…||So that…|
|S001||User||log in to the system||I can gain access to restricted areas of the site|
|Main||1||the user is not logged in|
|2||the user requests to view any “restricted” page|
|3||the system displays the log in page|
|4||the log in page includes data entry fields for user ID and password|
|5||the maximum length for user ID is 16 characters|
|6||The password field is a “masked” data entry field (it does not display the characters the user types)|
|7||the user enters a valid user ID and password, and requests to log in|
|8||the system changes user’s status to “logged in”|
|9||the system captures an audit trail of the date and time the user logged in|
|10||the system takes the use the the page they initially requested|
|7a||1||the user enters a valid user ID but an invalid password|
|2||the system re-displays the log in page|
|3||the system displays an error message: “Incorrect user ID or password”|
|7b||1||the user enters an invalid user ID|
|2||the system re-displays the log in page|
|3||the system displays an error message: “Incorrect user ID or password”|
- The “When…Then…” terminology is borrowed from Behaviour Driven Development, but really, what you are looking at here are use cases.
- The “Flow” column identifies the main (“happy day”) flow and the alternative flows of the use case. For example, flow 7a is the first alternative flow to branch off main flow step 7, and flow 7b is the second to branch off the same step.
I used to spend a lot of time deliberating over the exact language and level of detail in my use cases – making sure I used consistent terminology for all my actors and entities; trying not to get too drawn in to UI design. This time I threw caution to the wind and just wrote down, as quickly and as clearly as possible, what I wanted the system to do. Some acceptance criteria might be quite high level (e.g. “the system displays the log in page”) whereas other might be very specific (e.g. “the maximum length for user ID is 16 characters”). If there was some detail that I cared about, it went in. If I didn’t care too much, or if I was confident the developers would get it right without too much guidance, I left it out.
I also used to spend valuable time making sure my use cases were cumulative – so if I needed to modify or extend an existing system function, I would extend or modify the existing use case (being careful to mark up the changes to show what is new) – thus providing a useful document to refer back to in future. Whether or not this is worth the effort is a hotly debated topic. At the moment I fall on the “it’s not worth the effort” side of the debate, and so if I want to modify or extend an existing function, I just write a new story to describe the changes. Easy!
A Day In The Life
A large part of what made the FS agile is more to do with how (and when) I created it.
One key point is that I wrote the FS very much “just in time”. We were operating an incremental development process – each increment was 3 weeks long, so in any given 3 week period I was working on elaborating the stories that would go in the next increment. More precisely, I elaborated enough stories to be sure of having enough scope for the next increment – because the exact scope wasn’t known until we did the increment planning session at the start of the increment.
Stories were elaborated in priority order, and as we worked our way through the increments, the stories were continuously being re-prioritised, based on our unfolding understanding of the overall system. After some of the increments we made the system available to users (a sort of soft UAT) to get their feedback. This led to a whole raft of proposed tweaks, which were added into later increments as new stories – duly prioritised alongside all the other stories.
As mentioned earlier, I also developed an HTML prototype in parallel with the functional specification, which mainly served to illustrate the concepts in the functional specification but also provided look & feel guidance for the developers.
On this particular project, I was lucky enough to have ready access to my business stakeholders. So I ran twice-weekly workshops in which we went through the latest stories. Using my laptop and a projector, I had the HTML prototype up on the big screen for all to see and the FS on my laptop screen, so the stakeholders had something visual to look at but I had the details in the FS to refer back to. I would read out the acceptance criteria, get agreement around the room (or amend as appropriate) and then insert today’s date in the “business agreed” column in the FS. The design was signed off there and then, and the users never even had to read the spec. To my mind that was a big advantage of this process.
A Living Document
The FS was, of course, a living document – whilst the developers were using it to work on increment N, I was adding stories for increment N+1, and also amending the stories in increment N when the dev team spotted holes in the plot.
It lived on a shared network drive so that everybody could easily access it. To keep things simple, we didn’t use the “sharing” feature of MS Excel, so that meant only one person could edit it at a time (MS Excel handles the locking and notification for you – pretty nifty). Mostly this wasn’t a problem, but occasionally we had a bit of contention. That said, I don’t think this approach would scale beyond a team of around 6 people.
Not content with using a spreadsheet as a CASE tool, we also doubled it up to use it as a test tool too! The testers (myself and one other) used the use cases directly as test scripts, and if you look at columns N through R on the ‘Details’ tab, you will see that the spreadsheet also captured the results of testing, as well as any defects found. Again, this was a pretty lo-fi solution which wouldn’t scale that well, but it did have the big advantage that the developers were able to see exactly what has failed testing and in what context, without having to cross-refer to multiple documents.
Tracking and Progress Monitoring
If you know your agile, you will know that burn-down charts are all the rage for tracking progress. In my FS there was a separate tab showing a burn-down chart for each iteration. Actually, my charts were burn-up charts because they tracked work done, rather than work remaining, but the principle was the same. The burn-up charts were calculated automatically based on the date entered in column K of the ‘Summary’ sheet. Note that they only showed development complete, not test complete.
As I mentioned at the start of this article – I don’t claim that my FS is an agile analyst’s silver bullet. Here are some thoughts on its overall success on my particular project.
What went well:
- Everyone liked the low-tech approach and lack of enslavement to tools
- Business users enjoyed not having to read and sign off documentation
- Business analyst enjoyed the freedom to be slap-dash with language and terminology!
- No complaints over lack of clarity in requirements
Could do better:
- Solution won’t scale well (beyond a team of around 6)
- As the document grew it became a bit tricky to track through it looking for outstanding defects to fix/test.
There is a lot of talk these days about the system being documented as a series of executable acceptance tests. It seems to me that the next natural step would be to move towards the BA and/or QA writing these tests themselves (currently it is seen as a developer task). Behaviour Driven Development seems to be heading in this direction, but can we get to a point where executable tests can be written by a non-programmer? Or at least co-written? I have tried writing executable acceptance tests directly using Selenium, but I find that I get too bogged down in the technical aspects of getting the tests to work, and get drawn away from the analysis task. So we’re not quite there yet.
Heading off in a totally different direction, on my current project we are creating functional specs using Visio – and focusing on a much more visual format for our designs. The jury is out on how well that works, but when I get some feedback I might share it with you!