UAT can also be Specifications, by applying Specification by Examples

Specification must contains measurements. It is too obvious. Like dining tables & electronic component. But it is not the case in the Software industry, at least in my past 20 years as software developer. Only recently did I learn how to write Software Specifications with measurements from Mr. Gojko Adzic's books -- Specification by Examples & Bridging the Communication Gap.

However, I found "Example" somehow too casual and not mean for serious use. Hence, I prefer calling it "Specification with Measurements". If you look at the specificaiton of an electronic component, there is a description in English as well as accurate measurements in tables and graphs for you to determine "fit for your circuit". First,  measurements determine "fit for purpose". Can the dining table fit your house? Second, measurements determine acceptance. The delivered dining table not according to the measurements should be returned.

Unfortunatly, I come from a company with a "Control" first and then "Competence" culture (ref. An Agile Adoption and Transformation Survival Guide by Michael Sahota). I have to adopt instead of transform. Hence, Water-Scrum-Fall is the maximum agility allowed. That is, we need to have an initial planning and scoping phase to produce something for the management to signoff. Then, we can execute the agreed number of Scrum iterations. Finally, users will accept the system when successfully executed a UAT.

Apart from "Water" and "Fall", the Scrum part is pretty stright forward. Specification by Examples is very effective and we pick it up quickly.

As shown on the left, unit test cases are our program specifications and SpecFlow story tests are our functional specifications.

How about the initial signoff document and the UAT plan?

I found that traditional user acceptance testing (UAT) is having a bad name in the Agile circle. 

"agile acceptance testing has very little to do with user acceptance testing. In fact, they are on completely opposite sides of a software project. User acceptance testing happens after development and it is often the final confirmation before money changes hands" -- Bridging the Communication Gap

The book Bridging the Communication Gap mentions that we can "have one or two acceptance tests for the whole larger workflow to verify key paths through the steps". Draw the workflow on the wall and keep an eye on the big picture when discussing user stories. However, "Agile acceptance testing is very different from user acceptance testing".

What a lonely path. I have to find my way.

The "Water" Phase and the signoff Document

From a high level, our business process oriented system is best described by a Busines Processes flow diagram and a set of Crediable Scenarios. Credible Scenarios can be "Procedure to handle complaints from our customer" or "Procedure to handle users applying for a refund". These Credible Scenarios are critical to the business and will form the core of the final UAT.

Hence, these Credible Scenarios, like user story examples, are also specification of the system in the highest level.

On the left is the Business Processes of the system and one of the Credible Scenarios marked in red. In our project, we have identified a total of 23 Credible Scenarios.

Horizontal segments are user parties responsible for a particular Business Process.

Next, we expand each Credible Scenerio into execution steps with description, purpose and expected result, as shown on the right.


In this initial scoping stage, the User Story and UI manual Reference column are left blank.

Next, we arrange meetings with working level stakeholders to go through each Credible Scenario, the execution steps and identified user stories. UI manual Reference is still blank for now.

These stories are prioritized, estimated and plugged into the initial release plan, say, of certain iterations. Besides prioritizing based on business value, users may choose to complete higher priority Business Scenarios first, such that they can execute them eariler.

The initial signoff documents are:

1. The Business Process Flow diagram
2. The Credible Scenarios with detail steps (Specification as well as UAT scripts)
3. A list of initial user stories identified
4. The initial release plan of a fixed number of iterations

The Implementation Phase

We establish a separate QA team to revise the affected Business Scenerios and execution steps after each iteration. 

They also develope the UI Guide with detail explaination of each step, screen dumps, fields and button definitions

On the left is an example of a revised Business Scenerio execution steps.

On the right is an example of a UI reference.

You can see that in parallel of Agile development iterations, the QA team gradually develope and complete the UAT plan with UI guide.

Actually, users will not wait for the completion of all iterations before executing UAT. Even when some Business Scenerios are half-baked, users are eager to partially execute them to gave feedbacks. Besides testing the system, they also test the execution steps and UI Guide. They also use this UAT document as their business process documents and training guide. When asked how to handle user complaints, they will refer to the execution steps and the UI guide.

The "Fall" Phase

During final UAT, end users will study the Business Scenarios and execute the scenario steps inside, like doing exercises in a text book.

Credible Scenarios and execution steps are similar to story test examples. It is the specification in the Business Process level.

The refinement of steps and the UI guide is like coding the SpecFlow test the make the UAT plan executable "manually".

On the left is the three levels of Specification by Examples

Addition Benefits

In additon to the UAT docuemnt, we also work out a Business Scenario & User Stories matrix for our future maintenance team. To change a Business Scenerio, you immediately know the  impact on user stories. To change a user story, you immediately know the impact on Business Scenerios.

Last but not the least, don't forget to measure the Business Goal. It is also a Specification with Measurement.