The idea of Measurement came from the incident of buying a dining table (http://www.novadeko.com.au/media/pdf/DIN-0301-0001.pdf). My wife was about to place order while I, as a peer and have my say in that consensus, stopped her. I took a copy of the table’s specification home and did a careful fit-for-purpose analysis based on the measurements (width, length and height). To my surprised, my wife’s eyeball judgment was so accurate. We had exactly 1803mm wide to fit the table between two cabinets. The table was ordered online and got delivered the following day. However, we failed to fit the table into the space. I measured and found that the width was 1805mm. When I demanded a replacement of the defective table, the sales person pointed out to me the last sentence in the specification -- tolerance of +/- 5mm. 1805 is still within specification! Although I had to deal with the unfit table, I admired the simplicity & explicitness of the specification.
In a more generic sense, Measurements can also be processing steps, such as steps for defrost, reheat, grilling & baking of a microwave oven. The act of measuring is to determine if the documented outcome can be produced by following the steps.
These measurements can be used for
- Fit-for-purpose analysis (packaged) / Deliberate Discovery (custom build)
- Specifications for developers
- Acceptance criteria
- Documentation & training
After agreeing upon the measurements with customer, the next thing is NOT to produce an automated measuring robot (like TDD). Showing the unfinished product to customer and obtain feedback is more important. Lastly, we build the automated measuring robot for final acceptance. Because we have the measurements and we know that final acceptance is done by the robot, testability is by design. There is no need to write the test first.
"Testing" or "Verification" are too "IT" but Measurement is very common in many disciplines. Different levels has different Measuring methods, targeting different audiences.
|Business Goal||Business Process||User Story||Program|
and their business
|As a ..., I want...
so that ...
e.g. a number
how to measure
|Execution steps.||Given, When, Then,
|Fit for purpose
|Do we need to
meet this target?
|Can we accommodate
|Are the features
fit our work?
|Are these our
|Test detailing after|
Add UI references.
|Prepare test data,|
|Write the test code|
|Months to Years||In parallel with|
|Within 1 iteration||In hours to 1 day.|
|Acceptance||Target met.||UAT Successfully
|All examples pass
with green lights
|All test cases pass
with green lights
Business Goal Measurement
Measuring method of the Business Goal should be specific. Is it a moving average? Should we eliminate maximum and minimum? Should we set a standard deviation? What is the baseline? Should we allow for +/- 5% tolerance? After fixing measurement methods, "measuring points" must be built into the application to make it "measurable". We will use these measuring points to measure after go-live for a period and even after several enhancement projects and determine if the Business Goal has been met.
Credible Scenarios as Specifications
Credible Scenarios are the scenarios you would encounter in real life. The measurement is the execution steps and UI guide. To illustrate the concept, I would like to borrow the publicly available SAP Best Practices Scenarios map for illustration. Click on the Scenario Map link and you can find all 21 credible scenarios related to the Business Process of "Sales", such as
- Sale from stock
- Using 3rd party with shipping
- Free of charge delivery
- Sales of planned service
To view the details of a particular credible scenario, click on the credible scenario and click the "i" icon. For example, execution steps of "Sales Order Processing: Sale from Stock" are:-
- Sales Order Entry
- Delivery Creation
- Post Goods Issue
- Create Billing
The big picture is the UML Process Activity Diagram on the left. It shows the start, all credible scenario flows and the termination across different stakeholders' boundaries. The red line marks one of the credible scenario flow, which we are going to decompose into execution steps.
Execution steps and UI guide as Measurements
However, I found that the traditional "waterfall" project management approach is more appropriate in managing "Test detailing", completing Fig 1 by producing Fig 2 & 3.
2011 to 2012 are two fruitful years full of innovations in the software specification arena. Gojko Adzic's Specification by Example; Paul Gerrard's The Business Story Pocketbook and my little attempt to add Measurements to my hybrid-Agile project. Surprisingly, SAP also published the whole set of SAP Best Practices Business Process Documents (BPD) -- Scenarios, test data, detail execution steps and UI operations. The only different from my version is the missing screen dumps! Below is extracted from SAP Best Practices' home page. Note (1) fit for purpose determination & (2) test execution. Because they are executable, they are also living documents.
SAP Best Practices include clear methodologies and step-by-step approaches. Every package has extensive reusable documentation for self-study, evaluation, as well as for project team and end-user training. SAP Best Practices provide complete pre-configuration settings that give you everything you need to run specific key processes with minimal installation effort.
These are the well known "As a ... I want ... so that" user story specification and the Gherkin "Given / When / Then / Examples" measurements.
Program (Business Rule) Specification & Measurement
Below is a business rule Specification example of converting a requisition into a purchase order:
A requisition item cannot be converted into a purchase order in the following cases:
- if the master data is faulty or incomplete
(for example, the vendor is blocked or there is no current order price)
- if Customizing is faulty or incomplete
- if input data is missing (for example, no account assignment)
Measurements are the unit test cases -- examples of when requisitions are not converted in PO and when requisitions are converted to PO.
We are not doing TDD. We design by specifying the measurements before coding but not writing the test before coding. We write code and invite user feedback before writing automated testing for these measurements. Code quality is still guaranteed because first, measurement is the design and second, we code with testing in mind (i.e. write testable code).