What's Wrong With Agile Methods - Part 4
Some Principles and Values to Encourage Quantification

Tom Gilb, http://www.gilb.com/
Lindsey Brodie, Middlesex University

The customers responded very favorably (see Figure 9).

"I just wanted to let you know how appreciative we are of the new ‘entire report’ export functionality you recently incorporated into the Reportal. It produces a fantastic looking report, and the table of contents is a wonderful feature. It is also a HUGE time saver."

Figure 9. An example of pilot customer (Microsoft) feedback

On the second release (Confirmit 9.0) using Planguage, and specifically the Evo method, the Vice President (VP) of Marketing proudly named the Evo development method on the FIRM website (see Figure 10. A line executive bragging about a development method is somewhat exceptional!).

"FIRM, through evolutionary development, is able to substantially increase customer value by focusing on key product qualities important for clients and by continuously asking for their feedback throughout the development period. Confirmit is used by the leading market research agencies worldwide and Global 1000 companies, and together, we have defined the future of online surveying and reporting, represented with the Confirmit 9.0."

Figure 10. Comments by FIRM’s VP of Marketing, Kjell Øksendal

Details of the quantified improvements were also given to their customers (see Figure 11, which is an extract from the product release for Confirmit 9.0 published on the organization’s website).

News release

2004-11-29: Press Release from FIRM

New version of Confirmit increases user productivity up to 80 percent

NOVEMBER 29th, 2004: FIRM, the world’s leading provider of online survey & reporting software, today announced the release of a new version of Confirmit delivering substantial value to customers including increased user productivity of up to 80 percent.

FIRM is using Evolutionary (EVO) development to ensure the highest focus on customer value through early and continuous feedback from stakeholders. A key component of EVO is measuring the effect new and improved product qualities have on customer value. Increased customer value in Confirmit 9.0 includes:

* Up to 175 percent more intuitive user interface*

* Up to 80 percent increased user productivity in questionnaire design and testing*

* Up to 1500 percent increased performance in Reportal and Panel Management*

Figure 11. Confirmit 9.0 release announcement from the FIRM website http://www.firmglobal.com. It gives detail about the method and the quantified product results

Impact on the developers

Use of Evo has resulted in increased motivation and enthusiasm amongst the FIRM developers, because it has opened up ‘empowered creativity’ (Trond Johansen, FIRM Project Director). The developers can now determine their own design ideas, and are not subject to being dictated the design ideas by marketing and/or customers, who often tend to be amateur technical designers. Daily, and more often, product builds, called Continuous Integration (CI, using Cruise Control), were introduced. Evo combined with CI, is seen as a vehicle for innovation and inspiration. Every week, the developers get their work out onto the test servers, and receive feedback.

By May 2005, FIRM had adopted the approach of using a ‘Green Week’ once monthly. In a Green Week, the internal stakeholders are given precedence over the client stakeholders and can choose what product improvements they would like to see implemented. The FIRM developers chose to focus on the evolutionary improvement of about 12 internal stakeholder qualities (such as testability and maintainability).

Initial difficulties in implementing Planguage

Even though Planguage was embraced, there were parts of Planguage that were initially difficult to understand and execute at first. These included:

Defining good requirements (‘Scales’ of measure) sometimes proved hard; (they only had one day training initially, but after the first release saw the value in a weeks training!)
It was hard to find ‘Meters’ (that is, ways of measuring numeric qualities, to test the current developing quality levels), which were practical to use, and at the same time measured real product qualities;
Sometimes it took more than a week to deliver something of value to the client; (this was mainly a test synchronization problem they quickly overcame).
Testing was sometimes ‘postponed’ in order to start the next step. Some of these test postponements were then not in fact done in later testing.

Lessons learned with respect to Planguage, especially the Evo method

Some of the lessons learnt about the use of Planguage, and especially the Evo method, included:

Planguage places a focus on the measurable product qualities. Defining these clearly and testably requires training and maturity. It is important to believe that everything can be measured and to seek guidance if it seems impossible;
Evo demands dynamic re-prioritization of the next development steps using the ratio of delivering value for clients versus the cost of implementation. Data to achieve this is supplied by the weekly feedback. The greatest surprise was the power of focusing on these ratios. What seemed important at the start of the project may be replaced by other solutions based on gained knowledge from previous steps;
an open architecture is a pre-requisite for Evo;
management support for changing the software development process is another pre-requisite, but this is true of any software process improvement;
The concept of daily builds, CI, was valuable with respect to delivering a new version of the software every week;
It is important to control expectations. ‘Be humble in your promises, but overwhelming in your delivery’ is a good maxim to adopt;
There needed to be increased focus on feedback from clients. The customers willing to dedicate time to providing feedback need identifying. Internal stakeholders (like sales and help desk staff) can give valuable feedback, but some interaction with the actual customers is necessary;
Demonstrate new functionality automatically, with screen recording software or early test plans. This makes it easier for internal and external stakeholders to do early testing;
Tighter integration between Evo and the test process is necessary.

Conclusions of the Case Study

The positive impacts achieved on the Confirmit product qualities has proved that the Evo process is better suited than the Waterfall process (used formerly) to developing the Confirmit product.

Overall, the whole FIRM organization embraced Planguage, especially Evo. The first release, Confirmit 8.5 showed some of Planguage’s great potential. By the end of November 2004, with the second release (Confirmit 9.0), there was confirmation that the Evo method can, consistently and repetitively, produce the results needed for a competitive product. Releases 9.5 and 10.0 of Confirmit continued this pattern of successful product improvements delivered to the customers (as of November 2005).

It is expected that the next versions of Confirmit will show even greater maturity in the understanding and execution of Planguage. The plan is to continue to use Planguage (Evo) in the future.

Article Summary

Use of quantified requirements throughout the implementation of a project can provide many benefits as has been demonstrated by the FIRM organization’s use of Planguage (including Evo).

The key messages of this article can be summarized in twelve Planguage principles (see Figure 12). By adopting such principles, agile methods would be much better suited for use in the development of industrial products.

Twelve Planguage Principles

Control projects by a small set of quantified critical results (that is, not stories, functions, features, use cases, objects, etc.). Aim for them to bestated on one page!
Make sure those results are business results, not technical.
Align your project with your financial sponsor’s interests!
Identify a set of designs. Ensure you decompose the designs into increments of the smallest possible deliverables.
Estimate the impacts of your designs, on your quantified goals.
Select designs with the best performance to cost ratios; do them first.
Decompose the workflow and/or deliveries, into weekly (or 2% of budget) time boxes.
Give developers freedom, to find out how to deliver those results.
Change designs, based on quantified experience of implementation (feedback).
Change requirements, based in quantified experience (new inputs).
Involve the stakeholders, every week, in setting quantified goals.
Involve the stakeholders, every week, in actually using increments.

Figure 12. Twelve Gilb Planguage principles for project management/software development.

References

Agile Alliance, 2006, URL: http://www.agilealliance.com/.
Beck, Kent, Extreme Programming Explained: Embrace Change, 2000, Addison-Wesley. ISBN 0201616416.
Gilb, Tom, Competitive Engineering: A Handbook For Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage,2005, Elsevier Butterworth-Heinemann, ISBN 0750665076.
Johansen, Trond, FIRM: From Waterfall to Evolutionary Development (Evo) or How we rapidly created faster, more user-friendly, and more productive software products for a competitive multi-national market, Proceedings of European Software Process Improvement (EuroSPI), Trondheim, Norway, November 10-12, 2004, Torgeir Dingsøyr (Ed.), Lecture Notes in Computer Science 3281, Springer 2004, ISBN 3-540-23725-9. See also Proceedings of INCOSE 2005 (Johansen and Gilb 2005) and FIRM website, http://www.confirmit.com/news/release_20041129_confirmit_9.0_mr.asp/.

Page 3 Back to the archive list