Methods & Tools Software Development Magazine

Software Development Magazine - Project Management, Programming, Software Testing

 

What's Wrong With Agile Methods
Some Principles and Values to Encourage Quantification

Tom Gilb, http://www.gilb.com/
Lindsey Brodie, Middlesex University

Abstract

Current agile methods could benefit from using a more quantified approach across the entire implementation process (that is, throughout development, production and delivery). The main benefits of adopting such an approach include improved communication of the requirements and, better support for feedback and progress tracking.

This article first discusses the benefits of quantification, then outlines a proposed approach (Planguage) and, finally describes an example of its successful use (a case study of the ‘Confirmit’ product within a Norwegian organization, ‘FIRM’).

Introduction

Agile Software Methods (Agile Alliance 2006) have insufficient focus on quantified performance levels (that is, metrics stating the required qualities, resource savings and workload capacities) of the software being developed. Specifically, there is often no quantification of the main reasons why a project was funded (that is, metrics stating the required business benefits, such as business advancement, better quality of service and financial savings). This means projects cannot directly control the delivery of benefits to users and stakeholders. In turn, a consequence of this is that projects cannot really control the corresponding costs of getting the main benefits. In other words, if you don’t estimate quantified requirements, then you won’t be able to get a realistic budget for achieving them. See Figure 1 for a scientist’s (Lord Kelvin’s) opinion on the need for numerical data!

"In physical science the first essential step in the direction of learning any subject is to find principles of numerical reckoning and practicable methods for measuring some quality connected with it. I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be."

Lord Kelvin, 1893

Figure 1. A statement made by Lord Kelvin on the importance of measurement. 
From http://zapatopi.net/kelvin/quotes.html

Further, quantification must be utilized throughout the duration of an agile project, not just to state requirements but, to drive design, assess feedback and, track progress. To spell this last point out, quantification of the requirements (what do we want to control?) is only a first step in getting control. The next steps, based on this quantification, are design estimation (how good do we think our solutions are?) and measurement of the delivered results (how good were the solutions in practice?). The key issue here is the active use of quantified data (requirements, design estimates and feedback) to drive the project design and planning.

One radical conclusion to draw, from this lack of quantification, is that current conventional agile methods are not really suitable for development of industrial products. The rationale for this being that industry is not simply interested in delivered ‘functionality’ alone; they probably already have necessary business functions at some level. Projects must produce competitive products, which means projects must deliver specific performance levels (including qualities and savings). To address this situation, it is essential that the explicit notion of quantification be added to agile concepts.

See Figure 2 for a list of the benefits to agile development of using quantification.

Benefits of the Use of Quantification in Agile Development

  • Simplify requirements (if the top few requirements are quantified, there is less need for copious documentation as the developers are focused on a clearer, simpler ‘message’);
  • Communicate quality goals much better to all parties (that is, users, customers, project management, developers, testers, and lawyers);
  • Contract for results. Pay for results only (not effort expended). Reward teams for results achieved. This is possible as success is now measurable;
  • Motivate technical people to focus on real business results;
  • Evaluate solutions/designs/architectures against the quantified quality requirements;
  • Measure evolutionary project progress towards quality goals and get early & continuous improved estimates for time to completion;
  • Collect numeric historical data about designs, processes, organizational structures for future use. Use the data to obtain an understanding of your process efficiency, to bid for funding for improvements and to benchmark against similar organizations!

Figure 2. What can we do better in agile development (or ‘at all’), if we quantify requirements

Defining Quality

The main focus for discussion in this article will be the quality characteristics, because that is where most people have problems with quantification. A long held opinion of one of the authors of this article (Tom Gilb) is that all qualities are capable of being expressed quantitatively (see Figure 3).

The Principle Of 'Quality Quantification’

All qualities can be expressed quantitatively, 'qualitative' does not mean unmeasurable.

Tom Gilb

Figure 3. Tom Gilb’s opinion that all qualities can be expressed numerically

A Planguage definition of ‘quality’ is given in Figure 4. Planguage is a planning language and a set of methods developed by Tom Gilb over the last three decades (Gilb 2005). This next part of the article will outline the Planguage approach to specifying and using quantitative requirements to drive design and determine project progress.

Definition of Quality

Quality is characterized by these traits:

  • A quality describes ‘how well’ a function is done. Qualities each describe the partial effectiveness of a function (as do all other performance attributes).
  • Relevant qualities are either valued to some degree by some stakeholders of the system - or they are not relevant. Stakeholders generally value more quality, especially if the increase is free, or lower cost, than the stakeholder-perceived value of the increase.
  • Quality attributes can be articulated independently of the particular means (the designs and architectures) used for reaching a specific quality level, even though achievement of all quality levels depend on the particular designs used to achieve quality.
  • A particular quality can potentially be a described in terms of a complex concept, consisting of multiple elementary quality concepts, for example, ‘Love is a many-splendored thing!’
  • Quality is variable (along a definable scale of measure: as are all scalar attributes).
  • Quality levels are capable of being specified quantitatively (as are all scalar attributes).
  • Quality levels can be measured in practice.
  • Quality levels can be traded off to some degree; with other system attributes valued more by stakeholders.
  • Quality can never be perfect (no fault and no cost) in the real world. There are some valued levels of a particular quality that may be outside the state of the art at a defined future time and circumstance. When quality levels increase towards perfection, the resources needed to support those levels tend towards infinity.

(Gilb 2005)

Figure 4. Planguage definition of ‘quality’

Quantifying Requirements

Planguage enables capture of quantitative data (metrics) for performance and resource requirements. A scalar requirement, that is, either a performance or resource requirement, is specified by identifying a relevant scale of measure and stating the current and required levels on that scale. See Figure 5, which is an example of a performance requirement specification. Notice the parameters used to specify the levels on the scale that is, Past, Goal. And Fail.

Tag:
Scale:
Meter:
Past:
Goal:
Fail:

Figure 5. Planguage parameters used to specify a performance requirement

Copyright © 2006 by Tom Gilb

Page 2   Back to the archive list

Software Testing
Magazine


The Scrum Expert