What's Wrong With Agile Methods - Part 2
Some Principles and Values to Encourage Quantification

Tom Gilb, http://www.gilb.com/
Lindsey Brodie, Middlesex University

Evaluating Designs

Impact Estimation (IE) is the Planguage method for evaluating designs. See Table 1, which shows an example of a simple IE table. The key idea of an IE table is to put the potential design ideas against the quantified requirements and estimate the impact of each design on each of the requirements. If the current level of a requirement is known (its baseline, 0%), and the target level is known (its goal or budget depending on whether a performance requirement (an objective) or a resource requirement respectively, 100%), then the percentage impact of the design in moving towards the performance/resource target can be calculated. Because the values are converted into percentages, then simple arithmetic is possible to calculate the cumulative effect of a design idea (sum of performance and sum of cost) and the performance to cost ratio (see Table 1). You can also sum across the designs (assuming the designs are capable of being implemented together and that their impacts don’t cancel each other out) to see how much design you have that is addressing an individual requirement.

Table 1 also shows how you can take into account any uncertainties in your estimates. An additional feature, not shown here, is to assess the credibility of each estimate by assigning a credibility factor between 0.0 and 1.0. Each estimate can then be multiplied by its credibility factor to moderate it.

While such simple arithmetic does not represent the complete picture, it does give a convenient means of quickly identifying the most promising design ideas. Simply filling in an IE table gives a much better understanding of the strengths and weaknesses of the various designs with respect to meeting all the requirements.

Design Ideas-> Requirements: Goals and Budgets	Idea 1 Impact Estimates	Idea 2 Impact Estimates	Sum for Requirement/ (Sum of Percentage Impacts)	Sum of Percentage Uncertainty Values	Safety Deviation
Reliability 300 <-> 3000 hours MTBF	1650hr ±0	840hr ±240	92%	±9%	-108%
	61%±0	31%±9%
Usability 20 <-> 10 minutes	1min. ±4	6 min. ±9	70%	±130%	-130%
	10%±40%	60%±90%
Sum of Performance	71%	91%
Capital 0 <-> 1 million US$	500K ±200K	100K ±200K	60%	±40%	-10%
	50%±20	10%±20
Maintenance 1.1M <-> 100K/year US$	0 K$/Y ±180K	1 M$/Y ±720K	100%	±90%	-50%
	0%± 18%	100%±72%
Sum of Cost	50%	110%
Performance to Cost Ratio	1.42 (71/50)	0.83 (91/110)

Table 1. An example of a simple IE table (Gilb 2005)

Table 1 simply shows estimates for potential design ideas. However, you can also input the actual measurements (feedback) from implementing the design ideas. There are two benefits to this: you learn how good your estimates where for the design ideas implemented, and you learn how much progress you have made towards your target levels. You can then use all the IE table data as a basis to decide what to implement next.

Evolutionary Delivery

The final Planguage method we will discuss is Evolutionary Project Management (Evo). Evo demands include the following:

• that a system is developed in a series of small increments (each increment typically taking between 2% and 5% of the total project timescale to develop);
• that each increment is delivered for real use (maybe as Beta or Field trial) by real ‘users’ (any stakeholder) as early as possible (to obtain business benefits, and feedback, as soon as possible).
• that the feedback from implementing the Evo steps is used to decide on the contents of the next Evo step;
• that the highest value Evo steps are delivered earliest, to maximize the business benefit.

Note that ‘delivery’ of requirements is the key consideration. Each delivery is done within an Evo step. It may, or may not, include the building or creation of the increment (Some Evo steps may simply be further roll-out of existing software);

Development of necessary components will occur incrementally, and will be continuing in parallel while Evo steps are being delivered to stakeholders. Most development will only start when the decision has been taken to deliver it as the next Evo step. However, there probably will be some increments that have longer lead-times for development, and so their development will need to start early in anticipation of their future use. A project manager should always aim to ‘buffer’ his developers in case of any development problems by having in reserve some components (readied in the ‘Backroom’) ready for delivery. The ‘Frontroom’ being the term for the interface between developers and stakeholders – for implementation of the steps.

Planguage approach to Change

It is important to note that the quantified requirements, designs and implementation plans are not ‘frozen,’ they must be subject to negotiated change, over time. As Beck points out, "Everything in software changes. The requirements change. The design changes. The business changes. The technology changes. The team changes. … The problem isn’t change, per se,… the problem, rather, is the inability to cope with change when it comes" (Beck 2000).

Planguage’s means of dealing with change is as follows:

• performance and resource requirements are quantified to allow rapid communication of any changes in levels;
• IE tables allows dynamic reprioritization of design ideas and helps track progress towards targets;
• Evo enables all types of change to be catered for ‘in-flight’, as soon as possible. There is regular monitoring of what the best next Evo step to take.

Description of the Planguage process

To summarize and show how the methods (for quantifying requirements, evaluating designs and evolutionary delivery) described earlier in this article fit together, here is a description of the Planguage process for a project:

1. Gather from all the key stakeholders the top few (5 to 20) most critical goals that the project needs to deliver. Give each goal a reference name (a tag).
2. For each goal, define a scale of measure and a ‘final’ goal level. For example:

Reliable:
Scale: Mean Time Before Failure,
Goal: 1 month.

3. Define approximately 4 budgets for your most limited resources (for example, time, people, money, and equipment).
4. Write up these plans for the goals and budgets (Try to ensure this is kept to only one page).
5. Negotiate with the key stakeholders to formally agree the goals and budgets.
6. Draw up a list of design ideas: Ensure that you decompose the design ideas down into the smallest increments that can be delivered (these are potential Evo steps). Use Impact Estimation (IE) to evaluate your design ideas contributions towards meeting the requirements. Look for small increments with large business value. Note any dependencies, and draw up an initial rough Evo plan, which sequences the Evo steps. In practice, decisions about what to deliver for the next Evo step will be made in the light of feedback (that is when the results from the deliveries of the previous Evo steps are known). Plan to deliver some value (that is, progress towards the required goals) in weekly (or shorter) increments (Evo steps). Aim to deliver highest possible value as soon as possible.
7. Deliver the project in Evo steps.

Report to project sponsors after each Evo step (weekly, or shorter) with your best available estimates or measures, for each performance goal and each resource budget. On a single page, summarize the progress to date towards achieving the goals and the costs incurred.

Discuss with your project sponsors and stakeholders what design ideas you should deliver in the next Evo step. This should be done in the light of what has been achieved to date and what is left to do. Maximizing the business benefit should be the main aim.

8. When all goals are reached: ‘Claim success and move on.’ Free remaining resources for more profitable ventures

Figure 6. Planguage’s ten values for an agile project based around Beck’s Four Values for XP (Beck 2000 Page 29)

Figure 7. Planguage policy for project management

Copyright © 2006 by Tom Gilb

Page 1    Page 3    Back to the archive list