Methods & Tools Software Development Magazine

Software Development Magazine - Project Management, Programming, Software Testing

Agile Crash Course: Agile Project Management & Delivery - Master the most important concepts & tools of Agile

Your Company Name Here - Reach 30'000 visitors/month and 35'000 software development professionals for $145.

Click here to view the complete list of archived articles

This article was originally published in the Summer 2006 issue of Methods & Tools

Project Failure Prevention: 10 Principles for Project Control- Part 3

Tom Gilb,

Copyright © 2005 by Tom Gilb. Published and used by Methods & Tools with permission.

P5: DESIGN MUST MEET BUSINESS NEEDS:Design Review must be based on a ‘clean’ specification, and should be focused on whether the designs meet the business needs.

Before we can review a design specification regarding relevance, we must review it for ‘compliance to rules of specification’,that assures us about the intelligibility of the specification.

Figure 5. Four stages of checking a design specification

For example, there is no point in reviewing a design specification (such as ’Reusable Modules’), when

  • The specification is unnecessarily ambiguous;
  • There is no assertion of which requirements the design is supporting;
  • There is no assertion or estimation as to how well the design is supporting those requirements;
  • There are no resource estimates (costs) for the design.

To put it more directly: Check that a design specification is well written first, only then do you have a basis for checking to see ifit is a good design. I believe it is unreasonable to judge a design ideaitself on the basis of a poorly written specification. Only well-specifieddesigns should be evaluated by senior responsible people. Badly specifieddesigns – defined as those not following our own design specification rules– need to be returned to the designer for rework. See Figure 6 for an exampleof real design rules designed to force us to specify designs in sufficientdetail. When these rules are followed, then, I believe we have the basis for deciding whether a design is appropriate in relation to its requirements.

Rules: Design Specification

Tag: Rules.DS. Version: October 7, 2004. Owner: TG. Status: Draft.

Note: Design specifications are either for optional design ideas (possible solutions) or required design constraints (that is, actualrequirements AND consequently, pre-selected solutions).

Base: The rules for generic specification, Rules.GS apply. If the design idea is a design constraint (a requirement), the rules forrequirement specification, Rules.RS also apply.

R1: Design Separation: Only design ideas that are intentionally ‘constraints’ (Type: Design Constraint) are specified in the requirements. Any other design ideas are specified separately (Type: Design Idea). Note all the design ideas specified as requirements should be explicitly identified as ‘Design Constraints.’

R2: Detail: A design specification should be specified in enough detail so that we know precisely what is expected, and do not, and cannot, inadvertently assume or include design elements, which are not actually intended. It should be 'foolproof'. For complex designs, the detailed definition of its sub-designs can satisfy this need for clarity, the high level design description does not need to hold all the detail.

R3: Explode: Any design idea, whose impact on attributes can be better controlled by detailing it, should be broken down into a list of the tag names of its elementary and/or complex sub-design ideas. Use the parameter ‘Definition’ for Sub-Designs. If you know it can be decomposed; but don't want to decompose it just now, at least explicitly indicate the potential of such a breakdown. Use a Comment or Note parameter.

R4: Dependencies: Any known dependencies for successful implementation of a design idea need to be specified explicitly. Nothing should be assumed to be ‘obvious’. Use the parameter, Dependency (or Depends On), or other suitable notation such as [qualifiers].

R5: Impacts: For each design idea, specify at least one main performance attribute impacted by it. Use an impact arrow ‘->’ or the Impacts parameter. Comment: At early stages of design specification, you are just establishing that the design idea has some relevance to meeting your requirements. Later, an IE table can be used to establish the performance to cost ratio and/or the value to cost ratio of each design idea.

Design Idea 1 -> Availability.
Design Tag 2: Design Idea.
Impacts: Performance X.

R6: Side Effects: Document in the design specification any side effects of the design idea (on defined requirements or other specified potential design ideas) that you expect or fear. Do this using explicit parameters, such as Risks, Impacts [Side Effect] and Assumptions.

Design Idea 5: Have a <circus> -> Cost A.
Risk [Design Idea 5]: This might cost us more than justified.
Design Idea 6: Hold the conference in Acapulco.
Risk: Students might not be able to afford attendance at such a place?
Design Idea 7: Use Widget Model 2.3.
Assumption: Cost of purchasing quantities of 100 or more is 40% less due to discount.
Impacts [Side Effects]: {Reliability, Usability}.

Do not assume others will know, suspect or bother to deal with risks, side effects and assumptions. Do it yourself. Understanding potential side effects is a sign of your system engineering competence and maturity. Don’t be shy!

R7: Background Information: Capture the Background information for any estimated or actual impact of a design idea on a performance/cost attribute. The evidence supporting the impact, the level of uncertainty (the error margins), the level of credibility of any information and, the source(s) for all this information should be given as far as possible. For example, state a previous project’s experience of using the design idea. Use Evidence, Uncertainty, Credibility, and Source parameters. Note the source icon (<-) usually represents the Source parameter. Comment: This helps ‘ground’ opinions on how the design ideas contribute to meeting the requirements. It is also preparation for filling out an IE table.

Design Tag 2 -> Performance X <- Source Y.

R8: IE Table: The set of design ideas specified to meet a set of requirements should be validated at an early stage by using an Impact Estimation (IE) table.

Does the selected set of design ideas produce a good enough set of expected attributes, with respect to all requirements and any other proposed design ideas? Use an IE table as a working tool when specifying design ideas and also, when performing quality control or design reviews on design idea specifications.

R9: Constraints: No single design specification, or set of design specifications cumulatively, can violate any specified constraint. If there is any risk that this might occur the system engineer will give a suitable warning signal. Use the Risk or Issues parameters, for example.

R10: Rejected Designs: A design idea may be declared ‘rejected’ for any number of reasons. It should be retained in the design documentation or database, with information showing that it was rejected, and also, why it was rejected and by whom.

Design Idea D: Design Idea.
Status: Rejected.
Rationale [Status]: Exceeds Operational Costs.
Authority: Mary Fine. Date [Status]: April 20, This Year.


Figure 6. Specification rules for design, which lay a proper basis for judging the power and cost of the design in relation to the requirements. See also Chapter 7 in (Gilb 2005)

P6: VALIDATE STRATEGIES EARLY: The high-risk strategies need to be validated early, or swapped with better ones.

It should be possible to identify your high-risk strategies (Note, ‘strategies’ are also known as designs, solutions and architectures). They are the ones that you are not sure of the impact levels on performance and cost. They are the ones that can potentially ruin your project, if they turn out to be worse than you are expecting. If you try to estimate all impacts of a given strategy on an Impact Estimation table, and get estimates such as 50%±40% (100% = Goal level attained), then you have exposed a high-risk design. If the credibility estimate (a defined Impact Estimation method) on a scale of 0.0 to 1.0 is low, like under 0.5, then you also have a high-risk strategy. If no one will guarantee the result in a binding contract, then you also have a high-risk strategy.

Engineers have always had a number of techniques for validating risky strategies early. Trials, pilots, experiments, and prototypes are common tactics. One approach I particularly favor is scheduling the delivery of the risky strategy in an early evolutionary delivery step, to measure what happens – and thus get rid of some of the risk. Early evolutionary delivery steps usually integrate a new strategy with a real system, and with real users, and are therefore more trustworthy, than, for example, an expert review panel, which is relatively theoretical.

Policies for Risk Management

Explicit Risk Specification: All managers/planners/engineers/testers/quality assurance people shall specify any uncertainty, and any special conditions, which can imaginably lead to a risk of deviation from defined target levels of system performance. This must be done at the earliest opportunity in writing, and integrated into the main plan.

Numeric Expectation Specification: The expected levels of all quality and cost attributes of the system shall be specified in a numeric way, using defined scales of measure, and at least an outline of one or more appropriate ‘meters’ (that is, a proposed test or measuring instrument for determining where we are on a scale).

Conditions Specified: The requirements levels shall be qualified with regard to when, where and under which conditions the targets apply, so there is no risk of us inadvertently applying them inappropriately.

Complete Requirement Specification: A complete set of all critical performance and cost aspects shall be specified, avoiding the risk of failing to consider a single critical attribute.

Complete Design Specification and Impact Estimation: A complete set of designs or strategies for meeting the complete set of performance and cost targets will be specified. They will be validated against all specified performance and cost targets (using Impact Estimation Tables). They will meet a reasonable level of safety margin. They will then be evolutionarily validated in practice before major investment is made. The Evo steps will be made at a rate of maximum 2% of total project budget, and 2% of total project timescales, per increment (Evo step) of designs or strategies.

Specification Quality Control, Numerically Exited: All requirements, design, impact estimation and evolutionary project plans, as well as all other related critical documents, such as contracts, management plans, contract modifications, marketing plans, shall be ‘quality controlled’ using the Inspection method (Gilb 1993). A normal process exit level shall be that ‘no more than 0.2 maximum probable major defects per page can be calculated to remain, as a function of those found and fixed before release, when checking is done properly’ (that is, at optimum checking rates of 1 logical page or less per hour).

Evolutionary Proof of Concept Priorities: The Evolutionary Project Management method will be used to sense and control risk in mid-project. Dominant features will include:

  • • 2% (of budget and time-to-deadline) steps;

  • • High value-to-cost steps, with regard to risk, prioritized asap;

  • • High risk strategies tested ‘offline to customer delivery’, in the ‘backroom’ of the development process, or at cost-to-vendor, or with ‘research funds’ as opposed to using the official project budget.

Figure 7. Policy Ideas for Risk Management

P7: RESOURCES FOR DESIGNS: Adequate resources need to be allocated to deliver the design strategies.

It is not sufficient that your design strategies will meet your performance targets. We have to have the resources needed toimplement them. And the resources used must not result in an unprofitablesituation, even if we can get them. The resources we must consider are both fordevelopment and operation, even decommissioning. The resources are of many types, and include money, human effort and calendar time.

I observe that it is unusual for me to see design specifications with detailed cost calculations at the level of individualdesigns. We do not even seriously try to consider individual design ideacosts (at best we estimate a total cost); so it is not surprising that we constantly exceed the resource constraints that apply to our projects.

Page 2   Page 4    Back to the archive list

Methods & Tools
is supported by

Simpliv IT Courses

Software Testing

The Scrum Expert