Click here to view the complete list of archived articles

This article was originally published in the Fall 2006 issue of Methods & Tools

Introduction to the Emerging Practice off Software Product Line Development

Charles W. Krueger, http://www.biglever.com/

In today's customer-driven environment, most companies target the needs of their prospective customers by creating a product line – a portfolio of closely related products with variations in features and functions – rather than just a single product. For companies that utilize standalone or embedded software in their products, this product diversity poses a serious problem. Tools and techniques for software development tend to focus on individual products. As a result, developing software for a product line is extremely complex due to multiple intertwined products, features and production deadlines – all aimed at moving target markets.

These tactical software development challenges are big enough to impede the realization of business-critical goals and strategies. More specifically, they can greatly hinder a company’s ability to hit market windows, provide competitive pricing, maximize product quality, and expand the scale or scope of their product line portfolio.

A new class of software development methods, tools and techniques – collectively referred to as software product line development – is emerging to address this problem, offering improvements in development time-to-market, cost, quality, and portfolio scale and scope. What is most interesting is the magnitude of tactical and strategic improvements that are possible, not measured in percentage points, but more commonly in factors of two to ten. These improvements are so large that they impact the fundamentals of how companies compete. The following sections provide insight into the emerging practice of software product line development, its underlying key concepts and primary benefits

The Genesis of Software Product Line Development Methods

Manufacturers have long used analogous engineering techniques to create a product line of similar products using a common factory that assembles and configures parts designed to be reused across the varying products in the product line. For example, automotive manufacturers can now create thousands of unique variations of one car model using a single pool of carefully architected parts and one factory specifically designed to configure and assemble those parts.

The idea of manufacturing software from reusable parts has been around for decades, but success has been elusive. Recent advances in the software product line field have demonstrated that narrow and strategic application of these concepts can yield order of magnitude improvements in software engineering capability. The result is often a discontinuous jump in competitive business advantage, similar to that seen when manufacturers adopt mass production and mass customization paradigms.

The characteristic that distinguishes software product lines from previous efforts is predictive versus opportunistic software reuse. Rather than put general software components into a library in hopes that opportunities for reuse will arise, software product lines only call for software artifacts to be created when reuse is predicted in one or more products in a well defined product line.

Software product lines can be described in terms of four simple concepts, as illustrated in the figure below:

• Software asset inputs: a collection of software assets – such as requirements, source code components, test cases, architecture, and documentation – that can be configured and composed in different ways to create all of the products in a product line. Each of the assets has a well defined role within a common architecture for the product line. To accommodate variation among the products, some of the assets may be optional and some of the assets may have internal variation points that can be configured in different ways to provide different behavior.
• Decision model and product decisions: The decision model describes optional and variable features for the products in the product line. Each product in the product line is uniquely defined by its product decisions - choices for each of the optional and variable features in the decision model.
• Production mechanism and process: the means for composing and configuring products from the software asset inputs. Product decisions are used during production to determine which software asset inputs to use and how to configure the variation points within those assets.
• Software product outputs: the collection of all products that can be produced for the product line. The scope of the product line is determined by the set of software product outputs that can be produced from the software assets and decision model.

These concepts illustrate the key objectives of software product lines: to capitalize on commonality and manage variation in order to reduce the time, effort, cost and complexity of creating and maintaining a product line of similar software systems.

• Capitalize on commonality through consolidation and sharing within the software asset inputs, thereby avoiding duplication and divergence.
• Manage variation by clearly defining the variation points and decision model, thereby making the location, rationale, and dependencies for variation explicit.

Software product line development approaches provide a shift in perspective, so that development organizations can engineer their entire portfolio as though it were a single system – the production line – rather than a multitude of products.

Binding Times

The primary distinction between software product line engineering and conventional software engineering is the presence of variation in some or all of the software assets. In the early stages of a software product line lifecycle, software assets contain variation points that represent unbound options about how the software will behave. At some point during the production process, product decisions are utilized to select among the options for each variation point, after which the behavior of the variation point in the final product is fully specified. The time at which the decisions for a variation point are bound is referred to as the binding time.

Examples of different binding times for software product lines include:

• Source reuse time. Decisions bound when reusing a configurable source artifact
• Development time. Decisions bound during architecture, design, and coding
• Static code instantiation time. Decisions bound during assembly of code just prior to a build
• Build time. Decisions bound during compilation or related processing
• Package time. Decisions bound while assembling binary & executable collections
• Customer customizations. Decisions bound during custom coding at customer site
• Install time. Decisions bound during the installation of the software product
• Startup time. Decisions bound during system startup
• Runtime. Decisions bound when the system is executing

It is possible to utilize multiple binding times in a software product line. This allows some decisions to be bound earlier in the lifecycle and other decisions to be deferred until later in the process. For example, some decisions might best be made by a product manager at the company developing the software, while other decisions might best be made by the end customer that will use the software.

With multiple binding times, the software product outputs from binding decisions at one production stage become partially instantiated software asset inputs for binding decisions at the next production stage. The figure below illustrates two binding times, though more are possible.

Production

The production operation creates partially or fully instantiated product outputs from software asset inputs, using product decisions to determine how to bind variations in the assets. Production is a key discriminator between different software product line approaches and those differences are described in this section.

Automation. Production in software product lines can be fully automated, completely manual, or somewhere in between. An example of a fully automated production approach is application generators and product configurators, where product decisions provide sufficient information to automatically generate the product outputs. An example of a completely manual approach is a textual production plan, where software engineers interpret and follow directions in the plan and the product decisions to tailor, integrate, and provide "glue code" around the software assets in order to create products.

Periodicity. As with conventional software engineering, production of a product is typically not a one-shot activity. Products may need to be periodically reproduced to reflect enhancements to the software assets or decisions. The frequency of periodic production may be measured in terms of hours in an agile approach that utilizes automation or in terms of years in a waterfall and manual production approach.

Roles. Some production approaches define a separate human role for the production activity called application engineering, distinct from the domain engineering role responsible for engineering the software asset inputs. Other approaches do not distinguish between these two roles. Separate roles are common in approaches with manual production while a single role is more common in approaches with fully automated production.

Representation. The different formats used to represent product decisions range from informal textual descriptions to formal machine-interpreted languages.

Guidance. Some software product line approaches provide guidance when making decisions for a product, ranging from written heuristics, to constraint checking, to expert system guidance. Guidance helps to reduce the time, complexity, and errors in the decision-making process, particularly for large numbers of interdependent decisions.

Role. The decision-making role for product creation may be an engineering role such as an application engineer or a domain engineer, or it may be a non-engineering role such as a product marketer, a sales person, or the customer. The role should be given to the person who can make the best decisions at the best time. This will in turn influence the binding time (or times) for production.

Replayable. For periodic production of products, previously-made decisions for products can be automatically "replayed" rather than manually recreated. This requires stored decision representations and automated replay mechanisms.

Representation. The software asset inputs for software product lines can come in many different formats, ranging from immutable binary executables to mutable source code that is manually modified, tailored, or extended during production. Assets are not limited to source code and can include test cases, documentation, requirements, use cases, architecture and design descriptions, and so forth.

Variation points. There are many approaches for representing points of variation in the software assets as well as mechanisms to instantiate them. The best choice is influenced by the desired binding time and the degree of production automation desired. For example, a variation point for development-time binding without automation could be an empty block of source code that an application developer fills in. Variation points for runtime binding with full automation might be a programming language if-then-else statement whose behavior is determined by user preferences settings made by the customer.

Representation. The product outputs can take on many different formats, such as binary versus source and mutable versus immutable.

Partial instantiation. If the product outputs are partially instantiated in one stage of variation binding, the unbound variations become the variation points in software asset inputs for the next stage.

As with conventional software engineering, the software artifacts in a software product line (the assets, decisions, and products) are subject to maintenance and evolution. That variation over time, due to evolution, is distinct from the variation found in the variation points in the software assets, due to the variation among products in the product line space. This distinction is characterized as variation in time versus variation in space.

Conventional configuration management can manage some of the issues of variation in time for a software product line. However, the problem is complicated by dependencies among the different software artifacts in the production process. As illustrated below, evolutionary changes to an artifact may require that those changes be propagated to interdependent downstream or upstream artifacts in the production workflow.

Update paths. A change made to an upstream artifact may need to be reflected in all existing downstream artifacts previously produced from the original artifact.

Feedback paths. If downstream artifacts are mutable, then a change made to a downstream artifact may need to be fed back to the original artifact or artifacts from which the original downstream artifact was produced. Automated production and replay can eliminate the need for manual updates and all feedback paths.

The scope of a single software product is defined by the bounds of the capabilities provided in that product. Similarly, the scope of a software product line is determined by the bounds of the capabilities provided by the collection of products in the product line. The scope of the products in any product line can be characterized in terms of six dimensions in product usage scenarios: who, what, when, where, how, and why. Multiplicity along these dimensions leads to multiple products in a product line.

There are two primary approaches to managing scope software product lines, proactive and reactive. In the case of a pure proactive approach, all of the products needed on the foreseeable horizon are supported by the product line. In the case of a pure reactive approach, only the products needed in the immediate term are supported by the product line and new products are incrementally added as the needs change. There is, of course, a continuum between the two.

The reactive approach requires less up-front effort than the proactive approach since the initial scope coverage is smaller. To implement the same scope coverage, the reactive approach incrementally defers the cost and effort over longer period of time compared to the proactive approach. With the proactive approach, the up-front investment for implementing a broader scope is amortized over time when new products within the scope are deployed with little or no effort. It is possible for early stage artifacts in a software product line, such as the architecture, to have a more proactive scope than later stage artifacts, such as the source code. However, the overall scope of the product line is defined by the scope of the product collection that can be produced, tested, and deployed.

There are a variety of approaches for transitioning to, or adopting, a new software product line approach. Some case studies have reported transition efforts of 5 years while others have reported making the transition in as little as 2 months. This section outlines some of the characteristics that influence this surprising diversity in transition profiles.

The software asset inputs for a software product line might come from a variety of sources, including reusable artifacts from an existing library, wrapped or re-engineered artifacts from an existing product, or artifacts developed from scratch. Considerable time and effort can be saved if assets can be reused or extracted from existing products, existing repositories, commercial libraries, and so forth, rather than creating the software assets from scratch.

Another distinguishing characteristic in the transition approach for software product lines is whether it is a brand new product line or whether it is a transition from an existing product line to a more effective approach. Note that if the initial state has multiple products, managed with even the most primitive, ad hoc, conventional techniques such as clone-and-own or language preprocessor conditionals such as the C language #ifdefs, it is still a product line. "Clone-and-own" refers to the practice of creating a new product by "cloning" a copy of the source code of another product and then modifying it, thereby taking ownership of the newly cloned product which now has a development and maintenance lifecycle of its own. Existing products, assets (including requirements, architecture, source, and so forth), decision models, and production mechanisms can often be reused and re-engineered when enhancing a software product line approach in order to save time and effort.

Early case studies of software product line transitions reported typical adoption times of 2 to 5 years. For most organizations, this time and effort represents a significant barrier to adopting a product line approach, regardless of the potential benefits. Recently, advances have been made in lightweight approaches that lower the required transition effort, with some case studies reporting adoption efforts as low as 2 months. Lightweight techniques employed include:

• minimize differences between single-system and product line engineering in order to minimize impact on organization, processes, software, and infrastructure
• utilize an incremental adoption strategy to initially transition a small subset of assets, products, subsystems or people
• offer off-the-shelf software product line tools and technology
• use reactive approaches to defer product development and deployment efforts
• structure production to minimize the need for complex and costly merging, feedback, and product-specific configuration management overhead

The benefits of the software product line approach come in the form of tactical improvements in software engineering -- deploying software products faster, cheaper, and better. However, what is most interesting is that these tactical improvements are often large enough to have an impact well beyond the borders of the engineering department, offering strategic competitive benefits to the way that a company conducts its business.

The following tactical engineering benefits can be gained from software product lines. Some organizations have reported improvements ranging from factors of 3 to factors of 50.

• reduction in the average time to create and deploy a new product
• reduction in the average number of defects per product
• reduction in the average engineering effort to deploy and maintain a product, and therefore reduction in the average engineering cost per product
• increase in the total number of products that can be effectively deployed and managed

These tactical engineering benefits translate into a very powerful set of strategic business benefits:

• reduced time-to-market and time-to-revenue for new products
• improved competitive product value
• higher profit margins
• improved ability to hit market windows
• better product quality and improved company reputation for quality
• improved scalability of business model in terms of products and markets
• increased agility to expand into new markets
• reduced risk in product deployments

The order-of-magnitude benefits offered by software product lines can be attributed to strategic software reuse. Software product line techniques explicitly consolidate and capitalize on commonality throughout the product line. They formally manage and control the variations among the products in the product line. They aggressively eliminate all duplication of effort in the engineering processes. As a result, the only unique engineering effort required for any product in the product line is for the product variations that are truly unique to the product.

The remaining sections in this chapter provide additional details and descriptions on the tactical and strategic benefits of software product lines.

If new products in a software product line can be engineered and brought to market faster, the following strategic business benefits result:

• shorter time to revenue
• improved ability to hit critical market windows
• increased bandwidth to pursue more markets and thus generate more revenue
• greater agility to survive in turbulent market conditions

Companies with software product line success stories have reported decreasing their time-to-market for new products by factors of 2 to 50 when compared to the conventional techniques they were using before.

Some of the tactical pains experienced by engineering organizations that might be able to benefit from the time-to-market improvements of software product lines include:

• contentious relationship between marketing and engineering teams due to engineering's inability to accommodate business requests for more product variations
• complex and time consuming coordination of multiple parallel development efforts that share common software (for example, diverging and merging code bases or bushy configuration management branch structures)
• brittle source code that is difficult and error-prone to extend with new product variations

Software product line approaches improve time-to-market by enabling delta engineering. Delta engineering means that the only new software development required for a new product instance is for new variation points in the software assets to accommodate capabilities that are truly unique to the new product (that is, capabilities that don't already exist for other products). Beyond that, the new product instance can be created from the stable collection of existing common assets, variation points, the decision model, and the production mechanism.

Quality benefits of software product lines can be measured in two ways. The first is how well each product matches the needs of each customer. The mass customization capabilities of software product lines directly address this measure of quality. The second is the rate of defects found in each of the products in the product line, which can also be significantly improved by software product line techniques. Companies have reported reductions in defect rates as high as 96%.

Higher product quality with software product lines has both strategic business and tactical engineering impact:

• satisfied customers provide a continuing revenue stream and provide positive recommendations to other potential customers
• fewer product defects means lower overhead throughout the software lifecycle
• fewer support calls and thus a smaller support organization
• fewer failure/find/fix/retest cycles in the test organization leads to shorter test cycles and smaller test teams
• fewer debug/fix activities means that developers can focus more time on developing new products and new product capabilities

The quality benefits - in terms of defect reduction - can be directly attributed to the commonality in a software product line. By optimizing the reuse of software assets across the product line and throughout the life of a product line, very high quality assets emerge. Finding and fixing a single defect in a common software asset will impact all of the products using that asset, even if the defect was only observed in one of the products.

The following graph illustrates the downward trend of defects found when testing a collection of products in a software product line, both in a single release of products and across multiple releases of the products. Each colored curve represents a sequence of product tests for a simultaneous release of products in a product line. Each downward-sloping curve shows that defects found in early product tests (e.g., Product A) reduce the number of defects found in subsequent product tests (e.g., Product B). The downward trend of overall defects from Release 1 to Release 2 to Release 3 illustrates that high quality software assets are emerging due to high levels of commonality and carefully managed variations from release to release.

Productivity and Cost Benefits of Software Product Lines

Software product line techniques can significantly increase the productivity of software engineers, seen as a reduction in the effort and cost required to develop, deploy, and maintain a collection of similar software products. Typical productivity improvements reported in case studies range between a factor of 2 to 3, though higher factors are not uncommon. Because the labor costs often dominate the cost of building software systems, productivity and cost benefits go hand-in-hand with software product lines.

The following graph illustrates the effort – and thus cost – required to develop, deploy, and maintain a collection of similar software products. The greater the total number of products, the greater the total effort and cost. The red line represents a conventional productivity line. The blue line represents a productivity line of the pioneering efforts in software product lines, where a significant up-front effort was typically required (seen as the high Y-axis intercept) to launch a software product line and to achieve the 3-to-1 productivity improvements (seen as the more gradual slope). The green line represents the new generation of software product line methods that can achieve 3-to-1 or better productivity gains with much less up-front effort.

The productivity and cost benefits of software product lines can be directly attributed to the very effective reuse of software – capitalizing on the commonality and efficiently managing the variation among products in the product line.

While productivity improves the bottom line in a company's finances, there are also strategic top line benefits. For example, productivity benefits can simultaneously lower the prices and increase the profit margins for products, which provides significant strategic competitive advantage. Furthermore, the large increases in productivity can counteract the pressures to export software development jobs to offshore development locations.

Scalability Benefits of Software Product Lines

A company that takes a software product line approach needs to scale, without constraints, to whatever number of products are optimal for the business. Looking more carefully at the productivity graph from the previous section, both conventional and software product line approaches do have limits to the number of products that can be effectively developed and maintained. As shown in the graph below, a steep rise in effort is encountered when an approach reaches its complexity limit. The benefit of software product line approaches is that they can often scale to orders-of-magnitude more products than conventional software engineering techniques.

There are strategic competitive advantages to being able to scale to larger numbers of products. For example,

• going into larger numbers of markets with tailored products
• creating larger numbers of more narrowly focused products within a market
• providing greater variety and more targeted products than competitors

The complexity limit typically manifests itself as an overwhelming increase in errors and defects. The results is that engineers spend all of their time finding and fixing defects in existing products rather than developing the next product. The errors and complexity can be attributed to limitations in (1) how effectively commonality leveraged across products and (2) how effectively variation is managed among products. The more effectively these two things are done, the greater the scalability of the approach.

Conclusions and How to Learn More

The emerging practice of software product line development offers significant tactical software engineering improvements as well as strategic business advantages. In much the same way that manufacturers advanced from manual labor to mass production to mass customization, software product line development allow software development organizations to advance from labor intensive to highly efficient and automated portfolio development methods.

To learn more about these methods and successes stories, see the Software Product Line Development community web site, www.SoftwareProductLines.com.

Back to the archive list