Scoping and Pricing Agile Software Development
Special challenges arise in defining an end state and putting a price tag on the work.
By Donald Shannon
This second article in a two-part series about software and contracts explores how to determine when agile software development is complete and how to price it. The first article in the series appeared in the March 2023 issue of Contract Management.
Traditional Estimating
Traditional cost estimation is a specialized discipline. The formula is quantity x cost = total cost. The art and science come in quantifying the work to be done and obtaining authoritative cost-per-unit pricing.
Historically, software project size and costs have been estimated using one of at least three successive quantification techniques and one of several development strategies. We have progressed from counting software lines of code (SLOC), to function points based on use cases, to agile estimating techniques.
We generally used waterfall development and hard coding in first- through third-generation languages. Now, we have progressed to non-linear methods (agile) using fourth- (e.g., C++, Perl, Ruby, SQL) and fifth-generation languages (e.g., Prolog and OPS5). The non-linear development process requires its own measures of productivity to derive estimations of work, schedule, and cost.
Estimating Agile Development Costs
The first step in estimating agile development is, as in any other project, to quantify the items and work required to fulfill the buyer’s requirements. Traditional buyer-seller tensions are raised to a higher level when it comes to software because of its intangible nature. As a result, critical steps, such as defining “done” are difficult to impossible.
Buyers fear they will open their checkbooks to sellers only to watch helplessly as charge after charge hits a dwindling balance. Sellers fear a never-ending battle with scope creep as their costs escalate and any chance of profit goes out the door.
Why Agile Is Different
Agile software development is a management philosophy and set of techniques born in the late 1990s and popularized in 2001 with the publication of The Agile Manifesto.1 Agile is not a single technique, but rather a synthesis of multiple techniques that share common characteristics. Agile differs from traditional project management and software development approaches and requires a revised project cost estimation approach.
Used properly, agile gets around the basic issues of trust, because it2 is organized around the creation of value. The agile process contains two special roles: the product owner (from the buyer team) and the scrum master (from the developer/seller team). Owner and scrum master work as a team to create a product that fulfills the buyer’s needs while addressing the seller’s concerns. Translating this collaborative approach to fit within the guidelines of the Federal Acquisition Regulation (FAR) is challenging but achievable.
The first step in estimating is to identify the items or work that comprise the product or service being estimated. The result must include everything in the owner’s vision of the finished product (or services or both). That vison often is imprecise or poorly communicated. With agile, the most common way to capture these requirements is to create a vision of what “done” looks like and then document it in a work breakdown structure (WBS).
Frequently, the WBS is initially documented at only a very top level until a contract is awarded, whereupon a more detailed plan and release roadmap is developed. This lack of up-front planning is a key reason why producing a traditional cost estimate for agile projects is difficult and why fixed-price contracts (even level of effort) are difficult to negotiate during the pre-award process. This leads to one of the following outcomes:
1. The seller guesses at the requirements and bids a firm fixed price only to discover many of the requirements were not identified or were misidentified. The result: frequent change orders and loss of cost control.
2. The award of a cost-type contract for a moving target causes frequent change orders and ballooning costs and run-away scope.
3. Adopting a two-step approach as discussed in part one of this article: Develop the scope of the project as one activity, then quote the development effort (all or just the initial increment) based on the scope analysis. This two-step approach makes firm fixed price a more viable option.
Once the WBS and decomposition of effort starts, it leads to a list of things to do. The list comprises epics and features (see definitions in the sidebar, "Agile Terminology," on page 55) which are further decomposed into user stories. The list of things to do is thus created, prioritized, and is identified as the project backlog.
Buyers’ appetites often are larger than their budgets or there is more to do than the allotted time and resources will support. Sacrifices will be needed. This appetite reduction leads to a minimum viable product (MVP) – a less feature-packed version of the software that provides core capabilities but without many of the desired add-ons that users ultimately will want.
Often, the initial focus in the development effort is on reaching agreement on the MVP, addressing the buyer’s immediate concerns, and easing their pain. The MVP agreement is followed by one or more iterative development cycles to refine and expand the MVP to include features or improvements based on user experience and feedback.
This iterative (and cyclic) improvement technique is often described as the development security operations (DevSecOps) infinity loop. It is rapidly becoming a de facto process in government, especially defense-related, software acquisition. In DevSecOps, software is created, incrementally released, used, evaluated, and feedback is provided that leads to another round of development and a subsequent release, as shown in Figure 2. The cycle gives rise to the sense that software is never done.
The DevSecOps incremental approach is consistent with modular development of software as described in FAR 39.103. Not all the work is done in a single production effort. It is performed in manageable increments paced by user needs and feedback and available resources.
The development blueprint for the overall effort is documented in a product roadmap that identifies each planned software release (or major versions such as 2.0, 3.0, etc.) and notionally describes when the remainder of the backlog will be developed.
Developing a Detailed Estimate
The detailed agile estimating process begins with the aforementioned description of the need, frequently limited to one increment (e.g., the MVP) of a multi-phased effort. That overall effort (e.g., an epic or number of epics) is then decomposed into additional detail and elaborated upon through guided discussions with various members of the buyer’s team – principally the users, the administrators, and other support personnel. This leads to the creation of functions and/or user stories, as shown in Figure 3.
Quantifying Work Using Agile Techniques
Because development teams vary in size and composition, the amount of work (capacity) a team can produce is variable. However, a team has the capacity to perform some defined amount of work consistently during each sprint, or unit of time.
Quantifying the amount of work the team can accomplish in one sprint is an essential and foundational element of agile cost estimation and scheduling. The term used for this work volume is “capacity” or sometimes “velocity” and is measured with an abstract unit called “story points” per sprint. The unit has no standard value, is highly subjective, and is meaningful only to the team that uses it.
The conversion process from a pool of requirements into an organized form that can be assessed and estimated is accomplished during the scoping phase if done separately. If that task is not segregated, it is done at the beginning of the project.
During this process the various requirements are decomposed into finer detail starting with concepts like release and epic to describe high level or conceptual entities. The process progresses and evolves into user stories that describe the lowest level requirements.
The underlying principle of story points is to select the features and user stories to be implemented, then, subjectively assign each a value based on expert opinion or consensus using a size scale. In principle, the least difficult feature or story would be rated as a one, and each higher level of difficulty would be the multiple of such work units required to accomplish the work.
The evaluation can be done using any number of abstractions – a common one being T-shirt sizes where an Extra Large = 20 points, a Large = 12 points, a Medium = 5 points, and a Small = 1 point. This concept is shown in Figure 4.
A significant aspect and perhaps criticism of this technique is that story points only have a meaning relevant to the team doing the work. This is not a problem provided the scale is understood by the team and is consistently applied.
Production Capability
Because agile is a team-production technique, productivity is directly related to the number and size of teams employed. Team number and size, in turn, control the number of story points per unit of time each team is capable of producing.
Scrum teams usually are small. They typically have less than nine people and more often are limited to seven,3 including a scrum master and a product owner. For large projects, it is conceivable (and indeed practical) to logically divide the product backlog and assign portions to multiple teams.
The team’s baseline productivity is created from prior work and used to estimate the number of story points the team can produce in a typical sprint. The important thing is to anchor the evaluation scale to past projects based on common experience. We call the average number of story points or tasks that an agile team can deliver each time “velocity.”
Since scrum teams are self-contained, it is possible to add more teams to the project to reduce the overall development time. This is provided the backlog lends itself to further division. Consequently, if the total backlog is 300 story points and each team can accomplish 12 points per sprint, then the likely duration of the development phase of the project is 25 sprints. If two teams are used, the development time potentially could be halved assuming all teams are of equal velocity and the project scales well.
Total capacity (for the contractor) refers to the total number of story points the organization (comprising multiple teams) has available to support projects. Also, it is a benchmark for determining the overall product development time and cost.
Capacity planning helps management understand the amount of productive engineering time available in a sprint,4 especially when multiple scrum teams are employed. Team velocity and organizational capacity are the factors needed to translate a backlog of work into a meaningful estimate of production time (schedule) and total cost.
Since team size and composition are defined and the length of the sprint is time-boxed, it is relatively simple to convert the cost of the entire team (based on capacity) into a cost per sprint and per story point.
Determining a Price
Once a buyer’s need is defined and scoped, the seller can translate the buyer’s overall shopping list (product backlog in agile speak) into units of work (sprints) that can then be priced.
Regardless of the technique used, the estimating process quantifies the work to be done. As shown in Figure 4, in this case it quantifies the work as the sum of all the story points (i.e., the product backlog) that is then divided by the offeror’s capacity (rate) to convert these story points into sprints and ultimately dollars.
By using the estimated number of story points that the offeror’s team can effectively process during a time-boxed work session (i.e., sprint) staffed by a known team size and composition, the costs are easily quantifiable.
Sometimes this approach can take on the form of a hybrid project with elements of both the agile method and the waterfall method combined, as shown in Figure 5. This is especially adaptable for commercial software implementations of enterprise resource management or large database applications where the two-step approach advocated earlier is used.
Many commercial implementations use this technique since the waterfall elements are typically the same from project to project and more easily defined. The agile elements are then estimated by the number of sprints needed to adapt the product to a specific customer’s use case.
In either case – waterfall or hybrid – going from sprints to dollars is relatively easy as shown in Figure 6. A sprint is easy to calculate, auditable, and transparent on a team-by-team basis. If the project comprises multiple teams and multiple sprints, the basic building blocks of the estimate are known and simply need to be assembled.
This technique is sometimes referred to as “renting the software factory.” That’s because the cost is essentially the number of sprints multiplied by the cost per sprint for the agile development cost (as above). Plus project management if separately priced. Plus any non-agile work if a hybrid approach is used.
Conquering the Concepts
Buying software, especially software produced using agile methodology, need not be any more difficult than other acquisitions. However, like other niche acquisitions – construction, architecture and engineering services, research and development – software procurement benefits from a particular strategy.
The biggest challenge in agile software acquisition may not be the actual procurement process but rather understanding the concepts and the lexicon of the agile process itself. CM
Don Shannon is the owner of The Contract Coach LLC. He is a government contracts, project management and risk management educator and consultant with over 30 years of experience in the aerospace and manufacturing sectors.
ENDNOTES
https://agilemanifesto.org
https://www.scaledagileframework.com/agile-teams/
https://www.scrum-institute.org/Scrum_Roles_The_Scrum_Team.php
https://www.smartsheet.com/content/agile-sprint-scrum-capacity-planning
Agile Terminology
As with most methodologies, agile has its own lexicon. It is this lexicon that can create a barrier to those wishing to become familiar with the technique. Many of the terms have equivalent concepts in traditional project management.
However, unlike the differences between the imperial measurements (inches, ounces, etc.) and the metric equivalents (millimeters, centimeters, etc.), agile terms might not convert exactly to the project management lexicon nor vice versa. There is a definite hierarchy among the terms in both systems, and smaller units roll up into larger ones. Here are some of the most common agile terms.
Epic
An epic is a major unit of work decomposed into subordinate units of work known as features. These are further decomposed into user stories. On large projects, the epic might be worked on by two or more development teams. Each focuses on specific aspects or features. An epic is a major building block of the overall project or product. It may correlate to a major element in the project’s work breakdown structure (WBS).
Feature
Features are a subcategory of epics. They are generally a collection of the next lower-level items (user stories) that relate to a common design or functional objective. For example, an epic related to user identification and authentication for a portal application might have multifactor authentication as a feature along with new-user enrollment, forgotten credential recovery, etc. Often these features correlate to one or more of the project requirements. Or the features may be a logical grouping of these requirements. Usually, features are listed in the WBS as subelements of their respective epic and correlate to “work packages” in the traditional project management approach. Features may be too complex to develop in a single work session (sprint). However, they can usually be completed in the time increment associated with an individual software release.
User Story
User stories are how the user interacts with the software product to access the various features. A user attempting to access a portal is presented with various screens and dialogues to navigate in order to gain entry. The user story documents this interaction. The user story also documents the various attributes of the interaction. These include the data that is exchanged during the interaction, the criteria that must be satisfied for the interaction to be successful, and any related user stories. In many respects, user stories correspond to tasks – the lowest level of detail in traditional project management. Like tasks, user stories are not specifically called out in the WBS. User stories generally are small enough in scope that they can be developed in a short period of time, usually a single work session or sprint.
Backlog
Backlog refers to the totality of the work in the overall project (i.e., the WBS) less work that has been accomplished. It is a continually updated to-do list or queue of remaining work. Items in the backlog traditionally are ranked by importance.
Sprint
A sprint is a time-blocked work session during which the team withdraws waiting items (user stories or features) from the backlog to work on to develop finished and tested software. Sprints typically (but not always) are uniform blocks of time — two to three weeks is common. The result is the development and testing of an incremental addition to the finished product, and the release of a new software version/revision.
Release
A release is an incremental improvement of the completed product. It is comprised of usable and functional software that has been tested and validated to meet specific quality and functional requirements. It is suitable for release to the user community.