Can a solution be achieved within available business constraints?

imageAs we pursue target business objectives, we often find the landscape to be traversed filled to the brim with uncertainty and constraints. As teams attempt to navigate this hazardous terrain, they must discover a path from often nebulous business needs to worthwhile solutions, without ending in crevasses along the way.

A clear, unwavering vision can serve like a lighthouse that can help draw the team through their inevitable slogging along this journey. Solution envisioning is essential so possible alternative routes can be evaluated. These evaluations determine when and how pivots can and should be performed, and so the resources required can be judiciously metered out.

Yet adopting a compelling, shared vision, while necessary, is insufficient. A Japanese proverb warns us that "Vision without action is a daydream. Action without vision is a nightmare." And Jack Ring says it like it is: "A vision without a plan is an hallucination".

At the beginning of our journey, countless alternatives and options often present themselves, sunk costs are still low, risks are dismissed as mere worries, and everyone thinks they can get everything they want. But as we progress into the middle of this journey, we often find that our options have rapidly diminished, our celebrated prior accomplishments often have failed to earn sufficient value, and our sponsor's endorsements may suddenly shift like the wind.

Successful new product development and product engineering require us to develop and apply capabilities that will satisfy specific criteria for cost, schedule, and quality. Conditions often emerge that force leaders into positions that they previously were unwilling or unprepared to take. Reality thus emerges and often bites stakeholders in the posterior region, revealing the weaknesses that central planning so often obscures:

Today's development organizations are managed like centrally-controlled economies of the twentieth century. Resource is allocated by an infinitely wise centralized decision maker. There is no explicit cost for requesting priority or excess resources. In theory, the decision maker has the scarce global knowledge needed to allocate resources. In reality, this creates a massive incentive for projects to exaggerate their needs. Skillful lobbying can often get more resources than legitimate needs.

Why is the achievement of even limited goals so often elusive to project teams? To answer this question, let's walk through the dynamics of an example project to better understand how such uncertainty unfolds. Let's assume that our business is creating a new solution that will be subsequently manufactured and offered into the marketplace, but must contain its investment costs within affordable limits.  In our example, let's also assume that some critical resource is not yet available at the exact time they are needed- say someone who possesses specialized knowledge that is limited supply. This situation introduces a constraint into the project that blocks progress until access to this resource can be obtained. When such resources are in scarce supply, a queue of activities will inevitably form which are waiting on such resources.

To achieve success in a project as the project unfolds, we must balance lots of these constraints. The choices that we have made in the past constrain the options we have available to us in the present. Similarly, the choices we make in the present will limit what choices we have available to us in the future. Each of these choices must trade off desirable attributes for solutions that initially are abstract qualities, like scalability, reuse, and lots of other platitudes. Until the specific characteristics are adequately defined and elaborated from the viewpoint of the concerns of their stakeholders, the definition of success may not hold still.

If crossing this terrain were easy, or creating these plans were straightforward, businesses would always perform at peak efficiency. In practice, even simple things are complex, constraints abound, and many dimensions must be aligned, including:

  • reconciling the availability and appetite for resources among performing units,
  • applying disciplines at just the right level
  • extracting domain knowledge from subject matter experts whose availability is in high demand and thus nearly always limited and
  • trading off short and longer term interests.

Businesses must properly evaluate this uncertainties, just as designers must optimize within the solution space of what they are designing. These dynamics are likely to drive projects to hedge on their commitments by blurring crisp definitions of success, overstating their asks, and extending their planned completion date of activities to buy themselves time in the hope for sunnier weather in the future. This approach hopes to buffer the impacts of these constraints, yet if the underlying constraints lie on the critical path, and remain, resolution will be less likely later, rather than more so; in the meantime, opportunities and resources are squandered. Yet unless the fidelity of planning has been sufficiently robust, and objective evidence has verified that disciplined execution is routinely practiced, a rebalancing of resources may need to be performed across the entire project portfolio, so that the overall returns from available resources can meet expectations.

The corresponding tradeoffs of these constraints should be considered across all facets of the project management triangle, with particular emphasis on the emerging situation at hand. Regardless of which factor is favored, additional unknowns will inevitably be encountered as progress continues. Many drivers influence the quantity and types of resources that will be required to accomplish development activities, yet few may be knowable in advance, or are under the control of the project team. Yet without these insights, projections may be just wild guesses, with little confidence behind them, or just a synthesis of good intentions; neither will provide much traction on the slippery slope of overly ambitious expectations.

image

As the fog of these unknown unknowns condenses into even more constraints, agility emerges as the most important capability. Attempts to nail down reliable estimates presume that solution elements can be sufficiently anticipated in advance of constraints which will emerge in the future. The final option is to try to provision sufficient resources so they are deployed on the most important endeavors. This rebalancing may only be partially effective, since resources are rarely interchangeable, and since major pivots are usually only entertained after easier options have been exercised.

Yet even in the ideal and rare situation when teams make the right assumptions about initial conditions, the impacts from emerging constraints and risks can still remain unacceptably high. Interventions to address these changing conditions effectively exert a tax on each team's ability to ramp up their throughput and incorporate learning in a more steady state environment.

If we toss our dice under identical conditions for in any small series of tosses, their cumulative sum will vary; in projects, these random effects result in variation of performance, which manifests itself by expansions of project duration and cost. This variation takes the form of a power law distribution, rather than a normal one like our dice tosses. This variation has confirmed by industry estimating models, which have attempted to calibrate known project parameters, and evaluate their effect on possible future outcomes, projecting the likelihood of a particular outcome like we can rolls of the project dice. Here is an example estimate for the project resources required for a complex software-intensive system. As this example indicates, the outcome we project depends upon whether we are projecting the future under optimistic, pessimistic, or most likely scenarios.

Regardless of how we evaluate the future potential of candidate solutions, the effectiveness of our planning and execution will have a greater influence on what can actually be delivered to a customer than any other factor. Donald Reinertsen describes the mistake that organizations often fall prey to when they are faced with these competing perspectives:

Unhappy with late deliveries, a project manager decides he can reduce variability by inserting a safety margin or buffer in his schedule. He reduces uncertainty in the schedule by committing to an 80 percent confidence schedule. But, what is the cost of this buffer? The project manager is actually trading cycle time for variability. We can only know if this is a good trade-off if we quantify both the value of the cycle time and the economic benefit of reduced variability.

imageTeams often propose safety margins to absorb risks that are likely to emerge during execution. Yet rules of thumb like 'make your estimate and double it', are rarely tolerated​ by decision-makers, because of concerns about the downside of such buffers. There is also a bias to be hopeful that things will start getting easier in the future. Further, decision-makers are rightly concerned that additional resources provisioned for such contingency may expand to fill available time (a phenomenon which occurs frequently enough that has a name, the Hawthorne effect). In other words, when we have extra time, we use that time to create a better and better solution, so it is as good as it possibly can be, rather than give it back to the rest of the project to mitigate the problems others may encounter. They also recognize that sometimes, we even put off some of our work until the last minute (another circumstance called the student syndrome by behavioral psychologists).

When businesses add such insurance to projects to insulate them from this uncertainty, this safety buffer does not provide us with license to extend our estimates arbitrarily, or avoid focusing on meeting our goals. Any insurance we end up negotiating will likely be a compromise between opposing views, and each of these views may be legitimate under different future scenarios as the project plays out. But the effects of overestimating and underestimating do not have offsetting impacts on project outcomes. Steve McConnell points out that underestimates have a greater negative impact on project outcomes than the benefits overestimating are assumed to offer; underestimates so often result in nonlinear penalties arising from associated planning errors, accumulation of technical debt, and utilization of high-risk practices. When headcount is locked in, overruns of one project result in delays to others, yet those delays just erode the feasibility of completion, and start the cycle all over again.

Ray Madachy suggests such flawed mental models underlie (and can amplify) many project performance problems on projects:.

One reason management action contributes to a runaway condition is the tendency to respond too late to deviations, which then forces management to take big actions, which themselves have nonlinear consequences.

As a portfolio of products and services grows for a business, the complexity of these interactions increases with time. Once the 'low-hanging' fruit have been harvested, the marginal value that can be realized from additional investments may plateau or begin to decrease. At the same time, there will be a natural trend towards specialization of resources to develop, support, or manage these products. As a result, the effort required to realize these enhanced capabilities usually increases, since with inevitable turnover,  the pool of resources who are really proficient in these specialty areas is diluted, which increases the likelihood that shortages will occur.

As a result, leaders do everything they can to squeeze out every bit of contingency they sense is present in our schedule. Yet research indicates as they do this, just the opposite occurs, because individual developers are more likely to over-commit than they are to pad their estimates. The actions of these developers are rational because they really want to be successful at pleasing their customers. But after traversing this path over and over, we usually find ourselves torn between the adverse complications of adding too much of a buffer, and adding too little. As we gain experience in this dance, we realize that delivering on our commitments is only possible through combinations of discipline, risk management, and good fortune. Successful achievements require us to have a good plan, be realistic about what it will take to implement that plan, and position ourselves to quickly adapt to emerging conditions.

AttachmentSize
PDF icon Software intensive system estimate.pdf160.38 KB

Rating: