Evaluating alternatives

image

The affairs of life embrace a multitude of interests, and he who reasons in any one of them, without consulting the rest, is a visionary unsuited to control the business of the world. - James Fenimore Cooper

There is a classic depiction of the interrelationships between cost, quality, and delivery ("pick two") which suggests that each can be easily traded for the other. Yet this mindset often only examines such trades at a particular point in time, ignoring the longer-term shifting of interests as a product matures. The mental models of decision-makers are often clouded by their underlying assumptions and biases, such as believing that synergy will produce an on-demand value stream, that estimates are reliable, that intuitive decision-making can substitute for disciplined elaboration of details, and that innovations can be discovered on demand. Unfortunately, the impacts of these delusions often manifest as significant downstream cost increases and delays - usually after opportunities to contain these distractions have fallen short of what is necessary for presenting circumstances. Economist Thomas Sowell describes the precondition of necessary decisions:

The most basic of all decisions is who shall decide. This is easily lost sight of in discussions that proceed directly to the merits of particular issues, as if they could be judged from a unitary, or God's eye, viewpoint. A more human perspective must recognize the respective advantages and disadvantages of different decision-making processes, including their widely varying costs of knowledge, which is a central consideration often overlooked in analyses which proceed as if knowledge were either complete, costless, or of a "given" quantity. Decision-making processes differ not only in the quantity, quality, and cost of knowledge brought to bear initially, but also and perhaps still more so, in the feedback of knowledge and its effectiveness in modifying  the initial decision.

The book Building Systems from Commercial Components suggests beginning with a clearly defined goal, and highlights examples of such objectives

Objective

Description

When to Use

Conceptualize

Develop criteria for an ideal alternative

When the decision maker has trouble defining the decision that needs to be made.

Classify

Define categories of alternatives, for example "desirable," "acceptable, and "undesirable", so that each alternative can be placed exactly one category.

When the utility value of alternatives cluster in distinct regions, or where the decision maker likes some ambiguity  

Choose

Define two categories: "best" and “rest”, placing at most one alternative

in the "best” category

When the decision maker likes simplicity and prefers to focus on the leading candidate only.

Sort

Place each alternative into a single category, ranked from best to worst.  

When the decision maker likes detail, or likes to discover patterns or categories

Describe

When options are tagged with their most pronounced features

When the decision maker distrusts numerical decision methods, or is better with words than numbers  

Threshold

Select some aggregate utility threshold for an acceptable alternative and evaluate alternatives one at a time until an acceptable one is found, discarding other alternatives.

When the decision maker cannot fit some reason support all options but will accept a “first fit” (as opposed to “best fit”) alternative.

Opportunities should be channeled through project decision gates using a standardized workflow pattern which will guide candidates through a more rigorous commitment and portfolio management effort. A reasonable set of criteria for describing alternatives for subsequent sorting are used by Google in their 10 to the 100 Project. In that project, ideas compete for allocations by using the following factors:

  • Scope: How broadly will the proposed solution benefit the business?
  • Impact: How deeply will the business be affected by the proposed solution?
  • Urgency: How pressing and perishable will the value be that is made available if this opportunity is pursued?
  • Attainability: How achievable will the solution be within the constraints of time and resources anticipated to be available?
  • Sustainability: What investments will it take to ensure that this solution's benefits will persist over time?

The robustness of the analysis of each opportunity become increasingly important as the claimed sizes of these prizes grows, their expected delivery timing lengthens, their levels of required  investments increases, and the uncertainty of their realization broadens. Businesses must calibrate their maturity in performing this analysis, and incorporating their results into business decisions for each proposed investment. As alternative options are characterized, and opportunities are shaped through subsequent decision gates, tensions between stakeholder objectives inevitably escalate. Decisions in this context require better information and control of relevant factors than may be accessible early in an endeavor's lifecycle, so this intrinsic uncertainty must be accounted for in the analysis. In most situations, investments must yield increased revenue, fewer people, and less things, rather than promises built on intangible platitudes.

Ranking and factoring such criteria across multiple opportunities can become a daunting task. The resulting decisions depend on the clarity and precision of the corresponding business vision and strategies, the cohesion and fidelity of the underlying concepts, facts, and data upon which opportunities are based, and the verifiability of selections from a list of system quality attributes defined for implementation. Each selected attribute should reflect a primary area of concern of some stakeholder. There are many ways to integrate the viewpoints of these stakeholders, but the Analytic hierarchy process has the advantage of being able to incorporate human judgments as well as quantitative facts and data. An example of an online AHP implementation for a product development project is described here.

Other scoring systems (such as Technology Readiness Level and Manufacturing Readiness Level assessments) can be helpful when objectives like attainability must evaluated. For these assessments to be reliable, they should be performed by expert, independent evaluators (to avoid biases and gaming), based upon an examination of evidence, rather than testimony. As an example of the challenges which arise in such evaluations, consider this exchange on solar power economics, and the communications challenges which it reveals. Or tensions between rationality and beliefs, which stretch the ability to change, and accommodate entrenched positions everywhere. Or the lenses that cause us all to see the same thing differently, a phenomenon you can see here.

Once the criteria for evaluations and stakeholder priorities have been defined, a down-selected set of candidates should be evaluated according to three additional tests, to validate that they are ready for implementation:

  1. Is the vision for the proposed project clear and viable?
  2. Is the business positioned to capture the value offered by the endeavor?
  3. Is the proposed investment the best use of available resources within expected environments?

Investment should only be pursued after these questions can be answered satisfactorily, and their corresponding risks have been identified and can realistically be mitigated over time. Endeavors are successful when they focus on the most important of these attributes, when they can be tracked through a well formed set of technical performance measures and measures of effectiveness, and once confidence has been demonstrated that planned work is likely to deliver a balanced set of outcomes across these dimensions.

AttachmentSize
PDF icon Value attributes.pdf14.97 KB

Rating: