When a particular job can be performed by a single, experienced individual, the effort required for planning, decision-making, and status reporting can be kept to a minimum. But when a job instead requires a team to develop a complex product, which must be implemented within a multi-tiered architecture, the time, effort, and skills required to coordinate among the team members to accomplish these same activities can quickly become substantial. Sadly, when we plan for such jobs, we rarely give this coordination the forethought it deserves before these needs are upon us.
Jim McCarthy describes the ideal we should strive for when we perform this coordination:
The goal is to create a network of self-motivated individual commitments. Just as the goal of software design is to get all of the best ideas of the team into the box, the goal of managing deliverables (or, as we call them, handshakes) is to get each of the team's individual promises into the box.
What is being developed is not clear. To avoid going entirely dark, we need to see that the real development task is to create a community capable of making and keeping hundreds of small but vital promises. This has little to do with technology per se and much to do with integrity in the face of uncertainty.
Many of the challenges we face in achieving this vision come into focus when we consider a representative example of these complex products, such as when a hardware product requires sophisticated software to implement critical functionality. These types of complex products usually result in projects which are complex as well, and the path we must follow in coordinating all the players that must contribute to such an effort requires competent execution, communications and interactions at many different levels, and across many different disciplines.
First, we must craft an effective, multi-leveled architecture that is suitable for both the product we are producing and the structure of the team which must produce it. Second, the activities required to implement this architecture will require careful allocations of responsibility across many different disciplines. As an example, to develop a software-intensive embedded system, we typically must draw on approaches from systems engineering, electronic design, network design, structural and mechanical design and fabrication, and software design; we must weave all of these approaches together in a way that will converge into a useful, integrated, and robust solution.
Such products also require deep domain knowledge about the product itself. Domains can be as simple and predictable as they are within the commodity computing platforms and environments we find delivered within competent information technology organizations; these commodity solutions follow well-established design patterns, and build upon mature platform functionality and interfaces. Alternatively, when we instead engineer systems from scratch, domains are much more complex. For example, consider the domain knowledge required for the engineering development of new guidance, navigation and control functionality, and integrating the resulting subsystem into an automobile, ship, aircraft, or spacecraft. Across or within such engineering developments, many different parameters must be optimized simultaneously in order to address the critical constraints of the environment. For example, in all these environments, weight, fuel, payload, and range must all be optimized, though the specific mix varies because of the mission and environment of each type of vehicle. Yet no single contribution from any particular group in each of these settings can be solely responsible for achieving these goals; everything is simply connected to everything else.
A highly simplified representation of this type of development environment is shown in the above diagram. In this depiction, three separate work cells are represented, and labeled as Group I, Group J, and Group K. This is a gross oversimplification of most complex projects, since the number of performing organizations in real-world, complex products can easily exceed this number by several orders of magnitude. As the diagram shows, concurrent with this top-level shaping, each of the work cells that are responsible for implementing selected domains and disciplines must plan the steps involved in performing the work assigned to their group; this is known as a work package. These plans for developing work packages will be implemented to produce the components X, Y, and Z in the illustration shown above. These work cells then must execute these plans while collaborating with the other groups who are concurrently implementing their work package. While a top-level plan may provide targets for all these contributors, it is the lower-level details that will determine the duration required to develop the components required by the solution. Finally, the work products which result from this work must then be integrated and refined into a form that is of acceptable quality, expandable over time to support evolving customer needs, and supportable by the business.
To effectively distribute work across all of the organizations which must perform this work, and subsequently integrate their components into the framework of this multi-tiered architecture, the participants all need to understand their roles, and the outcomes they are responsible for producing. To act in a single-minded manner, team leaders need to integrate the techniques which will be employed by the individual members of the team, and rationalize the perspectives and beliefs they have about this work. The rate and degree to which these different perspectives and capacity to execute can be shaped, integrated and synthesized towards common ends will determine how long it will take and what it will cost for this community to deliver a coherent and robust solution.
It is neither practical nor efficient to attempt to involve everyone in all the decisions which will be required to perform this work. One alternative involves assigning proxies who have the knowledge and authority to communicate and speak for the needs of their represented areas, and negotiate across their many different perspectives. Another is to identify an architect with designated authority to dictate the structure, rule set, scope, and phasing for design, construction and assembly of both the whole and the parts. A third approach is to create an framework which will structure these interactions, and monitor their effectiveness.
Regardless of the choices made, the technical details of how teams are expected to work together in accomplishing the overall system's engineering activities must be one of the primary areas of emphasis across domains. These details typically are described in a Systems Development Plan, which defines the structure of the product being produced and the teams who are producing it. In turn, everyone on these teams should use this plan to organize and weave together their activities. This information forms the playbook for collectively managing the technical details of what work needs to be done, when it needs to be done, who is going to do it, and how progress will be tracked over time.