Past Book Review (November 23, 2008): "Organizational Patterns of Agile Software Development"

Past book review (i.e. posted prior to starting this blog) for Organizational Patterns of Agile Software Development, by James O. Coplien and Neil B. Harrison, Prentice Hall PTR, 2004, reposted here:

There must be other readers who find a bit awkward the following comments provided within the first page of the introduction: "O.K., we'll be frank: we chose 'Agile' for the title out of marketing concerns. It seems to be the current term of choice for the kinds of things we describe in this book. It is a term that rolls off the tongue more easily than other clever names that clamor for your attention on today's bookshelves". Now, the authors quickly follow these comments with the argument that many of the ideas shared here had their genesis prior to the Agile software development movement, they also note that the scope of this book is broader than "so-called" agile development. The authors attempt to describe their interest in "effective" rather than "agile" software development, and that some of the ideas provided within the pages of this book "would fit nobody's definition of agile". In addition, some of the patterns shared here, the authors continue, were criticized by one of the founders of Extreme Programming (XP), who noted that "anyone who worked on organizational issues was avoiding real work".

This reviewer provides this background here because the book starts off on a strange note, provided that anyone is willing to read the introduction. In the conclusion to their introduction, the authors save themselves by getting down to the core purpose of this book: "We have captured the good things organizations do, and we have recorded them as patterns. We hope these patterns will be as interesting and useful to you as they have been to us". It is with this frame of mind that one needs to read this book, if it is to be of any use to them in their business of software development. The history behind this book is what might make it particularly attractive to potential readers. As stated, "this work draws on data gathered as part of the Pasteur process research program at AT&T Bell Laboratories. In the Pasteur program, we studied software development organizations in many companies worldwide, covering a wide spectrum of development cultures".

The Pasteur studies used social network diagrams and adjacency diagrams to make some sense out of the network of organizational roles and the communication paths between them. While this reviewer found these diagrams to be of some aid in understanding the associated dialogue with each, at first glance they sometimes appear as like large entity relationship (ERD) diagrams with entity relationships liberally overlapping each other to a degree that can pose a challenge to even the most attentive reader. Another type of diagram that the authors provide, the interaction grid, which is intended to convey a communication matrix for a given organization, sometimes adds to the discussion but often does not, especially when the number of roles included is very high.

The authors, however, seem to understand the costs and benefits of process usage very well, noting that they started their studies on process and organization just as software development standards increased their influence in the 1980s and 1990s, and while process can add significant value, these cultures "often ignored important variations in organizational behavior that are key in dealing with market uncertainties or the uncertainties that arise in any process rooted in human intellect and instinct". And three problems typically arise with the process specification document as the final word on development activities in practice: lack of empirical conformance between practice and process specifications, incompleteness of process models, and inability to capture long-term stable process abstractions.

One of the best quotes that the authors provide in the entire book is as follows: "In Contextual Design, we always try to build on natural human ways of interacting. It is easier to act, not out of a long list of rules, but out of a simple, familiar model of relationship. A list of rules says, 'Do all these things' – you have to concentrate so much on following the rules you can't relate to the customer. It's too much to remember. A relationship model says, 'Be like this' – stay in the appropriate relationship, and you will naturally act appropriately." (For a much more in-depth look at this debate, see my review for "Balancing Agility and Discipline" by Boehm and Turner.)

While the project management pattern language, piecemeal growth pattern language, organizational style pattern language, and people and code pattern language portions of this text hold the most content, it is unfortunately a fact that much of the content provided in these sections, however truly relevant, can now often be found elsewhere in late-2008 industry literature. The difference that this content provides is that organizational design and construction patterns are presented as patterns, such as "Community of Trust", "Architect Also Implements", and "Incremental Integration", much like other books of the patterns genre.

In the opinion of this reviewer, the most insightful portions of this text that might be of interest to individuals already familiar with many of these ideas are contained within the organizational principles and anthropological foundations portions of the book. Essentially, the former section, as the authors indicate, "offers practical advice that will make it easier to apply the patterns. You need to know when an organization is ready to try out patterns, and, once it is ready, you need to know how to apply the patterns. You also need to know what to do when a pattern doesn't seem to be working. This isn't an academic exercise: The future of your organization is on the line". Among the topics discussed here are dissonance, team burnout, crisis management, change, empowerment, schismogenesis, team building, piecemeal growth, and the role of management. The latter section follows the thought of the authors that "it would be a good idea to build on the insights and foundations from anthropology, the formal world of the study of culture". Among the topics discussed are structure and values, shortcomings of process, roles and communication, and social network theory.

Some readers might be interested in the fact that one of the closing chapters to this text is a rather interesting case study on the Borland Quattro Pro for Windows project, which was adapted from an article written by one of the authors, Coplien, and Jon Erickson, that appeared in a 1994 industry publication. The answer of "probably not" is provided to the following question: "Can we capture the architecture of the Borland development organization and process and expect phenomenal results if we apply it to large development projects such as those that we have at AT&T?" The productivity that was demonstrated on that project, however, demonstrates what is possible.