« Introduction to the weblog | Main

November 12, 2004

7.0 Process frameworks

Blog root page
previous post in category

Process frameworks have the most unjustified reputation of any aspect of the software development environment.  Much of the reputation is passed mostly via word of mouth among developers and that portion is steeped in ignorance.  Unfortunately some of it is perpetrated by a couple of overly zealous advocates of the agile OOP-based processes who, most charitably, don't know what they are talking about.  The result is that a number of myths have become part of the Conventional Wisdom of software development:

Myth No 1: All process frameworks are bureaucratic.  This is nonsense.  A process framework is only as bureaucratic as its implementors make it.  Alas, this myth arose because there have been a lot of very poor implementations of process frameworks.  Some of the blame for that resides in the way process frameworks are expounded.  For example, the exposition of the CMM in its main bible takes all its examples from the world of DoD mega-projects.  But the framework itself is not inherently bureaucratic despite that exposition.

Myth No. 2: Process frameworks stifle creativity.  Again, nonsense.  Process frameworks only define what one should do, not how one does it.  In addition, most process frameworks say nothing at all about the intellectual content of software.  That sort of creative intellectual contribution is left to specific A&D methodologies.

Myth No. 3: Process frameworks impose a waterfall model.  This is generally true.  However, the implication is that the waterfall is large scale (i.e., one waterfall per project).  A few antiquated process frameworks did have this weakness.  But newer process frameworks fully embrace incremental, iterative development life cycles.  So the scale of the waterfall can anything down to the code fragment level.  Even the agile, OOP-based processes embrace the waterfall for their iterations.

Myth No. 4: Process frameworks are meant for huge projects with a cast of thousands.  Much of the research that produced process frameworks was, indeed, triggered by such projects because they are very difficult to manage and historically they have had high failure rates.  However, the principles behind modern process frameworks are borrowed from the well-established field of process engineering and they can be applied independently of scale.  What differs between large and small scale implementations is the way the framework is implemented.

Myth No. 5: Process frameworks are rigid and inflexible.  Some are.  But the good process frameworks, which includes almost all of the modern ones developed after 1980, are designed to be flexible.  For example, Level 5 of the CMM is devoted exclusively to ensuring that the framework implementation is flexible and self-correcting.  [I have always found it ironic that advocates of XP tend to be among the most vocally opposed to process frameworks, yet XP is probably the most rigidly defined software development process I have ever encountered.  In my opinion the greatest weakness of XP is that it has no built-in practices for process improvement or to deal with change in the development environment.]

Now that the myths are out of the way it is time to talk about what process frameworks are really about.  A process framework is just a skeleton that organizes some set of processes in the development environment.  The main purpose of a process framework is to define what should be done to produce a quality product.  That is, the process framework defines what processes the organization needs to have in place and what basic activities those processes need to address.  The actual processes that an organization defines are the implementation of the framework.  The key phrase here is that an organization defines.  The framework does not define processes, the local organization defines the processes as it implements the framework.

This distinction between specification (by the framework) and implementation (by the local organization) is at the root of most of the misconceptions about process frameworks.  (At least in the software industry; other industries are not at all confused about the distinction.)  In fact, the best way to initially implement process frameworks is with a minimalist approach where one implements processes that are the absolute minimum necessary to meet the framework specification using a broad interpretation.  One can then employ ongoing incremental process improvement to fill the gaps when it becomes apparent that the implementation was a tad too liberal.

[Anecdotal data point.  I had the good fortune to work for many years in a small organization where everyone was a grownup.  In that environment many of the processes we implemented were based on verbal Hallway Decisions where there were no paper trails signed in blood and stored in vault under the Rockies.  Thus no one physically signed a specification cover sheet; their attendance noted in the review meeting's minutes was sufficient.  Our shop was knee-deep in well-defined processes specified by the CMM but there was very little bureaucracy.  That was demonstrated by the fact that we maintained productivity that was integer factors better than figures published by industry Players for comparable software and our defect rates were at 5-Sigma for commercial grade software.  I attribute that degree of success to the fact that we were a process-oriented shop.]

Another thing to note about process frameworks is that there are a lot of different flavors, each with different goals.  Frameworks like TQM and Baldrige were initially focused purely on process improvement techniques rather than the processes themselves and they had much broader application than just software development.  Frameworks like ISO and CMM also have much broader application than software but they focus on what processes are necessary in a given environment.  Frameworks like RUP are a hybrid of process and methodology specification.  This segues to a distinction that I find useful when contrasting process vs. methodology.

Process.  A series of activities necessary to achieve a defined goal, usually expressed as a sequence of steps.  In software development environments the activities manipulate physical artifacts like work products.

Methodology.  A process where the activities are primarily intellectual and only the final goal may be physically manifested (e.g., a stored model or code).

Generally processes in the sense above are not very creative, other than the act of designing the process itself (which is a lot like OO development because one uses abstraction to capture invariants).  The processes that a process framework specify usually just provide a step as a place holder for a methodology that is executed to produce a work product for that step.  Thus the notion of creativity is largely irrelevant to process frameworks.  One employs process frameworks to organize the many rote activities that surround the more creative aspects of software development.

Blog root page
previous post in category

Posted by HS in Processes | Permalink