09 July 2009

"Process Performance Failure"

I recently came across an article titled "Understanding the Roots of Process Performance Failure." One of the authors is Dr. Bob Charette, the guy who wrote the great article for IEEE Spectum titled "What's Wrong With Weapons Acquisitions?"

The article asks why "there is an increasing gap between program cost, schedule and technical performance requirements and the capabilitiy of program teams to realize them," particularly given the "increased process focus within DoD programs."

The analysis indicates "software, systems engineering and management processes... were primary contributors to poor program performance." [emphasis added]. The authors also observe that "in 80 percent of the assessed programs where no process adherence issues of merit were found, process capability issues were still discovered." In other words, 80 percent of the time the project teams followed the process, but still had performance problems. The processes themseles were causing poor performance.

How could this be? Isn't "Process" supposed to improve things? The authors explain "Process adherence is mistakenly seen by too many program teams to automatically equate to process capability..." Further, "adherence-oriented models... are based on a generalized organizational standard of what most projects require, not on what any specific project requires. While these process models are intended to be tailored for specific program needs, the data suggests that in practice they often are not..." [emphasis in original].

The point is that there's a big difference between compliance & adherance on one hand, and creative, imaginative application on the other. Processes are supposed to be tailored. If we don't tailor them, we miss the whole point, because as Alex said in yesterday's post, "projects are unique."


Andrew Meyer said...


isn't there also another part? One can look at process adherence, process capabilities or even tailoring projects, but doesn't that beg the question about the results? You could have the most tailored processes with wonderful adherence and tons of imagination, but if you're not aiming for the right result, it's doesn't matter.

Let's take my peanut butter and jelly sandwiches as an example. I could apply the most imagination, tailor my approach and adhere to the most exacting process standards, but if I've told the girl I'm making her pheasant under glass, no PB&J sandwich is going to be satisfactory. (Don't ask me how I know this...)

The Dan Ward said...

Indeed! This whole process discussion presupposes we've established a good, meaningful, well-defined objective.

And of course, the process-advocates will offer a process for determining the desired results.

I'm more of a fan of being open to emerging requirements, which develop over time (aka the "make it up as you go along" approach). Expecting to determine the full requirement set before Day 1 always struck me as a bit silly (unless delivery is on Day 2).

Andrew Meyer said...


having initially desirable results is critical to getting the project going and also to discovering emergent requirements. Before you can work on emergent requirements, someone had to budget for the project or at least convince others to join them. How can you do that without having an initial, desirable result identified?

For example, when I invite a girl over to the boat, I still convince her to come with the pheasant under glass story. Ulterior motives may well emerge, but delivering on my initial and desirable result will surely help with the emergence.

Now, whether I'm ultimately judged on the initial result or on the emergent one (however impressive that one might be...), my process should be to deliver the result initially promised.

Projects, whether professional or prurient, probably stand a better chance of success if the initially advertised result is desirable. Furthermore, desirable initial results help emergent ones, don't you think?

Gabe said...

For my two cents, I would have to disagree with you Andrew. There have been several times where I've set out with an initial objective, and failed miserably, at least in the narrowly defined specfics of what I would have labeled success. However, the failure turned out to actually be much better, albeit different, than what I intended. Being open to surprise is the key here and a success as defined doesn't necessarily get you where you want to go.

The Dan Ward said...

@Andrew - I think we're not too far off from each other.

As a general rule, I'd say there's always more than one way to do things. I've found it's entirely possible to secure funding, support, etc for a system development project in which the desirable results are only partially defined. But yes, it is probably essential to have SOME kind of objective in mind, some sort of initial requirements.

And I'd suggest that the process needs to be able to handle not delivering the "result initially promised" when that result (as so often happens) turns out to be undesireable, despite our initial perception of desirability.

Andrew Meyer said...

@Gabe - I think we look at the world differently. I'll have to work on it, but I don't think there is such a thing as a "different" result which is "better" than my initial objective.

I'm sorry, but if I start out chasing Megan Fox and fail, there is no definition of the word "different" that makes Rosanne Barr "better".

@Dan - I would agree with you and take it one step further. When I invite Megan to the boat, I may well only partially define my desired result.

You are right, the process needs to be very flexible to adjust to changes from the result initially promised (pheasant under glass) to make what is possible, desirable (PB&J).

And if I find that there are some back door surprise that is different and better than my initial objective, well... I might have to agree with Gabe.

Gabe said...

An example to describe my theory of surprise is photography. As an amatuer, I'm always tinkering with how and what to photograph. One time (several times actually) I took a shot where the result was completely foreign to what I was trying to capture. In essence, I had failed at getting the shot I was attempting. But the result was so hauntingly beautiful, and so at odds with my intent, that I never could have set out to do it. This surprise left me with a new perspective on my choices and how I define a shot.

The Dan Ward said...

@Andrew - I'm with Gabe on this one, my friend. Let me turn your example around to show you how different = better.

You start out chasing Rosanne Barr, then end up with Megan Fox. That's different. That's better.

From a technology perspective, this is particularly evident in LONG projects. We start out thinking we want to be able to do X at some point in the distant future. Unexpected things happen in the world of technology, in the market, etc... and it turns out that pursuing Y (which is different) would have been better. Gabe & I think it's important to anticipate that need for flexibility.

The basic principle here is that we don't always get it right on the first attempt, and that applies even to our definition of what "right" is.

Andrew Meyer said...

My midwest Mother had a saying: "He who pays the piper calls the tune." Maybe I can reinterpret the tune, but if I'm going to get paid, I better play what they want.

@Gabe - I see your point. The question is one of transaction costs.

A picture, let me rephrase that, a digital picture, has virtually no transaction costs. So snap away and let serendipidity work in your favor. If you take 1,000 pictures for every one you sell, that's fine.

However, when Ansel Adams had to set up, shoot and then develop all his own pictures; the transaction cost for each picture he sold were totally different.

Are your projects closer to digital photography or Ansel Adams?

@Dan - It's a question of project expectations and degrees. A certain amount of reinvention and discovery is expected, but the first time I sell Megan Fox and deliver Rosanne Barr, it's probably the last time they call me to play a tune.

And I don't agree with your reframing of my example. If the "Big is Beautiful" convention paid me to deliver Rosanne Barr and I delivered Megan Fox, I don't see them defining that as better.

Maybe I can reinterpret the tune I'm paid to play, but I better play the tune.

If I promise Megan Fox and deliver Keira Knightley, I'm probably fine. But if they bought Barr and I brought Fox, I'll be boxed and booed by the big and beautiful.

dcwork said...

Coming from the contractor perspective, I definitely see Andrew's perspective.

The fixed time and money approach that I take Dan to be talking about is in my mind essentially what most agile software development is about.

However, I have never directly come across a solicitation for a project that puts forward that agile perspective. The one possible exception is service/support contracts.

We can resovle the apparent conflict by changing the terms of the contract.

Who pays the piper always calls the tunes. It's a quetion of whether the piper and caller agree to the playlist before hand, or whether they're willing to be more interactive about it.

The second approach requires a lot more trust. Which is hard to develop in a competitive lowest-cost bidding situation.

Don C.