What Is Good?

Apparently my theme this week has something to do with measuring progress (I didn't plan to have a theme - it just sorta happened). While previous posts have mostly focused on the measurement part, let's take a look at the "progress" part today.

Specifically, I'd like to talk about priorities and values. What attributes do we consider desirable for our projects? What are our measures of merit? What indicators give us reassurance that we've got a good program?

Regular readers of this blog know I think it's "important and good" for a program to be Fast, Inexpensive, Simple and Tiny. I apply these values to every part of the program, from the requirements documents and system architectures to the organizational structure and operational processes. My research indicates that the FIST values not only support programmatic success (i.e. delivering on time), but also operational success (i.e. performing well in the field).

The funny thing about values is they are often assumed, not discussed. I seldom see explicit statements about a project's values, but there are plenty of indicators that show what the project leaders really think are important.

Look at any number of recently cancelled projects, and I think you'll find that nobody thought it was particularly important for the thing to be developed quickly or inexpensively, or for it to be simple. Rather, complexity is often treated as a sign of sophistication. Big budgets are a sign that the system is important. Take a long time to build it? Great! That means you're clearly doing a good job and exercising due diligence.

Here's the thing: when we aren't deliberate with our values, when we don't examine and discuss them, we run the risk of being guided by values that are counterproductive and ineffective. I'm not saying that FIST is the only value set that works - I'm just saying we should be purposeful with our values, and should be aware of the way they shape our decisions and behavior.

Two Guesses & A Phenomenon

I can't believe I haven't already mentioned Dr. Roger Atkinson's paper titled Cost, Time and Quality: Two Best Guesses and a Phenomenon,

It's a great read. He points out that we come up with budgets and schedules "at a time when least is known about the project," then measure success based on the degree to which we live up to those guesses. Anyone else see something funny about that plan?

I would simply add that the longer the project, the less we know about it on Day 1 (or on Day -100, which is probably when we actually put the budget & schedule together).

Just one more reason to keep our schedules short... along with a compelling argument that the way we measure progress is often misleading, even when it's "accurate."

Earned Value Management

As a follow-on to yesterday's post, I'd like to say a few words in praise of something called Earned Value Management (aka EVM).

EVM is basically a way of measuring progress in terms of actual work accomplished, not simply in terms of the number of discrete tasks that have been marked Done. It seems to me that EVM is just another word for honest accounting.

It boggles the mind that a project leader would use anything other than EVM to measure, assess and report progress towards their goals.

Some people try to say it's too complicated, too cumbersome and frankly too dang hard to do EVM. I've also heard that EVM isn't well suited for small projects, but in fact there are ways to do simple implementations, without relying on excessive overhead. For that matter, the simple implementations can probably be useful even on big projects.

Now, applying EVM to certain types of research projects, where future work paths are unknown, requires a slightly different tack... but it's possible to use some of the principles even in this area.

EVM - it's not just for breakfast anymore. Check it out!

Statistics

I recently read a quote that said something along the lines of "95% of acquisitions are done on time, on budget."

My first thought was - huh? What about all the GAO reports that say (over and over and over again) that military acquisition projects take too long, cost too much, and under perform when fielded.

And then I realized that it's entirely possible for both to be true.

When we say a project comes in on time & on budget, it sounds like a good thing. But what if it had too much time & money to start with? It's possible to spent too much money AND be on budget.

And what do we really mean by "on budget." Does that mean we delivered the stated capability using the original amount of funding? Or are we taking credit for staying under a budget line that has increased over the years, as additional funds get added to the baseline (because we've increased the schedule, added requirements, etc)? If we define "on budget" broadly enough, then I'll bet almost everyone is on budget. Maybe even 95% of projects.

But there's more to this story. The idea that 95% of projects are on time & on budget doesn't actually mean as much as it might seem. Imagine a portfolio of 10 projects. Nine of them each cost $100, and each deliver on time, on budget. The 10th project was supposed to cost $100,000, but ended up coming in at $150,000. Do we praise this portfolio for being "90% on budget," or do we acknowledge it spent $50K more than planned? I'm thinking the latter assessment is more accurate.

The flaw in the 95% on budget reasoning is that it is measuring performance on a per-attempt basis, using a straight count of the number of projects. What we should be measuring is the performance of our dollars (i.e. the proverbial bang for the buck).

Similarly, let's say we have a project that has 10 steps. Nine of the steps each take one day to accomplish. The 10th step takes a year. At the end of nine days, if we say we're 90% done with the project, because we have finished nine of the steps, we're seriously misrepresenting how much work has been done (and how much is remaining).

Statistics are funny things. It's entirely possible for a statistic to be true and misleading all at once. It's important to be careful and make sure the things we're measuring and reporting provide an accurate representation of the situation.

Faster, Better, Cheaper?

I've been doing some research into NASA's Faster, Better, Cheaper initiate from the 1990's. It's quite a story, and I must admit I'm stumped as to why FBC got scrubbed.

The results of the FBC missions were fantastic. For less than the price of the Cassini mission, NASA launched 16 mission, including the Pathfinder mission to Mars and the Near Earth Asteroid Rendezvous (NEAR), which no kidding, landed on an asteroid, after collecting 10 times more data than they'd expected. Of those 16 mission, only 10 succeeded, but that's still 10 successful missions for less than the price of a single planetary mission using the "traditional" Slower, Suckier, More Expensive approach (excuse me - I mean, the deliberate, modernist, Scientific Management approach, which never ever fails ever).

The crazy thing is, many people treat FBC as if it was not only a failure, but an embarrassing failure. As if NASA should have known better than to try something so absurd as reducing costs and delays in their pursuit of knowledge and adventure in space. As if 10 successes for the price of 1 is an inadequate track record. As if the data clearly shows we must "pick two," instead of pursuing simultaneous improvements in all three dimensions. I don't get it.

OK, I'm not really stumped. I have my theories about why the approach was rejected. But it's becoming increasingly clear that the rejection and ridicule of the FBC approach has nothing to do with the initiative's actual results, which were admirable and impressive.

How about you? What do you think when you hear the phrase "faster, better, cheaper"?

Rethinking the Principles of Program Management

Some interesting concepts that come out of asymmetrical warfare strategy talk to the importance of taking advantage of opportunity and agility in maneuvering against an enemy. Often, these principles are linked to the conduct of war as an art form. As an art form, principles devised to guide conduct are not considered immutable or inviolable. One author even states,

The crucial element of its artistic application is recognizing unique contexts, the contingent factors, and the opportunities to create advantage purposely by violating principles or rules when needed. [A]rt accepts the existence of principles and rules, but only as guides. Each case has its own features - which modify the application of the rule, and may even make it at times wholly inapplicable.

- Frank Hoffman, Chap 17, Rethinking the principles of War, Naval institute Press, Oct 2005

If this applies to the messy work of warfare, why not program management? Isn't program management as much an art form as warfare? I think so. Yet too often (if not exclusively), the same defense department that conducts war treats the business of weapons technology development as a precise science to be quantified, measured, predicted and proven. This behavior is as if the human factor of "business" doesn't exist. But it does and with it comes the art of business. A craft that can't be accounted for by implementing rigid processes or precise controls. The artistic practice of business allows for unknowns and flexes to let humans use their ingenuity to solve problems rather than forcing them to stick strictly to the script. The artistic practice of business is attuned to serendipity.

Removing an over reliance on process and procedure is one step toward practicing the craft. Increasing tolerance of failure and allowing deviant practices as a matter of course are others. Anything that puts people back into the drivers seat of business - that's how to practice the art of business.

Windtunnel?

I recently came across the idea of an "acquisition wind tunnel" that would allow us to test our organizations and projects, to see how streamlined and efficient they are.

I like that imagery... I think. At the same time, I wonder if some of the points of friction and resistance might not be some of the more interesting and productive parts of the project. I guess it depends on how we define friction.

It occurs to me that one person's friction is another person's traction. Similarly, one person's momentum is another person's inertia. So, in this hypothetical wind tunnel, we'd have to be specific about how we distinguish between negative friction and positive traction. And the tricky thing is, the line between friction and traction is probably pretty fuzzy (and it probably moves).

This wind tunnel concept no doubt could be implemented under a Theory of Constraints sort of framework. No doubt the Lean crowd would find that this idea resonates with them. And frankly that makes me hesitant about the whole thing.

At the same time, I do like the idea of setting up some well-defined streams of "wind" that we could point at a project, organization, etc, to assess its flight worthiness. A FIST-based wind tunnel could have real value in identifying opportunities to unleash the creative power of constraints and restraint.

The One Thing

If there was one thing I could accomplish professionally, one contribution to the corpus of program management practice and theory, it would be to dispell the Myth of Inevitability.

What is this myth, you ask? It's the idea that high tech system development projects (weapons, spacecraft, commercial products, etc) inevitably take a long time, cost a lot and are complex. The myth is expressed in phrases like "better, faster, cheaper - pick two." A corollary to this myth is the idea that adding time and money to the project improves the outcome, as if the problem with our failed system development efforts was that our schedule was too short and our budget too small.

No kidding -that's what they said when the 18 year, $7B (billion-with-a-b) Comanche helicopter was cancelled. They needed more money. They needed more time. Yeah, that would have helped. Right. A similar chorus rises from any number of failed high tech projects. My assessment is that they had too much time & money, not too little. And at the core, they believed that the costs and delays were inevitable, simply an inherent part of this kind of work. It's a tragic belief and a self-fulfilling prophecy. It's also not grounded in reality.

The truth is, there is nothing inherent in military technology (for example) that requires it to cost so much, take so long or be so complex. Yes, systems like the F-22 took a long time and cost a lot. But there's a difference between "it took a long time" and "it takes a long time." We could have done it better (both programmatically and operationally). If we set up our values correctly, if we cherish our constraints and pursue intelligent simplicity, we don't have to spend billions and decades. We can do it for millions and in years (or thousands in months). The F-117, the SR-71, NASA's Near Earth Asteroid Rendezvous (NEAR) mission and the Pathfinder Mars mission are all examples of high-tech projects built on a FIST (Fast, Inexpensive, Simple, Tiny) foundation.

Despite the evidence of a significant portfolio of FISTy projects (documented in my master's degree thesis), the Myth of Inevitability insists that things like this have to be complicated and expensive. Believers in the myth see no alternative to bureaucracies, technologies and processes that are complex, expensive and slow.

Their failure to see through the myth reflects ignorance of the past and a lack of imagination. There are alternatives. Download a free copy of The FIST Handbook for more details on the principles, activities and examples of how to use constraints to foster creativity and deliver systems that are simultaneously faster, better and cheaper.

Everything is...

I keep coming across the phrase "everything is a process."

Um, I don't think that's quite true.

Rocks are not processes (although they can be described as the product of a process). For that matter, I don't think any nouns are processes. I'm going to go out on a limb and say that only a verb has the potential to be described as a process. This means an outcome is not a process. Hold that thought...

While we're talking about words, let's consider semantics. It might be true to say "everything you do can be represented as a process." That's a bit different than "everything IS a process," wouldn't you agree? I suspect that's what people mean when they say "everything is." But why say everything when you mean every activity? And why say is when you mean can be represented as? Those concepts are quite different.

More to the point, even if everything we do can be represented as a process, that doesn't mean everything should. Some activities are best performed with an intuitive, craftsman-like touch. Not because it's easier, but because the end product is better. As my favorite poet ee cummings pointed out:

since feeling is first
who pays any attention
to the syntax of things
will never wholly kiss you;

Process, it seems to me, is all about "the syntax of things." It's not about the outcome. Process may be about kissing, but it's not about The Kiss.

Another observation: some activities are not repeated / repeatable. Some projects are unique, some activities are unique. But repeatable or not, it's important to consider the amount of time, money, effort, etc that goes in to developing, documenting and validating a process. Sometimes, it's just not worth the expense. For example, do we really need to document a process if we're only going to do it once? What's the point of documenting a process if the people doing the activity are already effective and efficient? It's a calculation we should consider.

Sure, if you've got a large number of people doing well-bounded, well-defined, repeatable tasks, in which variations are undesirable and the environment is generally predictable, by all means, document your process and do it that way. But not everything is in that category.

Is everything a process? I'll take "Answers That Begin With No" for 500, Alex. Can everything be represented as a process? Sure. Are there some activities that should not get the process treatment? You bet...

The Realm of the Possible

In a recent discussion about improving defense acquisitions, someone suggesting things would be better if we could get rid of Congress. Everyone laughed at this humerous suggestion, and one particularly literal-minded person said "Oh, that's too big of a change, it can't be done, let's move on..."

Well, hold on.

Maybe there's a way to "eliminate" Congress from the project without disbanding one third of our constitutionally-established government.

See, the problem isn't Congress' existence. The problem is that sometimes Congress provides, as Shrek so delightfully put it, "the opposite of help." So, what if we could minimize this anti-help?

It turns out, we can. The FIST (Fast, Inexpensive, Simple, Tiny) approach does exactly that.

A FISTy project has a small budget, which attracts less interest and oversight from our congressional leadership. A short schedule provides fewer chronological opportunitites for, let's call it involvement (instead of meddling). A small team and simple organization doesn't get spread across multiple congressional districts and states, and voila, that means fewer legislators with a dog in the fight.

These decreases are all appropriate and above-board. It's not a matter of shutting Congress out, but rather of keeping the project sufficiently small that it doesn't require their, ahem, help.

Process Application

Most of the process advocates I've met recently have come from background like manufacturing and logistics. That makes a lot of sense - these are well-bounded, repeatable, concretely-defined disciplines. The problem is when we try to cut and paste successful manufacturing or logistics approaches onto other areas, like research & development or program management.

R&D is typically - and appropriately - an unbounded, unique, loosely-defined activity. That doesn't mean there are zero processes involved... just that the most significant parts of R&D are beyond the realm of process.

So, when I say that I'm a critic of the process-centric approach, what I'm really objecting to is the misapplication and the overapplication of these approaches.

I also want to point out that I'm not a skeptic of the utility of process-centric approaches in R&D and program management. I'm emphatically a critic. The term skeptic sounds like a person is unsure whether a thing will work or not. Not me - I'm not agnostic about this stuff. I've done my research, taken the classes, got the t-shirt, and came to a conclusion. I'm well beyond skepticism.

Process Is Neither The Problem Nor The Solution

I've been thinking about this whole process-centric approach for a while now, and I think I figured out a way to state my position in words the Lean and Theory Of Constraints crowd can understand:

Process is not the bottleneck. Talent is.

This means process is neither the problem nor the solution. Talent drives action, talent dictates outcomes, and at the risk of sounding like Tom Peters, talent is, well, it's everything. [not that there's anything wrong with sounding like Tom Peters - the guy's amazing].

Process is a poor substitute for talent. You can't make up for a talent shortage by simply instituting more and more (and more and more) processes. If there is indeed a talent shortage, the way to deal with it is by... unleashing more talent! And a compliance-based, process-centric, dictatorial, command & control approach squelches and repels talent.

Now, there's nothing wrong with using and improving our processes. But when our approach to improving performance is process-centric instead of talent-centric, we end up discounting and preventing the very thing that matters most.

Presentation Skills Are Critical

The ability to stand up in front of a group of people and communicate effectively is a critical professional skill for program managers and other project leaders. It's particularly necessary for military engineers, defense contractors, and leaders in general. I can't say strongly enough how important this is as a matter of professional competence.

For better or worse, PowerPoint is the tool of choice for making presentations. When anyone gives a presentation with dense, complicated, unreadable, illegible and otherwise horrible-horrible-horrible presentation charts, they are demonstrating sheer professional incompetence.

In case you can't tell, I get kinda worked up about this.

For all the time we spend giving and receiving briefings, it's inconceivable to me that honing one's presentation technique is not treated as an important discipline and a core element of professional development. I've even seen professional educators, who brag about their ability to facilitate discussions and interact with audiences, use powerpoint charts that are frankly embarrassingly bad.

This stuff isn't hard. Sure, it's a little bit more work than doing it badly, but once you get the hang of it, it's not bad. Go check out Garr Reynold's Presentation Zen blog for some great examples of how to do it right.

Change

I like to say that FIST isn't a new way to do acquisitions. It's just how we do acquisitions when we do it well.

But frankly, I'm not convinced my current attempt to drive FISTy change into the system is going to succeed. I am very aware that the probability of success is low. Five or ten years from now, there's a pretty good chance the defense acquisition community is going to look very much like it does right now.

Having said that, the environment does seem more change-friendly than it ever has before. Our experiences in Iraq and Afghanistan are a big part of that, along with the new administration, changes in the economy and advances in technology - particularly UAV's. This combination of operational, political, economic and technical factors is sort of a perfect storm and gives me hope that significant, meaningful change is possible. If the FIST approach ever had a chance at widespread adoption, it's in the current climate.

Our troops overseas are increasingly emphasizing the importance of rapid access to systems that are simple and effective. They have less tolerance for the complexity and delays of previous years. The president ran on a platform of change, and military leaders from the SECDEF on down are reimagining the way we do business. The economy is forcing people to think really hard before spending money. And along come these UAV's, which offer relatively inexpensive and simple ways to accomplish missions that used to require much more time, money and effort.

So will FIST take off? I don't know. I certainly hope so. All I know is that I've got to try to make a difference, even if the odds are against me, 'cause these are the best odds I'm ever likely to see.

It's Not Inevitable

OK, let me summarize the whole FIST thing as concisely as possible. It really comes down to this:

"There is nothing inherent in military technology that requries it to cost so much, take so long, or be so complex."

There, I wrote it down for everybody on the Interwebs to see, and it feels good.

The more I read, the more I do, the more I hear people's stories, the more convinced I am that complexity is not inevitable. We do it to ourselves, technologically and organizationally and procedurally... but we dont' have to. Similarly, these huge cost overruns and schedule delays are not inevitable. Until we believe that - I mean REALLY believe it - we're going to keep getting the results we've been getting.

Incidentally, this applies to NASA's space projects as much as the DoD's systems... and no doubt to commercial & industrial projects as well.

CogBlog Homerun

Check out this (rather lengthy) post over at the CogBlog. It's an insightful look at the saying-doing gap in defense acquisitions, primarily focused on the IT side of things. A few nuggets to whet your appetite:

DoD acquisition regulations do not actually mandate stupidity … they just encourage it.

...the Joint Capability Integration Development System, or “JCIDS“, mandates a change from “system-based” to “capability-based” requirements... However formal OPTEST evaluates systems, not enterprise capability.

What is the gap between the capability provided by Travelocity vs. Defense Travel System? [Dan here - holy cow DTS is a pain to use!]

We need pre-approved catalog offerings available on GSA schedules and IDIQ vehicles. We need “dating services” to put consumers with like requirements in touch with each other and expert providers. We need “consumer reports”... to provide basis of objective, convenient comparison.

Cancelling Projects

From a justice perspective, it's important for an accused person to have a good lawyer. Even if the defense attorney thinks the client is guilty, they are still supposed to provide a robust defense and make the strongest case possible.

Unfortunately, program managers often act like defense attorneys for their projects. They sometimes act as if their job is to keep the project alive, and defend it to all comers... even if the project is a dog and needs to be cancelled.

The program manager (arguably) knows the most about the health and viability of the program. If it's heading over a cliff, aiming for irrelevance or otherwise doomed, the PM is usually among the first to know, and should speak up. Sadly, this does not always happen.

PM's are not defense attorneys. Their job is not to keep the program alive and out of jail. It's to guide and shepherd the development. And if the program needs to be cancelled, the PM should lead the charge.

UAV's: Imagine The Possibilities

In the Jul 09 issue of National Defense magazine, Col Eric Mathewson, director of the Air Force's Unmanned Aerial Systems Task Force discusses some of the things UAV's will never do: "There's no way you can replace an F-35 or F-22 or something like that... No way."

In related news, "Lt. Gen. Dave Deptula, the Air Staff's head of intelligence, surveillance, and reconnaissance, said it is not unrealistic to imagine an unmanned aerial vehicle as successor to fifth-generation fighters-the F-22 and yet-to-field F-35-in our national military strategy, however he added that UAVs still require some technology maturation before they would be ready to assume that mantle. (AF Magazine Daily Report, 27 Jul 09 )

Which reminds me of a quote I heard somewhere, that went something like this: When the esteemed and aged expert says a thing can be done, he is most certainly right. When he says it cannot be done, he is most certainly wrong. (can anyone help with that reference?)

The Answer

One of the truisms of program management is that the answer to just about any question is "It depends." So I had to chuckle as I was leaving one of the buildings on the Defense Acquisition University campus and this little marble pillar caught my eye.

There it was, engraved for all time, the answer to every program management question.

Class dismissed.

FIST Presentation

Gabe and I recently had an opportunity to do our FIST (Fast, Inexpensive, Simple, Tiny) presentation for the Defense Acquisition University. It was a fun time and a great audience. They even let us take home some of the cool posters they made to advertise the show.

The more time I spend in DC, the more convinced I am that this message needs to get out there. There is such an infatuation with complexity among military technologists, and there's a widespread belief that things would be better if we just had more money, more time, and bigger teams. And don't get me started on the preference for formal processes, control and mandatory activities.

While the FIST ideas generally get a good reception and seem to resonate with our audiences, I'm coming to see exactly how counter-cultural these ideas are. We've got quite a fight ahead of us if we're going to try to get wider acceptance of the FIST approach. Wish us luck!