What Is Good?

Apparently my theme this week has something to do with measuring progress (I didn't plan to have a theme - it just sorta happened). While previous posts have mostly focused on the measurement part, let's take a look at the "progress" part today.

Specifically, I'd like to talk about priorities and values. What attributes do we consider desirable for our projects? What are our measures of merit? What indicators give us reassurance that we've got a good program?

Regular readers of this blog know I think it's "important and good" for a program to be Fast, Inexpensive, Simple and Tiny. I apply these values to every part of the program, from the requirements documents and system architectures to the organizational structure and operational processes. My research indicates that the FIST values not only support programmatic success (i.e. delivering on time), but also operational success (i.e. performing well in the field).

The funny thing about values is they are often assumed, not discussed. I seldom see explicit statements about a project's values, but there are plenty of indicators that show what the project leaders really think are important.

Look at any number of recently cancelled projects, and I think you'll find that nobody thought it was particularly important for the thing to be developed quickly or inexpensively, or for it to be simple. Rather, complexity is often treated as a sign of sophistication. Big budgets are a sign that the system is important. Take a long time to build it? Great! That means you're clearly doing a good job and exercising due diligence.

Here's the thing: when we aren't deliberate with our values, when we don't examine and discuss them, we run the risk of being guided by values that are counterproductive and ineffective. I'm not saying that FIST is the only value set that works - I'm just saying we should be purposeful with our values, and should be aware of the way they shape our decisions and behavior.

Statistics

I recently read a quote that said something along the lines of "95% of acquisitions are done on time, on budget."

My first thought was - huh? What about all the GAO reports that say (over and over and over again) that military acquisition projects take too long, cost too much, and under perform when fielded.

And then I realized that it's entirely possible for both to be true.

When we say a project comes in on time & on budget, it sounds like a good thing. But what if it had too much time & money to start with? It's possible to spent too much money AND be on budget.

And what do we really mean by "on budget." Does that mean we delivered the stated capability using the original amount of funding? Or are we taking credit for staying under a budget line that has increased over the years, as additional funds get added to the baseline (because we've increased the schedule, added requirements, etc)? If we define "on budget" broadly enough, then I'll bet almost everyone is on budget. Maybe even 95% of projects.

But there's more to this story. The idea that 95% of projects are on time & on budget doesn't actually mean as much as it might seem. Imagine a portfolio of 10 projects. Nine of them each cost $100, and each deliver on time, on budget. The 10th project was supposed to cost $100,000, but ended up coming in at $150,000. Do we praise this portfolio for being "90% on budget," or do we acknowledge it spent $50K more than planned? I'm thinking the latter assessment is more accurate.

The flaw in the 95% on budget reasoning is that it is measuring performance on a per-attempt basis, using a straight count of the number of projects. What we should be measuring is the performance of our dollars (i.e. the proverbial bang for the buck).

Similarly, let's say we have a project that has 10 steps. Nine of the steps each take one day to accomplish. The 10th step takes a year. At the end of nine days, if we say we're 90% done with the project, because we have finished nine of the steps, we're seriously misrepresenting how much work has been done (and how much is remaining).

Statistics are funny things. It's entirely possible for a statistic to be true and misleading all at once. It's important to be careful and make sure the things we're measuring and reporting provide an accurate representation of the situation.

Windtunnel?

I recently came across the idea of an "acquisition wind tunnel" that would allow us to test our organizations and projects, to see how streamlined and efficient they are.

I like that imagery... I think. At the same time, I wonder if some of the points of friction and resistance might not be some of the more interesting and productive parts of the project. I guess it depends on how we define friction.

It occurs to me that one person's friction is another person's traction. Similarly, one person's momentum is another person's inertia. So, in this hypothetical wind tunnel, we'd have to be specific about how we distinguish between negative friction and positive traction. And the tricky thing is, the line between friction and traction is probably pretty fuzzy (and it probably moves).

This wind tunnel concept no doubt could be implemented under a Theory of Constraints sort of framework. No doubt the Lean crowd would find that this idea resonates with them. And frankly that makes me hesitant about the whole thing.

At the same time, I do like the idea of setting up some well-defined streams of "wind" that we could point at a project, organization, etc, to assess its flight worthiness. A FIST-based wind tunnel could have real value in identifying opportunities to unleash the creative power of constraints and restraint.

Values

I haven't talked about the topic of values in a while, and after some lively discussions with B. Smitty, I figured I'd spend a little time on the topic again, because assumptions about values are at the heart of most of the problems in project leadership (and a lot of the disagreements too).

The values I have in mind are "the things we think are important." In engineering speak, values are measures of merit. They are the signs of sophistication and other desirable attributes (and the context I have in mind is organizations, technologies and processes). This stuff matters because our values shape our objectives, and at the end of the day, values are the yardstick we use to assess whether good things have happened or not.

In my FIST-related writings, I talk about values as the answer to the question "What is important and good?" and I often describe FIST as a values-based approach. It basically says "It is important and good to be Fast, Inexpensive, Simple and Tiny." There are other value sets out there, and mostly they go unstated, unexamined, unconsidered and untested. That's a HUGE problem.

See, if we value complexity (either explicitly or unconsciously), we'll make the system more complicated and think we've done something good - even if it's not actually any better. You see this a lot in PowerPoint presentations - people who think they are effective communicators because they included every word, every comma, every data point and every possible nuance of every diagram in their 800-chart presentation (for their 15 minute time slot). You also see this in over-engineered systems, chock-full of a million good ideas. I'm going to suggest these are examples of values out of whack. Overvaluing complexity drives unproductive behavior.

Similarly, if we think spending a lot of money guarantees quality, we'll feel reassured by a high price tag, even if a smaller price might have delivered better results, better quality. Same thing with time - if we value the slow-and-steady approach, the fact that it took 20 years to deliver a system gives us a warm feeling, even if such expenditures were unnecessary.

So, the questions are: what values drive your project? What values contribute to positive outcomes, and improved operational effectiveness? And to come full circle to the afore-mentioned discussion, when people praise the F-22 as "the most capable aircraft" in our inventory, what are they really praising? What values are driving that assessment? What basis is there for those values?

Frankly, this question doesn't come up often enough in program management circles, which is a bummer because I think it's at the core of the whole discipline.

The One Thing

If there was one thing I could accomplish professionally, one contribution to the corpus of program management practice and theory, it would be to dispell the Myth of Inevitability.

What is this myth, you ask? It's the idea that high tech system development projects (weapons, spacecraft, commercial products, etc) inevitably take a long time, cost a lot and are complex. The myth is expressed in phrases like "better, faster, cheaper - pick two." A corollary to this myth is the idea that adding time and money to the project improves the outcome, as if the problem with our failed system development efforts was that our schedule was too short and our budget too small.

No kidding -that's what they said when the 18 year, $7B (billion-with-a-b) Comanche helicopter was cancelled. They needed more money. They needed more time. Yeah, that would have helped. Right. A similar chorus rises from any number of failed high tech projects. My assessment is that they had too much time & money, not too little. And at the core, they believed that the costs and delays were inevitable, simply an inherent part of this kind of work. It's a tragic belief and a self-fulfilling prophecy. It's also not grounded in reality.

The truth is, there is nothing inherent in military technology (for example) that requires it to cost so much, take so long or be so complex. Yes, systems like the F-22 took a long time and cost a lot. But there's a difference between "it took a long time" and "it takes a long time." We could have done it better (both programmatically and operationally). If we set up our values correctly, if we cherish our constraints and pursue intelligent simplicity, we don't have to spend billions and decades. We can do it for millions and in years (or thousands in months). The F-117, the SR-71, NASA's Near Earth Asteroid Rendezvous (NEAR) mission and the Pathfinder Mars mission are all examples of high-tech projects built on a FIST (Fast, Inexpensive, Simple, Tiny) foundation.

Despite the evidence of a significant portfolio of FISTy projects (documented in my master's degree thesis), the Myth of Inevitability insists that things like this have to be complicated and expensive. Believers in the myth see no alternative to bureaucracies, technologies and processes that are complex, expensive and slow.

Their failure to see through the myth reflects ignorance of the past and a lack of imagination. There are alternatives. Download a free copy of The FIST Handbook for more details on the principles, activities and examples of how to use constraints to foster creativity and deliver systems that are simultaneously faster, better and cheaper.

Twitter Is Mission Critical

Gabe & I collaborated with our AT&L editor Carol on a fun little article titled Twitter Is Mission Critical. It'll be in the Oct issue of Signal magazine, and you can read a short preview here. We'll let you know when the full version is up.

New Wars, New Friend

A hearty Rogue Welcome to blogger Mike Burlson, who writes the New Wars blog! His blog is, as Gabe would say, awesome. Be sure to check it out.

Burlson's vision for the future Navy lines up very closely with my own imaginings of a future AF. Specifically, he's saying we should move away from small fleets of big, expensive, complex systems, and towards larger fleets of systems that are, to coin a phrase, fast-inexpensive-simple-tiny. Here's a short excerpt:

We contend here at New Wars that modern computer technology added to guided missiles has doomed the heavily armored, over-priced weaponry of the Cold War/WW 2 eras. With this in mind we could safely cut such complicated arms as the manned fighter, the heavy tank, and large surface warships. Their replacements would be unmanned aerial vehicles, light armored cars, plus submarines and light patrol ships.

Amen, brother!

Capabilities

Let's say I can hold my breath for two minutes. That would be an impressive capability.

However, being able to hold my breath that long would not make me a more capable writer or engineer - the two main areas of professional expression I'm currently engaged in. I would not include "can hold breath for 2 minutes" in a resume looking for an engineering or writing job -it's just not relevant to those tasks.

So, when I hear people say that the F-22 Raptor is "the most capable aircraft ever," I have to object and ask what they mean by "most capable."

When I assess a system's capability, I'm looking at its ability to contribute to the fight. Since 2001, any system that doesn't provide capabilities we need in Iraq or Afghanistan is, by my definition, not very capable. And that's precisely the case with the Raptor. It first went operational in 2005, and it has yet to fly a combat mission in either war. It's not bringing anything (a-n-y-t-h-i-n-g) to the fight. In what sense is it, therefore, the "most capable" jet around?

To put it plainly, the F-22 does things we don't currently need to do. We may need to do them someday, although I think that's not as likely as some people do. But it is a demonstrable fact that we don't need these capabilities today... or any time soon. The SECDEF himself said the Raptor is "irrelevant" to the fights in Iraq and Afghanistan. To my mind, that makes it one of the least capable aircraft in the inventory (and that's not even factoring in its maintenance issues, low availability rates, etc). Combine that with the $65B we've spent on the thing and you can see why we've decided to not buy any more of them.

So yeah, it can hold its breath for a long time, but that capability doesn't line up with the near- or mid-term needs. What we need are aircraft that are capable of accomplishing the mission. The Raptor clearly is not. I'm not saying we should scrap the whole fleet. I'm just saying we should stop trying to paint it as the "most capable" system we've got.

Catching Up With BRITE

One of the first projects I really had an impact on was a little imagery dissemination project for the National Geospatial-Intelligence Agency called BRITE (originally Broadcast-Request Imagery Technology Experiment... I think they changed the E to something else later). It was a small system designed to provide overhead imagery to forward deployed SpecOps guys who had very limited bandwidth and were highly mobile.

The BRITE project was one of the keys to developing my FIST approach to technology development. We had a very small team - in fact, I was the only government guy working on the project (and I was a junior Captain). On the contractor side, we just had a handful of people, and most of them were only assigned to the project part time, as I recall. The budget was quite small, the deployment schedule was short (this was 2001 - 2002). The system was designed to operate in austere conditions and it worked like magic.

That experience had an enormous impact on my thinking and my perspective on what can be done with small teams of talented people, working on tight budgets and tight schedules. It also hammered home the importance of simplicity - organizationally, technically and procedurally.

So, just for fun I googled it and came up with a great story about using BRITE to support the Hurricane Katrina disaster relief effort. It's nice to see that my little project still had legs after I'd moved on to other things.

Congress Redux

One other comment while we're talking about Congress.

Representatives and Senators are elected to represent and protect the interests of their constituencies. That's their job, and we should not complain and object when they do what they were elected to do. The problems in military acquisitions are not Congress' fault.

See, when we deliberately spread out development of a project across 44 states and several hundred congressional districts, we are making a cynical move that's designed to ensure the project can't be cancelled. We can't then turn around and object that those doggone people in Congress are forcing us to do something. If we built smaller, more focused projects, we'd get a lot less Congressional involvement.

Similarly, when we launch a hugely expensive project, it is entirely appropriate for Congress to insist on performing oversight. It's a lot of money, and they owe it to their constituencies to make sure it's spent appropriately. If we spent less money, we'd get less Congressional involvement.

The FIST approach is a good way to reduce intrusive oversight without denying legislators the right and ability to perform their legitimate, constitutionally appointed roles.

Not Easily Imitated

Ever since I wrote the nightmare fiction story Acquisition As Deterrent, I've been a little bit haunted by the thought that it might be true. In fact, when the idea first hit me, it came down on my head like a ton of bricks and sorta bummed me out.

What if the reason for all the complexity, cost and delay in military technology projects is to prevent the rest of the world from imitating us?

Well, that still might be the case, but if so, it's a bad reason. The thing is, the FIST approach is not easily imitated, because the US has such direct access to a wide range of mature, advanced technology, compliments of our various military laboratories, etc.

So, the hostile dude in some other part of the world who wants to use the FIST approach is starting off with a significant disadvantage, in terms of the pieces and parts available to use. Yeah, the barriers to accessing and adopting technology are lowering, but we've still got a significant advantage.

And then there's the whole personnel part, the training, talent, education, courage and creativity of the people who actually develop and use these systems. That's pretty hard to imitate.

FIST Video!

Yesterday's post was the short version of the FIST (Fast, Inexpensive, Simple, Tiny) approach to system development.

Today, I've got a short post with a link to a long video. Yes, this is the much anticipated Rogue-a-palooza aka Rogue Fest 2009 aka the briefing that Gabe and I did at the Defense Acquisition University back in July.

I confess I haven't actually watched the whole thing yet, but my mom and dad did and they said I did real good. They also said the video includes the Q&A at the end of the presentaiton - I wasn't sure if that part would be included, but it was. That's cool.

So, if you've got an hour or so to kill, and you want to see me and Gabe jump around like monkeys and tell stories about why FIST is so good, click the link.

TSPR, Trust & Strength

In the late 90's, before I really had any idea what was going on in the world of defense acquisitions, some in the DoD tried an approach called Total System Performance Responsibility (TSPR). It came and went largely unnoticed by little ole me. Now, like Total Quality Management and other past attempts at making things better, TSPR has become a bad word among military technology circles.

Since people are still throwing spears at it, I figured I should know more about it than I do. It turns out, the basic idea was to reduce the government's involvement and allow contractors to do things their own way, as efficiently as possible, without undue influence or involvement by the government.

Maj Henry Pandes wrote a pretty good article about it, if you want to read more. James Gill wrote a short response to Maj Pandes' article. Also a good read.

Unfortunately, many people today seem to have decided that the problem with TSPR was that it involved too much trust. Trust is bad, they conclude. We can't trust those slimy contractors. They're bad, they're bad!

Hold on. Let's not get the wrong conclusion here.

After a bit more investigation and reflection, it seems to me that the problem with TSPR wasn't trust, but rather ignorance. Instead of reasonable delegation, we turned it into abdication. The government wasn't merely hands-off, it was eyes-shut. Anyone surprised that didn't work out very well?

The truth is, I think the government should trust contractors more than we do (as I've written elsewhere). But that doesn't mean we should take the "wake me up when it's ready" approach. We can still be involved and informed, without turning it into excessive meddling.

More to the point, trust isn't weakness, naivete and stupidity. Trust requires strength, judgment and wisdom. A lack of trust indicates, among other things, a lack of trustworthiness and a significant degree of unreflective foolishness. Yeah, I said it - foolishness. Real rogues trust their partners, 'cause that's a smart and strong thing to do.

The TSPR approach may have been fatally flawed from the start, or it may have been a good idea badly executed. I still don't know. But what I do know is that if we think the lesson of TSPR's failure is to not trust people, we've learned a horribly wrong lesson.

Cancelling Projects

From a justice perspective, it's important for an accused person to have a good lawyer. Even if the defense attorney thinks the client is guilty, they are still supposed to provide a robust defense and make the strongest case possible.

Unfortunately, program managers often act like defense attorneys for their projects. They sometimes act as if their job is to keep the project alive, and defend it to all comers... even if the project is a dog and needs to be cancelled.

The program manager (arguably) knows the most about the health and viability of the program. If it's heading over a cliff, aiming for irrelevance or otherwise doomed, the PM is usually among the first to know, and should speak up. Sadly, this does not always happen.

PM's are not defense attorneys. Their job is not to keep the program alive and out of jail. It's to guide and shepherd the development. And if the program needs to be cancelled, the PM should lead the charge.

Twitter Is Mission Critical

I've been pushing for greater adaption of social media (and other Web 2.0 stuff) for a while now. I recently came across the CogBlog, which had this great comment on the US military's current approach to computer network security and operations.

To fight kinetic wars, the military gives its soldiers armor and modern weapons and trains them to exploit the native battlefield infrastructure and terrain. In contrast, to fight information wars, the military attempts to encase its soldiers in impenetrable bubbles that insulate them from the “information battlefield.” The analogy in kinetic space would be to encase our soldiers their vehicles and equipment in giant “hamster balls” and have them roll as best they can across the native infrastructure and terrain.

Exactly! When we decide to block off the most important and interesting parts of the internet, not allowing military types to use them to accomplish the mission, we're basically putting people in hamster balls, which doesn't exactly help.

FIST Handbook - Updated

Team Rogue and I put together an updated version of the FIST Handbook (pun intended - sorry!), which collects the best of our articles from Defense AT&L.

Unfortunately, the file format has some incompatabilities with Lulu, so the version you can download on RoguePress is out of date. The new version is 12Mb, and rather than try to email it, I stuck it on Rapid Share. You can download your own very version right here.

Enjoy!

UAV Pizza Delivery!

Looks like we're getting closer to my vision of UAV's delivering coffee, pizza and Chinese food. A report from Inside The Air Force recently announced that officials at Rockwell Collins project commercial unmanned aircraft being approved as early as 2012.

UAV pizza delivery - I'm telling you, it's coming! (and I love the phrase "groundswell of non-naysayers").

* * *
Inside the Air Force
August 14, 2009

ROCKWELL: INDUSTRY AIMING TO HAVE UAVS IN COMMERCIAL AIRSPACE BY 2012

Officials at Rockwell Collins see unmanned and manned aircraft sharing airspace across the globe by as early as 2012, one of the company's top managers for unmanned aerial vehicle control technologies announced this week.

While presenting numerous UAV control technologies being pioneered by the company, Dave Vos, Rockwell's senior manager for unmanned aerial system control technology, claimed that industry is quickly building a "groundswell of non-naysayers" who will work out the challenges associated with flying drones in the same airspace as manned aircraft on a routine basis.

The "2012 is the time line that we're all looking to, to allow and enable anyone who has a new business case and has managed to raise venture money . . . to go and exploit putting unmanned systems in commercial airspace," said Vos during an Aug. 12 briefing at an unmanned vehicle conference in Washington.

SOCOM Truths

A former classmate of mine is about to deploy with the Spec Ops guys, and he just sent me an excerpt from a briefing about SOCOM acquisitions. It contained the following truths:

FAST Requires MORE DISCIPLINE
RISK Must Be Managed NOT AVOIDED
FASTER Does Not Have To Increase COST/RISK
COMPETITION Can Be Done QUICKLY
UNCONVENTIONAL THINKING Is An ENABLER
CREDIBILITY Enables FREEDOM OF ACTION

I've always said that SOCOM is one of the shining stars in the defense acquisition community. Their track record of delivering on time, on schedule, AND being operationally effective is quite impressive. Embracing FISTy truths like these probably has a lot to do with that.

UAV's: Imagine The Possibilities

In the Jul 09 issue of National Defense magazine, Col Eric Mathewson, director of the Air Force's Unmanned Aerial Systems Task Force discusses some of the things UAV's will never do: "There's no way you can replace an F-35 or F-22 or something like that... No way."

In related news, "Lt. Gen. Dave Deptula, the Air Staff's head of intelligence, surveillance, and reconnaissance, said it is not unrealistic to imagine an unmanned aerial vehicle as successor to fifth-generation fighters-the F-22 and yet-to-field F-35-in our national military strategy, however he added that UAVs still require some technology maturation before they would be ready to assume that mantle. (AF Magazine Daily Report, 27 Jul 09 )

Which reminds me of a quote I heard somewhere, that went something like this: When the esteemed and aged expert says a thing can be done, he is most certainly right. When he says it cannot be done, he is most certainly wrong. (can anyone help with that reference?)

Colbert Speaks Our Language

To follow up yesterday's post, here's Dr. Stephen Colbert with the best commentary on the F-22 yet!

The Colbert Report	Mon - Thurs 11:30pm / 10:30c
ThreatDown - Henry Louis Gates Jr., Bill Gates & Wilford Brimley
http://www.colbertnation.com/
Colbert Report Full Episodes Political Humor Mark Sanford

Yeah buddy!