By paul ~ August 29th, 2008. Filed under: Systems Engr..
Take a look at the following article from the Washington Post:
Here’s a key quote from the conclusion:
Defense Department officials have tried to improve the procurement process, the GAO said, by doing more planning and review in the early stages of a contract. But “these significant policy changes have not yet translated into best practices on individual programs,” Gene L. Dodaro, acting comptroller general of the GAO, wrote in the report.
“Flagship acquisitions, as well as many other top priorities in each of the services, continue to cost significantly more, take longer to produce, and deliver less than was promised,” Dodaro said. “This is likely to continue until the overall environment for weapon system acquisitions changes.”
One of the keys to risk reduction in complex, technology-heavy systems development is up front analysis. Everybody understands this and salutes it, but few really do it. As mentioned in an early article, the reality is that a defense contractor is really incentivized to “do it wrong” a couple of times before doing it right. This is at least somewhat due to the pressure to deliver demonstrate-able versions as quickly as possible, rushing the project through the critical requirements analysis and design phases. There is also an instinctive understanding that getting the next round of funding is easier if I can quickly deliver something tangible, even if it isn’t designed right. I can refine the design a number of times before delivery.
Unfortunately, this approach is extremely expensive in time and money, as any observer can easily witness. Modeling & Simulation provide a powerful mitigator to this problem when used religiously from the requirements phase on through the project. M & S not only improves the analysis process by flushing out the detail devils, but helps satisfy the strong engineering urge to have some tangible deliverable to interact with. Essentially, a “virtual prototype” is produced that stakeholders can interact with in any number of ways to determine whether requirements are met, performance will be as expected, etc.
Some will argue that models can be made to lie, and that is certainly true. However, a model (and it’s embedded assumptions) is analyzable (by demonstration, test, and inspection) to determine whether it can be considered credible. Contrast this with the hundreds of slides in presentation packages presented at design reviews which present just what the presenter wants the reviewer to see.
Sadly, it appears that simulatable models are only sporadically required as deliverables in government programs. This is baffling considering how valuable such models can be in evaluating proposals and system designs. Acquisition offices need to ensure that this valuable tool is written into RFPs and contracts for several reasons:
- The modeling and simulation process will greatly assist in the analysis and design of the system, resulting in better system designs.
- The resulting simulatable model will be extremely useful to the evaluators at various milestones.
- The resulting model will be owned by the customer and can become part of a knowledge-base that can be useful to client programs, future increments of the same system, and the development of similar systems.
Modeling and simulation can be an expensive activity for complex systems, but don’t we have enough data to convince ourselves that not doing it is even more expensive?