About eight months ago the GAO issued a report concerning body armor (SAPI plate) recall, and outlined a number of findings concerning the testing the Army had performed.  We summarized a few of the findings in DoD Testing Requirements for Body Armor and Army Recall.

COPD is “Contract Purchase Description,” PEO is “Program Executive Officer,” and BFD means “Back Face Deformation.”  This last concept becomes important in the overall picture.  Turning to the specifics of the report, several key findings are outlined below for the purpose of providing examples of the investigation.

The inconsistencies that we identified concerned the treatment of over velocity shots.  During first article testing conducted on February 20 and November 7, 2007, shots on six of the plates were over the required velocity. Because none of the shots resulted in a complete penetration, the shots should have been considered fair, and the test should have proceeded, according to the COPD. During the November 7, 2007, test, the testing facility official complied with the COPD and correctly proceeded with testing. However, even though the scenario was exactly the same for the February 20, 2007, test, the testing facility official conducted retests on additional plates. The testing facility official documented all of the shots, including the retests, and provided the test results to PEO Soldier for scoring.  When scoring the test results for the February 20, 2007, first article test (design M3D2S2), the PEO Soldier scoring official chose to use the test results for the retested plates when he computed the test score. Use of the retested plates resulted in a score of 5.5 points, and the contractor passed the first article test. Had the scoring official followed the fair shot acceptance criteria as stated in the COPD and used the initial plates that withstood the over velocity shot, the contractor would have accumulated an additional 1.5 points (complete penetration on the second shot) and would have failed the first article test with 7 points.

Translation: When an over-velocity shot is taken on a plate, the testing may proceed if the plate is not penetrated under the assumption that a lower velocity shot would not have penetrated either.  This is a reasonable assumption.  However, if the plate is penetrated by the second shot it fails the testing, even if weakened by the initial shot.  The PEO made the decision to exclude the plates that had sustained over-velocity shots on the initial testing and to perform retests, but not consistently (as later records show).  A second example of the Inspector General’s findings pertains to measurements of BFD (back face deformation).

PEO Soldier instructed the testing facility to deviate from the COPD and use an offset correction technique (a mathematical formula used to adjust the BFD) when measuring the BFD. The testing facility official used this technique during 2 of the 21 first article tests conducted under Contract 0040. The COPD required that the testing facility officials measure the BFD at the deepest point in the clay depression after the bullet impacted the plate. However, PEO Soldier officials stated that contractors complained that the BFD measurement was not fair if the deepest point in the clay was not behind the point of impact. Therefore, a PEO Soldier official instructed the testing facility in an April 25, 2005, e-mail to use the offset correction technique if the deepest point in the clay depression was not behind the bullet’s point of impact.

Translation: The contractors complained when the measurement of deepest penetration was made at any point other than the point of bullet impact, which is the point of highest risk to the Soldier.  Therefore, the PEO made a decision that a correction would be applied to account for this effect and bring consistency to the program.

The Captain’s Journal initially concurs with both of the program deviations discussed above, since it isn’t fair to penalize one plate as compared to another if an over-velocity shot happened to be taken against it, and also since the highest risk to the Soldier does happen to be the point of bullet impact.

And it is also fair to point out that these aren’t the only problems discussed in the report.  But there are deeper problems that discussed even in the report.  With respect to the over-velocity shots, our judgment is that not enough SAPI plates are being included in the test samples (i.e., the sample size is not large enough) and the boundary conditions (such as shot velocity) are not being well-managed.  With respect to the deformation, the question naturally arises why the most severe deformation is occurring anywhere other than the point of bullet impact?  What’s happening to the ESAPI plates that is causing deformation in other than impact locations?

These questions (and other such technical questions) are not posed or answered in the Inspector General’s report, since the investigation is done by a government office.  The investigation focuses on programs, QA, adherence to procedures, consistency of application of rules and the like.  True enough, there are problems with some of the above.

But Senators and Representatives who have infinite trust in the power of government to solve problems leave the technology to the experts when a government office is the the sole arbiter of the strength of any technical program – and technological expert doesn’t usually define government offices.  In this particular case, as we have suggested before, there is no shame in assistance from industry experts.

Questions have been raised above which point to the need for completely independent consultative services focusing on QA, programmatic controls, statistical analysis of sample size, control over testing boundary conditions, and most especially the SAPI plates themselves and the underlying fracture mechanics of bullet impacts by finite element analysis.

At this point the business of body armor investigations wasn’t complete at the Government Accountability Office.  Hence, in October 2009 they issued Warfighter Support: Independent Expert Assessment of Army Body Armor Test Results and Procedures Needed Before Fielding.  In the executive summary they state:

To determine what effect, if any, the problems GAO observed had on the test data and on the outcomes of First Article Testing, the Army should provide for an independent ballistics evaluation of the First Article Testing results by ballistics and statistical experts external to the Department of Defense before any armor is fielded to soldiers under this contract solicitation. Because DOD did not concur with this recommendation, GAO added a matter for congressional consideration to this report suggesting that Congress direct DOD to either conduct such an independent external review of these test results or repeat First Article Testing.

To better align actual test practices with established testing protocols during future body armor testing, the Army should assess the need to change its test procedures based on the outcome of the independent experts’ review and document these and all other key decisions made to clarify or change the testing protocols during future body armor testing. Although DOD did not agree that an independent expert review of test results was needed, DOD stated it will address protocol discrepancies identified by GAO as it develops standardized testing protocols. DOD also agreed to document all decisions made to clarify or change testing protocols.

To improve internal controls over the integrity and reliability of test data for future testing as well as provide for consistent test conditions and comparable data among tests, the Army should provide for an independent external peer review of Aberdeen Test Center’s body armor testing protocols, facilities, and instrumentation to ensure that proper internal controls and sound management practices are in place. DOD generally concurred with this recommendation, but stated that it will also include DOD members on the review team.

Consistent with our own recommendations, they counsel in the strongest possible terms that outside independent consultative support be obtained.  But as soon as the GAO released its report, the DoD released a statement claiming confidence in the safety of the SAPI plates – a completely irrelevant rejoinder to the overall recommendations of the GAO report to procure consultative support for the program.  The same day that the DoD announced that they had full confidence in their body armor tests, they announced several new QA positions concerning ballistics and body armor testing.

The Army is sounding defensive and unwilling to open their program to outside expert inspection and assessment.  Here at The Captain’s Journal we haven’t recommended draconian measures such as jettisoning the Army test program, or complete replacement of the SAPI (at least until an equivalent, lighter weight ballistic insert can be developed).  We have only recommended the engagement of outside consultative services for the Army, just as did the GAO.

For the Army to reject that recommendation is very small and in extremely bad form.  When counsel has been given to open your programs to outside inspection and that counsel is rejected, it constitutes poor engineering.  There are many industries which “live in a glass house,” so to speak: nuclear, commercial air transport, pharmaceutical and medical, just to mention a few.  There is no valid technical or budgetary reason whatsoever that the Army cannot open their program to inspection by people who know as much or more than they do.