NBAF Risk Assessment is a Theory With Limited Predictive Value
Posted by tmanney on March 8, 2012
Our Kansas congressional delegation has hailed the recently released updated Site Specific Risk Assessment for the proposed National Bio and Agro-Defense Facility (NBAF) as a confirmation of its safety. As stated on Senator Moran’s web site, “I am pleased by the updated assessment’s confirmation of what we always knew, Manhattan is the safest and best place for this facility to be built,” Sen. Roberts said. “The de minimis risk of outbreak should help allay concerns from any opposition, in the public or Congress.”
So, as a vocal member of that opposition, I wish to share my thoughts on the updated NBAF risk assessment for release of foot-and-mouth disease virus, or any of the NBAF germs that are actually deadly to humans. After studying the ca. 1000-page statistical modeling analysis, I cannot question its findings; it seems to be a legitimate response to Congress’s mandate. Colleagues more authoritative in their knowledge of probability theory and statistical analysis agree. But I do question and doubt the legitimacy of incomplete, uninformed, and apparently disingenuous public interpretation of the report’s findings by the administrators and politicians who seem singularly focused on bringing this facility to Manhattan at any cost. KSU officials and the Kansas Congressional delegation, as reported nation wide, have dwelled almost exclusively on the risk of an FMD outbreak during the NBAF’s 50-year operating life being down from 70 percent to 0.11 percent.
But these interpretations have ignored 1) the caveat stated repeatedly throughout the report that the calculated probabilities cannot be taken as absolute values, 2) that statistical models are only theories and are no more accurate in predicting actual events than the assumptions that are plugged into them, and 3) the uncertainty of many of the input assumptions, and therefore of the conclusion, are extremely large. These too are clearly stated in the report.
For example, on page 403: “it is fundamentally a modeling‐based approach and therefore has limited ability to predict the absolute probability of an outbreak occurring and the corresponding consequences.” Statistical analysis is highly technical, intensely mathematical, and universally mysterious to ordinary people. I have yet to read, even in the DHS report itself, an intelligible statement of what a risk of 0.11 percent actually means in the real world. Yet Senator Pat Roberts has declared that the risk is “de minimis”, which is even more obscure. One highly knowledgeable scientist explains that it means, for example, that if 10,000 identical NBAFs were built, then 10 of them would be certain to release FMD viruses over their 50-yr life times. But as with all such probability estimates, it predicts absolutely nothing about any single case – a single NBAF. It is just a theory. What is worse is that the problem of understanding what a probability of 0.11 percent means in real life carries over to understanding what the uncertainty of such a value means. The same computer model that calculated the value of 0.11 percent also calculated that there is a 95 percent chance that it could be as high as 2.5 percent. What does that mean to you? De minimis?
Perhaps in an attempt to help ordinary people understand these obscure, theoretical concepts, the authors converted from percent to dollars based on the economic consequences of an FMD release. In these units they conclude, “The uncertainty (standard deviation) in the 50-year cumulative risk was found to be approximately $15B, regardless of whether catastrophic events are included.” To me, this says the economic risk is somewhere between 0 and $15 Billion. De minimis?
We hope that the National Research Council commission has a more realistic grasp of limits of a statistical model based on limited and uncertain input data.
Professor Emeritus, Physics & Biology
Kansas State University
Sorry, the comment form is closed at this time.