No NBAF in Kansas

Real Biosecurity for the Heartland

Archive for the ‘“site-specific biosafety and biosecurity risk assessment (SSRA)’ Category

NBAF Risk Assessment is a Theory With Limited Predictive Value

Posted by tmanney on March 8, 2012


Our Kansas congressional delegation has hailed the recently released updated Site Specific Risk Assessment for the proposed National Bio and Agro-Defense Facility (NBAF) as a confirmation of its safety.  As stated on Senator Moran’s web site, “I am pleased by the updated assessment’s confirmation of what we always knew, Manhattan is the safest and best place for this facility to be built,” Sen. Roberts said. “The de minimis risk of outbreak should help allay concerns from any opposition, in the public or Congress.”

So, as a vocal member of that opposition, I wish to share my thoughts on the updated NBAF risk assessment for release of foot-and-mouth disease virus, or any of the NBAF germs that are actually deadly to humans.  After studying the ca. 1000-page statistical modeling analysis, I cannot question its findings; it seems to be a legitimate response to Congress’s mandate. Colleagues more authoritative in their knowledge of probability theory and statistical analysis agree. But I do question and doubt the legitimacy of incomplete, uninformed, and apparently disingenuous public interpretation of the report’s findings by the administrators and politicians who seem singularly focused on bringing this facility to Manhattan at any cost.  KSU officials and the Kansas Congressional delegation, as reported nation wide, have dwelled almost exclusively on the risk of an FMD outbreak during the NBAF’s 50-year operating life being down from 70 percent to 0.11 percent.

But these interpretations have ignored 1) the caveat stated repeatedly throughout the report that the calculated probabilities cannot be taken as absolute values, 2) that statistical models are only theories and are no more accurate in predicting actual events than the assumptions that are plugged into them, and 3) the uncertainty of many of the input assumptions, and therefore of the conclusion, are extremely large.  These too are clearly stated in the report.

For example, on page 403: “it is fundamentally a modeling‐based approach and therefore has limited ability to predict the absolute probability of an outbreak occurring and the corresponding consequences.”  Statistical analysis is highly technical, intensely mathematical, and universally mysterious to ordinary people.  I have yet to read, even in the DHS report itself, an intelligible statement of what a risk of 0.11 percent actually means in the real world. Yet Senator Pat Roberts has declared that the risk is “de minimis”, which is even more obscure. One highly knowledgeable scientist explains that it means, for example, that if 10,000 identical NBAFs were built, then 10 of them would be certain to release FMD viruses over their 50-yr life times.  But as with all such probability estimates, it predicts absolutely nothing about any single case – a single NBAF.  It is just a theory.  What is worse is that the problem of understanding what a probability of 0.11 percent means in real life carries over to understanding what the uncertainty of such a value means.  The same computer model that calculated the value of 0.11 percent also calculated that there is a 95 percent chance that it could be as high as 2.5 percent.  What does that mean to you?  De minimis?

Perhaps in an attempt to help ordinary people understand these obscure, theoretical concepts, the authors converted from percent to dollars based on the economic consequences of an FMD release.  In these units they conclude, “The uncertainty (standard deviation) in the 50-year cumulative risk was found to be approximately $15B, regardless of whether catastrophic events are included.”  To me, this says the economic risk is somewhere between 0 and $15 Billion.  De minimis?

We hope that the National Research Council commission has a more realistic grasp of limits of a statistical model based on limited and uncertain input data.

Tom Manney

Professor Emeritus, Physics & Biology

Kansas State University

Posted in "site-specific biosafety and biosecurity risk assessment (SSRA), Uncategorized | Tagged: , , | Comments Off on NBAF Risk Assessment is a Theory With Limited Predictive Value

NRC Report Adds to Doubts About NBAF Safety

Posted by tmanney on December 3, 2010

More than two years ago the Governmental Accountability Office (GAO) questioned the safety of doing Foot and Mouth Disease research on the U. S. mainland.  The motivated our opposition to the selection of Manhattan, or any other mainland location for the NBAF. It subsequently motivated Congress to require the Department of Homeland Security (DHS) to conduct a Site Specific Risk Assessment (SSRA) and to have an independent evaluation of that Assessment to determine its adequacy and validity. That Assessment was conducted by a Committee of the National Research Council, a subsidiary of the National Academy of Science.

NBAF Risk Assessment Report

The text of the 146 page report can be purchased or read online at http://nap.edu/catalog.php?record_id=13031

We have abstracted the major findings here:

OVERALL ASSESSMENT

The committee evaluated the SSRA’s methods, facility design plans, and mitigation strategies. The committee found that the models used in performing the SSRA appear to be appropriate and that many of the SSRA’s general conclusions are valid. The SSRA has considered the major release pathways (aerosols, fomites, liquid waste, and solid waste), as recommended in the committee’s preliminary letter report (see Appendix B), and has addressed mitigation strategies for each. DHS has also appropriately responded to GAO’s prior criticism that it had inappropriately dealt with a potential plume from an airborne release of foot-and- mouth disease virus (FMDv); the SSRA uses a state-of-the-art puff dispersion model to simulate the aerosol transport of pathogens, which turned out to be a less critical pathway of FMDv spread than the near-site exposure of cattle. However, as described in the findings below, the committee found that the SSRA had several major shortcomings with respect to potential risks and impact scenarios, and there are some critical limitations in the SSRA’s execution and analysis.

The committee found that the SSRA has many legitimate conclusions, but the SSRA is not entirely adequate or valid. The SSRA does not account for the overall risks associated with operating the NBAF and conducting FMDv work in Manhattan, Kansas. The inputs and assumptions for the models are inadequate because they do not fully account for how a biosafety level 3 agriculture (BSL-3Ag) and BSL-4 facility would operate, how pathogens might be released, and which animal populations might be exposed. The SSRA sometimes used arbitrary assumptions and did not account for uncertainties, some of which require experimental data that are currently not available but that could greatly alter the outputs. Consequently, the committee is concerned about the validity of the actual risk and impact levels determined by the SSRA’s outcomes from the models.

Given more time, the SSRA may have progressed further and may have better addressed some of the concerns expressed in this report. The committee thus views this as a notable first step in an iterative process aimed at identifying and minimizing risk and determining actions that will need to be taken.

FINDINGS

The SSRA shows that constructing the NBAF in Manhattan, Kansas, carries a number of risks and that the impact of an FMDv release could potentially have significant economic, health, and national security impacts. Some risks and impacts are generic to any high-containment large- animal facility, whereas others are specific to the Manhattan, Kansas, site. The risk of release is primarily a generic concern, whereas the risk of infection, spread, and impact is largely related to the site. The SSRA’s estimates indicate that the probability of an infection resulting from a laboratory release of FMDv from the NBAF in Manhattan, Kansas approaches 70% over 50 years (see Figure 3-1) with an economic impact of $9-50 billion. The committee finds that the risks and costs could well be significantly higher than that, and elaborates on those findings below.

Finding 1: The SSRA lacks evidence to support the conclusion that the risk of release that results in infection is very low relative to the risk of infection introduced from an external source.

Finding 2: The SSRA overlooks some critical issues, both site-specific and non-site-specific, that could significantly elevate the risk of accidental release and spread of pathogens.

Finding 3: The SSRA has several methodological flaws related to dispersion modeling, tornado assessment, and epidemiological modeling. Thus the committee believes that questions remain about the validity of the overall risk estimates. Finding 3: The SSRA has several methodological flaws related to dispersion modeling, tornado assessment, and epidemiological modeling. Thus the committee believes that questions remain about the validity of the overall risk estimates.

Finding 4: The committee agrees with the SSRA’s conclusion that for FMDv, long-distance plume transport will likely be less important than the near-site exposure of cattle.

Finding 5: Substantial gaps in knowledge make predicting the course of an FMD outbreak very difficult, which led to weaknesses in the SSRA.

Finding 6: Although the economic modeling was conducted with appropriate methods, the epidemiological estimates used as inputs to the SSRA were flawed.

Finding 7: The committee agrees with the SSRA’s conclusion that early detection and rapid response can limit the impact of an FMDv release from the NBAF, but is concerned that the SSRA does not describe how the NBAF could rapidly detect such a release.

Finding 8: The SSRA lacks a comprehensive mitigation strategy developed with stakeholder input for addressing major issues related to a pathogen release. The mitigation strategies that are provided do not realistically demonstrate current or foreseen capacity for how federal, state, and local authorities would effectively respond to and control a pathogen release.

Finding 9: The committee agrees with the SSRA’s conclusion that human error will be the most likely cause of an accidental pathogen release, and fomite carriage is the most likely way that a pathogen would escape the facility’s outer biocontainment and biosecurity envelope.

Finding 10: The committee agrees with the SSRA’s conclusion that investment in biosafety and biosecurity engineering and the training of personnel and responders can reduce the risks, but is concerned about current design plans that potentially compromise safety measures.

Finding 11: The SSRA’s qualitative risk assessment of work with BSL-4 pathogens in large animals was inadequate.

ADDITIONAL REMARKS

The SSRA team should be applauded for its effort in conducting an extensive risk assessment in such a short period of time. Although the committee’s findings express major concerns about the validity of some of the SSRA’s conclusions, the work that was completed constitutes a huge step forward compared with previous risk assessments of its kind and should be viewed as a solid starting point.

The nation clearly needs an institution to support comprehensive research programs for the study of foreign animal and zoonotic diseases, including detection, diagnosis, and means of mitigation (drugs, vaccines, and genomic forensics). Such activities require a capability to work with all known threat agents (not just the eight infectious agents listed in the SSRA), multiple pathogen introductions, and emerging and unknown disease threats. For these reasons, the committee agrees that there is a need for a facility like the NBAF to be constructed and operated in the United States.

Constructing a BSL-3Ag and BSL-4 facility of the magnitude planned for the NBAF, one that is capable of large animal work on a scale greater than other high-containment laboratories, undoubtedly presents new and unknown risks that could not be accounted for in the SSRA because of a lack of data and experience. Given the constraints of the design framework and the short timeframe available for data collection and analysis, the committee finds that the limitations of the data, facility design details, and operating practices may have limited the scope that the SSRA could adequately address at this time. As more data, facility designs, and operational plans emerge, updated analyses may be appropriate to better evaluate the risks posed by a BSL-3Ag and BSL-4 large animal facility in Manhattan, Kansas.

The SSRA and the committee identify some sources of risk that can be addressed as part of the design, preparation, and long-term operation of the NBAF to reduce risk wherever it is located. Though the SSRA and the committee offer several points for consideration to reduce the risk of a pathogen release and its consequences, further risk analysis is needed to determine the extent to which these measures would reduce risk. Ultimately, policymakers will need to decide whether the risks are acceptable related to constructing and operating the NBAF in Manhattan, Kansas, and DHS will need to determine steps to minimize risk and impact if construction and operation should proceed as planned.

CONCLUDING REMARKS

The SSRA team should be applauded for its effort in conducting an extensive risk assessment in such a short period of time. Although the committee’s findings express major concerns about the validity of some of the SSRA’s conclusions, the work that was completed constitutes a huge step forward compared with previous risk assessments of its kind and should be viewed as a solid starting point.

The nation clearly needs an institution to support comprehensive research programs for the study of foreign animal and zoonotic diseases, including detection, diagnosis, and means of mitigation (drugs, vaccines, and genomic forensics). Such activities require a capability to work with all known threat agents (not just the eight infectious agents listed in the SSRA), multiple pathogen introductions, and emerging and unknown disease threats. For these reasons, the committee agrees that there is a need for a facility like the NBAF to be constructed and operated in the United States.

Constructing a BSL-3Ag and BSL-4 facility of the magnitude planned for the NBAF, one that is capable of large animal work on a scale greater than other high-containment laboratories, undoubtedly presents new and unknown risks that could not be accounted for in the SSRA because of a lack of data and experience. Given the constraints of the design framework and the short timeframe available for data collection and analysis, the committee finds that the limitations of the data, facility design details, and operating practices may have limited the scope that the SSRA could adequately address at this time. As more data, facility designs, and operational plans emerge, updated analyses may be appropriate to better evaluate the risks posed by a BSL-3Ag and BSL-4 large animal facility in Manhattan, Kansas.

The SSRA and the committee identify some sources of risk that can be addressed as part of the design, preparation, and long-term operation of the NBAF to reduce risk wherever it is located. Though the SSRA and the committee offer several points for consideration to reduce the risk of a pathogen release and its consequences, further risk analysis is needed to determine the extent to which these measures would reduce risk. Ultimately, policymakers will need to decide whether the risks are acceptable related to constructing and operating the NBAF in Manhattan, Kansas, and DHS will need to determine steps to minimize risk and impact if construction and operation should proceed as planned.

Posted in "site-specific biosafety and biosecurity risk assessment (SSRA) | Comments Off on NRC Report Adds to Doubts About NBAF Safety