Systematic review specialists say a recent peer-review of EPA’s draft TSCA model for applying that tool to risk evaluations shows the agency still has not acted on what they say is repeated advice from expert panels for fundamental changes to its approach, spurring “frustration” with a process they say is slowing chemical rules crucial to protecting public health.
In an exclusive interview with Inside TSCA, Nick Chartres, associate director of science and policy at the University of California San Francisco’s Program on Reproductive Health and the Environment (PRHE), says the latest draft Toxic Substances Control Act (TSCA) model for systematic review repeats flaws from prior attempts to craft such a tool, even after EPA advisors and stakeholders warned the agency to address them.
“The frustration is they are supposed to be implementing one of the most important statutes, TSCA, using the best science and it's not being used,” he said.
Systematic review is a method of identifying, selecting and reviewing research for inclusion in analyses such as TSCA risk evaluations. But although academics have cheered EPA’s adoption of the discipline, they have been highly critical of the specific approaches it has crafted.
The agency abandoned its Trump-era model after a harsh National Academy of Sciences (NAS) peer review, and the Science Advisory Committee on Chemicals (SACC) recently finalized a review of the Biden administration’s draft replacement that urges a host of fixes -- many of which echo NAS recommendations for the prior version.
Chartres says that process is hurting TSCA implementation.
“I think sometimes people forget the bigger picture. It's delaying getting regulations getting made,” he says, noting that EPA has spent years working to develop a unique systematic review approach to use in TSCA evaluations.
“Six years have passed since Congress updated [TSCA], yet EPA still does not have an appropriate method for conducting systematic reviews of chemical risks -- the method by which EPA evaluates potential health harms -- which is critical to protecting the public’s health,” reads a July 28 PRHE blog post.
PRHE’s post argues that critique “is the clear conclusion” from SACC’s peer review of the Biden administration draft. “EPA’s failure to establish scientifically defensible systematic review methods for its TSCA risk evaluations means further delays in EPA action to protect the public from toxic chemical exposures.”
Although the committee described EPA’s new draft method as “an improved approach compared with the 2018 method,” it also repeated key advice that both the EPA advisors and NAS offered for the Trump-era version.
Those findings followed comments by several members of the SACC peer review panel that they had “major concerns” with how EPA crafted its new methods during their April 19-21 meeting.
PRHE’s post outlines several such repeat concerns, including EPA’s use of a “quantitative scoring” model that assigned studies numerical scores for quality and used those figures to determine which would be included in an evaluation. NAS found that method to be flawed, but SACC’s latest review says the new version still uses a form of numeric scoring and urges EPA to drop that approach entirely.
“These problems with EPA’s approach to study quality assessment continued in its 2021 TSCA method,” PRHE writes.
But both Chartres and PRHE’s post argue that the latest review further underscores prior arguments raised by both SACC and NAS on the 2018 version that EPA should consider abandoning its effort to craft a TSCA-specific systematic review approach entirely, and instead adopt one of several existing systematic review approaches that have been adapted from evidence-based medical approaches to environmental health research.
Those calls have generally held up three candidates for EPA to consider: a revised version of the approach used by the agency’s own Integrated Risk Information System (IRIS) program; a method developed by the National Toxicology Program’s Office of Health Assessment and Translation; and the Navigation Guide approach developed by PRHE.
“Previous recommendations from SACC and [NAS] pointed out existing methodologies that could, and should, be used to both improve the basis for the decisions made and to save time and money,” SACC wrote in its latest report. “These recommendations should be re-reviewed.”
Chartres says SACC is “basically pointing out [the TSCA systematic review approach] has been reviewed multiple times and you’re getting the same recommendations: address all the flaws and there are methodologies that could have been used from the outset. TSCA risk evaluations are going to be further delayed and this is something that could have been [avoided] and there are [existing] methods that could have been used.”
EPA researchers and official have repeatedly rejected those calls, saying the TSCA office’s needs are fundamentally different from those other programs’ because they focus on hazard assessment rather than the toxics law’s model of risk analysis. In particular, they say the existing models are ill-equipped to consider exposure data that is key to TSCA evaluations.
But Chartres argues that it would be far more efficient for EPA to adapt one of the existing tools to those data types.
“We recognize that they need something unique to consider exposure -- it doesn’t justify developing a unique [method] from the principles of systematic review, which is really what this is. And SACC and NAS are saying the same thing, that there are principles you can apply,” he says.
And he points to an ongoing project where PRHE has assisted World Health Organization (WHO) and International Labour Organization (ILO) researchers to adapt the Navigation Guide approach to consider occupational exposure or “prevalence of exposure” studies as evidence those revisions are possible.
“One of the things EPA often mentions is the unique need to apply systematic review to exposure and other types of data, so they can’t just use IRIS or OHAT [methods]. But our method is being used by WHO/ILO for different occupational risk factors, it's been done for 15 systematic reviews. It's really groundbreaking stuff … The method used for evaluating health effects is similar to hazard identification for risk assessment,” he said.
Of the 15 systematic reviews and meta-analyses, the WHO/ILO group has conducted, five are “of studies estimating the prevalence or level of exposure to occupational risk factors, where they still used the Navigation Guide but adapted it for evaluating the study quality and assessing the quality of body of evidence,” Chartres said.
He explains that a systematic review approach has seven or eight steps in total, with most “amenable to any other data stream. The unique consideration is how you assess study quality. All the other steps in systematic review would be the same.”
And Chartres adds, “If you follow most of the IRIS method that means you have confidence in the science. The one step that needs to be adapted, and WHO has done this, is . . . create a study quality tool to assess this type of data because it's different to health effects. It doesn’t mean you create an entirely new type of method.”
While PRHE researchers have offered training or method development assistance to EPA, Chartres says, “It's disappointing our offers to help have not been taken up. We’re trying to ensure the best available science has been used.” -- Maria Hegstad (firstname.lastname@example.org)