Feeding vaccine hesitancy

The FDA moved slowly in using all available tools to search for adverse events

I’m resending this story to include a response from the Food and Drug Administration.

Over half of U.S. adults have gotten at least one COVID-19 vaccine shot. President Biden now wants to reach a 70% vaccination rate by July 4th. It’s a high bar. Anywhere from 15% to 30% of Americans say they won’t get vaccinated, according to recent polls.

Nearly half of the vaccine-hesitant fear the vaccines’ side effects. Over a third say they will wait for definitive evidence the vaccine is safe before reconsidering.

Some of that reluctance is political. Distrust of government runs high in certain circles, especially among Trumpian conservatives who refuse to accept assurances from public health officials that the vaccines are safe. Even some liberals are wary about safety evidence drawn from short-term clinical trials run by the drug companies selling the vaccines.

The fear factor is fed by a flood of anecdotal reports by friends, social media and the press about the flu-like symptoms and other minor side effects that some experience shortly after vaccination. For many, the brief pause in distributing the Johnson & Johnson one-shot vaccine, triggered by a handful of reported patient blood clots, reinforced those anxieties.  

It’s those fears that drove me to spend much of the past week delving into the government’s ability to quickly monitor the patient experience with the new vaccines – both for side effects and their success at preventing COVID-19. Here’s what I found: Like the personal protective equipment and hospital surge capacity fiascos of the early months of the pandemic, government officials moved slowly over the past year – the period when it knew innovative vaccines were being developed – to use tools already in hand that would have given Americans a far better vaccine safety and effectiveness monitoring system than the ones currently in use.

This isn’t to say that the vaccines now being administered to hundreds of millions of Americans aren’t safe and effective. By any regulatory standard, they overwhelmingly meet those tests.

Given the carnage already caused by an uncontrolled pandemic among a population culturally and politically averse to alternative public health measures like masking, social distancing and economic lockdowns, the benefits of these highly effective vaccines far outweigh their risks. Even the writers on Saturday Night Live had the perspicacity to note that the risks from blood clots from birth control pills are far greater than the rare events associated with the COVID-19 vaccines.

That said, it’s still worth reviewing the Food and Drug Administration’s missteps in setting up a new post-approval vaccine safety and effectiveness monitoring system. Americans deserve one that is as technologically advanced as the products the agency reviews. If one was already in place and its findings were fully transparent, it would likely have provided by this point far more convincing assurances about the vaccines’ safety and helped reduce vaccine hesitancy.

The current system

From the early 1990s until one year into the pandemic, the Centers for Disease Control and Prevention and the FDA primarily relied on the Vaccine Adverse Events Reporting System, the Vaccine Safety Datalink and the MedWatch programs to identify the risks of vaccines approved for general use. Each system relies on voluntary reporting by physicians and patients.

Voluntary reporting can provide hints that a vaccine may be causing a significant problem. But by itself, voluntarism generates neither the evidence researchers and regulators need to draw conclusions nor the reassurances that people crave.

The CDC also relies on voluntary reporting to count so-called breakthrough cases, where individuals contract a disease after getting vaccinated. For the COVID-19 vaccines, the agency through April 26 had counted just 9,245 infections among the 95 million people who had been fully vaccinated.

If taken at face value, that number would suggest an efficacy rate 3 ½ times better than the 95% effectiveness shown in the Pfizer/BioNTech clinical trial (1 breakthrough case per 10,000 vaccinations versus 3.6 per 10,000 in the trial). The obvious conclusion is that the CDC’s low number reflects shortcomings in voluntary reporting, not some medical miracle.

The FDA and CDC’s inability to generate near real-time data about the safety and effectiveness of the new vaccines is just the latest example of the U.S.’s lack of pandemic preparedness, an across-the-board failure that has been tragically exposed by COVID-19. In this case, it was a failure more than ten years in the making.

In 2007 the FDA announced plans for the Sentinel initiative, a compilation of multiple electronic health information databases that could be mined to monitor the safety of approved medical products. Over the decade, the agency awarded Harvard Pilgrim Health Care, a Massachusetts-based non-profit health plan, nearly $40 million in grants to organize and run the system.

By 2011, agency officials declared they already had 60 million persons’ de-identified health records in a readily accessible database. It allowed regulators and researchers to review treatments and outcomes and answer specific safety questions. Today, the Sentinel initiative, which knits together both claims and medical records from dozens of payers and providers across the country, counts records for over 100 million covered lives in its database.

So why is Sentinel use slowing?

Practical results from the use of Sentinel are few and have been slow to arrive, according to a search of the medical literature and the FDA’s records. One of the few cases involving vaccines arrived a little over a year ago. Researchers from Harvard Pilgrim published a report based on five-year-old data that identified a slightly elevated risk of seizures among children under 2 from the pneumococcal vaccine. They deemed it insignificant compared to the benefits of protecting small children from the disease. The same study found no added risk from giving small children the flu vaccine.

The initiative’s website reveals the FDA conducted just 55 risk analyses using Sentinel’s database in the year before the COVID shutdown. That was less than half the number conducted during 2016. Even so, the database has proved useful.

In an assessment published two months ago, the FDA used the Sentinel system to confirm anecdotal reports that a new class of anticoagulants caused severe uterine bleeding, a rare side effect that had not shown up in the clinical trials leading to their approval. The agency changed the drugs’ labeling to warn physicians about that risk when treating women of reproductive age and those already at risk of uterine bleeding.

Nearly two years before the current pandemic, the FDA began positioning the Sentinel initiative to monitor treatments and outcomes during a public health crisis. Harvard Pilgrim collaborated with HCA Healthcare, the nation’s largest for-profit hospital chain, to set up a pilot project that monitored the annual influenza outbreak – “a proxy for a public health emergency,” Sentinel’s managers noted in an article published a month ago in Pharmacoepidemiology and Drug Safety. Based on that pilot, “Sentinel can actively monitor the characteristics, treatments, and outcomes of patients with COVID-19,” they wrote.

Their article emphasized the importance of using both financial claims records, which offer the procedure codes used for payment, and EHR data, which include clinical details and nurses’ notes. When they began using the HCA pilot project to monitor hospitalized COVID-19 patients last year, they discovered its data-rich EHRs identified 75% of patients had used oxygen or mechanical ventilation as compared to just 26% reported through the procedural codes used for payment. “Indicators of disease severity are likely underestimated if only procedure codes are relied upon,” the Sentinel researchers wrote.

Given the years and money poured into what turned out to be a very sophisticated and very large real-time monitoring system, why didn’t the FDA ask the Sentinel initiative’s managers to take on the job of active surveillance of the soon-to-be-deployed COVID-19 vaccines? And why didn’t the government set up a multi-agency coordinating body to oversee post-vaccination surveillance and routinely analyze data about their performance?

Last July, with Operation Warp Speed to develop the vaccines in full gear, several top former officials at FDA warned the agency that it needed to move “as soon as feasible after vaccine safety is established in trials” to study the possibility of rare events occurring. Dr. Nicole Lurie, who runs the Coalition for Epidemic Preparedness Innovations at Harvard Medical School, former FDA deputy commissioner Dr. Joshua Sharfstein, now at the Johns Hopkins Bloomberg School of Public Health, and former FDA chief scientist Dr. Jesse Goodman, now an infectious disease professor at Georgetown University, recalled how the swine flu vaccine campaign in 1976 had collapsed after reports surfaced that it caused Guillain-Barre syndrome in 1 in 100,000 vaccinations. “To ensure that such rare events, however unexpected, are rapidly detected, reported, and addressed, extensive near real-time monitoring will be critical during the roll out of COVID-19 vaccines,” they wrote.

They also pointed to the “strong model” developed during the H1N1 flu outbreak of 2009, where the federal government brought together federal and private sector databases in a system called PRISM (Post-Licensure Rapid Immunization Safety Monitoring), which used mostly claims data to actively monitor the rollout of vaccines. “An independent assessment group, including nongovernmental experts, reviewed data at least biweekly, helping ensure and communicate that the vaccines used in the U.S. were safe,” they wrote.

Unfortunately, early in the Trump administration, the FDA, then under the command of Dr. Scott Gottlieb, abandoned the idea of a more centralized and coordinated approach to monitoring vaccine rollouts. In October 2017, the agency moved the job of using “big data” for vaccine monitoring into a separate program called BEST (Biologics Effectiveness and Safety System).

Second best?

The BEST program still isn’t fully up and running and as of now is less capable than Sentinel in monitoring real-time adverse events. Why? It still relies mostly on claims data, not EHRs. Moreover, as reported this week by Kaiser Health News, BEST only adds to the hodgepodge of different systems collecting vaccine safety data. As Harvard Pilgrim epidemiologist Katherine Yih told KHN reporter JoNel Aleccia, “It’s worth stating that it’s not as good as it could be.”

BEST, which started by incorporating PRISM’s limited set of claims data, has only recently been expanded to include data from elecronic health records. An FDA spokesperson said it currently includes over 100 million lives, just like Sentinel. The CDC’s Vaccine Safety Datalink is significantly smaller, collecting data from just four health plans and providers in seven states.

However, BEST at present only monitors 15 pre-specified adverse events among Medicare beneficiaries, according to its website. And it won’t begin collecting data from commercial insurers, who cover the non-senior population, before the end of June, according to KHN. BEST’s draft master protocol for monitoring of the COVID-19 vaccine rollout, posted on the agency’s website in late March, stated the new system will only use administrative claims data to assess “the risk of safety outcomes from COVID-19 vaccination.”

An FDA spokesperson said that is no longer the case. “BEST data sources have now been expanded to include electronic health records which did not exist in PRISM when BEST was launched,” the prepared statement said. “So the EHR data sources not only have increased access to health care data of a larger patient population for use in vaccine effectiveness studies but also provide more granular health care data. BEST is currently performing active surveillance studies on all of the COVID-19 vaccines which have received Emergency Use Authorization from FDA. These studies are focusing on the safety of the vaccines first and will expand evaluate effectiveness as well.”

The FDA’s delays in moving aggressively to pursue using a wider array of data to look for rare adverse events and measure real-world efficacy hasn’t stopped former FDA commissioner Gottlieb from advocating that the agency take full control of vaccine monitoring. Writing in JAMA Health Forum last week, Gottlieb, now at the American Enterprise Institute and a paid adviser to Pfizer and other drug companies, called for consolidating all existing safety monitoring programs at the agency.

“The FDA has led a substantial effort in recent years to create a post-market safety surveillance system that relies on near–real-time reporting using information gathered from electronic health records,” he wrote. “The BEST system comprises data derived from medical claims, EHRs, and linked claims-EHR databases.”

Former FDA chief scientist Goodman’s biggest concern now is that “we haven’t yet seen data from any large databases, EHR or claims data,” he said. “Given the heated anxiety about Covid vaccines – we’ve never had mRNA vaccines before – we needed a really strong approach to getting all the data we could so we could report on that to the public.”