Opinion masquerading as science

Public trust in FDA decisions will plummet if Commissioner Marty Makary abandons the agency's "gold standard" principles.

FDA Commissioner Marty Makary had every right to revisit the benefits and risks of taking hormones to treat severe menopause symptoms. The agency currently warns they may increase the risk of heart attacks, strokes, blood clots, and breast cancer.

But he had no right to stack the panel that was convened last week so it would come up with the results he wanted. That violates the Federal Advisory Committee Act (FACA), which prohibits conflicts of interest on federal advisory committees. According to the New York Times report on the session, “the panelists were selected by … the FDA commissioner, who has been an outspoken supporter of hormone therapy, and echoed his belief that the benefits of hormone therapy outweigh any risks.”

There was no presentation by FDA scientists. The public didn’t get a chance to speak. And, most significantly, the panel included four physician-members of Let’s Talk Menopause, an advocacy group backed by Bayer, Versalie, Pfizer and Astellas, all makers of products to treat menopause symptoms. That’s a crude violation of FACA.

The outcome was foreordained. The panel recommended removing the warning, at least on the low-dose estrogen used on vaginal creams.

The controversy

There may be some validity to a limited change in the label of female hormone replacement products. The original study showing the potential for harm from taking estrogen and progestin was called the Women’s Health Initiative (WHI), which began in the early 1990s and was funded by the National Institutes of Health. The study used higher doses of the hormones than are contained in the oral pills taken now. The estrogen/progestin arm of the study was stopped in 2002 when women in that arm showed an increased risk of developing breast cancer — a widely reported effect that scared millions of women away from taking the drugs.

Much has changed since then. Doses are lower. Practice has become more nuanced. It is generally conceded that it is mostly older women who are at greatest risk given it was mostly older women in the WHI.

But Makary isn’t interested in a hearing a scientific analysis that allows professionals at the FDA to make nuanced distinctions in product labeling. In his recent book Blind Spots (reviewed by GoozNews here), he claimed hormone therapy may actually prevent cognitive decline, heart disease and some cancers. Here’s what he said at the meeting, according to the Times:

“When a woman starts estrogen or estrogen and progesterone within 10 years of the onset of menopause, there is somewhere between roughly a 25 and 50 percent reduction in fatal heart attacks and cardiovascular disease. That is comparable to or greater than the risk reduction of a statin.”

He presented no evidence to back that claim, which were also included in his recent book. Last December, I spent a full day reviewing the evidence about estrogen/progestin therapy for my book review. Here’s what I wrote:

An evidenciary double standard

One of the most disturbing aspects of Makary’s book is his inconsistent use of medical evidence. He is quick to criticize the quality of studies used to justify the expert opinions he questions, but enthusiastically embraces studies with even lower levels of evidence when they suggest potential upsides from the therapies he supports.

This double standard is on full display in his chapter deriding the research indicating there are health risks associated with taking HRT to relieve symptoms of menopause. There’s no question HRT benefits women experiencing hot flashes and vaginal dryness. But for over two decades, HRT has carried an FDA black box warning label based on the Women’s Health Initiative (WHI), an NIH-funded clinical trial that compared more than 20,000 post-menopausal women taking hormones to two similarly-sized groups taking other, non-hormonal therapies. The trial also compared the hormone-taking group to a much larger “matched cohort” not in the trial.

The FDA warning states women taking an estrogen/progestin combination drug experience “increased risks of stroke, deep vein thrombosis, pulmonary embolism and myocardial infarction.” The trial “also demonstrated an increased risk of invasive breast cancer.” It is that latter finding that draws fire from Makary.

He correctly points out the trial’s finding that HRT increased the risk of breast cancer was problematic. The range of data points in the original study, known as the confidence interval, straddled the line between statistical significance and insignificance. (In statistics, “significance” refers to the likelihood the results are correct, not that there is a major effect.) There was a trend in the data indicating increased cancer risk, not conclusive proof.

Since breast cancer is always a hot button issue, the interim results led the organizers, over the protests of some of its participating clinicians, to stop the trial and hold a press conference. The widespread publicity triggered an immediate drop in HRT use and an ongoing reluctance to begin estrogen supplementation to relieve symptoms of menopause.

More than two decades later, Makary interviewed the primary organizers and authors of the study, who reluctantly confirmed his critique of the original study. “The finding (on breast cancer) is very borderline,” Dr. JoAnn Manson, a professor of medicine at Harvard Medical School and the study’s leader, told him. Yet earlier this year (2024), Manson and 18 co-investigators published an updated review of clinical implications of the WHI in JAMA. It included a 2020 study with longer follow-up and found elevated cancer risk. And, this time, it was statistically significant.

After critiquing the original breast cancer finding, Makary goes on to make broad claims for the health benefits of HRT, including delaying the onset of Alzheimer’s disease; prevention of cardiovascular disease and heart attacks; and reduced colon cancer risk. “Women taking estrogen have a 35% lower incidence of Alzheimer’s,” he declares. What evidence does he cite? A 1996 study of 8,877 women in a southern California retirement community mailed a general health survey in 1981, less than half of whom had died by 1995. The researchers then examined death records and found 248 mentioning Alzheimer’s or other forms of dementia, with about 35% more mentions in women who had not taken HRT than in the group that had.

This type of research is known as an observational study of matched cohorts, which is considered a much less trustworthy source of medical evidence than randomized clinical trials like the WHI. It is quite possible that unmeasured or unidentified variables such as underlying health conditions or family history will confound the results. The best one can say about conclusions drawn from observational studies is that they offer clues for further research but are inadequate for making definitive claims about efficacy.

Makary’s claim that HRT reduces the risk of heart disease is based on another observational study called the Nurses Health Study, first published in 1991 and updated in 2000. This frequently criticized study recorded the self-reported health habits, treatments and outcomes among over 70,000 nurses over two decades. “Heart disease is the leading cause of death in American women. HRT reduces that risk by about 50%,” he declares.

Yet in 2015, the Cochrane collaboration (“a highly respected group of experts who conduct extensive scientific reviews,” to use Makary’s description from another section of his book) found “no evidence that hormone therapy provides any protective effects against death from any cause, and specifically death from cardiovascular disease, non-fatal heart attacks or angina, either in healthy women or women with pre-existing heart disease.” The Cochrane group further concluded HRT increased risk of stroke for post-menopausal women.”

The updated review of the WHI, which was a randomized controlled clinical trial, again found no cardiovascular benefits. The review, which appeared this past May in JAMA, showed HRT significantly increased the risk of stroke and blood clots. The overall incidence of heart disease among women also increased, although that finding was not statistically significant.

And, as for Makary’s major claim that HRT presented no risk of cancer, this latest WHI review confirmed the trial’s original suspicion about increased cancer risk. Except this time, the findings were statistically significant. “At 20-year follow-up, [estrogen plus progestin] compared with placebo significantly increased breast cancer incidence” by 28%, the review reported. If taking estrogen alone, there was a slight decrease in breast cancer risk, but that was a statistically insignificant finding.

The bottom line: The overall risk of developing cancer from HRT is small and is greater in older women. That’s why the WHI investigators still suggest short-term HRT for women under 60 is an appropriate therapy for relieving menopause symptoms as long as they aren’t already at risk of developing heart disease or cancer.”

The fallout from opinion-driven decision-making

Makary’s penchant for substituting his own opinions for scientific analysis has not gone unnoticed at the FDA. Earlier this month, STAT reported that FDA staffers are “fed up and demoralized,” and are leaving in droves. Reporter Lizzy Lawrence …

… spoke with 15 employees across the agency’s drug, device, biologics, and food centers on how the FDA has functioned since the April 1 layoffs, which coincided with the start of FDA Commissioner Marty Makary’s tenure. All requested anonymity due to fear of retaliation from leadership. They told STAT that morale is incredibly low as departures mount and distrust of leaders grows among the staff that remain.

Last week’s hearing will only make things worse. What must internal scientists think when they are not even allowed to make a presentation to an advisory board empaneled by the new FDA commissioner?

At some point, this internal uproar will reach a broader audience — the nearly 200 million Americans who take prescription drugs each year. As with the recent decision questioning the safety of Covid vaccines, nothing less than the public’s faith in the FDA is at stake. If patients begin thinking they cannot believe what the FDA says and stop taking drugs and vaccines, the medical harm will be incalculable.