Last month, the FDA allowed Pfizer to change the warning label for Chantix — a drug prescribed for smoking cessation — so that it no longer has to have a prominent “black box” warning for psychiatric effects. Instead, it has a warning lower in the label that says:

Neuropsychiatric Adverse Events: Postmarketing reports of serious or clinically significant neuropsychiatric adverse events have included changes in mood (including depression and mania), psychosis, hallucinations, paranoia, delusions, homicidal ideation, aggression, hostility, agitation, anxiety, and panic, as well as suicidal ideation, suicide attempt, and completed suicide. Observe patients attempting to quit smoking with CHANTIX for the occurrence of such symptoms and instruct them to discontinue CHANTIX and contact a healthcare provider if they experience such adverse events. (5.1)

Kaleigh Rogers at Vice hails the change: “Though this is a win for Pfizer, it’s also a win for science: years of independent scientific research has shown there’s no link between the drug and an increased risk of negative psychiatric effects.” That sort of loose analysis?—?“no link,” instead of the more accurate “no statistically significant difference among users with no prior history of psychiatric conditions”?—?is what gets us into trouble. The problems only get worse once we look at the clinical trial that supposedly justified this change, a trial so plagued with problems that the FDA disregarded many of the investigators’ conclusions and initiated an investigation into protocol violations.
 


Put simply, we don’t have adequate scientific data to conclusively prove or disprove a link between Chantix and neuropsychiatric adverse events, but the evidence we do have is quite worrying.
 


Chantix, or varenicline, is a psychiatric drug with unknown effects on the brain. It works by inhibiting the ?4?2 acetylcholine nicotinic receptor. That’s the “on target” action. The “off target” actions of varenicline, i.e., the other stuff it does in the brain, aren’t well understood, but we know it has some. For example, a study in 2011 noted varenicline is “likely to have such off-target effects because, along with partial agonism of ?4?2*-nAChR, they are full agonists for the ?3?4-nAChR of autonomic ganglia and homomeric ?7-nAChR found in brain and other tissues,” and researchers in 2013 found “significant activity at ?7 and ?3* receptors, which may limit their utility and generate side effects.” Earlier this year, researchers found varenicline could even inhibit sugar cravings.
 


A drug that helps you stop smoking and stop eating so many sweets might sound like a good thing, but it underscores an important point: we don’t know the full scope of what these drugs do to the brain. Worse, we really don’t know what these drugs do to someone with pre-existing mental illness. All of those same “off target” actions identified above are also implicated in brain diseases like Alzheimer’s and schizophrenia. Hence, the researchers in 2011 suggested “Numerous adverse neuropsychiatry effects have been reported for varenicline therapies, leading the Food and Drug Administration to issue a black-box warning. It may be the case that these adverse effects were caused at least in part by off-target effects on ?7 receptors in patients already suffering from disease-related decreases in ?7 function.”
 


Let’s talk about those “adverse neuropsychiatry effects.” The FDA didn’t add a black box to Chantix willy-nilly. The FDA added it because, within the first year that Chantix was on the market, they received 153 adverse event reports in which doctors reported their patients experiencing suicidal ideation or their patients attempting/completing suicide.
 


Even today, the “adverse event” data for Chantix is worrying. Using Advera Health, I pulled up the adverse event data for Chantix then narrowed it to serious psychiatric events with more than 100 reports in which the doctor considered the Chantix the “primary suspect” for the problem. The “ROR” is the Reporting Odds Ratio, which shows how often that particular adverse event was reported for Chantix as compared to all other drugs.
 


To call those numbers “disturbing” is an understatement. Doctors report a connection between Chantix and suicidal behavior 24.42 times more often than they do for other drugs. Suicidal ideation 12.84 times more often. Suicide attempt 9.65 times more often. Completed suicide 1.85 times more often. That’s a big deal.
 


So why is the warning coming off? For the long story, see the FDA’s Advisory Committee Briefing Document from September 2016. The biggest difference was this clinical trial, conducted at the FDA’s request and paid for by Pfizer, which makes Chantix, and GlaxoSmithKline, which makes Zyban (bupropion, the same chemical in Wellbutrin).
 


Before we get to the results of that clinical trial, consider the source. The trial had ten listed authors. Six work for Pfizer or GlaxoSmithKline. Of the four non-employee authors, three are regularly paid to consult on and promote Chantix, including the lead author, the second named author, and the last named author. The remaining author is from the UK, and so payment data is unavailable.
 


The clinical trial was, in essence, several trials at once. 4,028 patients with no psychiatric history and 4,116 patients with a psychiatric history were assigned to receive Chantix, Zyban, a nicotine patch, or a placebo. That’s 8,144 patients, but only 6,379 (78%) completed the study. Nonetheless, the researchers used even the incomplete data for 8,058 (99%) to assess safety. (Yes, this sort of fudging?—?I mean extrapolating?—?is commonplace.) Those 8,058 participants were broken down as follows:

• 990 people with no psychiatric history on varenicline
• 1026 people with a psychiatric history on varenicline
• 989 people with no psychiatric history on bupropion
• 1017 people with a psychiatric history on bupropion
• 1006 people with no psychiatric history on a nicotine patch
• 1016 people with a psychiatric history on a nicotine patch
• 999 people with no psychiatric history on a placebo
• 1015 people with a psychiatric history on a placebo

By “psychiatric history,” the researchers mean “Unipolar and bipolar mood disorders” (70–71% in each group), “Anxiety disorders” (19–20% in each group), “Psychotic disorders” (9–10% in each group), “Personality disorders” (1% in each group). The trial lasted a mere 12 weeks, then they followed up for 12 weeks afterwards.
 


The authors concluded, “The study did not show a significant increase in neuropsychiatric adverse events attributable to varenicline or bupropion relative to nicotine patch or placebo,” but the key word there is “significant.” In fact, users of Chantix were 1.59 times more likely to suffer “moderate and severe neuropsychiatric adverse events,” but the difference wasn’t statistically significant, for whatever that’s worth. (Read this post of mine on the problem with statistical significance, which discusses the American Statistical Association’s own criticism of the concept of “statistical significance.”)
 


But the problem doesn’t end there: the clinical trial was done so poorly that the FDA’s Office of Scientific Investigation has an ongoing investigation into protocol violations at several of the clinical trial sites.
 


As they say in computer science, “garbage in, garbage out,” and the Pfizer study was garbage. On page 42 of the FDA review, the FDA included a special section on “general issues on data quality and reviewability.” The FDA tried to independently review the case narratives, but, given how poorly the study was done, “it was determined that it was neither feasible nor possible to attempt to adjudicate the cases based on the provided information.”
 


Nonetheless, the FDA did find that “many [adverse] events of potential interest were not flagged, and no narratives were constructed,” that Pfizer repeatedly attributed emotional symptoms to “stress,” and that Pfizer “almost always” coded worsening symptoms of pre-existing psychiatric illness as “not related to study drug.” The FDA also noted a litany of clinical trial protocol violations including,

inconsistencies between electronic case report forms and source data, missing documents, missing safety assessments and failure to record adverse events, and personnel performing diagnostic interviews and mental health evaluations who did not meet the mental health professional qualification requirements.

The data was such garbage that, for the FDA’s review, “[Pfizer’s] investigator assessment of relatedness has been altogether disregarded.”
 


Here’s just one example: a trial participant was coded as having a “skull fracture.” Although Pfizer didn’t code it, the FDA’s “subsequent information requests revealed the injury was inflicted by her boyfriend, also a trial participant.” And that wasn’t enough for Pfizer to indicate a potential connection to psychiatric problems? A trial participant fractures the skull of another participant and Pfizer doesn’t even write that down?
 


The disturbing part in all of this is how the FDA softened the warning anyway. So what does this mean for pharmaceuticals going forward? Can drug companies really push the FDA to minimize warnings by way of garbage clinical trials that fail to collect adequate data and fail to properly report the data they do collect? What kinds of incentives does that create for drug companies going forward?