|
Woloshin Web Exclusive
D A T A W A T C H
D R U G B E N E F I T D A T A
W E B E X C L U S I V E
28 April 2004
The Value Of Benefit Data
In Direct-To-Consumer Drug Ads
Simple tables showing drugs’
expected outcomes
are understood and valued by consumers.
By Steven Woloshin, Lisa M.
Schwartz, and H. Gilbert Welch
ABSTRACT:
Direct-to-consumer (DTC) pharmaceutical ads typically describe drug benefits
in qualitative terms; they rarely provide data on how well the drug works. We
describe an evaluation of a “prescription drug benefit box”—data
from the main randomized trials on the chances of various outcomes with and
without the drug. Most participants rated the information as “very important”
or “important”; almost all found the data easy to understand. Perceptions
of drug effectiveness were much lower for ads that incorporated the benefit
box than for ads that did not. Most people we interviewed want benefit data
in drug ads, can understand these data, and are influenced by them.
Proponents of direct-to-consumer (DTC) drug advertising claim that the ads teach
consumers about new medicines and treatment options.1
For ads to serve this educational function, however, physicians and patients
need to know to what extent the drug works. This is particularly true for drugs
whose effects are not directed at symptoms and thus not directly experienced
by patients, such as cholesterol-lowering drugs to reduce the risk of heart
disease and platelet inhibitors to reduce the risk of recurrent heart attack
or stroke. Because it expands the market for pharmaceuticals so dramatically,
the passage of a Medicare drug benefit in November 2003 adds some urgency to
the need for more informative DTC drug ads.
In practice, DTC ads provide limited information on how well the drug works.
The U.S. Food and Drug Administration (FDA) requires information about potential
harms, which appears in fine print on the “brief summary” page.2
In contrast, information on drug benefit is not required (the ad must simply
note the drug’s indication), and most ads assert that drugs work using
vague, qualitative terms (for example, “Zyrtec works,” “lower
your number”) rather than presenting actual data.3
Only about a tenth of DTC ads provide data about drug benefit in either the
ad or the accompanying brief summary.4
Ironically, the absence of benefit data may lead some patients to assume that
the drug always works; physicians cited exaggerated perception of benefit as
the most significant problem these ads create for their practice, according
to a recent FDA- sponsored survey.5 Nearly one-quarter
of Americans believe that only “extremely effective” drugs can be
advertised to consumers.6 To make drug benefit data
readily available, we created a “prescription drug benefit box”—a
standardized table with published data on the chances of various outcomes with
and without the drug—to supplement the brief summary provided with three
current DTC ads. The benefit box was modeled on the FDA’s “Nutrition
Facts” box (appearing on food packaging). We then conducted a study to
learn whether consumers really understand and value drug benefit data.
Methods
Study advertisements.
We selected three current print DTC drug advertisements that met the following
criteria: The advertised drug was for a common medical condition, and the advertisement
clearly specified the purpose of the medication in the main part of the ad.
We constructed two versions of each ad (for a total of six ad versions): a standard
version, which replicated the published ad and brief summary, and a benefit
box version, which had the identical ad and a brief summary that included the
benefit box (note that all versions of the advertisements are available online).7
Standard version.
The standard version of the ad replicated the published ad except for three
changes. To avoid preconceived notions about effectiveness from people already
taking these medications, we devised alternative names for the drugs and drug
companies in the ads and brief summaries using modified pig Latin. To make the
standard brief summary version as similar as possible to the benefit box version,
we also made the following changes. We slightly rearranged the position of some
of the text (we did not alter the text size) to clear an approximately 3.5 inch
by 2 inch space for the benefit box. For one ad (pravastatin), we edited the
content so that it only focused on consumers who had not had a heart attack.
The three drug advertisements were as follows.
Pravastatin. Pravastatin is a cholesterol-lowering agent to reduce heart
attack risk, sold under the brand name Pravachol. The Pravachol ad shows a pencil
with a list of facts, and the headline reads, “Maybe it’s time to
learn the facts. Pravachol is the only cholesterol lowering drug proven to help
protect against 1st and 2nd heart attack and stroke.” To focus the ad on
primary prevention, we changed the headline to “…proven to help protect
against heart attack” (the approved indication for primary prevention)
and omitted “1st and 2nd heart attack and stroke” (the approved indication
for secondary prevention) from the rest of the ad. In the study advertisement,
Pravachol was renamed Avastat.
Rofecoxib. Rofecoxib is a cyclooxygenase-2 (COX-2) inhibitor to reduce
arthritis pain, sold under the brand name Vioxx. The Vioxx ad shows a man digging
for clams with his dog at low tide, and the headline reads, “The clams
were the only ones that benefited from my arthritis. Sorry guys, I’m back.”
In the study advertisement, Vioxx was renamed Ofecox.
Clopridogrel. Clopridogrel is an antiplatelet agent to reduce second
heart attacks and strokes, sold under the brand name Plavix. The Plavix ad shows
a man standing at the edge of the Grand Canyon, and the headline reads, “You
don’t want another heart attack or another stroke to sneak up on you.”
In the study advertisement, Plavix was renamed Pridclo.
Benefit box version.
The benefit box version of each ad was identical to the standard version except
for the addition of the benefit box to the lower right-hand corner of the brief
summary. The benefit box is simply a table presenting the proportion of people
experiencing various outcomes with and without the drug.8
We expressed numbers as percentages and rounded them to the nearest whole number.9
We also included a one-word summary to describe the direction of the effect
(for example, “fewer”). For each drug, the efficacy data came from
the published article of the randomized trial cited in the FDA drug approval
document, which matched the indication and outcome in the advertisement.
Pravastatin. These data came from the only randomized trial of primary
prevention for people with high cholesterol cited in the FDA approval document
(and the approved label) for pravastatin: the West of Scotland Coronary Prevention
Group study.10 The benefit box presented data on
the five-year risk of having a heart attack, dying from a heart attack, and
dying for any reason (all-cause mortality) for pravastatin versus placebo (Exhibit
1).
Rofecoxib. These data came from the largest published randomized trial
cited in the FDA approval document, which compared Vioxx to ibuprofen for the
treatment of osteoarthritis (this was one of the two “pivotal studies”
submitted comparing Vioxx against ibuprofen). The benefit box presented data
on patients’ rating of the medication’s effect on their arthritis
symptoms (excellent, good, fair, poor, or no effect) for rofecoxib versus ibuprofen
at six weeks (Exhibit
2).11
Clopidogrel. These data came from the only randomized clinical trial
cited in the FDA approval document (and approved label) for the secondary prevention
of heart attack and stroke: Clopidogrel versus Aspirin in Patients at Risk of
Ischemic Events (CAPRIE).12 The benefit box presented
data on the two-year risk of having a stroke or heart attack, dying from a stroke
or heart attack, or dying for any reason for clopidogrel versus aspirin (Exhibit
3).
Advertisement evaluation.
Trained field interviewers from the Center for Survey Research at the University
of Massachusetts, Boston, were assigned to census blocks in the following towns
in the greater Boston area: Andover, North Reading, Newburyport, Fall River,
New Bedford, Chelmsford, Lowell, Brookline, Brighton, Cambridge, Belmont (MA);
Providence (RI); and Newport (NH). On each block, they knocked on doors and
interviewed up to four English-speaking adults who had no obvious cognitive
impairment. A total of 203 in-person interview were conducted; participants
received $20 for completing the interview. The interviews took an average of
twenty-three minutes. The interview was pretested with six subjects recruited
via newspaper ads (these subjects are not included in this study).
Before-after comparisons.
After a training task to familiarize people with the three elements of interest
(the ad, the brief summary, and the drug benefit box), each participant was
shown the standard version of the first drug advertisement. They were then asked
to state how effective they thought the drug was (using a five-point Likert
scale from “not effective” to “extremely effective”). Participants
were then given the benefit box version of the ad and were again asked to rate
drug effectiveness. In addition, participants were asked a question to test
their ability to use the table. This procedure was repeated for the second drug.13
Randomized comparison.
Respondents were subsequently asked a few general questions about the benefit
box itself (for example, whether they thought the information was important,
should be required, and was easy to understand) and then were randomized. People
in the intervention group were shown only the benefit box version of the third
ad (clopidogrel); the control group saw only the standard version of the ad.
Analysis.
For ease of presentation, we collapsed the five-level Likert response categories
for perceptions of drug effectiveness to three levels (extremely/very effective,
somewhat effective, and a little/not at all effective). P values for
the five-level comparisons were quite similar to those for the three-level comparisons.
For the before-after comparisons, we used an extended version of McNemar’s
test because the data are paired. For the randomized comparisons (where the
data are unpaired), we tested differences in proportions using the chi-square
test. All reported p values are based on two-sided tests. For all questions
answered by the full sample (N = 203), the maximum margin of error (that is,
the 95 percent confidence interval) was approximately 7 percent. All analyses
were done using STATA version 8.
Study Results
The 203 study participants represented a wide range of ages, incomes (37 percent
reported less than $25,000 total household income, while 18 percent reported
over $100,000), and education levels (15 percent reported less than high school
education, while 22 percent had done some graduate work) (Exhibit
4).
Desire for benefit data.
Before
having heard about the benefit box, 92 percent said that they would require
data on both benefits and side effects in drug ads (Exhibit
5). After seeing the benefit box, three-quarters of respondents said that
they would “pay a lot of attention” to the benefit box if it were
included in drug ads, and 67 percent said that they would trust the information
in the box more than information on the main page of the ad itself. Almost all
(93 percent) preferred the benefit box version to the standard brief summary,
5 percent had no preference, and only 2 percent preferred the standard brief
summary.14
Understanding of benefit
box. After
seeing the benefit box, respondents felt that the data in it were easy to understand:
They assigned the data a mean rank of 8.5 (median rank 9) on a scale from 0
(extremely hard) to 10 (extremely easy to understand). This rating was supported
by the fact that almost every respondent was able to correctly extract the appropriate
percentages from the table in two test questions (95 percent and 97 percent
responded correctly, respectively). Comprehension was high even among those
with the least formal educational attainment: 90 percent and 100 percent of
those with less than a high school education answered these two questions correctly.
Perceptions of drug effectiveness.
Before-after comparisons. Seeing the standard version of the ad, most
respondents rated the drugs as “extremely or very effective”: 51 percent
gave this rating to Avastat (15 percent rated it as extremely effective); 65
percent, to Ofecox (17 percent rated it as extremely effective) (Exhibit
6). After being presented with quantitative efficacy data from the randomized
trial in the benefit box version, the proportion of respondents rating the two
drugs as extremely or very effective fell—to 26 percent for Avastat and
28 percent for Ofecox (both p < .001). The presence of the benefit
box also caused many more respondents to correctly rate the effectiveness of
Ofecox as being “about the same” as that of ibuprofen (18 percent
with the standard version versus 57 percent with the benefit box version, p
< .001).
Randomized comparisons. Exhibit
7 shows the results for the randomized portion of the study, in which participants
received only one of the two ad versions for the third drug, Pridclo. Again
we found that the presence of the benefit box was associated with much lower
perceived effectiveness: The proportion rating Pridclo as “extremely”
or “very” effective was 59 percent in the standard version group versus
23 percent in the benefit box version group (p < .001). The presence
of the benefit box also caused many more respondents to rate the effectiveness
of Pridclo as being about the same as that of aspirin (36 percent with the standard
version versus 57 percent with the benefit box version, p = .001).
Interpretation And Limitations
We found strong support for including drug benefit data in DTC ads; almost all
respondents thought that such data in ads should be required. Almost all respondents
also liked the benefit box as a vehicle for communicating these data, rating
it as easy to understand and saying that they would pay attention to it. And
when given a choice, more than 90 percent preferred an ad that included the
box to the standard version without it. Finally, for each ad tested, perceptions
of drugs’ effectiveness dropped after respondents saw the benefit box.
Our findings should be interpreted in light of several limitations. First, it
is important to acknowledge that our findings are based on an experiment—although
92 percent said that they would pay some or a lot of attention to a benefit
box, in real life people might not. Second, both the ads and the study subjects
represent convenience samples. The ads were the first ones we found (flipping
through current magazines in our local general store) that met our inclusion
criteria; subjects consisted of the people who happened to be home and who agreed
to participate when an interviewer knocked on their door. Consequently, our
results might differ with different ads or different subjects (or even in different
geographic locations). However, given the magnitude of our findings and the
sociodemographic diversity of our non-self-selected participants, only an extraordinarily
powerful confounder could alter our qualitative message.
The impact of the benefit box on perceptions of drug effectiveness raises some
legitimate concerns. Our participants were very optimistic about the effectiveness
of each study drug; in each case, these perceptions dropped after seeing the
actual data. The effect, however, was similar for all drugs. This is a matter
of concern, since one of the drugs, Avastat, showed a reduction of overall mortality
over five years from 4 percent to 3 percent.15
We suspect that many respondents did not appreciate the real magnitude of this
effect: Few drugs now being manufactured can match this reduction in all-cause
mortality among relatively healthy outpatients. But to really judge how well
an advertised drug works, people need context—that is, some sense of the
magnitude of the benefit of typical drugs. Undoubtedly, most people lack such
knowledge. This probably explains why our respondents were less enthusiastic
when they saw the actual data for each medication. We believe that reactions
to drug benefit data will change as people have more exposure to them; that
is, consumers will be better able to discriminate among drugs as they become
better calibrated to effect sizes and the importance of different outcome measures.
Policy Implications
DTC drug advertising has been controversial since its inception, with proponents
and opponents debating the educational value of the ads and their impact on
the physician-patient relationship. Most troubling is the potential for advertising
to stimulate inappropriate demand for drugs. The new Medicare prescription drug
benefit can only exacerbate the problem. While it laudably provides millions
of Americans with improved access to pharmaceuticals, it also has a less desirable
effect: It means that millions more consumers may be stimulated by direct marketing
to demand drugs inappropriately.
One way to avoid inappropriate demand for advertised medications is to ban DTC
ads altogether, a position the European Union has taken for years and recently
reaffirmed.16 European consumer groups argued that
the educational material on drugs should “help establish informed choice
for patients instead of just more brand awareness” and that this educational
function could be fulfilled only by independent authorities, not by the pharmaceutical
companies’ ads.17 Recently, the Ministry of
Health in New Zealand—the only country besides the United States where
DTC ads are legal—reconsidered its position on them and is pursuing a ban.18
Another approach is to regulate the content of DTC ads to improve their educational
content. The systematic provision of drug benefit data would educate consumers
and promote informed decision making by providing easy access to scientific
data on drug benefit (drug versus placebo or an appropriate alternative intervention)
whenever a drug advertisement appears. A major challenge to moving forward will
be creating practical and valid guidelines to decide which data should be provided—specifically,
which outcome measures would be required and which study (or studies) should
the data come from. When there is only one main trial supporting a drug’s
efficacy for a given indication, study selection will not be an issue. In some
cases, generating good benefit data will require a sizable effort to pool data
and conduct meta-analyses. The FDA could mandate how such studies should be
done or even designate an independent entity to conduct them (perhaps paid for
by the drug companies).
To improve the communication of health information to consumers, the FDA recently
released draft guidance to pharmaceutical manufacturers with suggestions on
how to improve the brief summary in print DTC ads.19
The guidance, based on research by the FDA and independent investigators and
on consumers’ participation at a public FDA meeting in September 2003,
is intended to maximize the usefulness of information in DTC ads to consumers
and to help inform doctor-patient discussions about “the key risks and
benefits of a product.”20 Unfortunately, the
draft guidance contains no provisions for the communication of drug benefit
data.
Based on our findings, we suggest that the FDA mandate benefit data in the ads.
Our study shows that people want such data and have little trouble extracting
it from tables as implemented in the benefit box. Presenting benefit and harm
data together (in the same tabular format) would make it easy to put data on
side effects into context (that is, is this benefit worth the chance of these
side effects?) and would promote meaningful patient-physician discussion by
giving physicians access to data that they often lack. Requiring drug benefit
data in DTC ads is an important step in educating the public about prescription
drugs and promoting informed decision making.
Lisa Schwartz and Steven Woloshin contributed equally to the creation of
this manuscript; the order of their names is arbitrary. They are supported by
Veterans Affairs (VA) Career Development Awards in Health Services Research
and Development and by Robert Wood Johnson Generalist Faculty Scholar Awards.
This study was supported by a grant from the National Cancer Institute (CA91052-01)
and by an HSR&D Research Enhancement Award from the Department of Veterans
Affairs (REA-03-098). The views expressed herein do not necessarily represent
the views of the Department of Veterans Affairs or the United States government.
The authors thank Patricia Gallagher and Carol Cosenza (Center for Survey Research,
University of Massachusetts) for help in developing the survey and Kathryn Aikin
(Division of Drug Marketing, Advertising, and Communications, U.S. Food and
Drug Administration) for technical advice.
NOTES
1. M. Rosenthal et al., “Promotion of Prescription Drugs
to Consumers,” New England Journal of Medicine 346, no. 7 (2002):
498–505; and L. Fintor, “Direct-to-Consumer Marketing: How Has It
Fared?” Journal of the National Cancer Institute 94, no. 5 (2002):
329–331.
2. The brief summary includes a list of all harms noted in the
drug package insert and sometimes includes a table with data on the frequency
of side effects. The brief summary is typically an excerpt from the Physician’s
Desk Reference (PDR). The relevant regulations are found in “Prescription
Drug Advertisements,” 21 CFR, Part 202.1 (Washington: U.S. Government Printing
Office, 2003).
3. Ibid; S. Woloshin et al., “Direct to Consumer Drug Advertisements:
What Are Americans Being Sold?” Lancet 358, no. 9288 (2001): 1141–1146;
and R. Bell, M. Wilkes, and R. Kravitz, “The Educational Value of Consumer-Targeted
Prescription Drug Print Advertising,” Journal of Family Practice
49, no. 12 (2000): 1092–1098.
4. Bell et al., “The Educational Value.”
5. K.J. Aikin, Direct-to-Consumer Advertising of Prescription
Drugs: Physician Survey Preliminary Results, 13 January 2003, www.fda.gov/cder/ddmac/globalsummit2003/index.htm
(28 January 2004).
6. R. Bell, R. Kravitz, and M. Wilkes, “Direct-to-Consumer
Prescription Drug Advertising and the Public,” Journal of General Internal
Medicine 14, no. 11 (1999): 651–657.
7. See content.healthaffairs.org/cgi/content/full/hlthaff.w4.234v1/DC3.
8. The exhibits in this paper do not follow the benefit boxes’
formatting exactly as it was presented to study participants but merely provide
the information in the boxes. Each ad, the standard brief summary, and the brief
summary with the benefit box are available on the Health Affairs Web
site, as above.
9. We presented data using absolute rates of outcomes since
we have found in prior work that this format is the best understood. See L.M.
Schwartz et al., “The Role of Numeracy in Understanding the Benefit of
Screening Mammography,” Annals of Internal Medicine 127, no. 11
(1997): 966–972.
10. J. Shepherd et al., “Prevention of Coronary Heart
Disease with Pravastatin in Men with Hypercholesterolemia,” New England
Journal of Medicine 333, no. 20 (1995): 1301–1307.
11. R. Day et al., “A Randomized Trial of the Efficacy
and Tolerability of the COX-2 Inhibitor Rofecoxib vs. Ibuprofen in Patients
with Osteoarthritis,” Archives of Internal Medicine 160, no. 12
(2000): 1781–1787. Note that this article only presented mean differences
in the five-level Likert scale between entry and six-week follow-up (for example,
rofecoxib 2.4 levels lower versus ibuprofen 2.3 levels lower). Because we thought
that the actual distribution of responses by treatment group would be more easily
understood by readers, we obtained these data from the study authors and presented
three categories of responses (percentage reporting excellent effect; percentage
good effect; and percentage fair, poor, or no effect) in the benefit box.
12. CAPRIE Steering Committee, “A Randomised, Blinded
Trial of Clopidogrel versus Aspirin in Patients at Risk of Ischaemic Events
(CAPRIE),” Lancet 348, no. 9038 (2000): 1329–1339.
13. Because the drugs are so different in terms of indication
and efficacy, we randomly assigned participants to see the ads in different
orders (that is, pravastatin ads then rofecoxib ads, or rofecoxib ads then pravastatin
ads). Although we were concerned that the order of viewing might affect perceptions
of effectiveness, it did not, so we report pooled data here.
14. Preference for the benefit box version was the same in
both arms of the randomized comparison: 94 percent (standard ad group) versus
91 percent (benefit box group) preferred the benefit box version, p =
.56.
15. Shepherd et al., “Prevention of Coronary Heart Disease.”
16. R. Watson, “EU Health Ministers Reject Proposal for
Limited Direct to Consumer Advertising,” British Medical Journal
326, no. 7402 (2003): 1284.
17. Ibid.
18. B. Burton, “New Zealand Moves to Ban Direct Advertising
of Drugs,” British Medical Journal 328, no. 7431 (2004): 68.
19. U.S. Food and Drug Administration, “New FDA Draft
Guidances Aim to Improve Health Information (P04-12),” Press Release, 4
February 2004, www.fda.gov/bbs/topics/NEWS/2004/NEW01016.html
(17 March 2004).
20. Ibid; and FDA, “Direct-to-Consumer Promotion: Public
meeting, September 22 and 23, 2003,” www.fda.gov/cder/ddmac/DTCmeeting2003.html
(28 January 2004). Preliminary results of our evaluation were presented at this
meeting.
Read related papers by Joel
Weissman et al., Pat
Kelly, David
Riggs et al., James
Jeffords, Henry
Waxman, and Peter
Pitts.
Steven Woloshin (steven.woloshin{at}dartmouth.edu)
and Lisa Schwartz are senior research associates in the Veterans Affairs (VA)
Outcomes Group, White River Junction, Vermont, and associate professors of medicine
and of community and family medicine, Center for the Evaluative Clinical Sciences,
Dartmouth Medical School, in Hanover, New Hampshire. H. Gilbert Welsch is codirector
of the VA Outcomes Group and a professor of medicine and community and family
medicine, Dartmouth Medical School.
DOI: 10.1377/hlthaff.W4.234
©2004 Project HOPE–The People-to-People Health Foundation, Inc.
|
