Surrogate endpoints: pitfalls of easier questions

Posted on April 8, 2016

Tags: , , ,

“Do we still remember the question we are trying to answer?
Or have we substituted an easier one?”
(Daniel Kahneman) [1]

Good news: Fluoride helps raise bone mineral density.

Sounds promising, doesn’t it?

But wait a second, will that change translate into a reduction in fracture risk?

Actually, no [2]. Don’t cry Joni. Life’s not that easy.

The thing is, when dealing with a difficult question, we tend to seek the answer to a simpler one, that seems to be relevant. Instead of talking directly about fracture risk, we try a detour, conduct research into bone mineral density, then continue as if an increase in bone mineral density equals a reduction in fracture risk.

At first glance, this seems reasonable.

However, a seductive trap awaits us here.

We can be misled by focusing too much on the surrogate endpoints.

Because surrogate endpoints are not the same as “direct” endpoints. The former doesn’t take into account patients’ real benefits.

Figure 2, Tran

Why do people use surrogate endpoints?

A drug or other treatment must show its efficacy before it can be approved for marketing.

However, it’s not easy to wait for a long time to count complication events, or to assess survival rate. Conducting a study, in that case, will be heavily costly, lengthy, and arduous (with a large sample size to manage). Surrogate endpoints help keep studies simple.

In some cases, proving effect on “direct” endpoints may not be feasible. For example, in the case of preventive treatment for asymptomatic deep venous thrombosis, clinical events (such as death, pulmonary embolism, etc.) don’t occur frequently. So follow-ups with ultrasound to screen for blood clots (an outcome that is not “felt” by patients) are reasonable.

Pitfalls of surrogate endpoints

As the above examples indicate, in some cases, treatments do improve surrogate outcomes, but may fail to show real benefit to patients.

Just like how we may sometimes have faith in people who are wrong.

It’s understandable. Because our knowledge of the relevant pharmacologic and biologic events is always imperfect and incomplete.

We don’t fully know what lies behind the relationship between a surrogate endpoint and the pathology of a disease. We make an inference based on surrogate endpoints just because it seems logical to do that. Well, it also seemed reasonable that no bacteria could live inside an acidic stomach. Until Helicobacter pylori was found.

Some surrogate endpoints may not lie in the causal pathway of a disease. For example, the risk that HIV-infected pregnant women will transmit the infection to their infants is correlated with CD4 count (an indicator of how well the immune system is working). However, interleukin-2 (a cancer treatment) fails to reduce transmission risk, despite its beneficial impact on CD4 count. It turns out that there is no causal relationship between them. Maternal CD4 and transmission risk are both influenced by a third factor: maternal viral load (i.e. the amount of HIV in the mother’s blood [7]). Therefore, focusing directly on the outcome of the proportion of infants infected with HIV would be fair, and most appropriate, in this case.

Drugs may have other unfavorable effects, despite having a “beneficial” impact on surrogate endpoints. As in the case of hormone replacement therapy for women after menopause. The treatment raises HDL level (i.e. concentration of “good” cholesterol). However, it may also increase the risk of deep vein thrombosis and breast cancer [6].

Well, in short, many pitfalls can be detected. So caution is needed when dealing with surrogate endpoints, in order to avoid false hope.

What to do now, as medical practitioners?

Are you looking for something that brings direct benefits to patients?

Seek for POEMs!

POEMs stands for Patient-Oriented Evidence that matters. It is direct evidence that a medical intervention, on average, lengthens life, decreases symptoms, and improves life quality.

If a study showed that a new drug reduces total cholesterol by 15%, the evidence is not yet persuasive. We could remain confident if that drug is also shown to reduce cardiovascular mortality. That would be better for patients.

However, POEMs is not always available.

Don’t worry!

Just follow the endpoint hierachy [7]. You should start with level 1 (that is, POEMs). Then if level-1 evidence is not ready for use, move on to level 2. And so on.

Now you have an invaluable compass to explore the medical evidence jungle.

Feel free to send me a postcard if you have any problems during the trip.

Bon voyage!

REFERENCES

1/ Kahneman, Daniel. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.

2/ Haguenauer D, Welch V, Shea B, Tugwell P, Adachi JD, Wells G. Fluoride for the treatment of postmenopausal osteoporotic fractures: a meta-analysis. Osteoporos Int. 2000;11:727–738.

3/ Reporting Information Regarding Falsification of Data” Proposed Rule. http://www.fda.gov/downloads/drugs/newsevents/ucm440796.pdf

4/ Oliver MF, et al. A co-operative trial in the primary prevention of ischaemicheart disease using clofibrate. Br Heart J. 1978 Oct;40(10):1069–1118.

5/ Gerstein HC, Miller ME, Byington RP, Goff DC, Bigger JT, Buse JB,et al. Effects of intensive glucose lowering in type 2 diabetes. N Engl JMed. 2008 Jun;358(24):2545–2559.

6/ Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women’s Health Initiative randomized controlled trial. JAMA. 2002 Jul;288(3):321–333.

7/ Fleming TR, Powers JH. Biomarkers and Surrogate Endpoints In Clinical Trials. Statistics in medicine. 2012;31(25):2973-2984.

Read more of Tran’s blogs here…

Efficacy of drugs: 3 examples to get you to truly understand Number Needed to Treat (NNT)

Key to statistical result interpretation: P-value in plain English

How did they determine diagnostic thresholds: the stories of anemia and diabetes

Why should medical students know about kappa value?

Tran Quang Hung

I'm currently a resident in Hematology. I love learning statistics, interpreting scientific results and evaluating health news on media. Feel free to contact me at drtranquanghung91@gmail.com

More Posts - Website

Related Post

creative commons license
Surrogate endpoints: pitfalls of easier questions by Tran Quang Hung is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Unless otherwise stated, all images used within the blog are not available for reuse or republication as they are purchased for Students 4 Best Evidence from shutterstock.com.

15 thoughts on “Surrogate endpoints: pitfalls of easier questions

Leave a Reply

Your email address will not be published. Required fields are marked *