• Home
  • Health & Wellness
  • Disclaimer
    • Terms of Use
    • Privacy Policy
    • DMCA Notice
Podcast
  • Home
  • Health & Wellness
  • Disclaimer
    • Terms of Use
    • Privacy Policy
    • DMCA Notice
24/7 Health News
No Result
View All Result
Home Article

Major study shows the need to improve how scientists approach early-stage cancer research

March 20, 2022
in Article
0
Major study shows the need to improve how scientists approach early-stage cancer research
file 20220316 15 1tmf3hp
Preclinical research — the kind that takes place before testing on humans — often guides decisions about which potential treatments should continue to clinical trials. But attempts to replicate 50 studies found the odds of getting the same results were only about 50-50. (Pexels/Artem Podrez)

Preclinical studies, the kind that scientists perform before testing in humans, don’t get as much attention as their clinical counterparts. But they are the vital first steps to eventual treatments and cures. It’s important to get preclinical findings right. When they are wrong, scientists waste resources pursuing false leads. Worse, false findings can trigger clinical studies with humans.

Last December, the Center for Open Science (COS) released the worrying results of its eight-year $US 1.5 million Reproducibility Project: Cancer Biology study. Done in collaboration with research marketplace Science Exchange, independent scientists found that the odds of replicating results of 50 preclinical experiments from 23 high-profile published studies were no better than a coin toss.

Praise and controversy have followed the project from the beginning. The journal Nature applauded the replication studies as “the practice of science at its best.” But the journal Science noted that reactions from some scientists whose studies were chosen ranged from “annoyance to anxiety to outrage,” impeding the replications. Although none of the original experiments was described in enough detail to allow scientists to repeat them, a third of the original authors were unco-operative, and some were even hostile when asked for assistance.

A person wearing PPE using a multi-channel pipette in a laboratory
It’s important to get preclinical findings right. When they are wrong, scientists waste resources pursuing false leads.
(Shutterstock)

COS executive director Brian Nosek cautioned that the findings pose “challenges for the credibility of preclinical cancer biology.” In a tacit acknowledgement that biomedical research has not been universally rigorous or transparent, the American National Institutes of Health (NIH), the largest funder of biomedical research in the world, has announced that it will raise requirements for both of these qualities.

I have taught classes and written about good scientific practice in psychology and biomedicine for over 30 years. I’ve reviewed more grant applications and journal manuscripts than I can count, and I’m not surprised.

A stack of journal articles, with passages highlighted in the top one, with a pen resting on top.
Independent scientists found that the odds of replicating results of 50 preclinical experiments from 23 high-profile published studies were no better than a coin toss.
(Shutterstock)

The twin pillars of trustworthy science — transparency and dispassionate rigour — have wobbled under the stress of incentives that enhance careers at the expense of reliable science. Too often, proposed preclinical studies — and surprisingly, published peer-reviewed ones — don’t follow the scientific method. Too often, scientists do not share their government-funded data, even when required by the publishing journal.

Table of Contents

  • Controlling for bias
  • Study design and statistics
  • A better future

Controlling for bias

Many preclinical experiments lack the rudimentary controls against bias that are taught in the social sciences, though rarely in biomedical disciplines such as medicine, cell biology, biochemistry and physiology. Controlling for bias is a key element of the scientific method because it allows scientists to disentangle experimental signal from procedural noise.

Confirmation bias, the tendency to see what we want to see, is one type of bias that good science controls by “blinding.” Think of the “double-blind” procedures in clinical trials in which neither the patient nor the research team knows who is getting the placebo and who is getting the drug. In preclinical research, blinding experimenters to samples’ identities minimizes the chance that they will alter their behaviour, however subtly, in favour of their hypothesis.

Seemingly trivial differences, such as whether a sample is processed in the morning or afternoon or whether an animal is caged in the upper or lower row, can also change results. This is not as unlikely as you might think. Moment-to-moment changes in the micro-environment, such as exposure to light and air ventilation, for example, can change physiological responses.

A row of clear acrylic animal cages, each housing a white rat.
Seemingly trivial differences, such as whether an animal is caged in the upper or lower row, can change results.
(Shutterstock)

If all animals who receive a drug are caged in one row and all animals who do not receive the drug are caged in another row, any difference between the two groups of animals may be due to the drug, to their housing location or to an interaction between the two. You can’t honestly choose between the alternative explanations, and neither can the scientists.

Randomizing sample selection and processing order minimizes these procedural biases, makes the interpretation of the results clearer, and makes them more likely to be replicated.

Many of the replication experiments blinded and randomized, but it’s not known if the original experiments did. All that is known is that for the 15 animal experiments, only one of the original studies reported randomization and none reported blinding. But it would not be surprising if many of the studies neither randomized nor blinded.

Study design and statistics

According to one estimate, over half of the one million articles published each year have biased study designs, contributing to 85 per cent of US$100-billion spent each year on (mostly preclinical) research being wasted.

In a widely reported commentary, industry scientist and former academic Glenn Begley reported being able to reproduce the results of only six of 53 academic studies (11 per cent). He listed six practices of reliable research, including blinding. All six of the studies that replicated followed all six practices. The 47 studies that failed to replicate followed few or, sometimes, none of the practices.

Three people in white coats with a microscope in the foreground, superimposed with bar graphs and data points.
Misuse of statistics is a common in biomedical research despite calls for better data analysis practices.
(Shutterstock)

Another way to bias findings is by misusing statistics. As with blinding and randomization, it’s not known which, if any, of the original studies in the reproducibility project misused statistics, because of the studies’ lack of transparency. But that, too, is common practice.

A dictionary of terms describes a slew of poor data analysis practices that can manufacture statistically significant (but false) findings, such as HARKing (Hypothesizing After the Results are Known), p-hacking (repeating statistical tests until a desired result is produced) and following a series of data-dependent analysis decisions known as a “garden of forking paths” to publishable findings.

These practices are common in biomedical research. Decades of pleas from methodologists, and an unprecedented statement from the American Statistical Association to change data analysis practices, however, have gone unheeded.

A better future

A woman wearing a lab coat and safety glasses and green gloves examining lab samples
Incentives and standards should reward practices that produce trustworthy science and censor practices that do not, without killing innovation.
(Shutterstock)

Those who are anti-science should not take heart in these findings. Preclinical science’s accomplishments are real and impressive. Decades of preclinical research led to the development of the COVID-19 mRNA vaccines, for example. And most scientists are doing the best they can within a system that rewards quick flashy results over slower reliable ones.

But science is done by humans with all the strengths and weaknesses that go with it. The trick is to reward practices that produce trustworthy science and to censor practices that do not, without killing innovation.

Changing incentives and enforcing standards are the most effective ways to improve scientific practice. The goal is to improve efficiency by ensuring scientists who value transparency and rigour over speed and flash are given a chance to thrive. It’s been tried before, with minimal success. This time may be different. The Reproducibility Project: Cancer Biology study and the NIH policy changes it prompted may be just the push needed to make it happen.

The Conversation

Robert Nadon does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

ShareTweetSharePin
Previous Post

Ukraine: disease control is a casualty of war – so a surge in COVID cases is likely

Next Post

Returning to exercise postpartum: Supporting women’s physical activity after the birth of a child

Next Post
Returning to exercise postpartum: Supporting women’s physical activity after the birth of a child

Returning to exercise postpartum: Supporting women's physical activity after the birth of a child

Most Read

What causes stuttering? A speech pathology researcher explains the science and the misconceptions around this speech disorder

What causes stuttering? A speech pathology researcher explains the science and the misconceptions around this speech disorder

December 15, 2022
morning back pain

Morning Again Ache Trigger Is Not the Mattress

October 11, 2021
lower back pain relief exercises

5 decrease again ache aid workouts

October 11, 2021
3 years after legalization, we have shockingly little information about how it changed cannabis use and health harms

3 years after legalization, we have shockingly little information about how it changed cannabis use and health harms

October 15, 2021
Nasal vaccines promise to stop the COVID-19 virus before it gets to the lungs – an immunologist explains how they work

Nasal vaccines promise to stop the COVID-19 virus before it gets to the lungs – an immunologist explains how they work

December 14, 2022
bleeding in gum

When The Bleeding in gum Is Severe ?

October 11, 2021
Good Night Sleep

6 Causes of Good Evening Sleep

October 11, 2021

COVID vaccines: how one can pace up rollout in poorer international locations

October 5, 2021
Ten small changes you can make today to prevent weight gain

Ten small changes you can make today to prevent weight gain

October 12, 2021
Support and collaboration with health-care providers can help people make health decisions

Support and collaboration with health-care providers can help people make health decisions

December 16, 2021
Greece to make COVID vaccines mandatory for over-60s, but do vaccine mandates work?

Greece to make COVID vaccines mandatory for over-60s, but do vaccine mandates work?

December 1, 2021

Multiple sclerosis: the link with earlier infection just got stronger – new study

October 12, 2021
News of war can impact your mental health — here’s how to cope

Binge-eating disorder is more common than many realise, yet it’s rarely discussed – here’s what you need to know

December 2, 2022
Nurses’ attitudes toward COVID-19 vaccination for their children are highly influenced by partisanship, a new study finds

Nurses’ attitudes toward COVID-19 vaccination for their children are highly influenced by partisanship, a new study finds

December 2, 2022
Kick up your heels – ballroom dancing offers benefits to the aging brain and could help stave off dementia

Kick up your heels – ballroom dancing offers benefits to the aging brain and could help stave off dementia

January 3, 2023
How hot is too hot for the human body? Our lab found heat + humidity gets dangerous faster than many people realize

How hot is too hot for the human body? Our lab found heat + humidity gets dangerous faster than many people realize

July 6, 2022
The promise of repairing bones and tendons with human-made materials

The promise of repairing bones and tendons with human-made materials

January 4, 2022
As viral infections skyrocket, masks are still a tried-and-true way to help keep yourself and others safe

As viral infections skyrocket, masks are still a tried-and-true way to help keep yourself and others safe

December 14, 2022
How regulatory agencies, not the courts, are imposing COVID-19 vaccine mandates

How regulatory agencies, not the courts, are imposing COVID-19 vaccine mandates

October 24, 2021
Heart disease risk from saturated fats may depend on what foods they come from – new research

Heart disease risk from saturated fats may depend on what foods they come from – new research

November 29, 2021
Late night eating may cause greater weight gain – new research points to why

Late night eating may cause greater weight gain – new research points to why

October 12, 2022

10 Things I Learned During My Body Transformation

October 12, 2021

7 Health Benefits of Sweet Potatoes

October 12, 2021
Nobel prize: how chilli peppers helped researchers uncover how humans feel pain

Nobel prize: how chilli peppers helped researchers uncover how humans feel pain

October 12, 2021
How air filters can make COVID wards safer for patients and staff

How air filters can make COVID wards safer for patients and staff

December 1, 2021
GPs don’t give useful weight-loss advice – new study

GPs don’t give useful weight-loss advice – new study

December 16, 2022
Four ways to avoid gaining weight over the festive period – but also why you shouldn’t fret about it too much

Four ways to avoid gaining weight over the festive period – but also why you shouldn’t fret about it too much

December 22, 2022
Five ways to avoid pain and injury when starting a new exercise regime

Five ways to avoid pain and injury when starting a new exercise regime

December 30, 2022
Why suicide prevention support is crucial for people with fetal alcohol spectrum disorder

Why suicide prevention support is crucial for people with fetal alcohol spectrum disorder

September 28, 2022
Too hot to handle: Climate considerations for youth sport during the hottest years on record

Too hot to handle: Climate considerations for youth sport during the hottest years on record

July 19, 2022
  • Home
  • Health & Wellness
  • Disclaimer

© 2020 DAILY HEALTH NEWS

  • Home
  • Health & Wellness
  • Disclaimer
    • Terms of Use
    • Privacy Policy
    • DMCA Notice

© 2020 DAILY HEALTH NEWS