Knowing how to easily spot bad scientific research is a useful skill especially today when experts who are supposed to do it are few and they don’t write for mass media. I will give you some tips to do it by yourself. I am focusing on the studies regarding humans. 

  1. Most of the time you hear about the new scientific discoveries from the media. If you see reporting of scientific study in mass media don’t automatically assume that what you read is the same as what is in the study. Journalists don’t understand science and often write nonsense. 

*Who understands the science of COVID-19 which is the only thing we are thinking about is explained in detail by me in this post. 

  1. Check scientific studies by yourself. Look for link to the study. If you can’t find any then it’s likely fake news. Dismiss! 
  2. If you are able to find the original study then it is time to find out if it is good or not. It may seem difficult but isn’t.
  3. You don’t need to read the whole study. Read the Abstract where normally you could find what’s about, and the section called Materials and Methods where you could see how many people were included. 

What to look for more specifically?

  1. Before even starting!!! Check where the authors work and how they are funded. If funded by foundation or people who earn money from their research don’t even waste time to read the study. 
  2. If the study is regarding humans it must be done on humans! Studies on animals or in cell lines or organs cannot tell much about what could be the thing which is investigated in humans so dismiss. Math models – same. We are too complicated to be assessed by math modeling. This should be mentioned in the abstract, and materials and methods.
  3.  Check the sample size – it means how many people are included. The more the better. 100 is a good number to start with.This should be mentioned in the abstract or materials and methods.
  4.  Check how the results are obtained. I read a study where people self reported symptoms on an app. It was in Nature Journal, but absolutely unreliable. Results must be collected by researchers or medical personnel. Self-reporting is big NO! It has the same merit as twitter polls.  

What if the study comes from a government website (FDA, CDC, NHS, RKI) or World Health Organization? 

That doesn’t mean reliable. Follow the steps above. If there isn’t a link with an original peer-reviewed study, dismiss it!

Unfortunately even reliable journals publish really bad studies especially regarding COVID-19. Truth is that 99% of COVID-19 science is low quality. Imagine how many real meaningful studies can be done with the waste of money for COVID-19 research.

If you wonder if an article is worth or need help for your work or organisation contact me at letstalk@genuineprospect.com

Advertisement