It's difficult to read health and science news or peruse the articles of Healthkicker without being flooded with statements of things like "x substance causes cancer" or "miracle fruit reduces risk of stroke." Oftentimes, readers take the news summary at face value without delving into the actual research (or sometimes even getting past the title). In these news articles, there will often by a hyperlink to the study to which the article refers and it's important as a reader to be able to access and evaluate information.
Here are some quick and basic rules about evaluating research studies:
Sample Size and Who's Involved The first thing to look at is how many people were involved in the study. Studies with few participants have little validity. Without going into the details of Biostatistics and confidence intervals, understand that a study investigating exposures or diseases that are not rare or easily controlled have absolutely no reason not to have a very large study group. If only a few dozen people are studied, the findings are much less likely to be valid. Findings may not be applicable to other people and the effects of random chance are stronger when only a few people are studied.
Also, take note of whether or not the study involves people or animals. While animal models are necessary beginning parts to studies, published findings are not necessarily applicable to humans. It's also much easier to give large doses of an exposure to an animal to artificially create a disease association that would be impossible to replicate in people.
Cases and Controls Did the study have both a case group and control group? Other than the exposure or disease, were the two groups comparable to one another? Also, whenever you compare a study regarding a working population versus the overall population, remember that the overall population is inherently less healthy than the working population (known as the Healthy Worker Effect). Studies that have bad definitions for the cases or that lack controls are poor studies.
Where'd the Money Come From? Unfortunately, there's not always great integrity within science and researchers are confined to only publishing information that benefits those who funded the research. Take note of who funded the research and if they stood to gain from specific conclusions.
Biological Plausibility This is a pretty basic one; essentially, does the study make biological sense? If something seems highly biologically unlikely or far fetched, it is definitely something that you should research further.
So, the next time you read a health news article, you will be able to determine whether or not the study it summarizes is credible.
Recommend
Comments (7)
SOOO important! I'm afraid you might wind up preaching to the choir on this one, but there are those who will learn something by reading this. Particularly the bit about finding out who funded it -- the vast majority of the population don't seem to think of that. My university is loosing our chicken research grant because the things the students (Masters and Doctoral candidates) want to study are things the farming/processing industries aren't interested. It's not necessarily they are afraid the study/ies would reveal aspects of the industries they don't want risking their profits, simply that they want the researchers to research something else they think will benefit their companies.
So they're taking their business to our rival uni.
Because that's exactly what it is: a business. That's the one that always gets my goat.
Great post, very informative, concise, and understandable in laymans' terms.
Hm, good point. I took stat and a few other classes like auditing and risk management oddly, that should have already ingrained this in my head. My problem isn't even reading the headlines and taking them for face value. I generally take all health news with a grain of salt. My problem is I listen to Dr.Oz and who knows where he gets his info.
good points.
Going off the point of money, studies that are sponsored by a particular brand (Kotex funding a tampon study or Kashi funding a study about granola bars, for some very basic examples) are going to be more likely to be biased.
Also, check to see what types of data are being published (i.e. mean, median, mode) because if used incorrectly, they can skew results.
Honestly, whenever I go back and look at the original studies, I find that the biggest problem is usually in the conclusion or the interpretation of their results. Sometimes, there's nothing seriously wrong with their sample size, controls, or study design, etc, but they just make conclusions that really aren't appropriate given their results and a lot of researchers make huge logical leaps in order to make their research "important." In my field, I run into this all of the time when trying to read literature for my own research.
It's bad, bad, bad. Everything in the popular media should DEFINITELY be taken with a grain of salt.
I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
Thank you for spending some time to talk about this, I'm strongly about it and love learning read more about this topic.