Information on Research Studies
A Resource for Service Members and Their Families
Each day you most likely hear about recent medical research findings from various sources. Newly learned information can be interesting and exciting, confusing or even frightening. How do you judge which findings are important or relevant to you? Here are some questions that you can consider.
Should I pay attention to this new research finding?
It can be hard based on a short news presentation to decide if a new research finding has personal meaning for you.
Studies done with people similar to yourself are likely to be of most importance to you - but this information may
not be known when you first hear or learn of the new finding. If the information is provided, the study is
potentially more important to you if the study was done:
- with people similar to you
- if the [good or bad] result is a big enough difference to make you think twice about a lifestyle change or treatment
- if it's a change or treatment that you would be able to do
Even if this type of information is not available, you may still feel that the study (or information) is personally meaningful or it may be that the news report left you concerned. If you find you're left with a lot of questions, it is important to ask your doctor or a knowledgeable person for more information.
Why do results of multiple studies about the same issue have different findings?
It is important to remember that the scientific process is a road of discovery. It is very unlikely that one study
can answer a question completely. This is usually why multiple studies on a given topic will be done. In this
way, scientists can tease out what the findings and implications are and under what conditions or situations it
might be true or not true. For example, it could be that a finding was true in only one situation and another
study shows that by modifying the situation you get a different result. Published "research studies" should be
viewed as discussions among scientists. In these discussions, almost no one gets to have the final word, as it is
rare that a single study provides a final complete answer. In fact, with the benefit of new information or
technology, a scientist sometimes sees old results in a new light. For example, consecutive research studies might
reveal that bypass surgery does not prevent future heart attacks in the general population. However, other studies
on bypass surgery might reveal that for a certain population, such as a group of people with diabetes, some types
of bypass surgery do prevent future heart attacks. Based on characteristics of the population studied, findings
may be different. Ask your doctor or some knowledgeable person if you are unsure of how a research finding fits in
with other existing research and current knowledge.
How believable is the study?
Research studies can be different in terms of the quality of the study. Some are "stronger" than others and their
results can be more trusted. It is not important that you know all the details of the study, but it is important
to get some estimate of how "good" the study is or how much do experts trust the results. The quality of a study
is based on a variety of factors. Here are a few that you may want to keep in mind
- Design of the study: Usually one can find more information in a study that uses comparison groups (subjects that are treated exactly alike but do not receive the intervention or question under study). If subjects are assigned to the groups randomly (with equal chance of getting into either group, like based on the flip of a coin) the study findings may be more meaningful because everyone in the study had an equal chance of being in either group. Alternatively, if there is only one group under study, it is important to look at changes across time and ask about what type of people are in that group.
- Size of sample: If a large group of people were in the study the same outcomes may be found if the study was repeated. If it was a very small group of people studied, there are more chances that it is something particular to that specific group of patients that affected the results. Finally, a larger study where a greater percentage of the people who started the study completed the entire study, in other words, few people dropped out, also increases a study's strength and credibility.
- Length of follow-up: If it was a treatment being studied, it is important to know how long that outcomes of the treatment lasted for. In general the longer people are followed; the more one can trust the conclusion.
- Blindness/Minimize Bias: Sometimes researchers can unintentionally be more careful with the group they want positive outcomes from or unintentionally treat groups differently. Stronger studies try to limit the information given to study participants and those responsible for measuring outcomes or administering treatments so that the participants and experimenters can not unintentionally influence the outcome of the study just because they know what the scientists expect to happen.
What was the population that the study was conducted on?
It is important to ask how similar the people in the study are to you. A study can look at people with a range of
symptoms and backgrounds or only include a narrowly defined group of people. Studies may use only a very select
type of person - so maybe it's only people older than 60 or people with high blood pressure and chronic pain.
Some medical research starts with studies done on animals. If findings are positive from that, future studies may
be done on humans. If the study is with animals or with people who are different from you, the results may not
apply to you. It is important to ask how likely it that the results would occur for different people is.
How should I interpret the numbers?
Deciding what the numbers mean is difficult and may in part depend on the choice of numbers used to present the
results Sometimes numbers are presented without a comparison. So a news article may say that there was a 300% or
3-fold increase in strokes. While at first this might seem like a high number, it might make you feel differently
if you knew more information. Maybe it was an increase from 2 to 6 people out of 10,000 having strokes. Risk
information can also mean more to you depending on whether you are given a full number or a percentage (absolute
difference or relative difference). If you were told that Sydney has 30% more than Pat, would that mean something
different than if you were told that Sydney has $30,000 more than Pat. If you were given the account balances of
Sydney having $130,000 and Pat having $100,000 would that give more information? So to interpret information about
a study it may help to ask your doctor or someone who has reviewed the full the study to give you more numbers or
to present the numbers in a different format. Finding out how meaningful the numbers are is important as well.
What else does the study show?
Sometimes a very interesting finding is highlighted but side-effects or secondary outcomes are not mentioned. To
make an informed decision it is important to know as many of the related facts as you can find. This is similar
to the need to understand both the benefits and risks when making a medical decision.
Conclusions:
In conclusion it is important to think critically about research findings. One should consider not only the
strength of a study result but also whether it is relevant to you as an individual. Personal decisions about the
implications of the study may focus on two primary questions:
- How big or meaningful is the potential benefit compared to the potential risk?
- How do the implications of the study fit into your preferences and values?
Adapted from: National Institute on Aging- Understanding Risk: What do those headlines really mean? International Food Information Council (IFIC) Foundation - IFIC Review: How to understand and interpret food and health-related scientific studies.
Click here for more information on research studies.
Click here to view current research studies at the WRIISC.
This document was developed by the War Related Illness
and Injury Study Center (WRIISC)
VA-New Jersey Health Care System in East Orange, NJ
Last Updated December 16, 2004.