Psychologists examined more than sixteen hundred online reviews of weight loss pills and high cholesterol treatments on Amazon.com.
They found the reviews portrayed the products in a far more positive light than the clinical trial data would suggest.
And they have warned that customers should be “cautious about using reviews like these when deciding about health choices”.
Reviews and clinical data don’t match up
Dr Mícheál de Barra, from the University of Aberdeen conducted the research,
He found that the average drop in cholesterol using ‘Benecol’ was more than three times larger in Amazon reviews than was found in carefully controlled trials.
Similarly, reviewers on weight loss pill Orlistat lost about twice as much weight (14kg) as those in clinical trials (7kg).
Dr de Barra said: “These treatments are not entirely ineffective.
“However, what we show is that the reputation as described in these reviews is much more positive than the clinical trial data show.”
But Dr de Barra was also keen to stress that these reviews are not a deliberate attempt to mislead.
He said that the reviews appear exaggerated but are perhaps a result of a bias towards sharing positive outcomes rather than negative ones.
He added: “Only some people who try a treatment will then go on to tell other people about their experience.
“However, this subset of people are usually only those who have good outcomes.
“So, you hear a friend of yours had a good result using a treatment of some kind, and you think ‘well maybe this works’.
“Your friend is probably not lying - but the problem is that people with average or poor outcomes don’t tend to share their experiences.
“This means you get a positively skewed view of the treatment.”
Caution required when making health choices
Dr de Barra warned: “We should be cautious about using reviews like these when deciding about health choices.
“These narratives have a powerful influence on our own future health behaviour because they provide simple and clear anecdotes, but this study shows that they can be very misleading.
“These results also shine a new light on medical overuse, the use of treatments that are unnecessary and ineffective.
“Medical overuse is estimated to cost the $226bn in the US alone, and patient demand for medicines with limited value is one important cause.
“This study shows how a demand for ineffective medicines can easily develop when people rely on hearsay and narratives alone.
“Realistically, however, it is ridiculous to think that every health decision we make will be informed by systematic reviews - people have lives to lead.
“This study shows though that it’s important to be aware of the biases that can make informally acquired information unreliable.”
It is the first study to compare clinical trial data with user-generated online reviews and is published in the journal Social Science and Medicine this week.