Can you believe what you read in the papers?

Abstract

The number of reports of clinical trials grows by hundreds every week. However, this does not mean that people making decisions about healthcare are finding it easier to obtain reliable knowledge for these decisions. Some of the information is unreliable. Systematic reviews are helping to resolve this by bringing together the research on a topic, appraising and summarising it. But the quality of these reviews depends greatly on the quality of the studies, and this usually means the quality of their reports. If there are fundamental flaws within a study, such as the use of inappropriate 'randomisation' techniques in the context of reviews of the effects of interventions, the reviewers will not be able to fix these. Worse still, if they are not aware of underlying flaws, they might make incorrect judgements about the quality of the research in their review. A study by Wu and colleagues of 'randomised trials' from China provides a reminder of the cautious approach needed by users of scientific articles. They contacted the authors of more than 2000 research articles, which purported to be reports of randomised trials; and concluded that ten of every 11 studies claiming to be a randomised trial probably did not use random allocation. Better education of researchers, peer reviewers and editors about what is, and is not, a properly randomised trial is needed; along with better reporting of the details for how participants were allocated to the different interventions. Systematic reviewers must be cautious in making assumptions about the conduct of trials based on simple phrases about the trial methodology, rather than a full description of the methods actually used. It's not that you can't believe anything that you read in the papers, just that you cannot believe everything.