Publication bias

Publication bias - the tendency that studies that are easy to locate are more likely to show "positive" effects - is an important concern for systematic reviews; a useful review of this subject can be found elsewhere.15 Publication bias results from allowing factors other than the quality of the study to influence its acceptability for publication. Several studies have shown that factors such as sample size, direction and statistical significance of findings, or investigators' perception of whether the findings are "interesting", are related to the likelihood of publication.16,17 Language bias may also be a problem - studies that are "positive" have a tendency to be published in an English language journal and also more quickly than inconclusive or negative studies.16,17 A thorough systematic review should therefore include a search for high-quality, unpublished trials and should not be restricted to journals written in English. Studies with small samples are less likely to be published, especially if they have negative results.16,17 By emphasising only those studies that have positive results, this type of publication bias jeopardises one of the main goals of meta-analysis (i.e. an increase in power when pooling results of small studies). Creation of study registers and advance publication of research designs have been proposed as ways to prevent publication bias.18,19 Publication bias can be detected by using a simple graphic test (funnel plot) or by calculating the "fail-safe N",20,21 but these techniques are of limited value when fewer than 10 randomised controlled trials are included. In addition, for many diseases, the studies published are dominated by pharmaceutical-company-sponsored trials of new expensive treatments. This bias in publication can result in data-driven systematic reviews that draw more attention to those medicines. In contrast, question-driven systematic reviews answer the sorts of clinical questions of most concern to practitioners. In many cases, studies that are of most relevance to doctors and patients have not been done in the field of dermatology because of inadequate sources of independent funding. Systematic reviews that have been sponsored directly or indirectly by industry are also prone to bias by over-inclusion of unpublished "positive" studies that are kept "on file" by that company. Until it becomes mandatory to register all clinical trials conducted on human beings in a central register and to make all of the results available in the public domain, all sorts of distortions may occur as a result of selective withholding or release of data.

Generally reviews that have been conducted by volunteers in the Cochrane Collaboration are of better quality than non-Cochrane reviews, but even despite this, potentially serious errors have been noted in up to a third of such reviews.1,7

0 0

Post a comment