Reviews in Health Care 2011; 2(3): 195-202
Letter to the editor
How to report a review?
Come si scrive una review?
Giuseppe La Torre 1, Brigid Unim 1
1 Department of Public Health and Infectious Diseases, Sapienza University of Rome, Italy
Disclosure
The authors declare that they have no financial competing interests
Dear Editor,
In the editorial from Maina and Di Napoli [1], the important issue concerning the number of authors of a review was considered in its essential aspects. The authors stated that, in order «to avoid the uncontrolled proliferation of authors and co-authors», Reviews in Health Care decided to fix a maximum of 10 authors for each article.
In this contribution we would like to underline another issue: how to report a review?
The need of specific reporting guidelines for medical journals is beyond dispute. With that regard a number of statements were developed by groups of experts to facilitate reporting of research studies. Most medical journals, including British Medical Journal (BMJ), the Journal of the American Medical Association (JAMA), The Lancet, and New England Journal of Medicine, often require compliance to all or some of the following guidelines: CONSORT Statement (reporting of randomized controlled trials), STARD (reporting of diagnostic accuracy studies), STROBE (reporting of observational studies in epidemiology), PRISMA (reporting of systematic reviews and meta-analyses) and MOOSE (reporting of meta-analyses of observational studies).
The present letter concerns the standards of reporting in research, publication in medical journals and wishes to give an emphasis to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement.
There is no doubt that, according to the PRISMA statement [2], the number of authors for a systematic review needs to be at least 2 (“Eligibility assessment to be performed independently in an unblended standardized manner by 2 reviewers”).
PRISMA is an evidence-based set of 27-item checklist (Table I) and a four-phase flow diagram (Figure 1) and it’s basically an expansion of the QUOROM Statement developed in 1996 (QUality Of Reporting Of Meta-analyses).
The Statement was developed by a group of 29 review authors, methodologists, clinicians, medical editors, and consumers [3]. The flow diagram originally proposed by QUOROM was modified to show numbers of identified records, excluded articles, and included studies. Items considered essential for transparent reporting of a systematic review were also included in the checklist. After 11 revisions the group approved the checklist and the flow diagram. The items are presented numerically from 1-27, but it is not necessary to address them in that particular order in a report. Fundamentally, what is important is that the information for each item is given somewhere within the report [2].
|
Section/topic |
Item No |
Checklist item |
Reported on page No |
|
Title |
|||
|
Title |
1 |
Identify the report as a systematic review, meta-analysis, or both |
|
|
Abstract |
|||
|
Structured summary |
2 |
Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number |
|
|
Introduction |
|||
|
Rationale |
3 |
Describe the rationale for the review in the context of what is already known |
|
|
Objectives |
4 |
Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS) |
|
|
Methods |
|||
|
Protocol and registration |
5 |
Indicate if a review protocol exists, if and where it can be accessed (eg web address), and, if available, provide registration information including registration number |
|
|
Eligibility criteria |
6 |
Specify study characteristics (eg PICOS, length of follow-up) and report characteristics (eg years considered, language, publication status) used as criteria for eligibility, giving rationale |
|
|
Information sources |
7 |
Describe all information sources (eg databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched |
|
|
Search |
8 |
Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated |
|
|
Study selection |
9 |
State the process for selecting studies (ie, screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis) |
|
|
Data collection process |
10 |
Describe method of data extraction from reports (eg piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators |
|
|
Data items |
11 |
List and define all variables for which data were sought (eg PICOS, funding sources) and any assumptions and simplifications made |
|
|
Risk of bias in individual studies |
12 |
Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis |
|
|
Summary measures |
13 |
State the principal summary measures (such as risk ratio, difference in means) |
|
|
Synthesis of results |
14 |
Describe the methods of handling data and combining results of studies, if done, including measures of consistency (eg I2 statistic) for each meta-analysis |
|
|
Risk of bias across studies |
15 |
Specify any assessment of risk of bias that may affect the cumulative evidence (eg publication bias, selective reporting within studies) |
|
|
Additional analyses |
16 |
Describe methods of additional analyses (eg sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified |
|
|
Results |
|||
|
Study selection |
17 |
Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram |
|
|
Study characteristics |
18 |
For each study, present characteristics for which data were extracted (eg study size, PICOS, follow-up period) and provide the citations |
|
|
Risk of bias within studies |
19 |
Present data on risk of bias of each study and, if available, any outcome-level assessment (see item 12) |
|
|
Results of individual studies |
20 |
For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group and (b) effect estimates and confidence intervals, ideally with a forest plot |
|
|
Synthesis of results |
21 |
Present results of each meta-analysis done, including confidence intervals and measures of consistency |
|
|
Risk of bias across studies |
22 |
Present results of any assessment of risk of bias across studies (see item 15) |
|
|
Additional analysis |
23 |
Give results of additional analyses, if done (such as sensitivity or subgroup analyses, meta-regression) (see item 16) |
|
|
Discussion |
|||
|
Summary of evidence |
24 |
Summarise the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (eg health care providers, users, and policy makers) |
|
|
Limitations |
25 |
Discuss limitations at study and outcome level (eg risk of bias), and at review level (eg incomplete retrieval of identified research, reporting bias) |
|
|
Conclusions |
26 |
Provide a general interpretation of the results in the context of other evidence, and implications for future research |
|
|
Funding |
|||
|
Funding |
27 |
Describe sources of funding for the systematic review and other support (eg supply of data); role of funders for the systematic review |
|
Table I. Checklist of items to include when reporting a systematic review (with or without meta analysis) [2]
Figure 1. Flow of information through the different phases of a systematic review
Systematic reviews (SRs) and meta-analyses are essential tools for summarising evidence accurately and reliably. They help clinicians keep up-to-date; provide evidence for policy makers to judge risks, benefits, and harms of health care behaviours and interventions; gather together and summarise related research for patients and their carers; provide a starting point for clinical practice guideline developers; provide summaries of previous research for funders wishing to support new research [4]; and help editors judge the merits of publishing reports of new studies [5].
Poor reporting of key information is often in SRs and that diminishes their potential usefulness [6]; therefore, PRISMA focuses on ways in which authors can ensure the transparent and complete reporting of their researches. It does not address directly or in a detailed manner the conduct of SRs, for which other guides are available [7,8].
Several research and publication ethics standards have been set up. These include the Nuremberg Code, the Declaration of Helsinki, the International Committee of Medical Journal Editors (ICMJE) Uniform Requirements for Manuscripts to Biomedical Journals, recommendations of the World Association of Medical Editors and the Committee on Publication Ethics [9].
The study carried out in 2010 by Pitak-Arnnop revealed the lack of disclosures of human subject protection (obtaining ethical approval and subject’s consent), financial conflicts, and academic-industry relationship in oral-maxillofacial surgery (OMS) journals and innovations. Funding sources were disclosed in only 26.4% of controlled trials published in OMS journals. Their recent studies demonstrated that 9 of 29 clinical studies (31%) on piezoelectric OMS procedures were dual or fragmented publications in journals of different disciplines or different languages and that OMS authors had a considerably different understanding of research ethics. Multiple factors may contribute to such scientific misconduct. These include inadequate research experience, bias from career self-interest or financial gains, lack of knowledge about research and publication ethics, or a combination of these [9]. Moreover, guidelines for authors are usually limited and inconsistent among different journals [10,11].
Other factors could lead to a bias in evidence, for instance unpublished grey literature or publications in non-electronic journals are difficult to identify and are often not detected in systematic searches. Even though a study is published in electronic journals not every journal is listed on the major databases like PubMed.gov or Embase which hinders its identification. Most of the published studies are not accessible openly and in addition many large clinical centres do not have all the necessary licenses for all the relevant publications which limit the number of studies they can identify [12]. Therefore, knowing that publication bias is always present it’s advisable to take it into account when reading meta-analyses and systematic reviews.
Publication bias occurs when «investigators, reviewers, and editors submit or accept manuscripts for publication based on the direction or strength of the study findings» [13]. The ICMJE underlines that negative studies should be published: «Editors should seriously consider for publication any carefully done study of an important question, relevant to their readers, whether the results for the primary or any additional outcome are statistically significant. Failure to submit or publish findings because of lack of statistical significance is an important cause of publication bias».
The impact of publication bias has been widely examined for clinical trials [14-16], for which it has been suggested that studies with statistically positive results and large effect sizes can exaggerate a treatment’s effectiveness by 20% [17].
The PRISMA Statement calls for an international registry for SR protocols [18], which is under development [19]. An international registry may decrease the number of unpublished SRs and will hopefully decrease redundancy, increase transparency and collaboration within the SR community.
The interpretation of SR results may be improved by using the PRISMA Statement and GRADE (Grading of Recommendations Assessment, Development, and Evaluation) which considers four factors in grading the strength of recommendations: quality, benefit versus harm, values, preferences and resources [20,21]. A categorisation guide for meta-analysis results would be also useful. Such efforts will increase the applicability and relevance of the SR findings and may help to ensure adequate interpretation of the results.
The purpose of this letter is to encourage health care journals and editorial groups, such as the World Association of Medical Editors and the International Committee of Medical Journal Editors, to endorse PRISMA in much the same way as they have endorsed other reporting guidelines, such as CONSORT. We also encourage editors of health care journals to support PRISMA by updating their ‘‘Instructions to Authors’’ and including the PRISMA Web address, and by raising awareness through specific editorial actions.
References
1. Maina S, Di Napoli M. How many authors are needed to write a review? Reviews in Health Care 2011; 2: 83-6
2. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and Elaboration. Ital J Public Health 2009; 6: 354-91
3. Moher D, Liberati A, Tetzlaff J, Altman DG. The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med 2009; 151: 264-9
4. Canadian Institutes of Health Research (2006) Randomized controlled trials registration/application checklist. Available at: www.cihr-irsc.gc.ca/e/documents rct_reg_e.pdf
5. Young C, Horton R. Putting clinical trials into context. Lancet 2005; 366: 107-8
6. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med 2007; 4: e78
7. Atkins D, Fink K, Slutsky J. Better information for better health care: The Evidence-based Practice Center program and the Agency for Healthcare Research and Quality. Ann Intern Med 2005; 142: 1035-41
8. Centre for Reviews and Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in health care. York: University of York; 2009. Available at: http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf
9. Pitak-Arnnop P, Sader R, Rapidis AD, Dhanuthai K, Bauer U, Hervé C, et al. Publication bias in oral and maxillofacial surgery journals: an observation on published controlled trials. J Craniomaxillofac Surg 2010; 38: 4-10
10. Atlas MC. Emerging ethical issues in instructions to authors of high-impact biomedical journals. J Med Libr Assoc 2003; 91: 442-9
11. Sorinola O, Olufowobi O, Coomarasamy A, Khan KS. Instructions to authors for case reporting are limited: a review of a core journal list. BMC Med Educ 2004; 4: 4
12. Knobloch K, Yoon U, Vogt PM. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and publication bias. J Craniomaxillofac Surg 2011; 39: 91-2
13. Dickersin K. How important is publication bias? A synthesis of available data. AIDS Educ Prev 1997; 9: 15-21
14. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991; 337: 867-72
15. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990; 263: 1385-9
16. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998; 279: 281-6
17. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess 2010; 14: iii, ix-xi, 1-193
18. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009; 339: b2700
19. Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. An international registry of systematic-review protocols. Lancet 2011; 377: 108-9
20. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ 2008; 336: 1049-51
21. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336: 924-6
Email this article 
