Overview of methods used in reviews

 Search strategies

Cochrane Register of Studies

The Eyes and Vision Group's Trials Register is maintained in the Collaboration’s software, the Cochrane Register of Studies (CRS). Currently there are 18757 reports of RCTS and CCTs in the Register. The Register is updated from two sources: searching the Cochrane Central Register of Controlled Trials (CENTRAL) and handsearching. CENTRAL receives a monthly download from MEDLINE of reports of controlled and randomised trials. The TSC runs a search strategy on CENTRAL which is designed to identify all eye and vision related reports of CCTs and RCTs. These reports are screened for relevance and then added to the Group’s register. Handsearching (ophthalmology and optometry journals and conference proceedings) is undertaken by our editorial base satellite in the US. Handsearching is a useful tool to identify reports of CCTs and RCTs in publications not indexed in the major databases and for conference proceedings which are not widely disseminated. Also handsearching is extremely effective in identifying CCTs and RCTs that have been missed by electronic searching alone.

The TSC searches the Eyes and Vision Group's Trials Register for each review and also searches MEDLINE, EMBASE and the following ongoing trials registers as standard: ISRCTN, clinicaltrials.gov and the ICTRP. Thus, aiming to have a broad coverage of information sources to enable us to identify as many studies as possible that may be eligible for inclusion in a review.

Additional search strategies

We recommend that authors:
(1) search the reference lists in trial reports and in existing reviews relevant to their topic area for citations to additional trials;
(2) use the Science Citation Index to identify articles that have cited the studies that are included in the review;
(3) search other databases that they have access to.

Study selection

We recommend that:
(1) authors restrict their reviews to randomised controlled trials. For certain topics, this may be too restrictive and authors may then include quasi-randomised studies. Where necessary, non-randomised studies should be referred to in the background and discussion sections of the review. This should be discussed with a member of the editorial team;
(2) selection of studies for inclusion in a review should be done independently by more than one review author;
(3) differences in study selection should be resolved by discussion and the disagreement documented briefly in the results of the review if relevant.

Assessment of risk of bias

We recommend that:
(1) risk of bias is assessed independently by more than one review author where possible. The level of agreement may be reported in the review;
(2) differences in risk of bias assessment between authors should be resolved by discussion and reported in the review if relevant.

Data collection

We recommend that:
(1) extraction of data from trial reports is performed independently by more than one author;
(2) differences in data interpretation between review authors are resolved by discussion and reported in the review if relevant;
(3) where possible, data are verified with the person responsible for the primary study.

Analysis

We advise our authors to follow the guidance provided in the Cochrane Handbook for Systematic Reviews of Interventions. If authors decide to deviate from these, an explanation should be included in the review. Ideally, data should be entered into RevMan 5 using a method to minimise transcription errors- for example, cutting and pasting from a spreadsheet. One author should put the data in RevMan and another author check that it is correct in the review. 

Assessment of heterogeneity
We recommend that authors check for heterogeneity by examining:
(1) the characteristics of the study;
(2) the forest plot of results of the studies;
(3) the results of the Chi2 test for statistical heterogeneity;
(4) the results of the I2 test.

If significant heterogeneity is detected, either by the Chi2 test or by observation, we recommend that authors should not combine results but present a descriptive summary of results. Review authors may explore heterogeneity by examining subgroups, which should be stated a priori in the protocol. If post-hoc subgroup analyses are performed these should be identified as post-hoc in the review.

Measure of effect
For dichotomous data, we recommend calculation of the risk ratio or odds ratio. If the outcome is common it may be preferable to use the risk ratio for ease of interpretation. For continuous data, we recommend calculation of the mean difference. If the outcome has been measured using different instruments, but they are similar enough to be combined, we recommend presentation of the standardised mean difference.

We recommend that the point estimate and 95% confidence intervals (CI) are reported. 

Fixed-effect versus random-effects models
Techniques for calculating summary measures can be divided into fixed-effect and random-effects. There is considerable debate about the use of these two models. We recommend the following:

If there is a small number of trials in the analysis (three or fewer) the fixed-effect model may be appropriate, provided that heterogeneity has not been detected either statistically or by review. If the number of trials is greater than three and no heterogeneity has been detected we recommend the random-effects model. If heterogeneity has been detected it may be inappropriate to combine results to produce a single summary measure. In this case we recommend that review authors look at their forest plot and describe what the plot shows in the results section of the review.

Subgroup analysis
We recommend against subgroup analyses unless these are clearly stated a priori. If post-hoc subgroup analyses are performed these should be identified as post-hoc in the review.

Sensitivity analysis
We recommend that sensitivity analyses should be conducted to assess how robust review results are to key decisions and assumptions that were made during the review.

We recommend repeat analysis with the following adjustments:
(1) exclude studies at higher risk of bias
(2) exclude unpublished studies;
(3) exclude industry-funded studies

A note on graphical display or forest plots. It can be useful to enter data into RevMan to calculate the relevant measures of effect. We recommend that forest plots with two or more trials only should be included in the final publication. Forest plots that contain only one single trial should be deleted prior to publication and the results reported in the text, or tables, only.

Problems specific to ophthalmic studies
Ophthalmic papers can be difficult to review. People have two eyes and different studies deal with this in different ways. Some studies treat both eyes, others treat a single eye and either ignore the fellow eye or just observe it. Other studies assign one eye of an individual to treatment and assign the fellow eye to placebo or another intervention; this is known as a within-person study. It is not always evident from a paper whether eyes or participants are being analysed and review authors may need to contact study authors in order to clarify what has been done.

It may be necessary to consider laterality - it is plausible that some diseases may develop first in the left eye. We suggest that authors never assume that two eyes in an individual are independent data. Where investigators have made this assumption, we advise that review authors flag this in the review. Since RevMan does not have the resources for dealing with paired data, data for these studies should not be entered into RevMan. Authors

Reporting of reviews

See the Cochrane Handbook for Systematic Reviews of Interventions.

Review authors are advised to follow the style guidelines set out in the help section of RevMan 5.

Abstracts
Cochrane reviews are indexed on MEDLINE. It is important, therefore, that abstracts are well structured and contain relevant information. Guidelines on how to structure abstracts are available from the editorial base.

Editorial process

Titles

Details of our title registration process can be found here.

Protocols

Review authors are expected to write their protocols in RevMan (Cochrane software) and save their drafts in Archie.

When a first draft of the protocol is submitted to the editorial base, the Managing Editor will review the work, copy edit it and liaise with the Contact Person if further amendments need to be made. The protocol is then sent to the Contact Editor (if one has been assigned) or the Co-ordinating Editor for approval prior to peer review. If further amendments need to be made, the Managing Editor will liaise with the Contact Person. When the first draft of the protocol is submitted, the Trials Search Co-ordinator (TSC) reads the draft, contacts the review team to discuss search terms and creates search strategies for the electronic databases. The TSC also checks and edits the references at this point.

When the draft is ready for peer review, it is sent to the Statistical Editor for the Group and usually two content experts. If a consumer person has been identified, the protocol will also be sent to this person. The peer review process usually takes between four to six weeks.

All peer review comments are reviewed by the Managing Editor and Contact Editor or Co-ordinating Editor. The comments are then collated in to an Authors' Response document and sent to the Contact Person. Peer referee names are disclosed to the review teams unless anonymity has been specified by the peer referee. Peer referees are acknowledged in the protocol as well as on the Group's module.

Once the review team has responded to the peer review comments, the amended protocol is submitted to the editorial base and the Authors' Response Document emailed to the Managing Editor. The Managing Editor will check that all comments have been responded to and copy edit the protocol before requesting approval for publication from the Contact Editor (if one has been assigned), Co-ordinating Editor or both.

The protocol will also be sent off for copy editing by the Wiley copy edit team before submission for publication in the Cochrane Database of Systematic Reviews.

All members of the review team will be sent a License for Publication form and a Declarations of Interest form for completion through their Archie accounts. Signed copies from each member of the review team must be received by the editorial base before the protocol can be released for publication.

CEVG expects a first draft of the review to be submitted to the editorial base six months after publication of the protocol and the completed review to be published within one year of publication of the protocol. If these deadlines are not met and no reason for delay has been communicated to the Managing Editor, the protocol will be withdrawn from The Cochrane Library and a new review team identified to complete the review.

Reviews

Once the protocol has been submitted for publication, the TSC will run the electronic searches and send the search results to the review team to assess.

When a first draft of the review is submitted to the editorial base, the Managing Editor will review the work, copy edit it and liaise with the Contact Person if further amendments need to be made. The review is then sent to the Contact Editor (if one has been assigned) or the Co-ordinating Editor for approval prior to peer review. If further amendments need to be made, the Managing Editor will liaise with the Contact Person.

Before the draft is submitted for peer review, the TSC will read the draft, write the search sections of the review and include a PRISMA diagram for the studies identified in the electronic searches.

When the draft is ready for peer review, it is sent to the Statistical Editor of the Group and usually the same content experts that were used at the protocol stage. If a consumer person was used, the review will also be sent to this person. If a consumer was not identified at protocol stage we will check to see if there is one available for the review. The peer review process usually takes between four to six weeks.

All peer review comments are reviewed by the Managing Editor and Contact Editor or Co-ordinating Editor. The comments are then collated in to an Authors' Response document and sent to the Contact Person. Peer referee names are disclosed to the review teams unless anonymity has been specified by the peer referee. Peer referees are acknowledged in the review as well as on the Group's module.

Once the review team has responded to the peer review comments, the amended review is submitted to the editorial base and the Authors' Response Document emailed to the Managing Editor. The Managing Editor will check that all comments have been responded to and copy edit the review before requesting approval for publication from the Contact Editor (if one has been assigned), Co-ordinating Editor or both.

The review will also be sent off for copy editing by the Wiley copy edit team before submission for publication in the Cochrane Database of Systematic Reviews.

All members of the review team will be sent a License for Publication form and a Declarations of Interest form for completion. Signed copies from each member of the review team must be received by the editorial base before the review can be released for publication.

Disagreements between the editorial base and the review authors, or between the authors themselves, at any point during the review process should be resolved by discussion, with arbitration from the Co-ordinating Editor.

Updating

CEVG expects updates to the review to take place within two years from the date of publication. Any review published in The Cochrane Library that has not been updated in The Cochrane Library within 10 publication cycles of the first publication of the review will have their complimentary copy of The Cochrane Library withdrawn by the publishers until such time as their review has been updated.

For any review that needs to be updated, the TSC re-runs the electronic searches and sends the search results to the review team three to six months before the updated review needs to be submitted for publication. If no new trials are identified from the search, and no changes to the review are made by the author team or editorial base, this is indicated in the review when updating. If new trials are added and the conclusions of the review change, the updated review will be sent off for peer review.

If a review in the Cochrane Database of Systematic Reviews becomes out of date in an important way we will write to the author concerned requesting that they update the review and give a date for submission of the update. If the author fails to update the review within two years of the last update, we will remove the review from the database until the update has been completed.

Inclusion criteria

The Eyes and Vision Group's Trials Register contains reports of trials of therapeutic interventions for the ophthalmic diseases and disorders listed in the topics section. The register also contains trials of interventions to aid rehabilitation from visual impairment or blindness and trials on diagnostic interventions.

The register contains reports of randomised controlled trials (RCTs) and controlled clinical trials (CCTs). The following criteria are used to assess reports of studies for inclusion in the register:

Randomised controlled trials - studies in which there is an explicit statement that an intervention is allocated at random.

Controlled clinical trials - studies that state that an intervention is masked or placebo controlled but does not mention randomisation. Studies that use quasi random methods of allocation such as date of birth, alternation, hospital record number etc.