Chapter 2.  Methodology

Recruitment of Experts

	We identified a core group of four clinically oriented technical experts who provided extensive input throughout the project.  The group included two representatives from our principal partner, the American Academy of Ophthalmology (AAO).  The core group of technical experts were:  Paul Lee, MD, JD; Stephen Obstbaum, MD; Joseph Caprioli, MD (AAO representative); and Ronald L. Gross, MD (AAO representative). 

	Representatives from a range of other stakeholder organizations including the American Glaucoma Society, the American Optometric Association, the American Society of Cataract and Refractive Surgery, the Eyes and Vision Group of the Cochrane Collaboration, the Glaucoma Research Foundation, the Health Care Financing Administration, and The Glaucoma Foundation were identified as peer reviewers.  The representatives included two glaucoma patients.  A list of these peer reviewer representatives is attached as Appendix A.

Patient Population

	The target population addressed by this evidence report is adult patients (age 18 and older) with coexisting cataract and glaucoma.  Both open-angle and primary closed-angle glaucoma were considered.  The target audience is health care professionals involved in the care of adult patients with coexisting cataract and glaucoma.

Identifying the Specific Questions

	The study team devised an initial list of potential questions.  The preliminary list of questions was discussed with ophthalmologists external to the study team to add other relevant questions.  Following these discussions, the study investigators prepared a letter which included a list of the potential questions regarding the treatment of coexisting cataract and glaucoma (Appendix B).  Through the use of this letter, each technical expert and peer reviewer was asked to rate the importance of each proposed question from the perspective of his/her own profession or organization.  Based on the responses to the letter and a conference call with the technical experts, consensus was reached regarding the final questions to be addressed by this evidence report.  The following questions are addressed in the evidence report:

	1.	What are the risks and benefits of staged versus combined procedures for coexisting cataract and glaucoma?

	2.	What is the effect of cataract surgery on short- and long-term intraocular pressure control (IOP) in open-angle glaucoma patients?

	3.	What is the effect of glaucoma surgery (i.e., filtration surgery) upon the development and progression of cataract?

	4.	What are the risks and benefits of anti-fibrosis agents in combined cataract and glaucoma procedures?

	5.	What are the risks and benefits of trabeculectomy versus endoscopic laser versus deep sclerectomy/viscocanulostomy for coexistent cataract and glaucoma?

	6.	What are the risks and benefits of single- versus two-site operations for coexistent cataract and glaucoma?

	7.	What are the risks and benefits of nuclear expression or phacoemulsification in patients with glaucoma and cataract?

	Studies reporting short-term outcomes were defined as those whose reported followup was 24 hours.  Studies reporting long-term outcomes were defined as those reporting followup of 6 months or greater in length.

Causal Pathway

 	A model of the causal pathway for treatment of coexisting cataract and glaucoma was constructed (Figure 1.).  Patient outcomes are influenced by all the factors shown.  Provider factors such as subspeciality and geographic location, patient factors such as age and severity of ocular disease, and health system factors all influence the management strategy chosen for treating coexisting cataract and glaucoma.  In turn, these factors all influence patient outcomes.

Literature Search Methods

Sources

	A number of literature sources were used in order to identify all studies that were potentially relevant to the study questions examined in this evidence report.  Preliminary searches were completed on PubMed, the web-based access to MEDLINE provided by the National Library of Medicine.  The results of the searches were used in refinement of the questions and formed the basis for the database of citations used for this evidence report.

	The searches were conducted using two electronic databases, PubMed and CENTRAL, the Cochrane Collaboration's database.  The electronic searches were augmented by a hand search of five ophthalmology journals identified by the principal investigators as high priority (Appendix C).  The hand search included articles published through April 2000.

Search Terms and Strategies

	A core search strategy was developed and was used to search both the CENTRAL and PubMed databases.  For the PubMed searches, the core search strategy was combined with an optimal search strategy for the identification of controlled clinical trials.4  Text words and the Medical Subject Heading (MeSH) term “case report” were added to the search strategy to allow for the inclusion of case studies.  Because case studies are a source of information on reported complications of surgery, we felt it was important to include them in the literature review.  Multiple searches were completed.  The first search of PubMed was completed November 22, 1999.  The strategies used in the searches are included in Appendix D.  

Abstract Review

	The results of all searches were downloaded or hand entered into a database of studies that were considered potentially relevant to the literature review.  The abstracts of all identified citations were independently reviewed by two members of the study team to determine if they met eligibility criteria.  An abstract review form was developed based on the abstract review form used in the Evidence Report on Management of New Onset of Atrial Fibrillation5 and the Evidence Report on Anesthesia Management During Cataract Surgery.6  A copy of the abstract review form may be found in Appendix E.  All the citations not meeting the eligibility criteria were coded with a reason for exclusion.  For those articles without an abstract or where the reviewer could not determine whether the abstract met the eligibility criteria, a full copy of the article was obtained and reviewed.  Disagreements between reviewers about eligibility were adjudicated at a meeting of the study team.  A database, developed using ProCite software, was used to facilitate the tracking of search results, the abstract review process, and the printing of reports regarding the relevant literature.  It is possible that the contents of an abstract may not accurately convey the findings of the published paper.  In order to insure that pertinent articles were not wrongly excluded based on review of the abstract, we reviewed the full article for a five percent sample of rejected abstracts.  The methods used for this review were the same as for the abstract review.  Results of this review confirmed that all (100 percent) were not eligible for inclusion in the full article review.

	The following criteria were applied at the abstract review phase to exclude articles from further consideration: 

	1.	did not include human data

	2.	did not include adults as part of study population

	3.	had no original data

	4.	did not address open-angle glaucoma or primary angle closure glaucoma

	5.	was neither a controlled trial nor a case series greater than or equal to 100 eyes

	6.	only addressed intracapsular cataract extraction

	7.	only addressed full thickness glaucoma surgery

	8.	was a meeting abstract

	9.	was not in English.

	We chose to exclude case series with less than 100 eyes because we felt they were unlikely to include sufficient data about complications.  We did not include foreign language articles because we did not have the resources or the personnel to search, retrieve, and translate them.	

	The abstract review was also used to identify which of the following study topics each article addressed: development of cataract after glaucoma surgery; acute control of IOP after cataract surgery in glaucoma patients; long-term control of IOP after cataract surgery in glaucoma patients; and preferred surgical technique.

Article Review

Selection of Articles

	Each article identified by the search process, and determined to meet eligibility criteria through the abstract review process, was reviewed by two reviewers.  At least one of the reviewers was an individual trained in research methodology and at least one (and usually both) trained in ophthalmology or internal medicine.  In addition to Drs. Bass, Friedman, Jampel, and Lubomski, four ophthalmologists assisted in the review process: Dr. Nathan Congdon, M.D., M.P.H, an ophthalmologist trained in study methodology; Dr. Hana Levkovitch-Verbin, M.D., an ophthalmologist; Dr. John Kempen, M.D., M.P.H., an ophthalmologist trained in study methodology; and Dr. Harry Quigley, M.D., an ophthalmologist.

Abstraction of Qualitative and Quantitative Data

	Two data abstraction forms were developed and extensively pilot tested and revised prior to their use.  The quality assessment form is included as Appendix F and the article content assessment form is included as Appendix G.  

	The quality assessment form included the same exclusion criteria included on the abstract review form, and questions designed to assess the quality of the study. The form was developed based on the form used in the Evidence Report on Anesthesia Management During Cataract Surgery.6  The form contained 25 questions which where grouped into the following categories: representativeness of study population - how completely the authors described the study subjects; bias and confounding - including completeness of randomization and masking; description of therapy - completeness of the description of the study protocol and other treatments (if applicable); outcomes and followup - which includes explicit description of the outcomes reported (e.g., assessment of IOP, cataract assessment) and length of followup; and statistical quality and interpretation.  Each response to a question was assigned a value from zero to two.  The score for each category of study quality is the percentage of the total points available in each category.  The overall quality score is the average of its five categorical scores.  Quality assessment was completed for controlled trials and cohort studies only.

	The article content assessment form was developed through an iterative process.  The study team discussed which outcomes should be included on the form. Following these discussions,  selected relevant articles were reviewed to create a standardized abstraction of reported outcomes in a manner that was applicable across the range of studies. Finally, the form was pilot tested on sample manuscripts and revised as necessary.  Instructions for the use of the form were developed by consensus for the purpose of consistency.  The article content assessment form was completed for all articles meeting eligibility criteria.

	The article review was conducted serially. Each article was assigned a primary reviewer.  The primary reviewer performed the data abstraction and completed the data abstraction forms.  The secondary reviewer, using the data abstraction forms completed by the primary reviewer, reviewed the data abstraction.  Differences between the reviewers were reconciled by the two abstracters in a conference to reach consensus.  If a consensus could not be reached, the differences were discussed and reconciled at a meeting of all of the study investigators.  The reviewers were not masked with regard to author, institution, or journal.  

	A relational database was developed to collect, maintain, and analyze the data abstracted during the article review phase.  This database was subsequently used for construction of evidence tables.

Evidence Tables

	Three summary evidence tables (Evidence Tables 1, 2, and 3) were constructed to present data retrieved respectively for: overall description of each study, summary description of the interventions, and the methodologic quality of the studies.

Evidence Table 1: Overall summary of literature on treatment of coexisting cataract and glaucoma.  

	This table includes information on the source of the article, study design, main study aim, sample size (number of eyes), proportion of African-American patients, mean patient age, principal outcomes reported, mean length of study followup, study limitations, overall quality score, and a concluding statement of each study's results.  

Evidence Table 2: Summary of interventions reported in the literature on treatment of coexisting cataract and glaucoma. 

	This table includes information on the source of the article, the study treatment(s), location of conjunctival incision, description of cataract surgery, description of trabeculoplasty, description of endoscopic laser, concentration and duration of mitomycin-C administered, concentration and duration of intraoperative 5-fluorouracil administered, and dose and number of injections of postoperative 5-fluorouracil received.

Evidence Table 3: Methodologic quality of literature on treatment of coexisting cataract and glaucoma.  

	This table includes information on the source of the article, exclusion criteria, scores on the study quality categories, and the overall study quality score.  Scores have been calculated for randomized controlled trials, non-randomized controlled trials, and cohort studies only.  Studies of case-control or clinical series design are therefore not listed in this table.

	The remaining nine evidence tables present synthesis of the data, based on the organization of the abstracted data around specific individual questions.  The specific questions, and the evidence table in which they are addressed, are presented in Appendix H.  For example, data concerning the effect of cataract surgery (with or without other procedures/treatments) on short-term IOP control in open-angle glaucoma are presented in Evidence Table 5.  Manuscripts that provide information relevant to this question were grouped together in the evidence table along with their principal results.  A narrative description of the principal findings for each individual question is provided in Chapter 3.

Grading the Evidence

	Evidence grades were assigned to statements comparing alternative techniques (e.g., IOP control using mitomycin-C agents versus no mitomycin-C, or IOP control following combined phacoemulsification/trabeculectomy versus trabeculectomy alone).  The summary for each relevant section ends with a “conclusion” statement and the evidence grade.  The evidence underlying the concluding statement was independently reviewed by Drs. Bass, Congdon, Friedman, Jampel, Kempen, Lubomski, and Quigley.  The method used to grade the evidence is based on the method used in the Evidence Report on Anesthesia Management During Cataract Surgery.6  That scheme was based on a modification of the grading scheme employed by Garbutt and colleagues.7  The following strategy was used to grade the strength of the evidence.

	Grade A (strong):  Appropriate data are available for evaluating efficacy/safety of the interventions in question, including at least one well done randomized controlled trial; overall, the population of patients studied is sufficiently large, and adequate controls have been used; data are consistent and indicate that the efficacy/safety of the intervention have been accurately described and that one intervention is clearly superior, equivalent, or inferior to another in terms of one or more well-defined outcomes.

	Grade B (moderate):  Appropriate data are available for evaluating safety/efficacy of the interventions in question; overall, the population of patients studied is sufficiently large and adequate controls have been used; data are reasonably but not entirely consistent and indicate that the efficacy/safety of the intervention have been accurately described; the data indicate superiority (or equivalence) of one intervention over another for specific, well-defined outcomes; however, there is insufficient evidence to conclude definitively that the efficacy/safety of one intervention are clearly superior or inferior to another.

	Grade C (weak): Some data are available for evaluating safety/efficacy of the interventions in question; overall, the population of patients studied is reasonably large; data indicate a trend supporting a benefit (or equivalence) of one intervention over another for specific, well-defined outcomes; however, there is insufficient evidence to conclude that one intervention is truly superior (or equivalent to) to another.

	Grade I (insufficient):  Appropriate data are not available or there is an insufficient number of patients studied to assess safety/efficacy of the intervention in question alone or in comparison to alternatives.

	Disagreements between reviewers regarding the evidence grade were discussed and reconciled at a meeting of all of the participants in the evidence grading process.  The initial evidence grades assigned by the participants may be found in Appendix I.  The grades resulting from the adjudication process may be found at the conclusion of the relevant section of Chapter 3.

Meta-analysis

	Based upon our completed review of the literature (as described in Chapter 3:  Results) the scientific assumptions underlying the justification for a meta-analysis were not met.  Most importantly, studies were sufficiently heterogeneous with regard to study aims, content, design, and the way in which the outcomes were measured, to prevent pooling of the data in a meaningful or acceptable fashion.  

Supplemental Analyses

	No supplemental analyses were completed as a part of this project.