Lead Author(s): Clay Johnston, MD, PhD
The clinical literature is overwhelming. On any topic important to clinical decision making, there are numerous articles, sometimes with conflicting findings and recommendations. A clinician has been expected to synthesize these articles to create a sense of truth, from which decisions about care can be made, but this expectation is unrealistic given the breadth of the literature and the other demands of providing care. The traditional approach to this problem was to go to an expert review in a journal or textbook. A thoughtful senior practitioner was expected to provide a rational synthesis of the literature and make recommendations on care. However, such reviews are unreliable (disagreements between reviewers) and often inaccurate (failure to capture important results). Informal, traditional reviews tend to focus on the author's experience and research, and cannot be expected to provide an unbiased, balanced, and complete review. Meta-analysis is meant to be a solution to this problem.
Meta-analysis is a systematic and quantitative approach to review and synthesis of the literature on a given topic. The final goal is a complete review of a topic with one final result that captures the data from all the included studies. Although synthesis of results from randomized trials is the easiest and most useful function of meta-analysis, it can also be used to synthesize observational studies. It can declare an answer when one is not apparent from review of the individual studies or it can identify the need for additional studies.
There are four major steps of meta-analysis-identifying the studies to include, defining eligibility of potential studies, abstracting data, and analysis-which are briefly discussed below. From the perspective of a clinical researcher, one of the most appealing things about meta-analysis is that anyone can do it with a little guidance, and they can have major impact. One of my most cited publications is a meta-analysis on risk of stroke with oral contraceptives; a medical student is the first author and this was her first publication (7). For a more detailed discussion that provides adequate guidance to actually do a meta-analysis, see Diana Petitti's excellent book (3).
- Identifying studies to include: One of the most important characteristics of a meta-analysis is its systematic, explicit approach to gathering the literature. Good meta-analysis use a combination of data sources to find articles, including Medline searches with specific MeSH headings and key words, review of bibliographies of key references, review of non-English literature, letters to experts about unpublished studies, and review of key meeting abstracts. The results of the searches are recorded and any potentially important articles are collected. It is not unusual to review 10,000 citations to come up with a final meta-analysis that includes 20.
- Defining eligibility of studies to include: Strict inclusion/exclusion criteria for studies should be established. These may include characteristics of the study design, the population studied, or the outcome measure used. It is best to define these criteria before reviewing results of studies so that the selection of studies to include is completely free of bias. In the best meta-analyses, two independent reviewers then judge whether a given study should be included, with a third adjudicating disagreements.
- Abstracting data: Key information from the studies should be abstracted using strict definitions. These data will include details about the study design, as well as the results. These data will be used to summarize the results and may be useful for doing subgroup analyses. Again, the best studies use two independent abstracters with disagreements adjudicated by a third.
- Analyzing the data: The abstracted data are then synthesized using standard methodologies. These techniques allow aggregation of results from many studies, with weighting based on the size of the study. They also provide a measure whether differences in results between studies are too great to be explained by chance alone (test of heterogeneity), in which case it is best not to provide a single overall summary estimate of effect. There are two major models used to combine results. With fixed-effects models, the assumption is made that all important studies are included. With random-effects models, the included studies are considered a sample of the total studies done (or to be done). Random-effects models tend to have broader confidence intervals and are generally preferred. There are several statistical programs that support meta-analysis. Stata has a particularly nice program for meta-analysis, but we generally use Excel because it is more transparent.
In the end, a meta-analysis is just a review, but it is an excellent source for unbiased information on an important topic. The Cochrane Collaboration provides a collection of meta-analyses that meet numerous quality standards and cover a wide variety of topics (www.cochrane.org ).