Jump to: Page Content, Site Navigation, Site Search,
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
BMJ 2007;334:1257 (16 June), doi:10.1136/bmj.39217.473275.55 (published 4 June 2007)
Ros Collins, research fellow1, Jane Burch, research fellow1, Gillian Cranny, research fellow1, Raquel Aguiar-Ibáñez, research fellow in health economics1, Dawn Craig, research fellow in health economics1, Kath Wright, information officer1, Elizabeth Berry, senior lecturer2, Michael Gough, consultant vascular surgeon3, Jos Kleijnen, director4, Marie Westwood, senior research fellow1
1 Centre for Reviews and Dissemination, University of York, York YO10 5DD, 2 Academic Unit of Medical Physics, University of Leeds, Leeds, 3 Leeds Teaching Hospitals NHS Trust, Leeds, 4 Kleijnen Systematic Reviews Ltd, York
Correspondence to: Ros Collins rc14{at}york.ac.uk
Design Systematic review.
Methods Searches of 11 electronic databases (to April 2005), six journals, and reference lists of included papers for relevant studies. Two reviewers independently selected studies, extracted data, and assessed quality. Diagnostic accuracy studies were assessed for quality with the QUADAS checklist.
Results 107 studies met the inclusion criteria; 58 studies provided data on diagnostic accuracy, one on outcomes in patients, four on attitudes of patients, and 44 on adverse events. Quality assessment highlighted limitations in the methods and quality of reporting. Most of the included studies reported results by arterial segment, rather than by limb or by patient, which does not account for the clustering of segments within patients, so specificities may be overstated. For the detection of stenosis of 50% or more in a lower limb vessel, contrast enhanced magnetic resonance angiography had the highest diagnostic accuracy with a median sensitivity of 95% (range 92-99.5%) and median specificity of 97% (64-99%). The results were 91% (89-99%) and 91% (83-97%) for computed tomography angiography and 88% (80-98%) and 96% (89-99%) for duplex ultrasonography. A controlled trial reported no significant differences in outcomes in patients after treatment plans based on duplex ultrasonography alone or conventional contrast angiography alone, though in 22% of patients supplementary contrast angiography was needed to form a treatment plan. The limited evidence available suggested that patients preferred magnetic resonance angiography (with or without contrast) to contrast angiography, with half expressing no preference between magnetic resonance angiography or duplex ultrasonography (among patients with no contraindications for magnetic resonance angiography, such as claustrophobia). Where data on adverse events were available, magnetic resonance angiography was associated with the highest proportion of adverse events, but these were mild. The most severe adverse events, although rare, were mainly associated with contrast angiography.
Conclusions Contrast enhanced magnetic resonance angiography seems to be more specific than computed tomography angiography (that is, better at ruling out stenosis over 50%) and more sensitive than duplex ultrasonography (that is, better at ruling in stenosis over 50%) and was generally preferred by patients over contrast angiography. Computed tomography angiography was also preferred by patients over contrast angiography; no data on patients' preference between duplex ultrasonography and contrast angiography were available. Where available, contrast enhanced magnetic resonance angiography might be a viable alternative to contrast angiography.
Intra-arterial contrast angiography is regarded as the reference standard. The drawbacks are those associated with arterial puncture, ionising radiation, and potential nephrotoxicity of iodinated contrast agents. Several alternative imaging techniques are available, including magnetic resonance angiography, computed tomography angiography and duplex ultrasonography. These techniques are less invasive than contrast angiography, although computed tomography angiography carries risks relating to ionising radiation, and both contrast enhanced magnetic resonance angiography and computed tomography angiography carry risks associated with the use of contrast agents.
We carried out a systematic review to examine the evidence regarding the performance of magnetic resonance angiography, computed tomography angiography, and duplex ultrasonography as alternatives to contrast angiography to try to identify a technique that is safer and more acceptable to patients but as effective as contrast angiography for the assessment of symptomatic peripheral arterial disease. Here we present the systematic review of the evidence on effectiveness. The full report with economic evaluation is available elsewhere.4
|
We have presented a narrative synthesis for studies evaluating the impact of the method of assessment on management and outcome in patients, attitudes of patients, and studies of adverse events.
|
There is evidence that the availability of appropriate clinical data increases the accuracy of interpretation.8 The availability of clinical data was poorly reported, with only one study that evaluated magnetic resonance angiography and duplex ultrasonography reporting that clinical data were available when the imaging results were interpreted. Full details of included studies and quality assessment are on www.york.ac.uk/inst/crd/projects/peripheralarterialdisease.htm.
Assessment of stenosis/occlusion
Fifty eight diagnostic accuracy studies met the inclusion criteria. One evaluated 2D phase contrast magnetic resonance angiography,w1 11 evaluated 2D time of flight magnetic resonance angiography,w2-w12 14 evaluated contrast enhanced magnetic resonance angiography,w8w13-w25 7 evaluated computed tomography angiography,w26-w32 and 28 evaluated duplex ultrasonography.w4w8w33-w58 No studies evaluated 3D time of flight magnetic resonance angiography. Contrast angiography was the reference standard in all studies. As there was significant heterogeneity between individual studies we did not pool data and have presented results as medians (range).
Most of the included studies reported results by arterial segment, rather than by limb or by patient, which does not account for the clustering of segments within patients. Therefore, the increased number of segments assessed is likely to increase the number of true negative test results, and thus the specificities may be overstated. We report results only for studies where data were reported by arterial segment. Full diagnostic accuracy results are available elsewhere.4
Whole leg
Table 2 shows data for detection of stenosis 50% or more or occlusion.
Figures 2 and 3 show sensitivity and specificity data, respectively.![]()
Contrast enhanced magnetic resonance angiography had the highest diagnostic accuracy (seven studiesw13w16w17w19w21-w23), with median sensitivity 95% (range 92-99.5%) and median specificity 97% (64-99%). One study had a low specificity (64%) compared with the others; this was the only study to include assessment of foot vessels in the scan.w13 2D time of flight magnetic resonance angiography was less accurate (five studiesw2w6w7w10w12), with median sensitivity 92% (79-94%) and median specificity 88% (74-92%). The use of time of flight magnetic resonance angiography has largely been superseded by contrast enhanced magnetic resonance angiography. Only one studyw1 evaluated 2D phase contrast magnetic resonance angiography and this reported sensitivity and specificity of 98% and 74%, respectively.
|
|
|
Table 3 shows data for detection of occlusion.
Figures 4 and 5 show sensitivity and specificity data, respectively.![]()
Contrast enhanced magnetic resonance angiography (six studiesw17-w19w21-w23) had median sensitivity 94% (85-100%) and median specificity 99.2% (97-99.8%). 2D time of flight magnetic resonance angiography (four studiesw2w6w7,w12) had lower sensitivity; median 86% (77-100%) and comparable specificity; median 97% (85-98%). Computed tomography angiography (five studies,w26-w28w30w31 four of which used multidetector row computed tomography angiography) had median sensitivity 97% (89-100%) and median specificity 99.6% (99-100%).
|
|
|
Above and below the knee
Some studies provided separate results on diagnostic accuracy for arterial segments above and below the knee. The accuracy of the different techniques was similar for the detection of stenosis of 50% or more above and below the knee: with contrast enhanced magnetic resonance angiography the median sensitivity and specificity were 87% and 93%, respectively, above the kneew8w14w17w20 and 83% and 92% below the kneew15w17w24; with duplex ultrasonography the median sensitivity and specificity were 88% and 95% above the kneew8w35w39w40w47w55w56 and 84% and 93% below the knee.w35w40w43w55
Two studies assessed accuracy for the detection of occlusion in the foot: one evaluated 2D time of flight magnetic resonance angiographyw5 and the other contrast enhanced magnetic resonance angiography.w24 Sensitivities were 86% and 79%, respectively, and specificities 27% and 86%, respectively. One study assessed the accuracy of duplex ultrasonography for detecting target vessels suitable for surgery in the foot, with sensitivity and specificity of 64% and 80% respectively.w42 Although there was limited evidence, these data suggest that these techniques may be less accurate in the foot.
Impact of method of assessment on management and outcome
Only one controlled trial, a prospective assessment of duplex ultrasonography using a historical control group, met the inclusion criteria for assessing the impact of the assessment method on patients' management and outcome.w59 The study included consecutive patients with lower leg ischaemia whose treatment plans were based on the results of either duplex ultrasonography with contrast angiography where indicated (114 patients) or contrast angiography (control group 113 patients).
In 78% of cases the management plan was based on duplex ultrasonography without the need for contrast angiography. There were no significant differences between the groups in terms of immediate and intermediate outcomes. This trial seems to have been well conducted and the results are likely to be reliable. As it used a historical control group, however, other factors occurring within the timeframe of the trial may have affected the results. Treatment and characteristics of patients were not significantly different between the two groups, although the authors did not comment on some factors that could have influenced outcomes, such as the graft material used, continuation of smoking, and the use of antiplatelet drugs.
Patients' attitudes
Four studies reported results relating to patients' attitudes. Two evaluated magnetic resonance angiography and contrast angiography,w60w61 one duplex ultrasonography and magnetic resonance angiography,w62 and one computed tomography angiography, magnetic resonance angiography, and contrast angiography.w63 Significantly more patients (28/30 patients) stated that they would prefer contrast enhanced magnetic resonance angiography over contrast angiography if they had to undergo testing again in the future,w60 and contrast enhanced magnetic resonance angiography scored significantly better on a scale that rated patients' experience of the test compared with contrast angiography (P=0.0001 and P=0.0002).w60w61
Contrast angiography was reported as the most uncomfortable, followed by contrast enhanced magnetic resonance angiography, with computed tomography angiography being the least uncomfortable (P=0.016).w63 Fifty per cent of patients (who were not claustrophobic and had no metallic implants) had no preference between time of flight magnetic resonance angiography or duplex ultrasonography (49/98 patients). Of those who did express a preference, most preferred time of flight magnetic resonance angiography (40/49 patients).w62
Within the same population there was no significant difference between time of flight magnetic resonance angiography and duplex ultrasonography on a scale that rated how "bothersome" the tests were.w62 While some of the surveys potentially suffered from recall or sequential bias, they were generally well conducted and the results are probably reliable. As the studies included only patients who were suitable for magnetic resonance angiography, the results cannot be generalised to patients who are not suitable for magnetic resonance angiography, such as those with claustrophobia or metallic implants.
Adverse events
Nine of the diagnostic accuracy studies,w4-w7w19w28w29w32w37 two studies reporting patient attitudes,w60w62 and 44 additional studies, w64-w107 reported results relating to adverse events. The lack of reporting of data on adverse events cannot be interpreted as no adverse events having occurred. The criteria used in monitoring and recording adverse events varied and were not always reported. These results should therefore be regarded only as a guide to the spectrum of adverse events reported and not as an accurate assessment of their frequency.
The most commonly reported adverse events were minor pain or discomfort during or immediately after the procedure (17% for 2D time of flight magnetic resonance angiography (2/12 patients), 22% for duplex ultrasonography (22/98 patients), and up to 10% for contrast enhanced magnetic resonance angiography (10/98 patients)), acute symptoms in the digestive system associated with contrast enhanced magnetic resonance angiography (up to 10% (2/20) patients), anxiety associated with 2D time of flight magnetic resonance angiography (10% (4/40) patients), and acute adverse events in the central and peripheral nervous system associated with contrast enhanced magnetic resonance angiography (up to 10% (2/20) patients).
The highest proportion of adverse events was reported for magnetic resonance angiography. Major adverse events (death and severe vascular adverse events), however, were reported in a higher proportion of patients who underwent contrast angiography, although the overall proportion who experienced major adverse events was low (severe vascular adverse events: contrast angiography 5% (1/19 patients); contrast enhanced magnetic resonance angiography 0.5% (2/435 patients)). There were two deaths: one with contrast angiography and one with contrast enhanced magnetic resonance angiography.
Studies reported adverse events related to the contrast agent for a small proportion of patients in relation to contrast angiography (acute renal failure in 10% (4/42) of patients with baseline chronic renal insufficiency) and contrast enhanced magnetic resonance angiography (acute renal failure: 1% (3/218) of patients with baseline chronic renal insufficiency; acute change in renal function: 1% (2/136 patients); severe unspecified adverse events related to contrast agent: up to 1% (5/641 patients)). In one study, which was specifically designed to evaluate the dose response and safety of one contrast agent (gadofosveset trisodium),w83 a high proportion of patients (25%; 59/238) experienced unspecified adverse events related to the contrast agent after contrast enhanced magnetic resonance angiography.
The use of computed tomography angiography for the assessment of peripheral arterial disease is a relatively recent development, and its contribution to effective surgical planning remains to be explored. Patients found computed tomography angiography less uncomfortable than contrast angiography or magnetic resonance angiography, and only a few mild adverse events were reported.
The only controlled trial of the effectiveness of imaging procedures, in terms of surgical planning and outcome of patients, found that duplex ultrasonography and contrast angiography were comparable, a result that is seemingly at odds with poor estimates of the diagnostic accuracy for duplex ultrasonography. The sensitivity of duplex ultrasonography seems to be inferior to both contrast enhanced magnetic resonance angiography and computed tomography angiography, which means that duplex ultrasonography may miss some significant stenoses. This may be of particular concern if duplex ultrasonography were to be used to screen patients before surgical planning. Duplex ultrasonography, however, is unlikely to misclassify a whole limb as "normal" and thus inappropriately screen out a patient from further investigation. Fifty per cent of patients expressed no preference between time of flight magnetic resonance angiography or duplex ultrasonography, and those who did generally preferred time of flight magnetic resonance angiography. We had no data on patients' preferences between contrast enhanced magnetic resonance angiography and duplex ultrasonography. Some studies reported minor adverse events associated with duplex ultrasonographynamely, anxiety or minor pain or discomfort during or immediately after the procedure.
The area of leg assessed probably affects diagnostic performance. Contrast enhanced magnetic resonance angiography and duplex ultrasonography were less accurate for detecting stenoses in the foot. There was insufficient evidence to judge computed tomography angiography. The assessment of potential outflow vessels in the foot is known to be problematic9 and warrants further research, particularly with respect to newer technologies such as computed tomography angiography. Separate data on calf vessels and foot vessels are required as the inclusion of foot vessels in below knee imaging may lower the accuracy of results.
Strengths and weaknesses of the review
We conducted extensive literature searches to locate all relevant studies. The possibility of publication bias remains a potential problem for all systematic reviews. The extent to which publication bias is an issue for diagnostic studies remains unclear as such studies measure the agreement between the results of the index test and the reference standard, rather than assessing whether there is a significant difference in outcome between an intervention and control group. Studies reporting higher estimates of test performance are more likely to be published, but the extent to which this occurs is unclear. Similarly, tests might not perform as well in the clinical setting as indicated by reports from research studies.
Our review was limited by the lack of high quality, well reported studies. We found only one controlled trial, which used a historical control group that could be subject to bias. Most studies that provided data on diagnostic accuracy had small sample sizes (median 41.5, range 20-183) and reported results on a per segment rather than per patient basis. Our review therefore provides information on the ability of these techniques to detect stenosis within particular arterial segments but not for determining the presence or absence of disease on a per patient or per limb basis. Few included studies reported these data. Analysis by segment also means that the estimates of the 95% confidence intervals for sensitivity and specificity do not account for the clustering of segments within patients. This would also affect statistical testing of heterogeneity, but given the considerable heterogeneity observed, any conclusions are not likely to be affected. The estimates of specificity derived from this type of study may be raised as increasing the number of segments assessed is likely to increase the number of true negatives.
We did not collect data on variability between observers, although we note that the methods used to ascertain degree of stenosis were not generally well reported and few studies directly measured such variability. This is an important issue in the evaluation of tests that require subjective interpretation,10 and further investigation of its effects on estimates of the accuracy of vascular imaging techniques is needed.
The field of vascular imaging research is evolving rapidly, particularly in relation to the use of computed tomography angiography, which is a relatively recent development in the assessment of peripheral arterial disease. We did not find any study investigating the diagnostic accuracy of the new 64 slice computed tomography angiography as this is a very new development. Our results represent the imaging techniques available at the time the primary studies were undertaken and will become out of date as new techniques emerge.
Implications for clinical practice
From data that reported the accuracy of the imaging tests at assessing arterial segments, rather than the whole limb or areas of the limb, contrast enhanced magnetic resonance angiography seemed to have better overall diagnostic accuracy than computed tomography angiography and duplex ultrasonography, and was preferred by patients over conventional angiography. It might therefore be a viable alternative to conventional contrast angiography for assessing patients with peripheral arterial disease before treatment. We could not identify enough data to assess the effectiveness of the imaging tests in terms of surgical planning and postoperative outcomes. In addition, the lack of data on severity of disease and comorbidities reported by the included studies reduces the generalisability of these findings.
Implications for further research
Quality assessment highlighted limitations in the quality of methods and reporting of many included studies. Future evaluations of diagnostic tests should follow the STARD guidelines for reporting of diagnostic accuracy studies.11 12 They should also consider reporting results by patient or by limb, as well as by segment, if they would be relevant to clinical practice.
Further research should assess the performance and adverse effects of the imaging tests on different subgroups of patients, particularly those who may be at higher risk of certain adverse events, such as those with diabetes and renal insufficiency. Additional separate data are required regarding the performance of the different imaging tests for assessing calf and foot vessels. The use of newer technologies, such as computed tomography angiography, for the assessment of peripheral arterial disease should be assessed.
Data on the influence of imaging technologies on the surgical planning and postoperative outcome for patients with peripheral arterial disease are urgently needed. A simple comparison of the accuracy of a technique for defining the degree of stenosis cannot fully assess the ability of a procedure to produce the "vascular road map" as factors such as length and grouping of stenoses are not considered. In addition a comparative diagnostic accuracy study assumes that the result of the reference standard investigation (in this case contrast angiography) is always correct. Therefore, an evaluation with this method can never find that the new technology (index test) gives superior information to that provided by current practice (reference standard).
The most reliable and appropriate method for obtaining comparative data on different testing options would be a randomised controlled trial designed to provide information on the influence of tests on decisions about treatment and outcomes in patients with peripheral arterial disease. Data on health economics could be collected simultaneously. There may be ethical objections to a randomised controlled trial, such as the withholding of an available test, particularly when it is a routine part of assessment of peripheral arterial disease in the institution involved in the study. Such a trial would be difficult because of the refinements in technology over time, the availability of the technologies, and the potentially large sample size required. A large multicentre trial might be necessary.
|
Funding: Health Technology Assessment Programme (project No 03/07/04). The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the Department of Health.
Competing interests: EB is now director of a company that undertakes consulting associated with medical imaging research. Neither she nor JK received payment for their contributions to this review.
Ethical approval: Not required.
![]()
CiteULike
Complore
Connotea
Del.icio.us
Digg
Reddit
Technorati What's this?
Read all Rapid Responses