Published 7 July 2009, doi:10.1136/bmj.b2673
Cite this as: BMJ 2009;339:b2673

Editorials

QRISK or Framingham for predicting cardiovascular risk?

QRISK is better on every performance measure, and should be recommended in the UK

In the linked study (doi: 10.1136/bmj.b2584), Collins and Altman assess the performance of the QRISK cardiovascular risk prediction algorithm in a primary care setting in the United Kingdom,¹ and compare QRISK^{2 3} with equivalent Framingham algorithms.^{4 5}

The QRISK algorithm is based on the largest risk prediction study ever undertaken and highlights a potential use of large scale electronic health record systems.^{2 3 6} In just a few years, a small team has linked electronic health records from several million people to produce a cardiovascular risk prediction algorithm that is more accurate and better validated than previous ones. Although prediction algorithms are available for many conditions, most are based on small numbers, are poorly validated, infrequently updated, and not generalisable. Moreover, most prediction algorithms are weak predictors and are not used regularly.

The first QRISK prediction algorithm was generated by retrospectively extracting data on risk factors and subsequent cardiovascular events for almost two million people from the QRESEARCH primary care database of more than 10 million patients covering about 7% of the population of the United Kingdom.² It was validated by the developers in another large database,³ and a year later they published an updated and improved algorithm, QRISK2, which included several additional predictors.⁶

Collins and Altman¹ now provide an independent evaluation of the first (QRISK1) algorithm and compare its performance with three Framingham algorithms.^{4 5} They conclude that on every performance measure, QRISK1 is better than Framingham. Unfortunately, because of the timing of publications, they were unable to compare QRISK2 with a recently modified version of the Framingham algorithm recommended by the National Institute for Health and Clinical Excellence (NICE) in 2008. This modified Framingham algorithm includes adjustments for family history and ethnic origin.⁷ Although not an independent evaluation, Hippisley-Cox and colleagues have now compared their QRISK2 algorithm with the NICE modified Framingham algorithm and again QRISK performs better.⁶

Direct comparisons between QRISK and Framingham are perhaps a little unfair because Framingham algorithms have not been calibrated to the UK population, although this is a relatively easy mathematical adjustment.⁸ However, an algorithm’s ability to discriminate between patients who will have an event and those who will not cannot be so easily improved and this is where QRISK has a slight edge on Framingham. More importantly, because QRISK2 performs better than QRISK1, further improvements are likely in future iterations.

But a closer look at the Collins and Altman evaluation provides a sobering message about the current state of cardiovascular risk prediction.¹ Our figure uses scaled rectangles to re-present some of their data, and it illustrates more clearly the modest discrimination performance of both algorithms at recommended treatment thresholds.⁹ QRISK would classify one in 10 men in the UK as high risk—that is, having a 10 year cardiovascular risk above the threshold recommended by NICE for treatment with statins.⁷ However only 30% of the subsequent cardiovascular events in men occurred in this high risk group. In contrast, the Framingham algorithm would classify about twice as many men in the UK (one in five) as being at high risk, although this larger high risk group does not include twice as many of the men who had a cardiovascular event during follow-up (it included only 50%). Substantially fewer women were identified as high risk (about 4% by QRISK and 5% by Framingham), with surprisingly little overlap between the two high risk groups. These high risk groups included only 18% (QRISK) and 17% (Framingham) of women who subsequently had a cardiovascular event.

View larger version (29K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Proportions of men and women classified as high risk by QRISK and Framingham who had a subsequent cardiovascular event (derived from table 5 in Collins and Altman1)

 
Almost 80% of participants in QRISK had some missing risk prediction variables, which suggests that QRISK could be improved given more complete data. Furthermore, it indicates that most UK adults have not had a formal documented cardiovascular risk assessment, as recommended by NICE,⁷ and that the quality of cardiovascular risk management in the UK (as elsewhere) is suboptimal.

Although UK general practices using the EMIS electronic health record system will have free access to an integrated QRISK calculator, commercial restrictions on the use of the algorithm in other systems are a concern. Cost may become a barrier to the development of effective electronic decision support using QRISK algorithms. Our experience has taught us that developing and implementing a computerised cardiovascular risk assessment and decision support system is a highly specialised task. Three features are crucial to their success: automatic provision of decision support as part of clinician workflow; provision of recommendations rather than just assessments; and provision of support at the time and location of decision making.¹⁰ We have shown that decision support incorporating these features significantly increases cardiovascular risk assessment,¹¹ but substantial time, experimentation, and wide collaboration are needed.

A QRISK based algorithm should replace the currently recommended Framingham based algorithm for estimating cardiovascular risk in the UK. With increased use, the quality of data will improve and updated prediction algorithms should be more accurate. However, QRISK is just the first of many continuously updatable prediction algorithms that will become available worldwide as electronic health record systems replace current paper based systems. The planned UK General Practitioner Extraction Service, for example, should soon be capturing data relevant to risk prediction from most of the population.¹² We believe that freely sharing these algorithms is the best way to facilitate their effective implementation.

Cite this as: BMJ 2009;339:b2673

¹ School of Population Health, University of Auckland, Private Bag 92019, Auckland, New Zealand

Research, doi:10.1136/bmj.b2584

---

Competing interests: None declared.

Provenance and peer review: Commissioned; not externally peer reviewed.

References

1. Collins GS, Altman DG. An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study. BMJ 2009;339:b2584.[Abstract/Free Full Text]
2. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ 2007;335:136.[Abstract/Free Full Text]
3. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Brindle P. Performance of the QRISK cardiovascular risk prediction algorithm in an independent UK sample of patients from general practice: a validation study. Heart 2008;94:34-9.[Abstract/Free Full Text]
4. Anderson KV, Odell PM, Wilson PWF, Kannel WB. Cardiovascular disease risk profiles. Am Heart J 1991;121:293-8.[CrossRef][Web of Science][Medline]
5. D’Agostino R, Vasan R, Pencina M, Wolf PA, Cobain M, Massaro JM, et al. General cardiovascular risk profile for use in primary care: the Framingham heart study. Circulation 2008;117:743-53.[Abstract/Free Full Text]
6. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R, Sheikh A, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008;336:a332.
7. Cooper A, O’Flynn N; on behalf of the Guideline Development Group. Risk assessment and lipid modification for primary and secondary prevention of cardiovascular disease: summary of NICE guidance. BMJ 2008;336:1246-8.[Free Full Text]
8. D’Agostino RB, Sr, Grundy S, Sullivan LM, Wilson P. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 2001;286:180-7.[Abstract/Free Full Text]
9. Marshall RJ. Cardiovascular risk can be represented by scaled rectangle diagrams. J Clin Epidemiol (in press).
10. Kawamoto K, Houlihan C, Balas E, Lobach D. Improving clinical practice using clinical decision support systems: systematic review of trials to identify features critical to success. BMJ 2005;330:765-72.[Abstract/Free Full Text]
11. Wells S, Furness S, Rafter N, Horn E, Whittaker R, Stewart A, et al. Integrated electronic decision support increases CVD risk assessment four fold in routine primary care practice. Eur J Cardiovasc Prev Rehab 2008;15:173-8.[CrossRef]
12. NHS Information Centre. General Practice Extraction Service (GPES). 2009. www.ic.nhs.uk/services/in-development/general-practice-extraction-service.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Articles

ASSIGN, QRISK, and validation

Hugh Tunstall-Pedoe, Mark Woodward, and Graham Watt
BMJ 2009 339: b3514. [Extract] [Full Text]

An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study

Gary S Collins and Douglas G Altman
BMJ 2009 339: b2584. [Abstract] [Full Text] [PDF]

Risk assessment and lipid modification for primary and secondary prevention of cardiovascular disease: summary of NICE guidance

Angela Cooper, Norma O’Flynn on behalf of the Guideline Development Group
BMJ 2008 336: 1246-1248. [Extract] [Full Text] [PDF]

Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study

Julia Hippisley-Cox, Carol Coupland, Yana Vinogradova, John Robson, Margaret May, and Peter Brindle
BMJ 2007 335: 136. [Abstract] [Full Text] [PDF]

This article has been cited by other articles:

• Tunstall-Pedoe, H., Woodward, M., Watt, G. (2009). ASSIGN, QRISK, and validation. BMJ 339: b3514-b3514 [Full text]  

Rapid Responses:

Read all Rapid Responses

QRISK or Framingham for predicting cardiovascular risk? Start with the Pulse Mass Index

Prof. Enrique J. Sanchez-Delgado, MD
bmj.com, 20 Jul 2009 [Full text]

Time to develop a shortened life- span scale?

Mike Beary, et al.
bmj.com, 24 Jul 2009 [Full text]

Online poll

Find out more >>