Rapid Responses to:

EDITORIALS: Arlene S Bierman and Jocalyn P Clark Performance measurement and equity BMJ 2007; 334: 1333-1334 [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Equity, ethnicity and performance measures

Mark R D Johnson   (4 July 2007)

Performance measurement and health inequalities

Jocalyn Clark   (6 July 2007)

Impact of pay for performance on health inequities - reporting systems should include patient level data

Christopher J Millett, Kamlesh Khunti, Azeem Majeed   (10 July 2007)

Pay for Equity

Arlene S Bierman   (13 July 2007)

Tying pay-for-performance to healthcare disparities should await mastery of measurement issues

James P. Scanlan   (8 February 2009)

Equity, ethnicity and performance measures 4 July 2007
Mark R D Johnson, Professor of Diversity in Health & Social Care Mary Seacole Research Centre, De Montfort Unversity LE2 1RQ Send response to journal: Re: Equity, ethnicity and performance measures	I was delighted to have support from a BMJ editorial for the contention that Equity as well as Efficiency and individual Quality of Care benefits from ‘Performance Management’ (1). Those of us who have argued for ‘ethnic record keeping’ and monitoring have believed this for a long time (2). Yet, as Prof Wade (3) notes in the same issue of the BMJ, collecting patient-related data can only be justified if its long-term outcome can reasonably be expected to lead to improved health. But if the data are not collected, these issues can never be addressed, and ‘performance anxiety’ may be replaced by ‘performance complacency’. It s a pity that nearly all the evidence cited by Bierman & Clark in relation to reduction of ethnic inequality comes from the United States of America, (where ethnic record keeping has long been established as routine). This reflects the way in which research in the UK has too often been ‘colour blind’ or excluded black and minority ethnic (BME) groups from clinical trials (4). Our experience at the UK Centre for Evidence in Ethnicity Health and Diversity has been that in general UK practice and research has not sought to test ways of improving the health of ‘BME’ groups, but has been content merely to describe short-term initiatives as ‘good practice. The NHS Specialist Library for Ethnicity & Health (www.library.nhs.uk/ethnicity ) would be delighted to hear of any research studies which have subjected such initiatives (to reduce ethnic inequalities in health) to rigorous evaluation. 1: Bierman AS, Clark JP 2007 ‘Performance measurement and equity’ BMJ 334 :1333-4 (30 June). 2: Gill P, Johnson MRD 1995 'Ethnic monitoring and equity: Collecting data is just the beginning' (editorial) British Medical Journal 310 :890. 3: Wade D 2007 ‘Ethics of collecting and using healthcare data’ BMJ 334 :1330-1331 4: Mason, S., Hussain-Gambles, M., Leese, B., Atkin, K., Brown, J. 2003 Representation of South Asian people in randomised clinical trials: analysis of trials’ data British Medical Journal 326 :1244-1245. Competing interests: I am the 'clinical lead' for the NHS 'specialist library for ethnicity and health' and director of a research centre that has for a long time campaigned for better ethnicity data and services for minority ethnic groups in the NHS
Performance measurement and health inequalities 6 July 2007
Jocalyn Clark, Assistant professor of medicine St Michael's Hospital Send response to journal: Re: Performance measurement and health inequalities	My colleague Arlene Bierman and I were pleased to see another article published this month that raises similar questions as our editorial and those of the rapid respondent, focused specifically on pay for performance: Coleman K, Hamblin R. Can Pay-for-Performance Improve Quality and Reduce Health Disparities? PLoS Medicine Vol. 4, No. 6, e216 doi:10.1371/journal.pmed.0040216 The full text article can be accessed here http://dx.doi.org/10.1371/journal.pmed.0040216 Competing interests: I am an author of the editorial
Impact of pay for performance on health inequities - reporting systems should include patient level data 10 July 2007
Christopher J Millett, Specialist trainee in public health Department of Primary Care & Social Medicine, Imperial College, London. W6 8RP, Kamlesh Khunti, Azeem Majeed Send response to journal: Re: Impact of pay for performance on health inequities - reporting systems should include patient level data	In a recent BMJ Editorial, Bierman and Clark (1) highlight the importance of evaluating the impact of pay for performance incentives, such as those introduced in the new UK general practitioner contract, on known inequities in the delivery of healthcare and health outcomes. This is consistent with UK government policy (2), which recognises the importance of ensuring that new health policies are applied to all sectors of the population, including those people living in deprived areas and minority ethnic communities. However, assessing the impact of pay for performance on health inequities in the UK has been hampered by an absence of patient level data within the national reporting system for the new general practitioner contract. This has been further compounded by poor recording of patient based measures of ethnicity and of socio-economic status within primary care information systems. Findings from the Wandsworth Prospective Diabetes Study in south London, which has achieved > 90% ethnicity coding in primary care, suggest that the impact of pay for performance on health inequities has been mixed. We found that diabetes care was largely equitable between ethnic groups when assessed using process of care measures, such as the provision of smoking cessation advice (3). Yet while prescribing levels and intermediate clinical outcomes improved in all groups after the introduction of pay for performance incentives, inequities evident between ethnic groups before the contract persisted after it (4). The main lesson from these two studies is that pay for performance by itself may not be sufficient to address inequities in the quality of care. Financial incentives for physicians need to be combined with other initiatives to ensure that minority groups, who may particularly at risk of complications, receive the same quality of care as other patient groups. National reporting systems also need to be modified to allow patient level analyses of quality of care, in addition to the practice level measures of performance that are currently available. (1) Bierman A, Clark J. Performance measurement and equity. BMJ 2007; 334:1333-4 (2) Department of Health. Tackling Health Inequalities. A Programme for Action. London. Department of Health. 2003 http://www.dh.gov.uk/Home/fs/en Accessed 5th March 2007 (3) C Millett, J Gray, S Saxena, G Netuveli, A Majeed. Impact of a pay for performance incentive on support for smoking cessation and on smoking prevalence among people with diabetes. CMAJ 2007; 176(12):1705-10 (4) C Millett, J Gray, S Saxena, G Netuveli, K Khunti, A Majeed. Ethnic Disparities in Diabetes Management and Pay-for-Performance in the UK: The Wandsworth Prospective Diabetes Study. PLoS Medicine 2007; 4(6): e191. doi:10.1371/journal.pmed.0040191 Competing interests: None declared
Pay for Equity 13 July 2007
Arlene S Bierman, OWHC Chair in Women's Health Li Ka Shing Knowledge Institute St. Michael's Hospital M5B 1W8 Send response to journal: Re: Pay for Equity	Dr Johnson underscores that the collection and use of data on race and ethnicity is prerequisite for assessing and addressing health inequities in clinical practice and comments that the collection of these data in the US is routine. Unfortunately, this is not the case. The US has a long way to go in incorporating equity into performance measurement and improvement initiatives. While there is greater availability of racial and ethnic data in the US than in the UK, these data are not routinely collected in clinical settings, and data quality issues abound.1 Stratified reporting of performance indicators by sociodemographic characteristics is not required by any major reporting body. Furthermore, the contribution of social class to health inequities receives little attention. Progress has been made by engaging providers to address disparities. In 1999, I helped to organize a meeting cosponsored by the US Department of Health and Human Services and the Commonwealth Fund bringing together representatives of federal and state government, insurers, purchasers, and health plans to discuss issues related to obtaining racial and ethnic data for quality improvement and performance measurement in managed care.2 At that time both real and perceived barriers to the collection of these data resulted in great reluctance to move forward in this area. In 2002, the Institute of Medicine released its landmark report Unequal Treatment synthesizing a large body of literature that documented pervasive racial and ethnic disparities and calling for action.3 By 2004, major health insurers formed the National Health Plan Disparities Collaborative with the goal of reducing racial and ethnic disparities. The Collaborative has a major focus on collecting racial and ethnic data. (http://www.rwjf.org/files/publications/other/NHPCSummaryReport2006.pdf) Dr. Millett points out the need for the availability and analyses of these data at the individual level. In fact, when individual level is analyzed at the practice level, a great deal of variation in the magnitude of disparities varies across practices is found.4 By studying factors that influence the size of observed disparities we can learn what might work to close these gaps. Dr. Millet also argues that other initiatives need to be combined with pay for performance to reduce inequities. Perhaps we should consider “pay for equity” to accelerate progress. References 1. Fremont AM, Bierman A, Wickstrom SL, Bird CE, Shah M, Escarce JJ, et al. Use of geocoding in managed care settings to identify quality disparities. Health Aff (Millwood) 2005;24(2):516-26. 2. Bierman AS, Lurie N, Collins KS, Eisenberg JM. Addressing racial and ethnic barriers to effective health care: the need for better data. Health Aff (Millwood) 2002;21(3):91-102. 3. Smedley BD, Stith AY, Nelson AR. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, D.C.: National Academy Press, 2002. 4. Bird CE, Fremont AM, Bierman AS, Wickstrom S, Shah M, Rector T, et al. Does quality of care for cardiovascular disease and diabetes differ by gender for enrollees in managed care plans? Womens Health Issues 2007;17(3):131-8. Competing interests: I am an author of the editorial
Tying pay-for-performance to healthcare disparities should await mastery of measurement issues 8 February 2009
James P. Scanlan, Attorney James P. Scanlan, Attorney at Law, Washington, DC 20007, USA Send response to journal: Re: Tying pay-for-performance to healthcare disparities should await mastery of measurement issues	The editorial by Bierman and Clark [1] argues for including examination of healthcare disparities as a component of pay-for- performance programs. Such action should await a better understanding about, and consensus on, how to measure healthcare disparities With negligible exception, health and healthcare disparities research to date has employed various measures of differences between rates of experiencing a favorable or adverse outcome without recognition of the ways each measure tends to be affected by the overall prevalence of an outcome. As outcomes like receipt of beneficial health procedures or results increase in overall prevalence, relative differences in experiencing them tend to decrease while relative differences in failing to experience them tend to increase. Absolute differences between rates also tend to change as the overall prevalence of an outcome changes, though in a more complicated way. Roughly, as an uncommon outcome increases in overall prevalence, absolute differences tend to increase; as a common outcome becomes even more common, absolute differences tend to decline. Close to 100 references explaining these tendencies as they bear on the interpretation of data on group differences in the law and the social and medical sciences may be found on the Measuring Health Disparities (MHD) page of jpscanlan.com and a Pay-for-Performance sub-page addresses these issues with respect to perceptions about the likely effects of pay- for-performance programs on healthcare disparities and the wisdom of including measures of effects on healthcare disparities in pay-for- performance programs. Several of the articles referenced by Bierman and Clark illustrate some of the issues. They cite a study by Sehgal [2] as finding that performance measurement for patients with end stage renal disease insured by Medicare in the United States reduced racial and gender disparities in adequate haemodialysis but left unchanged disparities in anaemia management and nutritional status. In examining changing racial and gender disparities in rates of adequate haemodialysis, Sehgal relied on absolute differences between such rates during a period when black rates increased from 36% to 84% while white rates increased from 43% to 87% (reducing the absolute difference from 7 to 3 percentage points) and when male rates increased from 31% to 82% while female rates increased from 54% to 91% (reducing the absolute difference from 23 to 9 percentage points). The many researchers who appraise the size of healthcare disparities based on relative differences in favorable outcome rates would also have found substantial reductions in both disparities. But the United States National Center for Health Statistics (NCHS), which is committed to appraising healthcare as well as health disparities in terms of relative differences in adverse outcome rates, would have found both disparities, measured in terms of relative differences between rates of inadequate haemodialysis, to approximately double (from 12% to 23% for race and from 50% to 100% for gender). The only other issue on which Sehgal presented complete data reflected similar patterns. As the black rate of adequate hemoglobin increased from 24% to 73%, the white rate increased from 28% to 75% – halving the 4 percentage point absolute difference between rates (albeit not in statistically significant terms). At the same time, the relative difference between adequate hemoglobin rates declined substantially (from 14% to 3%), while the relative difference between inadequate hemoglobin rates increased by a third (from 6% to 8%). Bierman and Clark cite a study by Trivedi et al.,[3] which found that during periods of overall improvements in various process and control outcomes, racial disparities in all six of the former decreased while disparities in two of three of the latter increased. But here, too, the study relied on absolute differences between rates, and in most cases such differences changed in the usual direction for the rate levels at issue, complicating interpretation (as discussed in reference 4, a comment on a later study by the same authors). The procedure described on the Solutions sub-page of MHD, which, in theory, ought not to be subject to the effects of changes in the overall prevalence of an outcome, would show changes in disparities consistent with the patterns of changes in absolute differences. But NCHS, relying on relative differences in adverse outcomes, would find increasing disparities in four of the seven cases where Trivedi et al. found decreasing disparities, and NCHS statisticians so noted in a letter to the editor.[5] Bierman and Clark also cite Casalino et al.[6] for cautioning against the dangers of pay-for-performance. But Casalino et al. formed the view that improved healthcare would tend to increase disparities based on a study that found increases in absolute differences between rates when very uncommon procedures were generally increasing.[7] To the extent that the disparities examined in that study could be effectively measured, they in fact seem to have decreased.[8] While there remains a perception in the United States that pay-for- performance will increase healthcare disparities, a recent BMJ study of reduced absolute differences between blood pressure monitoring and control rates for different socioeconomic groups under pay-for-performance suggested that the pay-for-performance would reduce healthcare disparities in the United Kingdom.[9] But in this study, too, the directions of changes were of a kind that one should expect in the circumstances of further general increases of the high outcome rates at issue (though there may in fact have been meaningful reductions in disparities as well).[10]. Control of hypertension raises particularly complex measurement issues. General reductions in blood pressure will tend to reduce relative differences in control, and increase relative differences in absence of control, whether one examines the population at large or a subpopulation diagnosed as hypertensive. But the same types of general reductions in blood pressure will tend to reduce absolute differences in the general population (where control rates are fairly high) but increase them in a subpopulation diagnosed as hypertensive (where control rates are much lower).[11, 12] All to say, the measurement of healthcare disparities is more complex than is generally recognized, if, indeed, it can be reliably done at all. The above-mentioned approach that is described on the Solutions sub-page of MHD is plainly superior to the common reliance on one standard measure or another without regard to the way the measure tends to be affected by overall prevalence. But it is doubtful that even that approach is reliable enough to form the basis for pay-for-performance decisions. References: 1. Bierman AS, Clark JP. Performance measure and equity. BMJ 2007;334:1333-1334. 2. Sehgal AR. Impact of quality improvement efforts on race and sex disparities in hemodialysis. JAMA 2003;289:996-1000. 3. Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Trends in the quality of care and racial disparities in Medicare managed care. N Engl J Med 2005;353:692-700. 4. Scanlan JP. Understanding patterns of correlations between plan quality and different measures of healthcare disparities. Journal Review Aug. 30, 2007 (responding to Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Relationship between quality of care and racial disparities in Medicare health plans. JAMA 2006;296:1998-2004):: http://journalreview.org/v2/articles/view/17062863.html 5. Keppel KG, Pearcy JN, Weissman JS. Untitled letter. N Engl J Med 2005;353:2082-2083 6. Casalino LP, Elster A, Eisenberg A, et al. Will pay-for- performance and quality reporting affect health care disparities? Health Affairs 2007;26(3):405-414. 7. Werner, RM, Asch DA, Polsky D. Racial profiling: The unintended consequences of coronary artery bypass graft report cards. Circulation 2005;111:1257–1263 8. Scanlan JP. Pay-for-performance implications of the failure to recognize the way changes in prevalence of an outcome affect measures of racial disparities in experiencing the outcome. Journal Review Feb. 8, 2008 (responding to reference 7 above): http://journalreview.org/v2/articles/view/15769766.html 9. Ashworth M, Medina J, Morgan M. Effect of social deprivation on blood pressure monitoring and control in England: a survey of data from the quality and outcomes framework. BMJ 2008;337:a2030 10. Scanlan JP. Interpreting patterns of changes in absolute differences between rates when common outcomes become even more common. BMJ Dec. 7, 2008 (responding to reference 9 above): http://www.bmj.com/cgi/eletters/337/oct28_2/a2030 11. Scanlan JP. Can We Actually Measure Health Disparities?, presented at the 7th International Conference on Health Policy Statistics, Philadelphia, PA, Jan. 17-18, 2008 http://www.jpscanlan.com/images/2008_ICHPS_Oral.pdf 12. Scanlan JP. Measuring racial disparities in hypertension control. Ann Fam Med Jan. 25, 2009 (responding to Satcher D. Examining racial and ethnic disparities in health and hypertension control. Ann Fam Med 2008;6:483-485): http://www.annfammed.org/cgi/eletters/6/6/483 Competing interests: None declared