BMJ  2005;331 (3 December), doi:10.1136/bmj.331.7528.0-f

Editor's choice

In praise of uncertainty

Imagine that you have just been told that your child has cancer. As the news sinks in, you are likely to want to know what the best treatment is and to be assured that your child will get it. You may not want to hear that we don't know what the best treatment is and that the only way to find out for sure is to enter your child into a trial.

If a parent's decision is hard, so too is the job of the trialist. With a child's future in the balance, how comfortable are you that encouraging a parent to enrol is in the best interests of that child? This week's BMJ carries an important paper that may make your job easier. Ambuj Kumar and colleagues (p 1295) looked back at a cohort of consecutive trials of treatments for cancer in children. Their aim was to find out how often new treatments were better or worse than standard treatments. They wanted to know whether the pattern of success and failure for new treatments showed genuine uncertainty on the part of the trialists. Without this uncertainty—if you could predict with any degree of confidence the outcome of a trial—asking people to be randomised would be unethical.

What they found, from looking at more than 100 trials funded by the US National Cancer Institute, was that the new treatments being tested were just as likely to be worse as they were to be better than standard treatments. On average, the benefits (survival without an event) were greater with new treatments, but the harms (death related to the treatment) were also greater, so that overall survival was similar. The researchers also used time series analysis to show that each trial represented an independent experiment. Improved outcomes for children with cancer have not come, they say, from a series of successes but from empirical testing by trialists. They conclude that "the uncertainty principle"—the ethical foundation for randomising patients into trials—is alive and well.

Is a 50% success rate good enough? A quarter of a century ago, the US statistician Frederick Mosteller said that we should consider this rate of success a good investment. In fact, perhaps a higher success rate should be looked upon with suspicion. This week's paper makes an important distinction: all the trials they looked at were publicly funded. Does the health of the uncertainty principle extend to industry funded trials? Sadly this seems unlikely. A systematic review published in the BMJ (BMJ 2003;326: 1167-70[Abstract/Free Full Text]) found that industry funded trials are more likely to favour the company's drug. The two most likely explanations seem to be selective reporting of good outcomes, or violation of the uncertainty principle by, for example, choosing a comparator that is known to be inferior.

So should you encourage parents to enrol their children into randomised trials? If the trials are publicly funded, this paper says that you should. For industry funded studies the answer is probably also yes, but look hard at the control treatment and at the outcome measures.