The dangers of using nomograms to compare apples to oranges

A cross-specialty group of prostate cancer experts has reviewed data from > 13,000 patients on the accuracy of nomograms in predicting biochemical recurrence and actual prostate cancer-specific mortality after three different types of treatment.

Specifically, Lee et al. looked at the frequency of actual prostate cancer-specific mortality (PCSM) in the data from two large, high-volume, tertiary care, US hospitals and correlated those data to the nomogram-predicted probability of 5-year biochemical progression-free survival (NPP5bPFS) in the same men treated by radical prostatectomy (RP), by external beam radiation therapy (EBRT), and by brachytherapy (BT). And it is worth pointing out that this study was carried out by one of the most experienced cancer nomogram-development teams in the world.

Here are their basic findings:

  • The database encompassed 13,803 men who received first-line treatment for prostate cancer between 1995 and 2008.
  • At 10 years of follow-up, compared to men treated by RP, men treated with EBRT had a higher actual probability of PCSM for various nomogram-predicted risks
    • For men with an NPP5bPFS > 75 percent,
      • Men treated by EBRT had a PCSM of 3.0 percent.
      • Men treated by RP had a PCSM of 0.9 percent.
    • For men with an NPP5bPFS of 51 to 75 percent,
      • Men treated by EBRT had a PCSM of 6.8 percent.
      • Men treated by RP had a PCSM of 5.9 percent.
    • For men with an NPP5bPFS of 26 to 50 percent,
      • Men treated by EBRT had a PCSM of 26.6 percent.
      • Men treated by RP had a PCSM of 21.6 percent.
  • After adjusting for the NPP5bPFS, compared to treatment by RP, treatment with EBRT had significantly higher risk for PCSM (hazard ratio [HR] = 1.5; p = 0.006).
  • There was no evidence of a statistically significant difference in risk for PCSM between patients treated by RP and BT (but patient selection factors and lack of statistical power limited this analysis).

The authors conclude that:

EBRT patients with similar nomogram-predicted [NPP5bPFS] appear to have a significantly increased risk of PCSM compared with those treated by RP. Comparison of treatments using nomogram-predicted [biochemical recurrence] end points may not be valid.

It should be understood that this is not exactly a surprising finding. Rather, it provides confirmation of something that has long been suspected.

Definitions of biochemical recurrence and the post-treatment PSA kinetics can vary from treatment type to treatment type. Furthermore, there are racial, age-related, and other differences between the types of patients who are more likely to elect to have first-line surgery as compared to first-line EBRT (although we don’t know if that is true for brachytherapy).

From a patient perspective, Lee et al. note simply that biochemical recurrences after EBRT and RP seem to have different risks for consequent prostate cancer-specific mortality, so patients themselves (and of course their doctors) need to be careful about using nomogram-based predictions for biochemical recurrence-free survival as a surrogate for actual prostate cancer-specific survival. One is not comparing apples to apples.

6 Responses

  1. So we check the nomograms, then disregard the results because they are not valid comparisons. Its difficult to read and interpret all the studies ourselves. Do we then flip a coin to make our decision, or just do whatever the doctor recommends?

  2. Archie:

    The point that Dr. Kattan and his colleagues are making is that nomograms offer guidance, not “absolute truth”. In the examples they offer, the point that they are making is that nomograms usually tell you your risk for biochemical recurrence at points in time after certain types of treatment; they don’t usually tell you your risk for prostate cancer-specific mortality. Thus, just because Nomogram A tells you that you may have a 75% probability of biochemical progression-free survival at 10 years after surgery and a 90% probability of prostate cancer-specific mortality after 15 years does not mean that, when Nomogram B tells you that your have a 75% probability of biochemical progression-free survival at 10 years after external beam radiation therapy, you can assume that you also have a 90% probability of prostate cancer-specific survival at 15 years after radiation therapy.

  3. There are other outcomes that may be of more immediate interest to a patient than PCSM. The nomograms should be more reliable for predicting BCR. Although PSA-only outcomes may have little clinical significance, the patient may be interested in delaying initiation of ADT as long as possible, and may find predicting BCR to be useful for selecting treatment with that intent. Regarding efficacy of RP vs RT, it is only fair to add that RT has improved in the 20 year timeframe of the study, and developments have outstripped data on PCSM.

  4. Dear Archie:

    You appear to be missing the point of this study entirely. Some of the available nomograms are, in fact, very accurate in prediction of biochemical recurrence at specific time periods. What the authors are saying is that you cannot project information beyond what a particular nomogram tells you in order to compare apples to oranges. And the efficicy of every known form of treatment for prostate cancer has improved in the 20-year timeframe of the study (if one is treated by someone who knows what they are doing with the equipment, which can be a big “if”).

  5. I do not believe I am missing the point; I agree with you (and the esteemed authors) completely. I was delinquent and I apologize. Just musing and adding some ancillary points from my perspective as a patient and user of nomograms, which I trust highly for predicting BCR but less for distant outcomes. Thanks for presenting your commentary on this most interesting study, I would not have known about it otherwise.

    By the way, my lay understanding is that surgery was very good, in the olden days, at cancer control, but not very good at maintaining urinary and sexual function. RT back then was not as good at cancer control because a sufficient dose could not be delivered accurately enough to minimize collateral damage. Now both have improved regarding QOL issues, but RT had the most room for improvement for cancer control, and may have caught up with the new modalities such as IGRT.

  6. Perhaps the more important conclusion here is that using biochemical recurrence is not a valid surrogate for predicting mortality and that is why all the studies using biochemical recurrence to argue that HIFU, cryotherapy, and stereotactic radiation are similar to radical prostatectomy should not be interpreted as meaning they also have similar long-term survival.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: