Many clinicians and their patients use the so-called Kattan nomograms to help make decisions about outcomes following use of certain types of prostate cancer treatment. Apparently these prediction tools have also become known in at least some circles as “Prostagram” data. (This is the first time The “New” Prostate Cancer InfoLink has ever come across the use of this term.)
We have always understood that the accuracy of this type of nomogram is limited. Data from such nomograms should be used to offer general guidance, but should not be expected to be capable of providing predictive data with high levels of accuracy for any individual patient.
Frank et al. have recently examined whether the “Prostogram” for post-brachytherapy outcomes accurately predicted recurrence at 5 years in patients treated with iodine-125 brachytherapy at a specific tertiary cancer center (the M. D. Anderson Cancer Center to be precise).
The authors retrospectively reviewed the records of 208 consecutive patients with prostate cancer treated at M. D. Anderson with a permanent 125I implant without neoadjuvant androgen deprivation therapy between 1998 and 2006. In each patient the Prostogram brachytherapy formula was used to calculate the probability of 5-year biochemical recurrence-free survival based on clinical stage, Gleason sum, pretreatment PSA level, and the additional use or non-use of external beam radiotherapy. Recurrence was defined as clinical relapse, death from disease, post-treatment androgen deprivation therapy, secondary treatments administered before PSA failure or biochemical recurrence based on the Kattan modification of the American Society for Therapeutic Radiology and Oncology definition of biochemical recurrence after external beam radiation therapy.
Patients were divided into quartiles (four groups of equal size) based on their Prostogram-predicted probability of 5-year recurrence-free survival. The average Prostagram-predicted probability for each quartile was then compared to the average actual 5-year recurrence-free survival rate for each quartile.
The results of this analysis presented by the authors are as follows:
- Actual 5-year biochemical recurrence-free survival rates were superior to Prostogram predicted probabilities by quartile
- In quartile 1, actual 89 percent vs predicted 80 percent
- In quartile 2, actual 87 percent vs predicted 86 percent
- In quartile 3, actual 100 percent vs predicted 89 percent
- In quartile 4, actual 100 percent vs predicted 94 percent
- Statistical analysis suggests that the predictive accuracy of the nomogram in the patients being treated at M. D. Anderson Cancer Center was less than 50 percent.
The authors conclude that, “Institutional variability requires that clinicians be cautious when using the Prostogram to counsel patients about the probability of success after permanent prostate brachytherapy.”
The “New” Prostate Cancer InfoLink is hardly surprised by these results. Each individual patient is very different from the next and (as we say regularly) “statistics do not apply to individuals.” We are actually mildly surprised that the concordance between the predictive value of the Kattan nomogram and the M. D. Anderson data is this good!
In the first place, one would expect the recognized experts at an institution like M. D. Anderson Cancer Center to be able to deliver “better than average” outcomes from use of the treatments they offer their patients. That is certainly what patients are expecting when they travel across country or around the world to be treated at such institutions!
Second, one might expect M. D. Anderson clinicians to be “better than average” at picking the right patients for the right type of treatment. In other words, it is not unreasonable to expect their patients to do well on a specific first-line treatment, because the patients have presumably been carefully evaluated to ensure that this treatment is indeed an appropriate form of management.
What would perhaps be of much greater interest would be a similar analysis of the concordance of the actual and predicted outcomes of a couple of hundred consecutive patients from community brachytherapy practices, in which the pressure to actually treat everyone who walks through the door seeking treatment with brachytherapy (as opposed to any other modality) is presumably higher than at M. D. Anderson. It would be gratifying to discover that these practices’ actual outcomes were also better than (or at least closely correlated with) the projected Kattan nomogram outcomes.
Filed under: Management, Treatment | Tagged: accuracy, actual, correlation, Kattan nomogram, outcome, prediction, Prostagram |
THE "NEW" PROSTATE CANCER INFOLINK 
Good comments, Mike. Thanks.
The recent study done at University of Washington on undetectable circulating prostate cancer cells would seem to throw into doubt any of the “Prostograms” since someone can be a T1, apparent fully contained cancer in the gland, undetectable PSA post-prostatectomy for some years, and still have PC come back later. Apparently the prostograms have some general predictive value but they are certainly not something to hang your hat on. Or am I wrong?
Harvey
Dear Harvey:
The Kattan nomograms are what they are intended to be — predictive tools that any patient and his doctor can use at the time of diagnosis or post-treatment to get a basic idea of the likelihood of prostate cancer being progressive over time. And that is all they are … because statistics do not apply to individuals. Conversely, what the University of Washington data have helped to clarify is something we have also long known — that a small percentage of men may still have recurrent disease 10, 15, 20, and even 25 years after primary treatment … because statistics do not apply to individuals! This does not place doubt on the value of the Kattan nomograms. What it should do is simply remind people that the data offered by these nomograms are simply averages. They tell us what is likely to happen for most men. They can not tell you what will happen for you!
Hi Folks,
This is a nice website. I’ll try to clarify some of the confusion on this topic, because it comes up a lot.
The tools (nomograms) that I made are designed to predict, as accurately as I know how, what will happen to you, based on the (usually) thousands of patients that have come before you. In this particular case, statistics most definitely apply to the individual because these predictions are as tailored to the individual as we can provide.
Having said that, all nomograms are wrong in a sense. An individual patient will either experience an outcome or he won’t, and the tool provides this as a probability. So, the tool might predict that you have a 99% chance of 5-year survival, and you survive 5 years; well, the tool was wrong — it should have predicted 100% for you. My goal for tomorrow is to make a tool with less prediction error than I did today. That’s all I can do. I will never make a perfect prediction tool, nor have I ever seen one in medicine.
You can reject all prediction tools and declare yourself an individual because the tool cannot predict a probability of 0 or 1. Perhaps you would not be helped by knowing that, as best we can tell, if we had 100 patients just like you, we would expect 99 of them to survive 10 years (a prediction tailored to you but that is not perfect because it is not 0 or 1). Taking this argument to the extreme, when you ask your doctor what your chances are, perhaps he should just say “it doesn’t matter — no one can tell you for sure.” I agree with the second part, but not the first. Personally, I would not be comfortable with a doctor who said that, because I like to make my own treatment decisions based on predicted outcomes — predicted outcomes are all we have to go on when the decision must be made.
In the same issue of that study by MDACC, I wrote an editorial on that paper by Frank et al. Yes, the tool was wrong by about 7% in each risk group. If that error is too much, what is the alternative? Maybe physician judgment would predict more accurately. Every time I have compared the two, physician judgment was worse. And then again, when is the last time you asked a Ford dealer how well a Ford would work for you? At least the nomogram does not benefit from the choice that you make. In the Frank study, I asked how should the patient be counseled when tool #1 has a 7% error rate? What tool has less than that? Beats me, and hindsight is 20/20. But I would like physicians (especially my physician!) to use the most accurate tool presently available when he counsels me on my treatment choices.
And the tools never tell anyone what to do, per se. They only provide predictions, for those who want them. For example, I would like to know if a particular treatment for me had a 99% predicted probability of impotence when tailored to my characteristics. That, to me, is valuable information, despite the fact that it does not definitively say I will become impotent.
Good luck in your fight against cancer. I am 18 years out myself…