Over-inflation of the significance of PSA velocity in Kaiser paper

We still haven’t been able to see the full text of the article by Wallner et al. (reported yesterday) suggesting that PSA velocity was as good or better than a single PSA data point in predicting risk for prostate cancer — and for aggressive types of prostate cancer in particular. However, a commentary on the Medscape web site has clarified a number of points which had worried us, and calls into question the entire validity of this study.

Let’s see if we can make a number of simple points about the Kaiser Permanente data, as follows:

  • The study included all men > 45 years of age in the Kaiser database from January 1, 1998 who had at least three PSA data points and followed the patients until December 31, 2007 for a diagnosis of prostate cancer. However, …
  • A diagnosis of prostate cancer requires a biopsy, and there was no standardized PSA level at which men had to have a biopsy, so we have no idea how many men didn’t have a biopsy and therefore might have had undiagnosed prostate cancer. (Remember that in the big screening trials every patient in the screening arm had at least one biopsy at the end of the study.)
  • We also don’t know why there was no biopsy for the men who didn’t have one but arguably should have done because their PSA was greater than (say) 2.5 ng/ml and rising or they had a positive DRE. (Physician opinion? Patient opinion? What?)
  • Wallner et al. report in their study that PSA velocity provided a marginal improvement in prediction of overall risk for prostate cancer compared with a single PSA measurement (AUC, 0.963 vs 0.944). They also report that PSA velocity provided a much more accurate prediction of aggressive disease (AUC, 0.955 vs 0.727). However, …
  • This implies that, in their patients, on average, PSA testing was > 90 percent accurate in predicting risk for prostate cancer, and we know that this isn’t even close to being true.

What we are dealing with here is something known as “verification bias.” The so-called “gold standard” for prostate cancer diagnosis is a systematic, TRUS-guided biopsy. If every patient in a cohort study like the Kaiser Permanente study reported by Wallner et al. doesn’t get a biopsy based on standard PSA criteria, then what we are really looking at in the study is the accuracy of a biopsy (not a single PSA result or PSA velocity) in diagnosing prostate cancer. The accuracy of a biopsy in diagnosing prostate cancer in men who have positive biopsy results is — of course — 100 percent, by definition!

Now to be fair to Wallner et al. they do apparently acknowledge this issue in the full text of their article (as reported by Medscape), where they write that:

Verification bias resulting from the presence of subclinical prostate cancer among men defined as disease free cannot be ruled out. This differential misclassification could result in inflated estimates of the sensitivity and underestimates of the specificity [of a single PSA result or PSA velocity in predicting the probability of a diagnosis of prostate cancer].

However, the failure to acknowledge this fact in the abstract of the paper and in the Kaiser Permanente media release is a classic case of over-inflating the importance of their data in the interests of seeking media coverage. Despite the comforting conclusion that a rising PSA is an excellent indicator of risk for prostate cancer (and for aggressive prostate cancer in particular), the study does not, in fact, demonstrate any such thing.

We’d still like to be able to read the full text of this paper in order to fully confirm what we have written above, but we are now much more comfortable about how to appropriately interpret the proposed conclusions of the paper by Wallner et al. We don’t find them compelling.

6 Responses

  1. A further cautionary note: There is a tendency to assume that the increase in PSA after a cancer diagnosis is cancer related. Not always true. My PSA went from 12 to 14 something in 3 months. I refused surgery and we settled on removing calcification which CT showed as a cloudy area, It was at least five stones, quite distinct and separate, with a broken off biopsy needle tip included in the removed matter. Three months later my PSA 3.3.

    Do I have cancer? Yes. PSA 14+ cancer? No.

  2. What would be much more interesting would be the significance of Gleason score velocity.

  3. In theory, maybe, Gleason score velocity is meaningful, but I have never seen this be measured, and many men are going to have the same Gleason score for years and years, so their Gleason score velocity would be zero.

    The other problem with Gleason score velocity is that it would only really be accurate if one was sure one was biopsying the same area of the same tumor on a consistent basis, and that is extremely difficult to do with accuracy.

  4. “many men are going to have the same Gleason score for years and years”

    This possibly implies that there are many men with a Gleason score of 6 who are observed to stay at that score for years. And yet there are also many men with Gleason 6 who get RP. Assuming the former is true, what is it that decides whether men with Gleason 6 get RP or not (and subsequently have the same Gleason score for years and years)?

  5. Dear Chris:

    That’s a question researchers have been asking for most of the past 40 years. We have no clue.

  6. Good reading gents. I’m Gleason 8, starting pencil beam proton therapy at M.D. Anderson on Tuesday morning. Two weeks of Casodex and 60 days of Lupron has brought PSA to 2. Combo for the past 30 days and now PSA is 0.31. Wish me luck!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: