Alas … PPV not good for PI-RADS at all


For the past few years, some of us have been quietly concerned about the true value of MRI scans and the accuracy of PI-RADS scores in evaluation of a patient’s risk for the presence of clinically significant prostate cancer. This is a complex issue … but data from a large, recently completed study has now validated these concerns.

Westphalen et al. have published a detailed analysis of data from > 3,400 patients with > 5,000 suspicious lesions in the journal Radiology. An editorial commentary (by Milot) has also appeared in the same journal. And Diagnostic Imaging has reported bluntly that “PI-RADS not as effective with prostate MRI as hoped“. The last of these sources probably provides the simplest “read” for many patients.

What is a “positive predictive value” or PPV of a diagnostic test like a prostate MRI scan? It is the probability that a patient who has a positive test result really does have prostate cancer. In this particular study, it was used to assess the probability that each patient who was assigned a PI-RADS score of 2, 3, 4, or 5 really did have clinically significant prostate cancer as confirmed by a biopsy-proved Gleason score of 3 + 4 = 7 or higher.

What Westphalen and his colleagues did was to conduct a careful retrospective analysis of the available data provided by radiologists who used the PI-RADS evaluation methods at 26 centers in the USA, Canada, Brazil, the Netherlands, and South Korea. The patients whose data were re-evaluated all had either suspected or biopsy-confirmed, untreated prostate cancer. Prostate MRI imaging of most of these patients was carried out at either 1.5 T or 3 T.

Here are the key findings reported by Westphalen et al.:

  • MRI data from 5,082 lesions in 3,449 men were re-evaluated.
  • The average (mean) age of the patients was 65 ± 8 years.
  • The available biopsy data indicated that 2,082 of these men had 1,698 cancers with a Gleason score of 3 + 4 = 7 or higher (i.e., an International Society of Urological Pathology grade group of 2 or more).
  • The positive predictive values (PPVs) of the PI-RADS scores originally assigned to the patients’ MRI scans, across all centers, were:
    • 35 percent for a PI-RADS score of 3 or more
    • 49 percent for a PI-RADS score of 4 or 5
  • There was no apparent difference between the results when MRIs were carried out using 3 T as compared to 1.5 T MRI scans, and > 80% of the scans were carried out using 3 T MRI systems.

Westphalen et al. conclude that, based on their analysis:

The positive predictive value of the Prostate Imaging and Reporting Data System was low and varied widely across centers.

Obviously, this is not a good thing — at all!

Now … what does this actually mean (and not mean) from a patient perspective?

First and foremost, it does not mean that all MRI scans are a waste of time and money. They can be extremely helpful in specific cases.

What it really means is that

  • The interpretation of MRI scans can be extremely difficult, even for experienced and skilled radiologists.
  • There is a low level of consistency from center to center with respect to the accurate interpretation of the results of MRI scans by members of the radiology community.
  • The PI-RADS methodology for “scoring” risk based on MRI scans isn’t sufficiently accurate to be really helpful.

It is clear from these data that the radiology community needs to carry out a major re-assessment of the ways in which prostate cancer MRIs are carried out and how the consequent images are evaluated. It seems likely to your sitemaster that many scans can, in truth, only be said to have “inconclusive” results as to the presence or absence of clinically significant prostate cancer. It seems perfectly reasonable to your sitemaster that such a result should be reported in exactly that way, i.e., “Inconclusive”. It also opens the door to whether — when they become more widely available — gallium-67 PSMA scans may be a better way to assess risk for clinically significant prostate cancer than MRI scans (always assuming we can afford to do such scans on thousands of men each year).

In the meantime, should patients continue to ask for MRIs? We think that they should. But we also think that the MRI reports should be much simpler as to their conclusions until either there is far greater consistency about the use of the PI-RADS tool across the entire radiology community or a better way to assess the available data can be developed.

18 Responses

  1. Where does this leave men on AS who relying on mpMRI to monitor the prostate? Do they need to add back the biopsy and its risks? AS is a roller-coaster ride, not a merry-go-round.

  2. A single clearly biased study not showing peer and an all inclusive absolute headline. What MRIs were used and what was the skill of radio oncologist interpreting the results? The multiparametric MRI I went from Texas to London for and the doctors there provided superior diagnostic information than the American docs I consulted with on an inferior MRI. Without the mpMRI my treatment decisions would have been the poorer choices for my circumstances. As for PI-RADS, that’s are diagnostic tools – not conclusive. I again ask what is the business purpose of this site and the real intentions of the undisclosed sitemaster?

  3. Wonder what relevance this has, if any, to negative predictive value of prostate MRI?

  4. I know that UCLA, which has very experienced radiologists, now uses their own scoring system in addition to PI-RADS v2. While false positives may send patients for unnecessary biopsies, it is the negative predictive value (NPV) that I focus on more — telling a patient that there is no clinically significant prostate cancer (csPC), when there really is. NPVs of mpMRI for csPC range from 80 to 90% when biopsy is used as the reference standard. But the NPV falls as low as 50% (half the csPC is missed) in some studies where it is compared to whole-mount post-prostatectomy pathology. Relying on mpMRI imaging exclusively is never a good idea.

  5. Howard:

    If I was wearing your shoes I would go on getting mpMRIs. If you can’t see anything on the MRI in a situation like yours, then you probably don’t need a biopsy.

    On the other hand, I do think these data present a problem for men on AS with a Gleason score of 3 + 4 = 7.

    The real issue here is that all the radiologists who look at prostate mpMRIs need better training and a much more cautious approach to reporting what they think they are and aren’t seeing on these MRIs. The variation in data interpretation that was clearly evident in this trial looks to me like a case of people over-interpreting what they think they can see as opposed to reporting what they are 100% certain about.

  6. Dear Mr. Wadsworth …

    (1) The sitemaster is not undisclosed at all. You just haven’t done your homework. What the sitemaster does NOT do is promote himself as an individual.

    (2) This site has no “business purpose” at all. It is a disinterested news and information resource. You will note that it carries no advertising and it does not “promote” any particular form of therapy. Neither the sitemaster nor any other current staff member of Prostate Cancer International is paid or receives any other form of financial reward for the services we provide.

    (3) This study was done by people who (almost certainly) included — as authors and study participants — the ones you got your MRI done by in London. Have a look at the list of authors to see if I am correct. I would therefore point out to you that even if your MRI done in the UK happened to be correct, it by no means invalidates the study findings.

    (4) I have absolutely no idea why you would think this study was “biased”. It’s authors included many of the world’s top experts on the actual and potential roles of MRI scanning in the diagnosis and management of prostate cancer. They have basically just pointed out that the way that most of them have recommended “evaluation” of prostate MRI data seems to be fundamentally flawed in some way.

  7. Cliff: It may well be that there is another paper in the works from the same group of authors that will deal with exactly that question.

  8. Presumably this analysis also raises questions as to the accuracy of assessment of extracapsular extensions.

  9. Cliff:

    That may well be the case — at least for small extracapsular extensions and extensions in certain regions of the prostate where they may be more difficult to see on the MRI scans.

  10. Actually, the Sitemaster, and staff are highly compensated … not in money, but by much appreciation for all their hard work to provide good current information to so many who follow this web site!!

    Regarding mpMRIs, I agree that results depend “heavily” on the experience and expertise of the radiologist. And, I believe it is still advisable based on PSA and DRE for men to have a 3 T mpMRI prior to agreeing to a TRUS biopsy. IF, I were Howard, I would have the MRI at a COE and forego another needle biopsy unless the MRI shows a lesion that could be targeted. Maybe an annual MRI, if it is a 3 + 4 = 7, would be one to ask doctor about.

  11. Thank you Robert … The appreciation is always much appreciated! :O)

  12. So when my recent MRI indicated a couple of suspicious areas and one invasion but bone scan was clean, are you saying the MRI is unreliable?

  13. Dear Charles:

    What I am saying is that a large group of experts in the use of MRI scans to assess risk for localized prostate cancer using the PI-RADS system to report the results of what they think they are seeing have reported that in their opinion this seems to be somewhat unreliable.

  14. There was a study in July 2019, with 193 men, that demonstrated an 89.4% negative predictive value by mpMRI for clinically significant prostate cancer. Between the two predictive values, I would say it’s worse to be told you don’t have it and do, than to be told you do have it and don’t.

  15. I am wondering to what extent the poor PPV demonstrated by the study is driven by the CYA factor, i.e. a radiologist will think it better to mark something as cancerous when here is the slightest indication.

    I can only draw on my experience in the x-ray context when — on two occasions — radiologists indicated something that was not there. In one case, the radiologist thought he saw an aortic aneurysm, as a result of which I spent $400 on a CT scan for nothing. I had a discussion with a doctor friend of mine about it, and he had a similar occurrence with a radiologist saying he had some dread condition when he didn’t. But I suppose one could argue the high NPV runs counter to this.

  16. Cliff:

    The bottom line to all of this is that the interpretation of imaging data is as much an art as it is a science — and like all art, its quality is open to personal opinion.

    There is certainly a degree to which radiologists are often saying that something they can see is “suspicious” of X or Y as opposed to actually being X or Y, and that also has to be taken into account. The PI-RADS methodology was meant to be able to take the “suspiciousness” into account when looking at prostate mpMRIs. What the study reported above tells us is that it doesn’t do that very well.

  17. I have had two T-3 MRI scans in the past 6 months to try and determine if the PI-RAD 3 score warrants a biopsy. I also had a third radiologist review the findings. My most recent PSA was 2.98 ng/ml, down from 4 from 6 months ago. My 4K Score was 8, 7.5 being the cutoff where they recommend no biopsy. At the time of my test, my PSA was 4.1 ng/ml. If I took the test today my score is likely to be lower and in the safe zone. A final test I did just recently was the ExoDx test and it was high. So I have some confusing data all around. None of the radiologists felt I needed to have an immediate biopsy but should do continued follow up with an additional MRI. Your article now makes me wonder who to believe. If two MRIs showed the same results and three radiologists felt I could do AS, are they wrong based on this study. Does anyone have experience with the ExoDx and 4K tests?

  18. Dear Michael:

    First, let me be clear. There is no such thing as a “safe zone” when it comes to PSA levels. The probability of a diagnosis of prostate cancer drops significantly if one’s PSA is down below about 3.0 ng/ml compared to 4.0 ng/ml, but men get diagnosed with PSA levels much lower than that.

    Second, I think the question you are really struggling with s whether you are willing to take the risk of having a biopsy as opposed to the risk of having prostate cancer. If the biopsy came back as indicating low-risk disease, then you would be able to go on active surveillance, possibly for years or forever. You can’t actually be on active surveillance until you have a diagnosis, but you can regularly monitor your PSA level.

    The problem with all the tests we have available at present is that only one can tell a doctor and a patient definitively that that patient does have prostate cancer. That’s the biopsy. But even a biopsy can’t tell a doctor and a patient definitively that that patient doesn’t have prostate cancer. Why? Because the biopsy may simply “miss” any cancer that is present.

    I understand this can be as frustrating as heck. … Unfortunately it is just the truth.

    At this point in time, you are the only person who can make the decision whether to have a biopsy or not. All anyone else can do is give you their opinion on whether a biopsy is really necessary.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: