A meta-analysis is only a meta-analysis … not proof of a belief …


A recently published meta-analysis and review in the journal European Urology is probably going to help an awful lot of urologic surgeons believe what they want to believe — that radical prostatectomy really is a better first-line treatment for localized prostate cancer than radiation therapy. The truth may be rather different.

The problem is that, while this meta-analysis by Wallis et al. is quite certainly better than any other meta-analysis ever done on this subject, it is a meta-analysis that is based on what are still, generally, excessively diverse and “elderly” data sets, and that does not take account of the detailed diagnosis and treatment of each individual patient. We acknowledge that two of the authors hold positions in departments of radiation oncology and are experienced in the treatment of genitourinary cancer. Frankly, we are very surprised that they were willing to lend their names to this article as written.

Here are just a few of the critical problems:

  • We can’t and don’t know things like the Gleason scores, the pre-treatment PSA levels, or the clinical stages of all the patients in this study.
  • The time frame of when patients were actually being treated in the cohorts assessed in the meta-analysis goes back many years (and so the management they received may be very different from the types of treatment they could or should receive today).
  • Large numbers of the patients included in this study probably could and should (today) be managed by some form of simple monitoring as opposed to any form of invasive treatment.
  • There is absolutely no mention whatsoever of the relative levels of complications and side effects associated with the two types of treatment.

The authors themselves clearly note several of the problems associated with the data on which this meta-analysis is based (although they seem to under-emphasize the significance of those problems).

While The “New” Prostate Cancer InfoLink applauds the authors for their efforts and the statistical rigor of this meta-analysis, we believe that the conclusion expressed in the abstract to this paper is seriously flawed. That conclusion reads as follows:

Radiotherapy for prostate cancer is associated with an increased risk of overall and prostate cancer-specific mortality compared with surgery based on observational data with low to moderate risk of bias. These data, combined with the forthcoming randomized data, may help clinical decision making.

In our very humble opinion, a far better and more accurate conclusion to this study would read more like this:

This meta-analysis indicates that, historically, men treated with largely outdated modes of radiation therapy for prostate cancer had an increased risk of overall and prostate cancer-specific mortality compared to those treated by radical prostatectomy. However, it is inappropriate to use these data to make clinical recommendations about treatment for individual patients being diagnosed with prostate cancer today because of major changes in the guidelines for treatment, the current availability of more sophisticated forms of radiation therapy, and because we will soon have data from the randomized ProtecT trial which may give us far better guidance as to the quality of outcomes of a large and well-documented cohort of men with localized disease.

We strongly suggest to any patient who is told that this new meta-analysis “proves” that surgery is a better form of first-line treatment than radiation therapy for localized prostate cancer to ask the person telling him this whether: (a) they have actually even read the paper; (b) whether they understand that (as stated by the authors):

A meta-analysis depends on the validity of the included studies to draw conclusions. Therefore a key limitation of our study is the effect of residual confounding as this analysis is based on observational data. It is well established that patients treated by radiotherapy tend to be older and have a higher level of comorbidity. … Also, the use of salvage therapies may explain some of the survival differences between groups. … Also, for overall survival, there were insufficient data to assess the efficacy of IMRT, which has largely supplanted three-dimensional CRT in many jurisdictions. …

In fact, the data being analyzed in this study are so old that (as far as we can tell) only one of the radiotherapy studies included in the meta-analysis is based on IMRT at all, and only had 5 years of follow-up.

We expect this paper to receive considerable publicity. However, it is our opinion that such publicity will be largely inaccurate and inappropriate. This commentary on the ScienceDaily web site (and based on information provided by the European Association of Urology) typifies how the urology community is likely to “package” these data.

The really sad thing is that if the authors had made a small, additional effort to be extremely clear about the inherent problems with the data being used to conduct this meta-analysis, and that the value of the meta-analysis to clinical decisions about the treatment of men being diagnosed and treated today is really very limited, this might actually have been a valuable paper!

3 Responses

  1. I thought I’d add this link (http://www.thegreenjournal.com/article/S0167-8140(16)31033-7/fulltext) for any who are interested. This letter to the editor from Anthony Zietman (probably the most respected name in radiation oncology) and others very much supports the Sitemaster’s argument. The only point I’d take issue with is that the ProtecT clinical trial will solve the issue – the radiation dose used was low by today’s standards.

  2. Dear Allen:

    I don’t think the ProtecT study will “solve the issue” at all (and I don’t think I suggested that). It is only going to be relevant to men with relatively low-risk forms of prostate cancer. However, those men arguably don’t need high-dose radiation anyway (if they have radiation therapy), and so in the set of men enrolled in the ProtecT trial the outcomes may be extremely relevant.

  3. No, you didn’t suggest that, I meant that Zietman did in the link I provided. :-)

    I agree with you, the most important outcome of ProtecT will be the comparison to active surveillance.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: