Last week Paul Levy, one of the early hospital CEO bloggers who is now blogging at Not Running a Hospital, came across what he thought was a significant issue with Google’s new medical search knowledge panels that incorporate information from Mayo Clinic and other sources.
The main issue he raised related to the Prostate Cancer panel, and particularly that robotic surgery was listed first among surgical procedures (instead of alphabetically), with watchful waiting listed among “Other treatments” under the “Also common” heading. Or as he put it:
Observers want to know: Does this nonalphabetic list represent Mayo Clinic’s view of the most likely, highest priority, or most recommended approach to this disease?
Open the prostate cancer search in a new window and review the panel for yourself. Then come back here for some thoughts. I’ll wait.
Since he published his post, Mr. Levy has tweeted about the topic at least 15 times, starting with this one..
Also, on Saturday Mr. Levy published a related post, this time asking whether Mayo reviewed the “Ad” about Multiple Sclerosis, and questioning its accuracy.
Since he specifically mentioned me in the first post, and because I work at Mayo Clinic, I looked into the process. Far from being anything sinister, I think it’s a really positive educational service that puts reliable health information at the top of Google’s health search results, above any advertising.
Here’s how I understand what’s happening:
- Mayo Clinic provided a brief definition for each of about 400 diseases and conditions.
- Google drew information from various sources as listed here, (MayoClinic.org was one of them), and created a three-panel display for each topic with an overview, symptoms and treatments.
- Mayo Clinic staff reviewed the content of the panels, including treatments ranging from medicines to surgical procedures to diet and lifestyle modification. This did not include a review of list ordering, but only whether the treatments in the list are among the valid options.
- Each panel includes a link to the relevant page on MayoClinic.org, such as this one on prostate cancer, for a complete and balanced review of the topic.
- These Knowledge Panels aren’t ads.
Back to Mr. Levy’s question…
Observers want to know: Does this nonalphabetic list represent Mayo Clinic’s view of the most likely, highest priority, or most recommended approach to this disease?
The reality is, these panels do contain direct deep links to Mayo Clinic’s view for all 400+ diseases and conditions, ranging from ADHD and ALS to Heart Attack, PTSD and viral gastroenteritis.
And yes, prostate cancer too.
I’m more mystified at Mr. Levy’s criticism of the MS panel, especially as expressed in this tweet:
The issue he raised is with the number of cases:
Mr. Levy says he did some searching in the National MS Society Web site and found that the estimated number of cases worldwide (his emphasis) is only 2.3 million. But here is the quote in context:
More than 2.3 million people are affected by MS worldwide. Because the Centers for Disease Control and Prevention (CDC) does not require U.S. physicians to report new cases, and because symptoms can be completely invisible, the prevalence of MS in the U.S. can only be estimated. The Society continues to advocate for the establishment of a national registry that will track the number of people living with MS and has made a commitment to re-evaluate the current prevalence estimate and investigate the process by which an updated estimate can be identified. (Emphasis mine this time.)
The Google Knowledge Panel, which Mr. Levy says is “impairing Mayo Clinic’s credibility,” puts the number of U.S. cases of MS in the 200,000 to 3 million range. This WHO report estimates the prevalence at 135 per 100,000 population in the U.S.
The countries reporting the highest estimated prevalence of MS include Hungary (176 per 100 000), Slovenia (150), Germany (149), United States of America (135), Canada (132.5), Czech Republic (130), Norway (125), Denmark (122), Poland (120) and Cyprus (110).
That would put the number of cases in the U.S. at something well over 400,000. And the MS Society seems to indicate (from the passage quoted above) that U.S. cases are likely understated.
The Google Knowledge Panel uses four classes of disease prevalence, based on number of U.S. cases:
- Very Rare (<20K cases)
- Rare (20K to 200K)
- Common (200K to 3 million)
- Very Common (More than 3 million)
The categories are admittedly broad: Ebola is accurately in the Very Rare group, although the number of U.S. cases is way below 20K. It’s less than 20.
The range is big, but it’s not wrong.
And 400,000 cases of MS in the United States is in the Common range.
Finally, these Knowledge Panels would have happened with or without Mayo’s participation. We had the opportunity to share some of our knowledge via the definitions. We provided content review to help ensure that accurate, reliable health information appears at the top of search results, and and we offered direct links to deeper information on MayoClinic.org. We started with 400 medical topics and plan to work with Google to add up to 600 more.
I hope these answers to Mr. Levy’s questions are helpful.
I’m confident that the panels themselves will be.
