Treating Google’s “Snippets” with Suspicion

Occasionally when you search for an answer to a question on Google, you will not only find websites, but you may also find a “knowledge panel” that appears to have what search expert Danny Sullivan calls the “One True Answer,” an answer that appears on a knowledge panel on top of the results.

Sometimes Google pulls an answer from a source algorithmically. For example, in response to “How many men landed on the moon?,” this panel answers “12 men,” citing a Quora article.

Sometimes Google does not pull out the answer but makes the answer apparent in the blurb or headline of the card, as in this answer to the query, “last person to walk on the moon”:

This function of Google can be useful, but it malfunctions frequently enough that it should not be trusted without verifying the source and context of the answer. There are two major problems: false simplicity and false (or non-standard) information.

False Simplicity

Here’s a question: how many apostles are there in the Christian tradition? Google tells you, via a panel, even pulling out the number, thereby making it look decidedly objective: there are twelve!

If you click through to that Quora question, though, you’ll find that it answers a much more specific and simpler question: how many original apostles did Christ have (according to tradition). And for that answer they are correct. Including Judas, there are twelve.

But according to tradition, when Judas dies Matthias becomes an apostle, so that’s thirteen. Then, Paul is an apostle, so fourteen. And Barnabas, Timothy, and James. The truth is that this answer is pretty debatable: it’s certainly not twelve, and some versions of the Bible refer to up to 25 different people as “apostles.”

It gets worse. These numbers, which are already various, come from various Christian traditions. Many historians, on the other hand, see the twelve apostles as a creation of the early Church, that had no reality or significance during the lifetime of the historical Jesus and was later “retrojected” into the Gospels.

The fact is the whole question of how many apostles there were and who they were is inextricably bound up with complex questions of religion, history, and 1st century power struggles about who counted in the early church and who didn’t.

This may seem petty, but the truth is any extended discussion of this issue from any source, religious or historical, will surface these issues to the person who investigates. Google‘s panels, however, are oblivious to this kind of complexity and present a simple numerical answer where no simple answer actually exists.

Misleading Highlights

Google uses some programming to try and highlight relevant answers in the blurb, but the highlighting is confused or confusing. Here, Google, when asked how old Lee Harvey Oswald was when he shot Kennedy, highlights 18, 24, and 22.

In reality, the answer is 24 years old, though a quick glance at this might have you thinking 18 or 22.

Blatant Misinformation

Sometimes the panel presents blatant misinformation. Often this material is the product of highly politicized areas or of conspiracy-believing communities, which tend to rank highly on Google search results more generally.

Take for instance this search, where we ask Google which presidents were in the Ku Klux Klan. The Google panel provides what seems to be a definitive answer: there were five!

As Case University Western history professor Peter Schulman points out, this isn’t even remotely true. None of these presidents were members of the Ku Klux Klan (as far as we know), and if you click through to the article, you’ll find the source here is a Nigerian newspaper of uncertain stature that references a book by David Barton, a nationalist known for self-publishing dubious works of historical revisionism.

Google will also tell you that Lee Harvey Oswald didn’t assassinate John F. Kennedy, despite the overwhelming evidence to the contrary:

Confirmation Bias and Bad Snippets

A lot of times Google is just bad. But sometimes bad answers are often the result of asking questions in ways that tap into the language or concerns of pseudoscience, conspiracy theory, or fringe beliefs. For example, there is a very real problem some people have with monosodium glutumate, a food additive that triggers an allergic reaction in a small portion of the population. If you search on a phrase likely to by found in the medical literature like “msg sensitivity,” you get a fairly reliable result.

Healthline, in this case, is a recognized provider of reliable health information.

All this changes if you use the language of fringe groups that believe the medical community is suppressing a link between MSG and a variety of neurological disorders. Here’s what you get when you type in ‘msg dangers’:

The blurb says it all (brain damage! alzheimer’s! learning disabilities!), but if you look up the site (mercola.com) you’ll find it is run by a physician who has been warned by the FDA repeatedly to stop making false claims.

Our Advice

In general, simply treat the Google panel (“one true answer”) as you would any other top search result. Despite Google‘s claims to the contrary, it is not significantly more or less reliable than an average source. Click through, trace the claims on the page to a source, and investigate the source. Never trust its result without validating the source of the claim.

Source

Web Literacy for Student Fact-Checkers Copyright © 2017 by Michael A. Caulfield is licensed under a Creative Commons Attribution 4.0 International License