Will super-smart computers make our lives better?

Watson impressive on Jeopardy

I’m hearing a lot about Watson, the IBM super computer designed to process and analyze natural language that has just defeated the two most successful human champions ever to have appeared on the TV show “Jeopardy.” I even went to a presentation at the computer science building at the University of Maryland hosted by Dennis Shea, a senior manager at IBM’s Watson Research Center.* Before watching an early airing of the first show together, the large audience saw two short videos highlighting what the team is trying to accomplish and some of the ramifications for this kind of computing in aspect of our lives in the future that go well beyond answering Jeopardy questions. Imagine some of the applications of computers that understand complex natural language with a high degree of accuracy and respond with the information in vast data bases or the entire Internet.

There are already plans to collaborate with Columbia University and the University of Maryland to create a physician’s assistant service that will let doctors query a cybernetic assistant. And IBM is in discussions with a major consumer electronics retailer to develop a version of Watson that would be able to interact with consumers on a variety of subjects like buying decisions and technical support.

There’s a massive and increasing amount of data out there, and most of it is in the form of natural language, not coded data. There are revolutionary possibilities for science, medicine, law, business, and a variety of other fields, as natural language processing and machine learning develops.

During the question and answer session at the University of Maryland presentation, one of the computer science professors told the packed room that he was watching comments about Watson come through on twitter on his mobile device. (I couldn’t help reflecting on the somewhat ironically recursive use of information technology bringing real-time comments from a virtual community into the discussion by a large face-to-face group about this new natural language information technology breakthrough.) Most of the tweets mentioned were jokes about Watson diabolically taking control over the humans. It occurred to me that the IBM programmers could have decided to program Watson to make some such comment during the show. In a follow-up interview, Dr. David Ferrucci or some other lead programmer could momentarily deny that they had done so, claiming that Watson had come up with the comment on its own….

During the Jeopardy shows, Watson seemed brilliant most of the time, but occasionally it’s answers were way off, seeming stupid by normal human standards. For example: “In May 2010, five paintings worth $125 million by Braque, Matisse & three others left Paris’ Museum of this art period.” Latching on to the the keywords “3 others,” Watson responded “Picasso.” I think most people know Picasso was not an art period. The correct response was “modern art.”

In response to “Its largest airport was named for a World War II hero; its second largest, for a World War II battle,” under the category “US Cities,” Watson guessed “Toronto.” The responses are selected from a large number of candidate answers based on statistical algorithms; no “check” is performed to make sure an answer is eliminated because it’s outside a certain category. Besides, Watson had learned that the Jeopardy category headings often have a somewhat vague connection with the question. On the other hand, Watson would have given much more weight to U.S. cities if the phrase “What US city…” had been in the Jeopardy “answer” itself. Adding to the confusion for Watson, there are at least 7 U.S. cities named Toronto (though most of us haven’t even heard of them, and I highly doubt any of them has two airports); also Toronto, Canada has an American League baseball team. Watson’s confidence level was only about 30%, too low to buzz in under normal circumstances. Watson bet only $947 for this final Jeopardy category, leaving it with a combined total for the first two days of $35,734 compared to $10,400 for Brad Rutter and $4,800 for Ken Jennings.

Also, Watson’s designers thought it was extremely unlikely that one of the human Jeopardy champions would give a wrong answer that Watson would then select as its first choice, so they didn’t plan for it. Watson can’t see or hear; the “answer” is sent to it by text at the same moment Alex Trebek finishes reading it. So on a “Name That Decade” answer, when Ken Jennings responded incorrectly with “what is the 1920s?,” Watson buzzed in with his own response: “What is the 1920s?”

Chris Welty, a member of Watson’s algorithms team who was at the Jeopardy tapings, revealed that Watson almost managed to sneak a wrong answer past Alex Trebek. On an “Olympic Oddities” answer, Jennings responded incorrectly that Olympian gymnast George Eyser was “missing a hand.” Watson then buzzed in and responded, “What is a leg?” If a human competitor had responded this way, we might assume he or she was following up in the context of the previous incorrect response, but since Watson couldn’t hear Ken Jennings, no such leeway can be given. At first Trebek accepted Watson’s response, but the taping had to be stopped and the sequence re-shot because he’d forgotten that Watson wasn’t aware of the context created by Jenning’s response.

Final grand totals at the end of Day 3 were Brad Rutter: $21,600, Ken Jennings: $24,000, and Watson: $77,147.

If you’re interested in learning more about Watson, natural language processing, and machine learning, you might want to check out the PBS Nova special.

How do you think computers that can respond to natural language might enhance the quality of life? What problems do you see? If you are a social science researcher or other specialist, how would you use this kind of computational ability in your field? Feel free to leave any comments below.

*The Watson super computer, and the Watson Research Center in Yorktown Heights, NY where it was developed, are named for IBM founder Thomas J. Watson. I’m pretty sure it was IBM Research’s David Gondek (who you see on all the Watson videos) who was sitting in the middle of the audience and joining in on the conversation on Monday night at the presentation at the University of Maryland. Btw, for those of you who are curious, Watson’s software was written in programming languages Java and C++.

Meaning and Happiness HOME

Enter keywords...