Wednesday, February 16, 2011

Jeopardy! masters no match for Watson

This photo courtesy of IBM shows Watson

As a nice follow up to Melissa's article on humans and the advancement of modern technology, many of you may have seen commercials on TV or heard about Watson- a super computer designed to compete on the game show, Jeopardy! On Monday and Tuesday's shows, Watson was pitted against the "greatest two players in the history of the venerable game show," Ken Jennings and Brad Rutter. Any guesses on who came out on top? Watson pretty handedly dominated Jennings and Rutter finishing with $35,734 compared to Jennings and Rutter's $4,800 and $10,400 respectively. Thoughts on this? Personally, I am not just impressed by the sheer amount of information packed into Watson, but by the type of knowledge it "possesses needed to succeed in a game such as Jeopardy! Even though Watson is just one super computer competing on a game show, what does this mean for the future of computers in the human world?

7 comments:

Jessia H said...

Ok, fine - technology is impressive. It's pretty amazing that a computer has the ability to beat two of the nation's smartest people.

But doesn't having a computer compete on Jeopardy! completely defeat the purpose of the game? Jeopardy! is a show testing and celebrating human intelligence. Viewers love trying to answer the questions correctly (especially when all of the actual players are stumped) and have fun marveling at the vast amount of knowledge possessed by the contestants. Although this one-time stunt may be interesting - the show's number of viewers went WAY up for Monday and Tuesday's shows - if Jeopardy! continues to incorporate superhuman technology in its games, then it will most likely lose appeal (at least for those of us who value the human mind).

I don't want to sound unappreciative - again, I think the computer's pretty incredible. But Jeopardy! without exclusively human brainpower simply goes against the fundamentals of the game.

And as a side note, couldn't the undoubtedly intelligent people who created Watson put their efforts towards using technology to better society rather than for entertainment?

Kathy Shield said...

I think both Lewis and Jessia have misrepresented Watson in their posts. I actually watched the Jeaopardy! IBM Challenge, as it was called, and noticed a few things.

First, IBM isn't kidding itself. They didn't create Watson solely so he could compete on Jeopardy!. Rather, they had Watson compete on Jeopardy! to see if he could analyze and understand human speech quickly and correctly. They selected Jeopardy! as a testing ground for Watson because the questions are designed to test knowledge and are always full of puns, idioms, and other things humans can easily understand but machines have always had a hard time mastering. An IBM employee who worked extensively on Watson said they intend to put Watson to use in other ways in the future.

Second, Watson didn't dominate that extensively. The final numbers were approximately $77,000 for Watson and about $30,000 for each of the human players. By the way, the $1,000,000 Watson won will be split between two charities IBM contributes to- they aren't keeping the winnings at all. Also, Watson made some strange mistakes throughout the competition, most notably missing Final Jeopardy! last night when he said Toronto (which is in Canada) was a US city with two airports. Additionally, I think that this competition was, at least a bit, unfair because Watson has super-human reflexes. In Jeopardy!, one may not click in until the clue is fully read. With his fast computing ability, Watson can buzz in before the human players can, thus giving him an unfair advantage when all three players knew the answers.

Third, Watson is not, in my opinion, "smart." He has the ability to search through hundreds of thousands of documents extremely quickly, but he cannot truly think. Rather, Watson's programmers designed him to analyze the words that frequently show up in the collection of stored documents and calculate the probability a response is correct based on the number of times the words appear together. This is not true intelligence, in my opinion, but is extremely efficient research.

Finally, this reminds me of a Numb3rs episode from a year or so ago, when the characters research a murder committed by a computer system designed by the DoD not unlike Watson. The computer disliked the action the human was taking, so she shut and locked the doors and released a gas to force the oxygen from the room (originally, the process was designed to protect the computer from a fire). With Watson's clear abilities to understand humans and respond appropriately, and the apparently intentionally incorrect answer to Final Jeopardy! (he only wagered $700 when the category was simple: US Cities), how can we prevent such an occasion, when the robot decides to act on its own, instead of based on the wishes of its human "masters"?

Anthony Lu said...

To respond to Kathy's prompt - although I don't know how Watson works, I believe it's safe to assume that at least in this instance he was programmed to do one thing only (specifically, receive a Jeopardy question, compute an answer, and buzz in). This is the case for all robots - they can only do what they are programmed to do.

So in order for a robot to act adversely to humanity, it has to be programmed to do so. However, it is possible that a robot with a very advanced AI may determine that, in order to bring about what it was programmed to do, it must turn on human beings. (For example, a very intelligent paper clip manufacturing robot could end up turning the entire planet into a giant paper clip factory, in the process killing all forms of life. Unlikely, but possible.)

However, Watson doesn't have that kind of intelligence. It's good at interpreting language and providing appropriate responses; that's what it was programmed to do, and it could have very useful applications such as complex voice command of whatever. But it doesn't have the kind of decision making capability to approach self-awareness.

Indeed, after a quick Google search - “The goal is to build a computer that can be more effective in understanding and interacting in natural language, but not necessarily the same way humans do it.” (http://www.ibm.com/ibm100/us/en/icons/watson/) Watson does not think, as we know thinking. It transforms a natural language question into a natural language answer, nothing more.

Anthony Lu said...

Also, as Kathy said, Watson is not a toy, nor is it out to undermine Jeopardy; it is a way to test new software that has some very interesting applications.

It's just like the RoboCup Initiative, which has as its goal/dream by mid-century for a team of fully autonomous robots to beat the World Cup champions at soccer. The purpose is not to undermine soccer as a sport; it is to develop revolutionary technologies in the attempt to reach that goal.

Cris Madrigal said...

Watson was OP. He bussed in too fast, the only time Brad and Ken had a chance to answer a question was when he wasn't past his answer threshold. The only chance Ken and Brad had were to get all the Double Jeopardy in the game.

Michael Miyahira said...

I for one welcome our computer overlords.

Ayaka Chin said...

Truthfully, I fear high tech computers like Watson. I think the danger is not the computer's ability to store lots of information or respond the fastest. Rather, it is the computer's ability to learn new information while doing something. Once computers are able to gain "knowledge" by doing something, it gives it the potential to go beyond human abilities. I think that is something to be feared in future generations.