IBM Watson

Watson uses IBM's DeepQA software and the Apache UIMA (Unstructured Information Management Architecture) framework implementation. The system was written in various languages, including Java, C++, and Prolog, and runs on the SUSE Linux Enterprise Server 11 operating system using the Apache Hadoop framework to provide distributed computing.[11][12][13]

The system is workload-optimized, integrating massively parallel POWER7 processors and built on IBM's DeepQA technology,[14] which it uses to generate hypotheses, gather massive evidence, and analyze data.[1] Watson employs a cluster of ninety IBM Power 750 servers, each of which uses a 3.5 GHz POWER7 eight-core processor, with four threads per core. In total, the system uses 2,880 POWER7 processor threads and 16 terabytes of RAM.[14]

According to John Rennie, Watson can process 500 gigabytes (the equivalent of a million books) per second.[15] IBM master inventor and senior consultant Tony Pearson estimated Watson's hardware cost at about three million dollars.[16] Its Linpack performance stands at 80 TeraFLOPs, which is about half as fast as the cut-off line for the Top 500 Supercomputers list.[17] According to Rennie, all content was stored in Watson's RAM for the Jeopardy game because data stored on hard drives would be too slow to compete with human Jeopardy champions.[15]

The sources of information for Watson include encyclopedias, dictionaries, thesauri, newswire articles and literary works. Watson also used databases, taxonomies and ontologies including DBPedia, WordNet and Yago.[18] The IBM team provided Watson with millions of documents, including dictionaries, encyclopedias and other reference material, that it could use to build its knowledge.[19]

Ken Jennings, Watson, and Brad Rutter in their Jeopardy! exhibition match

Watson's basic working principle is to parse keywords in a clue while searching for related terms as responses. This gives Watson some advantages and disadvantages compared with human Jeopardy! players.[21] Watson has deficiencies in understanding the contexts of the clues. Watson can read, analyze, and learn from natural language which gives it the ability to make human-like decisions.[22] As a result, human players usually generate responses faster than Watson, especially to short clues.[19] Watson's programming prevents it from using the popular tactic of buzzing before it is sure of its response.[19] However, Watson has consistently better reaction time on the buzzer once it has generated a response, and is immune to human players' psychological tactics, such as jumping between categories on every clue.[19][23]

In a sequence of 20 mock games of Jeopardy, human participants were able to use the six to seven seconds that Watson needed to hear the clue and decide whether to signal for responding.[19] During that time, Watson also has to evaluate the response and determine whether it is sufficiently confident in the result to signal.[19] Part of the system used to win the Jeopardy! contest was the electronic circuitry that receives the "ready" signal and then examined whether Watson's confidence level was great enough to activate the buzzer. Given the speed of this circuitry compared to the speed of human reaction times, Watson's reaction time was faster than the human contestants except when the human anticipated (instead of reacted to) the ready signal.[24] After signaling, Watson speaks with an electronic voice and gives the responses in Jeopardy!'s question format.[19] Watson's voice was synthesized from recordings that actor Jeff Woodman made for an IBM text-to-speech program in 2004.[25]

The Jeopardy! staff used different means to notify Watson and the human players when to buzz,[24] which was critical in many rounds.[23] The humans were notified by a light, which took them tenths of a second to perceive.[26][27] Watson was notified by an electronic signal and could activate the buzzer within about eight milliseconds.[28] The humans tried to compensate for the perception delay by anticipating the light,[29] but the variation in the anticipation time was generally too great to fall within Watson's response time.[23] Watson did not attempt to anticipate the notification signal.[27][29]

Since Deep Blue's victory over Garry Kasparov in chess in 1997, IBM had been on the hunt for a new challenge. In 2004, IBM Research manager Charles Lickel, over dinner with coworkers, noticed that the restaurant they were in had fallen silent. He soon discovered the cause of this evening's hiatus: Ken Jennings, who was then in the middle of his successful 74-game run on Jeopardy!. Nearly the entire restaurant had piled toward the televisions, mid-meal, to watch Jeopardy!. Intrigued by the quiz show as a possible challenge for IBM, Lickel passed the idea on, and in 2005, IBM Research executive Paul Horn supported Lickel, pushing for someone in his department to take up the challenge of playing Jeopardy! with an IBM system. Though he initially had trouble finding any research staff willing to take on what looked to be a much more complex challenge than the wordless game of chess, eventually David Ferrucci took him up on the offer.[30] In competitions managed by the United States government, Watson's predecessor, a system named Piquant, was usually able to respond correctly to only about 35% of clues and often required several minutes to respond.[31][32][33] To compete successfully on Jeopardy!, Watson would need to respond in no more than a few seconds, and at that time, the problems posed by the game show were deemed to be impossible to solve.[19]

In initial tests run during 2006 by David Ferrucci, the senior manager of IBM's Semantic Analysis and Integration department, Watson was given 500 clues from past Jeopardy! programs. While the best real-life competitors buzzed in half the time and responded correctly to as many as 95% of clues, Watson's first pass could get only about 15% correct. During 2007, the IBM team was given three to five years and a staff of 15 people to solve the problems.[19] John E. Kelly III succeeded Paul Horn as head of IBM Research in 2007.[34] InformationWeek described Kelly as "the father of Watson" and credited him for encouraging the system to compete against humans on Jeopardy!.[35] By 2008, the developers had advanced Watson such that it could compete with Jeopardy! champions.[19] By February 2010, Watson could beat human Jeopardy! contestants on a regular basis.[36]

During the game, Watson had access to 200 million pages of structured and unstructured content consuming four terabytes of disk storage[11] including the full text of the 2011 edition of Wikipedia,[37] but was not connected to the Internet.[38][19] For each clue, Watson's three most probable responses were displayed on the television screen. Watson consistently outperformed its human opponents on the game's signaling device, but had trouble in a few categories, notably those having short clues containing only a few words.

Although the system is primarily an IBM effort, Watson's development involved faculty and graduate students from Rensselaer Polytechnic Institute, Carnegie Mellon University, University of Massachusetts Amherst, the University of Southern California's Information Sciences Institute, the University of Texas at Austin, the Massachusetts Institute of Technology, and the University of Trento,[9] as well as students from New York Medical College.[39] Among the team of IBM programmers who worked on Watson was 2001 Who Wants to Be a Millionaire? top prize winner Ed Toutant, who himself had appeared on Jeopardy! in 1989 (winning one game).[40]

Watson demo at an IBM booth at a trade show

In 2008, IBM representatives communicated with Jeopardy! executive producer Harry Friedman about the possibility of having Watson compete against Ken Jennings and Brad Rutter, two of the most successful contestants on the show, and the program's producers agreed.[19][41] Watson's differences with human players had generated conflicts between IBM and Jeopardy! staff during the planning of the competition.[21] IBM repeatedly expressed concerns that the show's writers would exploit Watson's cognitive deficiencies when writing the clues, thereby turning the game into a Turing test. To alleviate that claim, a third party randomly picked the clues from previously written shows that were never broadcast.[21] Jeopardy! staff also showed concerns over Watson's reaction time on the buzzer. Originally Watson signaled electronically, but show staff requested that it press a button physically, as the human contestants would.[42] Even with a robotic "finger" pressing the buzzer, Watson remained faster than its human competitors. Ken Jennings noted, "If you're trying to win on the show, the buzzer is all", and that Watson "can knock out a microsecond-precise buzz every single time with little or no variation. Human reflexes can't compete with computer circuits in this regard."[23][29][43] Stephen Baker, a journalist who recorded Watson's development in his book Final Jeopardy, reported that the conflict between IBM and Jeopardy! became so serious in May 2010 that the competition was almost cancelled.[21] As part of the preparation, IBM constructed a mock set in a conference room at one of its technology sites to model the one used on Jeopardy!. Human players, including former Jeopardy! contestants, also participated in mock games against Watson with Todd Alan Crain of The Onion playing host.[19] About 100 test matches were conducted with Watson winning 65% of the games.[44]

To provide a physical presence in the televised games, Watson was represented by an "avatar" of a globe, inspired by the IBM "smarter planet" symbol. Jennings described the computer's avatar as a "glowing blue ball crisscrossed by 'threads' of thought—42 threads, to be precise",[45] and stated that the number of thought threads in the avatar was an in-joke referencing the significance of the number 42 in Douglas Adams' Hitchhiker's Guide to the Galaxy.[45] Joshua Davis, the artist who designed the avatar for the project, explained to Stephen Baker that there are 36 trigger-able states that Watson was able to use throughout the game to show its confidence in responding to a clue correctly; he had hoped to be able to find forty-two, to add another level to the Hitchhiker's Guide reference, but he was unable to pinpoint enough game states.[46]

A practice match was recorded on January 13, 2011, and the official matches were recorded on January 14, 2011. All participants maintained secrecy about the outcome until the match was broadcast in February.[47]

In a practice match before the press on January 13, 2011, Watson won a 15-question round against Ken Jennings and Brad Rutter with a score of $4,400 to Jennings's $3,400 and Rutter's $1,200, though Jennings and Watson were tied before the final $1,000 question. None of the three players responded incorrectly to a clue.[48]

The first round was broadcast February 14, 2011, and the second round, on February 15, 2011. The right to choose the first category had been determined by a draw won by Rutter.[49] Watson, represented by a computer monitor display and artificial voice, responded correctly to the second clue and then selected the fourth clue of the first category, a deliberate strategy to find the Daily Double as quickly as possible.[50] Watson's guess at the Daily Double location was correct. At the end of the first round, Watson was tied with Rutter at $5,000; Jennings had $2,000.[49]

Watson's performance was characterized by some quirks. In one instance, Watson repeated a reworded version of an incorrect response offered by Jennings. (Jennings said "What are the '20s?" in reference to the 1920s. Then Watson said "What is 1920s?") Because Watson could not recognize other contestants' responses, it did not know that Jennings had already given the same response. In another instance, Watson was initially given credit for a response of "What is a leg?" after Jennings incorrectly responded "What is: he only had one hand?" to a clue about George Eyser (the correct response was, "What is: he's missing a leg?"). Because Watson, unlike a human, could not have been responding to Jennings's mistake, it was decided that this response was incorrect. The broadcast version of the episode was edited to omit Trebek's original acceptance of Watson's response.[51] Watson also demonstrated complex wagering strategies on the Daily Doubles, with one bet at $6,435 and another at $1,246.[52] Gerald Tesauro, one of the IBM researchers who worked on Watson, explained that Watson's wagers were based on its confidence level for the category and a complex regression model called the Game State Evaluator.[53]

Watson took a commanding lead in Double Jeopardy!, correctly responding to both Daily Doubles. Watson responded to the second Daily Double correctly with a 32% confidence score.[52]

However, during the Final Jeopardy! round, Watson was the only contestant to miss the clue in the category U.S. Cities ("Its largest airport was named for a World War II hero; its second largest, for a World War II battle"). Rutter and Jennings gave the correct response of Chicago, but Watson's response was "What is Toronto?????" with five question marks appended indicating a lack of confidence.[52][54][55] Ferrucci offered reasons why Watson would appear to have guessed a Canadian city: categories only weakly suggest the type of response desired, the phrase "U.S. city" did not appear in the question, there are cities named Toronto in the U.S., and Toronto in Ontario has an American League baseball team.[56] Chris Welty, who also worked on Watson, suggested that it may not have been able to correctly parse the second part of the clue, "its second largest, for a World War II battle" (which was not a standalone clause despite it following a semicolon, and required context to understand that it was referring to a second-largest airport).[57] Eric Nyberg, a professor at Carnegie Mellon University and a member of the development team, stated that the error occurred because Watson does not possess the comparative knowledge to discard that potential response as not viable.[55] Although not displayed to the audience as with non-Final Jeopardy! questions, Watson's second choice was Chicago. Both Toronto and Chicago were well below Watson's confidence threshold, at 14% and 11% respectively. Watson wagered only $947 on the question.

The game ended with Jennings with $4,800, Rutter with $10,400, and Watson with $35,734.[52]

During the introduction, Trebek (a Canadian native) joked that he had learned Toronto was a U.S. city, and Watson's error in the first match prompted an IBM engineer to wear a Toronto Blue Jays jacket to the recording of the second match.[58]

In the first round, Jennings was finally able to choose a Daily Double clue,[59] while Watson responded to one Daily Double clue incorrectly for the first time in the Double Jeopardy! Round.[60] After the first round, Watson placed second for the first time in the competition after Rutter and Jennings were briefly successful in increasing their dollar values before Watson could respond.[60][61] Nonetheless, the final result ended with a victory for Watson with a score of $77,147, besting Jennings who scored $24,000 and Rutter who scored $21,600.[62]

The prizes for the competition were $1 million for first place (Watson), $300,000 for second place (Jennings), and $200,000 for third place (Rutter). As promised, IBM donated 100% of Watson's winnings to charity, with 50% of those winnings going to World Vision and 50% going to World Community Grid.[63] Similarly, Jennings and Rutter donated 50% of their winnings to their respective charities.[64]

In acknowledgement of IBM and Watson's achievements, Jennings made an additional remark in his Final Jeopardy! response: "I for one welcome our new computer overlords", paraphrasing a joke from The Simpsons.[65][66] Jennings later wrote an article for Slate, in which he stated:

IBM has bragged to the media that Watson's question-answering skills are good for more than annoying Alex Trebek. The company sees a future in which fields like medical diagnosis, business analytics, and tech support are automated by question-answering software like Watson. Just as factory jobs were eliminated in the 20th century by new assembly-line robots, Brad and I were the first knowledge-industry workers put out of work by the new generation of 'thinking' machines. 'Quiz show contestant' may be the first job made redundant by Watson, but I'm sure it won't be the last.[45]

Philosopher John Searle argues that Watson—despite impressive capabilities—cannot actually think.[67] Drawing on his Chinese room thought experiment, Searle claims that Watson, like other computational machines, is capable only of manipulating symbols, but has no ability to understand the meaning of those symbols; however, Searle's experiment has its detractors.[68]

On February 28, 2011, Watson played an untelevised exhibition match of Jeopardy! against members of the United States House of Representatives. In the first round, Rush D. Holt, Jr. (D-NJ, a former Jeopardy! contestant), who was challenging the computer with Bill Cassidy (R-LA, later Senator from Louisiana), led with Watson in second place. However, combining the scores between all matches, the final score was $40,300 for Watson and $30,000 for the congressional players combined.[69]

IBM's Christopher Padilla said of the match, "The technology behind Watson represents a major advancement in computing. In the data-intensive environment of government, this type of technology can help organizations make better decisions and improve how government helps its citizens."[69]

In healthcare, Watson has been used to analyze medical data and assist doctors in making diagnoses and treatment decisions, including in areas such as oncology and radiology.[94] Watson's natural language, hypothesis generation, and evidence-based learning capabilities are being investigated to see how Watson may contribute to clinical decision support systems.[95] To aid physicians in the treatment of their patients, once a physician has posed a query to the system describing symptoms and other related factors, Watson first parses the input to identify the most important pieces of information; then mines patient data to find facts relevant to the patient's medical and hereditary history; then examines available data sources to form and test hypotheses;[95] and finally provides a list of individualized, confidence-scored recommendations.[96] The sources of data that Watson uses for analysis can include treatment guidelines, electronic medical record data, notes from healthcare providers, research materials, clinical studies, journal articles and patient information.[95] Despite being developed and marketed as a "diagnosis and treatment advisor", Watson has never been actually involved in the medical diagnosis process, only in assisting with identifying treatment options for patients who have already been diagnosed.[97]

In February 2011, it was announced that IBM would be partnering with Nuance Communications for a research project to develop a commercial product during the next 18 to 24 months, designed to exploit Watson's clinical decision support capabilities. Physicians at Columbia University would help to identify critical issues in the practice of medicine where the system's technology may be able to contribute, and physicians at the University of Maryland would work to identify the best way that a technology like Watson could interact with medical practitioners to provide the maximum assistance.[98]

In September 2011, IBM and WellPoint (now Anthem) announced a partnership to utilize Watson to help suggest treatment options to physicians.[99] Then, in February 2013, IBM and WellPoint gave Watson its first commercial application, for utilization management decisions in lung cancer treatment at Memorial Sloan–Kettering Cancer Center.[8]

IBM announced a partnership with Cleveland Clinic in October 2012. The company has sent Watson to the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, where it will increase its health expertise and assist medical professionals in treating patients. The medical facility will utilize Watson's ability to store and process large quantities of information to help speed up and increase the accuracy of the treatment process. "Cleveland Clinic's collaboration with IBM is exciting because it offers us the opportunity to teach Watson to 'think' in ways that have the potential to make it a powerful tool in medicine", said C. Martin Harris, MD, chief information officer of Cleveland Clinic.[100]

In 2013, IBM and MD Anderson Cancer Center began a pilot program to further the center's "mission to eradicate cancer".[101][102] However, after spending $62 million, the project did not meet its goals and it has been stopped.[103]

On February 8, 2013, IBM announced that oncologists at the Maine Center for Cancer Medicine and Westmed Medical Group in New York have started to test Watson in an effort to recommend treatment for lung cancer.[104]

On July 29, 2016, IBM and Manipal Hospitals[105][106][107] (a leading hospital chain in India) announced the launch of IBM Watson for Oncology, for cancer patients. This product provides information and insights to physicians and cancer patients to help them identify personalized, evidence-based cancer care options. Manipal Hospitals is the second hospital[108] in the world to adopt this technology and first in the world to offer it to patients online as an expert second opinion through their website.[105][109] Manipal discontinued this contract in December 2018.

On January 7, 2017, IBM and Fukoku Mutual Life Insurance entered into a contract for IBM to deliver analysis to compensation payouts via its IBM Watson Explorer AI, which resulted in the loss of 34 jobs. The company said it would speed up compensation payout analysis via analyzing claims and medical records, and increase productivity by 30%. The company also said it would save ¥140m in running costs.[110]

Several startups in the healthcare space have been effectively using seven business model archetypes to take solutions based on IBM Watson to the marketplace. These archetypes depends on the value generate for the target user (e.g. patient focus vs. healthcare provider and payer focus) and value capturing mechanisms (e.g. providing information or connecting stakeholders).[111]

By 2022, IBM Watson Health was generating about a billion dollars in annual gross revenue,[112] but was facing a lack of profitability and increased competition. One expert assessed to CNN that "IBM was clearly not gaining much traction in the healthcare market". A 2021 post from the Association for Computing Machinery (ACM) titled "What Happened To Watson Health?" described the portfolio management challenges of IBM Watson Health given the number of acquisitions involved in the Watson Health division creation in 2015, as well as technical limitations that existed at the time regarding where the Watson AI framework could be deployed.[113] In February 2021, the Wall Street Journal reported that Watson Health was exploring a sale.[114] On January 21, 2022, IBM announced the sell-off of its Watson Health unit to Francisco Partners.[115]

On January 9, 2014, IBM announced it was creating a business unit around Watson, led by senior vice president Michael Rhodin.[116] IBM Watson Group will have headquarters in New York's Silicon Alley and will employ 2,000 people. IBM has invested $1 billion to get the division going. Watson Group will develop three new cloud-delivered services: Watson Discovery Advisor, Watson Engagement Advisor, and Watson Explorer. Watson Discovery Advisor will focus on research and development projects in pharmaceutical industry, publishing, and biotechnology, Watson Engagement Advisor will focus on self-service applications using insights on the basis of natural language questions posed by business users, and Watson Explorer will focus on helping enterprise users uncover and share data-driven insights based on federated search more easily.[116] The company is also launching a $100 million venture fund to spur application development for "cognitive" applications. According to IBM, the cloud-delivered enterprise-ready Watson has seen its speed increase 24 times over—a 2,300 percent improvement in performance and its physical size shrank by 90 percent—from the size of a master bedroom to three stacked pizza boxes.[116] IBM CEO Virginia Rometty said she wants Watson to generate $10 billion in annual revenue within ten years.[117] In 2017, IBM and MIT established a new joint research venture in artificial intelligence. IBM invested $240 million to create the MIT–IBM Watson AI Lab in partnership with MIT, which brings together researchers in academia and industry to advance AI research, with projects ranging from computer vision and NLP to devising new ways to ensure that AI systems are fair, reliable and secure.[118] In March 2018, IBM's CEO Ginni Rometty proposed "Watson's Law," the "use of and application of business, smart cities, consumer applications and life in general."[119]

Watson helped a team of chefs create five new poutines for the 2015 La Poutine Week food festival in Toronto and Montreal. It analyzed the demographics and popular cuisines of the cities and drew from a database of tens of thousands of recipes to create fusion pairings for each city.[120] IBM and Bon Appétit magazine co-created an AI cooking app known as Chef Watson.[121]

Watson is being used via IBM partner program as a chatbot to provide the conversation for children's toys.[122]

In 2015, the engineering firm ENGEO created an online service via the IBM partner program named GoFetchCode. GoFetchCode applies Watson's natural language processing and question-answering capabilities to the International Code Council's model building codes.[123]

IBM Watson is being used for several projects relating to education, and has entered partnerships with Pearson Education, Blackboard, Sesame Workshop and Apple.[124][125]

In its partnership with Pearson, Watson is being made available inside electronic text books to provide natural language, one-on-one tutoring to students on the reading material.[126]

As an individual using the free Watson APIs available to the public, Ashok Goel, a professor at Georgia Tech, used Watson to create a virtual teaching assistant to assist students in his class.[127] Initially, Goel did not reveal the nature of "Jill", which was created with the help of a few students and IBM. Jill answered questions where it had a 97% certainty of an accurate answer, with the remainder being answered by human assistants.[128]

The research group of Sabri Pllana developed an assistant for learning parallel programming using the IBM Watson.[129] A survey with a number of novice parallel programmers at the Linnaeus University indicated that such assistants will be welcomed by students that learn parallel programming.

In August 2016, IBM announced it would be using Watson for weather forecasting.[130] Specifically, the company announced they would use Watson to analyze data from over 200,000 Weather Underground personal weather stations, as well as data from other sources, as a part of Project Deep Thunder.[131]

IBM Watson together with Marchesa designed a dress that changed the colour of the fabric depending on the mood of the audience. The dress lit up in different colours based on the sentiment of Tweets about the dress. Tweets were passed through a Watson tone analyzer and then sent back to a small computer inside the waist of the dress.[132]

On February 5–6, 2017, tax preparation company H&R Block began nationwide use of a Watson-based program.[133]

In September 2017, IBM announced that with its acquisition of The Weather Company's advertising sales division, and a partnership with advertising neural network Cognitiv, Watson will provide AI-powered advertising solutions.[134][135][136]

• Jeopardy! Show #6086 – Game 1, Part 1
• Jeopardy! Show #6087 – Game 1, Part 2
• Jeopardy! Show #6088 – Game 2

• PBS NOVA documentary on the making of Watson
• Building Watson – A Brief Overview of the DeepQA Project on YouTube (21:42), IBMLabs
• How Watson Answers a Question on YouTube
• David Ferrucci, Dan Cerutti and Ken Jennings on IBM's Watson at Singularity Summit 2011 on YouTube
• A Computer Called Watson on YouTube – November 15, 2011, David Ferrucci at Computer History Museum, alternate
• IBM Watson and the Future of Healthcare on YouTube – 2012
• IBM Watson-Introduction and Future Applications on YouTube – IBM at EDGE 2012
• IBM Watson for Healthcare on YouTube – Martin Kohn, 2013
• IBM Watson playlist, IBMLabs Watson playlist