CSCE 330 Programming Language Structures

CSCE 330 Programming Language Structures

CSCE 390 Professional Issues in Computer Science and Engineering How Does Watson Work? Spring 2011 Marco Valtorta [email protected] UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering

What is Watson? A computer system that can compete in realtime at the human champion level on the American TV quiz show Jeopardy. Adapted from: David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John Prager, Nico Schlafer, and Chris Welty. Building Watson: An Overview of the DeepQA Project. AI Magazine, 31, 3 (Fall 2010), 59-79. This is the reference for much of this presentation. UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering

How Does Watson Fit in? Systems that think like humans The exciting new effort to make computers think machines with minds, in the full and literal sense. (Haugeland, 1985) [The automation of] activities that we associate with human thinking, activities such as Richard Bellman decision-making, (1920-84) problem solving, learning (Bellman, 1978) Systems that act like humans The art of creating machines that perform functions that require intelligence when performed by people (Kurzweil, 1990) The study of how to make computers Alan Turing (1912do things at which, at the moment, 1954) people are better (Rich and Knight,

UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY 1991) Systems that think rationally The study of mental faculties through the use of computational models. (Charniak and McDermott, 1985) The study of the computations that make it possible to perceive, reason, and act. (Winston, 1972) Aristotle (384BC 322BC) Systems that act rationally The branch of computer science that is concerned with the automation of intelligent behavior. (Luger and Stubblefield, Thomas Bayes 1993) (1702-1761) Computational intelligence

is the study of the design of intelligent agents. (Poole et al., 1998) Department ofComputer Computer Scienceand and Department of Science AI is concerned with intelligent Engineering Watson is Designed to Act Humanly Watson is supposed to act like a human on the general question answering task Watson needs to act as well as think It needs to push the answer button at the right time This is a Jeopardy requirement. The IBM

design team wanted to avoid having to use a physical button The Jeopardy game is a kind of limited Turing test UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Acting Humanly: the Turing Test Operational test for intelligent behavior: the Imitation Game In 1950, Turing predicted that by 2000, a machine might have a

30% chance of fooling a lay person for 5 minutes Anticipated all major arguments against AI in following 50 years Suggested major components of AI: knowledge, reasoning, language understanding, learning Problem: Turing test is not reproducible, constructive, or amenable to mathematical analysis UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Watson is Designed to Act Rationally Watson needs to act rationally by choosing a strategy that maximizes its

expected payoff Some human players are known to choose strategies that do not maximize their expected payoff. UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Acting Rationally Rational behavior: doing the right thing The right thing: that which is expected to maximize goal achievement, given the available information Doesn't necessarily involve thinking (e.g.,

blinking reflex) but thinking should be in the service of rational action Aristotle (Nicomachean Ethics): Every art and every inquiry, and similarly every action and pursuit, is thought to aim at some good UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Game Playing Computer programs usually do not play games like

people A Min-Max tree of moves: (from wikipedia) UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Tuomas Sandholm. The State of Solving Large IncompleteInformation Games, and Application to Departmentof ofComputer ComputerScience Scienceand and Poker. Department

Engineering Computer Play Games Very Well After 18-and-a-half years and sifting through 500 billion billion (a five followed by 20 zeroes) checkers positions, Dr. Jonathan Schaeffer and colleagues at the University of Alberta have built a checkers-playing computer program that cannot be beaten. Completed in late April this year, the program, Chinook, may be played to a draw but will never be defeated. (http://www.sciencedaily.com/releases/2007/07/07071914351 7.htm, accessed 2011-02-15) Jonathan Schaeffer of the University of Alberta Checkers is a forced draw (like tic-tactoe) Connect-4 is a forced win for the first player UNIVERSITYOF

OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Chess and Go Chess is not a solved game, but the best computer program are at least as good as the best human players Human players are better than the best computer programs at the

game of Go UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Jeopardy Requires a Broad Knowledge Base Factual knowledge History, science, politics Commonsense knowledge E.g., nave physics and gender Vagueness,

obfuscation, uncertainty E.g., KISSing music UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering The Questions: Solution Methods Factoid questions Decomposition Puzzles

UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering The Domain Example: castling is a maneuver in chess UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer

ComputerScience Scienceand and Department Engineering Precision vs. Percentage Attempted Upper line: perfect confidence estimation UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Champion Human

Performance Dark dots correspond to Ken Jennings games UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Baseline Performance (IBM) PIQUANT system UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY

Departmentof ofComputer ComputerScience Scienceand and Department Engineering The DeepQA Approach Adapting PIQUANT did not work out The system we have built and are continuing to develop, called DeepQA, is a massively parallel probabilistic evidence-based architecture. For the Jeopardy Challenge, we use more than 100 different techniques for analyzing natural language, identifying sources, finding and generating hypotheses, finding and scoring evidence, and merging and ranking hypotheses. What is far more important than any particular technique we use is how we combine them in DeepQA such that overlapping approaches can bring their strengths to bear and contribute to improvements in accuracy, confidence, or speed. UNIVERSITYOF

OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Overarching Principles Massive parallelism Many experts Facilitate the integration, application, and contextual evaluation of a wide range of loosely coupled probabilistic question and content analytics. Pervasive confidence estimation Integrate shallow and deep knowledge UNIVERSITYOF OFSOUTH

SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering High-Level Architecture UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and

Department Engineering Content Acquisition UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Question Analysis The DeepQA approach encourages a mixture of experts at this stage, and in the Watson system we produce shallow parses, deep parses (McCord 1990), logical forms, semantic role labels,

coreference, relations, named entities, and so on, as well as specific kinds of analysis for question answering. UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Hypothesis Generation The operative goal for primary search eventually stabilized at about 85 percent binary recall for the top 250 candidates; that is, the system generates the correct answer as a candidate answer for 85 percent of the questions somewhere within the top 250 ranked candidates.

If the correct answer(s) are not generated at this stage as a candidate, the system has no hope of answering the question. This step therefore significantly favors recall over precision, with the expectation that the rest of the processing pipeline will tease out the correct answer, even if the set of candidates is quite large. UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Hypothesis and Evidence Scoring Nixon pardon example

UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering Search Engine Failure UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience

Scienceand and Department Engineering Progress UNIVERSITYOF OFSOUTH SOUTHCAROLINA CAROLINA UNIVERSITY Departmentof ofComputer ComputerScience Scienceand and Department Engineering

Recently Viewed Presentations

  • Decision Trees on MapReduce CS246: Mining Massive Datasets

    Decision Trees on MapReduce CS246: Mining Massive Datasets

    Decision Tree Learning. Give one attribute (e.g., lifespan), try to predict the value of new people's lifespans by means of some of the other available attribute. Input attributes: d. ... Our new goal: Find tree .
  • Relations and Functions - DANGMATH.com

    Relations and Functions - DANGMATH.com

    2.1: Relations and Functions. Definitions. Relation is a pairing of . 2 items (like (x, y)) Domain is the . X -values in a relation. Also known as . INDEPENDENT. Variable. Range is the . Y -values in a relation....
  • RETS Developer Seminar - Corvelle

    RETS Developer Seminar - Corvelle

    RETS the standard RETS transactions Identify issues, concerns and recommendations Sample session workflow RETS Software Development Topic Objectives Improve understanding of the RETS architecture Build familiarity with the RETS document Enable more productive RETS software development RETS Principles - 1...
  • Chapter 21

    Chapter 21

    Inflation is . not. the prices of a certain good going up due to a shift in demand or supply for that good. Inflation is the increase in the price level (the price of all goods). Inflation by itself will...
  • State of Minnesota State of Minnesota  New Employee

    State of Minnesota State of Minnesota New Employee

    A 2-year pre-existing condition exclusion clause applies. Disability Insurance = Paycheck Protection. Next, let's review your Long Term Disability Benefit. This coverage is also a form of income protection.
  • Chapter 22

    Chapter 22

    Times Beach - In Missouri, was an area contaminated with dioxin from buried waste oil. Love Canal Superfund Site. A method that seeks to develop as many options as possible, to reduce environmental harm and cost. ... Chapter 22 Last...
  • PowerPoint Slides for Professors Spring 2010 Version This

    PowerPoint Slides for Professors Spring 2010 Version This

    Munich Re, Swiss Re, Berkshire Hathaway…, Mega-mergers and acquisitions internationally. Reinsurance departments (of insurance companies) Directly. Through underwriting agents. Suppliers of Reinsurance. Pools.
  • Special Senses - Swallows Charter Academy

    Special Senses - Swallows Charter Academy

    External Anatomy. Eyelids. Rehydrate and wash out dust and bacteria. Eyebrows and Eyelashes. keep dirt and sweat out of the eye. Internal Anatomy. Tears. Produced by lacrimal gland located above the eye. Wash dirt and bacteria from the eye.