Implicatures and Experiments

Implicatures and Experiments

Assessing the pragmatics of experiments with crowdsourcing: The case of scalar implicature Pranav Anand, Caroline Andrews, Matthew Wagers University of California, Santa Cruz Experiments & Pragmatic Processing Case Study: (Embedded) Implicatures Each of the critics reviewed some of the movies.

but not all ? Depending on the study: - no evidence of EIs evidence for EIs, with different response choices Worry: Are we adequately testing the influence of methodologies on our data?

Previous Limitation: Lack of Subjects and Money Crowd-sourcing addresses both problems Pragmatics of Experimental Situations Evaluation Apprehension subjects know they are being judged Teleological Curiosity - Subjects hypothesizing expected behavior, matching an ideal The experiment itself is part of the pragmatic context

See Rosenthal & Rosnow. (1975) The Volunteer Subject. Elements of Experimental Context Protocol Social Context / Task Specification Response Structure Response choices available to the subject e.g. True / False, Yes / No, 1-7 scale Prompt the Question directions for the Response Structure Immediate Linguistic/Visual Context

Our Goal: Explore variations of these elements in a systematic way Experimental Design Is this an accurate description? Some of the spices have red lids. Linguistic Contexts All Relevant, All Irrelevant, No Context

Protocol Experimental normal experiment instructions Annotation checking the work of unaffiliated annotators 4 Implicature Targets, 6 Some/All Controls, 20 Fillers Experiment 1: Social Context Focus on Protocol Annotation vs Experiment All Irrelevant No Story

All-Relevant Experiment Annotation Accuracy Prompt - Is this an accurate description? Response Categories - Yes, No, Dont Know Population: Undergraduates Experiment 1:

Social Context Finding: Social context even when linguistic context does not. Linguistic Context: No Effect Experiment 1: Social Context

Finding: Social context even when linguistic context does not. Lower SI rate for Annotation (p<0.05) Experiment 2 Prompt Type

Accuracy Prompt - Is this an accurate description? Response Categories - Yes, No, Dont Know Informativity Prompt - How Informative is this sentence? Response Categories - Not Informative Enough Informative Enough Too Much Information False Population: Mechanical Turk Workers Systematic Debriefing Survey

Experiment 2 Prompt Type Effect for Prompt Experiment 2 Prompt Type Effect for Prompt (p<0.001)

Effect for Context (p<0.001) Experiment 2 Prompt Type Effect for Prompt (p<0.001)

Effect for Context (p<0.001) Weak Interaction: Prompt x Context (p<0.06)

Experiment 2 Prompt Type No Effect for Protocol Experiment 2 Prompt Type Low SI rates overall But the debriefing survey

indicates that (roughly) 70% of participants were aware of some/all contrast Populations Turkers More sensitive to Linguistic Context Less sensitive to changes in changes in social context/ evaluation apprehension Undergraduates More sensitive to Protocol

Take Home Points Methodological variables should be explored alongside conventional linguistic variables Ideal: models of these processes (cf. Schutze 1996) Crowdsourcing allows for cheap/fast exploration of parameter spaces New Normal: Dont guess, test. Controls, norming, confounding all testable online

A potential check on exuberance Undergraduates may be WEIRD*, but crowdsourcing engenders its own weirdness High evaluation apprehension Uncontrolled backgrounds, skillsets, focus levels Unknown motivations Ignorance does not necessarily mean diversity This requires study if we rely on such participants more

* Heinrich et al. (2010) The Weirdest People in the World? BBS Acknowledgments Thanks Jaye Padgett and to the attendees of two Semantics Lab presentations and the XPRAG conference for their comments, to the HUGRA committee for their generous award and support, and thanks to Rosie Wilson-Briggs for stimuli construction.

Recently Viewed Presentations

  • Teori Keperawatan Terpilih: King, Neuman, Orem, Leininger

    Teori Keperawatan Terpilih: King, Neuman, Orem, Leininger

    IMOGENE M. KING. King's Conceptual System and Theory of Goal Attainment and Transactional Process
  • Session Timeline

    Session Timeline

    It also requires the teacher to explain and define the term "unlike fractions." The teacher is then supposed to demonstrate the process for using multiples to create equivalent fractions, and finally have children practice this skill. The following are student...
  • Getting Back to Work AFTER A TRADE-RELATED LAYOFF Hardinge ...

    Getting Back to Work AFTER A TRADE-RELATED LAYOFF Hardinge ...

    getting back to work after a trade-related layoff petition # 80,405schweizer aircraft corporation a subsidiary of sikorsky aircraft corporation Youare eligible to apply for Adjustment Assistance. Totally or partially separated from employment on or after August 30, 2010.
  • Machine Learning Techniques in Image Processing

    Machine Learning Techniques in Image Processing

    CSCI 8810 Course Project MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING By Kaan Tariman M.S. in Computer Science Outline Introduction to Machine Learning The example application Machine Learning Methods Decision Trees Artificial Neural Networks Instant Based Learning What is Machine Learning...
  • The Lemma - fac.ksu.edu.sa

    The Lemma - fac.ksu.edu.sa

    or guns, which create tiny granules like hair or grits. These machines are becoming cheaper with increased technology. Ball Bearings: Little firm balls of snow that form around or under skis. Blowing Snow: Grounded snow that has been moved around...
  • Simulation of the turbulent flow in a 3D channel and over a ...

    Simulation of the turbulent flow in a 3D channel and over a ...

    Fig. 2 Cross-section of a rough micro tube. (Picture courtesy: Microgroup, Inc.) Numerical simulation of Newtonian fluid through a cylindrical tube with a rough surface; Study of the relationship between tube length and pressure loss; Generic modeling of flow in...
  • Coastal State Indicators for Coastal Erosion

    Coastal State Indicators for Coastal Erosion

    Coastal State Indicators for Coastal Erosion James Sutherland HR Wallingford Coastal State Indicators for Coastal Erosion Indicators used in many fields to assess progress in implementing policy Coastal State Indicators are a "reduced set of parameters that can simply, adequately...
  • Ionic Equations - mskingschemistryclass.weebly.com

    Ionic Equations - mskingschemistryclass.weebly.com

    This is LEO He says GER Redox Reactions A reaction in which electrons are transferred from one atom to another is called an _____ . For simplicity, chemists often refer to oxidation-reduction reactions as _____ .