The Cognitive Reflection Test Under Pressure

Abstract

Background
The original Cognitive Reflection Test (CRT) is a widely-known three-item psychological measure which claims to test individuals on intuitive (Type 1) vs. analytic (Type 2) cognitive processing. The CRT became popular because the questions lead the decision-maker to an incorrect, yet intuitive, response that must be avoided in order to arrive at the correct answer. Previous research has shown that emergency physicians and interns are interrupted, on average, 7.1 times per hour (Westbrook et al., 2010). Further, working in a high-pressure and dynamic environment can greatly affect decision-making (Flowerdew, Brown, Russ, Vincent, & Woloshynowych, 2012). We aim to use the CRT to study the role of interruptions and time pressure on the balance of intuitive and analytic kinds of thinking.

Methods
We recruited participants through Amazon’s Mechanical Turk platform and they responded to a series of CRT-like questions. Every question was presented across three separate pages, and we allowed participants to go back and forth between the pages in order to re-read parts of the question. All of the questions had a multiple choice answer set, and participants selected the answer they believed to be correct. After every question, participants indicated their level of confidence on a slider ranging from “uncertain/guessing” to “absolutely certain”. We are interested in performance under pressure, and so we manipulated the type of pressure participants received. There were three conditions - 1) a baseline condition with no additional pressure, 2) a countdown timer condition in which participants only had a set amount of time to answer each question, and 3) an interruption condition where participants were sometimes interrupted in the middle of attempting to solve the problems.

Predicted Results
Data collection is ongoing, and so we present only predicted results here. We will first analyse performance based on the proportion of correct and intuitive answers. The second and third dependent variables will be summary statistics for the number of times that participants navigate back and forth to re-read the questions, and the time they spend on each page. It is predicted that participants in the baseline condition will make more correct responses, and will spend less time and need fewer re-reads of the pages, than participants in the time-pressure and interruption conditions.

Conclusions
These predicted results should give us insight into the way that the added stress of getting interrupted or being under time pressure influences the processing of information and the type and incidence of errors in decision making.