ASP/WOP/HDS Research projects 2016-2017
Project title: Heuristics and biases: Meta-analytic reviews
Each of the students will conduct a meta-analytic review of the literature on a specific bias or heuristic in judgement and decision making (e.g., action-effect, negativity effect etc.).
In the meta-analytic review project, we will examine all the literature on a classic bias or heuristic and provide a quantitative meta-analytic review and summary of the findings. We will review the literature and call other researchers to share their unpublished findings, and then proceed to run a statistical method to compute the combined effect-size of all findings. Since different papers will show different effect-sizes, we will also look for moderators to explain these differences (sample size, publication bias, etc. and other theoretically meaningful moderators). We will then write a general review and the meta-analytic findings as a high-quality top-tier journal submission.
More information is available on: http://wiki.mgto.org/doku.php/wop_research_projects_2016-2017
People do not always act rationally. When making decisions, people employ ‘rules of thumb’ (heuristics) that sometimes lead to biases that may seem as irrational. As an example, the “action-effect” by Nobel prize winning Kahneman and Tversky (1982) describes a phenomenon in which people tend to regret negative outcomes more when they are a result of action (acting) compared to inaction (not acting) (more details can be read in my recent publication: http://bit.ly/2c7hKEm). This is an exciting and prolific area of research with many interesting findings highlighting the bounded rationality of the human mind with fairly strong and consistent effects.
In the last few years psychology has been facing new challenges with failed replications for classic findings (sometimes referred to as the “replication crisis”) raising the need for both more replication work and meta-analytic summaries of the existing literature and data. In a meta-analytic review, we examine all the literature on a specific topic, in this case of a specific bias or effect, and call other researchers to share their unpublished findings to then run a statistical method to compute the combined effect-size of all findings. Since different papers show different effect-sizes, we also look for moderators to explain these differences (sample size, publication bias, etc. and other theoretically meaningful moderators).
The project will be mainly a quantitative review of the literature. Whatever we find, even if we find no effects, is interesting and important to communicate, and meta-analyses typically enjoy higher rates of acceptance for publication and higher impact. This means that, as students, conducting a meta-analysis offers the benefits of higher data availability, lower risk of failure, and higher chances for publication and impact, with the added advantages of becoming an expert in the topic of interest, and connecting with other scholars working on the topic. Importantly, my expectations are that the end goal of this project will be a high-quality meta-analytic summary of a specific bias written up as top-tier journal submission.
For examples of published meta-analyses and what we will be aiming for, please browse the Psychological Bulletin journal articles at (years 2015 or earlier will have available full-text when on campus or using a VPN) : http://browzine.com/libraries/1006/journals/21076/issues/7974273
If you want a specific example, see “The Actor–Observer Asymmetry in Attribution: A(Surprising) Meta-Analysis” by (Malle, 2006, PsycBull)
Regarding data collection, FPN research guidelines indicate special instructions for metas:
Students preferably test (real) participants; this is also to the student’s advantage, as he/ she will be able to get in touch with the future work field. In case this is not possible, the data must be at least originating from participants (such as in research with databases, questionnaire research, existing image and sound material). In case this is not possible (e.g., in case of a meta–analysis), more emphasis must be put on one or more other parts of the empirical cycle (e.g., more advanced statistics, and/ or more background/ introduction to the topic, leading to the research question).
This means that we will aim to supplement the meta with a pretest participant data collection using Amazon Mechanical Turk online labour market. You can read more about it in my blog post. This will be a well-powered pre-registered replication of the chosen effect.
Working with me
IMPORTANT - When you contact me please :
- Indicate your familiarity with meta-analysis procedure (none/read a meta-analysis paper/read about the meta-analysis procedure/learned how to do metas/conducted a meta).
- Rate your general familiarity with heuristics and biases (0 - none; 10 - expert).
- Browse the list of biases and select atleast two that you're interested in : https://en.wikipedia.org/wiki/List_of_cognitive_biases , explain in one sentence why those are of interest to you.
- Briefly go over my publications list: http://mgto.org/publications/ and check that you find my research and methods a good fit for you.
- Indicate that you understand and accept to the goal of turning the thesis to a high-quality top-tier journal article submission.
Action/inaction and harm/responsibility
- Omission bias: People prefer harm through omission (inaction) than harm through commission (action).
- The action principle: People judge harm caused by action as morally worse than equivalent harm caused by omission (inaction).
- Omission bias, individual differences, and normality (this article shows that this topic and the findings so far are controversial. So this is great for a replication+meta).
- Another good review - COUNTERFACTUAL THINKING AND SOCIAL PERCEPTION: THINKING ABOUT WHAT MIGHT HAVE BEEN (focus especially on IIb)
- The effects of action, normality, and decision carefulnesson anticipated regret: Evidence for a broad mediatingrole of decision justifiability (2010) - they generally oppose Baron & Ritov
Status quo bias
Description: People generally prefer the status quo over change.
Mere ownership effect
Description: Simply thinking you own something makes you value it more.
Also can look up: The endowment effect (but stay away from articles in Economics or Organizational Behavior)
- Good recent review http://www.sciencedirect.com/science/article/pii/S1364661315000789 (see section about Psychological ownership)
Decoy effect / Asymmetric dominance / attraction effect
Description: Introducing an irrelevant bad option to a choice between two options shifts preferences from one option to the other.
- Example of a meta-analysis from 1995: http://bit.ly/2iZcreM and we’ll do the decoy effect meta on an updated literature and from a different angle (factors).
- I strongly suggest reading Ariely's experiments in predictability irrational. Quite entertaining. (see Ariely's book in the Dropbox) (see Dan's TED video, start from 11:00 for the Decoy Effect)
- The classic experiment - Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis (Huber, Payne, & Puto, 1982) - This one is a bit hard to read/follow
- Seeking Subjective Dominance in Multidimensional Space: An Explanation of the Asymmetric Dominance Effect (Ariely & Wallsten, 1995) - much clearer
- More evidence challenging the robustness and usefulness of the attraction effect - good paper about the problems with findings - see their experiments in the supplementary materials
- Regret salience and accountability in the decoy effect (2013) - links decoy effect with the other students' biases
- It also works in frogs - Irrationality in mate choice revealedby túngara frogs (Science, 2015) (but we won't try and replicate that one)
How social norms and past behavior affects judgments or feelings of regret. I suggest we focus on normality of action-inaction and regret
- See my preprint paper under review - https://osf.io/6jvh8/?view_only=4d181e034b974b59890650ed0f45d58b
- Another good review - COUNTERFACTUAL THINKING AND SOCIAL PERCEPTION: THINKING ABOUT WHAT MIGHT HAVE BEEN (focus especially on IIa2, IIIa, Va and Vb)
- The effects of action, normality, and decision carefulnesson anticipated regret: Evidence for a broad mediatingrole of decision justifiability (2010) - good experimental design, but no effects for normality
- Counterfactual Processing of Normal and Exceptional Events (1989) - simple design, easy to follow
- The inaction effect in the psychology of regret (2002) - I replicated their study.
- The Undoing of Scenarios (JPSP, 1987) - this one is a bit complicated, only if you feel like it
Inaction inertia (not chosen)
People who pass once (inaction) tend to pass again in the future (inertia).
Counterfactuals/mutabilty with blame attributions (not chosen)
people attribute more blame when it’s easier to think of alternatives (the outcome is mutable).
- http://bit.ly/2jgi5bT (see section “Moral Judgments”)