How to guides

Before submission / reviews

Learning R and R scripts

General R guides

Statistical tests

Graphics / Plotting


SEM in R

R Studio

Text mining

General stats courses

T-test and Equivalence testing

Basic correlations




ANOVA two way interaction (with constrasts:

Basic code:


(might want to change the “EMMEANS=TABLES(IV1*IV2) COMPARE(IV2)” to “EMMEANS=TABLES(IV1*IV2) COMPARE(IV1)” to see if more convinient)

Excel Magic

General SPSS magic

Effect size


  • The Power Dialogues (basic conversational explanation of power with links - Brent Roberts UIUC)


Interpreting effect sizes, Andy Field summarizes in his methods book:

Cohen (1992, 1988) has made some widely accepted suggestions about what constitutes a large or small effect: •r = 0.10 (small effect): in this case, the effect explains 1% of the total variance. •r = 0.30 (medium effect): the effect accounts for 9% of the total variance. •r = 0.50 (large effect): the effect accounts for 25% of the variance.

Table: effect-size.png

Readings about effect size estimates in psychology:


Multi-level / HLM

Meta analysis





Data Transformations

Multi choice data bank

Detecting cheating

Common method bias using AMOS


Absence of an effect

Analyzing change

APA style

Helping others with their MTurk

In the last few lab meetings some of you mentioned that you are interested in collecting data with Amazon Mechanical Turk but that you do not have an account and asked if I might be able to help. I have an account, and have had some experience with this, so I’d be happy to help you run your studies and to help you adjust them to MTurk to increase the chances of getting reliable high-quality data. The email below is to give you an idea of how we can work on that.

First, a quick background. Although MTurk does have some limitations, many of the issues that came up in the various lab meetings can definitely be addressed with MTurk data collection – a replication attempt, scale validation, a pre-test to determine power, quick access to working people and/or underprivileged sectors, etc. I personally believe that MTurk is also good enough for running an independent study complementary to other collected data – if done correctly. In my experience, MTurk data collection is comparable if not better than student participant pool data. I also use a tool called TurkPrime that rides on top on MTurk and intended for academics, that completely automates academic data collection and increases data reliability (preventing duplicates and lots of potential problems, etc.).

My experience is summarized in the following two posts:

If you’d like my help to run something with MTurk, what you’ll need to do is:

  • Have a working Qualtrics survey that you’ve tested. I can share a Qualtrics demo with you designed for MTurk that includes a consent form, funneling section, demographics, and debriefing.
  • Make sure you can get reimbursed and have the money available. Once MTurk runs it automatically pays from my account, so I would ask that you transfer the funds to me before I start running this, so that I don’t have to bear the financial costs of waiting for everyone’s reimbursements. If you can’t, we can talk about that.
  • Plan how many participants you’ll need (N), the intended pay (minimum for IRB in some places is 0.05US$ a minute, most do a minimum of 0.10US$ a minute), sample characteristics (location, qualifications, etc.).

Costs you’ll need to take into account : (N + 10+ participants for pretest run) * pay + 20% Amazon MTurk commission [Note: The default Amazon commission is actually 40%, but TurkPrime uses a feature called Micro-Batch that reduces that to 20%. In some study designs this feature cannot be used, and I’ll alert you if that is the case]

The procedure would usually be:

  • You pretest your study with one or two other persons to make sure it’s working well (preferably a number of times to test all conditions, if you have manipulations).
  • We run a pretest of 10 participants on MTurk to see it all goes well (possibly more if you have many conditions).
  • We run a full sample.

Depending on the kind of survey you’re running and your intended sample size, data collection could take anything from half an hour to a few days.

Please let me know if you have any other questions.


You’ll need to work out your sample size (+small pretest) and payment for each participant with the formula before, confirm your reimbursement, and then transfer that money to me in advance. In the meanwhile, make a copy of your survey on Qualtrics and share the copy with me, and I’ll make sure it’s all set.

Also, it’s not a must, ofcourse, but I would strongly encourage you :

  • Calculate needed sample size based on power analysis using G*Power and an expected effect size.
  • Take 15-30min to pre-register your survey layout and your main predictions on OSF or “As predicted”.

To me, personally, as a reviewer, these two things significantly increase my confidence in the findings (and, in a way, in the researcher).

Control variables

IRB (Maastricht)

Build a scholar website

Twitter research

Games in surveys


Public goods:

Dealing with outliers

Country level analyses

Spatial dependence analyses


Lab participant management

Mobile data collection

Create a mobile app

  • PACO (mobile platform for behavioral science) / info

Machine learning


how_to_guides.txt · Last modified: 2017/02/20 06:28 by filination
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0