how_to_guides

How to guides

Before submission / reviews

Learning R and R scripts

General R guides

Courses:

Data cleaning

Styling

Research design

Data prep

Data check

Statistical tests

Citations in R

Graphics / Plotting

Visualization galleries:

Reporting

Reproducibility

SEM in R

Pre-registering SEM analyses

R Studio

Text mining

Advanced R uses

Developing packages:

Cheat sheets

General stats courses

T-test and Equivalence testing

Basic correlations

Regressions

Mediation

Interactions

-@fredericksolt's interplot R, interflex R (through Twitter)

ANOVA two way interaction (with constrasts:

Basic code: 

UNIANOVA DV BY IV1 IV2
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /POSTHOC=IV1(TUKEY) 
  /PLOT=PROFILE(IV1*IV2)
  /EMMEANS=TABLES(IV2) COMPARE ADJ(LSD)
  /EMMEANS=TABLES(IV1) COMPARE ADJ(LSD)
  /EMMEANS=TABLES(IV1*IV2)  COMPARE(IV2)
  /CONTRAST(IV1)=Simple
  /CONTRAST(IV2)=Simple
  /PRINT=OPOWER ETASQ HOMOGENEITY DESCRIPTIVE
  /CRITERIA=ALPHA(.05)
  /DESIGN=IV1 IV2 IV1*IV2.

(might want to change the “EMMEANS=TABLES(IV1*IV2) COMPARE(IV2)” to “EMMEANS=TABLES(IV1*IV2) COMPARE(IV1)” to see if more convinient)

U-Curve / quadratic regressions

Excel Magic

General SPSS magic

Generally highly recommended - UITS Tutorials and Working Papers.

Effect size

General

Tools:

Interpreting effect sizes, Andy Field summarizes in his methods book:

Cohen (1992, 1988) has made some widely accepted suggestions about what constitutes a large or small effect: •r = 0.10 (small effect): in this case, the effect explains 1% of the total variance. •r = 0.30 (medium effect): the effect accounts for 9% of the total variance. •r = 0.50 (large effect): the effect accounts for 25% of the variance.

Table: effect-size.png

Readings about effect size estimates in psychology:

Power analyses

Moderation

Mediation

Multi level power analysis calculations

Multi-level / HLM

CFA

Meta analysis

Forest plots:

Combining two metas (SEM):

Best way to assess publication bias:

Pooling means and SD/variance:

New directions:

Power:

Promising packages:

Reporting

Writing

Pre-registration

Versioning/Github

Data Transformations

Multi choice data bank

Detecting cheating

Common method bias using AMOS

Bayesian

If you still want to do NHST of the null hypothesis:

Absence of an effect

Analyzing change

In experiments:

APA style

Helping others with their MTurk

In the last few lab meetings some of you mentioned that you are interested in collecting data with Amazon Mechanical Turk but that you do not have an account and asked if I might be able to help. I have an account, and have had some experience with this, so I’d be happy to help you run your studies and to help you adjust them to MTurk to increase the chances of getting reliable high-quality data. The email below is to give you an idea of how we can work on that.

First, a quick background. Although MTurk does have some limitations, many of the issues that came up in the various lab meetings can definitely be addressed with MTurk data collection – a replication attempt, scale validation, a pre-test to determine power, quick access to working people and/or underprivileged sectors, etc. I personally believe that MTurk is also good enough for running an independent study complementary to other collected data – if done correctly. In my experience, MTurk data collection is comparable if not better than student participant pool data. I also use a tool called TurkPrime that rides on top on MTurk and intended for academics, that completely automates academic data collection and increases data reliability (preventing duplicates and lots of potential problems, etc.).

My experience is summarized in the following two posts:

If you’d like my help to run something with MTurk, what you’ll need to do is:

  • Have a working Qualtrics survey that you’ve tested. I can share a Qualtrics demo with you designed for MTurk that includes a consent form, funneling section, demographics, and debriefing.
  • Make sure you can get reimbursed and have the money available. Once MTurk runs it automatically pays from my account, so I would ask that you transfer the funds to me before I start running this, so that I don’t have to bear the financial costs of waiting for everyone’s reimbursements. If you can’t, we can talk about that.
  • Plan how many participants you’ll need (N), the intended pay (minimum for IRB in some places is 0.05US$ a minute, most do a minimum of 0.10US$ a minute), sample characteristics (location, qualifications, etc.).

Costs you’ll need to take into account : (N + 10+ participants for pretest run) * pay + 20% Amazon MTurk commission [Note: The default Amazon commission is actually 40%, but TurkPrime uses a feature called Micro-Batch that reduces that to 20%. In some study designs this feature cannot be used, and I’ll alert you if that is the case]

The procedure would usually be:

  • You pretest your study with one or two other persons to make sure it’s working well (preferably a number of times to test all conditions, if you have manipulations).
  • We run a pretest of 10 participants on MTurk to see it all goes well (possibly more if you have many conditions).
  • We run a full sample.

Depending on the kind of survey you’re running and your intended sample size, data collection could take anything from half an hour to a few days.

Please let me know if you have any other questions.

Also:

You’ll need to work out your sample size (+small pretest) and payment for each participant with the formula before, confirm your reimbursement, and then transfer that money to me in advance. In the meanwhile, make a copy of your survey on Qualtrics and share the copy with me, and I’ll make sure it’s all set.

Also, it’s not a must, ofcourse, but I would strongly encourage you :

  • Calculate needed sample size based on power analysis using G*Power and an expected effect size.
  • Take 15-30min to pre-register your survey layout and your main predictions on OSF or “As predicted”.

To me, personally, as a reviewer, these two things significantly increase my confidence in the findings (and, in a way, in the researcher).

Control variables

IRB (Maastricht)

Build a scholar website

Twitter research

Games in surveys

Negotiations:

Public goods:

Dealing with outliers or invalid repsonses

Country level analyses

Spatial dependence analyses

MPlus

Lab participant management

Mobile data collection

Create a mobile app

  • PACO (mobile platform for behavioral science) / info

Machine learning

Collaborations

Share presentations and data

Simulations

Social network analysis

Network analysis

Machine learning

Conducting replications

Various

See all kinds of great resources from the SIPS workshop:

Content mining

Open Science

Research workflow

Lab manuals

Some examples:

Remove line breaks in office (when copying from PDFs)

Do this macro: 

Sub RemoveParaOrLineBreaks()
Dim oRng As Range
Dim oFind As Range
    Set oRng = Selection.Range
    Set oFind = Selection.Range
    With oFind.Find
        .Replacement.Text = "\1"
        Do While .Execute(FindText:="[^13^l]{1,}([a-z])", _
                          MatchWildcards:=True, _
                          Replace:=wdReplaceAll)
        Loop
    End With
lbl_Exit:
    Set oRng = Nothing
    Set oFind = Nothing
    Exit Sub
End Sub

Add a keyboard shortcut from options→ribbbon→keyboard→macros→RemoveParaOrLineBreaks

  • how_to_guides.txt
  • Last modified: 2019/04/25 19:10
  • by filination