Attachment 'hlp_workers.html'
DownloadHLP Lab MTurk Workers
Do we know stuff about Turkers? What do we know? Let’s find out!
Andrew Watts
2018-05-14
How many HITs per year?
Basic info about workers
semi_distinct_workers <- anonyhits %>%
select(UUID, HITcount, Sex, Race, Ethnicity) %>%
distinct()
nrow(semi_distinct_workers)## [1] 14571distinct_workers <- anonyhits %>%
select(UUID, HITcount, Sex, Race, Ethnicity, Experiment, Experimenter) %>%
distinct(UUID, HITcount, .keep_all = TRUE) %>%
mutate(Sex = Sex %>% replace_na("Unknown"),
Ethnicity = Ethnicity %>% replace_na("Unknown"))
nrow(distinct_workers)## [1] 13603Sex breakdown
| Sex | count |
|---|---|
| Female | 3597 |
| Male | 3852 |
| Unknown | 6154 |
Ethnicity breakdown
| Ethnicity | count |
|---|---|
| Hisp | 536 |
| NonHisp | 6784 |
| Unknown | 6283 |
Race breakdown
| Race | count |
|---|---|
| American Indian / Alaska Native | 66 |
| Asian | 509 |
| Black or African American | 519 |
| More Than One Race | 341 |
| Native Hawiiian or Other Pacific Islander | 10 |
| Other | 82 |
| Unknown or Not Reported | 6217 |
| White | 5859 |
How many HITs do Turkers do?
Not many on average
Your median worker does one HIT for us ever, but there are some crazies who have done a ton.
summary(distinct_workers$HITcount)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 1.000 1.000 1.000 3.668 2.000 1506.000How does when they first did a HIT effect how many they do?
Turns out that workers from the first two years (when we often split HITs into chunks of one to four trials) drive most of the deviation from doing < 10.
How many HITs do Turkers 2011 or later (after split HIT experiment) do?
Let’s zoom the boxplot a bit by discarding those outlier-y first two years.
Frequency of HIT acceptance by hour and day
For workers who have done more than 10 HITs
How long did HITs take
Time in minutes, outliers (by Z-Score > 3) removed
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0500 0.6667 2.8500 9.3701 14.0167 116.0833Let’s look at just grad students and postdocs currently in the lab
## Warning: Removed 6 rows containing non-finite values (stat_boxplot).How many workers did each experiment?
| Experimenter | Experiment | count |
|---|---|---|
| Alex Fine | Adapt SPR | 90 |
| Alex Fine | Adapt2 | 1000 |
| Alex Fine | adaptallRC_NoOverlap | 80 |
| Alex Fine | adaptallrcsc | 369 |
| Alex Fine | scverbrep | 121 |
| Alex Fine | VWPNorm | 1034 |
| BCS206 Group | Baese-Berk Replication | 90 |
| Ben Anible | CRM Recall | 768 |
| Bozena Pajak | DLPrior | 138 |
| Camber Hansen Carr | EMOPRM.1 | 72 |
| Dan Pontillo | Adaptation and Attention | 551 |
| Dave Kleinschmidt | bp_adapt | 124 |
| Dave Kleinschmidt | bp-var | 66 |
| Dave Kleinschmidt | MT Adapt | 833 |
| Dave Kleinschmidt | supunsup | 372 |
| Esteban Buz | BaeseBerkGoldrickRep1 | 349 |
| Esteban Buz | Heller.Rep1 | 40 |
| Esteban Buz | Rate Intelligibility | 167 |
| Esteban Buz | sz_context | 84 |
| Esteban Buz | sz_mp | 26 |
| Florian Jaeger | CC SPR | 72 |
| Florian Jaeger | Florian Completion | 768 |
| Florian Jaeger | Phononset | 276 |
| Florian Jaeger | Phonrime | 180 |
| Florian Jaeger | RC SPR | 74 |
| Florian Jaeger | RCPred | 9088 |
| Florian Jaeger | SemRegCompl.1 | 57 |
| Florian Jaeger | SemRegCompl.2 | 70 |
| Ilker Yildirim | Semantic adaptation | 884 |
| Jackie Gutman | Jackie Alien Language | 85 |
| Jeremy Ferris | EMOPRM.1 | 198 |
| Judith Degen | JDSome1 | 1760 |
| Judith Degen | JDSome3 | 1030 |
| Klinton Bicknell | loco-vw-norming | 98 |
| Kodi Weatherholtz | SocAlign1 | 316 |
| Kodi Weatherholtz | SocAlign1.NoContext | 26 |
| Kodi Weatherholtz | socalign2 | 45 |
| Kodi Weatherholtz | socalign2_pilot | 36 |
| Kodi Weatherholtz | socalign2pd | 81 |
| Linda Liu | Bradlow and Bent | 1113 |
| Linda Liu | Causal Reasoning | 1810 |
| Linda Liu | Clarke and Garrett | 278 |
| Linda Liu | Kraljic Replication | 489 |
| Linda Liu | Mimo | 144 |
| Linda Liu | Norming | 20 |
| Linda Liu | Subtitle | 1413 |
| Linda Liu | Tongue Twister | 148 |
| Linda Liu | TT | 1672 |
| Maryam Seifeldin | Generalization | 856 |
| Masha Fedzechkina | IT Learn | 84 |
| Masha Fedzechkina | Masha KL | 460 |
| Masha Fedzechkina | MultiDay | 458 |
| Neal Snider | Judge the naturalness of sentences | 12065 |
| Neal Snider | PB Cloze | 3500 |
| Nicole Craycraft | ncspradapt | 765 |
| Scott Fraundorf | BeDropPGH | 60 |
| Scott Fraundorf | BeDropUS | 160 |
| Scott Fraundorf | BeModalUS | 270 |
| Scott Fraundorf | NeedsWashed | 550 |
| Shaorong Yan | Chinese Radical Norming | 56 |
| Tal Linzen | LexicalizedAdaptationSPR | 156 |
| Tal Linzen | subcatent | 188 |
| Thomas Farmer | 3wayPT | 147 |
| Thomas Farmer | TFarmer Completion | 40 |
| Wednesday Bushong | Bicknell Replication | 49 |
| Wednesday Bushong | BrownReplicationDefiniteness | 192 |
| Wednesday Bushong | Clash Trial | 120 |
| Wednesday Bushong | Maintenance Norming | 32 |
| Wednesday Bushong | Response Extension | 48 |
| Xin Xie | Accent Priming | 123 |
| Xin Xie | Baese-Berk Replication | 238 |
| Xin Xie | Goodness Rating | 128 |
| Xin Xie | Identification | 48 |
| Zach Burchill | Pupillometry | 214 |
How long do Turkers hang around?
When are we likely never to see them again?
Attached Files
To refer to attachments on a page, use attachment:filename, as shown below in the list of files. Do NOT use the URL of the [get] link, since this is subject to change and can break easily.You are not allowed to attach a file to this page.