Size: 1606
Comment:
|
Size: 2622
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 8: | Line 8: |
== Creating Balanced Lists == Amazon gives you: * Hit``Id - identifier for a given HIT (aka "trial" to us) * Worker``Id - unique identifier of the user doing the HIT * Assignment``Id - unique identifier for the assignment. HitId + some sort of hash * any user created annotation - I use Trial``Id Amazon creates a HIT for each trial, and creates as many assignments as you tell it to of each HIT. We want to: * show each worker items from only one list * use each list the same number of times * use each item from each list the same number of times Problems: * Workers can start with any HIT (trial) in a given assignment * Workers can return HITs at any time, making them available to a new worker, but given the information Amazon gives you, there's no way for you to know when this happens, so you can automatically start the new worker where the old one left off Solution: ??? |
|
Line 11: | Line 33: |
Via Neal Snider from Robert Munro: | Via Neal Snider from Robert Munro (w/ minor changes by Andrew Watts for formatting and to make it valid HTML): |
Line 19: | Line 41: |
<p><input type="hidden" name="userDisplayLanguage" /> <input type="hidden" name="browserInfo" /> <input type="hidden" name="ipAddress" /> <input type="hidden" name="country" /> <input type="hidden" name="city" /> <input type="hidden" name="region" /></p> <script language="Javascript" src="http://gd.geobytes.com/gd?after=-1&variables=GeobytesCountry,GeobytesCity,GeobytesRegion,GeobytesIpAddress"> |
<p> <input type="hidden" name="userDisplayLanguage" /> <input type="hidden" name="browserInfo" /> <input type="hidden" name="ipAddress" /> <input type="hidden" name="country" /> <input type="hidden" name="city" /> <input type="hidden" name="region" /> </p> <script type="text/javascript" src="http://gd.geobytes.com/gd?after=-1variables=GeobytesCountry,GeobytesCity,GeobytesRegion,GeobytesIpAddress"> |
Line 27: | Line 53: |
<script language="Javascript"> | <script type="text/javascript"> |
Mechanical Turk
1. Creating Balanced Lists
Amazon gives you:
HitId - identifier for a given HIT (aka "trial" to us)
WorkerId - unique identifier of the user doing the HIT
AssignmentId - unique identifier for the assignment. HitId + some sort of hash
any user created annotation - I use TrialId
Amazon creates a HIT for each trial, and creates as many assignments as you tell it to of each HIT.
We want to:
- show each worker items from only one list
- use each list the same number of times
- use each item from each list the same number of times
Problems:
- Workers can start with any HIT (trial) in a given assignment
- Workers can return HITs at any time, making them available to a new worker, but given the information Amazon gives you, there's no way for you to know when this happens, so you can automatically start the new worker where the old one left off
Solution:
- ???
2. Helpful Code
2.1. Geographic Info
Via Neal Snider from Robert Munro (w/ minor changes by Andrew Watts for formatting and to make it valid HTML):
If you place it in the design-view of your template, and it will use the IP address and browser settings of each Turker to populate fields with some useful demographics like 'City', 'Region', 'Country', and 'User Display Language'.
<p> <input type="hidden" name="userDisplayLanguage" /> <input type="hidden" name="browserInfo" /> <input type="hidden" name="ipAddress" /> <input type="hidden" name="country" /> <input type="hidden" name="city" /> <input type="hidden" name="region" /> </p> <script type="text/javascript" src="http://gd.geobytes.com/gd?after=-1variables=GeobytesCountry,GeobytesCity,GeobytesRegion,GeobytesIpAddress"> </script> <script type="text/javascript"> <!-- function getUserInfo() { var userDisplayLanguage = navigator.language ? navigator.language : navigator.userDisplayLanguage; var browserInfo = navigator.userAgent; var ipAddress = sGeobytesIpAddress; var country = sGeobytesCountry; var city = sGeobytesCity; var region = sGeobytesRegion; document.mturk_form.userDisplayLanguage.value = userDisplayLanguage; document.mturk_form.browserInfo.value = browserInfo; document.mturk_form.ipAddress.value = ipAddress; document.mturk_form.country.value = country; document.mturk_form.city.value = city; document.mturk_form.region.value = region; } getUserInfo(); // --> </script>