Differences between revisions 2 and 4 (spanning 2 versions)
Revision 2 as of 2009-09-01 20:27:01
Size: 1606
Editor: platypus
Comment:
Revision 4 as of 2009-10-05 18:34:24
Size: 2622
Editor: platypus
Comment:
Deletions are marked like this. Additions are marked like this.
Line 8: Line 8:
== Creating Balanced Lists ==

Amazon gives you:
 * Hit``Id - identifier for a given HIT (aka "trial" to us)
 * Worker``Id - unique identifier of the user doing the HIT
 * Assignment``Id - unique identifier for the assignment. HitId + some sort of hash
 * any user created annotation - I use Trial``Id

Amazon creates a HIT for each trial, and creates as many assignments as you tell it to of each HIT.

We want to:
 * show each worker items from only one list
 * use each list the same number of times
 * use each item from each list the same number of times

Problems:
 * Workers can start with any HIT (trial) in a given assignment
 * Workers can return HITs at any time, making them available to a new worker, but given the information Amazon gives you, there's no way for you to know when this happens, so you can automatically start the new worker where the old one left off

Solution:
 ???
Line 11: Line 33:
Via Neal Snider from Robert Munro: Via Neal Snider from Robert Munro (w/ minor changes by Andrew Watts for formatting and to make it valid HTML):
Line 19: Line 41:
<p><input type="hidden" name="userDisplayLanguage" /> <input
type="hidden" name="browserInfo" /> <input type="hidden"
name="ipAddress" /> <input type="hidden" name="country" /> <input
type="hidden" name="city" /> <input type="hidden" name="region" /></p>
<script language="J
avascript"
src="http://gd.geobytes.com/gd?after=-1&variables=GeobytesCountry,GeobytesCity,GeobytesRegion,GeobytesIpAddress">
<p>
  
<input type="hidden" name="userDisplayLanguage" />
  <input
type="hidden" name="browserInfo" />
  <input type="hidden"
name="ipAddress" />
  <input type="hidden" name="country" />
  <input type="hidden" name="city" />
  <input type="hidden" name="region" />
</p>

<script type="text/j
avascript" src="http://gd.geobytes.com/gd?after=-1variables=GeobytesCountry,GeobytesCity,GeobytesRegion,GeobytesIpAddress">
Line 27: Line 53:
<script language="Javascript"> <script type="text/javascript">

Mechanical Turk

1. Creating Balanced Lists

Amazon gives you:

  • HitId - identifier for a given HIT (aka "trial" to us)

  • WorkerId - unique identifier of the user doing the HIT

  • AssignmentId - unique identifier for the assignment. HitId + some sort of hash

  • any user created annotation - I use TrialId

Amazon creates a HIT for each trial, and creates as many assignments as you tell it to of each HIT.

We want to:

  • show each worker items from only one list
  • use each list the same number of times
  • use each item from each list the same number of times

Problems:

  • Workers can start with any HIT (trial) in a given assignment
  • Workers can return HITs at any time, making them available to a new worker, but given the information Amazon gives you, there's no way for you to know when this happens, so you can automatically start the new worker where the old one left off

Solution:

  • ???

2. Helpful Code

2.1. Geographic Info

Via Neal Snider from Robert Munro (w/ minor changes by Andrew Watts for formatting and to make it valid HTML):

If you place it in the design-view of your template, and it will use the IP address and browser settings of each Turker to populate fields with some useful demographics like 'City', 'Region', 'Country', and 'User Display Language'.

<p>
  <input type="hidden" name="userDisplayLanguage" />
  <input type="hidden" name="browserInfo" />
  <input type="hidden" name="ipAddress" />
  <input type="hidden" name="country" />
  <input type="hidden" name="city" />
  <input type="hidden" name="region" />
</p>

<script type="text/javascript" src="http://gd.geobytes.com/gd?after=-1variables=GeobytesCountry,GeobytesCity,GeobytesRegion,GeobytesIpAddress">
</script>

<script type="text/javascript">
<!--
function getUserInfo() {
   var userDisplayLanguage = navigator.language ? navigator.language :
navigator.userDisplayLanguage;
   var browserInfo = navigator.userAgent;
   var ipAddress = sGeobytesIpAddress;
   var country = sGeobytesCountry;
   var city = sGeobytesCity;
   var region = sGeobytesRegion;

   document.mturk_form.userDisplayLanguage.value = userDisplayLanguage;
   document.mturk_form.browserInfo.value = browserInfo;
   document.mturk_form.ipAddress.value = ipAddress;
   document.mturk_form.country.value = country;
   document.mturk_form.city.value = city;
   document.mturk_form.region.value = region;
}

getUserInfo();

// -->
</script>

MechanicalTurk (last edited 2011-08-10 18:29:00 by echidna)

MoinMoin Appliance - Powered by TurnKey Linux