Guidelines for running experiments

1. Well in advance of starting an experiment

1.1. RSRB

At least one month before running an experiment, the experimenter needs to check whether there is RSRB approval for the study they plan to conduct. In particular, is both the method and the language covered by valid RSRB approval? Will the approval still be valid throughout the entire expected duration of the study (which may take several months)? Will the experiment require consent forms in a language that we do not yet have available (e.g. we have approval to run studies on Chinese, but only on Chinese speakers on campus, all of whom speak English, so we do NOT have a consent form in Chinese yet)?

1.2. EPRP

If you have not gotten an EPRP (Ethical Principals in Research Program) number, please go to the [http://www.rochester.edu/ohsp/certification/eprp.html EPRP website] and take the test to get one before you run any subjects and sign consent forms. It is mandatory that any faculty, grad students, post-docs, or staff who are on the protocol as investigators or co-investigators have one. It is a good idea even for the undergrads to get an EPRP number though, because you are required to have one if you ever run subjects for us, even though you most likely won't be on the protocol and consent forms.

EPRP numbers do expire every few years. Be sure to know when yours does, and take steps to get it renewed in time that your ability to run experiments is not interrupted.

2. Before you start designing stimuli

Create a wiki-page for your experiment. It should contain detailed information about the design, the type of stimuli, the methods you plan to use, the number of items you will need, how you plan to analyze the data, etc. This page should only be readable to hlp lab members (see [wiki:HlpLab/WikiEditing how to restrict access to wiki pages]). If you do not know how to edit the wiki, meet with the lab manager to get instructions (it's easy). You can find an [wiki:HlpLab/Projects/ChineseClassifiers/LabLog example for a good experiment summary wiki page here]. This page should later also contain a link to the annotation guidelines for RAs -- even if you use a standard annotation scheme.

3. While the experiment is running

3.1. Deviations and violations

Unfortunately, there are going to be deviations and violations of RSRB protocols every once in a while. Although we need to do all we can to avoid them, we will keep track of those that occur. It's still better to do that than not to do it. So, if you are in doubt about something, you should ask the lab manager and/or Florian. In short, please inform us about possible deviations or violations. The consequences for the lab, if you fail to report such deviations, are quite severe.

Below, you'll first learn about general considerations about paying subjects, consent forms, payment sheets (who got paid how much), and subject sheets (who you're running in what list). Then read on to see where all these forms should be stored and how should they be organized. It is very important that you follow these guidelines.

3.2. Paying subjects

Money for paying subjects comes from Chris Dambra or Jennifer Gillis in the BCS office in Meliora 360. You can only get money or return payment forms from 10am-12pm and 2pm-4pm Monday through Friday. Be sure to get the money before your subjects are scheduled.

Each subject must sign a payment form and provide their home address, which cannot be a P.O. Box or campus box. If the subject is a student and willing, it would be helpful for accounting purposes to get their student ID number. In the past, we got SSNs, but that is no longer allowed. Payment forms are available from Andrew or Dana, as well as being available on this page: attachment:Subject_Payment_Form.xls. Payment forms must be returned to Meliora 360, so the department can be reimbursed and we can keep getting money to pay subjects.

3.3. Consent forms

Anyone running experiments in the HLP/Jaeger lab paid for out of Florian's account must use a Cross-linguistic Studies on Human Communication consent form.

Only consent forms with an RSRB stamp are valid. Even if a copy is translated literally from an approved consent form, it is only valid to use that form if it itself has been approved and hence has an RSRB stamp on it. Also, the first page of the consent form must be on University of Rochester letterhead.

We currently have consent forms in a variety of languages and for studies in several languages, as well as an Amazon Mechanical Turk consent form (English only) online at [http://www.hlp.rochester.edu/consent/English_2010-05-13.pdf]. Before you run an experiment, make sure our RSRB protocol covers that language and that you are using the consent form in the right language.

If you are on the protocol as a co-investigator, you can go to the [https://rsrb01.urmc.rochester.edu/ RSRB website], sign in and go to the Documents tab on the protocol to download the current approved consent form. These are not on letterhead, so you'll have to print them on letterhead if you download them yourself. Andrew will also keep master copies of the current consent forms in the lab, and will copy them for you if you let him know you are running a study. He will also email the lab if the consent forms change so you always have the most up to date.

Consent forms expire once a year after the Continuing Review, at which point we get new versions with the next year's stamp. Make sure that you have the current version of the consent form. The expiration date is on the RSRB stamp on the form. The Amazon Mechanical Turk consent forms are stored on the webserver with the expiration date in the filename, i.e Languagename-YYYY-MM-DD.pdf

Each time you run a subject, you are required to get a consent form for them, even if they have run in experiments in the past, or even another day of the same experiment. (i.e. If you have a 5 day experiment, you need 5 consent forms for each subject.) If the consent form changes while you are in the middle of a study, you need to start using the new consent form immediately, not when you finish the current study. The RSRB also now requires that we give each subject a copy of the consent form. Previously they only required that we offer the subject a copy. You can give them an unsigned consent form; you don't have to photocopy the signed one. Most subjects don't actually want a copy of the consent form, so please stress that we are required to give it to them, but inform them that if they don't want it they can leave it on the table on their way out.

Do not leave forms with personally identifying information (e.g. consent forms and subject running lists) out in the lab! When you leave the lab, make sure all paperwork is in its folder in a filing drawer.

• Each (series of) experiment(s) should have a green folder in the filing cabinet in the main room to the left of Andrew's office door. That folder needs to be accurately labeled with the name of the experiment, the abbreviation code for the experiment if there is one, and the name of the main investigator (the student(s) who designed the study.
• Within each of these green folders, there should be separate beige folders for each experiment and for (a) consent forms, (b) payment forms, and (c) whatever other documents you use to keep track of how many subjects have been run in which list of the experiment. Each folder should be labeled with the experiment name, experiment code, experimenter's name, and the name(s) of the RAs running the subjects.
• Participant surveys should be stored in the folder on the right in that cabinet (top drawer). Since they are the same for all experiments, they should all be stored in the same folder.
• Please do not keep copies of unused forms in the experiment folders. That leads to outdated forms being used. Please always use forms from the "Unused Forms" folder on the right of the top drawer. If you need to make more copies, only use the master copies in the "Master copies" folder on the right in the top drawer.
• If this suggested routine causes problems, please inform me about them, so that we can adapt.
• If you have not already organized your forms according to the outline above, please do so as soon as possible.

3.4. Subject sheets

Keep a sheet with information about subjects that run in your experiment. Write the date, the list number, the subject's name, which computer, and any relevant notes for each subject. For future reference, keep this list in the same folder as the consent forms.

3.5. Coordinating with other experiments

We want to use the time we have with each subject as much as possible, so please coordinate with other experimenters to arrange running subjects in multiple experiments at the same time, especially if each experiment is under one hour. Even if the experiment is long and it doesn't make sense to combine, try to get the subject scheduled for another experiment they haven't done, if there are any they can do. (Coordinate with the other experimenter, of course.) Keep in mind though that some experiments shouldn't be run together and that some experiments shouldn't be done if particular experiments have been done in the past. For the time being, we have a spreadsheet on Google Docs with the experiments. Check it to see what all is running. If you need access, contact Andrew or Katrina to get it. Eventually we will have a subject scheduling system that keeps track of what experiments subjects have run and what experiments block other experiments.

4. After the experiment is completed

This means after all subjects have run, not after each subject...

FILL THIS IN

4.1. At the end of a Mechanical Turk experiment

Make sure to submit the invoice from Amazon to Kathy for reimbursement as soon as you get it. Also CC Florian on that email, as Mechanical Turk charges his personal American Express card, and he needs to know the charge occurred.