Vernier Project

Part of Claudia's work.

How is space represented in the visual system? In an immobile eye, a stationary stimulus is necessarily encoded by the pattern of active receptors in the retina. But, the eyes are always in motion, even during fixation, and eye movements make spatial information available in the temporal domain. Our recent work has provided strong evidence that the visual system also uses the temporal modulations resulting from fixational eye movements to encode fine spatial detail (Rucci et al 2007; Kuang et al., 2012). Here we investigated the mechanisms of this encoding process.

Contents

  1. Description
  2. Experiments

    • 2.1 Subjects
      2.2 Experiment 1: CRT monitor
      2.3 Experiments 2: LED monitor (20 ms stimulus exposure)
      2.4 Experiments 3: LED monitor (10 ms stimulus exposure)
      2.5 Experiment 4: Stimulus deflector + CRT
      2.6 Experiment 5: stimulus deflector + LED monitor

  3. Ongoing work and notes

    • 3.1 10/16/2014
      3.2 10/10/2014
      3.3 08/07/2014
      3.4 08/21/2014

  4. Lab meetings slides
  5. Project Reports

Description

attachment:VernierProject_experimental_procedure.png
Experimental procedure: Observers are asked to report whether the top bar of a vernier stimulus is to the left or to the right of the bottom one. The two bars are shown at two different temporal interval t1 and t3 with a temporal delay (t2). The two bars of the vernier stimulus are shown at the same horizontal position on the retina aligned with the Preferred Retinal Locus of Fixation (PRFL), thus no actual displacement is present in the retinal stimulus. ‎

To determine whether the visual system is capable of establishing spatial representations from delays in neuronal responses, we designed an experiment in which observers were asked to report the configuration of a standard Vernier viewed through a narrow aperture. The two lines forming the vernier stimulus were never visible at the same time, and their offset was exclusively determined by eye movements. This was achieved showing the two bars so that on the retina were vertically aligned with the Preferred Retinal Locus of Fixation (PRLF). The intervals in which the two bars were exposed were separated by a period of 100 ms (t2). Since the eye drifted in this interval, the two lines were actually presented at different locations on the monitor (but at the same horizontal position on the retina). In this way the input to the retina did not contain any explicit spatial cue signaling the offset between the two lines.

In all experiments, observers faced the same task. They were asked to report the spatial configuration of the Vernier stimulus, that is whether the top bar was located to the left or to the right of the bottom one. If the visual system is not capable of reconstructing space from the temporal modulation resulting from ocular drift, subjects should not be able to perform this task, and discrimination performance are expected to be at chance levels.

Experiments

For a schematic representation of the instructions given to the observers see: instructions.

There are five experiments. They differed for the monitor (LED or CRT) and two of them used the stimulus deflector.

Exp 3-5 were run with two durations, 100 ms and 500 ms.

For a summary of the characteristics of the main experiments of the project see: main experiments characteristics. PLEASE ADD FUNCTIONS USED TO RUN EXPERIMENT.

Subjects

(as of 11/17/2014)

Exp 1

Exp 2

Exp 2

Exp 3

Exp 4

Exp 5

CRT

LED (low)

LED (high)

LED flashes

Deflector CRT

Deflector LED

exposure duration

n

20ms

20ms

10ms

20ms

10ms

#

Manuscript #

Exp 1

Exp 2

Exp 2

Exp 3

Exp 4

Exp 5

Reason

1

S1

1

2

4

3

5

2

S2

1

moved/not interested

3

S3

1

moved

4

S4

1

2

3

4

5

--

only training

6

--

2

1

dropped

7

--

dropped: for unrelated reasons

8

S5

3

1

2

4

5

6

9

--

1

moved/not interested

10

S6

1

2

3

11

--

only training

12

--

left during training

13

S7

2

1

14

S8

1

2

15

S9

1

2

positive result

numbers indicate the order of data collection

negative result/not clear

data collection in progress

only training

not completed

Experiment 1: CRT monitor

Subjects. We have data from 6 observers, all with positive results. One subject was excluded (was giving negative results and left).

EyeRIS Code. The EyeRIS code used for Experiment 1 can be found at: \\opus.cvs.rochester.edu\aplab\ClaudiaProjects\EncodingSpaceInTime\DataAndCodes\Experiment1

Stimuli and procedure. Stimuli were displayed on a fast phosphor monitor (Iyamaya HM204DT) at a resolution of 1024x768 pixels and vertical refresh rate of 200 Hz. The lines forming the vernier stimulus were 1.35' wide and 28.35' long and the vertical gap between them was 8'.

Both lines were displayed on the retina at the same position on the horizontal axis (the Preferred Retinal Locus of Fixation, PRLF). The duration of the exposure of the each line varied according to the eye movements of the observers: the line remained visible until the line of sight moved away from it. Then, after 100 ms, the second line of the Vernier stimulus was shown at the same horizontal retinal position of the first line and it remained visible as long as the line of sight was at that particular location. The order of presentation of the two lines was random. At the end of each trial, a bright random noise mask was presented for 1333 ms to signal to the observer the conclusion of the trial and avoid dark adaptation.

Observers refined the calibration procedure for a central marker every 5 experimental trials (this operation remains valid in Exp.1,2,3).

Experimental Data. Data can be found at \\opus.cvs.rochester.edu\aplab\ClaudiaProjects\EncodingSpaceInTime\DataAndCodes\Experiment1

Subject .eis data are contained in data\S#. Raw matlab data are in \data. Folder \results contain processed data: all trials, valid trials, and performance---saved but not really used to plot figure.

Subject pressed left or right when they saw two bars. Selection of trials was done online and then double-checked by means of Experiment1_FilterTrials.m. Trials excluded are those with (1) no two exposures; (2) presence of blinks and/or saccades/microsaccades from first stimulus on to second stimulus off; (3) possible inaccuracy in retinal stabilization.

Performance file (PerfS1.mat) contains a summary of performance. Variable perf gives number of trials, number of hits (pressed left), false alarms (pressed left, stimulus right), misses (pressed right, stimulus left), correct rejections (pressed right, stimulus right). Catch trials are the zero gap size, subject pressed left or right. dprime report d' on total number of trials without considering different gap sizes.

per.Detail contains information divided in gap sizes. StimD is the gap size.

To generate the main result figure: Figure_Experiment1_Performance .

Results. See Exp1Res.pdf.


Performance in Experiment 1. a: Proportion of correct responses and d' as a function of the gap size. The d' values are plotted on the right axis (filled data point). b: Proportion of correct responses as a function of the gap size according to the main direction of ocular drift. Data represent averages across observers (N=6). Error bars indicate one standard deviation.

Panel a shows the average performance across observes. The average percentage of correct responses across observer was 69% with a standard deviation of 6 at gap size 1.4'. Performance increased significantly to 84% ± 6 when the displacement between the two lines also increased to 2.7' (p = 0.0028, paired t-test). On average across observers the sensitivity index (d') was 0.99 at gap size 1.4' and it increased to 2.02 at gap size 2.7' reflecting the facility in which observers discriminated the stimuli.

Comments. Even if the order of the presentation top/bottom line was randomized, the configuration of the two stimuli was dictated by the direction of ocular drift on the horizontal axis. It is therefore possible that the results were consequences of a combination of a general bias in the direction of ocular drift and a bias in the subject response: for example, the observer may not have access to an extra-retinal drift signal but have a general knowledge that his eye tends on average to drift in a given direction. For this reason, we analyzed performance as a function of the direction of ocular drift. Panel b shows that the average performance across observers was similar for both direction of motion. This result demonstrate that our findings were not a consequence of biases toward a particular direction of ocular drift or in the response of the observers.

Concerns. Although Experiment 1 was designed to avoid explicit spatial cues, two cues may have occurred: (1) motion on the stimulus on the retina because of the finite steps of retinal stabilization and (2) the persistence of the phosphors on the CRT. These cues may leave an undesired spatial trace of the configuration of the Vernier stimulus. To circumvent this problem, we designed a second experiment (Experiment 2) in which stimuli were displayed for much shorter intervals on a custom-made discrete LED monitor specifically designed to minimize phosphor persistence.

Experiments 2: LED monitor (20 ms stimulus exposure)

This was the first experiment with the LED monitor. The eyeris code was similar to experiment 1, checking eye movements online. The bar was displayed when drift became unidirectional.

For a description of the characteristics of the LED monitor see: LED monitor.

Subjects. We have data from three subjects. S1, S5, S6, all with positive results.

EyeRIS Code. Eye movements were monitored online. The first bar was presented when drift became unidirectional. Trials with blinks, saccades, microsaccades in the critical interval were eliminated. The EyeRIS code used for Experiment 2 can be found at: \\opus.cvs.rochester.edu\aplab\ClaudiaProjects\EncodingSpaceInTime\DataAndCodes\Experiment2

Stimuli and procedure. In this experiment, each line was displayed in a position vertically aligned with the preferred retinal locus of fixation, for 20 ms. The interstimulus time interval was either 100. The order of presentation of the two lines (bottom or top) was random.

The calibration procedure was performed with the isolated LEDs of the LED monitor.

Experimental Data. Data can be found at \\opus.cvs.rochester.edu\aplab\ClaudiaProjects\EncodingSpaceInTime\DataAndCodes\Experiment2

Subject .eis data are contained in data\S#. Raw matlab data are in \data. Folder \results contain processed data: all trials, valid trials, and performance---saved but not really used to plot figure.

Subject pressed left or right when they saw two bars. Selection of trials was done online and then double-checked by means of Experiment2_FilterTrials.m. Trials excluded are those with (1) no two exposures; (2) presence of blinks and/or saccades/microsaccades from first stimulus on to second stimulus off;

Performance file (PerfS1.mat) contains a summary of performance. Variable perf gives number of trials, number of hits (pressed left), false alarms (pressed left, stimulus right), misses (pressed right, stimulus left), correct rejections (pressed right, stimulus right). Catch trials are the zero gap size, subject pressed left or right. dprime report d' on total number of trials without considering different gap sizes.

per.Detail contains information divided in gap sizes. StimD is the gap size.

To generate the main result figure: Figure_Experiment2_Performance .

Results. See Exp2Res.pdf.

Experiments 3: LED monitor (10 ms stimulus exposure)

This is the main experiment. Four subjects were run in the two conditions 100 and 500 ms. There is no online detection of drift or anything. Trials are filtered in data analysis, so to have a low level of curvature on the X axis.

Subjects. We have data from four subjects. S1, S4, S5, S7, all with positive results.

Stimuli and procedure. In this experiment, each line was displayed in a position vertically aligned with the preferred retinal locus of fixation, for 10 ms. The interstimulus time interval was either 100 or 500 ms in different sessions (different days). The order of presentation of the two lines (bottom or top) was random.

The calibration procedure was performed with the isolated LEDs of the LED monitor, like in Experiment 2. For a description of the characteristics of the LED monitor see: LED monitor.

EyeRIS Code. The EyeRIS code used for Experiment 3 can be found at: \\opus.cvs.rochester.edu\aplab\ClaudiaProjects\EncodingSpaceInTime\DataAndCodes\Experiment3

Experimental Data. Data can be found at \\opus.cvs.rochester.edu\aplab\ClaudiaProjects\EncodingSpaceInTime\DataAndCodes\Experiment3

Subject .eis data are contained in data\S# for 100 and 500 ms. Raw matlab data are in \data. Folder \results contain processed data: all trials, valid trials, and performance---saved but not really used to plot figure.

Subject pressed left or right when they saw two bars. They pressed X when they saw only one bar or nothing.

In this experiment all trials were saved. Selection of trials was done by Experiment3_FilterTrials.m. Trials excluded are those with (1) microsaccades/saccades within the interval TimeFirstBar-50 ms --- TimeSecondBar+50 ms; (2) too curved on the x axis. Curvature (length/amplitude) threshold of 0.65; (3) blinks within the interval TimeFirstBar-100 ms --- TimeSecondBar+100 ms; (4) Retinal error, measured as difference between eye position and stimulus position, smaller than 1 arcmin; (5) offset from center of the display more than 7 deg.

Performance file (PerfS1.mat) contains a summary of performance. Variable perf gives number of trials, number of hits (pressed left), false alarms (pressed left, stimulus right), misses (pressed right, stimulus left), correct rejections (pressed right, stimulus right). Catch trials are the zero gap size, subject pressed left or right. NotDet contains the number of trials subject pressed X. dprime report d' on total number of trials without considering different gap sizes.

per.Detail contains information divided in gap sizes. StimD is the gap size.

To generate the main result figure: Figure_Experiment3_Conditions (comparison 100 vs. 500 ms) and Figure_Experiment3_Performance (100 ms in both directions).

Figure_Experiment3_Performance: generates data for all subjects.

Results. The 100 ms results are plotted in Exp3Res.pdf. The comparison between 100 and 500 ms are plotted in Exp3Res_100VS500.pdf.

Experiment 4: Stimulus deflector + CRT

To ensure that imperfections in retinal stabilization did not give explicit spatial information, we also designed a third experiment, which used a different technique for retinal stabilization. In this experiment (Experiment 4), we used a stimulus deflector (see Crane and Clark, 1978) to stabilize the stimulus on the retina. Stimuli were displayed on a fast CRT monitor as in Experiment 1 (Iyamaya HM204DT).

This is the first experiment with the stimulus deflector with the CRT. Data were collected in arcmins by means of calibration executed with the CRT before mounting the stimulus deflector. There was no interruption in tracking while mounting the deflector.

Stimuli and procedure. In this experiment, each line was displayed in a position vertically aligned with the preferred retinal locus of fixation, for 20 ms. The interstimulus time interval was either 100 or 500 ms. The order of presentation of the two lines (bottom or top) was random.

Because of the different apparatus, the calibration procedure of Experiment 4 differed from those of the previous two experiment. Since the stimulus deflector stabilizes the entire visual field, it does not allow an approach similar to the one in Experiment 1.

For a description of the calibration of the stimulus deflector see: stimulus deflector (LINK).

EyeRIS Code. The EyeRIS code used for Experiment 4 can be found at: \\opus.cvs.rochester.edu\aplab\ClaudiaProjects\EncodingSpaceInTime\DataAndCodes\Experiment4

Subjects. Three subjects available. S1, S5, S6. S1 gave clear positive results. S5 uncertain: this subject had very high percentages in Exp.1-2, but quite low here. S6 gave a negative result and explicitly complained about the length of the procedure.

Experimental Data. Directories data contain the three directories of the subjects only for the 100 ms condition. For example, visualize the x coordinate of the first trial upload S1.mat in matlab and plot(Data.x{1}).

Data Analysis. Data were binned to reduce effect of noise. Each bin is 4 arcmins. Experiment4_all convert eis into matlab, preprocesses and filters trials according to selected parameters. Current parameters are: 100 ms free blinks and microsaccades. If desired, th filters trials to stay within the distance from the center of the unstabilized marker. Ends by saving the matrices S1, S1_alltrials, and S1_validTrials. Figure_Experiment4_performance generates the figure of the data. It gives this figure: Exp4Res.pdf

Results. The 100 ms results are plotted in Exp4Res.pdf.

Advantages and limits of Experiment 4. In Experiment 4 no spatial offset was ever present either on the display or on the retina no matter the eye movement performed by the observer. This was an advantage of Experiment 4: it allowed exposure to high contrast stimuli stabilized on both the horizontal and vertical axes while excluding possible spatial cues resulting from phosphor persistence. It was also a limitation, because possible persistence played against the effect we are after.

Also, the calibration procedure was very time consuming. This, together with the need for high cooperation from the observer, limited the window of time available for data acquisition and only allowed collection of data from 2 experienced observers in this experiment.

Experiment 5: stimulus deflector + LED monitor

This is the stimulus deflector with the LED. Data were collected in voltages and converted into arcmins by means of an average calibration.

Subjects. Only one subject available. Another subject (S7) could not go through the procedure, as she would not perceive the afterimage. S5, for which calibration data are included here, did not actually take part in the experiment. She went through all the other experiments giving positive results, but gave a negative result in Exp.4, for which she explicitly complained about the length of the procedure.

Experimental Data. Directories data and data_calibration. Data contains the results from two experimental attempts:

  1. S1 Pilot. This is a first intermediate experimental session with CC. To speed up the procedure, the after-image was not used (CC had the impression that this was good enough anyway), and data were collected directly (skipping the calibration) but without explicitly asking EyeRIS for a voltage output. This means that EyeRIS probably used a previously stored calibration.
  2. S1 Data. In this experiment (two sessions in the same day) the procedure was as follows: 1. The unstabilized reference was set by CC on transparent film. 2. In dim light, turned on stimulus deflector and adjusted offset to center LED monitor. 3. Positioned another monitor (IPAD) with bar; CC adjusted offset of bar to center. 4. CC adjusted both gain and offset based on after-image. When she removed the IPAD the LED monitor was still centered. MA tried this procedure, but for him the LED monitor had shifted after removing the IPAD.


Calibration data. A set of calibration data collected in a dedicated experimental sessions. The subject moved out of the eyetracker, bite bar was removed, eyetracker moved, and chair position changed. Calibration was performed again. For CC the last four calibration session (10-14) were collected sequentially, without altering positions.

Data Analysis. AnalyzeData_Calibration loads the calibration data and estimates an average calibration. Experiment5_all convert eis into matlab, preprocesses and filters trials according to selected parameters. Ends by saving the matrices S1, S1_alltrials, and S1_validTrials. Figure_Experiment5_performance generates the figure of the data.

Results. Results from CC can be found in Exp5_Results.pdf

Possible improvements. A better procedure could be: 1. Have an unstabilized central reference fixed at the center of the stimulus deflector. 2. In dim light, turn on stimulus deflector and adjust offset to center LED monitor. 3. Position another monitor (IPAD) with bar; subject adjusts offset of bar to center. 4. Subject adjusts gain based on after-image. Possible problem is that offset may change too, forcing tuning from step 1. One possibility is to use a better calibration figure than a bar, maybe two bars vertically connected by a thin segment.

Ongoing work and notes

10/17/14 MR: According to MA the stimulus deflector works as m=KV +o. I am not really sure of this, because the manual is for the amplifier, not for the stimulus deflector itself. But if it is correct, then the procedure should be as follows: (1) the subject looks at the center dot of the well-mounted reticule; (2) a bar is presented on an ipad or other means at the stabilized center; the subject adjusts the position of the bar to make it at the center. CC will look into a mouse to control this. Even just the ipad keyboard will do. (3) subjects adjusts gain looking at eccentric unstabilized dots in the reticule. This should not affect offset. After gain is adjusted, experiment can begin.

I still would like to have an unstabilized luminous reference to be brought in periodically, to recenter gaze and ensure that we are at the center of the operating range.

10/16/2014

To DO:


10/16/2014 MR: CC presented refined table of subjects. It clarifies concerns on who dropped out from experiments and why.

There are now three subjects running the LED experiments, but results at the moment are not positive. MA suggested starting with very clear stimuli with 100 ms flashes so that subjects can see what is going on.

There remains concerns on how the stimulus deflector exactly works and thus what is the best calibration strategy. Three possibilities are that: m = k DV + o; m = k V + o; m = k [V+o]; where m is the position of the mirror, V is the eyetracker voltage, k and o gain and offset.

MA and CC should review data analysis together. Depending on how the deflector works a simple calibration strategy needs to be found. MA agreed on mounting the unstabilized reference.

10/10/2014

MR Notes: MA tried the calibration, not the experiment. He wasn't able to make it work: there was an asymmetry between shifts to the left and to the right. MA raised the need to use a precise unstabilized reference at the center of the visual field---we are still currently using a hand-made film loosely adjusted by the subject, even though we long purchased reticules. The issue seems to be simply mounting the reticule. MA said he would look into that.

I am still not convinced that the current procedure of adjusting stabilization gains after saccades works. It obviously shifts the offset. In my view, shifts in offset will be minimized by looking first back at the center. CC says that current procedure is more intuitive and MA argues that it converges, but I don't really see why.

There are three subjects now 'training'. But are results trustworthy if calibration is not accurate as in MA case? CC seems to be sure that this does not happen. The plan is that MA should be sitting more extensively in the experimental set up to actually get to experience the full experiment and identify issues.

Agreed on going over the list of results next week to see how many subjects were rejected and criteria for that.

08/07/2014

CC ran another subject with the stimulus deflector. She had two subjects that sort of worked (herself and last year's RA). But this new subject gave chance level responses at all eccentricities. I am puzzled. There are three main possibilities that I see for this pattern of results:

  1. The result is an artifact (for example, the previous two subjects had under-adjusted the gain of the compensation);
  2. The experiment, with its extremely long calibrations, is simply too difficult for the subject;
  3. The CRT persistence is biasing results. In this specific experiment, monitor persistence will tell the subjects that the two lines are aligned, effectively bringing down performance.

My suggestion: we can try running a simpler version of this experiment, which takes care of points 2 and 3 above. Forget about all calibrations, start experiments with the deflector in place, and directly acquire voltages (or arcmin using a dummy calibration or a calibration through the deflector). Place 2 aligned LEDS in front of the deflector ---or use LED monitor to this end---. Run an experiment in which we check performance with increasing amplitudes of eye movements, even if we don't know whether eye movement amplitude is correct. Place great care in adjusting the gain of the stabilization. If we still see an effect, the result must be real. One possibility is to eliminate a dedicated stimulus deflector figure, and insert these data as points in one of the previous figures.

08/21/2014


Lab Meeting Slides

Project Reports

11/5/2014 Report Manuscript_Aperture.pdf.

PLEASE INSERT LATEST REPORT

Claudia-VernierProject (last edited 2018-04-17 14:28:42 by ChelseaMarsh)

MoinMoin Appliance - Powered by TurnKey Linux