TITLE: Testing Framework Protocol
AUTHORS: Alex Lepauvre, Rony Hirschorn, Lucia Melloni, mudrikli

[DESCRIPTION]
This protocol provides a step-by-step guide for testing and validating experimental setups in visual event-related research. It includes Testing Preparation required to conduct the protocol, Visual Angle Check for stimulus size accuracy, Controlled and Uncontrolled Event Content Logging to verify the correctness of event and response logs, Controlled and Uncontrolled Event Timing Logging to assess timing precision, Validating Experimental Design Parameters to confirm design adherence, and Peripheral Testing for devices like EEG, MEG, and eye trackers. By standardizing setup testing, this protocol enhances reproducibility, robustness, and cross-study integration. Shared on protocols.io, it supports transparent and reliable research practices.

[STEPS]
SECTION: Controlled Event content logging
5.1. Number of tested events conditions
 
 
 



SECTION: Controlled Event content logging
5. Describe method used to test:
 
 
SECTION: Controlled Event content logging
5.2. Number of tested events per condition: 
 

SECTION: Controlled event timing logging
7. Describe method used to test:

 

SECTION: Controlled event timing logging
7.1. Select threshold   for binarization:
 

SECTION: Controlled event timing logging
7.7. Compute and report the log file average timing inaccuracies (  )
  
 

SECTION: Testing preparation
1. Adjust the experiment scripts to conduct the testing protocol
SECTION: Controlled event timing logging
7.2. Binarize the signal:
  
Where   is the recorded signal

SECTION: Controlled event timing logging
7.4. Detect event observed onset (  ):
  
SECTION: Controlled event timing logging
7.5. Compute  : 
  
Where n is the number of detected photodiode events
SECTION: Controlled event timing logging
7.6. Compute  : 
  
Where   is the log file time stamp of all events (n) in the log file

SECTION: Peripheral testing
11. Peripherals that will be used to collect data during the experiment should be tested and reported according in the step case below. 
SECTION: Peripheral testing
12. The experiment was conducted entirely while recording data on the Eyetracker. For each event, a trigger was sent from the experimental computer to the device's computer to identify each event's content as well as timing. The triggers' timing accuracy was computed by comparing them to the photodiode. The triggers' content accuracy was computed by comparing  them to the log file event content.
SECTION: Peripheral testing
12.1. Compute finite difference of each Eyetracker trigger time stamp  : 
  
SECTION: Peripheral testing
12.2. Compute and report the device triggers average timing inaccuracies (  ):
  

 

Where   is the finite difference of photodiode onsets computed in step 3.4
SECTION: Peripheral testing
12.3. Compute the log file file timing inaccuracies standard deviation (  )
   

 

SECTION: Peripheral testing
12.4. Compare the   information to the log file using scripted test
 

SECTION: Peripheral testing
12.5. Tested events counts (should be equal to the total number of events in one experimental run): 

 

SECTION: Peripheral testing
12.6. Confirmation that the test was performed and that no discrepancies remain (between the logged responses and the planned response sequence)
 

SECTION: Peripheral testing
12. The experiment was conducted entirely while recording data on the EEG system. For each event, a trigger was sent from the experimental computer to the device's computer to identify each event's content as well as timing. The triggers' timing accuracy was computed by comparing them to the photodiode. The triggers' content accuracy was computed by comparing  them to the log file event content.
SECTION: Peripheral testing
12.1. Compute finite difference of each EEG trigger time stamp  : 
  
SECTION: Peripheral testing
12.2. Compute and report the device triggers average timing inaccuracies ( ):
  

 

Where   is the finite difference of photodiode onsets computed in step 3.4
SECTION: Peripheral testing
12.3. Compute the log file file timing inaccuracies standard deviation ( )
   

 

SECTION: Peripheral testing
None. Select threshold   for binarization:

 


SECTION: Peripheral testing
None. Binarize the signal:
  
Where   is the recorded signal

SECTION: Peripheral testing
None. Detect event observed onset (  ):
  
SECTION: Peripheral testing
None. Compute  : 
  

SECTION: Peripheral testing
None. Compute  : 
  
Where   is the log file time stamp of a given event t

SECTION: Peripheral testing
None. Compute and report the log file average timing inaccuracies (  )
  


 

SECTION: Peripheral testing
None. Compute and report the log file timing inaccuracies standard deviation ( )
  

 


None. Tested events counts: 
 

None. Compute the log file file timing inaccuracies standard deviation (  )
   
 

None. Compare the   event content and the Log file:
None. Incorrect trigger event content count: 
 

SECTION: Testing preparation
2. Create a pre-defined response sequence to be executed during the test run to compare against the log file to identify any log file issues. The event sequence must sample all possible answers, as well as unexpected button presses to assess the robustness of the experiment. 
SECTION: Testing preparation
3. Prepare for a test run:
SECTION: Testing preparation
1.1. Present a black square on a corner of the screen that switches to white on the same frame as the stimulus onset and then turns back to black after three frames.
SECTION: Testing preparation
1.2. Add functionality to record sound from a microphone during the execution of the experiment to record button presses - "click" sounds - to assess response box latencies.
SECTION: Testing preparation
3.1. Attach a photodiode recording device (see material) over the displayed black/white square flashed upon stimulus onset and record the measured voltage to a file for later processing
SECTION: Testing preparation
3.2. Place a contact microphone (see materials) on the response devices used to record the sound made by button presses and ensure quietness in the room (avoid speaking, opening/closing doors, etc.)  to facilitate later processing stages 
SECTION: Testing preparation
3.3. Prepare to take notes of the presented events on the screen for assessing logging content accuracy. In the case of a fast paced experiment, set up a camera to capture the screen for slow paced annotations of events presentation. 

SECTION: Peripheral testing
12.4. Compare the   information to the log file using scripted test
 

SECTION: Peripheral testing
12.5. Tested events counts (should be equal to the total number of events in one experimental run): 

 

SECTION: Peripheral testing
12.6. Confirmation that the test was performed and that no discrepancies remain (between the logged responses and the planned response sequence)
 

SECTION: Peripheral testing
12. The experiment was conducted entirely while recording data on the MEG system. For each event, a trigger was sent from the experimental computer to the device's computer to identify each event's content as well as timing. The triggers' timing accuracy was computed by comparing them to the photodiode. The triggers' content accuracy was computed by comparing  them to the log file event content.
SECTION: Peripheral testing
12.1. Compute finite difference of each EEG trigger time stamp M: 
  
SECTION: Peripheral testing
12.2. Compute and report the device triggers average timing inaccuracies (  ):
  

 

Where   is the finite difference of photodiode onsets computed in step 3.4
SECTION: Peripheral testing
12.3. Compute the log file file timing inaccuracies standard deviation (  )
   

 

SECTION: Peripheral testing
12.4. Compare the   information to the log file using scripted test
 

SECTION: Peripheral testing
12.5. Tested events counts (should be equal to the total number of events in one experimental run): 

 

SECTION: Peripheral testing
12.6. Confirmation that the test was performed and that no discrepancies remain (between the logged responses and the planned response sequence)
 

SECTION: Visual angle check
4. The visual angle of each stimulus of interests was tested using by measuring the size of the stimulus presented on the screen with a ruler. The size of each stimulus was converted to degrees of visual angle using this converter: https://www.sr-research.com/visual-angle-calculator/ with the participant's expected distance from the display. The eccentricity was computed using the same converter (if applicable). 
SECTION: Visual angle check
4.3. Distance between participant nasion and screen
 

SECTION: Visual angle check
4.4. Measured stimuli sizes (if more than one, comma separated)
 

SECTION: Visual angle check
4.5. Eccentricity (if more than one, comma separated)
 

SECTION: Uncontrolled events timing
8. The experiment was conducted entirely while a human actuator executed a pre-defined sequence of button presses. Each press content and timing was saved in the log file. In addition, a contact microphone was located close to the keys to be pressed and recorded on the experimental computer. The response devices' latencies were computed as the difference between the detected onset of button press in the response device and the recorded and parsed sound file.
SECTION: Uncontrolled events timing
8.1. Select threshold   for binarization:
 

SECTION: Uncontrolled events timing
8.2. Binarize the signal:
  
Where   is the recorded audio signal containing the button presses sounds

SECTION: Uncontrolled events timing
8.5. Compute  : 
  
Where n is the number of detected audio events

SECTION: Uncontrolled events timing
8.6. Compute  : 
  
Where   is the log file time stamp of response events (n) in the log file

SECTION: Uncontrolled events timing
8.7. Compute and report the log file average timing inaccuracies (  )
  
 

SECTION: Uncontrolled events timing
8.8. Compute and report the log file timing inaccuracies standard deviation ( )
  
 


SECTION: Uncontrolled events content logging
6. The experiment was conducted entirely while a human actuator executed a pre-defined sequence of button presses. Each press content and timing was saved in the log file. In addition, a contact microphone was located close to the keys to be pressed and recorded on the experimental computer. The response devices' latencies were computed as the difference between the detected onset of button press in the response device and the recorded and parsed sound file.
SECTION: Uncontrolled events content logging
6.1. Compare the logged response description against the planned response sequence. An inaccurate response logging would be identified as a discrepancy between the two, such as the log file recording a response as "No" when in fact the button mapped to the "Yes" logging was pressed for example. Such issues must be addressed before data collection and the report i below must be 0. 
SECTION: Uncontrolled events content logging
6.2. Number of tested responses types: 
 

SECTION: Uncontrolled events content logging
6.3. Number of tested responses per response types: 
 

SECTION: Uncontrolled events content logging
6.4. Confirmation that the test was performed and that no discrepancies remain (between the logged responses and the planned response sequence)
 

SECTION: Controlled Event content logging
5.3. Confirmation that the test was performed and that no discrepancies remain
 




SECTION: Controlled event timing logging
7.8. Compute and report the log file timing inaccuracies standard deviation ( )
  
 


SECTION: Controlled event timing logging
7.3. Compute discrete difference on the binarized photodiode signal:
  
Where n is the number of samples in the signal
SECTION: Uncontrolled events timing
8.3. Compute discrete difference on the binarized audio signal:
  
Where n is the number of samples in the signal
SECTION: Uncontrolled events timing
8.4. Detect event observed onset ( ):
  
SECTION: Visual angle check
4.1. Obtain Screen Height and Width in pixels
 

SECTION: Visual angle check
4.2. Measure screen height and width in cm
 

SECTION: Validating experimental design parameters
9. Test Experiment counter-balancing
Describe method used to test:

 

SECTION: Validating experimental design parameters
9.1. Total number of trials


 



SECTION: Validating experimental design parameters
10. Test observed stimulus duration against the expected stimulus duration

Describe the method used:

 


SECTION: Validating experimental design parameters
9.2. Number of trials per condition

 

SECTION: Validating experimental design parameters
10.1. Compute observed stimulus duration:
  

SECTION: Validating experimental design parameters
10.2. Compute and report the mean difference between observed and expected ( ):
 

 

SECTION: Validating experimental design parameters
10.3. Compute and report the standard deviation of the difference between observed and expected ( ):