DOI: 10.5061/dryad.d2547d8h2
title: Data from: Action observation responses in macaque frontal cortex

description: Neurons that are active during action execution and action observation
 (i.e., Action Observation/Execution Neurons, AOENs) are distributed across
 the brain in a network of parietal, motor, and prefrontal areas. In a
 previous study, we showed that most AOENs in ventral premotor area F5c,
 where they were discovered three decades ago, responded in a highly phasic
 way during the observation of a grasping action, did not require the
 perception of causality or a meaningful action, and even responded to
 static frames of the action videos. To assess whether these
 characteristics are shared with AOENs in other areas of the AOE network,
 we performed the first large-scale neural recordings during action
 execution and action observation in multiple frontal areas including
 dorsal premotor (PMd) area F2, primary motor (M1) cortex, ventral premotor
 area F5p, frontal eye field (FEF) and 45B. In all areas, AOENs displayed
 highly phasic responses during specific epochs of the action video and, in
 addition, strong responses to a moving object, similar to F5c. In
 addition, the population dynamics in PMv, PMd, and M1 showed a shared
 representation between action execution and action observation, with an
 overlap that was as large as the overlap between action execution and
 passive viewing of translation movements. These results pose important
 constraints on the interpretation of action observation responses in
 frontal cortical areas.
description: # Data from: Action observation responses in macaque frontal cortex This
 dataset was generated to assess the different neural responses in the
 macaque frontal cortex during the observation of actions. This study
 focused on neural responses of Action Observation/Execution neurons
 (AOENs, also known as mirror neurons), which are neurons that respond to
 both the execution and the observation of an action. To find AOENs,
 subjects performed two tasks during each recording session, and only
 neurons responding to both tasks were included in the analyses:  A) Action
 execution task: the subject performed a visually guided delayed grasping
 task in which he grasped an object after a variable delay time indicated
 by a go cue. B) Action observation task: the subject passively observed
 short action videos in which either a monkey or a human performed the
 action execution task. Additional control videos were also shown to assess
 neural action observation responses in more detail (for example, a video
 in which the hand was replaced by an ellipse that moved towards a
 graspable object, and a video in which the ellipse moved on a scrambled
 background). The study was conducted in six male macaques that were
 implanted with micro-electrode arrays in frontal brain areas. Monkey 1, 2,
 3, and 4 had multiple Utah arrays implanted in two or three of the
 following brain areas: ventral premotor (PMv) area F5c, dorsal premotor
 (PMd) area F2, or the primary motor cortex (M1). Monkey 5 and 6 were
 implanted with movable electrodes above the frontal cortex, and recordings
 in this dataset include the following brain areas: ventral premotor area
 F5p and the areas in the pre-arcuate sulcus (preAS, including the frontal
 eye field and area 45B). Overall, this study showed that AOENs in PMd and
 M1 displayed highly phasic neural responses during specific epochs of the
 action video and, in addition, strong responses to a moving object.
 Moreover, the population dynamics in PMv, PMd and M1 showed a shared
 representation between action execution and action observation, with an
 overlap that was as large as the overlap between action execution and
 passive viewing of ellipse movements. AOENs were only scarcely present in
 ventral premotor area F5p and preAS areas, suggesting a less important
 role of these brain areas in action observation. ## Description of the
 data and file structure 1. Description of dataset The Data.zip folder
 contains the data of each monkey in their respective folders (Monkey 1,
 2a, 2b, 3, 4, 5, and 6). Note that Monkey 2 has two folders (a and b)
 because he was implanted twice (once in each hemisphere) due to an implant
 failure after the first implantation. Each of the subject folders has two
 task folders that contain the data files for that specific task. The
 following .mat files correspond to the filtered data acquired during the
 study. The name of each file includes the task, the animal name, date of
 collection and hour of collection. For each .mat file, there is a
 corresponding .nex or.nex5 file that contains the neural activity of
 individual neurons. For files that were generated after 2020, there is an
 extra .mat file named *_ReplayedMUA.mat. This file contains the complete
 multi-unit activity, since only a part was recorded and stored in the
 original .mat file due to a change in the settings during recording of the
 signal. * Animal names: Monkey 1 = Isaac, Monkey 2 = Sky, Monkey 3 = Loki,
 Monkey 4 = Vino, Monkey 5 = Abu ('a' abbreviated), and Monkey 6
 = Sheldon ('shel' abbreviated). * Tasks: For each recording
 session, there is an action execution file and an action observation file
 (see .txt files labeled 'fnames' in each monkey's task
 folder to have a full list of the generated files per monkey and per
 task). There are two separate tasks which are listed here under there
 common name: a. Action execution task --&gt; 'vgrasp' b. Action
 observation task --&gt; 'mirror' or 'mirrormovies'
 General structure of the data folder: Data.zip \| - subject name \| | -
 task name \| | | - testfnames.txt (list of all files in that subfolder) \|
 | | - [Task][Subject][YYYYMMDD][HHMM][SessionID].mat (file with behavioral
 and neural data) \| | | -
 [Task]*[Subject]*[YYYYMMDD][HHMM][SessionID]sorted.nex (file with neural
 activity of individual neurons; .nex for Monkey 1, 5, and 6 and .nex5 for
 Monkey 2, 3, and 4) \| | | -
 [Task]*[Subject]*[YYYYMMDD]*[HHMM]_[SessionID]_ReplayedMUA.mat (file with
 correct multi-unit activity; only for Monkey 3 and 4) \| | | -
 [subject]_F5p.txt (list of all files in that subfolder, with for each file
 the electrodes that recorded neural activity of individual neurons in F5p;
 only for Monkey 5 and 6) \| | | - [subject]_preAS.txt (list of all files
 in that subfolder, with for each file the electrodes that recorded neural
 activity of individual neurons in preAS; only for Monkey 5 and 6) 1. File
 list A. files of the Action execution task (.mat file, corresponding
 .nex/.nex5 file, and if needed, an additional **ReplayedMua .mat file).
 The naming convention is slightly different between subjects. Monkey 1-4:
 [task][subject][YYYYMMDD][HHMM][SessionID] Monkey 5-6: [subject
 abbreviated]*[task][SessionID]*[DD_MM_YY]*[HHMM] ``` i.
 vgrasp_[Subject]_[YYYYMMDD]_[HHMM]_[SessionID].mat ``` Example:
 vgrasp_Loki_20210601_0953_A.mat (3 files of Monkey 1, 3 files of Monkey
 2a, 3 files of Monkey 2b, 6 files of Monkey 3, 3 files of Monkey 4, 17
 files of Monkey 5, and 9 files of Monkey 6) Description: These files
 contain the behavioral and neural data recorded while the subject
 performed the action execution task. The .mat file can be opened in
 MATLAB. The variables included in each .mat file are listed below. Content
 .mat files of Monkey 1-4: Each .mat file contains two variables that are
 necessary for the analysis of the data: cerebusDataA and tnsTrials. Some
 .mat files have an additional variable 'cerebusDataB' that
 contains the same structure as cerebusDataA, but for additional
 electrodes. When numbering electrodes, electrodes of cerebusDataB follow
 those of cerebusDataA. So if cerebusDataA contains 128 electrodes and
 cerebusDataB 64 electrodes, the electrodes of cerebusDataB will be
 electrode 129 until 192. All timings are in milliseconds and relative to
 the start of the task. A) cerebusDataA contains the neural data \-
 elecSpikes; timestamps of each electrode. \_0 in the name indicates that
 it contains multi-unit activity. Each timestep is the timing of a spike
 event, i.e., crossing the set threshold (in milliseconds). \- Cue; timing
 of the Go Cue (milliseconds) used to align the neural data to the task
 data. B) tnsTrials contains the task data \- Index; trial number \- Start;
 start of the trial (milliseconds) \- Stop; stop of the trial
 (milliseconds) \- Answer; answer is 1 if the animal performed the trial
 correctly. \- TargetObject; sphere that needed to be grasped [0:3] \-
 Light; moment the external light turned on and illuminated the object
 (milliseconds) \- LiftHand; moment the subject started to reach for the
 object (milliseconds) \- Target; moment the subject pulled the correct
 sphere (milliseconds) \- Cue; two values that indicate when the cue light
 went on and off (milliseconds). The second value is the Go Cue. Content
 .mat files of Monkey 5-6: Each .mat file contains one variable called
 'c.trial', which contains the behavioral (dsp_data) and the
 neural data (cerebus_data) per trial. A) cerebus_data \- spikeEvents:
 timestamps of each electrode (multi-unit activity). Each timestep is the
 timing of a spike event, i.e., crossing the set threshold (milliseconds).
 B) dsp_data \- Start; start of the trial (milliseconds) \- Stop; stop of
 the trial (milliseconds) \- Answer; answer is 1 if the animal performed
 the trial correctly. \- Condition; sphere that needed to be grasped [0:3]
 \- LiftHand; moment the subject started to reach for the object
 (milliseconds) \- PullObject; moment the subject pulled the correct sphere
 (milliseconds) \- Cue; two values that indicate when the cue light went on
 and off (milliseconds). The second value is the Go Cue. \- spikeEvents:
 timestamps of each electrode that had an offline sorted individual neuron
 (single unit activity). Each timestep is the timing of a spike event, i.e.
 crossing the set threshold (milliseconds). ``` ii.
 vgrasp_[Subject]_[YYYYMMDD]_[HHMM]_[SessionID]_ReplayedMUA.mat ```
 Example: vgrasp_Loki_20210601_0953_A_ReplayedMUA.mat (6 files of Monkey 3,
 3 files of Monkey 4) Description: For files that were generated after 2020
 (Monkey 3 and 4), there is an extra .mat file named *_ReplayedMUA.mat.
 This file contains one variable 'cerebusDataA'. Use this
 variable instead of the 'cerebusDataA' of the general .mat file
 (i) when the *_ReplayedMUA.mat file is present. The variable
 'elecSpikes' has the same structure in both .mat files. Content
 .mat file: A) cerebusDataA contains the neural data \- elecSpikes;
 timestamps of each electrode. \_0 in the name indicates that it is
 multi-unit activity. Each timestep is the timing of a spike event, i.e.
 crossing the set threshold (in milliseconds). ``` iii.
 vgrasp_[Subject]_[YYYYMMDD]_[HHMM]_[SessionID]_sorted.nex ``` Example:
 vgrasp_Loki_20210601_0953_A_cer1001_sorted.nex5 (3 files of Monkey 1, 3
 files of Monkey 2a, 3 files of Monkey 2b, 6 files of Monkey 3, 3 files of
 Monkey 4, 17 files of Monkey 5, and 9 files of Monkey 6) Description: The
 .nex and .nex5 files contain the neural data of offline sorted individual
 neurons, that were manually sorted with the 'Plexon Offline
 Sorter' software. Individual electrodes can have data from multiple
 sorted neurons. Content .nex /.nex5 file: timesteps (in milliseconds) of
 the sorted neurons per electrode. B. files of the Action observation task
 The data files of the Action observation task are similarly structured as
 the files of the Action execution task. So for each subject there will be
 a .mat file, a corresponding .nex/.nex5 file, and if needed an additional
 *_ReplayedMua mat file per recording session. Since the study focused on
 finding AOENs, each recording session includes an Action execution data
 file and an Action observation data file. Therefore, the number of files
 is identical in the two task folders per subject. ``` i.
 mirrormovies_[Subject]_[YYYYMMDD]_[HHMM]_[SessionID].mat ii.
 mirrormovies_[Subject]_[YYYYMMDD]_[HHMM]_[SessionID]_ReplayedMUA.mat iii.
 mirrormovies_[Subject]_[YYYYMMDD]_[HHMM]_[SessionID]_sorted.mat ```
 Description: These files are similarly structured as the Action execution
 task and contain the same variables that also apply to this task (Index,
 Start, Stop, Answer, elecSpikes, spikeEvents). Other variables are listed
 below. Content .mat file of Monkey 1-4: B) tnsTrials \- Stimulus; name of
 the video that is shown (POV = point-of-view of the monkey = Viewpoint 1;
 Side = filmed from the side = Viewpoint 2). Each stimulus name is paired
 with its corresponding name in the paper: Human Grasp =
 [viewpoint]_Human_Grip Human touch = [viewpoint]_Human_Fist Monkey =
 [viewpoint]_Obi_Monkey\ Static = [viewpoint]_Human_Nogo Ellipse =
 [viewpoint]_Object_Ellipse SCR background = [viewpoint]_SCR_Ellipse Static
 Interaction = [viewpoint]_Human_staticgrip Static Approach =
 [viewpoint]_Human_StaticApproach Human_Disappear videos were not used in
 this study. \- PhotoEvents; timing of photoevents (in milliseconds), 3 in
 a correct trial: (1) start of the trial, (2) start of the video, (3) end
 of the video. Content .mat file of Monkey 5-6: B) dsp_data \- Condition;
 number of the video shown (POV = point-of-view of the monkey = Viewpoint
 1; Side = filmed from the side = Viewpoint 2). Each stimulus name is
 paired with its corresponding name in the paper: Human Grasp =
 [viewpoint]_human_grip.avi Human Touch = [viewpoint]_human_fist.avi Monkey
 = [viewpoint]_obi.avi Static = [viewpoint]_nogo.avi Ellipse =
 [viewpoint]_obj_ellipsa.avi SCR background = [viewpoint]_scr_ellipse.avi
 \- PhotoEvents; timing of photoevent, i.e., the start of the trial (in
 milliseconds) C. testfnames.txt files (for each task per subject)
 Description: Each .txt file contains a list of all recording files used
 per task and per subject. Below each file name is one letter, indicating
 which recorded electrodes (also called channels or ch abbreviated) belong
 to which brain area. All data shown in the study are from either the
 primary motor cortex (M1), the dorsal premotor cortex (PMd), the ventral
 premotor area F5p, or the pre-arcuate areas (preAS). However, not all
 electrodes that are included in these data files are implanted in one of
 these areas. The recorded data may also contain data from electrodes
 implanted in the ventral premotor area F5c (PMv, see related public
 dataset: DOI: 10.5061/dryad.kwh70rzc7). The electrode numbering is the
 same in the action execution (vgrasp) and the action observation task
 (mirrormovies). Below is a list of the possible letters in the .txt files
 and the corresponding channel configurations: E pmv = [1:96]; pmd =
 [97:128]; F pmv = [1:64]; pmd = [65:128]; C pmv = [97:128]; pmd = [1:96];
 U pmv = [1:96]; m1 = [97:192]; R m1 = [1:96]; S pmd = [1:96]; Q m1 =
 [1:64,97:128]; pmd = [65:96,129:192]; pmv = [193:256]; O pmv =
 [1:64,97:128]; m1 = [65:96,129:192]; pmd = [193:256]; Note: if the letter
 is an 'o' or 'z' (in Monkey 5 and 6), the electrode
 configuration is written down in additional txt files (see D. below). D.
 other .txt files (for each task per subject; only Monkey 5 and 6) The
 naming convention is: *[Monkey]*[area].txt ``` example: _Abu_F5p.txt ```
 Description: These .txt files have a similar content as the
 'testfnames.txt' files. They list all file names in that folder,
 but below each file name are the electrodes listed that have offline
 sorted neurons (in the corresponding .nex file) in the brain area that is
 in the .txt name. For example, in 'Monkey6/Action execution
 task/_Sheldon_F5p.txt' the first few lines read:
 shelvgraspC_21_03_17_09h22 16 20 24 45 54 77 85 87 89 90 91 93 94 Thus, in
 this specific .nex file, there will be 13 offline sorted neurons (one on
 ch16, one on ch20, etc.). Each folder will have two of these .txt files:
 one for the offline sorted neurons in F5p and one for the offline sorted
 neurons in preAS. Missing data codes: None **Code/Software** MATLAB
 required for opening .mat files To open .nex or .nex5 files: Software:
 NeuroExplorer **SHARING/ACCESS INFORMATION** Links to publications that
 use the data: De Schrijver S, Decramer T, Janssen P. Action observation
 responses in macaque frontal cortex. (2026) NeuroImage: Reports.
 [https://doi.org/10.1016/j.ynirp.2026.100319](https://doi.org/10.1016/j.ynirp.2026.100319)