Reviewerly/exHarmony huggingface dataset:
Gaze-dependent brain activity during narrative perception and recall

Perception and recall recruit overlapping neural systems in the human brain. They are further linked on a behavioral level through gaze reinstatement, the recapitulation of encoding related gaze patterns during recall. Here, we ask whether eye movements during recall reflect the event structure of continuous narratives, and whether the aforementioned neural overlap is grounded in shared gaze patterns. We tested this by combining eye tracking and deep learning-based gaze predictions with fMRI data acquired while participants watched and recalled a movie. First, we tested whether frame-wise saliency maps of the movie predict eye movements during recall of the corresponding movie scenes. Second, by reconstructing gaze behavior from the MR-signal of the eyes and relating it to brain activity, we show that eye movements do indeed drive wide-spread brain activity in both domains. In a series of follow-up analyses, we then characterized the relationship between event-specific eye movements, the multi-voxel pattern of the eyes and brain activity in detail. Our preliminary results suggest that both perception and recall of continuous narratives recruit our oculomotor systems, which may help reassemble our perceptual experiences for recollection and mental imagery.
Does working memory decay constrain gaze location while walking and bicycling?

During locomotion in natural terrains, humans look closer to the body in more complex terrain presumably due to the need for more precise visual information about the 3D structure of the terrain in order to guide footholds (Bonnen et al., 2021). They also slow down as the terrain becomes more complex and gaze is found to fall on a location approximately 1.6 sec ahead, over a range of walking speeds from 1.4 to 0.9 m/s (Matthis et al., 2018). A similar look-ahead time has been observed in driving (Land & Lee, 1994) and other contexts. It is possible that this reflects the time constant of working memory decay that modulates the precision of the information needed to plan future steps. To explore the generality of this finding, we collected gaze data using a Pupil Labs Invisible eye tracker while subjects rode a bicycle around campus. As with walking, subjects alternated between stable gaze locations (fixations) on the ground plane and fixations to more distant locations. The ground plane was reconstructed using photogrammetry. Fixations were identified as periods separated by saccades, where the velocity of the eye-in-orbit was less than 100 deg/sec, as the eye counter-rotates in the orbit during forward motion. Fixations ranged from 1.9 to 9.2 meters ahead, with speeds varying between 0 and 7.9 m/s. Preliminary results suggest that subjects have a constant look-ahead time with a mean of 1.1 +/- 0.02 seconds. This is slightly shorter than the look ahead time while walking or driving, but still consistent with the idea that the quality of the sensory information may be limited by working memory decay. Other factors such as spatial resolution at these distances, and the momentum of the bicycle may also need to be considered.
The role of texture summary-statistics in material recognition from drawings and photographs

Material depictions in artwork are useful tools for revealing image features that support material categorization. For example, artistic ‘recipes’ for drawing specific materials make explicit the critical information leading to recognizable material properties (Di Cicco et al., 2020) and investigating the recognizability of material renderings as a function of their visual features supports conclusions about the vocabulary of material perception. Presently, we compared material categorization abilities between photographic stimuli (Sharan et al., 2014) and line drawings (Saito et al., 2015) in their original format and after texture synthesis. Specifically, our participants (N=52) completed a 4AFC material categorization task in which stimulus appearance was manipulated across participant groups via the Portilla-Simoncelli texture synthesis model. This manipulation allowed us to examine how categorization may be affected differently across materials and image formats when only summary-statistic information about appearance was retained. Our accuracy results revealed a three-way interaction (F(3,150)=4.80, p=0.003) between image format (photographs/drawings), material category (metal, stone, water, or wood) and appearance (original/texture-synthesized, driven by differential advantages for photographic vs. line drawing stimuli as a function of both material category and appearance. While line drawings supported better recognition of metal and wood across all stimulus manipulations, photographic water and stone were better recognized than drawings in original and synthetic versions, respectively. Do these patterns emerge purely from the image data? A linear SVM classifier assessed discriminability of the four materials in the eight layers of a deep convolutional neural network (AlexNet). Classification accuracy increased across layers, and photographs out-performed drawings. Category confusion rates were more correlated across humans and the classifier for photographs as well. Together, these results demonstrate that line drawings can make materials more recognizable to humans than photographs in some cases, perhaps by isolating critical image features used by human vision that are not captured by pre-trained dCNNs.
A Beta-Variational Auto-Encoder Model of Human Visual Representation Formation in Utility-Based Learning

The human brain is capable of forming informationally constrained representations of complex visual stimuli in order to achieve its behavioural goals, such as utility-based learning. Recently, methods borrowed from machine learning have demonstrated a close connection between the mechanisms of visual representation formation in primate brains with the latent representations formed by Beta-Variational Auto-Encoders (Beta-VAEs). While auto-encoder models capture some aspects of visual representations, they fail to explain how visual representations are adapted in a task-directed manner. We developed a model of visual representation formation in learning environments based on a modified Beta-VAE model that simultaneously learns the task-specific utility of visual information. We hypothesized that humans update their visual representations as they learn which visual features are associated with utility in learning tasks. To test this hypothesis, we applied the proposed model onto the data from a visual contextual bandit learning task [Niv et al. 2015; J. Neuroscience]. The experiment involved humans (N=22) learning the utility associated with 9 possible visual features (3 colors, shapes or textures). Critically, our model takes in as input the same visual information that is presented to participants, instead of the hand-crafted features typically used to model human learning. A comparison of predictive accuracy between our proposed model and models using hand-crafted features demonstrated a similar correlation to human learning. These results show that representations formed by our Beta-VAE based model can predict human learning from complex visual information. Additionally, our proposed model makes predictions of how visual representations adapt during human learning in a utility-based task. Further, we performed a comparison of our proposed model across a range of parameters such as information-constraint, utility-weight, and number of training steps between predictions. Results from this comparison give insight into how the human brain adjusts its visual representation formation during learning.
IAG Membership Information

Geostandards and Geoanalytical ResearchVolume 46, Issue 4 p. 871-871 Membership Information IAG Membership Information First published: 15 December 2022 https://doi.org/10.1111/ggr.12393AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat No abstract is available for this article. Volume46, Issue4December 2022Pages 871-871 RelatedInformation
A Conceptual Approach to an AI-Supported Adaptive Study System for Individualized Higher Education Services

In the context of the digital transformation, the targeted implementation of AI-based or AI-supported technologies in “teaching & learning,” as well as “administration & service,” holds considerable potential for organizational change and quality enhancement for higher education institutions. The use of AI in higher education teaching and services lags behind the level in research. Therefore, holistic solutions must be planned and implemented in unity of teaching and research for the AI-based support of the stakeholders’ inclusive administration, the further development or the establishment of new digital study programs and offers, as well as the prospective qualification of university staff in the field of AI. The applications that currently exist do not generally fit into a holistic concept. Therefore, they must be analyzed, systematized, and structured to generate a conceptual approach via an integrated architecture with adaptive services. A cross-university, transdisciplinary and modular system approach is being pursued to transfer AI methods as supporting technologies into the regular operation of teaching and associated services at the university in an interdisciplinary project over several years. From study orientation to the use of new study programs, the procedures at different process and structural levels of the university are made more flexible by means of AI and transferred into adaptive services for the individualization of learning and teaching. The paper explains the conceptual approach for the further development of the AI-based education system at a university and presents the first results of planning, developments, and applications. Selected case studies related to the characteristics of paths for study orientation and study specialization, as well as for special learning profiles serve as illustrations.
Big Book Media on Beginning Reading Skills of Grade 2 Elementary School Students

This study aims to determine the effect of Big Book media on the early reading skills of second grade students of SD Hafidz Al Qur'an Porong. This research uses quantitative research. The method used is a pre-experimental design method with a research design of One Group Pretest -Posttest . The sample in this study was the second grade students of SD Hafidz Al Qur'an Porong, totaling 18 students using a saturated sample. The data collection in this study used a test (Pretest and Posttest ) on initial reading skills. The results showed that there was an effect of Big Book media on the reading skills of second grade students of SD Hafidz Al Qur'an. This is evidenced by the results of the calculation of the hypothesis test with a level of a = 0.05 or 5% meaning 0.000 &lt;0.05 then Ho is rejected and Ha is accepted. So that learning using Big Book media will show differences, it can be shown that there is an effect of Big Book media on early reading skills of second grade students of SD Hafidz Al Qur'an Porong. The magnitude of the effect in this study was calculated using the eta squared test and obtained the results of 1.04 &gt; 0.14 which indicated the criteria for a large influence. So it can be concluded that the Big Book media has a big influence on the reading skills of elementary school students in grade II
Model Teams Games Tournament (TGT) Aided by Audio Visual Media on Learning Activeness of Elementary School Students

This research is motivated by a conventional science learning model that is teacher-centered, so that the level of student learning activity is still low, learning activity is influenced by teachers who can stimulate students to be more active in learning, discussing, expressing opinions, curiosity, seeking information. new (Zaeni 2017, Hidayah 2017). This study aims to determine the effect and how much influence the TGT learning model assisted by audio-visual media has on science learning activities at SDN Kepunten Tulangan for the 2021/2022 academic year. This study uses a quantitative research method of pre-experimental design, one-shot case study which is given posttest treatment, to determine the effect and magnitude of the influence the researcher uses the t-test one sample t- test to determine the effect of the TGT learning model assisted by audio-visual media on student learning activities. which obtained a significant value (2 – tailed) of 0.023 &lt;0.05, and a tcount of 4.333 &gt; ttable 2.04523, so it can be concluded that there is a positive and significant effect of the TGT learning model assisted by audio-visual media on student learning activities at SDN Kepunten Tulangan. In the F and R square tests to find out how much influence the TGT learning model has, the significant value is 0.00 &lt; 0.05 and the R Square value is 0.929 or equal to 92%, which means that the TGT learning model assisted by audio-visual media has a major influence, namely 92% on student learning activity. So in conclusion, the TGT learning model assisted by audio-visual media on science learning activities for class V at SDN Kepunte has an effect of 92%.
The Door Activity Method on Writing Skills for Grade 4 Students at Madrasah Ibtidaiyah

The research aims to determine the effect of relay writing method on student out door activity of description in the 4th grade at MI Al-Masyithoh Surabaya. The research used pre­-experimental design one group pre-test post-test. The population of the research was the student of 4th grade at MI Al-Masyithoh Surabaya which consist of 21 students. The sampling technique used in this research was saturated sampling. The sample of this research was the whole students in the 4th grade at MI Al-Masyithoh Surabaya. The instruments of this research were pre-test post-test. From the research results showed that the lowest score of pre-test was 45 and the highest was 80, and the mean score was 65,23%. While the lowest score of post-test was 70 and the highest was 95 and the mean score was 82,85%. It can be concluded that there is an effect of out door activity method on students description in the 4th grade at MI Al-Masyithoh Surabaya.
Can a saliency model using feature sets derived cityscapes predict cultural differences in visual search asymmetry?

Visual search asymmetries in line length (long or short), line orientation (tilted or vertical), and circle with line versus circle vary in presence and magnitude depending on where they are measured (e.g., Canada, Japan, and the United States, see Ueda et al., 2018). These results indicate that our visual cognition is affected by the environments around us. What induces these differences? Several previous studies suggest that it may due to the cityscape and orthographies of each location. It has been shown that visual attention and eye movements change after viewing Japanese and American cityscapes for a certain amount of time (Miyamoto et al. 2006; Ueda & Komiya, 2012). A study dealing directly with search asymmetry showed that visual saliency calculated using the Attention based on Information Maximization (AIM; Bruce & Tsotsos, 2009) model, which derives a set of visual features from orthographic characters (alphabets, Japanese hiragana, and Kanji characters) produced different saliency levels in line length asymmetry (Saiki, 2020). In this study, we used the AIM model to investigate whether differences in the scenery of different locations can produce different visual saliency in the search asymmetry. In the original AIM model, visual feature sets were derived from various scene images, while instead of this, we collected more than 10,000 photos of cities across Japan, and used only them to derive visual feature sets with independent component analysis (ICA). Comparing visual saliency calculated by visual features derived from the original image set with the image set of Japanese cities, consistent results across image sets were obtained for search asymmetry. These results suggest that the AIM model with feature sets derived from scenery can robustly predict visual search asymmetry, and cultural differences in search asymmetry may emerge not from the cityscapes in each culture, but rather from long-term experiences with orthographic letters.
Cognitive Differences in Comprehending Data Visualizations with Seductive Details

This study employed eye-tracking to examine the comprehension of data visualizations (DVs) with and without seductive details (Harp & Mayer, 1998) and representing either relative or absolute data. DVs serve to visualize numerical data and use visual dimensional objects. Given advancements in computer graphics, DVs are increasingly designed using (a) seductive details, and (b) relative data (e.g., data on monthly changes of COVID-19 cases). The purpose of this study was to explore how differences in undergraduates’ verbal and visuospatial working memory capacity (WMC) influence their processing of DVs with and without seductive details, and representing either relative or absolute data. The study was conducted with 70 undergraduates at a public US university. Each participant examined six COVID-19 DVs: (a) two with related seductive details, two with non-related seductive details, and two plain (no seductive details); as well as (b) three DVs representing absolute and three DVs with relative data representations. DV processing was assessed using the proportion of eye-gaze frequency data on DVs’ three regions of interest (ROIs)- instructions, visualizations, and assessment items. Additionally, participants completed ACCES test of reading span (rspan), visuospatial span (sspan), and operations span (ospan). Multiple regression analyses were performed with participants’ rspan, sspan, and ospan as predictors of average proportions of eye-gaze frequency on three ROIs for various conditions. The findings reveal that: (a) “related” details supported DV processing, whereas “non-related” details hindered it for people with higher operations and low visuospatial span scores, (b) all participants had difficulty with processing DVs with relative data regardless of their span scores, (c) participants with higher reading span needed significantly more time for examining visualizations compared to verbal information. Further studies will include analyses of participants’ fixations and saccades between ROIs. These findings motivate designing personalized DV environments sensitive to the differences in users’ WMC.
The relationship between gaze and foot placement is shaped by the visual discriminability and availability of footholds in an overground Augmented Reality stepping stone task

Successful human locomotion over complex terrain requires precise coupling between gaze and foot placement. Matthis, Yates, & Hayhoe (2018) found that walkers adapted distinct gaze strategies that were tuned to the demands imparted by locomotion over different terrains (e.g., flat ground vs. a rocky creek bed). However, because these were quasi-observational experiments in a natural environment, it is difficult to discern the specific aspects of the terrain that were driving these different strategies. Here, we used an Augmented Reality (UniCAVE) projected ground plane (~2mx10m) to parametrically control the visual discriminability and availability of footholds in an overground stepping stone task. Using a binocular eye tracker (Pupil Labs) and a spatiotemporally synchronized, marker-based, full body motion capture system (Qualisys) we estimate participants 3d gaze and calculate gaze-ground intersection as they traverse the 10m path, comprising pseudo-randomly distributed footholds (represented by Landolt C’s; diameter 110mm, line thickness 32mm) and distractors (represented by visually similar O’s). We manipulated visual discriminability and availability of footholds to discern the aspects of terrain that drive the emergence of different gaze-foothold strategies. Participants walked over six terrains, consisting of two levels of visual discriminability (controlled by manipulating the size in the C gap; 6mm vs. 36mm) crossed with three levels of foothold/distractor ratio (3:1, 1:1, and 1:3). With 21 trials for each of the 7 conditions (six terrains plus a free walking control condition) we record 147 trials (approximately 1960 steps) per participant. We mapped the gaze-foothold relationships from the six terrain conditions to those observed in natural environments (Matthis, Yates, & Hayhoe, 2018) to specify how the visual discriminability and availability of footholds result in different gaze strategies. We further explored whether saccades are driven by the biomechanically specified and energetically optimal footholds vs. the likelihood of finding potential footholds based on peripheral vision.
Audiovisual Semantic Relatedness of Real-World Objects

Semantic information is an important feature used to guide attention in real world environments. However, research into semantic guidance of attention has been limited by the difficulty of quantifying semantic relatedness, particularly between stimuli in different sensory modalities. To address this, we first created a constrained audiovisual stimulus set to derive similarity ratings between each item within three categories (animals, instruments, household items). A set of 140 participants judged which of two images was more similar to a given sound, and which of two sounds was more similar to a given image. These judgments were used to derive semantic relatedness values for pairs of images and sounds. Using these semantic relatedness values, in a separate experiment, we have been able to probe semantic guidance of attention in audiovisual contexts in a more nuanced continuous manner than previously possible. Current work moves beyond evaluating semantic guidance of attention in a categorical manner (either semantically related or not), and allows us to test it in a continuous manner using the semantic relatedness values derived from the similarity task. The derived quantified judgments have been made available in a database form to the wide research community to be used as a measure of semantic relatedness in cognitive psychology experiments, enabling more robust studies of semantic influences in audiovisual environments.