Miracle-dz/UniFAR-FTUs huggingface dataset:
Detection of Distraction-related Actions on DMD: An Image and a Video-based Approach Comparison

The recently presented Driver Monitoring Dataset (DMD) extends research lines for Driver Monitoring Systems. We intend to explore this dataset and apply commonly used methods for action recognition to this specific context, from image-based to video-based analysis. Specially, we aim to detect driver distraction by applying action recognition techniques to classify a list of distraction-related activities. This is now possible thanks to the DMD, that offers recordings of distracted drivers in video format. A comparison between different state-of-the-art models for image and video classification is reviewed. Also, we discuss the feasibility of implementing image-based or video-based models in a real-context driver monitoring system. Preliminary results are presented in this article as a point of reference to future work on the DMD.
I-ODA, Real-world Multi-modal Longitudinal Data for Ophthalmic Applications

Data from clinical real-world settings is characterized by variability in quality, machine-type, setting, and source. One of the primary goals of medical computer vision is to develop and validate artificial intelligence (AI) based algorithms on real-world data enabling clinical translations. However, despite the exponential growth in AI based applications in healthcare, specifically in ophthalmology, translations to clinical settings remain challenging. Limited access to adequate and diverse real-world data inhibits the development and validation of translatable algorithms. In this paper, we present a new multi-modal longitudinal ophthalmic imaging dataset, the Illinois Ophthalmic Database Atlas (I-ODA), with the goal of advancing state-of-the-art computer vision applications in ophthalmology, and improving upon the translatable capacity of AI based applications across different clinical settings. We present the infrastructure employed to collect, annotate, and anonymize images from multiple sources, demonstrating the complexity of real-world retrospective data and its limitations. I-ODA includes 12 imaging modalities with a total of 3,668,649 ophthalmic images of 33,876 individuals from the Department of Ophthalmology and Visual Sciences at the Illinois Eye and Ear Infirmary of the University of Illinois Chicago (UIC) over the course of 12 years.
SERS enhancement of gold nanospheres of defined size

Monodisperse, citrate-stabilized gold nanoparticles of sizes ranging from 15 to 40 nm were synthesized and characterized by small angle X-ray scattering and UV-vis experiments. Identical surface properties of nanoparticles of different sizes to avoid variation in the chemical surface-enhanced Raman scattering (SERS) enhancement, as well as selection of experimental conditions so that no aggregation took place, enabled the investigation of enhancement of individual nanospheres. Enhancement factors (EFs) for SERS were determined using the dye crystal violet (CV). EFs for individual gold nanospheres ranged from 102 to 103, in agreement with theoretical predictions. An increase of the EFs of individual spheres with size can be correlated to changes in the extinction spectra of nanoparticle solutions. This confirms that the increase in enhancement with increasing size results from an increase in electromagnetic enhancement. Beyond this dependence of EFs of isolated gold spheres on their size, EFs were shown to vary with analyte concentration as a result of analyte-induced aggregation. This has implications for the application of nanoparticle solutions as SERS substrates in quantitative analytical tasks. Copyright © 2011 John Wiley & Sons, Ltd.
Real-time Human Eye Resolution Ray Tracing in Mixed Reality

Mixed reality applications require natural visualizations. Ray tracing is one of the candidates for this purpose. Real-time ray tracing is slowly becoming a reality in consumer market mixed and virtual reality. This is happening due to development in display technologies and computer hardware. Some of these examples are foveated rendering enabled high resolution displays, like Varjo mixed reality headset, and parallel computing enablers, like GPUs getting ray tracing hardware acceleration enablers, such as for example Nvidia’s RTX. Currently, the challenge in ray tracing is resource need especially in mixed reality where low latency is wanted and with human eye resolution where high resolution needs are obvious. In this paper, we design and implement a novel foveated ray tracing solution called Human Eye Resolution Ray Tracer (HERR) that achieves real-time frame rates in human eye resolution in mixed reality.
BASH: Biomechanical Animated Skinned Human for Visualization of Kinematics and Muscle Activity

Biomechanical analysis of human motion is applied in medicine, sports and product design. However, visualizations of biomechanical variables are still highly abstract and technical since the body is visualized with a skeleton and muscles are represented as lines. We propose a more intuitive and realistic visualization of kinematics and muscle activity to increase accessibility for non-experts like patients, athletes, or designers. To this end, the Biomechanical Animated Skinned Human (BASH) model is created and scaled to match the anthropometry defined by a musculoskeletal model in OpenSim file format. Motion is visualized with an accurate pose transformation of the BASH model using kinematic data as input. A statistical model contributes to a natural human appearance and realistic soft tissue deformations during the animation. Finally, muscle activity is highlighted on the model surface. The visualization pipeline is easily applicable since it requires only the musculoskeletal model, kinematics and muscle activation patterns as input. We demonstrate the capabilities for straight and curved running simulated with a full-body musculoskeletal model. We conclude that our visualization could be perceived as intuitive and better accessible for non-experts than conventional skeleton and line representations. However, this has to be confirmed in future usability and perception studies.
Developing a Machine Learning Workflow to Explain Black-box Models for Alzheimer's Disease Classification

Many research articles used difficult-to-interpret black-box Machine Learning (ML) models to classify Alzheimer’s disease (AD) without examining their biological relevance. In this article, an ML workflow was developed to interpret black-box models based on Shapley values. This workflow enabled the model-agnostic visualization of complex relationships between model features and predictions and also the explanation of individual predictions, which is important in clinical practice. To demonstrate this workflow, eXtreme Gradient Boosting (XGBoost) and Random Forest (RF) classifiers were trained for AD classification. All models were trained on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) or Australian Imaging and Lifestyle flagship study of Ageing (AIBL) dataset and were validated for independent test datasets of both cohorts. The results showed improved performances for black-box models in comparison to simple Classification and Regression Trees (CARTs). For the classification of Mild Cognitive Impairment (MCI) conversion and the ADNI training dataset, the best model achieved a classification accuracy of 71.03 % for the ADNI test dataset and 67.65 % for the entire AIBL dataset. This RF used a logical long-term memory test, the count of Apolipoprotein E ε4 (ApoEε4) alleles and the volume of the left hippocampus as the most important features.
Convolutional Neural Networks with Fixed Weights

Improved computational power has enabled artificial neural networks to achieve great success through deep learning. However, visual classification is brittle; networks can be easily confused when a small amount of noise is added to an image. This position paper raises the hypothesis that using all the pixels of an image is wasteful of resources and unstable. Biological neural networks achieve greater success, and the outline of their architecture is well understood and reviewed in this paper. It would behove deep learning network architectures to take additional inspiration from biology to reduce the dimensionality of images and video. Pixels strike the retina, but are convolved before they get to the brain. It has been demonstrated that a set of five filters retains key visual information while achieving compression by an order of magnitude. This paper presents those filters. We propose that images should be pre-processed with a fixed weight convolution that mimics the filtering performed in the retina and primary visual cortex. Deep learning would then be applied to the smaller filtered image.
Using Geometric Graph Matching in Image Registration

Image registration is a fundamental task in many medical applications, allowing interpreting and analyzing images acquired using different technologies, from different viewpoints, or at different times. The image registration task is particularly challenging when the images have little high-frequency information and when average brightness changes over time, as is the case with infrared breast exams acquired using a dynamic protocol. This paper presents a new method for registering these images, where each one is represented in a compact form by a geometric graph, and the registration is done by comparing graphs. The application of the proposed technique consists of five stages: (i) pre-process the infrared breast image; (ii) extract the internal linear structures that characterize arteries, vascular structures, and other hot regions; (iii) create a geometric graph to represent such structures; (iv) perform structure registration by comparing graphs; and (v) estimate the transformation function. The Dice coefficient, Jaccard index, and total overlap agreement measure are considered to evaluate the results’ quality. The output obtained on a public database of infrared breast images is compared against SURF interest points for image registration and a state of the art approach for infrared breast image registration from the literature. The analyzes show that the proposed method outperforms others.
Machine Learning Studies of Non-coding RNAs based on Artificially Constructed Training Data

Machine learning (ML) methods are often used to identify members of non-coding RNA classes such as microRNAs or snoRNAs. However, ML methods have not been successfully used for homology search tasks. A systematic evaluation of ML in homology search requires large, controlled, and known ground truth test sets, and thus, methods to construct large realistic artificial data sets. Here we describe a method for producing sets of arbitrarily large and diverse snoRNA sequences based on artificial evolution. These are then used to evaluate supervised ML methods (Support Vector Machine, Artificial Neural Network, and Random Forest) for snoRNA detection in a chordate genome. Our results indicate that ML approaches can indeed be competitive also for homology search.
A New Generic Progressive Approach based on Spectral Difference for Single-sensor Multispectral Imaging System

Single-sensor RGB cameras use a color filter array to capture the initial image and demosaicking technique to reproduce a full-color image. A similar concept can be extended from the color filter array (CFA) to a multispectral filter array (MSFA). It allows us to capture a multispectral image using a single-sensor at a low cost using MSFA demosaicking. The binary tree based MSFAs can be designed for any k-band multispectral images and are preferred, however the existing demosaicking methods are either not generic or are of limited efficacy. In this paper, we propose a new generic demosaicking method applicable on any k-band MSFAs, designed using preferred binary-tree based approach. The proposed method involves applying the bilinear interpolation and estimating the spectral correlation differences appropriately and progressively. Experimental results on two different multispectral image datasets consistently show that our method outperforms the existing state-of-art methods, both visually and quantitatively, as per the different metrics.
Convolution Filter based Efficient Multispectral Image Demosaicking for Compact MSFAs

Using the multispectral filter arrays (MSFA) and demosaicking, the low-cost multispectral imaging systems can be developed that are useful in many applications. However, multispectral image demosaicking is a challenging task because of the very sparse sampling of each spectral band present in the MSFA. The selection of MSFA is very crucial for the applicability and for the better performance of demosaicking methods. Here, we consider widely accepted and preferred MSFAs that are compact and designed using binary tree based approach and for these compact MSFAs, we propose a new efficient demosaicking method that relies on performing filtering operations and can be used for different bands size multispectral images. We also present new filters for demosaicking based on the probability of appearance of spectral bands in binary-tree based MSFAs. Detailed experiments are performed on multispectral images of two different benchmark datasets. Experimental results reveal that the proposed method has wider applicability and is efficient; it consistently outperforms the existing state-of-the-art generic multispectral image demosaicking methods in terms of different image quality metrics considered.
Disentangled Rendering Loss for Supervised Material Property Recovery

In order to replicate the behavior of real world material using computer graphics, accurate material property maps must be predicted which are used in a pixel-wise multi-variable rendering function. Recent deep learning techniques use the rendered image to obtain the loss on the material property map predictions. While use of rendering loss defined this way results in some improvements in the quality of the predicted renderings, it has problems in recovering the individual property maps accurately. These inaccuracies arise due to the following: i) different property values can collectively generate the same image for limited light and view directions, ii) even correctly predicted property maps get changed because the loss backpropagates gradients to all, and iii) the heuristic chosen for number of light and view samples affects accuracy and computation time. We propose a new loss function, named disentangled rendering loss which addresses the above issues: each predicted property map is used with ground truth maps instead of the other predicted maps, and we solve for the integral of the L1 loss of the specular term over different light and view directions, thus avoiding the need for multiple light and view samples. We show that using our disentangled rendering loss to train the current state of the art network leads to a noticeable increase in the accuracy of recovered material property maps.
Boosting Self-localization with Graph Convolutional Neural Networks

Scene graph representation has recently merited attention for being flexible and descriptive where visual robot self-localization is concerned. In a typical self-localization application, the objects, object features and object relationships of the environment map are projected as nodes, node features and edges, respectively, on to the scene graph and subsequently mapped to a query scene graph using a graph matching engine. However, the computational, storage, and communication overhead costs of such a system are directly proportional to the number of feature dimensionalities of the graph nodes, often significant in large-scale applications. In this study, we demonstrate the feasibility of a graph convolutional neural network (GCN) to train and predict alongside a graph matching engine. However, visual features do not often translate well into graph features in modern graph convolution models, thereby affecting their performance. Therefore, we developed a novel knowledge transfer framework that introduces an arbitrary self-localization model as the teacher to train the GCN-based self-localization system i.e., the student. The framework, additionally, facilitated lightweight storage and communication by formulating the compact output signals from the teacher model as training data. Results on the Oxford RobotCar datasets reveal that the proposed method outperforms existing comparative methods and teacher self-localization systems.
EEG Classification for Visual Brain Decoding via Metric Learning

In this work, we propose CNN based approaches for EEG classification which is acquired from a visual perception task involving different classes of images. Our approaches involve deep learning architectures using 1D CNN (on time axis) followed by 1D CNN (on channel axis) and Siamese network (for metric learning) which are novel in this domain. The proposed approaches outperform the state-of-the-art methods on the same dataset. Finally, we also suggest a method to select fewer number of EEG channels.
Asset Detection in Railroad Environments using Deep Learning-based Scanline Analysis

This work presents an approach for the automated detection of railroad assets in 3D point clouds from mobile mapping LiDAR scans using established convolutional neural networks for image analysis. It describes how images of individual scan lines can be generated from 3D point clouds. In these scan lines, objects such as tracks, signal posts, and axle counters can be detected using artificial neural networks for image analysis, previously trained on ground-truth data. The recognition results can then be transferred back to the 3D point cloud as a semantic classification result, or they are used to generate geometry or map data for further processing in GIS applications. Using this approach, trained objects can be found with high automation. Challenges such as varying point density, different data characteristics of scanning devices, and the massive amount of data can be overcome with this approach.
A Random Walker Can Optimize the Exploration without the Large Capacity Memory

A random walker explores an unknown field and sometimes changes its movement property using new spatial information obtained by it during its exploration. An important matter is the relation between the movement property of a random walker and the use for acquired information. I recently developed a random walk model in which a walker coordinated its directional rule based on its experiences and found that this model presented an optimal random walk, which demonstrated a so-called Lévy walk with μ = 2.00. Here, I investigate the foraging efficiency for that model and verify whether a large memory capacity is required or not in order to maintain the foraging efficiency. My findings reveal that the proposed model can apply to biological processes where a random walker does not have a high memory capacity.
Cross-phase Emotion Recognition using Multiple Source Domain Adaptation

EEG signal, the brain wave, has been widely applied in detecting human emotion. Due to the human brain’s complexity, the EEG pattern varies from different individuals, leading to low cross-subject classification performance. What is more, even within the same subject, EEG data also shows diversity for the same reason. Many researchers have conducted experiments to deal with the variance between subjects by transfer learning or domain adaptation. However, most of them are still low-performance, especially when the new subject does not share generality with training samples. In this study, we examined using cross-phase data instead of cross-subject data because the discrepancy of different phase data should be smaller than that of different subjects. Different phases represent data recorded multiple times from the same subject with the same stimuli. Two neural networks are adopted to verify the effectiveness of the cross-phase domain adaptation. As a result, experiments on the public EEG dataset showed approximation level accuracy compared to the state-of-the-art method but much lower standard derivation. Moreover, multiple source domains promote accuracy in contrast to one single domain. This study helps develop a more robust and high-performance real-time EEG system by transferring knowledge from previous data phases.
A Survey of Possibilities and Challenges with AR/VR/MR and Gamification Usage in Healthcare

Software and applications of augmented reality (AR), virtual reality (VR), and mixed reality (MR) technology combined with game/gamification techniques in healthcare have increasingly been studied in academia. However, there is a need to explore the usage, challenges and opportunities of AR/VR/MR game/gamification software/applications in the healthcare system. To explore this, we present an online survey conducted in the healthcare-relevant system (including hospital-based system, homecare-based system, institute and university, and industry). Based on the answers, we found examples of digital games and AR/VR/MR applications used in healthcare, as well as some general information (name and feature, purpose, target user, and use occasion), usage situation, and user experience. This presented survey is beneficial for both researchers and developers in computer science and medical science. It can familiarise them with existing products and their current use, advantages and potential issues of AR/VR/MR and game applications in healthcare. In future work, the survey would be extended to obtain other user experiences and feedback of AR/VR/MR techniques and game/gamification technology applied to healthcare, as well as to study how to overcome the challenges, and develop the opportunities further.
High-resolution Controllable Prostatic Histology Synthesis using StyleGAN

For use of deep learning algorithms in clinical practice, detailed justification for diagnosis is necessary. Convolutional Neural Networks (CNNs) have been demonstrated to classify prostatic histology using the same diagnostic signals as pathologists. Using the StyleGAN series of networks, we demonstrate that recent advances in high-resolution image synthesis with Generative Adversarial Networks (GANs) can be applied to prostatic histology. The trained network can produce novel histology samples indistinguishable from real histology at 1024x1024 resolution and can learn disentangled representations of histologic semantics that separates at a variety of scales. Through blending of the latent representations, users have the ability to control the projection of histologic semantics onto a reconstructed image. When applied to the medical domain without modification, StyleGAN2 is able to achieve a Fréchet Inception Distance (FID) of 3.69 and perceptual path length (PPL) of 33.25.