Miracle-dz/UniFAR-FTUs huggingface dataset:
Client-wise Modality Selection for Balanced Multi-modal Federated Learning

Selecting proper clients to participate in the iterative federated learning (FL) rounds is critical to effectively harness a broad range of distributed datasets. Existing client selection methods simply consider the variability among FL clients with uni-modal data, however, have yet to consider clients with multi-modalities. We reveal that traditional client selection scheme in MFL may suffer from a severe modality-level bias, which impedes the collaborative exploitation of multi-modal data, leading to insufficient local data exploration and global aggregation. To tackle this challenge, we propose a Client-wise Modality Selection scheme for MFL (CMSFed) that can comprehensively utilize information from each modality via avoiding such client selection bias caused by modality imbalance. Specifically, in each MFL round, the local data from different modalities are selectively employed to participate in local training and aggregation to mitigate potential modality imbalance of the global model. To approximate the fully aggregated model update in a balanced way, we introduce a novel local training loss function to enhance the weak modality and align the divergent feature spaces caused by inconsistent modality adoption strategies for different clients simultaneously. Then, a modality-level gradient decoupling method is designed to derive respective submodular functions to maintain the gradient diversity during the selection progress and balance MFL according to local modality imbalance in each iteration. Our extensive experiments showcase the superiority of CMSFed over baselines and its effectiveness in multi-modal data exploitation.
Unsafe Probabilities and Risk Contours for Stochastic Processes using Convex Optimization

This paper proposes an algorithm to calculate the maximal probability of unsafety with respect to trajectories of a stochastic process and a hazard set. The unsafe probability estimation problem is cast as a primal-dual pair of infinite-dimensional linear programs in occupation measures and continuous functions. This convex relaxation is nonconservative (to the true probability of unsafety) under compactness and regularity conditions in dynamics. The continuous-function linear program is linked to existing probability-certifying barrier certificates of safety. Risk contours for initial conditions of the stochastic process may be generated by suitably modifying the objective of the continuous-function program, forming an interpretable and visual representation of stochastic safety for test initial conditions. All infinite-dimensional linear programs are truncated to finite dimension by the Moment-Sum-of-Squares hierarchy of semidefinite programs. Unsafe-probability estimation and risk contours are generated for example stochastic processes.
A Large-Scale Re-identification Analysis in Sporting Scenarios: the Betrayal of Reaching a Critical Point

Re-identifying participants in ultra-distance running competitions can be daunting due to the extensive distances and constantly changing terrain. To overcome these challenges, computer vision techniques have been developed to analyze runners’ faces, numbers on their bibs, and clothing. However, our study presents a novel gait-based approach for runners’ re-identification (re-ID) by leveraging various pre-trained human action recognition (HAR) models and loss functions. Our results show that this approach provides promising results for re-identifying runners in ultra-distance competitions. Furthermore, we investigate the significance of distinct human body movements when athletes are approaching their endurance limits and their potential impact on re-ID accuracy. Our study examines how the recognition of a runner’s gait is affected by a competition’s critical point (CP), defined as a moment of severe fatigue and the point where the finish line comes into view, just a few kilometers away from this location. We aim to determine how this CP can improve the accuracy of athlete re-ID. Our experimental results demonstrate that gait recognition can be significantly enhanced (up to a 9% increase in mAP) as athletes approach this point. This highlights the potential of utilizing gait recognition in real-world scenarios, such as ultra-distance competitions or long-duration surveillance tasks.
Prototype-Based Approach for One-Shot Segmentation of Brain Tumors using Few-Shot Learning

The potential for augmenting the segmentation of brain tumors through the use of few-shot learning is vast. Although several deep learning networks (DNNs) demonstrate promising results in terms of segmentation, they require a substantial quantity of training data in order to produce suitable outcomes. Furthermore, a major issue faced by most of these models is their ability to perform well when faced with unseen classes. To address these challenges, we propose a one-shot learning model for segmenting brain tumors in magnetic resonance images (MRI) of the brain, based on a single prototype similarity score. Leveraging the recently developed techniques of few-shot learning, which involve the utilization of support and query sets of images for training and testing purposes, we strive to obtain a definitive tumor region by focusing on slices that contain foreground classes. This approach differs from other recent DNNs that utilize the entire set of images. The training process for this model is carried out iteratively, with each iteration involving the selection of random slices that contain foreground classes from randomly sampled data as the query set, along with a different random slice from the same sample as the support set. In order to distinguish the query images from the class prototypes, we employ a metric learning-based approach that relies on non-parametric thresholds. We employ the multimodal Brain Tumor Image Segmentation (BraTS) 2021 dataset, which comprises 60 training images and 350 testing images. The effectiveness of the model is assessed using the mean dice score and mean Intersection over Union (IoU) score.
Distributional Reinforcement Learning-based Energy Arbitrage Strategies in Imbalance Settlement Mechanism

Growth in the penetration of renewable energy sources makes supply more uncertain and leads to an increase in the system imbalance. This trend, together with the single imbalance pricing, opens an opportunity for balance responsible parties (BRPs) to perform energy arbitrage in the imbalance settlement mechanism. To this end, we propose a battery control framework based on distributional reinforcement learning (DRL). Our proposed control framework takes a risk-sensitive perspective, allowing BRPs to adjust their risk preferences: we aim to optimize a weighted sum of the arbitrage profit and a risk measure while constraining the daily number of cycles for the battery. We assess the performance of our proposed control framework using the Belgian imbalance prices of 2022 and compare two state-of-the-art RL methods, deep Q learning and soft actor-critic. Results reveal that the distributional soft actor-critic method can outperform other methods. Moreover, we note that our fully risk-averse agent appropriately learns to hedge against the risk related to the unknown imbalance price by (dis)charging the battery only when the agent is more certain about the price.
Canada's Lobbying Industry: Business and Public Interest Advocacy from Harper to Trudeau

Abstract This article provides an empirical overview of federal lobbying in Canada, examining lobbying contacts by field and sector from 2011 to 2022. We track shifts in lobbying representation over this period, including across Harper Conservative and Trudeau Liberal administrations. The study reveals the dominance of business interests in lobbying in Canada and a high level of lobbying concentration. By sector, export-oriented industries with high environmental and climatic impacts—namely, agriculture, fossil fuel and manufacturing industries—predominate. With the transition to Trudeau, we find a significant increase in overall rates of lobbying and a modest increase in the ratio of public interest representation. Overall, the lobbying industry is characterized by greater access but unequal voice.
How are APOE4, changes in body weight, and longevity related? Insights from a causal mediation analysis

The {varepsilon}4 allele of the APOE gene (APOE4) is known for its negative association with human longevity, however, the mechanism is unclear. APOE4 was also linked to changes in body weight, and the latter changes were associated with survival in some studies. Here we explore the role of aging changes in weight in the connection between APOE4 and longevity, using a Causal Mediation Analysis (CMA) approach to uncovering mechanisms of genetic associations. Using the Health and Retirement Study (HRS) data, we tested a hypothesis, whether the association of APOE4 with reduced survival to age 85+ is mediated by key characteristics of age-trajectories of weight, such as age at reaching peak values, and slope of the decline in weight afterwards. Mediation effects were evaluated by the Total Effect (TE), Natural Indirect Effect, and Proportion Mediated. Controlled Direct Effect and Natural Direct Effect are also reported. The CMA results suggest that APOE4 carriers have 19%-22% (TE p=0.019-0.038) lower chances of surviving to age 85 and beyond in part because they reach peak values of weight at younger ages, and their weight declines faster afterwards, compared to non-carriers. This finding is in line with the idea that detrimental effect of APOE4 on longevity is in part related to accelerated physical aging of {varepsilon}4 carriers.
Robodoc: a conversational-AI based app for medical conversations

Artificial Intelligence (AI) evolved in trends. Currently, the trend is Conversational Artificial Intelligence (CAI). Those models of AI are focused on text-related tasks, and their commonest applications are chatbots. On this paper, we explore a smart chatbot using the Large Language Models (LLMs) from openAI. I have used a tool called Teachable Machine (TM) from Google to apply transfer learning and create image-based models. I have built two image-based model: for X-ray and for OCT. The model of X-ray is able to detect viral and bacterial pneumonia, whereas the Optical coherence tomography (OCT) model can detect Drusen, Choroidal Neovascularization (CNV) and (Diabetic Macular Edema DME) conditions on the patient's eyes image. I have also used TensorFlow.js from Google to create a diabetes detection model. All those models are integrated into a chatbot, that according to the message entered by a user, is able to use the models intelligently. Our results show a good integration between the models and the chatbot, with slight deviations from the expected behaviors. For the OCT model, we have also tested a stub function for medical appointments done by the bot, based on how serious is the patient condition. The future of artificial intelligence are public APIs, as I have shown that a complex model can be built, without a complex research infrastructure, and with low costs. Bioinformatics may have gained a new supporter towards more friendly interfaces on bioinformatics.
Automatic Dataset Creation from User-generated Recipes for Ingredient-centric Food Image Analysis

We aim to develop an application that automatically creates a nutrition facts label from food images for precise dietary control. Firstly, we constructed a new dataset with food category labels and a list of ingredients in a nutritionally calculable format using an image classification model and BERT for 1.6 million recipes accompanied by images. The nutritional value of the recipe can be calculated using a conversion table consisting of the food item number and unit class. Next, using deep learning techniques, we built models that estimate the list of food item numbers from food images. While the multi-task model that identifies the food category label and the ingredient list simultaneously is only effective within a limited number of recipes, the single-task model that only identified the ingredient list achieved a Micro-F1 of 53.32% in total.
Scan-less microscopy based on acousto-optic encoded illumination

Abstract Several optical microscopy methods are now available for characterizing scientific and industrial processes at sub-micron resolution. However, they are often ill-suited for imaging rapid events. Limited by the trade-off between camera frame-rate and sensitivity, or the need for mechanical scanning, current microscopes are optimized for imaging at hundreds of frames-per-second (fps), well-below what is needed in processes such as neuronal signaling or moving parts in manufacturing lines. Here, we present a scan-less technology that allows sub-micrometric imaging at thousands of fps. It is based on combining a single-pixel camera with parallelized encoded illumination. We use two acousto-optic deflectors (AODs) placed in a Mach–Zehnder interferometer and drive them simultaneously with multiple and unique acoustic frequencies. As a result, orthogonal light stripes are obtained that interfere with the sample plane, forming a two-dimensional array of flickering spots – each with its modulation frequency. The light from the sample is collected with a single photodiode that, after spectrum analysis, allows for image reconstruction at speeds only limited by the AOD’s bandwidth and laser power. We describe the working principle of our approach, characterize its imaging performance as a function of the number of pixels – up to 400 × 400 – and characterize dynamic events at 5000 fps.
Le cours moyen de la morge et de ses affluents (Puy-de-Dôme, France) au pléistocène : éléments de chronologie et de paléoenvironnement

The study of upper pleistocene alluvial deposits of the Morge river reveals two phases of volcanic ash falls. Frost phenomena have also been identified. Among several in-cuting phases, one has been reported to Allerod according with radiocarbone dating on a peatty layer. Geological and morphological mapping in Morge, Sardon and CHambaron valleys brings up paleogeographical and chronological data. Tectonism has involved important changes in Sardon and Morge river beds
Music-Graph2Vec: An Efficient Method for Embedding Pitch Segment

Learning low-dimensional continuous vector representation for short pitch segment extracted from songs is has been confirmed to contain tonal features of music, which is key to melody modeling that can be utilized in many music investigations, such as genre classification, emotion classification, and music retrieval, and so on. The skip-gram version of Word2Vec is ubiquitous, and widely used approach for music pitch segment embedding, but it poorly scales to large data sets due to its extremely long training time. In this paper, we propose a novel efficient graph-based embedding method, named Music-Graph2Vec, to tackle this concern. This approach converts music files into graphs, extracts the rhythmic sequence through random walking, and trains the rhythmic embedding model using skip-gram. Experimental results demonstrate that Music-Graph2Vec outperforms Word2Vec in training rhythmic embedding, with the advantage of being 55 times faster on the top-MAGD dataset (2,134.7s for Word2Vec and 38.9s for Music-Graph2Vec), with the same accuracy for Word2Vec in terms of music genre classification.
Adapting Object Detection to Fisheye Cameras: A Knowledge Distillation with Semi-Pseudo-Label Approach

In this paper, we introduce a lightweight object detection system, custom-designed for fisheye cameras and optimized for quick deployment on embedded systems. Given the constraints of training solely on standard images, our methodology centers on the effective knowledge transfer to accentuate object detection in fisheye scenarios. The integration of the Parallel Residual Bi-Fusion (PRB) Feature Pyramid Network (FPN) into the state-of-the-art YOLOv7 backbone specifically addresses the challenges of detecting tiny objects often present in fisheye images. Our unique two-phase training strategy operates as follows: Firstly, a comprehensive Teacher Model is trained on standard images, setting the stage for knowledge acquisition. Subsequently, in the second phase, this knowledge is distilled to a more compact Student Model. The twist is in using fisheye images as pseudo-information, ensuring the model’s adaptability to fisheye-centric environments. Combining knowledge distillation with semi-pseudo-label semi-supervised learning, this strategy guarantees optimal performance and embraces a lightweight design perfect for real-time applications on constrained devices. In essence, our contributions span the crafting of a specialized object detection framework for fisheye cameras, the proposition of a novel two-tiered training strategy, and the synergetic use of PRB with YOLOv7. Empirical results reinforce the efficacy of our approach, illustrating that while our model retains a compact footprint, it doesn’t compromise on performance, excelling in tasks with a comparable nature and offering swift inference.
Developing a VR-based contextualized language learning system to Enhance Junior High School Students' Pragmatic Competence

This paper addresses the significance of pragmatic competence development for language learners in a globalized world. To improve language learning experiences for Taiwanese junior high school students, a VR-based immersive English pragmatics learning game was developed. The game focuses on expressions of requests, apologies, and compliments, aligning with the junior high school English textbooks. Initial framework completion allowed for a SUS (System Usability Scale) survey with 15 urban and 15 rural students, evaluating the system's usability. The survey showed an average SUS score of 74, indicating user satisfaction while also revealing areas for improvement. Future work involves optimizing the system and conducting practical research to assess the game's effectiveness in enhancing students' English language learning outcomes.
Toward Optimal Real-time Dynamic Point Cloud Streaming over Bandwidth-constrained Networks

Point cloud is the emerging format for representing real-world objects in VR/AR applications. However, real-time streaming of dynamic point clouds presents challenges due to high data rates and low latency requirements. This paper introduces a novel and bandwidth-efficient streaming approach for scenes consisting of multiple dynamic point clouds over networks with limited bandwidth. The proposed approach dynamically adjusts the Level of Detail (LoD) of individual point clouds based on network conditions and user preferences to optimize the user’s Quality of Experience (QoE). The LoD version selection problem is formulated as a QoE optimization problem, and two real-time solutions are presented for deciding the LoD version for each point cloud. Experimental results demonstrate that the proposed method outperforms the existing methods in terms of visual quality while achieving remarkably low processing time, about 0.01 ms. These findings have the potential to advance seamless user experience.
MontageNet: Annotated Dataset of Furniture Components in Real-World Images

Indoor understanding is currently a topic that is widely studied in the field of machine learning. Furniture is the most common object in indoor scenes, just as various vehicles are most commonly seen in street scenes. Any object is made up of a combination of functional components. Functional component dismantling and reassembly is an important development for industrial manufacturing to improve efficiency and reduce costs. In the context of understanding indoor scenes, we focus on building a dataset that uses real-world furniture images as part labels. Building a large-scale furniture dataset is very challenging, first of all, the existing dataset has too few real images of furniture, mostly 3D model images, but the diversity of furniture in the real world far exceeds that of 3D models, and real images help improve the calculation speed of model training. Most of the published furniture data is poorly aligned with the annotation data, and even fewer materials perform component segmentation labeling using real furniture images. MontageNet has become a rich resource for part-level 3D shape analysis, semantic understanding, instance segmentation and 3D reconstruction, and other research. Our accompanying empirical studies provide an in-depth analysis of dataset characteristics and performance evaluation of several state-of-the-art methods against our benchmarks.
A multiscale model of striatum microcircuit dynamics

Building large-scale simulations of neuronal activity is currently a main challenge in neuroscience research. An efficient approach has been recently adopted which is based on the use of mean-field models to build whole-brain simulations, where large neuronal populations are represented by single mean-field models. However, these methods are normally based on generic mean-field models that do not capture the cellular and structural specificity of each brain region, which imposes a strong limitation to its applicability. Thus, the development of region specific mean-field models is currently of great relevance, which can provide a big advance to these methods. In this paper we present spiking network and mean-field models of the striatum microcircuit, the largest structure in the basal ganglia, an area known for its key role in functions such as learning and motor control. To this end we adopt a recent formalism based on a bottom-up approach that starts from single-cell models and spiking networks to build a mean-field model of populations of neurons with different cell types. We start with a brief introduction to the microcircuit of the striatum and we describe step by step the construction of a spiking network model, and its mean-field, for this area. Then we test the mean-field model by analyzing the response of the system to the main brain rhythms observed in the striatum and comparing our results to the corresponding spiking networks. Next, we study the effects of dopamine in striatal cells, a very important feature of the area, and we show its dynamical consequences at the network level, and how it can be captured by the mean-field model. Finally we introduce a basic implementation of reinforcement learning (one of the main known functions of the basal-ganglia) using the mean-field model of the striatum microcircuit.