Miracle-dz/UniFAR-FTUs huggingface dataset:
LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agents

Recent advancements in reasoning abilities of Large Language Models (LLM) has promoted their usage in problems that require high-level planning for robots and artificial agents. However, current techniques that utilize LLMs for such planning tasks make certain key assumptions such as, access to datasets that permit finetuning, meticulously engineered prompts that only provide relevant and essential information to the LLM, and most importantly, a deterministic approach to allow execution of the LLM responses either in the form of existing policies or plan operators. In this work, we propose LgTS (LLM-guided Teacher-Student learning), a novel approach that explores the planning abilities of LLMs to provide a graphical representation of the sub-goals to a reinforcement learning (RL) agent that does not have access to the transition dynamics of the environment. The RL agent uses Teacher-Student learning algorithm to learn a set of successful policies for reaching the goal state from the start state while simultaneously minimizing the number of environmental interactions. Unlike previous methods that utilize LLMs, our approach does not assume access to a propreitary or a fine-tuned LLM, nor does it require pre-trained policies that achieve the sub-goals proposed by the LLM. Through experiments on a gridworld based DoorKey domain and a search-and-rescue inspired domain, we show that generating a graphical structure of sub-goals helps in learning policies for the LLM proposed sub-goals and the Teacher-Student learning algorithm minimizes the number of environment interactions when the transition dynamics are unknown.
Generative Adversarial Training for Text-to-Speech Synthesis Based on Raw Phonetic Input and Explicit Prosody Modelling

We describe an end-to-end speech synthesis system that uses generative adversarial training. We train our Vocoder for raw phoneme-to-audio conversion, using explicit phonetic, pitch and duration modeling. We experiment with several pre-trained models for contextualized and decontextualized word embeddings and we introduce a new method for highly expressive character voice matching, based on discreet style tokens.
Sparse Index Tracking via Topological Learning

In this research, we introduce a novel methodology for the index tracking problem with sparse portfolios by leveraging topological data analysis (TDA). Utilizing persistence homology to measure the riskiness of assets, we introduce a topological method for data-driven learning of the parameters for regularization terms. Specifically, the Vietoris-Rips filtration method is utilized to capture the intricate topological features of asset movements, providing a robust framework for portfolio tracking. Our approach has the advantage of accommodating both $\ell_1$ and $\ell_2$ penalty terms without the requirement for expensive estimation procedures. We empirically validate the performance of our methodology against state-of-the-art sparse index tracking techniques, such as Elastic-Net and SLOPE, using a dataset that covers 23 years of S&P500 index and its constituent data. Our out-of-sample results show that this computationally efficient technique surpasses conventional methods across risk metrics, risk-adjusted performance, and trading expenses in varied market conditions. Furthermore, in turbulent markets, it not only maintains but also enhances tracking performance.
CoCoFormer: A controllable feature-rich polyphonic music generation method

This paper explores the modeling method of polyphonic music sequence. Due to the great potential of Transformer models in music generation, controllable music generation is receiving more attention. In the task of polyphonic music, current controllable generation research focuses on controlling the generation of chords, but lacks precise adjustment for the controllable generation of choral music textures. This paper proposed Condition Choir Transformer (CoCoFormer) which controls the output of the model by controlling the chord and rhythm inputs at a fine-grained level. In this paper, the self-supervised method improves the loss function and performs joint training through conditional control input and unconditional input training. In order to alleviate the lack of diversity on generated samples caused by the teacher forcing training, this paper added an adversarial training method. CoCoFormer enhances model performance with explicit and implicit inputs to chords and rhythms. In this paper, the experiments proves that CoCoFormer has reached the current better level than current models. On the premise of specifying the polyphonic music texture, the same melody can also be generated in a variety of ways.
Laboratory investigation of some characteristics of the Eppley pyrheliometer

The pyranometer of Eppley shows the following deviations of its scale value: (1) The temperature coefficient consists of coefficients of the thermopile and of the different heat exchange parameters. (2) Air convection within the glass bulb causes a dependency of the scale value on the angle between the receiving surface and the direction of gravity. (3) Rays from below the instrument plane may be reflected by the glass ball onto the receiving surface. (4) The existing black paint has a specular reflection which causes a dependency on the angle of incidence. All deviations can be brought under control by suitable modification or calibration.
Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning

Transformer-based models, even though achieving super-human performance on several downstream tasks, are often regarded as a black box and used as a whole. It is still unclear what mechanisms they have learned, especially their core module: multi-head attention. Inspired by functional specialization in the human brain, which helps to efficiently handle multiple tasks, this work attempts to figure out whether the multi-head attention module will evolve similar function separation under multi-tasking training. If it is, can this mechanism further improve the model performance? To investigate these questions, we introduce an interpreting method to quantify the degree of functional specialization in multi-head attention. We further propose a simple multi-task training method to increase functional specialization and mitigate negative information transfer in multi-task learning. Experimental results on seven pre-trained transformer models have demonstrated that multi-head attention does evolve functional specialization phenomenon after multi-task training which is affected by the similarity of tasks. Moreover, the multi-task training strategy based on functional specialization boosts performance in both multi-task learning and transfer learning without adding any parameters.
MOCHA: Real-Time Motion Characterization via Context Matching

Transforming neutral, characterless input motions to embody the distinct style of a notable character in real time is highly compelling for character animation. This paper introduces MOCHA, a novel online motion characterization framework that transfers both motion styles and body proportions from a target character to an input source motion. MOCHA begins by encoding the input motion into a motion feature that structures the body part topology and captures motion dependencies for effective characterization. Central to our framework is the Neural Context Matcher, which generates a motion feature for the target character with the most similar context to the input motion feature. The conditioned autoregressive model of the Neural Context Matcher can produce temporally coherent character features in each time frame. To generate the final characterized pose, our Characterizer network incorporates the characteristic aspects of the target motion feature into the input motion feature while preserving its context. This is achieved through a transformer model that introduces the adaptive instance normalization and context mapping-based cross-attention, effectively injecting the character feature into the source feature. We validate the performance of our framework through comparisons with prior work and an ablation study. Our framework can easily accommodate various applications, including characterization with only sparse input and real-time characterization. Additionally, we contribute a high-quality motion dataset comprising six different characters performing a range of motions, which can serve as a valuable resource for future research.
Evaluating Intelligent Algorithms for Gait Phase Classification in Lower Limb Robotic Systems

Accurate and rapid detection of gait phases is of utmost importance in achieving optimal performance of powered lower-limb prostheses and exoskeletons. With the increasing versatility and complexity of these robotic systems, there is a growing need to enhance the performance of gait detection algorithms. The development of reliable and functional gait detection algorithms holds the potential to enhance precision, stability, and safety in prosthetic devices and other rehabilitation technologies. In this systematic review, we delve into the extensive body of research and development in the domain of gait event detection methods, with a specific focus on their application to prosthetic devices. Our review critically assesses various proposed methods, aiming to identify the most effective approaches for gait phase classification in lower limb robotic systems. Through a comprehensive comparative analysis, we highlight the strengths and weaknesses of different algorithms, shedding light on their performance characteristics, applicability, and potential for further improvements. This comprehensive review was conducted by screening two databases, namely IEEE and Scopus. The search was limited to 204 papers published from 2010 to 2023. A total of 6 papers that focused on Heuristic, Thresholding, and Amplitude Zero Crossing involved techniques were identified and included in the review. 33.3% of implemented Algorithms used kinematic parameters such as joint angles, joint linear and angular velocity, and joint angular acceleration. This study purely focuses on threshold-based algorithms and thus paper focusing on other gait phase detection methods were excluded.
Bridging the Divide: Unraveling the Knowledge Gap in Data Visualization Research and Practice

Empirical research on perception and cognition has laid the foundation for visualization design, often yielding useful design guidelines for practitioners. However, it remains uncertain how well practitioners stay informed about such crucial visualization design knowledge. In this paper, we employed a mixed-method approach to explore the knowledge gap between visualization research and real-world design guidelines. We initially collected existing design guidelines from various sources and empirical studies from diverse publishing venues, analyzing their alignment and uncovering missing links and inconsistent knowledge. Subsequently, we conducted surveys and interviews with practitioners and researchers to gain further insights into their experiences and attitudes towards design guidelines and empirical studies, and their views on the knowledge gap between research and practice. Our findings highlight the similarities and differences in their perspectives and propose strategies to bridge the divide in visualization design knowledge.
Overconstrained Robotic Limb with Energy-Efficient, Omni-directional Locomotion

This paper studies the design, modeling, and control of a novel quadruped, featuring overconstrained robotic limbs employing the Bennett linkage for motion and power transmission. The modular limb design allows the robot to morph into reptile- or mammal-inspired forms. In contrast to the prevailing focus on planar limbs, this research delves into the classical overconstrained linkages, which have strong theoretical foundations in advanced kinematics but limited engineering applications. The study showcases the morphological superiority of overconstrained robotic limbs that can transform into planar or spherical limbs, exemplifying the Bennett linkage. By conducting kinematic and dynamic modeling, we apply model predictive control to simulate a range of locomotion tasks, revealing that overconstrained limbs outperform planar designs in omni-directional tasks like forward trotting, lateral trotting, and turning on the spot when considering foothold distances. These findings highlight the biological distinctions in limb design between reptiles and mammals and represent the first documented instance of overconstrained robotic limbs outperforming planar designs in dynamic locomotion.
Leveraging heterogeneous spillover effects in maximizing contextual bandit rewards

Recommender systems relying on contextual multi-armed bandits continuously improve relevant item recommendations by taking into account the contextual information. The objective of these bandit algorithms is to learn the best arm (i.e., best item to recommend) for each user and thus maximize the cumulative rewards from user engagement with the recommendations. However, current approaches ignore potential spillover between interacting users, where the action of one user can impact the actions and rewards of other users. Moreover, spillover may vary for different people based on their preferences and the closeness of ties to other users. This leads to heterogeneity in the spillover effects, i.e., the extent to which the action of one user can impact the action of another. Here, we propose a framework that allows contextual multi-armed bandits to account for such heterogeneous spillovers when choosing the best arm for each user. By experimenting on several real-world datasets using prominent linear and non-linear contextual bandit algorithms, we observe that our proposed method leads to significantly higher rewards than existing solutions that ignore spillover.
Optimizing Layerwise Polynomial Approximation for Efficient Private Inference on Fully Homomorphic Encryption: A Dynamic Programming Approach

Recent research has explored the implementation of privacy-preserving deep neural networks solely using fully homomorphic encryption. However, its practicality has been limited because of prolonged inference times. When using a pre-trained model without retraining, a major factor contributing to these prolonged inference times is the high-degree polynomial approximation of activation functions such as the ReLU function. The high-degree approximation consumes a substantial amount of homomorphic computational resources, resulting in slower inference. Unlike the previous works approximating activation functions uniformly and conservatively, this paper presents a \emph{layerwise} degree optimization of activation functions to aggressively reduce the inference time while maintaining classification accuracy by taking into account the characteristics of each layer. Instead of the minimax approximation commonly used in state-of-the-art private inference models, we employ the weighted least squares approximation method with the input distributions of activation functions. Then, we obtain the layerwise optimized degrees for activation functions through the \emph{dynamic programming} algorithm, considering how each layer's approximation error affects the classification accuracy of the deep neural network. Furthermore, we propose modulating the ciphertext moduli-chain layerwise to reduce the inference time. By these proposed layerwise optimization methods, we can reduce inference times for the ResNet-20 model and the ResNet-32 model by 3.44 times and 3.16 times, respectively, in comparison to the prior implementations employing uniform degree polynomials and a consistent ciphertext modulus.
Enhanced Edge-Perceptual Guided Image Filtering

Due to the powerful edge-preserving ability and low computational complexity, Guided image filter (GIF) and its improved versions has been widely applied in computer vision and image processing. However, all of them are suffered halo artifacts to some degree, as the regularization parameter increase. In the case of inconsistent structure of guidance image and input image, edge-preserving ability degradation will also happen. In this paper, a novel guided image filter is proposed by integrating an explicit first-order edge-protect constraint and an explicit residual constraint which will improve the edge-preserving ability in both cases. To illustrate the efficiency of the proposed filter, the performances are shown in some typical applications, which are single image detail enhancement, multi-scale exposure fusion, hyper spectral images classification. Both theoretical analysis and experimental results prove that the powerful edge-preserving ability of the proposed filter.
Hyperspectral Image Fusion via Logarithmic Low-rank Tensor Ring Decomposition

Integrating a low-spatial-resolution hyperspectral image (LR-HSI) with a high-spatial-resolution multispectral image (HR-MSI) is recognized as a valid method for acquiring HR-HSI. Among the current fusion approaches, the tensor ring (TR) decomposition-based method has received growing attention owing to its superior performance on preserving the spatial-spectral correlation. Furthermore, the low-rank property in some TR factors has been exploited via the matrix nuclear norm regularization along mode-2. On the other hand, the tensor nuclear norm (TNN)-based approaches have recently demonstrated to be more efficient on keeping high-dimensional low-rank structures in tensor recovery. Here, we study the low-rankness of TR factors from the TNN perspective and consider the mode-2 logarithmic TNN (LTNN) on each TR factor. A novel fusion model is proposed by incorporating this LTNN regularization and the weighted total variation which is to promote the continuity of HR-HSI in the spatial-spectral domain. Meanwhile, we have devised a highly efficient proximal alternating minimization algorithm to solve the proposed model. The experimental results indicate that our method improves the visual quality and exceeds the existing state-of-the-art fusion approaches with respect to various quantitative metrics.
Reading Between the Heat

Stress impacts our physical and mental health as well as our social life. A passive and contactless indoor stress monitoring system can unlock numerous important applications such as workplace productivity assessment, smart homes, and personalized mental health monitoring. While the thermal signatures from a user's body captured by a thermal camera can provide important information about the "fight-flight" response of the sympathetic and parasympathetic nervous system, relying solely on thermal imaging for training a stress prediction model often lead to overfitting and consequently a suboptimal performance. This paper addresses this challenge by introducing ThermaStrain, a novel co-teaching framework that achieves high-stress prediction performance by transferring knowledge from the wearable modality to the contactless thermal modality. During training, ThermaStrain incorporates a wearable electrodermal activity (EDA) sensor to generate stress-indicative representations from thermal videos, emulating stress-indicative representations from a wearable EDA sensor. During testing, only thermal sensing is used, and stress-indicative patterns from thermal data and emulated EDA representations are extracted to improve stress assessment. The study collected a comprehensive dataset with thermal video and EDA data under various stress conditions and distances. ThermaStrain achieves an F1 score of 0.8293 in binary stress classification, outperforming the thermal-only baseline approach by over 9%. Extensive evaluations highlight ThermaStrain's effectiveness in recognizing stress-indicative attributes, its adaptability across distances and stress scenarios, real-time executability on edge platforms, its applicability to multi-individual sensing, ability to function on limited visibility and unfamiliar conditions, and the advantages of its co-teaching approach. These evaluations validate ThermaStrain's fidelity and its potential for enhancing stress assessment.
A Lens Finder Map to Check Claimed High-z Galaxies behind SMACS J0723.3-7327

The first science image released by the James Webb Space Telescope (JWST) reveals numerous galaxies in the distant background of the galaxy cluster SMACS J0723.3-7327. Some have claimed redshifts of up to z ≃ 20, challenging standard cosmological models for structure formation. Here, we present a lens model for SMACS J0723.3-7327 anchored on five spectroscopically confirmed systems at 1.38 ≤ z ≤ 2.21 that are multiply lensed, along with 12 other systems with proposed image counterparts sharing common colors, spectral energy distributions, and morphological features, but having unknown redshifts. Constrained only by their image positions, and where available, redshifts, our lens model correctly reproduces the positions and correctly predicts the morphologies and relative brightnesses of all these image counterparts, as well as providing geometrically determined redshifts spanning 1.4 ≲ z ≲ 6.7 for the 12 candidate multiply lensed galaxies lacking spectroscopic measurements. From this lens model, we create a lens finder map that defines regions over which galaxies beyond a certain redshift are predicted to be multiply lensed. Applying this map to three galaxies claimed to be at 10 ≲ z ≲ 20, we find no image counterparts at locations (with an uncertainty of ∼0.″5) where they ought to be sufficiently magnified to be detectable—suggesting instead that these galaxies lie at z ≲ 1.7–3.2. In lieu of spectroscopy, the creation of reliable lens finder maps for cluster fields is urgently needed to test and constrain redshifts inferred from photometry for a rapidly increasing number of candidate high-z galaxies found with JWST.
Radiation Tolerance of Low-Noise Photoreceivers for the LISA Space Mission

This study investigates the effects of space environmental radiation on the performance of InGaAs Quadrant Photodiodes (QPDs) and assesses their suitability for the Laser Interferometer Space Antenna (LISA) mission. QPDs of 1.0, 1.5 and 2.0 mm have been irradiated with 20 and 60 MeV protons, 0.5 and 1 MeV electrons, and Co$^{60}$ gamma. An exposure corresponding to a displacement damage equivalent fluence of $1.0 \times 10^{+12}$ p/cm$^2$ for 20 and 60 MeV protons and a total ionizing dose of 237 krad were applied, surpassing the anticipated radiation levels for the LISA mission by a factor of approximately five. Experiments were conducted to measure changes in QPD dark current, capacitance, and responsivity. The QPDs are integrated with a low-noise DC-coupled transimpedance amplifier to form the Photoreceiver (QPR). QPR noise and performance in an interferometric system like LISA were also measured. Although radiation impacted their dark current and responsivity, almost all QPDs met LISA's validation criteria and did not demonstrate any critical failure. These findings prove that the tested QPDs are promising candidates for LISA and other space-based missions.
YOLOV7 for Mosquito Breeding Grounds Detection and Tracking

With the looming threat of climate change, neglected tropical diseases such as dengue, zika, and chikungunya have the potential to become an even greater global concern. Remote sensing technologies can aid in controlling the spread of Aedes Aegypti, the transmission vector of such diseases, by automating the detection and mapping of mosquito breeding sites, such that local entities can properly intervene. In this work, we leverage YOLOv7, a state-of-the-art and computationally efficient detection approach, to localize and track mosquito foci in videos captured by unmanned aerial vehicles. We experiment on a dataset released to the public as part of the ICIP 2023 grand challenge entitled Automatic Detection of Mosquito Breeding Grounds. We show that YOLOv7 can be directly applied to detect larger foci categories such as pools, tires, and water tanks and that a cheap and straightforward aggregation of frame-by-frame detection can incorporate time consistency into the tracking process.