Miracle-dz/UniFAR-FTUs huggingface dataset:
Multi-Agent Deep Reinforcement Learning with Adaptive Policies

We propose a novel approach to address one aspect of the non-stationarity problem in multi-agent reinforcement learning (RL), where the other agents may alter their policies due to environment changes during execution. This violates the Markov assumption that governs most single-agent RL methods and is one of the key challenges in multi-agent RL. To tackle this, we propose to train multiple policies for each agent and postpone the selection of the best policy at execution time. Specifically, we model the environment non-stationarity with a finite set of scenarios and train policies fitting each scenario. In addition to multiple policies, each agent also learns a policy predictor to determine which policy is the best with its local information. By doing so, each agent is able to adapt its policy when the environment changes and consequentially the other agents alter their policies during execution. We empirically evaluated our method on a variety of common benchmark problems proposed for multi-agent deep RL in the literature. Our experimental results show that the agents trained by our algorithm have better adaptiveness in changing environments and outperform the state-of-the-art methods in all the tested environments.
Image Based Identification of Ghanaian Timbers Using the XyloTron: Opportunities, Risks and Challenges

Computer vision systems for wood identification have the potential to empower both producer and consumer countries to combat illegal logging if they can be deployed effectively in the field. In this work, carried out as part of an active international partnership with the support of UNIDO, we constructed and curated a field-relevant image data set to train a classifier for wood identification of $15$ commercial Ghanaian woods using the XyloTron system. We tested model performance in the laboratory, and then collected real-world field performance data across multiple sites using multiple XyloTron devices. We present efficacies of the trained model in the laboratory and in the field, discuss practical implications and challenges of deploying machine learning wood identification models, and conclude that field testing is a necessary step - and should be considered the gold-standard - for validating computer vision wood identification systems.
Distributional Locational Marginal Pricing Based Optimization for Electric Vehicle Charging Management

Electric power generation, transmission, and distribution systems are attracting a large amount of interest from researchers with the development of the smart grid technologies. A smart grid aims at effective control and conditioning of the distribution of electricity. Pricing signal based distribution system are seen as one of the novel ways to achieve control in a smart grid. In our work, we propose to use a pricing signal modeled after the locational marginal price in the transmission system to locally provide price data to the users. The formulation and implementation of the distributional locational marginal price (DLMP) are achieved to develop a fair pricing model. The work is further practically implemented to a grid with Electric Vehicles in addition to the conventional load. The increasing popularity of EVs because of their ability to reduce greenhouse gas (GHG) emissions will pose a greater challenge in terms of demand on the power grid. We propose to use the DLMP modeling to alleviate congestion in the power grid and develop an optimal charging schedule for EVs.
Image segmentation of liver stage malaria infection with spatial uncertainty sampling

Global eradication of malaria depends on the development of drugs effective against the silent, yet obligate liver stage of the disease. The gold standard in drug development remains microscopic imaging of liver stage parasites in in vitro cell culture models. Image analysis presents a major bottleneck in this pipeline since the parasite has significant variability in size, shape, and density in these models. As with other highly variable datasets, traditional segmentation models have poor generalizability as they rely on hand-crafted features; thus, manual annotation of liver stage malaria images remains standard. To address this need, we develop a convolutional neural network architecture that utilizes spatial dropout sampling for parasite segmentation and epistemic uncertainty estimation in images of liver stage malaria. Our pipeline produces high-precision segmentations nearly identical to expert annotations, generalizes well on a diverse dataset of liver stage malaria parasites, and promotes independence between learned feature maps to model the uncertainty of generated predictions.
Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation

Achieving robust multi-person 2D body landmark localization and pose estimation is essential for human behavior and interaction understanding as encountered for instance in HRI settings. Accurate methods have been proposed recently, but they usually rely on rather deep Convolutional Neural Network (CNN) architecture, thus requiring large computational and training resources. In this paper, we investigate different architectures and methodologies to address these issues and achieve fast and accurate multi-person 2D pose estimation. To foster speed, we propose to work with depth images, whose structure contains sufficient information about body landmarks while being simpler than textured color images and thus potentially requiring less complex CNNs for processing. In this context, we make the following contributions. i) we study several CNN architecture designs combining pose machines relying on the cascade of detectors concept with lightweight and efficient CNN structures; ii) to address the need for large training datasets with high variability, we rely on semi-synthetic data combining multi-person synthetic depth data with real sensor backgrounds; iii) we explore domain adaptation techniques to address the performance gap introduced by testing on real depth images; iv) to increase the accuracy of our fast lightweight CNN models, we investigate knowledge distillation at several architecture levels which effectively enhance performance. Experiments and results on synthetic and real data highlight the impact of our design choices, providing insights into methods addressing standard issues normally faced in practical applications, and resulting in architectures effectively matching our goal in both performance and speed.
DLTPy: Deep Learning Type Inference of Python Function Signatures using Natural Language Context

Due to the rise of machine learning, Python is an increasingly popular programming language. Python, however, is dynamically typed. Dynamic typing has shown to have drawbacks when a project grows, while at the same time it improves developer productivity. To have the benefits of static typing, combined with high developer productivity, types need to be inferred. In this paper, we present DLTPy: a deep learning type inference solution for the prediction of types in function signatures based on the natural language context (identifier names, comments and return expressions) of a function. We found that DLTPy is effective and has a top-3 F1-score of 91.6%. This means that in most of the cases the correct type is within the top-3 predictions. We conclude that natural language contained in comments and return expressions are beneficial to predicting types more accurately. DLTPy does not significantly outperform or underperform the previous work NL2Type for Javascript, but does show that similar prediction is possible for Python.
Proximal Splitting Algorithms: Overrelax them all!

Many problems in statistics, machine learning, signal and image processing, or control can be formulated as convex optimization problems [1–6]. In the age of ‘big data’, with the explosion in size and complexity of the data to process, it is increasingly important to be able to solve convex optimization problems, whose solutions live in very high dimensional spaces [2, 7–10]. There is extensive literature about splitting methods for solving convex optimization problems, with applications in various fields [8, 11–17]. They consist of simple, easy to compute, steps that can deal with the terms in the objective function separately. In this paper, we present several splitting methods in the single umbrella of the forward–backward iteration to solve monotone inclusions, appliedwith preconditioning. In addition, we show that, when the smooth term in the objective function is quadratic, convergence is guaranteed with larger values of the relaxation parameter than previously known. By relaxation, we mean the following: let us consider an iterative algorithm of the form z = T (z), for some operator T , which converges to some fixed point and solution z. Then z tends to be closer than z to z, on average. So, we may want, starting at z, to move further in the direction z − z, which improves the estimate. This yields the relaxed iteration: ⌊ z 1 2 ) = T (z) z = z + ρ(z 1 2 ) − z) . (1)
Optimization for Reinforcement Learning: From a single agent to cooperative agents

Fueled by recent advances in deep neural networks, reinforcement learning (RL) has been in the limelight because of many recent breakthroughs in artificial intelligence, including defeating humans in games (e.g., chess, Go, StarCraft), self-driving cars, smart-home automation, and service robots, among many others. Despite these remarkable achievements, many basic tasks can still elude a single RL agent. Examples abound, from multiplayer games, multirobots, cellular-antenna tilt control, traffic-control systems, and smart power grids to network management.
Efficient Relaxed Gradient Support Pursuit for Sparsity Constrained Non-convex Optimization

Large-scale non-convex sparsity-constrained problems have recently gained extensive attention. Most existing deterministic optimization methods (e.g., GraSP) are not suitable for large-scale and high-dimensional problems, and thus stochastic optimization methods with hard thresholding (e.g., SVRGHT) become more attractive. Inspired by GraSP, this paper proposes a new general relaxed gradient support pursuit (RGraSP) framework, in which the sub-algorithm only requires to satisfy a slack descent condition. We also design two specific semi-stochastic gradient hard thresholding algorithms. In particular, our algorithms have much less hard thresholding operations than SVRGHT, and their average per-iteration cost is much lower (i.e., O(d) vs. O(d log(d)) for SVRGHT), which leads to faster convergence. Our experimental results on both synthetic and real-world datasets show that our algorithms are superior to the state-of-the-art gradient hard thresholding methods.
Prior-based Domain Adaptive Object Detection for Adverse Weather Conditions

Adverse weather conditions such as rain and haze corrupt the quality of captured images, which cause detection networks trained on clean images to perform poorly on these images. To address this issue, we propose an unsupervised prior-based domain adversarial object detection framework for adapting the detectors to different weather conditions. We make the observations that corruptions due to different weather conditions (i) follow the principles of physics and hence, can be mathematically modeled, and (ii) often cause degradations in the feature space leading to deterioration in the detection performance. Motivated by these, we propose to use weather-specific prior knowledge obtained using the principles of image formation to define a novel prior-adversarial loss. The prior-adversarial loss used to train the adaptation process aims to produce weather-invariant features by reducing the weather-specific information in the features, thereby mitigating the effects of weather on the detection performance. Additionally, we introduce a set of residual feature recovery blocks in the object detection pipeline to de-distort the feature space, resulting in further improvements. The proposed framework outperforms all existing methods by a large margin when evaluated on different datasets such as Foggy-Cityscapes, Rainy-Cityscapes, RTTS and UFDD.
Urban Driving with Conditional Imitation Learning

Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the majority of such systems have been primarily evaluated in simulation - a simple domain, which lacks real-world complexities. Motivated by these challenges, we focus on learning representations of semantics, geometry and motion with computer vision for IL from human driving demonstrations. As our main contribution, we present an end-to-end conditional imitation learning approach, combining both lateral and longitudinal control on a real vehicle for following urban routes with simple traffic. We address inherent dataset bias by data balancing, training our final policy on approximately 30 hours of demonstrations gathered over six months. We evaluate our method on an autonomous vehicle by driving 35km of novel routes in European urban streets.
Differences in the magnon diffusion length for electrically and thermally driven magnon currents in 
Y3Fe5O12

Recent demonstration of efficient transport and manipulation of spin information by magnon currents have opened exciting prospects for processing information in devices. Magnon currents can be driven both electrically and thermally, even in magnetic insulators, by applying charge currents in an adjacent metal layer. Earlier reports in thin yttrium iron garnet (YIG) films suggested that the diffusion length of magnons is independent of the biasing method, but different values were obtained in thicker films. Here, we study the magnon diffusion length for electrically and thermally driven magnon currents in the linear regime in a 2-\ensuremath{\mu}m-thick YIG film as a function of temperature and magnetic field. Our results show a decrease in the magnon diffusion length with magnetic field for both biasing methods and at all temperatures from 5 to 300 K, indicating that subthermal magnons dominate long-range transport. Moreover, we demonstrate that the value of the magnon diffusion length depends on the driving mechanism, suggesting that different nonequilibrium magnon distributions are biased for each method. Finally, we demonstrate that the magnon diffusion length for thermally driven magnon currents is independent of the YIG thickness and material growth conditions, confirming that this quantity is an intrinsic parameter of YIG.
Topic-aware chatbot using Recurrent Neural Networks and Nonnegative Matrix Factorization

We propose a novel model for a topic-aware chatbot by combining the traditional Recurrent Neural Network (RNN) encoder-decoder model with a topic attention layer based on Nonnegative Matrix Factorization (NMF). After learning topic vectors from an auxiliary text corpus via NMF, the decoder is trained so that it is more likely to sample response words from the most correlated topic vectors. One of the main advantages in our architecture is that the user can easily switch the NMF-learned topic vectors so that the chatbot obtains desired topic-awareness. We demonstrate our model by training on a single conversational data set which is then augmented with topic matrices learned from different auxiliary data sets. We show that our topic-aware chatbot not only outperforms the non-topic counterpart, but also that each topic-aware model qualitatively and contextually gives the most relevant answer depending on the topic of question.
Efficient Collaborative Multi-Agent Deep Reinforcement Learning for Large-Scale Fleet Management.

Large-scale online ride-sharing platforms have substantially transformed our lives by reallocating transportation resources to alleviate traffic congestion and promote transportation efficiency. An efficient fleet management strategy not only can significantly improve the utilization of transportation resources but also increase the revenue and customer satisfaction. It is a challenging task to design an effective fleet management strategy that can adapt to an environment involving complex dynamics between demand and supply. Existing studies usually work on a simplified problem setting that can hardly capture the complicated stochastic demand-supply variations in high-dimensional space. In this paper we propose to tackle the large-scale fleet management problem using reinforcement learning, and propose a contextual multi-agent reinforcement learning framework including three concrete algorithms to achieve coordination among a large number of agents adaptive to different contexts. We show significant improvements of the proposed framework over state-of-the-art approaches through extensive empirical studies.
Cosmic evolution of star-forming galaxies to z ≃ 1.8 in the faint low-frequency radio source population

We study the properties of star-forming galaxies selected at 610 MHz with the GMRT in a survey covering ∼1.86 deg2 down to a noise of ∼7.1 μJy beam−1. These were identified by combining multiple classification diagnostics: optical, X-ray, infrared, and radio data. Of the 1685 SFGs from the GMRT sample, 496 have spectroscopic redshifts whereas 1189 have photometric redshifts. We find that the IRRC of star-forming galaxies, quantified by the infrared-to-1.4 GHz radio luminosity ratio $\rm {\mathit{ q}_{IR}}$, decreases with increasing redshift: $\rm {\mathit{ q}_{IR}\, =\, 2.86\pm 0.04(1\, +\, \mathit{ z})^{-0.20\pm 0.02}}$ out to z ∼ 1.8. We use the $\rm {\mathit{ V}/\mathit{ V}_{max}}$ statistic to quantify the evolution of the comoving space density of the SFG sample. Averaged over luminosity our results indicate $\rm {\langle \mathit{ V}/\mathit{ V}_{max} \rangle }$ to be $\rm {0.51\, \pm \, 0.06}$, which is consistent with no evolution in overall space density. However, we find $\rm \mathit{ V}/\mathit{ V}_{max}$ to be a function of radio luminosity, indicating strong luminosity evolution with redshift. We explore the evolution of the SFGs radio luminosity function by separating the source into five redshift bins and comparing to theoretical model predictions. We find a strong redshift trend that can be fitted with a pure luminosity evolution of the form $\rm {\mathit{ L}_{610\, MHz}\, \propto \, (\, 1+\, \mathit{ z})^{(2.95\pm 0.19)-(0.50\pm 0.15)z}}$. We calculate the cosmic SFR density since $\rm {\mathit{ z} \sim 1.5}$ by integrating the parametric fits of the evolved 610 MHz luminosity function. Our sample reproduces the expected steep decline in the star formation rate density since $\rm {\mathit{ z}\, \sim \, 1}$.
PointEval: On the Impact of Pointer Analysis Frameworks

Pointer analysis is a foundational analysis leveraged by various static analyses. Therefore, it gathered wide attention in research for decades. Some pointer analysis frameworks are based on succinct declarative specifications. However, these tools are heterogeneous in terms of the underlying intermediate representation (IR), heap abstraction, and programming methodology. This situation complicates a fair comparison of these frameworks and thus hinders further research. Consequently, the literature lacks an evaluation of the strengths and weaknesses of these tools. In this work, we evaluate two major frameworks for pointer analysis, WALA and Doop, on the DaCapo set of benchmarks. We compare the pointer analyses available in Wala and Doop, and conclude that---even though based on a declarative specification---Doop provides a better pointer analysis than Wala in terms of precision and scalability. We also compare the two IRs used in Doop, i.e., Jimple from the Soot framework and IR from the Wala framework. Our evaluation shows that in the majority of the benchmarks Soot's IR gives a more precise and scalable pointer analysis. Finally, we propose a micro-benchmark \emph{PointerBench}, for which we manually validate the points-to statistics to evaluate the results of these tools.
RST-MODNet: Real-time Spatio-temporal Moving Object Detection for Autonomous Driving

Moving Object Detection (MOD) is a critical task for autonomous vehicles as moving objects represent higher collision risk than static ones. The trajectory of the ego-vehicle is planned based on the future states of detected moving objects. It is quite challenging as the ego-motion has to be modelled and compensated to be able to understand the motion of the surrounding objects. In this work, we propose a real-time end-to-end CNN architecture for MOD utilizing spatio-temporal context to improve robustness. We construct a novel time-aware architecture exploiting temporal motion information embedded within sequential images in addition to explicit motion maps using optical flow images.We demonstrate the impact of our algorithm on KITTI dataset where we obtain an improvement of 8% relative to the baselines. We compare our algorithm with state-of-the-art methods and achieve competitive results on KITTI-Motion dataset in terms of accuracy at three times better run-time. The proposed algorithm runs at 23 fps on a standard desktop GPU targeting deployment on embedded platforms.
Phasing a deployable sparse telescope

After launching and deploying a sparse space telescope, fine tuning is required to correct for inaccurate initial placement of its elements. We selected unique shapes and locations of these telescope aperture segments, to be able to distinguish between their diffraction patterns, while at the same time having a proper spatial frequency coverage. Then we improved the combined wave front, without measuring it directly: First we correlated each segment's focal image with its distinctive template, to correct its tilt. Next we interfered them with the other segments, pair by pair, using their limited coherence, to locate their mutual optical path differences. Finally, we optimized the combined focal image for fine alignment.
An HLL Riemann solver for the hybridised discontinuous Galerkin formulation of compressible flows

This work proposes a high-order hybridised discontinuous Galerkin (HDG) formulation of the Harten-Lax-Van Leer (HLL) Riemann solver for compressible flows. A unified framework is introduced to present Lax-Friedrichs, Roe and HLL Riemann solvers via appropriate definitions of the HDG numerical fluxes. The resulting high-order HDG method with HLL Riemann solver is evaluated through a set of numerical simulations of inviscid compressible flows in different regimes, from subsonic isentropic flows to transonic and supersonic problems with shocks. The accuracy of the proposed method is comparable with the one of Lax-Friedrichs and Roe numerical fluxes in subsonic and transonic flows. The superior performance of HLL is highlighted in supersonic cases, where the method provides extra robustness, being able to produce positivity preserving approximations without the need of any user-defined entropy fix.