Miracle-dz/UniFAR-FTUs huggingface dataset:
Convective Rainfall in Lake Victoria Watershed and Adjacent Equatorial Africa

An integrated satellite precipitation estimation dataset, namely, the Climate Prediction Center morphing method (CMORPH), was used to analyze precipitation regimes across Equatorial Africa between 3°S - 1°N and 24°E - 42°E from 2000 to 2014. This region includes the Rift Valley, part of the Congo Forest, and the Lake Victoria (LV) basin, the second largest lake in the area of the world. Hovmoller diagrams were obtained for all organized convective systems to estimate their spans, duration, and phase speeds. The analysis included 33,189 episodes of westward propagating convective systems. Within the study area, lake and land breezes tend to trigger convection and precipitation over LV as well as mountain-valley circulation trigger thunderstorms over the mountains east of LV and western Rift Valley. The statistics of convective systems streaks on longitude-time diagrams were obtained for yearly frequencies of starting and ending longitudes and times among other morphologic variables. Results indicate organized precipitation episodes tend to move westward across Rift valley and Congo forest with an average phase speed of 10.3 m·s-1. More than 50% of them are triggered over LV and propagate more than 600 km at an average phase speed of 12.1 m·s-1. These convective systems tend to produce high rainfall rates hundreds of kilometers away into the Congo Forest. Half of all episodes of organized convection analyzed have phase speeds between 8 m·s-1 and 16 m·s-1, lasting 8 hr to 16 hr. Most precipitating systems start east of LV and west of Rift Valley in the afternoon to early morning and propagates less than 400 km. Finally, hourly precipitation accumulation and lightning density analysis indicate three preferable regions for convective initiation: 1) The mountain range east of LV; 2) Midwest of LV, and; 3) The Congo Forest mountain range.
An Evaluation of MODIS Global Evapotranspiration Product as Satellite-Based Evapotranspiration Data for Supporting Precision Agriculture in West Papua - Indonesia

Precision Agriculture has been a significant issue since the middle of the 1980s. Evapotranspiration is one of the main parameters in precision agriculture to analyze real water needs in the agriculture area and managing water resources. Traditionally evapotranspiration estimates by directly measured methods, i.e., lysimeter, pan-evaporation, eddy covariance, Bowen ratio, soil water, and climate data analysis. These methods are expensive techniques with low spatial representativeness. The utilization of remote sensing technology is expected to be an alternative solution for providing evapotranspiration data with a cost-effective and high spatial representative. This research aims to evaluate the MODIS global evapotranspiration as satellite-based evapotranspiration in estimating evapotranspiration in West Papua. Four (4) statistical parameters, i.e., mean error (ME), root means square error (RMSE), relative bias (RB), and mean bias factor (MBF), are using for evaluation. The research showed that MODIS global evapotranspiration was overestimated in estimating evapotranspiration in West Papua. However, MODIS global evapotranspiration has an acceptable accuracy in estimating evapotranspiration in West Papua indicated by ME = 0.66 mm/day, RMSE = 0.94 mm/day, RB = 0.27, and MBF = 0.81. Therefore, MODIS global evapotranspiration can be used as an alternative solution for providing evapotranspiration data in West Papua with a cost-effective.
Topological lattice models with constant Berry curvature

Band geometry plays a substantial role in topological lattice models. The Berry curvature, which resembles the effect of magnetic field in reciprocal space, usually fluctuates throughout the Brillouin zone. Motivated by the analogy with Landau levels, constant Berry curvature has been suggested as an ideal condition for realizing fractional Chern insulators. Here we show that while the Berry curvature cannot be made constant in a topological two-band model, lattice models with three or more degrees of freedom per unit cell can support exactly constant Berry curvature. However, contrary to the intuitive expectation, we find that making the Berry curvature constant does not always improve the properties of fractional Chern insulator states. In fact, we show that an ``ideal flatband'' cannot have constant Berry curvature, equivalently, we show that the density algebra of Landau levels cannot be realised in any tight-binding lattice system.
The Art of the Meta Stream Protocol: Torrents of Streams

Context The rise of streaming libraries such as Akka Stream, Reactive Extensions, and LINQ popularized the declarative functional style of data processing. The stream paradigm offers concise syntax to write down processing pipelines to consume the vast amounts of real-time data available today. Inquiry These libraries offer the programmer a domain specific language (DSL) embedded in the host language to describe data streams. These libraries however, all suffer from extensibility issues. The semantics of a stream is hard-coded into the DSL language and cannot be changed by the user of the library. Approach We introduce an approach to modify the semantics of a streaming library by means of metaprogramming at both the run-time and compile-time level, and showcase its generality. Knowledge We show that the expressiveness of the meta-facilities is strong enough to enable push and pull semantics, error handling, parallelism, and operator fusion. Grounding We evaluate our work by implementing the identified shortcomings in terms of a novel stream meta-architecture and show that its design and architecture adhere to the design principles of a meta-level architecture. Importance The state of the art offers plenty of choice to programmers regarding reactive stream processing libraries. Expressing reactive systems is otherwise difficult to do in general purpose languages. Extensibility and fine-tuning should be possible in these libraries to ensure a broad variety of applications can be expressed within this single DSL.
Cataclysmic Variables in the Second Year of the Zwicky Transient Facility

Using a filter in the GROWTH Marshal based on color and the amplitude and timescale of variability, we have identified 372 objects as known or candidate cataclysmic variables (CVs) during the second year of the operation of the Zwicky Transient Facility. From the available difference imaging data, we found that 93 are previously confirmed CVs and 279 are strong candidates. Spectra of four of the candidates confirm them as CVs by the presence of Balmer emission lines, while one of the four has prominent He ii lines indicative of containing a magnetic white dwarf. Gaia EDR3 parallaxes are available for 154 of these systems, resulting in distances from 108–2096 pc and absolute magnitudes in the range of 7.5–15.0, with the largest number of candidates between 10.5 and 12.5. The total numbers are 21% higher than from the previous year of the survey with a greater number of distances available but a smaller percentage of systems close to the Galactic plane. Comparison of these findings with a machine-learning method of searching all the light curves reveals large differences in each data set related to the parameters involved in the search process.
MXDAG: A Hybrid Abstraction for Cluster Applications

Distributed applications, such as database queries and distributed training, consist of both compute and network tasks. DAG-based abstraction primarily targets compute tasks and has no explicit network-level scheduling. In contrast, Coflow abstraction collectively schedules network flows among compute tasks but lacks the end-to-end view of the application DAG. Because of the dependencies and interactions between these two types of tasks, it is sub-optimal to only consider one of them. We argue that co-scheduling of both compute and network tasks can help applications towards the globally optimal end-to-end performance. However, none of the existing abstractions can provide fine-grained information for co-scheduling. We propose MXDAG, an abstraction to treat both compute and network tasks explicitly. It can capture the dependencies and interactions of both compute and network tasks leading to improved application performance.
Backprop-Free Reinforcement Learning with Active Neural Generative Coding

In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input. This awareness largely depends on how the human agent interacts with the environment. In this work, we propose \emph{active neural generative coding}, a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. Specifically, we develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. We demonstrate on several control problems, in the online learning setting, that our proposed modeling framework performs competitively with deep Q-learning models. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
Emergence of gaussianity in the thermodynamic limit of interacting fermions

Systems of interacting fermions can give rise to ground states whose correlations become effectively free-fermion-like in the thermodynamic limit, as shown by Baxter for a class of integrable models that include the one-dimensional XYZ spin1 2 chain. Here, we quantitatively analyse this behaviour by establishing the relation between system size and correlation length required for the fermionic Gaussianity to emerge. Importantly, we demonstrate that this behaviour can be observed through the applicability of Wick’s theorem and thus it is experimentally accessible. To establish the relevance of our results to possible experimental realisations of XYZ-related models, we demonstrate that the emergent Gaussianity is insensitive to weak variations in the range of interactions, coupling inhomogeneities and local random potentials.
WeightScale: Interpreting Weight Change in Neural Networks

Interpreting the learning dynamics of neural networks can provide useful insights into how networks learn and the development of better training and design approaches. We present an approach to interpret learning in neural networks by measuring relative weight change on a per layer basis and dynamically aggregating emerging trends through combination of dimensionality reduction and clustering which allows us to scale to very deep networks. We use this approach to investigate learning in the context of vision tasks across a variety of state-of-the-art networks and provide insights into the learning behavior of these networks, including how task complexity affects layer-wise learning in deeper layers of networks.
Mixed reality technologies for people with dementia: Participatory evaluation methods

Technologies can support people with early onset dementia (PwD) to aid them in Instrumental Activities of Daily Living (IADL). The integration of physical and virtual realities in Mixed reality technologies (MRTs) could provide scalable and deployable options in developing prompting systems for PwD. However, these emerging technologies should be evaluated and investigated for feasibility with PwD. Survey instruments such as SUS, SUPR-Q and ethnographic methods that are used for usability evaluation of websites and apps are used to evaluate and study MRTs. However, PwD who cannot provide written and verbal feedback are unable to participate in these studies. MRTs also present challenges due to different ways in which physical and virtual realities could be coupled. Experiences with physical, virtual and the couplings between the two are to be considered in evaluating MRTs.
A Field Guide to Federated Optimization

Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection. The distributed learning process can be formulated as solving federated optimization problems, which emphasize communication efficiency, data heterogeneity, compatibility with privacy and system requirements, and other constraints that are not primary considerations in other problem settings. This paper provides recommendations and guidelines on formulating, designing, evaluating and analyzing federated optimization algorithms through concrete examples and practical implementation, with a focus on conducting effective simulations to infer real-world performance. The goal of this work is not to survey the current literature, but to inspire researchers and practitioners to design federated learning algorithms that can be used in various practical applications.
Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although penalizing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies which allow to tune their level of conservativity at run-time based on the desired comfort and utility. Using a proactive safety verification approach, the proposed framework can guarantee that actions generated from RL are failsafe according to the worst-case assumptions. Concurrently, the policy is encouraged to minimize safety interference and generate more comfortable behavior. We trained and evaluated the proposed approach and baseline policies using a high level simulator with a variety of randomized scenarios including several corner cases which rarely happen in reality but are very crucial. In light of our experiments, the behavior of policies learned using distributional RL is adaptive at run-time and robust to the environment uncertainty. Quantitatively, the learned distributional RL agent reduces the average driving time more than 50% compared to the normal DQN policy. It also requires 83% less safety interference compared to the rule-based policy while only slightly increasing the average driving time. We also study sensitivity of the learned policy in environments with higher perception noise and show that our algorithm learns policies that can still drive reliable when the perception noise is two times higher than in the training configuration in automated merging and crossing at occluded intersections.
One thousand and one stories: a large-scale survey of software refactoring

Despite the availability of refactoring as a feature in popular IDEs, recent studies revealed that developers are reluctant to use them, and still prefer the manual refactoring of their code. At JetBrains, our goal is to fully support refactoring features in IntelliJ-based IDEs and improve their adoption in practice. Therefore, we start by raising the following main questions. How exactly do people refactor code? What refactorings are the most popular? Why do some developers tend not to use convenient IDE refactoring tools? In this paper, we investigate the raised questions through the design and implementation of a survey targeting 1,183 users of IntelliJ-based IDEs. Our quantitative and qualitative analysis of the survey results shows that almost two-thirds of developers spend more than one hour in a single session refactoring their code; that refactoring types vary greatly in popularity; and that a lot of developers would like to know more about IDE refactoring features but lack the means to do so. These results serve us internally to support the next generation of refactoring features, as well as can help our research community to establish new directions in the refactoring usability research.
Multi-Channel Auto-Encoders and a Novel Dataset for Learning Domain Invariant Representations of Histopathology Images

Domain shift is a problem commonly encountered when developing automated histopathology pipelines. The performance of machine learning models such as convolutional neural networks within automated histopathology pipelines is often diminished when applying them to novel data domains due to factors arising from differing staining and scanning protocols. The Dual-Channel Auto-Encoder (DCAE) model was previously shown to produce feature representations that are less sensitive to appearance variation introduced by different digital slide scanners. In this work, the Multi-Channel Auto-Encoder (MCAE) model is presented as an extension to DCAE which learns from more than two domains of data. Additionally, a synthetic dataset is generated using CycleGANs that contains aligned tissue images that have had their appearance synthetically modified. Experimental results show that the MCAE model produces feature representations that are less sensitive to inter-domain variations than the comparative StaNoSA method when tested on the novel synthetic data. Additionally, the MCAE and StaNoSA models are tested on a novel tissue classification task. The results of this experiment show the MCAE model out performs the StaNoSA model by 5 percentage-points in the f1-score. These results show that the MCAE model is able to generalise better to novel data and tasks than existing approaches by actively learning normalised feature representations.
VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots

We present visual inertial lidar legged navigation system (VILENS), an odometry system for legged robots based on factor graphs. The key novelty is the tight fusion of four different sensor modalities to achieve reliable operation when the individual sensors would otherwise produce degenerate estimation. To minimize leg odometry drift, we extend the robot's state with a linear velocity bias term, which is estimated online. This bias is observable because of the tight fusion of this preintegrated velocity factor with vision, lidar, and inertial measurement unit (IMU) factors. Extensive experimental validation on different ANYmal quadruped robots is presented, for a total duration of 2 h and 1.8 km traveled. The experiments involved dynamic locomotion over loose rocks, slopes, and mud, which caused challenges such as slippage and terrain deformation. Perceptual challenges included dark and dusty underground caverns, and open and feature-deprived areas. We show an average improvement of 62% translational and 51% rotational errors compared to a state-of-the-art loosely coupled approach. To demonstrate its robustness, VILENS was also integrated with a perceptive controller and a local path planner.
FMNet: Latent Feature-Wise Mapping Network for Cleaning Up Noisy Micro-Doppler Spectrogram

Micro-Doppler signatures contain considerable information about target dynamics. However, the radar sensing systems are easily affected by noisy surroundings, resulting in uninterpretable motion patterns on the micro-Doppler spectrogram ( $\mu $ -DS). Meanwhile, radar returns often suffer from multipath, clutter, and interference. These issues lead to difficulty in, for example, motion feature extraction and activity classification using micro-Doppler signatures. In this article, we propose a latent feature-wise mapping strategy, called feature mapping network (FMNet), to transform measured spectrograms so that they more closely resemble the output from a simulation under the same conditions. Based on measured spectrogram and the matched simulated data, our framework contains three parts: an encoder which is used to extract latent representations/features, a decoder outputs reconstructed spectrogram according to the latent features, and a discriminator minimizes the distance of latent features of measured and simulated data. We demonstrate the FMNet with six activities data and two experimental scenarios, and final results show strong enhanced patterns and can keep actual motion information to the greatest extent. On the other hand, we also propose a novel idea which trains a classifier with only simulated data and predicts new measured samples after cleaning them up with the FMNet. From final classification results, we can see significant improvements.
A unified framework for bandit multiple testing

In bandit multiple hypothesis testing, each arm corresponds to a different null hypothesis that we wish to test, and the goal is to design adaptive algorithms that correctly identify large set of interesting arms (true discoveries), while only mistakenly identifying a few uninteresting ones (false discoveries). One common metric in non-bandit multiple testing is the false discovery rate (FDR). We propose a unified, modular framework for bandit FDR control that emphasizes the decoupling of exploration and summarization of evidence. We utilize the powerful martingale-based concept of"e-processes"to ensure FDR control for arbitrary composite nulls, exploration rules and stopping times in generic problem settings. In particular, valid FDR control holds even if the reward distributions of the arms could be dependent, multiple arms may be queried simultaneously, and multiple (cooperating or competing) agents may be querying arms, covering combinatorial semi-bandit type settings as well. Prior work has considered in great detail the setting where each arm's reward distribution is independent and sub-Gaussian, and a single arm is queried at each step. Our framework recovers matching sample complexity guarantees in this special case, and performs comparably or better in practice. For other settings, sample complexities will depend on the finer details of the problem (composite nulls being tested, exploration algorithm, data dependence structure, stopping rule) and we do not explore these; our contribution is to show that the FDR guarantee is clean and entirely agnostic to these details.
Objective Metrics to Evaluate Residual-Echo Suppression During Double-Talk

Human subjective evaluation is optimal to assess speech quality for human perception. The recently introduced deep noise suppression mean opinion score (DNSMOS) metric was shown to estimate human ratings with great accuracy. The signal-to-distortion ratio (SDR) metric is widely used to evaluate residual-echo suppression (RES) systems by estimating speech quality during double-talk. However, since the SDR is affected by both speech distortion and residual-echo presence, it does not correlate well with human ratings according to the DNSMOS. To address that, we introduce two objective metrics to separately quantify the desired-speech maintained level (DSML) and residual-echo suppression level (RESL) during double-talk. These metrics are evaluated using a deep learning-based RES-system with a tunable design parameter. Using 280 hours of real and simulated recordings, we show that the DSML and RESL correlate well with the DNSMOS with high generalization to various setups. Also, we empirically investigate the relation between tuning the RES-system design parameter and the DSML-RESL tradeoff it creates and offer a practical design scheme for dynamic system requirements.