Miracle-dz/UniFAR-FTUs huggingface dataset:
A Meta-Analysis of Distributionally-Robust Models

State-of-the-art image classiﬁers trained on massive datasets (such as ImageNet) have been shown to be vulnerable to a range of both intentional and incidental distribution shifts. On the other hand, several recent classiﬁers with favorable out-of-distribution (OOD) robustness properties have emerged, achieving high accuracy on their target tasks while maintaining their in-distribution accuracy on challenging benchmarks. We present a meta-analysis on a wide range of publicly released models, most of which have been published over the last twelve months. Through this meta-analysis, we empirically identify four main commonalities for all the best-performing OOD-robust models, all of which illuminate the consid-erable promise of vision-language pre-training.
CMNEROne at SemEval-2022 Task 11: Code-Mixed Named Entity Recognition by leveraging multilingual data

Identifying named entities is, in general, a practical and challenging task in the field of Natural Language Processing. Named Entity Recognition on the code-mixed text is further challenging due to the linguistic complexity resulting from the nature of the mixing. This paper addresses the submission of team CMNEROne to the SEMEVAL 2022 shared task 11 MultiCoNER. The Code-mixed NER task aimed to identify named entities on the code-mixed dataset. Our work consists of Named Entity Recognition (NER) on the code-mixed dataset by leveraging the multilingual data. We achieved a weighted average F1 score of 0.7044, i.e., 6% greater than the NER baseline.
First decay-time-dependent analysis of $B^{0} \to K_{S}^{0} \pi^{0}$ at Belle II

We report measurements of the branching fraction ( B ) and direct CP -violating asymmetry ( A CP ) of the charmless decay B 0 → K 0 π 0 at Belle II. A sample of e + e − collisions, corresponding to 189 . 8 fb − 1 of integrated luminosity, recorded at the Υ (4 S ) resonance is used for the ﬁrst decay-time-dependent analysis of these decays within the experiment. We reconstruct about 135 signal candidates, and measure B ( B 0 → K 0 π 0 ) = [11 . 0 ± 1 . 2(stat) ± 1 . 0(syst)] × 10 − 6 and A CP ( B 0 → K 0 π 0 ) = − 0 . 41 +0 . 30 − 0 . 32 (stat) ± 0 . 09(syst).
Comparison of Different Configurations of Saturated Core Fault Current Limiters in a Power Grid by Numerical Method

Short circuit fault currents are increasing due to growing demand for electricity and high complexity in power systems. Because the fault currents reach the highest value which the breakers are unable to restrict, the electrical grid security is under jeopardy. By entering a limiting impedance into a transmission line in series, these impedances restrict the rising amounts of fault currents to levels that are acceptable. Saturated core fault current limiters (SCFCLs) are a pivotal tool for limiting fault currents rise in power networks that have good performance characteristics. In a normal condition, these limiters have slight effects on the system and can effectively limit short circuit currents when occur. In this chapter, various structures of SCFCLs with different arrangements of ac windings&dc windings are presented and the currents passing through the FCLs under the normal and faulty system conditions are assessed and compared. The flux density in various regions of the core in different arrangements has been investigated as well and the desired analyzes have been performed. Simulation will be presented based on COMSOL Multiphysics 5.4, a finite element software package which can provide a precious assessment to compare these protective devices with different configurations.
Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

Many advances in cooperative multi-agent reinforcement learning (MARL) are based on two common design principles: value decomposition and parameter sharing. A typical MARL algorithm of this fashion decomposes a centralized Q-function into local Q-networks with parameters shared across agents. Such an algorithmic paradigm enables centralized training and decentralized execution (CTDE) and leads to efficient learning in practice. Despite all the advantages, we revisit these two principles and show that in certain scenarios, e.g., environments with a highly multi-modal reward landscape, value decomposition, and parameter sharing can be problematic and lead to undesired outcomes. In contrast, policy gradient (PG) methods with individual policies provably converge to an optimal solution in these cases, which partially supports some recent empirical observations that PG can be effective in many MARL testbeds. Inspired by our theoretical analysis, we present practical suggestions on implementing multi-agent PG algorithms for either high rewards or diverse emergent behaviors and empirically validate our findings on a variety of domains, ranging from the simplified matrix and grid-world games to complex benchmarks such as StarCraft Multi-Agent Challenge and Google Research Football. We hope our insights could benefit the community towards developing more general and more powerful MARL algorithms. Check our project website at https://sites.google. com/view/revisiting-marl.
Body Gesture Recognition to Control a Social Robot

—In this work, we propose a gesture based language to allow humans to interact with robots using their body in a natural way. We have created a new gesture detection model using neural networks and a custom dataset of humans performing a set of body gestures to train our network. Furthermore, we compare body gesture communication with other communication channels to acknowledge the importance of adding this knowledge to robots. The presented approach is extensively validated in diverse simulations and real-life exper- iments with non-trained volunteers. This attains remarkable results and shows that it is a valuable framework for social robotics applications, such as human robot collaboration or human-robot interaction.
Can Linear Programs Have Adversarial Examples? A Causal Perspective

The recent years have been marked by extended research on adversarial attacks, especially on deep neural networks. With this work we intend on posing and investigating the question of whether the phenomenon might be more general in nature, that is, adversarial-style attacks outside classiﬁcation. Speciﬁcally, we investigate optimization problems starting with Linear Programs (LPs). We start off by demonstrating the shortcoming of a naive mapping between the formalism of adversarial examples and LPs, to then reveal how we can provide the missing piece—intriguingly, through the Pearlian notion of Causality. Characteristically, we show the direct inﬂuence of the Structural Causal Model (SCM) onto the subsequent LP optimization, which ultimately exposes a notion of confounding in LPs (inherited by said SCM) that allows for adversarial-style attacks. We provide both the general proof formally alongside existential proofs of such intriguing LP-parameterizations based on SCM for three combinatorial problems, namely Linear Assignment, Shortest Path and a real world problem of energy systems.
Flexible Raman Amplifier Optimization Based on Machine Learning-Aided Physical Stimulated Raman Scattering Model

The problem of Raman amplifier optimization is studied. A differentiable interpolation function is obtained for the Raman gain coefficient using machine learning (ML), which allows for the gradient descent optimization of forward-propagating Raman pumps. Both the frequency and power of an arbitrary number of pumps in a forward pumping configuration are then optimized for an arbitrary data channel load and span length. The forward propagation model is combined with an experimentally-trained ML model of a backward-pumping Raman amplifier to jointly optimize the frequency and power of the forward amplifier's pumps and the powers of the backward amplifier's pumps. The joint forward and backward amplifier optimization is demonstrated for an unrepeatered transmission of 250 km. A gain flatness of $< $1 dB over 4 THz is achieved. The optimized amplifiers are validated using a numerical simulator.
Using Machine Learning to Augment Dynamic Time Warping Based Signal Classification

Modern applications such as voice recognition rely on the ability to compare signals to pre-recorded ones to classify them. However, this comparison typically needs to ignore differences due to signal noise, temporal offset, signal magnitude, and other external factors. The Dynamic Time Warping (DTW) algorithm quantifies this similarity by finding corresponding regions between the signals and non-linearly warping one signal by stretching and shrinking it. Unfor-tunately, searching through all “warps” of a signal to find the best corresponding regions is computationally expen-sive. The FastDTW algorithm improves performance, but sacrifices accuracy by only considering small signal warps. My goal is to improve the speed of DTW while maintaining high accuracy. My key insight is that in any particular application domain, signals exhibit specific types of variation. For example, the accelerometer signal measured for two different people would differ based on their stride length and weight. My system, called Machine Learning DTW (MLDTW), uses machine learning to learn the types of warps that are common in a particular domain. It then uses the learned model to improve DTW performance by limiting the search of potential warps appropriately. My results show that compared to FastDTW, MLDTW is at least as fast and reduces errors by 60% on average across four different data sets. These improvements will significantly impact a wide variety of applications (e.g. health monitoring) and enable more scal-able processing of multivariate, higher frequency, and longer signal recordings.
Energetic Variational Neural Network Discretizations to Gradient Flows

We propose structure-preserving neural-network-based numerical schemes to solve both L 2 -gradient ﬂows and generalized diﬀusions. In more detail, by using neural networks as tools for spatial discretization, we introduce a structure-preserving Eulerian algorithm to solve L 2 - gradient ﬂows and a structure-preserving Lagrangian algorithm to solve generalized diﬀusions. The Lagrangian algorithm for a generalized diﬀusion evolves the “ﬂow map” which determines the dy- namics of the generalized diﬀusion. This avoids the non-trivial task of computing the Wasserstein distance between two probability functions. Unlike most existing methods that construct numerical discretizations based on the strong or weak form of the underlying PDE, the proposed schemes are constructed using variational formula- tions of these PDEs for preserving their variational structures. Our schemes ﬁrst perform temporal discretization on these variational systems. By doing so, they are very computer-memory-eﬃcient. Moreover, instead of directly solving the obtained nonlinear systems after temporal and spatial discretization, the minimizing movement scheme is utilized to evolve the solutions. This guarantees the monotonic decay of the energy of the system, and is crucial for the long-term stability of numerical computation. Lastly, the proposed neural-network-based scheme is mesh-free, and enables us to solve gradient ﬂows in high dimensions. Various numerical experiments are presented to demonstrate the accuracy and energy stability of the proposed numerical approaches.
Towards Goal, Feasibility, and Diversity-Oriented Deep Generative Models in Design

I. ABSTRACT Deep Generative Machine Learning Models (DGMs) have been growing in popularity across the design community thanks to their ability to learn and mimic complex data distributions. DGMs are conventionally trained to minimize statistical divergence between the distribution over generated data and distribution over the dataset on which they are trained. While sufﬁcient for the task of generating “realistic” fake data, this objective is typically insufﬁcient for design synthesis tasks. Instead, design problems typically call for adherence to design requirements, such as performance targets and constraints. Advancing DGMs in engineering design requires new training objectives which promote engineering design objectives. In this paper, we present the ﬁrst Deep Generative Model that simultaneously optimizes for performance, feasibility, diversity, and target achievement. We benchmark performance of the proposed method against several Deep Generative Models over eight evaluation metrics that focus on feasibility, diversity, and satisfaction of design performance targets. Methods are tested on a challenging multi-objective bicycle frame design problem with skewed, multimodal data of different datatypes. The proposed framework was found to outperform all Deep Generative Models in six of eight metrics.
Steered quantum annealing: improving time efficiency with partial information

In the computational model of quantum annealing, the size of the minimum gap between the ground state and the ﬁrst excited state of the system is of particular importance, since it is inversely proportional to the running time of the algorithm. Thus, it is desirable to keep the gap as large as possible during the annealing process, since it allows the computation to remain under the protection of the adiabatic theorem while staying eﬃcient. We propose steered quantum annealing as a new method to enlarge the gap throughout the process, in the case of diagonal ﬁnal Hamiltonians, based on the exploitation of some assumptions we can make about the particular problem instance. In order to introduce this information, we propose beginning the anneal from a biased Hamiltonian that incorporates reliable assumptions about the ﬁnal ground state. Our simulations show that this method yields a larger average gap throughout the whole computation, which results in an increased robustness of the overall annealing process.
A Survey : Neural Networks for AMR-to-Text

AMR-to-text is one of the key techniques in the NLP community that aims at generating sentences from the Abstract Meaning Representation (AMR) graphs. Since AMR was proposed in 2013, the study on AMR-to-Text has become increasingly prevalent as an essential branch of structured data to text because of the unique advantages of AMR as a high-level semantic description of natural language. In this paper, we provide a brief survey of AMR-to-Text. Firstly, we introduce the current scenario of this technique and point out its difﬁculties. Secondly, based on the methods used in previous studies, we roughly divided them into ﬁve categories according to their respective mechanisms, i.e., Rules-based, Seq-to-Seq-based, Graph-to-Seq-based, Transformer-based, and Pre-trained Language Model (PLM)-based. In particular, we detail the neural network-based method and present the latest progress of AMR-to-Text, which refers to AMR reconstruction, Decoder optimization, etc. Furthermore, we present the benchmarks and evaluation methods of AMR-to-Text. Eventually, we provide a summary of current techniques and the outlook for future research.
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

Vision-language (VL) pre-training has recently received considerable attention. However, most existing end-to-end pre-training approaches either only aim to tackle VL tasks such as image-text retrieval, visual question answering (VQA) and image captioning that test high-level understanding of images, or only target region-level understanding for tasks such as phrase grounding and object detection. We present FIBER (Fusion-In-the-Backbone-based transformER), a new VL model architecture that can seamlessly handle both these types of tasks. Instead of having dedicated transformer layers for fusion after the uni-modal backbones, FIBER pushes multimodal fusion deep into the model by inserting cross-attention into the image and text backbones, bringing gains in terms of memory and performance. In addition, unlike previous work that is either only pre-trained on image-text data or on fine-grained data with box-level annotations, we present a two-stage pre-training strategy that uses both these kinds of data efficiently: (i) coarse-grained pre-training based on image-text data; followed by (ii) fine-grained pre-training based on image-text-box data. We conduct comprehensive experiments on a wide range of VL tasks, ranging from VQA, image captioning, and retrieval, to phrase grounding, referring expression comprehension, and object detection. Using deep multimodal fusion coupled with the two-stage pre-training, FIBER provides consistent performance improvements over strong baselines across all tasks, often outperforming methods using magnitudes more data. Code is available at https://github.com/microsoft/FIBER.
The BSE-property for vector-valued $L^p-$algebras

Let A be a separable Banach algebra, G be a locally compact Hausdorff group and 1 < p < ∞. In this paper, we first provide a necessary and sufficient condition, for which L(G,A) is a Banach algebra, under convolution product. Then we characterize the character space of L(G,A), in the case where A is commutative and G is abelian. Moreover, we investigate the BSE-property for L(G,A) and prove that L(G,A) is a BSE-algebra if and only if A is a BSE-algebra and G is finite. Finally, we study the BSE-norm property for L(G,A) and show that if L(G,A) is a BSE-norm algebra then A is so. We prove the converse of this statement for the case where G is finite and A is unital.
Automated image analysis in large-scale cellular electron microscopy: A literature survey

Large-scale electron microscopy (EM) datasets generated using (semi-) automated microscopes are becoming the standard in EM. Given the vast amounts of data, manual analysis of all data is not feasible, thus automated analysis is crucial. The main challenges in automated analysis include the annotation that is needed to analyse and interpret biomedical images, coupled with achieving high-throughput. Here, we review the current state-of-the-art of automated computer techniques and major challenges for the analysis of structures in cellular EM. The advanced computer vision, deep learning and software tools that have been developed in the last ﬁve years for automatic biomedical image analysis are discussed with respect to annotation, segmentation and scalability for EM data. Integration of automatic image acquisition and analysis will allow for high-throughput analysis of millimeter-range datasets with nanometer resolution.
Rethinking Initialization of the Sinkhorn Algorithm

While the optimal transport (OT) problem was originally formulated as a linear program, the addition of entropic regularization has proven beneficial both computationally and statistically, for many applications. The Sinkhorn fixed-point algorithm is the most popular approach to solve this regularized problem, and, as a result, multiple attempts have been made to reduce its runtime using, e.g., annealing in the regularization parameter, momentum or acceleration. The premise of this work is that initialization of the Sinkhorn algorithm has received comparatively little attention, possibly due to two preconceptions: since the regularized OT problem is convex, it may not be worth crafting a good initialization, since any is guaranteed to work; secondly, because the outputs of the Sinkhorn algorithm are often unrolled in end-to-end pipelines, a data-dependent initialization would bias Jacobian computations. We challenge this conventional wisdom, and show that data-dependent initializers result in dramatic speed-ups, with no effect on differentiability as long as implicit differentiation is used. Our initializations rely on closed-forms for exact or approximate OT solutions that are known in the 1D, Gaussian or GMM settings. They can be used with minimal tuning, and result in consistent speed-ups for a wide variety of OT problems.
How to Reduce Change Detection to Semantic Segmentation

Change detection (CD) aims to identify changes that occur in an image pair taken different times. Prior methods devise specific networks from scratch to predict change masks in pixel-level, and struggle with general segmentation problems. In this paper, we propose a new paradigm that reduces CD to semantic segmentation which means tailoring an existing and powerful semantic segmentation network to solve CD. This new paradigm conveniently enjoys the mainstream semantic segmentation techniques to deal with general segmentation problems in CD. Hence we can concentrate on studying how to detect changes. We propose a novel and importance insight that different change types exist in CD and they should be learned separately. Based on it, we devise a module named MTF to extract the change information and fuse temporal features. MTF enjoys high interpretability and reveals the essential characteristic of CD. And most segmentation networks can be adapted to solve the CD problems with our MTF module. Finally, we propose C-3PO, a network to detect changes at pixel-level. C-3PO achieves state-of-the-art performance without bells and whistles. It is simple but effective and can be considered as a new baseline in this field. Our code is at https://github.com/DoctorKey/C-3PO.
Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems

Stochastic optimization has found wide applications in minimizing objective functions in machine learning, which motivates a lot of theoretical studies to understand its practical success. Most of existing studies focus on the convergence of optimization errors, while the generalization analysis of stochastic optimization is much lagging behind. This is especially the case for nonconvex and nonsmooth problems often encountered in practice. In this paper, we initialize a systematic stability and generalization analysis of stochastic optimization on nonconvex and nonsmooth problems. We introduce novel algorithmic stability measures and establish their quantitative connection on the gap between population gradients and empirical gradients, which is then further extended to study the gap between the Moreau envelope of the empirical risk and that of the population risk. To our knowledge, these quantitative connection between stability and generalization in terms of either gradients or Moreau envelopes have not been studied in the literature. We introduce a class of sampling-determined algorithms, for which we develop bounds for three stability measures. Finally, we apply these discussions to derive error bounds for stochastic gradient descent and its adaptive variant, where we show how to achieve an implicit regularization by tuning the step sizes and the number of iterations.
New estimate for the contribution of the Geminga pulsar to the positron excess

The origin of the positron excess is one of the most intriguing mysteries in astroparticle physics. The recent discovery of extended γ-ray halos around the pulsars Geminga, Monogem and PSR J0621+3755 have brought indirect evidence that pulsar wind nebulae accelerate e ± up to very-high-energy. While the precision of previous data does not permit precise evaluation of the parameters for the pulsars, we are able to find the more precise shape of the injection spectrum using new data released by HAWC and LHAASO in 2020 and 2021. We find that this is well fitten by a power-law with an exponential cutoff. The spectral index is quite hard with values around 1 while the cutoff energy is roughly 100 TeV. We also derive the strength of the diffusion coefficient around the pulsars finding that it is two orders of magnitude lower than the average of the Galaxy. Finally, we use the above mentioned results to estimate the contribution of Geminga to the positron excess. This source alone can contribute to the entire positron excess at around 1 TeV.