Miracle-dz/UniFAR-FTUs huggingface dataset:
Almost-linear-time Weighted $\ell_p$-norm Solvers in Slightly Dense Graphs via Sparsification

We give almost-linear-time algorithms for constructing sparsifiers with $n\ poly(\log n)$ edges that approximately preserve weighted $(\ell^{2}_2 + \ell^{p}_p)$ flow or voltage objectives on graphs. For flow objectives, this is the first sparsifier construction for such mixed objectives beyond unit $\ell_p$ weights, and is based on expander decompositions. For voltage objectives, we give the first sparsifier construction for these objectives, which we build using graph spanners and leverage score sampling. Together with the iterative refinement framework of [Adil et al, SODA 2019], and a new multiplicative-weights based constant-approximation algorithm for mixed-objective flows or voltages, we show how to find $(1+2^{-\text{poly}(\log n)})$ approximations for weighted $\ell_p$-norm minimizing flows or voltages in $p(m^{1+o(1)} + n^{4/3 + o(1)})$ time for $p=\omega(1),$ which is almost-linear for graphs that are slightly dense ($m \ge n^{4/3 + o(1)}$).
Experimental Study of Gold and Platinum Solubility in a Complex Fluid under Hydrothermal Conditions

Abstract: The solubility of gold was studied in water and aqueous NaCl (1– 5 m) solutions under oxygen and sulfur buffered conditions between 300–500C at a constant pressure 1 kb. Two buffer assemblages HMP and PPM were used. Analysis of the scatter in measured values in log mAu–mNaCl–T frame fixed linear dependence between log mAu and T at any studied iso‐pleth (mNaCl) in the form of log mAu = a. T(C) + b. Coefficients of the equation were calculated for water and NaCl (1, 3, 5 m) solutions. The maximum solubility characterizes the NaCl‐free system in the presence of HMP. In the case, Au solubility increases from (log mAu) –6. 72 to –5. 04 at 300 and 500C, respectively. In the presence of PPM, maximum of Au solubility was obtained for the 5 mNaCl solution. In a similar manner solubility rises from –6. 54 to –5. 77 at 300 and 500C, accordingly. In studied fO2/fS2 area the behavior of Au solubility testified that: (i) – a composite interaction between chloride and hydrosulfide speciation of gold affects its total solubility; (ii) – in addition of NaCl up to about 1. 5 m the solubility decreases, more pronounced in the presence of HMP; (iii) – the contribution of chloride in total Au solubility is more for PPM despite of lower fO2value, than for HMP.
Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing

Sharing parameters in multi-agent deep reinforcement learning has played an essential role in allowing algorithms to scale to a large number of agents. Parameter sharing between agents significantly decreases the number of trainable parameters, shortening training times to tractable levels, and has been linked to more efficient learning. However, having all agents share the same parameters can also have a detrimental effect on learning. We demonstrate the impact of parameter sharing methods on training speed and converged returns, establishing that when applied indiscriminately, their effectiveness is highly dependent on the environment. We propose a novel method to automatically identify agents which may benefit from sharing parameters by partitioning them based on their abilities and goals. Our approach combines the increased sample efficiency of parameter sharing with the representational capacity of multiple independent networks to reduce training time and increase final returns.
Stochastic Convergence Rates and Applications of Adaptive Quadrature in Bayesian Inference

Abstract We provide the first stochastic convergence rates for a family of adaptive quadrature rules used to normalize the posterior distribution in Bayesian models. Our results apply to the uniform relative error in the approximate posterior density, the coverage probabilities of approximate credible sets, and approximate moments and quantiles, therefore guaranteeing fast asymptotic convergence of approximate summary statistics used in practice. The family of quadrature rules includes adaptive Gauss-Hermite quadrature, and we apply this rule in two challenging lowdimensional examples. Further, we demonstrate how adaptive quadrature can be used as a crucial component of a modern approximate Bayesian inference procedure for high-dimensional additive models. The method is implemented and made publicly available in the aghq package for the R language, available on CRAN.
Survey on Aerial Radio Access Networks: Toward a Comprehensive 6G Access Infrastructure

Current access infrastructures are characterized by heterogeneity, low latency, high throughput, and high computational capability, enabling massive concurrent connections and various services. Unfortunately, this design does not pay significant attention to mobile services in underserved areas. In this context, the use of aerial radio access networks (ARANs) is a promising strategy to complement existing terrestrial communication systems. Involving airborne components such as unmanned aerial vehicles, drones, and satellites, ARANs can quickly establish a flexible access infrastructure on demand. ARANs are expected to support the development of seamless mobile communication systems toward a comprehensive sixth-generation (6G) global access infrastructure. This paper provides an overview of recent studies regarding ARANs in the literature. First, we investigate related work to identify areas for further exploration in terms of recent knowledge advancements and analyses. Second, we define the scope and methodology of this study. Then, we describe ARAN architecture and its fundamental features for the development of 6G networks. In particular, we analyze the system model from several perspectives, including transmission propagation, energy consumption, communication latency, and network mobility. Furthermore, we introduce technologies that enable the success of ARAN implementations in terms of energy replenishment, operational management, and data delivery. Subsequently, we discuss application scenarios envisioned for these technologies. Finally, we highlight ongoing research efforts and trends toward 6G ARANs.
General Total Variation Regularized Sparse Bayesian Learning for Robust Block-Sparse Signal Recovery

Block-sparse signal recovery without knowledge of block sizes and boundaries, such as those encountered in multi-antenna mmWave channel models, is a hard problem for compressed sensing (CS) algorithms. We propose a novel Sparse Bayesian Learning (SBL) method for block-sparse recovery based on popular CS based regularizers with the function input variable related to total variation (TV). Contrary to conventional approaches that impose the regularization on the signal components, we regularize the SBL hyperparameters. This iterative TV-regularized SBL algorithm employs a majorization-minimization approach and reduces each iteration to a convex optimization problem, enabling a flexible choice of numerical solvers. The numerical results illustrate that the TV-regularized SBL algorithm is robust to the nature of the block structure and able to recover signals with both block-patterned and isolated components, proving useful for various signal recovery systems.
Learning Intra-Batch Connections for Deep Metric Learning

The goal of metric learning is to learn a function that maps samples to a lower-dimensional space where similar samples lie closer than dissimilar ones. Particularly, deep metric learning utilizes neural networks to learn such a mapping. Most approaches rely on losses that only take the relations between pairs or triplets of samples into account, which either belong to the same class or two different classes. However, these methods do not explore the embedding space in its entirety. To this end, we propose an approach based on message passing networks that takes all the relations in a mini-batch into account. We refine embedding vectors by exchanging messages among all samples in a given batch allowing the training process to be aware of its overall structure. Since not all samples are equally important to predict a decision boundary, we use an attention mechanism during message passing to allow samples to weigh the importance of each neighbor accordingly. We achieve state-of-the-art results on clustering and image retrieval on the CUB-200-2011, Cars196, Stanford Online Products, and In-Shop Clothes datasets. To facilitate further research, we make available the code and the models at https://github.com/dvl-tum/intra_batch_connections.
CP symmetry and symplectic modular invariance

We analyze CP symmetry in symplectic modular-invariant supersymmetric theories. We show that for genus g\ge 3g≥3 the definition of CP is unique, while two independent possibilities are allowed when g\le 2g≤2. We discuss the transformation properties of moduli, matter multiplets and modular forms in the Siegel upper half plane, as well as in invariant subspaces. We identify CP-conserving surfaces in the fundamental domain of moduli space. We make use of all these elements to build a CP and symplectic invariant model of lepton masses and mixing angles, where known data are well reproduced and observable phases are predicted in terms of a minimum number of parameters.
The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods

The adaptive stochastic gradient descent (SGD) with momentum has been widely adopted in deep learning as well as convex optimization. In practice, the last iterate is commonly used as the final solution to make decisions. However, the available regret analysis and the setting of constant momentum parameters only guarantee the optimal convergence of the averaged solution. In this paper, we fill this theory-practice gap by investigating the convergence of the last iterate (referred to as individual convergence), which is a more difficult task than convergence analysis of the averaged solution. Specifically, in the constrained convex cases, we prove that the adaptive Polyak's Heavy-ball (HB) method, in which only the step size is updated using the exponential moving average strategy, attains an optimal individual convergence rate of $O(\frac{1}{\sqrt{t}})$, as opposed to the optimality of $O(\frac{\log t}{\sqrt {t}})$ of SGD, where $t$ is the number of iterations. Our new analysis not only shows how the HB momentum and its time-varying weight help us to achieve the acceleration in convex optimization but also gives valuable hints how the momentum parameters should be scheduled in deep learning. Empirical results on optimizing convex functions and training deep networks validate the correctness of our convergence analysis and demonstrate the improved performance of the adaptive HB methods.
Mixed Nash Equilibria in the Adversarial Examples Game

This paper tackles the problem of adversarial examples from a game theoretic point of view. We study the open question of the existence of mixed Nash equilibria in the zero-sum game formed by the attacker and the classifier. While previous works usually allow only one player to use randomized strategies, we show the necessity of considering randomization for both the classifier and the attacker. We demonstrate that this game has no duality gap, meaning that it always admits approximate Nash equilibria. We also provide the first optimization algorithms to learn a mixture of classifiers that approximately realizes the value of this game, \emph{i.e.} procedures to build an optimally robust randomized classifier.
Learning Student-Friendly Teacher Networks for Knowledge Distillation

We propose a novel knowledge distillation approach to facilitate the transfer of dark knowledge from a teacher to a student. Contrary to most of the existing methods that rely on effective training of student models given pretrained teachers, we aim to learn the teacher models that are friendly to students and, consequently, more appropriate for knowledge transfer. In other words, at the time of optimizing a teacher model, the proposed algorithm learns the student branches jointly to obtain student-friendly representations. Since the main goal of our approach lies in training teacher models and the subsequent knowledge distillation procedure is straightforward, most of the existing knowledge distillation methods can adopt this technique to improve the performance of diverse student models in terms of accuracy and convergence speed. The proposed algorithm demonstrates outstanding accuracy in several well-known knowledge distillation techniques with various combinations of teacher and student models even in the case that their architectures are heterogeneous and there is no prior knowledge about student models at the time of training teacher networks.
Beyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers

We explore cross-lingual transfer of register classification for web documents. Registers, that is, text varieties such as blogs or news are one of the primary predictors of linguistic variation and thus affect the automatic processing of language. We introduce two new register-annotated corpora, FreCORE and SweCORE, for French and Swedish. We demonstrate that deep pre-trained language models perform strongly in these languages and outperform previous state-of-the-art in English and Finnish. Specifically, we show 1) that zero-shot cross-lingual transfer from the large English CORE corpus can match or surpass previously published monolingual models, and 2) that lightweight monolingual classification requiring very little training data can reach or surpass our zero-shot performance. We further analyse classification results finding that certain registers continue to pose challenges in particular for cross-lingual transfer.
Synthesis and characterization of temperature- and pH-sensitive poly(N,N-diethylacrylamide-co-methacrylic acid)

A series of pH- and temperature-responsive (N,N-diethylacrylamide-co-methacrylic acid) (DEAM–MAA) random copolymers were synthesized by free-radical copolymerization techniques, and their lower critical solution temperatures (LCSTs) at different pH values and their critical phase-transition pH values at different temperatures were determined by transmittance measurements, respectively. The LCSTs of DEAM–MAA copolymers depend on the MAA content in the copolymers and are significantly affected by the pH of the aqueous solution. Similarly, the critical phase-transition pH values also depend on the MAA content of the DEAM–MAA copolymers and are significantly affected by the temperature of the aqueous solution. The LCST of DEAM–MAA copolymers increases with an increase in the MAA content. The critical phase-transition pH increases with increasing MAA content in the copolymer and also increases with increasing aqueous solution temperature. © 2003 Wiley Periodicals, Inc. J Appl Polym Sci 90: 3563–3568, 2003
Coherent Emission in Pulsars, Magnetars, and Fast Radio Bursts: Reconnection-driven Free Electron Laser

We develop a model of the generation of coherent radio emission in the Crab pulsar, magnetars, and fast radio bursts (FRBs). Emission is produced by a reconnection-generated beam of particles via a variant of the free electron laser mechanism, operating in a weakly turbulent, guide field-dominated plasma. We first consider nonlinear Thomson scattering in a guide field-dominated regime, and apply it to explain emission bands observed in Crab pulsar and in FRBs. We consider particle motion in a combined field of the electromagnetic wave and the electromagnetic (Alfvénic) wiggler. Charge bunches, created via a ponderomotive force, Compton/Raman scatter the wiggler field coherently. The model is both robust to the underlying plasma parameters and succeeds in reproducing a number of subtle observed features: (i) emission frequencies depend mostly on the scale λ t of turbulent fluctuations and the Lorentz factor of the reconnection-generated beam, ω∼γb2(c/λt) —it is independent of the absolute value of the underlying magnetic field. (ii) The model explains both broadband emission and the presence of emission stripes, including multiple stripes observed in the high frequency interpulse of the Crab pulsar. (iii) The model reproduces correlated polarization properties: the presence of narrow emission bands in the spectrum favors linear polarization, while broadband emission can have an arbitrary polarization. (iv) The mechanism is robust to the momentum spread of the particle in the beam. We also discuss a model of wigglers as nonlinear force-free Alfvén solitons (light darts).
A hybrid variance-reduced method for decentralized stochastic non-convex optimization

This paper considers decentralized stochastic optimization over a network of $n$ nodes, where each node possesses a smooth non-convex local cost function and the goal of the networked nodes is to find an $\epsilon$-accurate first-order stationary point of the sum of the local costs. We focus on an online setting, where each node accesses its local cost only by means of a stochastic first-order oracle that returns a noisy version of the exact gradient. In this context, we propose a novel single-loop decentralized hybrid variance-reduced stochastic gradient method, called GT-HSGD, that outperforms the existing approaches in terms of both the oracle complexity and practical implementation. The GT-HSGD algorithm implements specialized local hybrid stochastic gradient estimators that are fused over the network to track the global gradient. Remarkably, GT-HSGD achieves a network topology-independent oracle complexity of $O(n^{-1}\epsilon^{-3})$ when the required error tolerance $\epsilon$ is small enough, leading to a linear speedup with respect to the centralized optimal online variance-reduced approaches that operate on a single node. Numerical experiments are provided to illustrate our main technical results.
Robust Lane Detection via Expanded Self Attention

The image-based lane detection algorithm is one of the key technologies in autonomous vehicles. Modern deep learning methods achieve high performance in lane detection, but it is still difficult to accurately detect lanes in challenging situations such as congested roads and extreme lighting conditions. To be robust on these challenging situations, it is important to extract global contextual information even from limited visual cues. In this paper, we propose a simple but powerful self-attention mechanism optimized for lane detection called the Expanded Self Attention (ESA) module. Inspired by the simple geometric structure of lanes, the proposed method predicts the confidence of a lane along the vertical and horizontal directions in an image. The prediction of the confidence enables estimating occluded locations by extracting global contextual information. ESA module can be easily implemented and applied to any encoder-decoder-based model without increasing the inference time. The performance of our method is evaluated on three popular lane detection benchmarks (TuSimple, CULane and BDD100K). We achieve state-of-the-art performance in CULane and BDD100K and distinct improvement on TuSimple dataset. The experimental results show that our approach is robust to occlusion and extreme lighting conditions.
Certifiably Robust Variational Autoencoders

We introduce an approach for training variational autoencoders (VAEs) that are certiﬁably robust to adversarial attack. Specif-ically, we ﬁrst derive actionable bounds on the minimal size of an input perturbation re-quired to change a VAE’s reconstruction by more than an allowed amount, with these bounds depending on certain key parameters such as the Lipschitz constants of the encoder and decoder. We then show how these parameters can be controlled, thereby providing a mechanism to ensure a priori that a VAE will attain a desired level of robustness. Moreover, we extend this to a complete practical approach for training such VAEs to ensure our criteria are met. Critically, our method allows one to specify a desired level of robustness upfront and then train a VAE that is guaranteed to achieve this robustness. We further demonstrate that these Lipschitz– constrained VAEs are more robust to attack than standard VAEs in practice.
Depth lower bounds in Stabbing Planes for combinatorial principles

Stabbing Planes (also known as Branch and Cut) is a proof system introduced very recently which, informally speaking, extends the DPLL method by branching on integer linear inequalities instead of single variables. The techniques known so far to prove size and depth lower bounds for Stabbing Planes are generalizations of those used for the Cutting Planes proof system. For size lower bounds these are established by monotone circuit arguments, while for depth these are found via communication complexity and protection. As such these bounds apply for lifted versions of combinatorial statements. Rank lower bounds for Cutting Planes are also obtained by geometric arguments called protection lemmas. In this work we introduce two new geometric approaches to prove size/depth lower bounds in Stabbing Planes working for any formula: (1) the antichain method, relying on Sperner's Theorem and (2) the covering method which uses results on essential coverings of the boolean cube by linear polynomials, which in turn relies on Alon's combinatorial Nullenstellensatz. We demonstrate their use on classes of combinatorial principles such as the Pigeonhole principle, the Tseitin contradictions and the Linear Ordering Principle. By the first method we prove almost linear size lower bounds and optimal logarithmic depth lower bounds for the Pigeonhole principle and analogous lower bounds for the Tseitin contradictions over the complete graph and for the Linear Ordering Principle. By the covering method we obtain a superlinear size lower bound and a logarithmic depth lower bound for Stabbing Planes proof of Tseitin contradictions over a grid graph.
GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent

In this paper, we present GradPIM, a processingin-memory architecture which accelerates parameter updates of deep neural networks training. As one of processing-in-memory techniques that could be realized in the near future, we propose an incremental, simple architectural design that does not invade the existing memory protocol. Extending DDR4 SDRAM to utilize bank-group parallelism makes our operation designs in processing-in-memory (PIM) module efficient in terms of hardware cost and performance. Our experimental results show that the proposed architecture can improve the performance of DNN training and greatly reduce memory bandwidth requirement while posing only a minimal amount of overhead to the protocol and DRAM area.