Miracle-dz/UniFAR-FTUs huggingface dataset:
Distributed stochastic projection-free solver for constrained optimization

—This paper proposes a distributed stochastic projection-free algorithm for large-scale constrained ﬁnite-sum optimization whose constraint set is complicated such that the projection onto the constraint set can be expensive. The global cost function is allocated to multiple agents, each of which computes its local stochastic gradients and communicates with its neighbors to solve the global problem. Stochastic gradient methods enable low computational cost, while they are hard and slow to converge due to the variance caused by random sampling. To construct a convergent distributed stochastic projection-free algorithm, this paper incorporates a variance reduction technique and gradient tracking technique in the Frank-Wolfe update. We develop a sampling rule for the variance reduction technique to reduce the variance introduced by stochastic gradients. Com-plete and rigorous proofs show that the proposed distributed projection-free algorithm converges with a sublinear convergence rate and enjoys superior complexity guarantees for both convex and non-convex objective functions. By comparative simulations, we demonstrate the convergence and computational efﬁciency of the proposed algorithm.
Hardness of approximation in p via short cycle removal: cycle detection, distance oracles, and beyond

We present a new technique for efficiently removing almost all short cycles in a graph without unintentionally removing its triangles. Consequently, triangle finding problems do not become easy even in almost k-cycle free graphs, for any constant k≥ 4. Triangle finding is at the base of many conditional lower bounds in P, mainly for distance computation problems, and the existence of many 4- or 5-cycles in a worst-case instance had been the obstacle towards resolving major open questions. Hardness of approximation: Are there distance oracles with m1+o(1) preprocessing time and mo(1) query time that achieve a constant approximation? Existing algorithms with such desirable time bounds only achieve super-constant approximation factors, while only 3− factors were conditionally ruled out (Pătraşcu, Roditty, and Thorup; FOCS 2012). We prove that no O(1) approximations are possible, assuming the 3-SUM or APSP conjectures. In particular, we prove that k-approximations require Ω(m1+1/ck) time, which is tight up to the constant c. The lower bound holds even for the offline version where we are given the queries in advance, and extends to other problems such as dynamic shortest paths. The 4-Cycle problem: An infamous open question in fine-grained complexity is to establish any surprising consequences from a subquadratic or even linear-time algorithm for detecting a 4-cycle in a graph. This is arguably one of the simplest problems without a near-linear time algorithm nor a conditional lower bound. We prove that Ω(m1.1194) time is needed for k-cycle detection for all k≥ 4, unless we can detect a triangle in √n-degree graphs in O(n2−δ) time; a breakthrough that is not known to follow even from optimal matrix multiplication algorithms.
SNE: an Energy-Proportional Digital Accelerator for Sparse Event-Based Convolutions

Event-based sensors are drawing increasing attention due to their high temporal resolution, low power consumption, and low bandwidth. To efficiently extract semantically meaningful information from sparse data streams produced by such sensors, we present a 4.5TOP/s/W digital accelerator capable of performing 4-bits-quantized event-based convolutional neural networks (eCNN). Compared to standard convolutional engines, our accelerator performs a number of operations proportional to the number of events contained into the input data stream, ultimately achieving a high energy-to-information processing proportionality. On the IBM-DVS-Gesture dataset, we report 80uJ/inf to 261uJ/inf, respectively, when the input activity is 1.2% and 4.9%. Our accelerator consumes 0.221pJ/SOP, to the best of our knowledge it is the lowest energy/OP reported on a digital neuromorphic engine.
Graph Gain: A Concave-Hull Based Volumetric Gain for Robotic Exploration

The existing volumetric gain for robotic exploration is calculated in the 3D occupancy map, while the sampling-based exploration method is extended in the reachable (free) space. The inconsistency between them makes the existing calculation of volumetric gain inappropriate for a complete exploration of the environment. To address this issue, we propose a concave-hull based volumetric gain in a sampling-based exploration framework. The concave hull is constructed based on the viewpoints generated by Rapidly-exploring Random Tree (RRT) and the nodes that fail to expand. All space outside this concave hull is considered unknown. The volumetric gain is calculated based on the viewpoints configuration rather than using the occupancy map. With the new volumetric gain, robots can avoid inefficient or even erroneous exploration behavior caused by the inappropriateness of existing volumetric gain calculation methods. Our exploration method is evaluated against the existing state-of-the-art RRT-based method in a benchmark environment. In the evaluated environment, the average running time of our method is about 38.4% of the existing state-of-the-art method and our method is more robust.
Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks

Logical approaches to representing language have developed and evaluated computational models of quantifier words since the 19th century, but today’s NLU models still struggle to capture their semantics. We rely on Generalized Quantifier Theory for language-independent representations of the semantics of quantifier words, to quantify their contribution to the errors of NLU models. We find that quantifiers are pervasive in NLU benchmarks, and their occurrence at test time is associated with performance drops. Multilingual models also exhibit unsatisfying quantifier reasoning abilities, but not necessarily worse for non-English languages. To facilitate directly-targeted probing, we present an adversarial generalized quantifier NLI task (GQNLI) and show that pre-trained language models have a clear lack of robustness in generalized quantifier reasoning.
Sparse and Dense Approaches for the Full-rank Retrieval of Responses for Dialogues

Ranking responses for a given dialogue context is a popular benchmark in which the setup is to re-rank the ground-truth response over a limited set of $n$ responses, where $n$ is typically 10. The predominance of this setup in conversation response ranking has lead to a great deal of attention to building neural re-rankers, while the first-stage retrieval step has been overlooked. Since the correct answer is always available in the candidate list of $n$ responses, this artificial evaluation setup assumes that there is a first-stage retrieval step which is always able to rank the correct response in its top-$n$ list. In this paper we focus on the more realistic task of full-rank retrieval of responses, where $n$ can be up to millions of responses. We investigate both dialogue context and response expansion techniques for sparse retrieval, as well as zero-shot and fine-tuned dense retrieval approaches. Our findings based on three different information-seeking dialogue datasets reveal that a learned response expansion technique is a solid baseline for sparse retrieval. We find the best performing method overall to be dense retrieval with intermediate training, i.e. a step after the language model pre-training where sentence representations are learned, followed by fine-tuning on the target conversational data. We also investigate the intriguing phenomena that harder negatives sampling techniques lead to worse results for the fine-tuned dense retrieval models. The code and datasets are available at https://github.com/Guzpenha/transformer_rankers/tree/full_rank_retrieval_dialogues.
Planarized aluminum metallization for sub-0.5 mu m CMOS technology

The authors describe an aluminum sputter process, called the Al-plug process, which results in complete filling of submicron contacts and vias of various sizes. The process can be done in a conventional sputtering system. Material and electrical characterization of the film and the implementation of this process in submicron integrated circuits are presented. <<ETX>>
Deterministic Self-Adjusting Tree Networks Using Rotor Walks

We revisit the design of self-adjusting single-source tree networks. The problem can be seen as a generalization of the classic list update problem to trees, and finds applications in reconfigurable datacenter networks. We are given a balanced binary tree T connecting n nodes V = {v1,…, vn}. A source node v0, attached to the root of the tree, issues communication requests to nodes in V , in an online and adversarial manner; the access cost of a request to a node v, is given by the current depth of v in T . The online algorithm can try to reduce the access cost by performing swap operations, with which the position of a node is exchanged with the position of its parent in the tree; a swap operation costs one unit. The objective is to design an online algorithm which minimizes the total access cost plus adjustment cost (swapping). Avin et al. [12] (LATIN 2020) recently presented RANDOM-PUSH, a constant competitive online algorithm for this problem, based on random walks, together with a sophisticated analysis exploiting the working set property.This paper studies analytically and empirically, online algorithms for this problem. In particular, we explore how to derandomize RANDOM-PUSH. In the analytical part, we consider a simple derandomized algorithm which we call ROTOR-PUSH, as its behavior is reminiscent of rotor walks. Our first contribution is a proof that ROTOR-PUSH is constant competitive: its competitive ratio is 12 and hence by a factor of five lower than the best existing competitive ratio. Interestingly, in contrast to RANDOM-PUSH, the algorithm does not feature the working set property, which requires a new analysis. We further present a significantly improved and simpler analysis for the randomized algorithm, showing that it is 16-competitive.In the empirical part, we compare all self-adjusting single-source tree networks, using both synthetic and real data. In particular, we shed light on the extent to which these self-adjusting trees can exploit temporal and spatial structure in the workload. Our experimental artefacts and source codes are publicly available.
A Generalization of Ripley's K Function for the Detection of Spatial Clustering in Areal Data

Spatial clustering detection has a variety of applications in diverse ﬁelds, including identifying infectious disease outbreaks, assessing land use patterns, pinpointing crime hotspots, and identifying clusters of neurons in brain imaging applications. While performing spatial clustering analysis on point process data is common, applications to areal data are frequently of interest. For example, researchers might wish to know if census tracts with a case of a rare medical con-dition or an outbreak of an infectious disease tend to cluster together spatially. Since few spatial clustering methods are designed for areal data, researchers often reduce the areal data to point process data (e.g., using the centroid of each areal unit) and apply methods designed for point process data, such as Ripley’s K function or the average nearest neighbor method. However, since these methods were not designed for areal data, a number of issues can arise. For example, we show that they can result in loss of power and/or a signiﬁcantly inﬂated type I error rate. To address these issues, we propose a generalization of Ripley’s K function designed speciﬁcally to detect spatial clustering in areal data. We compare its performance to that of the traditional Ripley’s K function, the average nearest neighbor method, and the spatial scan statistic with an extensive simulation study. We then evaluate the real world performance of the method by using it to detect spatial clustering in land parcels containing conservation easements and US counties with high pediatric overweight/obesity rates.
Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering

Knowledge-based visual question answering (QA) aims to answer a question which requires visually-grounded external knowledge beyond image content itself. Answering complex questions that require multi-hop reasoning under weak supervision is considered as a challenging problem since i) no supervision is given to the reasoning process and ii) high-order semantics of multi-hop knowledge facts need to be captured. In this paper, we introduce a concept of hypergraph to encode high-level semantics of a question and a knowledge base, and to learn high-order associations between them. The proposed model, Hypergraph Transformer, constructs a question hypergraph and a query-aware knowledge hypergraph, and infers an answer by encoding inter-associations between two hypergraphs and intra-associations in both hypergraph itself. Extensive experiments on two knowledge-based visual QA and two knowledge-based textual QA demonstrate the effectiveness of our method, especially for multi-hop reasoning problem. Our source code is available at https://github.com/yujungheo/kbvqa-public.
Emergent Communication for Understanding Human Language Evolution: What's Missing?

Emergent communication protocols among humans and artificial neural network agents do not yet share the same properties and show some critical mismatches in results. We describe three important phenomena with respect to the emergence and benefits of compositionality: ease-of-learning, generalization, and group size effects (i.e., larger groups create more systematic languages). The latter two are not fully replicated with neural agents, which hinders the use of neural emergent communication for language evolution research. We argue that one possible reason for these mismatches is that key cognitive and communicative constraints of humans are not yet integrated. Specifically, in humans, memory constraints and the alternation between the roles of speaker and listener underlie the emergence of linguistic structure, yet these constraints are typically absent in neural simulations. We suggest that introducing such communicative and cognitive constraints would promote more linguistically plausible behaviors with neural agents.
Human Preferences as Dueling Bandits

The dramatic improvements in core information retrieval tasks engendered by neural rankers create a need for novel evaluation methods. If every ranker returns highly relevant items in the top ranks, it becomes difficult to recognize meaningful differences between them and to build reusable test collections. Several recent papers explore pairwise preference judgments as an alternative to traditional graded relevance assessments. Rather than viewing items one at a time, assessors view items side-by-side and indicate the one that provides the better response to a query, allowing fine-grained distinctions. If we employ preference judgments to identify the probably best items for each query, we can measure rankers by their ability to place these items as high as possible. We frame the problem of finding best items as a dueling bandits problem. While many papers explore dueling bandits for online ranker evaluation via interleaving, they have not been considered as a framework for offline evaluation via human preference judgments. We review the literature for possible solutions. For human preference judgments, any usable algorithm must tolerate ties, since two items may appear nearly equal to assessors, and it must minimize the number of judgments required for any specific pair, since each such comparison requires an independent assessor. Since the theoretical guarantees provided by most algorithms depend on assumptions that are not satisfied by human preference judgments, we simulate selected algorithms on representative test cases to provide insight into their practical utility. Based on these simulations, one algorithm stands out for its potential. Our simulations suggest modifications to further improve its performance. Using the modified algorithm, we collect over 10,000 preference judgments for pools derived from submissions to the TREC 2021 Deep Learning Track, confirming its suitability. We test the idea of best-item evaluation and suggest ideas for further theoretical and practical progress.
A Note on the Regularity of Images Generated by Convolutional Neural Networks

The regularity of images generated by convolutional neural networks, such as the U-net, generative networks, or the deep image prior, is analyzed. In a resolution-independent, infinite dimensional setting, it is shown that such images, represented as functions, are always continuous and, in some circumstances, even continuously differentiable, contradicting the widely accepted modeling of sharp edges in images via jump discontinuities. While such statements require an infinite dimensional setting, the connection to (discretized) neural networks used in practice is made by considering the limit as the resolution approaches infinity. As practical consequence, the results of this paper in particular provide analytical evidence that basic L2 regularization of network weights might lead to over-smoothed outputs.
High rectifying performance of heterojunctions with interface between armchair C$_3$N nanoribbons with and without edge H-passivation

Two-dimensional polyaniline with C 3 N stoichiometry, is a newly fabricated layered material that has been expected to possess fascinating electronic, thermal, mechanical and chemical properties. The nature of its counterpart nano-ribbons / structures o ﬀ ering even more tunability in property because of the unique quantum conﬁnement and edge e ﬀ ect, however, has not been revealed su ﬃ ciently. Here, using the ﬁrst-principles calculation based on density functional theory and nonequilibrium Green’s function technique, we ﬁrst perform a study on the electron band structure of armchair C 3 N nanoribbons (AC 3 NNRs) without and with H-passivation. The calculated results show that the pristine AC 3 NNRs are metal, while the H-passivated ones are either direct or indirect band gap semiconductors depending on the detailed edge atomic conﬁgurations. Then we propose a lateral planar homogenous junction with an interface between the pristine and H-passivated AC 3 NNRs, in which forms a Schottky-like barrier. Interestingly, our further transport calculation demonstrates that this AC 3 NNRs-based heterojunction exhibits a good rectiﬁcation behavior. In speciﬁcation, the average rectiﬁcation ratio (RR) can reach up to 10 3 in the bias regime from 0.2 to 0.4 V. Particularly, extending the length of semiconductor part in the heterojunction leads to the decrease of the current through the junction, but the RR can be enlarged obviously. The average RR increases to the order of 10 4 in the bias from 0.25 to 0.40 V, with the boosted maximum up to 10 5 at 0.35 V. The ﬁndings of this work may be serviceable for the design of functional nanodevices based on AC 3 NNRs in the future.
Log-based Sparse Nonnegative Matrix Factorization for Data Representation

Nonnegative matrix factorization (NMF) has been widely studied in recent years due to its eﬀectiveness in representing nonnegative data with parts-based rep-resentations. For NMF, a sparser solution implies better parts-based representation. However, current NMF methods do not always generate sparse solutions. In this paper, we propose a new NMF method with log-norm imposed on the factor matrices to enhance the sparseness. Moreover, we propose a novel column-wisely sparse norm, named (cid:96) 2 , log -(pseudo) norm to enhance the robustness of the proposed method. The (cid:96) 2 , log -(pseudo) norm is invariant, continuous, and diﬀerentiable. For the (cid:96) 2 , log regularized shrinkage problem, we derive a closed-form solution, which can be used for other general problems. Eﬃcient multiplicative updating rules are developed for the optimization, which theoretically guarantees the convergence of the objective value sequence. Extensive experimental results conﬁrm the eﬀectiveness of the proposed method, as well as the enhanced sparseness and robustness.
Three-point intrinsic alignments of dark matter halos in the IllustrisTNG simulation

We use the IllustrisTNG suite of cosmological simulations to measure intrinsic alignment (IA) bispectra of dark matter subhalos between redshifts 0 and 1. We decompose the intrinsic shear field into E- and B-modes and find that the bispectra BδδE and BδEE, between the matter overdensity field, δ, and the E-mode field, are detected with high significance. We also model the IA bispectra analytically using a method consistent with the two-point non-linear alignment model. We use this model and the simulation measurements to infer the intrinsic alignment amplitude AIA and find that values of AIA obtained from IA power spectra and bispectra agree well at scales up to kmax = 2 hMpc−1. For example at z = 1 AIA = 2.13 ± 0.02 from the cross power spectrum between the matter overdensity and E-mode fields and AIA = 2.11 ± 0.03 from BδδE. This demonstrates that a single physically-motivated model can jointly model two-point and three-point statistics of intrinsic alignments, thus enabling a cleaner separation between intrinsic alignments and cosmological weak lensing signals.
Towards Involving End-users in Interactive Human-in-the-loop AI Fairness

Ensuring fairness in artificial intelligence (AI) is important to counteract bias and discrimination in far-reaching applications. Recent work has started to investigate how humans judge fairness and how to support machine learning (ML) experts in making their AI models fairer. Drawing inspiration from an Explainable AI (XAI) approach called \emph{explanatory debugging} used in interactive machine learning, our work explores designing interpretable and interactive human-in-the-loop interfaces that allow ordinary end-users without any technical or domain background to identify potential fairness issues and possibly fix them in the context of loan decisions. Through workshops with end-users, we co-designed and implemented a prototype system that allowed end-users to see why predictions were made, and then to change weights on features to"debug"fairness issues. We evaluated the use of this prototype system through an online study. To investigate the implications of diverse human values about fairness around the globe, we also explored how cultural dimensions might play a role in using this prototype. Our results contribute to the design of interfaces to allow end-users to be involved in judging and addressing AI fairness through a human-in-the-loop approach.
Development of an algorithm for medical image segmentation of bone tissue in interaction with metallic implants

This preliminary study focuses on the development of a medical image segmentation algorithm based on artiﬁcial intelligence for calculating bone growth in contact with metallic implants. Two databases consisting of computerized microtomography images have been used throughout this work: 100 images for training and 196 images for testing. Both bone and implant tissue were manually segmented in the training data set. The type of network constructed follows the U-Net architecture, a convolutional neural network explicitly used for medical image segmentation. In terms of network accuracy, the model reached around 98%. Once the prediction was obtained from the new data set (test set), the total number of pixels belonging to bone tissue was calculated. This volume is around 15% of the volume estimated by conventional techniques, which are usually overestimated. This method has shown its good performance and results, although it has a wide margin for improvement, modifying various parameters of the networks or using larger databases to improve training.