Miracle-dz/UniFAR-FTUs huggingface dataset:
Botnets Breaking Transformers: Localization of Power Botnet Attacks Against the Distribution Grid

Traditional botnet attacks leverage large and distributed numbers of compromised internet-connected devices to target and overwhelm other devices with internet packets. With increasing consumer adoption of high-wattage internet-facing"smart devices", a new"power botnet"attack emerges, where such devices are used to target and overwhelm power grid devices with unusual load demand. We introduce a variant of this attack, the power-botnet weardown-attack, which does not intend to cause blackouts or short-term acute instability, but instead forces expensive mechanical components to activate more frequently, necessitating costly replacements / repairs. Specifically, we target the on-load tap-changer (OLTC) transformer, which uses a mechanical switch that responds to change in load demand. In our analysis and simulations, these attacks can halve the lifespan of an OLTC, or in the most extreme cases, reduce it to $2.5\%$ of its original lifespan. Notably, these power botnets are composed of devices not connected to the internal SCADA systems used to control power grids. This represents a new internet-based cyberattack that targets the power grid from the outside. To help the power system to mitigate these types of botnet attacks, we develop attack-localization strategies. We formulate the problem as a supervised machine learning task to locate the source of power botnet attacks. Within a simulated environment, we generate the training and testing dataset to evaluate several machine learning algorithm based localization methods, including SVM, neural network and decision tree. We show that decision-tree based classification successfully identifies power botnet attacks and locates compromised devices with at least $94\%$ improvement of accuracy over a baseline"most-frequent"classifier.
Multi-Domain Multi-Definition Landmark Localization for Small Datasets

We present a novel method for multi image domain and multi-landmark definition learning for small dataset facial localization. Training a small dataset alongside a large(r) dataset helps with robust learning for the former, and provides a universal mechanism for facial landmark localization for new and/or smaller standard datasets. To this end, we propose a Vision Transformer encoder with a novel decoder with a definition agnostic shared landmark semantic group structured prior, that is learnt, as we train on more than one dataset concurrently. Due to our novel definition agnostic group prior the datasets may vary in landmark definitions and domains. During the decoder stage we use cross- and self-attention, whose output is later fed into domain/definition specific heads that minimize a Laplacian-log-likelihood loss. We achieve state-of-the-art performance on standard landmark localization datasets such as COFW and WFLW, when trained with a bigger dataset. We also show state-of-the-art performance on several varied image domain small datasets for animals, caricatures, and facial portrait paintings. Further, we contribute a small dataset (150 images) of pareidolias to show efficacy of our method. Finally, we provide several analysis and ablation studies to justify our claims.
Learning Neural Contextual Bandits through Perturbed Rewards

Thanks to the power of representation learning, neural contextual bandit algorithms demonstrate remarkable performance improvement against their classical counterparts. But because their exploration has to be performed in the entire neural network parameter space to obtain nearly optimal regret, the resulting computational cost is prohibitively high. We perturb the rewards when updating the neural network to eliminate the need of explicit exploration and the corresponding computational overhead. We prove that a $\tilde{O}(\tilde{d}\sqrt{T})$ regret upper bound is still achievable under standard regularity conditions, where $T$ is the number of rounds of interactions and $\tilde{d}$ is the effective dimension of a neural tangent kernel matrix. Extensive comparisons with several benchmark contextual bandit algorithms, including two recent neural contextual bandit models, demonstrate the effectiveness and computational efficiency of our proposed neural bandit algorithm.
EVALUATION OF THRIPS MANAGEMENT OPTIONS IN COTTON

Thrips are early-season pests that have the potential of causing 50-60% yield reduction in Arkansas cotton with the level of damage varying from year to year. The objective of this experiment was to evaluate seed treatment, in-furrow, and foliar insecticides for thrips management in cotton. Experiments were conducted in Lonoke Co. and Lee Co., AR, in 1999 and 2000, respectively. Thrips samples and ratings were taken at weekly intervals four times each year early in the growing season. In both years, all treatments significantly improved thrips control above that of the untreated check. In 1999, the Temik treatments outperformed the others with respect to thrips suppression, visual damage rating, and cotton yield. While thrips suppression was not significant among treatments in 2000, the Temik treatments achieved higher yields. The data presented from these growing seasons indicate Temik to be one of the best treatments for thrips control in Arkansas cotton.
Countable Ramsey

The celebrated Erd˝os–Hajnal Conjecture says that in any proper hereditary class of finite graphs we are guaranteed to have a clique or anti-clique of size n c , which is a much better bound than the logarithmic size that is provided by Ramsey’s Theorem in general. On the other hand, in uncountable cardinalities, the model-theoretic property of stability guarantees a uniform set much larger than the bound provided by the Erd˝os–Rado Theorem in general.Even though the consequences of stability in the finite have been much studied in the literature, the countable setting seems a priori quite different, namely, in the countably infinite the notion of largeness based on cardinality alone does not reveal any structure as Ramsey’s Theorem already provides a countably infinite uniform set in general. In this paper, we show that the natural notion of largeness given by upper density reveals that these phenomena meet in the countable: a countable graph has an almost clique or anti-clique of positive upper density if and only if it has a positive upper density almost stable set. Moreover, this result also extends naturally to countable models of a universal theory in a finite relational language. Our methods explore a connection with the notion of convergence in the theory of limits of dense combinatorial objects, introducing and studying a natural approximate version of the Erd˝os–Hajnal property that allows for a negligible error in the edges (in general, predicates) but requires linear-sized uniform sets in convergent sequences of models (this is much stronger than what stable regularity can provide as the error is required to go to zero). Finally, surprisingly, we completely characterize all hereditary classes of finite graphs that have this approximate Erd˝os–Hajnal property. The proof highlights both differences and similarities with the original conjecture.
Should Radar Replace Lidar in All-Weather Mapping and Localization?

Radar is a rich sensing modality that is a compelling alternative to lidar for its inherent robustness to precipitation, dust, and fog. In this paper, we investigate radarbased mapping and localization as an alternative to lidar by demonstrating the performance of our own radar-based pipeline. We present extensive comparisons against a lidarbased pipeline across varying seasonal and weather conditions using our own publicly available dataset collected on a repeated route over the course of one year. Our experiments show that while lidar achieves better overall localization accuracy, our radar-based pipeline achieves comparable SE(2) results using a fraction of the computational and memory resources required by the lidar-based pipeline. We additionally demonstrate our radar-based pipeline localizing against lidar maps with better accuracy than the current state of the art for radar-to-lidar localization. Code for this project can be found at: https: //github.com/utiasASRL/vtr3
Application of Quantum Density Matrix in Classical Question Answering and Classical Image Classification

: Quantum density matrix represents all the information of the entire quantum system, and novel models of meaning employing density matrices naturally model linguistic phenomena such as hyponymy and linguistic ambiguity, among others in quantum question answering tasks. Naturally, we argue that applying the quantum density matrix into classical Question Answering (QA) tasks can show more effective performance. Specifically, we (i) design a new mechanism based on Long Short-Term Memory (LSTM) to accommodate the case when the inputs are matrixes; (ii) apply the new mechanism to QA problems with Convolutional Neural Network (CNN) and gain the LSTM-based QA model with the quantum density matrix. Experiments of our new model on TREC-QA and WIKI-QA data sets show encouraging results. Similarly, we argue that the quantum density matrix can also enhance the image feature information and the relationship between the features for the classical image classification. Thus, we (i) combine density matrices and CNN to design a new mechanism; (ii) apply the new mechanism to some representative classical image classification tasks. A series of experiments show that the application of quantum density matrix in image classification has the generalization and high efficiency on different datasets. The application of quantum density matrix both in classical question answering tasks and classical image classification tasks show more effective performance.
Thermal Comptonization in a Changing Corona in the Changing-look Active Galaxy NGC 1566

We present broadband UV/X-ray spectral variability of the changing-look active galactic nucleus (AGN) NGC 1566, based on simultaneous near-ultraviolet and X-ray observations performed by the XMM-Newton, Swift, and NuSTAR satellites at five different epochs during the declining phase of the 2018 outburst. We found that the accretion disk, soft X-ray excess, and X-ray power-law components were extremely variable. Additionally, the X-ray power-law flux was correlated with both the soft excess plus disk and the pure disk fluxes. Our finding shows that at high-flux levels the soft X-ray excess and the disk emission both provided the seed photons for thermal Comptonization in the hot corona, whereas at low-flux levels, where the soft excess was absent, the pure disk emission alone provided the seed photons. The X-ray power-law photon index was only weakly variable (ΔΓhot ≤ 0.06), and it was not well correlated with the X-ray flux over the declining timescale. On the other hand, we found that the electron temperature of the corona increased from ∼22 to ∼200 keV with the decreasing numbers of seed photons from 2018 June to 2019 August. At the same time, the optical depth of the corona decreased from τ hot ∼ 4 to ∼0.7, and the scattering fraction increased from ∼1% to ∼10%. These changes suggest structural changes in the hot corona, such as it was growing in size and becoming hotter with the decreasing accretion rate during the declining phase. The AGN is most likely evolving with a decreasing accretion rate toward a state similar to the low/hard state of black hole X-ray binaries.
Explicit Quantum Circuits for Block Encodings of Certain Sparse Matrice

Many standard linear algebra problems can be solved on a quantum computer by using recently developed quantum linear algebra algorithms that make use of block encodings and quantum eigenvalue/singular value transformations. A block encoding embeds a properly scaled matrix of interest A in a larger unitary transformation U that can be decomposed into a product of simpler unitaries and implemented efficiently on a quantum computer. Although quantum algorithms can potentially achieve exponential speedup in solving linear algebra problems compared to the best classical algorithm, such gain in efficiency ultimately hinges on our ability to construct an efficient quantum circuit for the block encoding of A, which is difficult in general, and not trivial even for well-structured sparse matrices. In this paper, we give a few examples on how efficient quantum circuits can be explicitly constructed for some well-structured sparse matrices, and discuss a few strategies used in these constructions. We also provide implementations of these quantum circuits in MATLAB.
XTREME-S: Evaluating Cross-lingual Speech Representations

We introduce XTREME-S, a new benchmark to evaluate universal cross-lingual speech representations in many languages. XTREME-S covers four task families: speech recognition, classification, speech-to-text translation and retrieval. Covering 102 languages from 10+ language families, 3 different domains and 4 task families, XTREME-S aims to simplify multilingual speech representation evaluation, as well as catalyze research in"universal"speech representation learning. This paper describes the new benchmark and establishes the first speech-only and speech-text baselines using XLS-R and mSLAM on all downstream tasks. We motivate the design choices and detail how to use the benchmark. Datasets and fine-tuning scripts are made easily accessible at https://hf.co/datasets/google/xtreme_s.
Dense Siamese Network

This paper presents Dense Siamese Network (Dens-eSiam), a simple unsupervised learning framework for dense prediction tasks. It learns visual representations by maximizing the similarity between two views of one image with two types of consistency, i.e . , pixel consistency and region consistency. Concretely, DenseSiam ﬁrst maximizes the pixel level spatial consistency according to the exact location correspondence in the overlapped area. It also extracts a batch of region embeddings that correspond to some sub-regions in the overlapped area to be contrasted for region consistency. In contrast to previous methods that require negative pixel pairs, momentum encoders or heuristic masks, DenseSiam beneﬁts from the simple Siamese network and optimizes the consistency of different granularities. It also proves that the simple location correspondence and interacted region embeddings are effective enough to learn the similarity. We apply DenseSiam on ImageNet and obtain competitive improvements on various downstream tasks. We also show that only with some extra task-speciﬁc losses, the simple framework can directly conduct dense prediction tasks. On an existing unsupervised semantic segmentation benchmark, it surpasses state-of-the-art segmentation methods by 2.1 mIoU with 28% training costs.
Masked Discrimination for Self-Supervised Learning on Point Clouds

Masked autoencoding has achieved great success for self-supervised learning in the image and language domains. However, mask based pretraining has yet to show benefits for point cloud understanding, likely due to standard backbones like PointNet being unable to properly handle the training versus testing distribution mismatch introduced by masking during training. In this paper, we bridge this gap by proposing a discriminative mask pretraining Transformer framework, MaskPoint}, for point clouds. Our key idea is to represent the point cloud as discrete occupancy values (1 if part of the point cloud; 0 if not), and perform simple binary classification between masked object points and sampled noise points as the proxy task. In this way, our approach is robust to the point sampling variance in point clouds, and facilitates learning rich representations. We evaluate our pretrained models across several downstream tasks, including 3D shape classification, segmentation, and real-word object detection, and demonstrate state-of-the-art results while achieving a significant pretraining speedup (e.g., 4.1x on ScanNet) compared to the prior state-of-the-art Transformer baseline. Code is available at https://github.com/haotian-liu/MaskPoint.
Attention Aided CSI Wireless Localization

Deep neural networks (DNNs) have become a popular approach for wireless localization based on channel state information (CSI). A common practice is to use the raw CSI in the input and allow the network to learn relevant channel representations for mapping to location information. However, various works show that raw CSI can be very sensitive to system impairments and small changes in the environment. On the contrary, hand-designing features may hinder the limits of channel representation learning of the DNN. In this work, we propose attention-based CSI for robust feature learning. We evaluate the performance of attended features in centralized and distributed massive MIMO systems for ray-tracing channels in two non-stationary railway track environments. By comparison to a base DNN, our approach provides exceptional performance.
Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work

Creating presentation slides is a critical but time-consuming task for data scientists. While researchers have proposed many AI techniques to lift data scientists’ burden on data preparation and model selection, few have targeted the presentation creation task. Based on the needs identified from a formative study, this paper presents NB2Slides, an AI system that facilitates users to compose presentations of their data science work. NB2Slides uses deep learning methods as well as example-based prompts to generate slides from computational notebooks, and take users’ input (e.g., audience background) to structure the slides. NB2Slides also provides an interactive visualization that links the slides with the notebook to help users further edit the slides. A follow-up user evaluation with 12 data scientists shows that participants believed NB2Slides can improve efficiency and reduces the complexity of creating slides. Yet, participants questioned the future of full automation and suggested a human-AI collaboration paradigm.
FaiRR: Faithful and Robust Deductive Reasoning over Natural Language

Transformers have been shown to be able to perform deductive reasoning on a logical rulebase containing rules and statements written in natural language. Recent works show that such models can also produce the reasoning steps (i.e., the proof graph) that emulate the model’s logical reasoning process. Currently, these black-box models generate both the proof graph and intermediate inferences within the same model and thus may be unfaithful. In this work, we frame the deductive logical reasoning task by defining three modular components: rule selection, fact selection, and knowledge composition. The rule and fact selection steps select the candidate rule and facts to be used and then the knowledge composition combines them to generate new inferences. This ensures model faithfulness by assured causal relation from the proof step to the inference reasoning. To test our framework, we propose FaiRR (Faithful and Robust Reasoner) where the above three components are independently modeled by transformers. We observe that FaiRR is robust to novel language perturbations, and is faster at inference than previous works on existing reasoning datasets. Additionally, in contrast to black-box generative models, the errors made by FaiRR are more interpretable due to the modular approach.
Learning Affordance Grounding from Exocentric Images

Affordance grounding, a task to ground (i.e., localize) action possibility region in objects, which faces the challenge of establishing an explicit link with object parts due to the diversity of interactive affordance. Human has the ability that transform the various exocentric interactions to invariant egocentric affordance so as to counter the impact of interactive diversity. To empower an agent with such ability, this paper proposes a task of affordance grounding from exocentric view, i.e., given exocentric human-object interaction and egocentric object images, learning the affordance knowledge of the object and transferring it to the egocentric image using only the affordance label as supervision. To this end, we devise a cross-view knowledge transfer framework that extracts affordance-specific features from exocentric interactions and enhances the perception of affordance regions by preserving affordance correlation. Specifically, an Affordance Invariance Mining module is devised to extract specific clues by minimizing the intra-class differences originated from interaction habits in exocentric images. Besides, an Affordance Co-relation Preserving strategy is presented to perceive and localize affordance by aligning the co-relation matrix of predicted results between the two views. Particularly, an affordance grounding dataset named AGD20K is constructed by collecting and labeling over 20K images from 36 affordance categories. Experimental results demonstrate that our method outperforms the representative models in terms of objective metrics and visual quality. Code: github.com/lhc1224/Cross-View-AG.
Propagation of regularity for transport equations. A Littlewood-Paley approach

It is known that linear advection equations with Sobolev velocity fields have very poor regularity properties: Solutions propagate only derivatives of logarithmic order, which can be measured in terms of suitable Gagliardo seminorms. We propose a new approach to the study of regularity that is based on Littlewood-Paley theory, thus measuring regularity in terms of Besov norms. We recover the results that are available in the literature and extend these optimally to the diffusive setting. As a consequence, we derive sharp bounds on rates of convergence in the zero-diffusivity limit.
YouTube, The Great Radicalizer? Auditing and Mitigating Ideological Biases in YouTube Recommendations

Recommendations algorithms of social media platforms are often criticized for placing users in"rabbit holes"of (increasingly) ideologically biased content. Despite these concerns, prior evidence on this algorithmic radicalization is inconsistent. Furthermore, prior work lacks systematic interventions that reduce the potential ideological bias in recommendation algorithms. We conduct a systematic audit of YouTube's recommendation system using a hundred thousand sock puppets to determine the presence of ideological bias (i.e., are recommendations aligned with users' ideology), its magnitude (i.e., are users recommended an increasing number of videos aligned with their ideology), and radicalization (i.e., are the recommendations progressively more extreme). Furthermore, we design and evaluate a bottom-up intervention to minimize ideological bias in recommendations without relying on cooperation from YouTube. We find that YouTube's recommendations do direct users -- especially right-leaning users -- to ideologically biased and increasingly radical content on both homepages and in up-next recommendations. Our intervention effectively mitigates the observed bias, leading to more recommendations to ideologically neutral, diverse, and dissimilar content, yet debiasing is especially challenging for right-leaning users. Our systematic assessment shows that while YouTube recommendations lead to ideological bias, such bias can be mitigated through our intervention.