Miracle-dz/UniFAR-FTUs huggingface dataset:
Fabrication of Metasurfaces on Building Construction Materials for Potential Electromagnetic Applications in the Microwave Band

Energy self-sufficiency, as well as optimal management of power in buildings is gaining importance, while obtaining power from traditional fossil energy sources is becoming more and more expensive. In this context, millimeter-scale metasurfaces can be employed to harvest energy from microwave sources. They can also be used as sensors in the microwave regime for efficient power management solutions. In the current study, a simple spray printing method is proposed to develop metasurfaces in construction materials, i.e., plasterboard and wood. Such materials are used in the interior design of buildings; therefore, the implementation of metasurfaces in large areas, such as walls, doors and floors, is realized. The fabricated metasurfaces were characterized regarding their electromagnetic performance. It is hereby shown that the investigated metasurfaces exhibit an efficient electromagnetic response in the frequency range (4–7 GHz), depending on the MS. Thus, spray-printed metasurfaces integrated on construction materials can potentially be used for electromagnetic applications, for buildings’ power self-efficiency and management.
A Logical Framework with Higher-Order Rational (Circular) Terms

Logical frameworks provide natural and direct ways of specifying and reasoning within deductive systems. The logical framework LF and subsequent developments focus on finitary proof systems, making the formalization of circular proof systems in such logical frameworks a cumbersome and awkward task. To address this issue, we propose CoLF, a conservative extension of LF with higher-order rational terms and mixed inductive and coinductive definitions. In this framework, two terms are equal if they unfold to the same infinite regular B\"ohm tree. Both term equality and type checking are decidable in CoLF. We illustrate the elegance and expressive power of the framework with several small case studies.
Detecting A Single Change-point

This chapter overviews some of the work on detecting and estimating the location of a single change. We first consider the most common change-point problem, namely that of detecting a change in mean, before looking at extensions to detecting other types of change. The intuition from the problem of detecting a single change-point is helpful for understanding the variety of methods for detecting multiple changes.
Sapling Similarity outperforms other local similarity metrics in collaborative filtering

Many bipartite networks describe systems where a link represents a relation between a user and an item. Measuring the similarity between either users or items is the basis of memory-based collaborative ﬁltering, a widely used method to build a recommender system with the purpose of proposing items to users. When the edges of the network are unweighted, traditional approaches allow only positive similarity values, so neglecting the possibility and the e ﬀ ect of two users (or two items) being very dissimilar. Here we propose a method to compute similarity that allows also negative values, the Sapling Similarity. The key idea is to look at how the information that a user is connected to an item inﬂuences our prior estimation of the probability that another user is connected to the same item: if it is reduced, then the similarity between the two users will be negative, otherwise it will be positive. Using di ﬀ erent datasets, we show that the Sapling Similarity outperforms other similarity metrics when it is used to recommend new items to users.
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting

Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks. We propose MAPL, a simple and parameter-efficient method that reuses frozen pre-trained unimodal models and leverages their strong generalization capabilities in multimodal vision-language (VL) settings. MAPL learns a lightweight mapping between the representation spaces of unimodal models using aligned image-text data, and can generalize to unseen VL tasks from just a few in-context examples. The small number of trainable parameters makes MAPL effective at low-data and in-domain learning. Moreover, MAPL’s modularity enables easy extension to other pre-trained models. Extensive experiments on several visual question answering and image captioning benchmarks show that MAPL achieves superior or competitive performance compared to similar methods while training orders of magnitude fewer parameters. MAPL can be trained in just a few hours using modest computational resources and public datasets. We release our code and pre-trained model weights at https://github.com/oscmansan/mapl.
Biologically plausible solutions for spiking networks with efficient coding

Understanding how the dynamics of neural networks is shaped by the computations they perform is a fundamental question in neuroscience. Recently, the framework of efficient coding proposed a theory of how spiking neural networks can compute low-dimensional stimulus signals with high efficiency. Efficient spiking networks are based on time-dependent minimization of a loss function related to information coding with spikes. To inform the understanding of the function and dynamics of biological networks in the brain, however, the mathematical models have to be informed by biology and obey the same constraints as biological networks. Currently, spiking network models of efficient coding have been extended to include some features of biological plausibility, such as architectures with excitatory and inhibitory neurons. However, biological realism of efficient coding theories is still limited to simple cases and does not include single neuron and network properties that are known to be key in biological circuits. Here, we revisit the theory of efficient coding with spikes to develop spiking neural networks that are closer to biological circuits. Namely, we find a biologically plausible spiking model realizing efficient coding in the case of a generalized leaky integrate-and-fire network with excitatory and inhibitory units, equipped with fast and slow synaptic currents, local homeostatic currents such as spike-triggered adaptation, hyperpolarization-activated rebound current, heterogeneous firing thresholds and resets, heterogeneous postsynaptic potentials, and structured, low-rank connectivity. We show how the complexity of E-E connectivity matrix shapes network responses.
Structural Pruning via Latency-Saliency Knapsack

Structural pruning can simplify network architecture and improve inference speed. We propose Hardware-Aware Latency Pruning (HALP) that formulates structural pruning as a global resource allocation optimization problem, aiming at maximizing the accuracy while constraining latency under a predefined budget on targeting device. For filter importance ranking, HALP leverages latency lookup table to track latency reduction potential and global saliency score to gauge accuracy drop. Both metrics can be evaluated very efficiently during pruning, allowing us to reformulate global structural pruning under a reward maximization problem given target constraint. This makes the problem solvable via our augmented knapsack solver, enabling HALP to surpass prior work in pruning efficacy and accuracy-efficiency trade-off. We examine HALP on both classification and detection tasks, over varying networks, on ImageNet and VOC datasets, on different platforms. In particular, for ResNet-50/-101 pruning on ImageNet, HALP improves network throughput by $1.60\times$/$1.90\times$ with $+0.3\%$/$-0.2\%$ top-1 accuracy changes, respectively. For SSD pruning on VOC, HALP improves throughput by $1.94\times$ with only a $0.56$ mAP drop. HALP consistently outperforms prior art, sometimes by large margins. Project page at https://halp-neurips.github. io/.
HIGH-RESOLUTION ROTATION CURVES AND GALAXY MASS MODELS FROM THINGS

We present rotation curves of 19 galaxies from The H i Nearby Galaxy Survey (THINGS). The high spatial and velocity resolution of THINGS make these the highest quality H i rotation curves available to date for a large sample of nearby galaxies, spanning a wide range of H i masses and luminosities. The high quality of the data allows us to derive the geometric and dynamical parameters using H i data alone. We do not find any declining rotation curves unambiguously associated with a cut-off in the mass distribution out to the last measured point. The rotation curves are combined with 3.6 μm data from the Spitzer Infrared Nearby Galaxies Survey to construct mass models. Our best-fit dynamical disk masses, derived from the rotation curves, are in good agreement with photometric disk masses derived from the 3.6 μm images in combination with stellar population synthesis arguments and two different assumptions for the stellar initial mass function (IMF). We test the cold dark matter (CDM) motivated cusp model, and the observationally motivated central density core model and find that (independent of IMF) for massive, disk-dominated galaxies, all halo models fit apparently equally well; for low-mass galaxies, however, a core-dominated halo is clearly preferred over a cusp-like halo. The empirically derived densities of the dark matter halos of the late-type galaxies in our sample are half of what is predicted by CDM simulations, again independent of the assumed IMF.
How to Train Vision Transformer on Small-scale Datasets?

Vision Transformer (ViT), a radically different architecture than convolutional neural networks offers multiple advantages including design simplicity, robustness and state-of-the-art performance on many vision tasks. However, in contrast to convolutional neural networks, Vision Transformer lacks inherent inductive biases. Therefore, successful training of such models is mainly attributed to pre-training on large-scale datasets such as ImageNet with 1.2M or JFT with 300M images. This hinders the direct adaption of Vision Transformer for small-scale datasets. In this work, we show that self-supervised inductive biases can be learned directly from small-scale datasets and serve as an effective weight initialization scheme for fine-tuning. This allows to train these models without large-scale pre-training, changes to model architecture or loss functions. We present thorough experiments to successfully train monolithic and non-monolithic Vision Transformers on five small datasets including CIFAR10/100, CINIC10, SVHN, Tiny-ImageNet and two fine-grained datasets: Aircraft and Cars. Our approach consistently improves the performance of Vision Transformers while retaining their properties such as attention to salient regions and higher robustness. Our codes and pre-trained models are available at: https://github.com/hananshafi/vits-for-small-scale-datasets.
Multiplane NeRF-Supervised Disentanglement of Depth and Camera Pose from Videos

We propose to perform self-supervised disentanglement of depth and camera pose from large-scale videos. We introduce an Autoencoder-based method to reconstruct the input video frames for training, without using any ground-truth annotations of depth and camera. The model encoders estimate the monocular depth and the camera pose. The decoder then constructs a Multiplane NeRF representation based on the depth encoder feature
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities

Although deep learning has made great progress in recent years, the exploding economic and environmental costs of training neural networks are becoming unsustainable. To address this problem, there has been a great deal of research on *algorithmically-efficient deep learning*, which seeks to reduce training costs not at the hardware or implementation level, but through changes in the semantics of the training program. In this paper, we present a structured and comprehensive overview of the research in this field. First, we formalize the *algorithmic speedup* problem, then we use fundamental building blocks of algorithmically efficient training to develop a taxonomy. Our taxonomy highlights commonalities of seemingly disparate methods and reveals current research gaps. Next, we present evaluation best practices to enable comprehensive, fair, and reliable comparisons of speedup techniques. To further aid research and applications, we discuss common bottlenecks in the training pipeline (illustrated via experiments) and offer taxonomic mitigation strategies for them. Finally, we highlight some unsolved research challenges and present promising future directions.
Towards a Unified Multi-Dimensional Evaluator for Text Generation

Multi-dimensional evaluation is the dominant paradigm for human evaluation in Natural Language Generation (NLG), i.e., evaluating the generated text from multiple explainable dimensions, such as coherence and fluency. However, automatic evaluation in NLG is still dominated by similarity-based metrics, and we lack a reliable framework for a more comprehensive evaluation of advanced models. In this paper, we propose a unified multi-dimensional evaluator UniEval for NLG. We re-frame NLG evaluation as a Boolean Question Answering (QA) task, and by guiding the model with different questions, we can use one evaluator to evaluate from multiple dimensions. Furthermore, thanks to the unified Boolean QA format, we are able to introduce an intermediate learning phase that enables UniEval to incorporate external knowledge from multiple related tasks and gain further improvement. Experiments on three typical NLG tasks show that UniEval correlates substantially better with human judgments than existing metrics. Specifically, compared to the top-performing unified evaluators, UniEval achieves a 23% higher correlation on text summarization, and over 43% on dialogue response generation. Also, UniEval demonstrates a strong zero-shot learning ability for unseen evaluation dimensions and tasks. Source code, data, and all pre-trained evaluators are available at https://github.com/maszhongming/UniEval.
Language Models of Code are Few-Shot Commonsense Learners

We address the general task of structured commonsense reasoning: given a natural language input, the goal is to generate a graph such as an event or a reasoning-graph.To employ large language models (LMs) for this task, existing approaches ‘serialize’ the output graph as a flat list of nodes and edges.Although feasible, these serialized graphs strongly deviate from the natural language corpora that LMs were pre-trained on, hindering LMs from generating them correctly. In this paper, we show that when we instead frame structured commonsense reasoning tasks as code generation tasks, pre-trained LMs of code are better structured commonsense reasoners than LMs of natural language, even when the downstream task does not involve source code at all.We demonstrate our approach across three diverse structured commonsense reasoning tasks. In all these natural language tasks, we show that using our approach, a code generation LM (codex) outperforms natural-LMs that are fine-tuned on the target task (T5) and other strong LMs such as GPT-3 in the few-shot setting.
Schatten Class Hankel Operators on Weighted Bergman Spaces induced by regular weights

In this paper, for $1\leq p<\infty$, we provide several descriptions of Schatten $p$-class Hankel operators $H_f$ and $H_{\overline{f}}$ on the weight Bergman space $A^2_\omega$, in terms of a certain global and local mean oscillation of the symbol $f\in L^2_\omega$, provided $\omega$ is a class of regular weights. The approaches applied to rely on several classical methods, and simultaneously rely on a novel but more convenient construction associated with the atomic decomposition of $A^2_\omega$.
Explanations from Large Language Models Make Small Reasoners Better

Integrating free-text explanations to in-context learning of large language models (LLM) is shown to elicit strong reasoning capabilities along with reasonable explanations. In this paper, we consider the problem of leveraging the explanations generated by LLM to improve the training of small reasoners, which are more favorable in real-production deployment due to their low cost. We systematically explore three explanation generation approaches from LLM and utilize a multi-task learning framework to facilitate small models to acquire strong reasoning power together with explanation generation capabilities. Experiments on multiple reasoning tasks show that our method can consistently and significantly outperform finetuning baselines across different settings, and even perform better than finetuning/prompting a 60x larger GPT-3 (175B) model by up to 9.5% in accuracy. As a side benefit, human evaluation further shows that our method can generate high-quality explanations to justify its predictions, moving towards the goal of explainable AI.
Detection of barium in the atmospheres of the ultra-hot gas giants WASP-76b and WASP-121b

Context. High-resolution spectroscopy studies of ultra-hot Jupiters have been key in our understanding of exoplanet atmospheres. Observing into the atmospheres of these giant planets allows for direct constraints on their atmospheric compositions and dynamics while laying the groundwork for new research regarding their formation and evolution environments. Aims. Two of the most well-studied ultra-hot Jupiters are WASP-76b and WASP-121b, with multiple detected chemical species and strong signatures of their atmospheric dynamics. We take a new look at these two exceptional ultra-hot Jupiters by reanalyzing the transit observations taken with ESPRESSO at the Very Large Telescope and attempt to detect additional species. Methods. To extract the planetary spectra of the two targets, we corrected for the telluric absorption and removed the stellar spectrum contributions. We then exploited new synthetic templates that were specifically designed for ultra-hot Jupiters in combination with the cross-correlation technique to unveil species that remained undetected by previous analyses. Results. We add a novel detection of Ba+ to the known atmospheric compositions of WASP-76b and WASP-121b, the heaviest species detected to date in any exoplanetary atmosphere, with additional new detections of Co and Sr+ and a tentative detection of Ti+ for WASP-121b. We also confirm the presence of Ca+, Cr, Fe, H, Li, Mg, Mn, Na, and V on both WASP-76b and WASP-121b, with the addition of Ca, Fe+, and Ni for the latter. Finally, we also confirm the clear asymmetric absorption feature of Ca+ on WASP-121b, with an excess absorption at the bluer wavelengths and an effective planet radius beyond the Roche lobe. This indicates that the signal may arise from the escape of planetary atmosphere.
SOLNP+: A Derivative-Free Solver for Constrained Nonlinear Optimization

SOLNP+ is a derivative-free solver for constrained nonlinear optimization. It starts from SOLNP proposed in 1989 by Ye Ye with the main idea that uses ﬁnite diﬀerence to approximate the gradient. We incorporate the techniques of implicit ﬁltering, new restart mechanism and modern quadratic programming solver into this new version with an ANSI C implementation. The algorithm exhibits a great advantage in running time and robustness under noise compared with the last version by MATLAB. SOLNP+ is free to download at https: //github.com/COPT-Public/SOLNP_plus .
Language Agnostic Multilingual Information Retrieval with Contrastive Learning

Multilingual information retrieval (IR) is challenging since annotated training data is costly to obtain in many languages. We present an effective method to train multilingual IR systems when only English IR training data and some parallel corpora between English and other languages are available. We leverage parallel and non-parallel corpora to improve the pretrained multilingual language models' cross-lingual transfer ability. We design a semantic contrastive loss to align representations of parallel sentences that share the same semantics in different languages, and a new language contrastive loss to leverage parallel sentence pairs to remove language-specific information in sentence representations from non-parallel corpora. When trained on English IR data with these losses and evaluated zero-shot on non-English data, our model demonstrates significant improvement to prior work on retrieval performance, while it requires much less computational effort. We also demonstrate the value of our model for a practical setting when a parallel corpus is only available for a few languages, but a lack of parallel corpora resources persists for many other low-resource languages. Our model can work well even with a small number of parallel sentences, and be used as an add-on module to any backbones and other tasks.
Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions

A deep neural network using rectified linear units represents a continuous piecewise linear (CPWL) function and vice versa. Recent results in the literature estimated that the number of neurons needed to exactly represent any CPWL function grows exponentially with the number of pieces or exponentially in terms of the factorial of the number of distinct linear components. Moreover, such growth is amplified linearly with the input dimension. These existing results seem to indicate that the cost of representing a CPWL function is expensive. In this paper, we propose much tighter bounds and establish a polynomial time algorithm to find a network satisfying these bounds for any given CPWL function. We prove that the number of hidden neurons required to exactly represent any CPWL function is at most a quadratic function of the number of pieces. In contrast to all previous results, this upper bound is invariant to the input dimension. Besides the number of pieces, we also study the number of distinct linear components in CPWL functions. When such a number is also given, we prove that the quadratic complexity turns into bilinear, which implies a lower neural complexity because the number of distinct linear components is always not greater than the minimum number of pieces in a CPWL function. When the number of pieces is unknown, we prove that, in terms of the number of distinct linear components, the neural complexities of any CPWL function are at most polynomial growth for low-dimensional inputs and factorial growth for the worst-case scenario, which are significantly better than existing results in the literature.
Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition

Deep learning models have shown their vulnerability when dealing with adversarial attacks. Existing attacks almost perform on low-level instances, such as pixels and super-pixels, and rarely exploit semantic clues. For face recognition attacks, existing methods typically generate the l_p-norm perturbations on pixels, however, resulting in low attack transferability and high vulnerability to denoising defense models. In this work, instead of performing perturbations on the low-level pixels, we propose to generate attacks through perturbing on the high-level semantics to improve attack transferability. Specifically, a unified flexible framework, Adversarial Attributes (Adv-Attribute), is designed to generate inconspicuous and transferable attacks on face recognition, which crafts the adversarial noise and adds it into different attributes based on the guidance of the difference in face recognition features from the target. Moreover, the importance-aware attribute selection and the multi-objective optimization strategy are introduced to further ensure the balance of stealthiness and attacking strength. Extensive experiments on the FFHQ and CelebA-HQ datasets show that the proposed Adv-Attribute method achieves the state-of-the-art attacking success rates while maintaining better visual effects against recent attack methods.