Reviewerly/exHarmony huggingface dataset:
Masked Student Dataset of Expressions

Facial expression recognition (FER) algorithms work well in constrained environments with little or no occlusion of the face. However, real-world face occlusion is prevalent, most notably with the need to use a face mask in the current Covid-19 scenario. While there are works on the problem of occlusion in FER, little has been done before on the particular face mask scenario. Moreover, the few works in this area largely use synthetically created masked FER datasets. Motivated by these challenges posed by the pandemic to FER, we present a novel dataset, the Masked Student Dataset of Expressions or MSD-E, consisting of 1,960 real-world non-masked and masked facial expression images collected from 142 individuals. Along with the issue of obfuscated facial features, we illustrate how other subtler issues in masked FER are represented in our dataset. We then provide baseline results using ResNet-18, finding that its performance dips in the non-masked case when trained for FER in the presence of masks. To tackle this, we test two training paradigms: contrastive learning and knowledge distillation, and find that they increase the model's performance in the masked scenario while maintaining its non-masked performance. We further visualise our results using t-SNE plots and Grad-CAM, demonstrating that these paradigms capitalise on the limited features available in the masked scenario. Finally, we benchmark SOTA methods on MSD-E.
Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems (Extended Version)

This paper presents a policy parameterization for learning-based control on nonlinear, partially-observed dynamical systems. The parameterization is based on a nonlinear version of the Youla parameterization and the recently proposed Recurrent Equilibrium Network (REN) class of models. We prove that the resulting Youla-REN parameterization automatically satisfies stability (contraction) and user-tunable robustness (Lipschitz) conditions on the closed-loop system. This means it can be used for safe learning-based control with no additional constraints or projections required to enforce stability or robustness. We test the new policy class in simulation on two reinforcement learning tasks: 1) magnetic suspension, and 2) inverting a rotary-arm pendulum. We find that the Youla-REN performs similarly to existing learning-based and optimal control methods while also ensuring stability and exhibiting improved robustness to adversarial disturbances.
Influences of Fourier Completely Bounded Polynomials and Classical Simulation of Quantum Algorithms

We give a new presentation of the main result of Arunachalam, Bri\"et and Palazuelos (SICOMP'19) and show that quantum query algorithms are characterized by a new class of polynomials which we call Fourier completely bounded polynomials. We conjecture that all such polynomials have an influential variable. This conjecture is weaker than the famous Aaronson-Ambainis (AA) conjecture (Theory of Computing'14), but has the same implications for classical simulation of quantum query algorithms. We prove a new case of the AA conjecture by showing that it holds for homogeneous Fourier completely bounded polynomials. This implies that if the output of $d$-query quantum algorithm is a homogeneous polynomial $p$ of degree $2d$, then it has a variable with influence at least $Var[p]^2$. In addition, we give an alternative proof of the results of Bansal, Sinha and de Wolf (CCC'22 and QIP'23) showing that block-multilinear completely bounded polynomials have influential variables. Our proof is simpler, obtains better constants and does not use randomness.
Evaluation of ChatGPT Model for Vulnerability Detection

In this technical report, we evaluated the performance of the ChatGPT and GPT-3 models for the task of vulnerability detection in code. Our evaluation was conducted on our real-world dataset, using binary and multi-label classification tasks on CWE vulnerabilities. We decided to evaluate the model because it has shown good performance on other code-based tasks, such as solving programming challenges and understanding code at a high level. However, we found that the ChatGPT model performed no better than a dummy classifier for both binary and multi-label classification tasks for code vulnerability detection.
MasakhaNEWS: News Topic Classification for African languages

African languages are severely under-represented in NLP research due to lack of datasets covering several NLP tasks. While there are individual language specific datasets that are being expanded to different tasks, only a handful of NLP tasks (e.g. named entity recognition and machine translation) have standardized benchmark datasets covering several geographical and typologically-diverse African languages. In this paper, we develop MasakhaNEWS -- a new benchmark dataset for news topic classification covering 16 languages widely spoken in Africa. We provide an evaluation of baseline models by training classical machine learning models and fine-tuning several language models. Furthermore, we explore several alternatives to full fine-tuning of language models that are better suited for zero-shot and few-shot learning such as cross-lingual parameter-efficient fine-tuning (like MAD-X), pattern exploiting training (PET), prompting language models (like ChatGPT), and prompt-free sentence transformer fine-tuning (SetFit and Cohere Embedding API). Our evaluation in zero-shot setting shows the potential of prompting ChatGPT for news topic classification in low-resource African languages, achieving an average performance of 70 F1 points without leveraging additional supervision like MAD-X. In few-shot setting, we show that with as little as 10 examples per label, we achieved more than 90\% (i.e. 86.0 F1 points) of the performance of full supervised training (92.6 F1 points) leveraging the PET approach.
Constructing Temporal Networks of OSS Programming Language Ecosystems

One of the primary factors that encourage developers to contribute to open source software (OSS) projects is the collaborative nature of OSS development. However, the collaborative structure of these communities largely remains unclear, partly due to the enormous scale of data to be gathered, processed, and analyzed. In this work, we utilize the World Of Code dataset, which contains commit activity data for millions of OSS projects, to build collaboration networks for ten popular programming language ecosystems, containing in total over 290M commits across over 18M projects. We build a collaboration graph representation for each language ecosystem, having authors and projects as nodes, which enables various forms of social network analysis on the scale of language ecosystems. Moreover, we capture the information on the ecosystems' evolution by slicing each network into 30 historical snapshots. Additionally, we calculate multiple collaboration metrics that characterize the ecosystems' states. We make the resulting dataset publicly available, including the constructed graphs and the pipeline enabling the analysis of more ecosystems.
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence

Real-time and robust photorealistic avatars for telepresence in AR/VR have been highly desired for enabling immersive photorealistic telepresence. However, there still exists one key bottleneck: the considerable computational expense needed to accurately infer facial expressions captured from headset-mounted cameras with a quality level that can match the realism of the avatar's human appearance. To this end, we propose a framework called Auto-CARD, which for the first time enables real-time and robust driving of Codec Avatars when exclusively using merely on-device computing resources. This is achieved by minimizing two sources of redundancy. First, we develop a dedicated neural architecture search technique called AVE-NAS for avatar encoding in AR/VR, which explicitly boosts both the searched architectures' robustness in the presence of extreme facial expressions and hardware friendliness on fast evolving AR/VR headsets. Second, we leverage the temporal redundancy in consecutively captured images during continuous rendering and develop a mechanism dubbed LATEX to skip the computation of redundant frames. Specifically, we first identify an opportunity from the linearity of the latent space derived by the avatar decoder and then propose to perform adaptive latent extrapolation for redundant frames. For evaluation, we demonstrate the efficacy of our Auto-CARD framework in real-time Codec Avatar driving settings, where we achieve a 5.05x speed-up on Meta Quest 2 while maintaining a comparable or even better animation quality than state-of-the-art avatar encoder designs.
Evaluating Inter-Bilingual Semantic Parsing for Indian Languages

Despite significant progress in Natural Language Generation for Indian languages (IndicNLP), there is a lack of datasets around complex structured tasks such as semantic parsing. One reason for this imminent gap is the complexity of the logical form, which makes English to multilingual translation difficult. The process involves alignment of logical forms, intents and slots with translated unstructured utterance. To address this, we propose an Inter-bilingual Seq2seq Semantic parsing dataset IE-SEMPARSE for 11 distinct Indian languages. We highlight the proposed task's practicality, and evaluate existing multilingual seq2seq models across several train-test strategies. Our experiment reveals a high correlation across performance of original multilingual semantic parsing datasets (such as mTOP, multilingual TOP and multiATIS++) and our proposed IE-SEMPARSE suite.
DuETT: Dual Event Time Transformer for Electronic Health Records

Electronic health records (EHRs) recorded in hospital settings typically contain a wide range of numeric time series data that is characterized by high sparsity and irregular observations. Effective modelling for such data must exploit its time series nature, the semantic relationship between different types of observations, and information in the sparsity structure of the data. Self-supervised Transformers have shown outstanding performance in a variety of structured tasks in NLP and computer vision. But multivariate time series data contains structured relationships over two dimensions: time and recorded event type, and straightforward applications of Transformers to time series data do not leverage this distinct structure. The quadratic scaling of self-attention layers can also significantly limit the input sequence length without appropriate input engineering. We introduce the DuETT architecture, an extension of Transformers designed to attend over both time and event type dimensions, yielding robust representations from EHR data. DuETT uses an aggregated input where sparse time series are transformed into a regular sequence with fixed length; this lowers the computational complexity relative to previous EHR Transformer models and, more importantly, enables the use of larger and deeper neural networks. When trained with self-supervised prediction tasks, that provide rich and informative signals for model pre-training, our model outperforms state-of-the-art deep learning models on multiple downstream tasks from the MIMIC-IV and PhysioNet-2012 EHR datasets.
Analogy-Forming Transformers for Few-Shot 3D Parsing

We present Analogical Networks, a model that encodes domain knowledge explicitly, in a collection of structured labelled 3D scenes, in addition to implicitly, as model parameters, and segments 3D object scenes with analogical reasoning: instead of mapping a scene to part segments directly, our model first retrieves related scenes from memory and their corresponding part structures, and then predicts analogous part structures for the input scene, via an end-to-end learnable modulation mechanism. By conditioning on more than one retrieved memories, compositions of structures are predicted, that mix and match parts across the retrieved memories. One-shot, few-shot or many-shot learning are treated uniformly in Analogical Networks, by conditioning on the appropriate set of memories, whether taken from a single, few or many memory exemplars, and inferring analogous parses. We show Analogical Networks are competitive with state-of-the-art 3D segmentation transformers in many-shot settings, and outperform them, as well as existing paradigms of meta-learning and few-shot learning, in few-shot settings. Analogical Networks successfully segment instances of novel object categories simply by expanding their memory, without any weight updates. Our code and models are publicly available in the project webpage: http://analogicalnets.github.io/.
DemodNet: Learning Soft Demodulation from Hard Information Using Convolutional Neural Network

Soft demodulation is a basic module of traditional communication receivers. It converts received symbols into soft bits, that is, log likelihood ratios (LLRs). However, in the nonideal additive white Gaussian noise (AWGN) channel, it is difficult to accurately calculate the LLR. In this letter, we propose a demodulator, DemodNet, based on a fully convolutional neural network with variable input and output length. We use hard bit information to train the DemodNet, and we propose log probability ratio (LPR) based on the output layer of the trained DemodNet to realize soft demodulation. The simulation results show that under the AWGN channel, the performance of both hard demodulation and soft demodulation of DemodNet is very close to the traditional methods. In three non-ideal channel scenarios, i.e., the presence of frequency deviation, additive generalized Gaussian noise (AGGN) channel, and Rayleigh fading channel, the performance of channel decoding using the soft information LPR obtained by DemodNet is better than the performance of decoding using the exact LLR calculated under the ideal AWGN assumption.
Dynamics of Darwinian versus Baldwinian versus Lamarckian evolution

In recent years, studies in epigenetic inheritance in biological systems as well as studies on evolution in non-biological systems e.g., machine learning and robotics, have reopened the discussion of non-Darwinian methods of evolutionary optimization. In this paper, the three most prominent classical evolutionary strategies Lamarckian, Darwinian, and Baldwinian are implemented and compared in an agent-based simulation. The dynamics of optimization and learning are studied in constant as well as in dynamic environments, both as single evolutionary strategy populations and as mixed evolutionary strategy populations. The three different agent types are implemented as simple objects that need to gather resources to replicate. The agent fitness is defined through a bitstring match between the agent and the environment and to replicate each agent needs to gather sufficient resources. To make a fair comparison between the three different evolutionary methods, we can assume that both the ability to learn within an individual's lifetime and the learning process itself require additional resources compared to an evolutionary process without lifetime learning. When all three strategies coexist and we vary the replication costs for the different strategies we find the following general pattern: The evolutionary advantage decreases for Lamarckian and Baldwinian strategies as their learning costs increase and/or as their relative replication costs increase compared to the Darwinian strategies, and in most situations the Baldwinian strategy performs in between the Darwinian and Lamarckian strategies. Finally, as our investigations emphasize the superiority of the Lamarckian strategy in most cases with low learning cost, it is interesting to note that a corresponding biological inheritance mechanism did not evolve, while the Lamarckian strategy is currently extensively used in technology applications.
Ontologies of common sense, physics and mathematics

The view of nature we adopt in the natural attitude is determined by common sense, without which we could not survive. Classical physics is modelled on this common-sense view of nature, and uses mathematics to formalise our natural understanding of the causes and effects we observe in time and space when we select subsystems of nature for modelling. But in modern physics, we do not go beyond the realm of common sense by augmenting our knowledge of what is going on in nature. Rather, we have measurements that we do not understand, so we know nothing about the ontology of what we measure. We help ourselves by using entities from mathematics, which we fully understand ontologically. But we have no ontology of the reality of modern physics; we have only what we can assert mathematically. In this paper, we describe the ontology of classical and modern physics against this background and show how it relates to the ontology of common sense and of mathematics.
Unlimiformer: Long-Range Transformers with Unlimited Length Input

Since the proposal of transformers, these models have been limited to bounded input lengths, because of their need to attend to every token in the input. In this work, we propose Unlimiformer: a general approach that wraps any existing pretrained encoder-decoder transformer, and offloads the cross-attention computation to a single k-nearest-neighbor (kNN) index, while the returned kNN distances are the attention dot-product scores. This kNN index can be kept on either the GPU or CPU memory and queried in sub-linear time; this way, we can index practically unlimited input sequences, while every attention head in every decoder layer retrieves its top-k keys, instead of attending to every key. We evaluate Unlimiformer on several long-document and book-summarization benchmarks, showing that it can process even 500k token-long inputs from the BookSum dataset, without any input truncation at test time. We demonstrate that Unlimiformer improves pretrained models such as BART and Longformer by extending them to unlimited inputs without additional learned weights and without modifying their code. We make our code and models publicly available at https://github.com/abertsch72/unlimiformer .
NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds

This paper proposes NeuralEditor that enables neural radiance fields (NeRFs) natively editable for general shape editing tasks. Despite their impressive results on novel-view synthesis, it remains a fundamental challenge for NeRFs to edit the shape of the scene. Our key insight is to exploit the explicit point cloud representation as the underlying structure to construct NeRFs, inspired by the intuitive interpretation of NeRF rendering as a process that projects or "plots" the associated 3D point cloud to a 2D image plane. To this end, NeuralEditor introduces a novel rendering scheme based on deterministic integration within K-D tree-guided density-adaptive voxels, which produces both high-quality rendering results and precise point clouds through optimization. NeuralEditor then performs shape editing via mapping associated points between point clouds. Extensive evaluation shows that NeuralEditor achieves state-of-the-art performance in both shape deformation and scene morphing tasks. Notably, NeuralEditor supports both zero-shot inference and further fine-tuning over the edited scene. Our code, benchmark, and demo video are available at https://immortalco.github.io/NeuralEditor.
A Study on Blood-Cell Segmentation Method for the Identification of Hematological Disorders

A important task in the detection of hematological abnormalities is the automated segmentation of blood cells. It is essential for diagnosis, arranging treatments, and assessing results. This procedure uses a hybrid blood-cell segmentation technique based on RESNET50 Unet that may be utilized to identify a number of hematological diseases. Our main contributions are a more precise seed-point and better segmentation performance achieved by combining RESNET50 Unet techniques while keeping the advantages of both methods. It is a computationally effective strategy since it combines algebraic and non-iterative geometric algorithms. The minor and major axes should also be estimated using the residue and residue offset factors, according to our proposal. The residue offset parameter that is here presented results in better segmentation with appropriate EF. Modern approaches are contrasted with our method. It performs better than the current EF methods in terms of precision, Jaccard score, and F1 score as well as dice similarity. Other medical and cybernetics applications could benefit from it. Our suggested model beat current models on the test set, with an average accuracy of 97.5%. Also, we contrasted the performance of our model with that of other segmentation models, including DeepLabv3+, Mask R-CNN, and U-Net. Our ResNet50-based model outperformed these models in terms of accuracy and speed, according to the results. As a result, our suggested strategy utilising ResNet50 is a potential technique for precise and effective blood cell segmentation, which can help in the early identification of blood-related disorders
Few-shot Domain-Adaptive Visually-fused Event Detection from Text

Incorporating auxiliary modalities such as images into event detection models has attracted increasing interest over the last few years. The complexity of natural language in describing situations has motivated researchers to leverage the related visual context to improve event detection performance. However, current approaches in this area suffer from data scarcity, where a large amount of labelled text-image pairs are required for model training. Furthermore, limited access to the visual context at inference time negatively impacts the performance of such models, which makes them practically ineffective in real-world scenarios. In this paper, we present a novel domain-adaptive visually-fused event detection approach that can be trained on a few labelled image-text paired data points. Specifically, we introduce a visual imaginator method that synthesises images from text in the absence of visual context. Moreover, the imaginator can be customised to a specific domain. In doing so, our model can leverage the capabilities of pre-trained vision-language models and can be trained in a few-shot setting. This also allows for effective inference where only single-modality data (i.e. text) is available. The experimental evaluation on the benchmark M2E2 dataset shows that our model outperforms existing state-of-the-art models, by up to 11 points.