OpenRaiser/Intern-Atlas huggingface dataset:
What Can Secondary Predictions Tell Us? An Exploration on Question-Answering with SQuAD-v2.0
Venue: arXiv cs.CL 2022
Year: 2022

Performance in natural language processing, and specifically for the question-answer task, is typically measured by comparing a modelś most confident (primary) prediction to golden answers (the ground truth). We are making the case that it is also useful to quantify how close a model came to predicting a correct answer even for examples that failed. We define the Golden Rank (GR) of an example as the rank of its most confident prediction that exactly matches a ground truth, and show why such a match always exists. For the 16 transformer models we analyzed, the majority of exactly matched golden answers in secondary prediction space hover very close to the top rank. We refer to secondary predictions as those ranking above 0 in descending confidence probability order. We demonstrate how the GR can be used to classify questions and visualize their spectrum of difficulty, from persistent near successes to persistent extreme failures. We derive a new aggregate statistic over entire test sets, named the Golden Rank Interpolated Median (GRIM) that quantifies the proximity of failed predictions to the top choice made by the model. To develop some intuition and explore the applicability of these metrics we use the Stanford Question Answering Dataset (SQuAD-2) and a few popular transformer models from the Hugging Face hub. We first demonstrate that the GRIM is not directly correlated with the F1 and exact match (EM) scores. We then calculate and visualize these scores for various transformer architectures, probe their applicability in error analysis by clustering failed predictions, and compare how they relate to other training diagnostics such as the EM and F1 scores. We finally suggest various research goals, such as broadening data collection for these metrics and their possible use in adversarial training.
Chinese Word Sense Embedding with SememeWSD and Synonym Set
Venue: arXiv cs.CL 2022
Year: 2022

Word embedding is a fundamental natural language processing task which can learn feature of words. However, most word embedding methods assign only one vector to a word, even if polysemous words have multi-senses. To address this limitation, we propose SememeWSD Synonym (SWSDS) model to assign a different vector to every sense of polysemous words with the help of word sense disambiguation (WSD) and synonym set in OpenHowNet. We use the SememeWSD model, an unsupervised word sense disambiguation model based on OpenHowNet, to do word sense disambiguation and annotate the polysemous word with sense id. Then, we obtain top 10 synonyms of the word sense from OpenHowNet and calculate the average vector of synonyms as the vector of the word sense. In experiments, We evaluate the SWSDS model on semantic similarity calculation with Gensim's wmdistance method. It achieves improvement of accuracy. We also examine the SememeWSD model on different BERT models to find the more effective model.
Representation and Synthesis of C++ Programs for Generalized Planning
Venue: arXiv cs.AI 2022
Year: 2022

The paper introduces a novel representation for Generalized Planning (GP) problems, and their solutions, as C++ programs. Our C++ representation allows to formally proving the termination of generalized plans, and to specifying their asymptotic complexity w.r.t. the number of world objects. Characterizing the complexity of C++ generalized plans enables the application of a combinatorial search that enumerates the space of possible GP solutions in order of complexity. Experimental results show that our implementation of this approach, which we call BFGP++, outperforms the previous GP as heuristic search approach for the computation of generalized plans represented as compiler-styled programs. Last but not least, the execution of a C++ program on a classical planning instance is a deterministic grounding-free and search-free process, so our C++ representation allows us to automatically validate the computed solutions on large test instances of thousands of objects, where off-the-shelf classical planners get stuck either in the pre-processing or in the search.
Imaging the time series of one single referenced EEG electrode for Epileptic Seizures Risk Analysis
Venue: arXiv cs.LG 2022
Year: 2022

The time series captured by a single scalp electrode (plus the reference electrode) of refractory epileptic patients is used to forecast seizures susceptibility. The time series is preprocessed, segmented, and each segment transformed into an image, using three different known methods: Recurrence Plot, Gramian Angular Field, Markov Transition Field. The likelihood of the occurrence of a seizure in a future predefined time window is computed by averaging the output of the softmax layer of a CNN, differently from the usual consideration of the output of the classification layer. By thresholding this likelihood, seizure forecasting has better performance. Interestingly, for almost every patient, the best threshold was different from 50%. The results show that this technique can predict with good results for some seizures and patients. However, more tests, namely more patients and more seizures, are needed to better understand the real potential of this technique.
A note on a Code-Based Signature Scheme
Venue: arXiv cs.CR 2022
Year: 2022

In this work, we exploit a serious security flaw in a code-based signature scheme from a 2019 work by Liu, Yang, Han and Wang. They adapt the McEliece cryptosystem to obtain a new scheme and, on top of this, they design an efficient digital signature. We show that the new encryption scheme based on McEliece, even if it has longer public keys, is not more secure than the standard one. Moreover, the choice of parameters for the signature leads to a significant performance improvement, but it introduces a vulnerability in the protocol.
Contributions to Context-Aware Smart Healthcare: A Security and Privacy Perspective
Venue: arXiv cs.CR 2022
Year: 2022

The management of health data, from their gathering to their analysis, arises a number of challenging issues due to their highly confidential nature. In particular, this dissertation contributes to several security and privacy challenges within the smart health paradigm. More concretely, we firstly develop some contributions to context-aware environments enabling smart health scenarios. We present an extensive analysis on the security aspects of the underlying sensors and networks deployed in such environments, a novel user-centred privacy framework for analysing ubiquitous computing systems, and a complete analysis on the security and privacy challenges that need to be faced to implement cognitive cities properly. Second, we contribute to process mining, a popular analytical field that helps analyse business processes within organisations. Despite its popularity within the healthcare industry, we address two major issues: the high complexity of healthcare processes and the scarce research on privacy aspects. Regarding the first issue, we present a novel process discovery algorithm with a built-in heuristic that simplifies complex processes and, regarding the second, we propose two novel privacy-preserving process mining methods, which achieve a remarkable trade-off between accuracy and privacy. Last but not least, we present some smart health applications, namely a context-aware recommender system for routes, a platform supporting early mobilization programmes in hospital settings, and a health-oriented geographic information system. The results of this dissertation are intended to help the research community to enhance the security of the intelligent environments of the future as well as the privacy of the citizens regarding their personal and health data.
Should Social Robots in Retail Manipulate Customers?
Venue: arXiv cs.RO 2022
Year: 2022

Against the backdrop of structural changes in the retail trade, social robots have found their way into retail stores and shopping malls in order to attract, welcome, and greet customers; to inform them, advise them, and persuade them to make a purchase. Salespeople often have a broad knowledge of their product and rely on offering competent and honest advice, whether it be on shoes, clothing, or kitchen appliances. However, some frequently use sales tricks to secure purchases. The question arises of how consulting and sales robots should "behave". Should they behave like human advisors and salespeople, i.e., occasionally manipulate customers? Or should they be more honest and reliable than us? This article tries to answer these questions. After explaining the basics, it evaluates a study in this context and gives recommendations for companies that want to use consulting and sales robots. Ultimately, fair, honest, and trustworthy robots in retail are a win-win situation for all concerned.
Building a Secure Software Supply Chain with GNU Guix
Venue: arXiv cs.SE 2022
Year: 2022

The software supply chain is becoming a widespread analogy to designate the series of steps taken to go from source code published by developers to executables running on the users? computers. A security vulnerability in any of these steps puts users at risk, and evidence shows that attacks on the supply chain are becoming more common. The consequences of an attack on the software supply chain can be tragic in a society that relies on many interconnected software systems, and this has led research interest as well as governmental incentives for supply chain security to rise. GNU Guix is a software deployment tool and software distribution that supports provenance tracking, reproducible builds, and reproducible software environments. Unlike many software distributions, it consists exclusively of source code: it provides a set of package definitions that describe how to build code from source. Together, these properties set it apart from many deployment tools that center on the distribution of binaries. This paper focuses on one research question: how can Guix and similar systems allow users to securely update their software? Guix source code is distributed using the Git version control system; updating Guix-installed software packages means, first, updating the local copy of the Guix source code. Prior work on secure software updates focuses on systems very different from Guix -- systems such as Debian, Fedora, or PyPI where updating consists in fetching metadata about the latest binary artifacts available -- and is largely inapplicable in the context of Guix. By contrast, the main threats for Guix are attacks on its source code repository, which could lead users to run inauthentic code or to downgrade their system. Deployment tools that more closely resemble Guix, from Nix to Portage, either lack secure update mechanisms or suffer from shortcomings. Our main contribution is a model and tool to authenticate new Git revisions. We further show how, building on Git semantics, we build protections against downgrade attacks and related threats. We explain implementation choices. This work has been deployed in production two years ago, giving us insight on its actual use at scale every day. The Git checkout authentication at its core is applicable beyond the specific use case of Guix, and we think it could benefit to developer teams that use Git. As attacks on the software supply chain appear, security research is now looking at every link of the supply chain. Secure updates are one important aspect of the supply chain, but this paper also looks at the broader context: how Guix models and implements the supply chain, from upstream source code to binaries running on computers. While much recent work focuses on attestation -- certifying each link of the supply chain -- Guix takes a more radical approach: enabling independent verification of each step, building on reproducible builds, "bootstrappable" builds, and provenance tracking. The big picture shows how Guix can be used as the foundation of secure software supply chains.
NERDA-Con: Extending NER models for Continual Learning -- Integrating Distinct Tasks and Updating Distribution Shifts
Venue: arXiv cs.CL 2022
Year: 2022

With increasing applications in areas such as biomedical information extraction pipelines and social media analytics, Named Entity Recognition (NER) has become an indispensable tool for knowledge extraction. However, with the gradual shift in language structure and vocabulary, NERs are plagued with distribution shifts, making them redundant or not as profitable without re-training. Re-training NERs based on Large Language Models (LLMs) from scratch over newly acquired data poses economic disadvantages. In contrast, re-training only with newly acquired data will result in Catastrophic Forgetting of previously acquired knowledge. Therefore, we propose NERDA-Con, a pipeline for training NERs with LLM bases by incorporating the concept of Elastic Weight Consolidation (EWC) into the NER fine-tuning NERDA pipeline. As we believe our work has implications to be utilized in the pipeline of continual learning and NER, we open-source our code as well as provide the fine-tuning library of the same name NERDA-Con at https://github.com/SupritiVijay/NERDA-Con and https://pypi.org/project/NERDA-Con/.
A Multilingual Dataset of COVID-19 Vaccination Attitudes on Twitter
Venue: arXiv cs.CL 2022
Year: 2022

Vaccine hesitancy is considered as one main cause of the stagnant uptake ratio of COVID-19 vaccines in Europe and the US where vaccines are sufficiently supplied. Fast and accurate grasp of public attitudes toward vaccination is critical to address vaccine hesitancy, and social media platforms have proved to be an effective source of public opinions. In this paper, we describe the collection and release of a dataset of tweets related to COVID-19 vaccines. This dataset consists of the IDs of 2,198,090 tweets collected from Western Europe, 17,934 of which are annotated with the originators' vaccination stances. Our annotation will facilitate using and developing data-driven models to extract vaccination attitudes from social media posts and thus further confirm the power of social media in public health surveillance. To lay the groundwork for future research, we not only perform statistical analysis and visualisation of our dataset, but also evaluate and compare the performance of established text-based benchmarks in vaccination stance extraction. We demonstrate one potential use of our data in practice in tracking the temporal changes of public COVID-19 vaccination attitudes.
Assessing Intel's Memory Bandwidth Allocation for resource limitation in real-time systems
Venue: arXiv cs.DC 2022
Year: 2022

Industries are recently considering the adoption of cloud computing for hosting safety critical applications. However, the use of multicore processors usually adopted in the cloud introduces temporal anomalies due to contention for shared resources, such as the memory subsystem. In this paper we explore the potential of Intel's Memory Bandwidth Allocation (MBA) technology, available on Xeon Scalable processors. By adopting a systematic measurement approach on real hardware, we assess the indirect memory bandwidth limitation achievable by applying MBA delays, showing that only given delay values (namely 70, 80 and 90) are effective in our setting. We also test the derived bandwidth assured to a hypothetical critical core when interfering cores (e.g., generating a concurrent memory access workload) are present on the same machine. Our results can support designers by providing understanding of impact of the shared memory to enable predictable progress of safety critical applications in cloud environments.
Multi-scale Physical Representations for Approximating PDE Solutions with Graph Neural Operators
Venue: arXiv cs.LG 2022
Year: 2022

Representing physical signals at different scales is among the most challenging problems in engineering. Several multi-scale modeling tools have been developed to describe physical systems governed by \emph{Partial Differential Equations} (PDEs). These tools are at the crossroad of principled physical models and numerical schema. Recently, data-driven models have been introduced to speed-up the approximation of PDE solutions compared to numerical solvers. Among these recent data-driven methods, neural integral operators are a class that learn a mapping between function spaces. These functions are discretized on graphs (meshes) which are appropriate for modeling interactions in physical phenomena. In this work, we study three multi-resolution schema with integral kernel operators that can be approximated with \emph{Message Passing Graph Neural Networks} (MPGNNs). To validate our study, we make extensive MPGNNs experiments with well-chosen metrics considering steady and unsteady PDEs.
GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction
Venue: arXiv cs.CV 2022
Year: 2022

We present GO-Surf, a direct feature grid optimization method for accurate and fast surface reconstruction from RGB-D sequences. We model the underlying scene with a learned hierarchical feature voxel grid that encapsulates multi-level geometric and appearance local information. Feature vectors are directly optimized such that after being tri-linearly interpolated, decoded by two shallow MLPs into signed distance and radiance values, and rendered via surface volume rendering, the discrepancy between synthesized and observed RGB/depth values is minimized. Our supervision signals -- RGB, depth and approximate SDF -- can be obtained directly from input images without any need for fusion or post-processing. We formulate a novel SDF gradient regularization term that encourages surface smoothness and hole filling while maintaining high frequency details. GO-Surf can optimize sequences of $1$-$2$K frames in $15$-$45$ minutes, a speedup of $\times60$ over NeuralRGB-D, the most related approach based on an MLP representation, while maintaining on par performance on standard benchmarks. Project page: https://jingwenwang95.github.io/go_surf/
Key Factors of Wireless Real-Time Networks -- From Dependability to Timeliness
Venue: arXiv cs.NI 2022
Year: 2022

Offering support for real-time communications on top of a wireless network infrastructure is both a hot topic and still an open challenge. Wireless networks are not on the same level of safety, dependability, and timeliness observed in the wired realm, but they are evolving towards it. Instead of focusing on the results that need to be delivered, the key factors of wireless real-time networks are on the foundation of the network operation, defining their capability of being dependable, safe, and timely on their roots. IEEE 802.15.4 and ISA100.11a are part of this context, which we show how to be strengthened. From dealing with network inaccessibility to touching the needs of reliable communication protocols to ensure the safe and sound exchange of information, this white paper describes how we can go from dependability to timeliness. This is achieved by visiting the roots of the network operation for securing the provided communication service as a dependable, safe, and timely asset for industrial automation. Keywords: Dependability, Safety, Timeliness, Resilience, Real-Time Wireless Networks, Industrial Automation.
Distilling Model Failures as Directions in Latent Space
Venue: arXiv cs.LG 2022
Year: 2022

Existing methods for isolating hard subpopulations and spurious correlations in datasets often require human intervention. This can make these methods labor-intensive and dataset-specific. To address these shortcomings, we present a scalable method for automatically distilling a model's failure modes. Specifically, we harness linear classifiers to identify consistent error patterns, and, in turn, induce a natural representation of these failure modes as directions within the feature space. We demonstrate that this framework allows us to discover and automatically caption challenging subpopulations within the training dataset. Moreover, by combining our framework with off-the-shelf diffusion models, we can generate images that are especially challenging for the analyzed model, and thus can be used to perform synthetic data augmentation that helps remedy the model's failure modes. Code available at https://github.com/MadryLab/failure-directions
System-level Simulation of Reconfigurable Intelligent Surface assisted Wireless Communications System
Venue: arXiv cs.IT 2022
Year: 2022

Reconfigurable intelligent surface (RIS) is an emerging technique employing metasurface to reflect the signal from the source node to the destination node. By smartly reconfiguring the electromagnetic (EM) properties of the metasurface and adjusting the EM parameters of the reflected radio waves, RIS can turn the uncontrollable propagation environment into an artificially reconfigurable space, and thus, can significantly increase the communications capacity and improve the coverage of the system. In this paper, we investigate the far field channel in which the line-of-sight (LOS) propagation is dominant. We propose an antenna model that can characterize the radiation patterns of realistic RIS elements, and consider the signal power received from the two-hop path through RIS. System-level simulations of network performance under various scenarios and parameter.
Understanding Generalization via Leave-One-Out Conditional Mutual Information
Venue: arXiv cs.LG 2022
Year: 2022

We study the mutual information between (certain summaries of) the output of a learning algorithm and its $n$ training data, conditional on a supersample of $n+1$ i.i.d. data from which the training data is chosen at random without replacement. These leave-one-out variants of the conditional mutual information (CMI) of an algorithm (Steinke and Zakynthinou, 2020) are also seen to control the mean generalization error of learning algorithms with bounded loss functions. For learning algorithms achieving zero empirical risk under 0-1 loss (i.e., interpolating algorithms), we provide an explicit connection between leave-one-out CMI and the classical leave-one-out error estimate of the risk. Using this connection, we obtain upper and lower bounds on risk in terms of the (evaluated) leave-one-out CMI. When the limiting risk is constant or decays polynomially, the bounds converge to within a constant factor of two. As an application, we analyze the population risk of the one-inclusion graph algorithm, a general-purpose transductive learning algorithm for VC classes in the realizable setting. Using leave-one-out CMI, we match the optimal bound for learning VC classes in the realizable setting, answering an open challenge raised by Steinke and Zakynthinou (2020). Finally, in order to understand the role of leave-one-out CMI in studying generalization, we place leave-one-out CMI in a hierarchy of measures, with a novel unconditional mutual information at the root. For 0-1 loss and interpolating learning algorithms, this mutual information is observed to be precisely the risk.