OpenRaiser/Intern-Atlas huggingface dataset:
Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts
Venue: arXiv cs.CL 2022
Year: 2022

We present a robust methodology for evaluating biases in natural language generation(NLG) systems. Previous works use fixed hand-crafted prefix templates with mentions of various demographic groups to prompt models to generate continuations for bias analysis. These fixed prefix templates could themselves be specific in terms of styles or linguistic structures, which may lead to unreliable fairness conclusions that are not representative of the general trends from tone varying prompts. To study this problem, we paraphrase the prompts with different syntactic structures and use these to evaluate demographic bias in NLG systems. Our results suggest similar overall bias trends but some syntactic structures lead to contradictory conclusions compared to past works. We show that our methodology is more robust and that some syntactic structures prompt more toxic content while others could prompt less biased generation. This suggests the importance of not relying on a fixed syntactic structure and using tone-invariant prompts. Introducing syntactically-diverse prompts can achieve more robust NLG (bias) evaluation.
A survey on grading format of automated grading tools for programming assignments
Venue: arXiv cs.CY 2022
Year: 2022

The prevalence of online platforms and studies has generated the demand for automated grading tools, and as a result, there are plenty in the market. Such tools are developed to grade coding assignments quickly, accurately, and effortlessly. Since there are varieties of tools available to cater to the diverse options of programming languages and concepts, it is overwhelming for any instructor to decide which one suits one's requirements. There are several surveys studying the tools and giving insights into how they function and what they support. However other than knowing the functionality, it is important for an instructor to know how the assignments are graded and what is the format of the test cases. This is crucial since the instructor has to design the grading format and therefore requires a learning curve. This survey studies and evaluates the automated grading tools based on their evaluation format. This in turn helps a reader in deciding which tool to choose and provides an insight into what are the assessment settings and approaches used in grading the coding assignment in any specific grading tool.
Kernel Inversed Pyramidal Resizing Network for Efficient Pavement Distress Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Pavement Distress Recognition (PDR) is an important step in pavement inspection and can be powered by image-based automation to expedite the process and reduce labor costs. Pavement images are often in high-resolution with a low ratio of distressed to non-distressed areas. Advanced approaches leverage these properties via dividing images into patches and explore discriminative features in the scale space. However, these approaches usually suffer from information loss during image resizing and low efficiency due to complex learning frameworks. In this paper, we propose a novel and efficient method for PDR. A light network named the Kernel Inversed Pyramidal Resizing Network (KIPRN) is introduced for image resizing, and can be flexibly plugged into the image classification network as a pre-network to exploit resolution and scale information. In KIPRN, pyramidal convolution and kernel inversed convolution are specifically designed to mine discriminative information across different feature granularities and scales. The mined information is passed along to the resized images to yield an informative image pyramid to assist the image classification network for PDR. We applied our method to three well-known Convolutional Neural Networks (CNNs), and conducted an evaluation on a large-scale pavement image dataset named CQU-BPDD. Extensive results demonstrate that KIPRN can generally improve the pavement distress recognition of these CNN models and show that the simple combination of KIPRN and EfficientNet-B3 significantly outperforms the state-of-the-art patch-based method in both performance and efficiency.
GraphGDP: Generative Diffusion Processes for Permutation Invariant Graph Generation
Venue: arXiv cs.LG 2022
Year: 2022

Graph generative models have broad applications in biology, chemistry and social science. However, modelling and understanding the generative process of graphs is challenging due to the discrete and high-dimensional nature of graphs, as well as permutation invariance to node orderings in underlying graph distributions. Current leading autoregressive models fail to capture the permutation invariance nature of graphs for the reliance on generation ordering and have high time complexity. Here, we propose a continuous-time generative diffusion process for permutation invariant graph generation to mitigate these issues. Specifically, we first construct a forward diffusion process defined by a stochastic differential equation (SDE), which smoothly converts graphs within the complex distribution to random graphs that follow a known edge probability. Solving the corresponding reverse-time SDE, graphs can be generated from newly sampled random graphs. To facilitate the reverse-time SDE, we newly design a position-enhanced graph score network, capturing the evolving structure and position information from perturbed graphs for permutation equivariant score estimation. Under the evaluation of comprehensive metrics, our proposed generative diffusion process achieves competitive performance in graph distribution learning. Experimental results also show that GraphGDP can generate high-quality graphs in only 24 function evaluations, much faster than previous autoregressive models.
Vibration suppression of a state-of-the-art wafer gripper
Venue: arXiv cs.RO 2022
Year: 2022

In this paper the implementation of piezoelectrics to a state-of-the-art wafer gripper is investigated. The objective is to propose and validate a solution method, which includes a mechanical design and control system, to achieve at least 5% damping for two eigenmodes of a wafer gripper. This objective serves as a 'proof of concept' to show the possibilities of implementing a state-of-the-art damping method to an industrial application, which in turn can be used to dampen different thin structures. The coupling relation between the piezoelectrics and their host structure were used to design the placement of the piezoelectric patches, together with modal analysis data of the a state-of-the-art wafer gripper. This data had been measured through an experimental setup. Active damping has been succesfully implemented onto the wafer gripper where positive position feedback (PPF) is used as a control algorithm to dampen two eigenmodes.
Pairing-Friendly Elliptic Curves: Revisited Taxonomy, Attacks and Security Concern
Venue: arXiv cs.CR 2022
Year: 2022

Major families of pairing-friendly elliptic curves, including BN, BLS12, BLS24, KSS16, and KSS18 have recently been vulnerable to number field sieve (NFS) attacks. Due to the recent attacks on discrete logs in F_(q^k ), selecting such curves became relevant again. This paper revisited the topic of selecting pairing-friendly curves at different security levels. First, we expanded the classification given by Freeman et al. [1] by identifying new families that were not previously mentioned, such as a complete family with variable differentiation and new sparse families of curves. We discussed individual curves and a comprehensive framework for constructing parametric families. We estimated the security and assessed families of the pairing-friendly curve to discover families of curves better than BN, KSS, and BLS in terms of the required key size. We also evaluated the complexity of the optimal ate pairing that has never been discussed before, except by Barbulescu et al. [2]. We demonstrated that the recent attack (TNFS) on pairing needs to increase the key size. We compared families of curves in the context of key size and selected a suitable alternative to an elliptic curve.
Online Shielding for Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Besides the recent impressive results on reinforcement learning (RL), safety is still one of the major research challenges in RL. RL is a machine-learning approach to determine near-optimal policies in Markov decision processes (MDPs). In this paper, we consider the setting where the safety-relevant fragment of the MDP together with a temporal logic safety specification is given and many safety violations can be avoided by planning ahead a short time into the future. We propose an approach for online safety shielding of RL agents. During runtime, the shield analyses the safety of each available action. For any action, the shield computes the maximal probability to not violate the safety specification within the next $k$ steps when executing this action. Based on this probability and a given threshold, the shield decides whether to block an action from the agent. Existing offline shielding approaches compute exhaustively the safety of all state-action combinations ahead of time, resulting in huge computation times and large memory consumption. The intuition behind online shielding is to compute at runtime the set of all states that could be reached in the near future. For each of these states, the safety of all available actions is analysed and used for shielding as soon as one of the considered states is reached. Our approach is well suited for high-level planning problems where the time between decisions can be used for safety computations and it is sustainable for the agent to wait until these computations are finished. For our evaluation, we selected a 2-player version of the classical computer game SNAKE. The game represents a high-level planning problem that requires fast decisions and the multiplayer setting induces a large state space, which is computationally expensive to analyse exhaustively.
"Tell me, how do you know it's me?" Expectations of security and personalization measures for smart speaker applications
Venue: arXiv cs.CR 2022
Year: 2022

Voice-controlled smart speaker devices have gained a foothold in many modern households. Their prevalence combined with their intrusion into core private spheres of life has motivated research on security and privacy intrusions, especially those performed by third-party applications used on such devices. In this work, we take a closer look at such third-party applications from a less pessimistic angle: we consider their potential to provide personalized and secure capabilities and investigate measures to authenticate users (``PIN'', ``Voice authentication'', ``Notification'', and presence of ``Nearby devices''). To this end, we asked 100 participants to evaluate 15 application categories and 51 apps with a wide range of functions. The central questions we explored focused on: users' preferences for security and personalization for different categories of apps; the preferred security and personalization measures for different apps; and the preferred frequency of the respective measure. After an initial pilot study, we focused primarily on 7 categories of apps for which security and personalization are reported to be important; those include the three crucial categories finance, bills, and shopping. We found that ``Voice authentication'', while not currently employed by the apps we studied, is a highly popular measure to achieve security and personalization. Many participants were open to exploring combinations of security measures to increase the protection of highly relevant apps. Here, the combination of ``PIN'' and ``Voice authentication'' was clearly the most desired one. This finding indicates systems that seamlessly combine ``Voice authentication'' with other measures might be a good candidate for future work.
Understanding How Model Size Affects Few-shot Instruction Prompting
Venue: arXiv cs.CL 2022
Year: 2022

Large Language Models are affected by the phenomena of memorizing and forgetting their training data. But how do these vary by model size? We work towards this question by investigating how the model size affects the model's ability to discriminate a word's meaning in a given context. We introduce a dataset called DeltaWords, which evaluates a model's ability to follow instructions to select a sentence which replaces the target word with its antonym. We show a weak inverse scaling trend, where task accuracy degrades as model size increase, under extremely few-shot prompting regimes. We show that increasing the number of examples tend to disproportionately benefit larger models than smaller models.
Speech MOS multi-task learning and rater bias correction
Venue: arXiv cs.SD 2022
Year: 2022

Perceptual speech quality is an important performance metric for teleconferencing applications. The mean opinion score (MOS) is standardized for the perceptual evaluation of speech quality and is obtained by asking listeners to rate the quality of a speech sample. Recently, there has been increasing research interest in developing models for estimating MOS blindly. Here we propose a multi-task framework to include additional labels and data in training to improve the performance of a blind MOS estimation model. Experimental results indicate that the proposed model can be trained to jointly estimate MOS, reverberation time (T60), and clarity (C50) by combining two disjoint data sets in training, one containing only MOS labels and the other containing only T60 and C50 labels. Furthermore, we use a semi-supervised framework to combine two MOS data sets in training, one containing only MOS labels (per ITU-T Recommendation P.808), and the other containing separate scores for speech signal, background noise, and overall quality (per ITU-T Recommendation P.835). Finally, we present preliminary results for addressing individual rater bias in the MOS labels.
Cross-lingual Similarity of Multilingual Representations Revisited
Venue: arXiv cs.CL 2022
Year: 2022

Related works used indexes like CKA and variants of CCA to measure the similarity of cross-lingual representations in multilingual language models. In this paper, we argue that assumptions of CKA/CCA align poorly with one of the motivating goals of cross-lingual learning analysis, i.e., explaining zero-shot cross-lingual transfer. We highlight what valuable aspects of cross-lingual similarity these indexes fail to capture and provide a motivating case study \textit{demonstrating the problem empirically}. Then, we introduce \textit{Average Neuron-Wise Correlation (ANC)} as a straightforward alternative that is exempt from the difficulties of CKA/CCA and is good specifically in a cross-lingual context. Finally, we use ANC to construct evidence that the previously introduced ``first align, then predict'' pattern takes place not only in masked language models (MLMs) but also in multilingual models with \textit{causal language modeling} objectives (CLMs). Moreover, we show that the pattern extends to the \textit{scaled versions} of the MLMs and CLMs (up to 85x original mBERT).\footnote{Our code is publicly available at \url{https://github.com/TartuNLP/xsim}}
First Demonstration Experiment for Energy Trading System EDISON-X Using the XRP Ledger
Venue: arXiv cs.CR 2022
Year: 2022

We develop an energy trading system, EDISON-X, that uses blockchain technology to manage the buying and selling of electricity usage rights, i.e., tokens. UPX and SPX tokens purchase electricity from the utility company's distribution lines and the photovoltaic panels. On July 1, 17 students in our school dormitory participated in an experiment to confirm the operation of the EDISON-X system. Based on the results of this experiment, we discuss the energy trading system using blockchain technology for the effective usage of renewable energy. We develop topology and network science methodologies to understand the characteristics of energy trading. We test the hypothesis that market transactions become less active when ``cavities'' appear using persistent homology. The preliminary result implies that the hypothesis could be adopted. We, however, need more data samples.
Learning to Learn Better for Video Object Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Recently, the joint learning framework (JOINT) integrates matching based transductive reasoning and online inductive learning to achieve accurate and robust semi-supervised video object segmentation (SVOS). However, using the mask embedding as the label to guide the generation of target features in the two branches may result in inadequate target representation and degrade the performance. Besides, how to reasonably fuse the target features in the two different branches rather than simply adding them together to avoid the adverse effect of one dominant branch has not been investigated. In this paper, we propose a novel framework that emphasizes Learning to Learn Better (LLB) target features for SVOS, termed LLB, where we design the discriminative label generation module (DLGM) and the adaptive fusion module to address these issues. Technically, the DLGM takes the background-filtered frame instead of the target mask as input and adopts a lightweight encoder to generate the target features, which serves as the label of the online few-shot learner and the value of the decoder in the transformer to guide the two branches to learn more discriminative target representation. The adaptive fusion module maintains a learnable gate for each branch, which reweighs the element-wise feature representation and allows an adaptive amount of target information in each branch flowing to the fused target feature, thus preventing one branch from being dominant and making the target feature more robust to distractor. Extensive experiments on public benchmarks show that our proposed LLB method achieves state-of-the-art performance.
FaceQAN: Face Image Quality Assessment Through Adversarial Noise Exploration
Venue: arXiv cs.CV 2022
Year: 2022

Recent state-of-the-art face recognition (FR) approaches have achieved impressive performance, yet unconstrained face recognition still represents an open problem. Face image quality assessment (FIQA) approaches aim to estimate the quality of the input samples that can help provide information on the confidence of the recognition decision and eventually lead to improved results in challenging scenarios. While much progress has been made in face image quality assessment in recent years, computing reliable quality scores for diverse facial images and FR models remains challenging. In this paper, we propose a novel approach to face image quality assessment, called FaceQAN, that is based on adversarial examples and relies on the analysis of adversarial noise which can be calculated with any FR model learned by using some form of gradient descent. As such, the proposed approach is the first to link image quality to adversarial attacks. Comprehensive (cross-model as well as model-specific) experiments are conducted with four benchmark datasets, i.e., LFW, CFP-FP, XQLFW and IJB-C, four FR models, i.e., CosFace, ArcFace, CurricularFace and ElasticFace, and in comparison to seven state-of-the-art FIQA methods to demonstrate the performance of FaceQAN. Experimental results show that FaceQAN achieves competitive results, while exhibiting several desirable characteristics.
Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Prediction
Venue: arXiv cs.CV 2022
Year: 2022

Motion prediction is highly relevant to the perception of dynamic objects and static map elements in the scenarios of autonomous driving. In this work, we propose PIP, the first end-to-end Transformer-based framework which jointly and interactively performs online mapping, object detection and motion prediction. PIP leverages map queries, agent queries and mode queries to encode the instance-wise information of map elements, agents and motion intentions, respectively. Based on the unified query representation, a differentiable multi-task interaction scheme is proposed to exploit the correlation between perception and prediction. Even without human-annotated HD map or agent's historical tracking trajectory as guidance information, PIP realizes end-to-end multi-agent motion prediction and achieves better performance than tracking-based and HD-map-based methods. PIP provides comprehensive high-level information of the driving scene (vectorized static map and dynamic objects with motion information), and contributes to the downstream planning and control. Code and models will be released for facilitating further research.
Identity Management through a global Discovery System based on Decentralized Identities
Venue: arXiv cs.CR 2022
Year: 2022

Digital identities today continue to be a company resource instead of belonging to the actual person they represent. At the same time, the digitalization of everyday services intensifies the Identity Management problem and leads to a constant increase of users online identities and identity related data. This paper presents DIMANDS2, a framework capable of organizing identity data that allows service providers and identity issuers securely exchange identity related information in a privacy-enabled manner while the user maintains full control over any activity related to his/her identity data. The framework is format-agnostic and can accommodate any type of identifier (existing or new), without requiring from existing services and providers to implement and adopt another new global identifier.
Rethinking Generative Methods for Image Restoration in Physics-based Vision: A Theoretical Analysis from the Perspective of Information
Venue: arXiv cs.CV 2022
Year: 2022

End-to-end generative methods are considered a more promising solution for image restoration in physics-based vision compared with the traditional deconstructive methods based on handcrafted composition models. However, existing generative methods still have plenty of room for improvement in quantitative performance. More crucially, these methods are considered black boxes due to weak interpretability and there is rarely a theory trying to explain their mechanism and learning process. In this study, we try to re-interpret these generative methods for image restoration tasks using information theory. Different from conventional understanding, we analyzed the information flow of these methods and identified three sources of information (extracted high-level information, retained low-level information, and external information that is absent from the source inputs) are involved and optimized respectively in generating the restoration results. We further derived their learning behaviors, optimization objectives, and the corresponding information boundaries by extending the information bottleneck principle. Based on this theoretic framework, we found that many existing generative methods tend to be direct applications of the general models designed for conventional generation tasks, which may suffer from problems including over-invested abstraction processes, inherent details loss, and vanishing gradients or imbalance in training. We analyzed these issues with both intuitive and theoretical explanations and proved them with empirical evidence respectively. Ultimately, we proposed general solutions or ideas to address the above issue and validated these approaches with performance boosts on six datasets of three different image restoration tasks.