OpenRaiser/Intern-Atlas huggingface dataset:
FewFedWeight: Few-shot Federated Learning Framework across Multiple NLP Tasks
Venue: arXiv cs.CL 2022
Year: 2022

Massively multi-task learning with large language models has recently made substantial progress on few-shot generalization. However, this is usually performed in a centralized learning fashion, ignoring the privacy sensitivity issue of (annotated) data used in multiple tasks. To mitigate this issue, we propose FewFedWeight, a few-shot federated learning framework across multiple tasks, to achieve the best of both worlds: privacy preservation and cross-task generalization. FewFedWeight trains client models in isolated devices without sharing data. It broadcasts the global model in the server to each client and produces pseudo data for clients so that knowledge from the global model can be explored to enhance few-shot learning of each client model. An energy-based algorithm is further proposed to weight pseudo samples in order to reduce the negative impact of noise from the generated pseudo data. Adaptive model weights of client models are also tuned according to their performance. We use these model weights to dynamically aggregate client models to update the global model. Experiments on 118 NLP tasks show that FewFedWeight can significantly improve the performance of client models on 61% tasks with an average performance improvement rate of 30.5% over the baseline and substantially outperform FedAvg and other decentralized learning methods.
Geometric Rectification of Creased Document Images based on Isometric Mapping
Venue: arXiv cs.CV 2022
Year: 2022

Geometric rectification of images of distorted documents finds wide applications in document digitization and Optical Character Recognition (OCR). Although smoothly curved deformations have been widely investigated by many works, the most challenging distortions, e.g. complex creases and large foldings, have not been studied in particular. The performance of existing approaches, when applied to largely creased or folded documents, is far from satisfying, leaving substantial room for improvement. To tackle this task, knowledge about document rectification should be incorporated into the computation, among which the developability of 3D document models and particular textural features in the images, such as straight lines, are the most essential ones. For this purpose, we propose a general framework of document image rectification in which a computational isometric mapping model is utilized for expressing a 3D document model and its flattening in the plane. Based on this framework, both model developability and textural features are considered in the computation. The experiments and comparisons to the state-of-the-art approaches demonstrated the effectiveness and outstanding performance of the proposed method. Our method is also flexible in that the rectification results can be enhanced by any other methods that extract high-quality feature lines in the images.
Homonymy Information for English WordNet
Venue: arXiv cs.CL 2022
Year: 2022

A widely acknowledged shortcoming of WordNet is that it lacks a distinction between word meanings which are systematically related (polysemy), and those which are coincidental (homonymy). Several previous works have attempted to fill this gap, by inferring this information using computational methods. We revisit this task, and exploit recent advances in language modelling to synthesise homonymy annotation for Princeton WordNet. Previous approaches treat the problem using clustering methods; by contrast, our method works by linking WordNet to the Oxford English Dictionary, which contains the information we need. To perform this alignment, we pair definitions based on their proximity in an embedding space produced by a Transformer model. Despite the simplicity of this approach, our best model attains an F1 of .97 on an evaluation set that we annotate. The outcome of our work is a high-quality homonymy annotation layer for Princeton WordNet, which we release.
"We are a startup to the core": A qualitative interview study on the security and privacy development practices in Turkish software startups
Venue: arXiv cs.HC 2022
Year: 2022

Security and privacy are often neglected in software development, and rarely a priority for developers. This insight is commonly based on research conducted by researchers and on developer populations living and working in the United States, Europe, and the United Kingdom. However, the production of software is global, and crucial populations in important technology hubs are not adequately studied. The software startup scene in Turkey is impactful, and comprehension, knowledge, and mitigations related to software security and privacy remain understudied. To close this research gap, we conducted a semi-structured interview study with 16 developers working in Turkish software startups. The goal of the interview study was to analyze if and how developers ensure that their software is secure and preserves user privacy. Our main finding is that developers rarely prioritize security and privacy, due to a lack of awareness, skills, and resources. We find that regulations can make a positive impact on security and privacy. Based on the study, we issue recommendations for industry, individual developers, research, educators, and regulators. Our recommendations can inform a more globalized approach to security and privacy in software development.
Near-Field Wideband Channel Estimation for Extremely Large-Scale MIMO
Venue: arXiv cs.IT 2022
Year: 2022

Extremely large-scale multiple-input-multiple-output (XL-MIMO) at millimeter-wave (mmWave) and terahertz (THz) bands plays an important role in supporting extreme high beamforming gain as well as ultra-wideband spectrum resources. Unfortunately, accurate wideband XL-MIMO channel estimation suffers from the new challenge called as the near-field beam split effect. Prior works either neglect the accurate near-field channel model or fail to exploit the beam split effect, resulting in poor channel estimation accuracy for wideband XL-MIMO. To tackle this problem, this paper proposes a bilinear pattern detection (BPD) based approach to accurately recover the wideband XL-MIMO channel. Specifically, by analyzing the characteristics of near-field wideband channels, we first reveal the bilinear pattern of the near-field beam split effect, which implies that the sparse support set of near-field channels in both the angle and the distance domains can be regarded as a linear function against frequency. Then, inspired by the classical simultaneously orthogonal matching pursuit technique, we use the bilinear pattern to estimate the angle-of-arrival (AoA) and distance parameters of each near-field path component at all frequencies. In this way, the entire wideband XL-MIMO channel can be recovered by compressed sensing algorithms. Moreover, we provide the computational complexity of the proposed algorithm compared with existing algorithms. Finally, simulation results demonstrate that our scheme can achieve the accurate estimation of the near-field wideband XL-MIMO channel in the presence of near-field beam split effect.
Free-form 3D Scene Inpainting with Dual-stream GAN
Venue: arXiv cs.CV 2022
Year: 2022

Nowadays, the need for user editing in a 3D scene has rapidly increased due to the development of AR and VR technology. However, the existing 3D scene completion task (and datasets) cannot suit the need because the missing regions in scenes are generated by the sensor limitation or object occlusion. Thus, we present a novel task named free-form 3D scene inpainting. Unlike scenes in previous 3D completion datasets preserving most of the main structures and hints of detailed shapes around missing regions, the proposed inpainting dataset, FF-Matterport, contains large and diverse missing regions formed by our free-form 3D mask generation algorithm that can mimic human drawing trajectories in 3D space. Moreover, prior 3D completion methods cannot perform well on this challenging yet practical task, simply interpolating nearby geometry and color context. Thus, a tailored dual-stream GAN method is proposed. First, our dual-stream generator, fusing both geometry and color information, produces distinct semantic boundaries and solves the interpolation issue. To further enhance the details, our lightweight dual-stream discriminator regularizes the geometry and color edges of the predicted scenes to be realistic and sharp. We conducted experiments with the proposed FF-Matterport dataset. Qualitative and quantitative results validate the superiority of our approach over existing scene completion methods and the efficacy of all proposed components.
Check-worthy Claim Detection across Topics for Automated Fact-checking
Venue: arXiv cs.CL 2022
Year: 2022

An important component of an automated fact-checking system is the claim check-worthiness detection system, which ranks sentences by prioritising them based on their need to be checked. Despite a body of research tackling the task, previous research has overlooked the challenging nature of identifying check-worthy claims across different topics. In this paper, we assess and quantify the challenge of detecting check-worthy claims for new, unseen topics. After highlighting the problem, we propose the AraCWA model to mitigate the performance deterioration when detecting check-worthy claims across topics. The AraCWA model enables boosting the performance for new topics by incorporating two components for few-shot learning and data augmentation. Using a publicly available dataset of Arabic tweets consisting of 14 different topics, we demonstrate that our proposed data augmentation strategy achieves substantial improvements across topics overall, where the extent of the improvement varies across topics. Further, we analyse the semantic similarities between topics, suggesting that the similarity metric could be used as a proxy to determine the difficulty level of an unseen topic prior to undertaking the task of labelling the underlying sentences.
Fine-grained Czech News Article Dataset: An Interdisciplinary Approach to Trustworthiness Analysis
Venue: arXiv cs.CL 2022
Year: 2022

We present the Verifee Dataset: a novel dataset of news articles with fine-grained trustworthiness annotations. We develop a detailed methodology that assesses the texts based on their parameters encompassing editorial transparency, journalist conventions, and objective reporting while penalizing manipulative techniques. We bring aboard a diverse set of researchers from social, media, and computer sciences to overcome barriers and limited framing of this interdisciplinary problem. We collect over $10,000$ unique articles from almost $60$ Czech online news sources. These are categorized into one of the $4$ classes across the credibility spectrum we propose, raging from entirely trustworthy articles all the way to the manipulative ones. We produce detailed statistics and study trends emerging throughout the set. Lastly, we fine-tune multiple popular sequence-to-sequence language models using our dataset on the trustworthiness classification task and report the best testing F-1 score of $0.52$. We open-source the dataset, annotation methodology, and annotators' instructions in full length at https://verifee.ai/research to enable easy build-up work. We believe similar methods can help prevent disinformation and educate in the realm of media literacy.
An Ethical Trajectory Planning Algorithm for Autonomous Vehicles
Venue: arXiv cs.RO 2022
Year: 2022

With the rise of AI and automation, moral decisions are being put into the hands of algorithms that were formerly the preserve of humans. In autonomous driving, a variety of such decisions with ethical implications are made by algorithms for behavior and trajectory planning. Therefore, we present an ethical trajectory planning algorithm with a framework that aims at a fair distribution of risk among road users. Our implementation incorporates a combination of five essential ethical principles: minimization of the overall risk, priority for the worst-off, equal treatment of people, responsibility, and maximum acceptable risk. To the best of the authors' knowledge, this is the first ethical algorithm for trajectory planning of autonomous vehicles in line with the 20 recommendations from the EU Commission expert group and with general applicability to various traffic situations. We showcase the ethical behavior of our algorithm in selected scenarios and provide an empirical analysis of the ethical principles in 2000 scenarios. The code used in this research is available as open-source software.
Provable Fairness for Neural Network Models using Formal Verification
Venue: arXiv cs.LG 2022
Year: 2022

Machine learning models are increasingly deployed for critical decision-making tasks, making it important to verify that they do not contain gender or racial biases picked up from training data. Typical approaches to achieve fairness revolve around efforts to clean or curate training data, with post-hoc statistical evaluation of the fairness of the model on evaluation data. In contrast, we propose techniques to \emph{prove} fairness using recently developed formal methods that verify properties of neural network models.Beyond the strength of guarantee implied by a formal proof, our methods have the advantage that we do not need explicit training or evaluation data (which is often proprietary) in order to analyze a given trained model. In experiments on two familiar datasets in the fairness literature (COMPAS and ADULTS), we show that through proper training, we can reduce unfairness by an average of 65.4\% at a cost of less than 1\% in AUC score.
Rethinking Cooking State Recognition with Vision Transformers
Venue: arXiv cs.CV 2022
Year: 2022

To ensure proper knowledge representation of the kitchen environment, it is vital for kitchen robots to recognize the states of the food items that are being cooked. Although the domain of object detection and recognition has been extensively studied, the task of object state classification has remained relatively unexplored. The high intra-class similarity of ingredients during different states of cooking makes the task even more challenging. Researchers have proposed adopting Deep Learning based strategies in recent times, however, they are yet to achieve high performance. In this study, we utilized the self-attention mechanism of the Vision Transformer (ViT) architecture for the Cooking State Recognition task. The proposed approach encapsulates the globally salient features from images, while also exploiting the weights learned from a larger dataset. This global attention allows the model to withstand the similarities between samples of different cooking objects, while the employment of transfer learning helps to overcome the lack of inductive bias by utilizing pretrained weights. To improve recognition accuracy, several augmentation techniques have been employed as well. Evaluation of our proposed framework on the `Cooking State Recognition Challenge Dataset' has achieved an accuracy of 94.3%, which significantly outperforms the state-of-the-art.
IMAGINE: An Integrated Model of Artificial Intelligence-Mediated Communication Effects
Venue: arXiv cs.AI 2022
Year: 2022

Artificial Intelligence (AI) is transforming all fields of knowledge and production. From surgery, autonomous driving, to image and video creation, AI seems to make possible hitherto unimaginable processes of automation and efficient creation. Media and communication are not an exception, and we are currently witnessing the dawn of powerful AI tools capable of creating artistic images from simple keywords, or to capture emotions from facial expression. These examples may be only the beginning of what can be in the future the engines for automatic AI real time creation of media content linked to the emotional and behavioural responses of individuals. Although it may seem we are still far from there, it is already the moment to adapt our theories about media to the hypothetical scenario in which content production can be done without human intervention, and governed by the controlled any reactions of the individual to the exposure to media content. Following that, I propose the definition of the Integrated Model of Artificial Intelligence-Mediated Communication Effects (IMAGINE), and its consequences on the way we understand media evolution (Scolari, 2012) and we think about media effects (Potter, 2010). The conceptual framework proposed is aimed to help scholars theorizing and doing research in a scenario of continuous real-time connection between AI measurement of people's responses to media, and the AI creation of content, with the objective of optimizing and maximizing the processes of influence. Parasocial interaction and real-time beautification are used as examples to model the functioning of the IMAGINE process.
Distribution-aware Goal Prediction and Conformant Model-based Planning for Safe Autonomous Driving
Venue: arXiv cs.RO 2022
Year: 2022

The feasibility of collecting a large amount of expert demonstrations has inspired growing research interests in learning-to-drive settings, where models learn by imitating the driving behaviour from experts. However, exclusively relying on imitation can limit agents' generalisability to novel scenarios that are outside the support of the training data. In this paper, we address this challenge by factorising the driving task, based on the intuition that modular architectures are more generalisable and more robust to changes in the environment compared to monolithic, end-to-end frameworks. Specifically, we draw inspiration from the trajectory forecasting community and reformulate the learning-to-drive task as obstacle-aware perception and grounding, distribution-aware goal prediction, and model-based planning. Firstly, we train the obstacle-aware perception module to extract salient representation of the visual context. Then, we learn a multi-modal goal distribution by performing conditional density-estimation using normalising flow. Finally, we ground candidate trajectory predictions road geometry, and plan the actions based on on vehicle dynamics. Under the CARLA simulator, we report state-of-the-art results on the CARNOVEL benchmark.
A systematic literature review on Internet of Vehicles Security
Venue: arXiv cs.CR 2022
Year: 2022

The Internet of Vehicles IoV commonly referred to as connected automobiles is a vast network that connects various entities including users sensors and vehicles They will connect across a network to lessen traffic accidents and improve both the security and safety of smart vehicles The Internet of Vehicles is subject to a wide variety of threats including spoofing attacks recognition attacks privacy attacks and verification attacks Our the primary concern when creating any new smart gadget is the users safety which will be improved by identifying solutions to the various cyber threats Therefore we will cover the security of smart automobiles in this literature review including their attacks and solutions.
Improving Levenberg-Marquardt Algorithm for Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

We explore the usage of the Levenberg-Marquardt (LM) algorithm for regression (non-linear least squares) and classification (generalized Gauss-Newton methods) tasks in neural networks. We compare the performance of the LM method with other popular first-order algorithms such as SGD and Adam, as well as other second-order algorithms such as L-BFGS , Hessian-Free and KFAC. We further speed up the LM method by using adaptive momentum, learning rate line search, and uphill step acceptance.
Stateful Switch: Optimized Time Series Release with Local Differential Privacy
Venue: arXiv cs.CR 2022
Year: 2022

Time series data have numerous applications in big data analytics. However, they often cause privacy issues when collected from individuals. To address this problem, most existing works perturb the values in the time series while retaining their temporal order, which may lead to significant distortion of the values. Recently, we propose TLDP model that perturbs temporal perturbation to ensure privacy guarantee while retaining original values. It has shown great promise to achieve significantly higher utility than value perturbation mechanisms in many time series analysis. However, its practicability is still undermined by two factors, namely, utility cost of extra missing or empty values, and inflexibility of privacy budget settings. To address them, in this paper we propose {\it switch} as a new two-way operation for temporal perturbation, as opposed to the one-way {\it dispatch} operation. The former inherently eliminates the cost of missing, empty or repeated values. Optimizing switch operation in a {\it stateful} manner, we then propose $StaSwitch$ mechanism for time series release under TLDP. Through both analytical and empirical studies, we show that $StaSwitch$ has significantly higher utility for the published time series than any state-of-the-art temporal- or value-perturbation mechanism, while allowing any combination of privacy budget settings.
A Survey on Password Guessing
Venue: arXiv cs.CR 2022
Year: 2022

Text password has served as the most popular method for user authentication so far, and is not likely to be totally replaced in foreseeable future. Password authentication offers several desirable properties (e.g., low-cost, highly available, easy-to-implement, reusable). However, it suffers from a critical security issue mainly caused by the inability to memorize complicated strings of humans. Users tend to choose easy-to-remember passwords which are not uniformly distributed in the key space. Thus, user-selected passwords are susceptible to guessing attacks. In order to encourage and support users to use strong passwords, it is necessary to simulate automated password guessing methods to determine the passwords' strength and identify weak passwords. A large number of password guessing models have been proposed in the literature. However, little attention was paid to the task of providing a systematic survey which is necessary to review the state-of-the-art approaches, identify gaps, and avoid duplicate studies. Motivated by that, we conduct a comprehensive survey on all password guessing studies presented in the literature from 1979 to 2022. We propose a generic methodology map to present an overview of existing methods. Then, we explain each representative approach in detail. The experimental procedures and available datasets used to evaluate password guessing models are summarized, and the reported performances of representative studies are compared. Finally, the current limitations and the open problems as future research directions are discussed. We believe that this survey is helpful to both experts and newcomers who are interested in password security