OpenRaiser/Intern-Atlas huggingface dataset:
Applications of Federated Learning in Manufacturing: Identifying the Challenges and Exploring the Future Directions with Industry 4.0 and 5.0 Visions
Venue: arXiv cs.LG 2023
Year: 2023

In manufacturing settings, data collection and analysis are often a time-consuming, challenging, and costly process. It also hinders the use of advanced machine learning and data-driven methods which require a substantial amount of offline training data to generate good results. It is particularly challenging for small manufacturers who do not share the resources of a large enterprise. Recently, with the introduction of the Internet of Things (IoT), data can be collected in an integrated manner across the factory in real-time, sent to the cloud for advanced analysis, and used to update the machine learning model sequentially. Nevertheless, small manufacturers face two obstacles in reaping the benefits of IoT: they may be unable to afford or generate enough data to operate a private cloud, and they may be hesitant to share their raw data with a public cloud. Federated learning (FL) is an emerging concept of collaborative learning that can help small-scale industries address these issues and learn from each other without sacrificing their privacy. It can bring together diverse and geographically dispersed manufacturers under the same analytics umbrella to create a win-win situation. However, the widespread adoption of FL across multiple manufacturing organizations remains a significant challenge. This study aims to review the challenges and future directions of applying federated learning in the manufacturing industry, with a specific emphasis on the perspectives of Industry 4.0 and 5.0.
Physical Adversarial Attacks on Deep Neural Networks for Traffic Sign Recognition: A Feasibility Study
Venue: arXiv cs.CV 2023
Year: 2023

Deep Neural Networks (DNNs) are increasingly applied in the real world in safety critical applications like advanced driver assistance systems. An example for such use case is represented by traffic sign recognition systems. At the same time, it is known that current DNNs can be fooled by adversarial attacks, which raises safety concerns if those attacks can be applied under realistic conditions. In this work we apply different black-box attack methods to generate perturbations that are applied in the physical environment and can be used to fool systems under different environmental conditions. To the best of our knowledge we are the first to combine a general framework for physical attacks with different black-box attack methods and study the impact of the different methods on the success rate of the attack under the same setting. We show that reliable physical adversarial attacks can be performed with different methods and that it is also possible to reduce the perceptibility of the resulting perturbations. The findings highlight the need for viable defenses of a DNN even in the black-box case, but at the same time form the basis for securing a DNN with methods like adversarial training which utilizes adversarial attacks to augment the original training data.
Spherical Wavefronts Improve MU-MIMO Spectral Efficiency When Using Electrically Large Arrays
Venue: arXiv cs.IT 2023
Year: 2023

Modern MIMO communication systems are almost exclusively designed under the assumption of locally plane wavefronts over antenna arrays. This is known as the far-field approximation and is soundly justified at sub-6-GHz frequencies at most relevant transmission ranges. However, when higher frequencies and shorter transmission ranges are used, the wave curvature over the array is no longer negligible, and arrays operate in the so-called radiative near-field region. This letter aims to show that the classical far-field approximation may significantly underestimate the achievable spectral efficiency of multi-user MIMO communications operating in the 30-GHz bands and above, even at ranges beyond the Fraunhofer distance. For planar arrays with typical sizes, we show that computing combining schemes based on the far-field model significantly reduces the channel gain and spatial multiplexing capability. When the radiative near-field model is used, interference rejection schemes, such as the optimal minimum mean-square-error combiner, appear to be very promising, when combined with electrically large arrays, to meet the stringent requirements of next-generation networks.
Online Black-Box Confidence Estimation of Deep Neural Networks
Venue: arXiv cs.CV 2023
Year: 2023

Autonomous driving (AD) and advanced driver assistance systems (ADAS) increasingly utilize deep neural networks (DNNs) for improved perception or planning. Nevertheless, DNNs are quite brittle when the data distribution during inference deviates from the data distribution during training. This represents a challenge when deploying in partly unknown environments like in the case of ADAS. At the same time, the standard confidence of DNNs remains high even if the classification reliability decreases. This is problematic since following motion control algorithms consider the apparently confident prediction as reliable even though it might be considerably wrong. To reduce this problem real-time capable confidence estimation is required that better aligns with the actual reliability of the DNN classification. Additionally, the need exists for black-box confidence estimation to enable the homogeneous inclusion of externally developed components to an entire system. In this work we explore this use case and introduce the neighborhood confidence (NHC) which estimates the confidence of an arbitrary DNN for classification. The metric can be used for black-box systems since only the top-1 class output is required and does not need access to the gradients, the training dataset or a hold-out validation dataset. Evaluation on different data distributions, including small in-domain distribution shifts, out-of-domain data or adversarial attacks, shows that the NHC performs better or on par with a comparable method for online white-box confidence estimation in low data regimes which is required for real-time capable AD/ADAS.
Joint Task and Data Oriented Semantic Communications: A Deep Separate Source-channel Coding Scheme
Venue: arXiv cs.IT 2023
Year: 2023

Semantic communications are expected to accomplish various semantic tasks with relatively less spectrum resource by exploiting the semantic feature of source data. To simultaneously serve both the data transmission and semantic tasks, joint data compression and semantic analysis has become pivotal issue in semantic communications. This paper proposes a deep separate source-channel coding (DSSCC) framework for the joint task and data oriented semantic communications (JTD-SC) and utilizes the variational autoencoder approach to solve the rate-distortion problem with semantic distortion. First, by analyzing the Bayesian model of the DSSCC framework, we derive a novel rate-distortion optimization problem via the Bayesian inference approach for general data distributions and semantic tasks. Next, for a typical application of joint image transmission and classification, we combine the variational autoencoder approach with a forward adaption scheme to effectively extract image features and adaptively learn the density information of the obtained features. Finally, an iterative training algorithm is proposed to tackle the overfitting issue of deep learning models. Simulation results reveal that the proposed scheme achieves better coding gain as well as data recovery and classification performance in most scenarios, compared to the classical compression schemes and the emerging deep joint source-channel schemes.
Contracting edges to destroy a pattern: A complexity study
Venue: arXiv cs.DS 2023
Year: 2023

Given a graph G and an integer k, the objective of the $Π$-Contraction problem is to check whether there exists at most k edges in G such that contracting them in G results in a graph satisfying the property $Π$. We investigate the problem where $Π$ is `H-free' (without any induced copies of H). It is trivial that H-free Contraction is polynomial-time solvable if H is a complete graph of at most two vertices. We prove that, in all other cases, the problem is NP-complete. We then investigate the fixed-parameter tractability of these problems. We prove that whenever H is a tree, except for seven trees, H-free Contraction is W[2]-hard. This result along with the known results leaves behind three unknown cases among trees.
Evaluation of Automatically Constructed Word Meaning Explanations
Venue: arXiv cs.CL 2023
Year: 2023

Preparing exact and comprehensive word meaning explanations is one of the key steps in the process of monolingual dictionary writing. In standard methodology, the explanations need an expert lexicographer who spends a substantial amount of time checking the consistency between the descriptive text and corpus evidence. In the following text, we present a new tool that derives explanations automatically based on collective information from very large corpora, particularly on word sketches. We also propose a quantitative evaluation of the constructed explanations, concentrating on explanations of nouns. The methodology is to a certain extent language independent; however, the presented verification is limited to Czech and English. We show that the presented approach allows to create explanations that contain data useful for understanding the word meaning in approximately 90% of cases. However, in many cases, the result requires post-editing to remove redundant information.
PACCART: Reinforcing Trust in Multiuser Privacy Agreement Systems
Venue: arXiv cs.CR 2023
Year: 2023

Collaborative systems, such as Online Social Networks and the Internet of Things, enable users to share privacy sensitive content. Content in these systems is often co-owned by multiple users with different privacy expectations, leading to possible multiuser privacy conflicts. In order to resolve these conflicts, various agreement mechanisms have been designed and agents that could participate in such mechanisms have been proposed. However, research shows that users hesitate to use software tools for managing their privacy. To remedy this, we argue that users should be supported by trustworthy agents that adhere to the following criteria: (i) concealment of privacy preferences, such that only necessary information is shared with others, (ii) equity of treatment, such that different kinds of users are supported equally, (iii) collaboration of users, such that a group of users can support each other in agreement and (iv) explainability of actions, such that users know why certain information about them was shared to reach a decision. Accordingly, this paper proposes PACCART, an open-source agent that satisfies these criteria. Our experiments over simulations and user study indicate that PACCART increases user trust significantly.
Formation-of-Arrays Antenna Technology for High-Throughput Mobile Non-Terrestrial Networks
Venue: arXiv cs.IT 2023
Year: 2023

Effective integration of terrestrial and non-terrestrial segments is one of the key research avenues in the design of current and future wireless communication networks. To this aim, modern communication-satellite constellations intend to attain sufficiently high throughput in terms of bit rate per unit area on the ground by rather aggressive patterns of spatial frequency reuse. This goal calls for on-board narrow-beam antennas, whose size turns out to be in many cases incompatible with the size/mass and accommodation constraints of the hosting satellite. This paper investigates the attainable performance of large distributed arrays of antennas implemented as the ensemble of a few to many simpler sub-antennas of smaller sizes, carried by one (small) satellite each. The sub-antennas can in their turn be implemented like (regular) 2D arrays of simple radiating elements, realizing an overall (distributed) antenna architecture that we call "Formation of Arrays" (FoA). The satellites that implement this radiating architecture need to be relatively close to each other and constitute a formation of flying objects, to be coordinated and controlled as a whole. In this paper, we develop a theoretical analysis of an FoA antenna, and we show how to take advantage of this new technology to improve network throughput in a multi-beam S-band mobile communication network with low-earth or geostationary orbiting satellites directly providing 5G-like communication services to hand-held user terminals.
Neural networks for learning personality traits from natural language
Venue: arXiv cs.CL 2023
Year: 2023

Personality is considered one of the most influential research topics in psychology, as it predicts many consequential outcomes such as mental and physical health and explains human behaviour. With the widespread use of social networks as a means of communication, it is becoming increasingly important to develop models that can automatically and accurately read the essence of individuals based solely on their writing. In particular, the convergence of social and computer sciences has led researchers to develop automatic approaches for extracting and studying "hidden" information in textual data on the internet. The nature of this thesis project is highly experimental, and the motivation behind this work is to present detailed analyses on the topic, as currently there are no significant investigations of this kind. The objective is to identify an adequate semantic space that allows for defining the personality of the object to which a certain text refers. The starting point is a dictionary of adjectives that psychological literature defines as markers of the five major personality traits, or Big Five. In this work, we started with the implementation of fully-connected neural networks as a basis for understanding how simple deep learning models can provide information on hidden personality characteristics. Finally, we use a class of distributional algorithms invented in 2013 by Tomas Mikolov, which consists of using a convolutional neural network that learns the contexts of words in an unsupervised way. In this way, we construct an embedding that contains the semantic information on the text, obtaining a kind of "geometry of meaning" in which concepts are translated into linear relationships. With this last experiment, we hypothesize that an individual writing style is largely coupled with their personality traits.
Let's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations
Venue: arXiv cs.CL 2023
Year: 2023

The emergence of an AI-powered chatbot that can generate human-like sentences and write coherent essays has caught the world's attention. This paper discusses the historical overview of chatbots and the technology behind Chat Generative Pre-trained Transformer, better known as ChatGPT. Moreover, potential applications of ChatGPT in various domains, including healthcare, education, and research, are highlighted. Despite promising results, there are several privacy and ethical concerns surrounding ChatGPT. In addition, we highlight some of the important limitations of the current version of ChatGPT. We also ask ChatGPT to provide its point of view and present its responses to several questions we attempt to answer.
Denoising Diffusion Samplers
Venue: arXiv cs.LG 2023
Year: 2023

Denoising diffusion models are a popular class of generative models providing state-of-the-art results in many domains. One adds gradually noise to data using a diffusion to transform the data distribution into a Gaussian distribution. Samples from the generative model are then obtained by simulating an approximation of the time-reversal of this diffusion initialized by Gaussian samples. Practically, the intractable score terms appearing in the time-reversed process are approximated using score matching techniques. We explore here a similar idea to sample approximately from unnormalized probability density functions and estimate their normalizing constants. We consider a process where the target density diffuses towards a Gaussian. Denoising Diffusion Samplers (DDS) are obtained by approximating the corresponding time-reversal. While score matching is not applicable in this context, we can leverage many of the ideas introduced in generative modeling for Monte Carlo sampling. Existing theoretical results from denoising diffusion models also provide theoretical guarantees for DDS. We discuss the connections between DDS, optimal control and Schrödinger bridges and finally demonstrate DDS experimentally on a variety of challenging sampling tasks.
Do as You Say: Consistency Detection of Data Practice in Program Code and Privacy Policy in Mini-App
Venue: arXiv cs.CR 2023
Year: 2023

Mini-app is an emerging form of mobile application that combines web technology with native capabilities. Its features, e.g., no need to download and no installation, have made it popular rapidly. However, privacy issues that violate the laws or regulations are breeding in the swiftly expanding mini-app ecosystem. The consistency between what the mini-app does about the data in the program code and what it declares in its privacy policy description is important. But no work has systematically investigated the privacy problem of the mini-app before. In this paper, to our best knowledge, we are the first to conduct the compliance detection of data practice and policy description in mini-apps. In this paper, we first customize a taint analysis method based on data entity dependency network to adapt to the characteristics of the JavaScript language in the mini-apps. Then, we transform data types and data operations to data practices in program codes and privacy policies, so as to finish a fine-grained consistency matching model.We crawl 100,000 mini-apps on WeChat client in the wild and extract 2,998 with a privacy policy. Among them, only 318 meet the consistency requirements, 2,680 are inconsistent, and the proportion of inconsistencies is as high as 89.4%. The inconsistency in the mini-app is very serious. Based on 6 real-world cases analyzed, in order to reduce this potential data leakage risk, we suggest that the developer should reduce the collection of irrelevant information and the straightforward use of templates, and the platform should provide data flow detection tools and privacy policy writing support.
AI-Driven Container Security Approaches for 5G and Beyond: A Survey
Venue: arXiv cs.CR 2023
Year: 2023

The rising use of microservices based software deployment on the cloud leverages containerized software extensively. The security of applications running inside containers as well as the container environment itself are critical infrastructure in the cloud setting and 5G. To address the security concerns, research efforts have been focused on container security with subfields such as intrusion detection, malware detection and container placement strategies. These security efforts are roughly divided into two categories: rule based approaches and machine learning that can respond to novel threats. In this study, we have surveyed the container security literature focusing on approaches that leverage machine learning to address security challenges.
Argument Mining using BERT and Self-Attention based Embeddings
Venue: arXiv cs.CL 2023
Year: 2023

Argument mining automatically identifies and extracts the structure of inference and reasoning conveyed in natural language arguments. To the best of our knowledge, most of the state-of-the-art works in this field have focused on using tree-like structures and linguistic modeling. But, these approaches are not able to model more complex structures which are often found in online forums and real world argumentation structures. In this paper, a novel methodology for argument mining is proposed which employs attention-based embeddings for link prediction to model the causational hierarchies in typical argument structures prevalent in online discourse.
On Dillon's property of $(n,m)$-functions
Venue: arXiv cs.IT 2023
Year: 2023

Dillon observed that an APN function $F$ over $\mathbb{F}_2^{n}$ with $n$ greater than $2$ must satisfy the condition $\{F(x) + F(y) + F(z) + F(x + y + z) \,:\, x,y,z \in\mathbb{F}_2^n\}= \mathbb{F}_2^n$. Recently, Taniguchi (2023) generalized this condition to functions defined from $\mathbb{F}_2^n$ to $\mathbb{F}_2^m$, with $m>n$, calling it the D-property. Taniguchi gave some characterizations of APN functions satisfying the D-property and provided some families of APN functions from $\mathbb{F}_2^n$ to $\mathbb{F}_2^{n+1}$ satisfying this property. In this work, we further study the D-property for $(n,m)$-functions with $m\ge n$. We give some combinatorial bounds on the dimension $m$ for the existence of such functions. Then, we characterize the D-property in terms of the Walsh transform and for quadratic functions we give a characterization of this property in terms of the ANF. We also give a simplification on checking the D-property for quadratic functions, which permits to extend some of the APN families provided by Taniguchi. We further focus on the class of the plateaued functions, providing conditions for the D-property.
Efficient Informed Proposals for Discrete Distributions via Newton's Series Approximation
Venue: arXiv cs.LG 2023
Year: 2023

Gradients have been exploited in proposal distributions to accelerate the convergence of Markov chain Monte Carlo algorithms on discrete distributions. However, these methods require a natural differentiable extension of the target discrete distribution, which often does not exist or does not provide effective gradient guidance. In this paper, we develop a gradient-like proposal for any discrete distribution without this strong requirement. Built upon a locally-balanced proposal, our method efficiently approximates the discrete likelihood ratio via Newton's series expansion to enable a large and efficient exploration in discrete spaces. We show that our method can also be viewed as a multilinear extension, thus inheriting its desired properties. We prove that our method has a guaranteed convergence rate with or without the Metropolis-Hastings step. Furthermore, our method outperforms a number of popular alternatives in several different experiments, including the facility location problem, extractive text summarization, and image retrieval.
Generator Matrices by Solving Integer Linear Programs
Venue: arXiv cs.GR 2023
Year: 2023

In quasi-Monte Carlo methods, generating high-dimensional low discrepancy sequences by generator matrices is a popular and efficient approach. Historically, constructing or finding such generator matrices has been a hard problem. In particular, it is challenging to take advantage of the intrinsic structure of a given numerical problem to design samplers of low discrepancy in certain subsets of dimensions. To address this issue, we devise a greedy algorithm allowing us to translate desired net properties into linear constraints on the generator matrix entries. Solving the resulting integer linear program yields generator matrices that satisfy the desired net properties. We demonstrate that our method finds generator matrices in challenging settings, offering low discrepancy sequences beyond the limitations of classic constructions.