OpenRaiser/Intern-Atlas huggingface dataset:
Sharp asymptotics on the compression of two-layer neural networks
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M<N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimize the population L_2 loss between the outputs of the target and of the compressed network, under the assumption of Gaussian inputs. By using tools from high-dimensional probability, we show that this non-convex problem can be simplified when the target network is sufficiently over-parameterized, and provide the error rate of this approximation as a function of the input dimension and N. In this mean-field limit, the simplified objective, as well as the optimal weights of the compressed network, does not depend on the realization of the target network, but only on expected scaling factors. Furthermore, for networks with ReLU activation, we conjecture that the optimum of the simplified optimization problem is achieved by taking weights on the Equiangular Tight Frame (ETF), while the scaling of the weights and the orientation of the ETF depend on the parameters of the target network. Numerical evidence is provided to support this conjecture.
An Application of Scenario Exploration to Find New Scenarios for the Development and Testing of Automated Driving Systems in Urban Scenarios
Venue: arXiv cs.SE 2022
Year: 2022

Verification and validation are major challenges for developing automated driving systems. A concept that gets more and more recognized for testing in automated driving is scenario-based testing. However, it introduces the problem of what scenarios are relevant for testing and which are not. This work aims to find relevant, interesting, or critical parameter sets within logical scenarios by utilizing Bayes optimization and Gaussian processes. The parameter optimization is done by comparing and evaluating six different metrics in two urban intersection scenarios. Finally, a list of ideas this work leads to and should be investigated further is presented.
Towards Robotic Laboratory Automation Plug & Play: Survey and Concept Proposal on Teaching-free Robot Integration with the LAPP Digital Twin
Venue: arXiv cs.RO 2022
Year: 2022

The Laboratory Automation Plug & Play (LAPP) framework is an over-arching reference architecture concept for the integration of robots in life science laboratories. The plug & play nature lies in the fact that manual configuration is not required, including the teaching of the robots. In this paper a digital twin (DT) based concept is proposed that outlines the types of information that have to be provided for each relevant component of the system. In particular, for the devices interfacing with the robot, the robot positions have to be defined beforehand in a device-attached coordinate system (CS) by the vendor. This CS has to be detectable by the vision system of the robot by means of optical markers placed on the front side of the device. With that, the robot is capable of tending the machine by performing the pick-and-place type transportation of standard sample carriers. This basic use case is the primary scope of the LAPP-DT framework. The hardware scope is limited to simple benchtop and mobile manipulators with parallel grippers at this stage. This paper first provides an overview of relevant literature and state-of-the-art solutions, after which it outlines the framework on the conceptual level, followed by the specification of the relevant DT parameters for the robot, for the devices and for the facility. Finally, appropriate technologies and strategies are identified for the implementation.
Working with Affective Computing: Exploring UK Public Perceptions of AI enabled Workplace Surveillance
Venue: arXiv cs.HC 2022
Year: 2022

This paper explores public perceptions around the role of affective computing in the workplace. It uses a series of design fictions with 46 UK based participants, unpacking their perspectives on the advantages and disadvantages of tracking the emotional state of workers. The scenario focuses on mundane uses of biometric sensing in a sales environment, and how this could shape management approaches with workers. The paper structure is as follows: section 1 provides a brief introduction; section 2 provides an overview of the innovative design fiction methodology; section 3 explores wider shifts around IT in the workplace; section 4 provides some legal analysis exploring emergence of AI in the workplace; and section 5 presents themes from the study data. The latter section includes discussion on concerns around functionality and accuracy of affective computing systems, and their impacts on surveillance, human agency, and worker/management interactions.
Self-Sovereign Identity as a Service: Architecture in Practice
Venue: arXiv cs.SE 2022
Year: 2022

Self-sovereign identity (SSI) has gained a large amount of interest. It enables physical entities to retain ownership and control of their digital identities, which naturally forms a conceptual decentralized architecture. With the support of the distributed ledger technology (DLT), it is possible to implement this conceptual decentralized architecture in practice and further bring technical advantages such as privacy protection, security enhancement, high availability. However, developing such a relatively new identity model has high costs and risks with uncertainty. To facilitate the use of the DLT-based SSI in practice, we formulate Self-Sovereign Identity as a Service (SSIaaS), a concept that enables a system, especially a system cluster, to readily adopt SSI as its identity model for identification, authentication, and authorization. We propose a practical architecture by elaborating the service concept, SSI, and DLT to implement SSIaaS platforms and SSI services. Besides, we present an architecture for constructing and customizing SSI services with a set of architectural patterns and provide corresponding evaluations. Furthermore, we demonstrate the feasibility of our proposed architecture in practice with Selfid, an SSIaaS platform based on our proposed architecture.
Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation
Venue: arXiv cs.RO 2022
Year: 2022

In recent years, policy learning methods using either reinforcement or imitation have made significant progress. However, both techniques still suffer from being computationally expensive and requiring large amounts of training data. This problem is especially prevalent in real-world robotic manipulation tasks, where access to ground truth scene features is not available and policies are instead learned from raw camera observations. In this paper, we demonstrate the efficacy of learning image keypoints via the Dense Correspondence pretext task for downstream policy learning. Extending prior work to challenging multi-object scenes, we show that our model can be trained to deal with important problems in representation learning, primarily scale-invariance and occlusion. We evaluate our approach on diverse robot manipulation tasks, compare it to other visual representation learning approaches, and demonstrate its flexibility and effectiveness for sample-efficient policy learning.
GraphMapper: Efficient Visual Navigation by Scene Graph Generation
Venue: arXiv cs.CV 2022
Year: 2022

Understanding the geometric relationships between objects in a scene is a core capability in enabling both humans and autonomous agents to navigate in new environments. A sparse, unified representation of the scene topology will allow agents to act efficiently to move through their environment, communicate the environment state with others, and utilize the representation for diverse downstream tasks. To this end, we propose a method to train an autonomous agent to learn to accumulate a 3D scene graph representation of its environment by simultaneously learning to navigate through said environment. We demonstrate that our approach, GraphMapper, enables the learning of effective navigation policies through fewer interactions with the environment than vision-based systems alone. Further, we show that GraphMapper can act as a modular scene encoder to operate alongside existing Learning-based solutions to not only increase navigational efficiency but also generate intermediate scene representations that are useful for other future tasks.
Fronthaul Compression Control for shared Fronthaul Access Networks
Venue: arXiv cs.NI 2022
Year: 2022

There is a widely held belief that future Radio Access Network (RAN) architectures will be characterized by increased levels of virtualization, whereby base station functionalities, traditionally residing at a single location, will be scattered across different logical entities while being interfaced via high-speed fronthaul (FH) links. For the deployment of such FH links, operators are faced with the challenge of maintaining acceptable radio access performance while at the same time keeping deployment costs low. A common practice is to exploit statistical multiplexing by allowing several cells to utilize the same FH link. As a result, in order to cope with the resulting aggregated traffic, different techniques can be used to reduce the required FH data rates. Herein, we focus on FH compression control strategies for multiple-cell/multiple-user scenarios sharing a common FH link. We propose various methods for sounding reference signal (SRS) handling and analyze different FH-aware modulation data compression and scheduling strategies. Considering a full system setup, including the radio and FH access networks, numerical evaluation is conducted using a 5G NR system-level simulator implemented in ns-3. Simulation results show that, under stringent FH capacity constraints, optimized modulation compression strategies provide significant user-perceived throughput gains over baseline strategies (between 5.2x and 6.9x). On top of them, SRS handling methods achieve additional 2% to 41% gains.
On Uncertainty of Dynamic Systems via State Aggregation Coarse-Graining and State Decomposition Fine-Graining Ways
Venue: arXiv cs.IT 2022
Year: 2022

Uncertainty is an important feature of dynamic systems, and entropy has been widely used to measure this attribute. In this Letter, we prove that state aggregation and decomposition can decrease and increase the entropy, respectively, of dynamic systems. More than 20 popular entropies in the literature are summarized and analyzed, and it is noted that none of them breaks this property. Finally, pertinent proofs are given for four cases.
LPC-AD: Fast and Accurate Multivariate Time Series Anomaly Detection via Latent Predictive Coding
Venue: arXiv cs.LG 2022
Year: 2022

This paper proposes LPC-AD, a fast and accurate multivariate time series (MTS) anomaly detection method. LPC-AD is motivated by the ever-increasing needs for fast and accurate MTS anomaly detection methods to support fast troubleshooting in cloud computing, micro-service systems, etc. LPC-AD is fast in the sense that its reduces the training time by as high as 38.2% compared to the state-of-the-art (SOTA) deep learning methods that focus on training speed. LPC-AD is accurate in the sense that it improves the detection accuracy by as high as 18.9% compared to SOTA sophisticated deep learning methods that focus on enhancing detection accuracy. Methodologically, LPC-AD contributes a generic architecture LPC-Reconstruct for one to attain different trade-offs between training speed and detection accuracy. More specifically, LPC-Reconstruct is built on ideas from autoencoder for reducing redundancy in time series, latent predictive coding for capturing temporal dependence in MTS, and randomized perturbation for avoiding overfitting of anomalous dependence in the training data. We present simple instantiations of LPC-Reconstruct to attain fast training speed, where we propose a simple randomized perturbation method. The superior performance of LPC-AD over SOTA methods is validated by extensive experiments on four large real-world datasets. Experiment results also show the necessity and benefit of each component of the LPC-Reconstruct architecture and that LPC-AD is robust to hyper parameters.
Compatible deep neural network framework with financial time series data, including data preprocessor, neural network model and trading strategy
Venue: arXiv cs.LG 2022
Year: 2022

Experience has shown that trading in stock and cryptocurrency markets has the potential to be highly profitable. In this light, considerable effort has been recently devoted to investigate how to apply machine learning and deep learning to interpret and predict market behavior. This research introduces a new deep neural network architecture and a novel idea of how to prepare financial data before feeding them to the model. In the data preparation part, the first step is to generate many features using technical indicators and then apply the XGBoost model for feature engineering. Splitting data into three categories and using separate autoencoders, we extract high-level mixed features at the second step. This data preprocessing is introduced to predict price movements. Regarding modeling, different convolutional layers, an long short-term memory unit, and several fully-connected layers have been designed to perform binary classification. This research also introduces a trading strategy to exploit the trained model outputs. Three different datasets are used to evaluate this method, where results indicate that this framework can provide us with profitable and robust predictions.
Subdivisions and Crossroads: Identifying Hidden Community Structures in a Data Archive's Citation Network
Venue: arXiv cs.DL 2022
Year: 2022

Data archives are an important source of high quality data in many fields, making them ideal sites to study data reuse. By studying data reuse through citation networks, we are able to learn how hidden research communities - those that use the same scientific datasets - are organized. This paper analyzes the community structure of an authoritative network of datasets cited in academic publications, which have been collected by a large, social science data archive: the Interuniversity Consortium for Political and Social Research (ICPSR). Through network analysis, we identified communities of social science datasets and fields of research connected through shared data use. We argue that communities of exclusive data reuse form subdivisions that contain valuable disciplinary resources, while datasets at a "crossroads" broadly connect research communities. Our research reveals the hidden structure of data reuse and demonstrates how interdisciplinary research communities organize around datasets as shared scientific inputs. These findings contribute new ways of describing scientific communities in order to understand the impacts of research data reuse.
Decentralised Update Selection with Semi-Strategic Experts
Venue: arXiv cs.GT 2022
Year: 2022

Motivated by governance models adopted in blockchain applications, we study the problem of selecting appropriate system updates in a decentralised way. Contrary to most existing voting approaches, we use the input of a set of motivated experts of varying levels of expertise. In particular, we develop an approval voting inspired selection mechanism through which the experts approve or disapprove the different updates according to their perception of the quality of each alternative. Given their opinions, and weighted by their expertise level, a single update is then implemented and evaluated, and the experts receive rewards based on their choices. We show that this mechanism always has approximate pure Nash equilibria and that these achieve a constant factor approximation with respect to the quality benchmark of the optimal alternative. Finally, we study the repeated version of the problem, where the weights of the experts are adjusted after each update, according to their performance. Under mild assumptions about the weights, the extension of our mechanism still has approximate pure Nash equilibria in this setting.
Secure Summation: Capacity Region, Groupwise Key, and Feasibility
Venue: arXiv cs.IT 2022
Year: 2022

The secure summation problem is considered, where $K$ users, each holds an input, wish to compute the sum of their inputs at a server securely, i.e., without revealing any information beyond the sum even if the server may collude with any set of up to $T$ users. First, we prove a folklore result for secure summation - to compute $1$ bit of the sum securely, each user needs to send at least $1$ bit to the server, each user needs to hold a key of at least $1$ bit, and all users need to hold collectively some key variables of at least $K-1$ bits. Next, we focus on the symmetric groupwise key setting, where every group of $G$ users share an independent key. We show that for symmetric groupwise keys with group size $G$, when $G > K-T$, the secure summation problem is not feasible; when $G \leq K-T$, to compute $1$ bit of the sum securely, each user needs to send at least $1$ bit to the server and the size of each groupwise key is at least $(K-T-1)/\binom{K-T}{G}$ bits. Finally, we relax the symmetry assumption on the groupwise keys and the colluding user sets; we allow any arbitrary group of users to share an independent key and any arbitrary group of users to collude with the server. For such a general groupwise key and colluding user setting, we show that secure summation is feasible if and only if the hypergraph, where each node is a user and each edge is a group of users sharing the same key, is connected after removing the nodes corresponding to any colluding set of users and their incident edges.
Systematic Mapping Protocol: Variability Management in Dynamic Software Product Lines for Self-Adaptive Systems
Venue: arXiv cs.SE 2022
Year: 2022

Context: The Importance of Dynamic Variability Management in Dynamic Software Product Lines. Objective: Define a protocol for conducting a systematic mapping study to summarize and synthesize evidence on dynamic variability management for Dynamic Software Product Lines in self-adaptive systems. Method: Application the protocol to conduct a systematic mapping study according the guidelines of K. Petersen. Results: A validated protocol to conduct a systematic mapping study. Conclusions: First findings show that it is necessary to visualize new ways to manage variability in dynamic software product lines.
Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks
Venue: arXiv cs.ET 2022
Year: 2022

Neural networks are widely deployed models across many scientific disciplines and commercial endeavors ranging from edge computing and sensing to large-scale signal processing in data centers. The most efficient and well-entrenched method to train such networks is backpropagation, or reverse-mode automatic differentiation. To counter an exponentially increasing energy budget in the artificial intelligence sector, there has been recent interest in analog implementations of neural networks, specifically nanophotonic neural networks for which no analog backpropagation demonstration exists. We design mass-manufacturable silicon photonic neural networks that alternately cascade our custom designed "photonic mesh" accelerator with digitally implemented nonlinearities. These reconfigurable photonic meshes program computationally intensive arbitrary matrix multiplication by setting physical voltages that tune the interference of optically encoded input data propagating through integrated Mach-Zehnder interferometer networks. Here, using our packaged photonic chip, we demonstrate in situ backpropagation for the first time to solve classification tasks and evaluate a new protocol to keep the entire gradient measurement and update of physical device voltages in the analog domain, improving on past theoretical proposals. Our method is made possible by introducing three changes to typical photonic meshes: (1) measurements at optical "grating tap" monitors, (2) bidirectional optical signal propagation automated by fiber switch, and (3) universal generation and readout of optical amplitude and phase. After training, our classification achieves accuracies similar to digital equivalents even in presence of systematic error. Our findings suggest a new training paradigm for photonics-accelerated artificial intelligence based entirely on a physical analog of the popular backpropagation technique.
IEC-61850 Performance Evaluation in a 5G Cellular Network: UDP and TCP Analysis
Venue: arXiv cs.NI 2022
Year: 2022

This chapter summarizes the results obtained from a test bed, which is composed of a microgrid and a wireless network, both relying on real and practical premises. This test bed aims to evaluate application protocols considered in the transport layer of the standard IEC-61850. The application is a data gathering system, where IEC-61850 messages are transmitted from and between different elements in a microgrid system, such as IED, publishers, and subscribers, which are node elements that are part of the utility grid. The standard IEC-61850 defines many protocols, such as SV, GOOSE, and MMS, each of them with a variety of requirements and sharing similar transport communication protocols, such as TCP and UDP. Hence, a testing framework is used in the test bed in order to evaluate the performance of these protocols aiming to enable the communication in a real microgrid deployment. The IEC-61850 messages transported by TCP or UDP are transmitted to a centralized location, where a local database stores the information under different test scenarios and using different transmission sample times. The overall results are promising and indicate that the protocols based on TCP in combination with a 5G cellular network architecture would be suitable for most applications within the studied microgrid case. In the case of UDP, the performance indicated some specific constraints related to the multiple access nature of simultaneous transmissions and also based on the SINR.
RARITYNet: Rarity Guided Affective Emotion Learning Framework
Venue: arXiv cs.CV 2022
Year: 2022

Inspired from the assets of handcrafted and deep learning approaches, we proposed a RARITYNet: RARITY guided affective emotion learning framework to learn the appearance features and identify the emotion class of facial expressions. The RARITYNet framework is designed by combining the shallow (RARITY) and deep (AffEmoNet) features to recognize the facial expressions from challenging images as spontaneous expressions, pose variations, ethnicity changes, and illumination conditions. The RARITY is proposed to encode the inter-radial transitional patterns in the local neighbourhood. The AffEmoNet: affective emotion learning network is proposed by incorporating three feature streams: high boost edge filtering (HBSEF) stream, to extract the edge information of highly affected facial expressive regions, multi-scale sophisticated edge cumulative (MSSEC) stream is to learns the sophisticated edge information from multi-receptive fields and RARITY uplift complementary context feature (RUCCF) stream refines the RARITY-encoded features and aid the MSSEC stream features to enrich the learning ability of RARITYNet.