OpenRaiser/Intern-Atlas huggingface dataset:
SketchMeHow: Interactive Projection Guided Task Instruction with User Sketches
Venue: arXiv cs.HC 2021
Year: 2021

In this work, we propose an interactive general instruction framework SketchMeHow to guidance the common users to complete the daily tasks in real-time. In contrast to the conventional augmented reality-based instruction systems, the proposed framework utilizes the user sketches as system inputs to acquire the users' production intentions from the drawing interfaces. Given the user sketches, the designated task instruction can be analyzed based on the sub-task division and spatial localization for each task. The projector-camera system is adopted in the projection guidance to the end-users with the spatial augmented reality technology. To verify the proposed framework, we conducted two case studies of domino arrangement and bento production. From our user studies, the proposed systems can help novice users complete the tasks efficiently with user satisfaction. We believe the proposed SketchMeHow can broaden the research topics in sketch-based real-world applications in human-computer interaction.
High-Resolution Waveform Capture Device on a Cyclone-V FPGA
Venue: arXiv cs.AR 2021
Year: 2021

We introduce the waveform capture device (WCD), a flexible measurement system capable of recording complex digital signals on trillionth-of-a-second (ps) time scales. The WCD is implemented via modular code on an off-the-shelf field-programmable gate-array (FPGA, Intel/Altera Cyclone V), and incorporates both time-to-digital converter (TDC) and digital storage oscilloscope (DSO) functionality. The device captures a waveform by taking snapshots of a signal as it propagates down an ultra-fast transmission line known as a carry chain (CC). It is calibrated via a novel dynamic phase-shifting (DPS) method that requires substantially less data and resources than the state-of-the-art. Using DPS, we find the measurement resolution - or mean propagation delay from one CC element to the next - to be 4.91 +/- 0.04 ps (4.54 +/- 0.02 ps) for a pulse of logic high (low). Similarly, we find the single-shot precision - or mean error on the timing of the waveform - to be 29.52 ps (27.14 ps) for pulses of logic high (low). We verify these findings by reproducing commercial oscilloscope measurements of asynchronous ring-oscillators on FPGAs, finding the mean pulse width to be 0.240 +/- 0.002 ns per inverter gate. Finally, we present a careful analysis of design constraints, introduce a novel error correction algorithm, and sketch a simple extension to the analog domain. We also provide the Verilog code instantiating the our design on an FPGA in an Appendix, and make our methods available as an open-source Python library at https://github.com/Noeloikeau/fpyga.
Temporal complex networks modeling applied to vehicular ad-hoc networks
Venue: arXiv cs.SI 2021
Year: 2021

VANETs solutions use aggregated graph representation to model the interaction among the vehicles and different aggregated complex network measures to quantify some topological characteristics. This modeling ignores the temporal interactions between the cars, causing loss of information or unrealistic behavior. This work proposes the use of both temporal graphs and temporal measures to model VANETs applications. To verify the viability of this model, we initially perform a comparative analysis between the temporal and aggregated modeling considering five different real datasets. This analysis shows that the aggregated model is inefficient in modeling the temporal aspects of networks. After that, we perform a network evaluation through a simulation by considering the impact of temporal modeling applied to the deployment of RSUs. First, we compare a solution based on our temporal modeling with a greedy algorithm based on an aggregated model to choose the positions of RSUs. In a scenario with 70 RSUs, we have 77% and 65% of coverage in the temporal and aggregated model (greedy algorithm), respectively. Second, we evaluate the use of aggregated and temporal measures applied as features in a genetic algorithm. The approach with temporal betweenness had the better result with 90% of the coverage area against 61% of aggregated one applied to the same scenario.
FHAC at GermEval 2021: Identifying German toxic, engaging, and fact-claiming comments with ensemble learning
Venue: arXiv cs.CL 2021
Year: 2021

The availability of language representations learned by large pretrained neural network models (such as BERT and ELECTRA) has led to improvements in many downstream Natural Language Processing tasks in recent years. Pretrained models usually differ in pretraining objectives, architectures, and datasets they are trained on which can affect downstream performance. In this contribution, we fine-tuned German BERT and German ELECTRA models to identify toxic (subtask 1), engaging (subtask 2), and fact-claiming comments (subtask 3) in Facebook data provided by the GermEval 2021 competition. We created ensembles of these models and investigated whether and how classification performance depends on the number of ensemble members and their composition. On out-of-sample data, our best ensemble achieved a macro-F1 score of 0.73 (for all subtasks), and F1 scores of 0.72, 0.70, and 0.76 for subtasks 1, 2, and 3, respectively.
An Executable Structural Operational Formal Semantics for Python
Venue: arXiv cs.PL 2021
Year: 2021

Python is a popular high-level general-purpose programming language also heavily used by the scientific community. It supports a variety of different programming paradigms and is preferred by many for its ease of use. With the vision of harvesting static analysis techniques like abstract interpretation for Python, we develop a formal semantics for Python. A formal semantics is an important cornerstone for any sound static analysis technique. We base our efforts on the general framework of structural operational semantics yielding a small-step semantics in principle allowing for concurrency and interaction with an environment. The main contributions of this thesis are twofold: first, we develop a meta-theoretic framework for the formalization of structural operational semantics in tandem with the necessary tool support for the automated derivation of interpreters from such formal semantics, and, second, we validate the suitability of this approach for the formalization of modern programming languages developing a semantics for Python.
Recommendation Fairness: From Static to Dynamic
Venue: arXiv cs.IR 2021
Year: 2021

Driven by the need to capture users' evolving interests and optimize their long-term experiences, more and more recommender systems have started to model recommendation as a Markov decision process and employ reinforcement learning to address the problem. Shouldn't research on the fairness of recommender systems follow the same trend from static evaluation and one-shot intervention to dynamic monitoring and non-stop control? In this paper, we portray the recent developments in recommender systems first and then discuss how fairness could be baked into the reinforcement learning techniques for recommendation. Moreover, we argue that in order to make further progress in recommendation fairness, we may want to consider multi-agent (game-theoretic) optimization, multi-objective (Pareto) optimization, and simulation-based optimization, in the general framework of stochastic games.
Fair Comparison: Quantifying Variance in Resultsfor Fine-grained Visual Categorization
Venue: arXiv cs.CV 2021
Year: 2021

For the task of image classification, researchers work arduously to develop the next state-of-the-art (SOTA) model, each bench-marking their own performance against that of their predecessors and of their peers. Unfortunately, the metric used most frequently to describe a model's performance, average categorization accuracy, is often used in isolation. As the number of classes increases, such as in fine-grained visual categorization (FGVC), the amount of information conveyed by average accuracy alone dwindles. While its most glaring weakness is its failure to describe the model's performance on a class-by-class basis, average accuracy also fails to describe how performance may vary from one trained model of the same architecture, on the same dataset, to another (both averaged across all categories and at the per-class level). We first demonstrate the magnitude of these variations across models and across class distributions based on attributes of the data, comparing results on different visual domains and different per-class image distributions, including long-tailed distributions and few-shot subsets. We then analyze the impact various FGVC methods have on overall and per-class variance. From this analysis, we both highlight the importance of reporting and comparing methods based on information beyond overall accuracy, as well as point out techniques that mitigate variance in FGVC results.
On the Convergence of Decentralized Adaptive Gradient Methods
Venue: arXiv cs.LG 2021
Year: 2021

Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks. Meanwhile, given the need for distributed computing, distributed optimization algorithms are rapidly becoming a focal point. With the growth of computing power and the need for using machine learning models on mobile devices, the communication cost of distributed training algorithms needs careful consideration. In this paper, we introduce novel convergent decentralized adaptive gradient methods and rigorously incorporate adaptive gradient methods into decentralized training procedures. Specifically, we propose a general algorithmic framework that can convert existing adaptive gradient methods to their decentralized counterparts. In addition, we thoroughly analyze the convergence behavior of the proposed algorithmic framework and show that if a given adaptive gradient method converges, under some specific conditions, then its decentralized counterpart is also convergent. We illustrate the benefit of our generic decentralized framework on a prototype method, i.e., AMSGrad, both theoretically and numerically.
Fast random number generator based on optical physical unclonable functions
Venue: arXiv cs.CR 2021
Year: 2021

We propose an approach for fast random number generation based on homemade optical physical unclonable functions (PUFs). The optical PUF is illuminated with input laser wavefront of continuous modulation to obtain different speckle patterns. Random numbers are fully extracted from speckle patterns through a simple post-processing algorithm. Our proof-of-principle experiment achieves total random number generation rate of 0.96 Gbit/s with verified randomness, which is far faster than previous optical-PUF-based schemes. Our results demonstrate that the presented random number generator (RNG) proposal has great potential to achieve ultrafast random number generation rate up to several hundreds of Gbit/s.
CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems
Venue: arXiv cs.CR 2021
Year: 2021

Given the success of reinforcement learning (RL) in various domains, it is promising to explore the application of its methods to the development of intelligent and autonomous cyber agents. Enabling this development requires a representative RL training environment. To that end, this work presents CyGIL: an experimental testbed of an emulated RL training environment for network cyber operations. CyGIL uses a stateless environment architecture and incorporates the MITRE ATT&CK framework to establish a high fidelity training environment, while presenting a sufficiently abstracted interface to enable RL training. Its comprehensive action space and flexible game design allow the agent training to focus on particular advanced persistent threat (APT) profiles, and to incorporate a broad range of potential threats and vulnerabilities. By striking a balance between fidelity and simplicity, it aims to leverage state of the art RL algorithms for application to real-world cyber defence.
On the space of coefficients of a Feed Forward Neural Network
Venue: arXiv cs.LG 2021
Year: 2021

We define and establish the conditions for `equivalent neural networks' - neural networks with different weights, biases, and threshold functions that result in the same associated function. We prove that given a neural network $\mathcal{N}$ with piece-wise linear activation, the space of coefficients describing all equivalent neural networks is given by a semialgebraic set. This result is obtained by studying different representations of a given piece-wise linear function using the Tarski-Seidenberg theorem.
Mixup Decoding for Diverse Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

Diverse machine translation aims at generating various target language translations for a given source language sentence. Leveraging the linear relationship in the sentence latent space introduced by the mixup training, we propose a novel method, MixDiversity, to generate different translations for the input sentence by linearly interpolating it with different sentence pairs sampled from the training corpus when decoding. To further improve the faithfulness and diversity of the translations, we propose two simple but effective approaches to select diverse sentence pairs in the training corpus and adjust the interpolation weight for each pair correspondingly. Moreover, by controlling the interpolation weight, our method can achieve the trade-off between faithfulness and diversity without any additional training, which is required in most of the previous methods. Experiments on WMT'16 en-ro, WMT'14 en-de, and WMT'17 zh-en are conducted to show that our method substantially outperforms all previous diverse machine translation methods.
ADER:Adapting between Exploration and Robustness for Actor-Critic Methods
Venue: arXiv cs.LG 2021
Year: 2021

Combining off-policy reinforcement learning methods with function approximators such as neural networks has been found to lead to overestimation of the value function and sub-optimal solutions. Improvement such as TD3 has been proposed to address this issue. However, we surprisingly find that its performance lags behind the vanilla actor-critic methods (such as DDPG) in some primitive environments. In this paper, we show that the failure of some cases can be attributed to insufficient exploration. We reveal the culprit of insufficient exploration in TD3, and propose a novel algorithm toward this problem that ADapts between Exploration and Robustness, namely ADER. To enhance the exploration ability while eliminating the overestimation bias, we introduce a dynamic penalty term in value estimation calculated from estimated uncertainty, which takes into account different compositions of the uncertainty in different learning stages. Experiments in several challenging environments demonstrate the supremacy of the proposed method in continuous control tasks.
Pose-guided Inter- and Intra-part Relational Transformer for Occluded Person Re-Identification
Venue: arXiv cs.CV 2021
Year: 2021

Person Re-Identification (Re-Id) in occlusion scenarios is a challenging problem because a pedestrian can be partially occluded. The use of local information for feature extraction and matching is still necessary. Therefore, we propose a Pose-guided inter-and intra-part relational transformer (Pirt) for occluded person Re-Id, which builds part-aware long-term correlations by introducing transformers. In our framework, we firstly develop a pose-guided feature extraction module with regional grouping and mask construction for robust feature representations. The positions of a pedestrian in the image under surveillance scenarios are relatively fixed, hence we propose an intra-part and inter-part relational transformer. The intra-part module creates local relations with mask-guided features, while the inter-part relationship builds correlations with transformers, to develop cross relationships between part nodes. With the collaborative learning inter- and intra-part relationships, experiments reveal that our proposed Pirt model achieves a new state of the art on the public occluded dataset, and further extensions on standard non-occluded person Re-Id datasets also reveal our comparable performances.
Social Analysis of Young Basque Speaking Communities in Twitter
Venue: arXiv cs.CY 2021
Year: 2021

In this paper we take into account both social and linguistic aspects to perform demographic analysis by processing a large amount of tweets in Basque language. The study of demographic characteristics and social relationships are approached by applying machine learning and modern deep-learning Natural Language Processing (NLP) techniques, combining social sciences with automatic text processing. More specifically, our main objective is to combine demographic inference and social analysis in order to detect young Basque Twitter users and to identify the communities that arise from their relationships or shared content. This social and demographic analysis will be entirely based on the~automatically collected tweets using NLP to convert unstructured textual information into interpretable knowledge.
RefineCap: Concept-Aware Refinement for Image Captioning
Venue: arXiv cs.CL 2021
Year: 2021

Automatically translating images to texts involves image scene understanding and language modeling. In this paper, we propose a novel model, termed RefineCap, that refines the output vocabulary of the language decoder using decoder-guided visual semantics, and implicitly learns the mapping between visual tag words and images. The proposed Visual-Concept Refinement method can allow the generator to attend to semantic details in the image, thereby generating more semantically descriptive captions. Our model achieves superior performance on the MS-COCO dataset in comparison with previous visual-concept based models.
Data Aggregation in Synchronous Large-scale IoT Networks: Granularity, Reliability, and Delay Tradeoffs
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies data aggregation in large-scale regularly deployed Internet of Things (IoT) networks, where devices generate synchronized time-triggered traffic (e.g., measurements or updates). The data granularity, in terms of information content and temporal resolution, is parameterized by the sizes of the generated packets and the duty cycle of packet generation. The generated data packets at the devices are aggregated through static terrestrial gateways. Universal frequency reuse is adopted across all gateways and randomized scheduling is utilized for the IoT devices associated with each gateway. Such network model finds applications in environmental sensing, precision agriculture, and geological seismic sensing to name a few. To this end, we develop a novel spatiotemporal mathematical model to characterize the interplay between data granularity, transmission reliability, and delay. The developed model accounts for several IoT design parameters, which include packet sizes, generation duty cycle, devices and gateways spatial densities, transmission rate adaptation, power control, and antenna directivity. For tractable analysis, we propose two accurate approximations, based on the Poisson point process, to characterize the signal-to-interference-plus-noise-ratio (SINR) based transmission reliability. For the delay analysis, we propose a phase-type arrival/departure (PH/PH/1) queueing model that accounts for packet generation, transmission scheduling, and rate-sensitive SINR-based packet departure. The developed model is utilized to obtain the optimal transmission rate for the IoT devices that minimizes delay. The numerical results delineate the joint feasibility range of packet sizes and inter-arrival times for data aggregation and reveal significant gains when deploying directional antennas.
Formal Query Building with Query Structure Prediction for Complex Question Answering over Knowledge Base
Venue: arXiv cs.CL 2021
Year: 2021

Formal query building is an important part of complex question answering over knowledge bases. It aims to build correct executable queries for questions. Recent methods try to rank candidate queries generated by a state-transition strategy. However, this candidate generation strategy ignores the structure of queries, resulting in a considerable number of noisy queries. In this paper, we propose a new formal query building approach that consists of two stages. In the first stage, we predict the query structure of the question and leverage the structure to constrain the generation of the candidate queries. We propose a novel graph generation framework to handle the structure prediction task and design an encoder-decoder model to predict the argument of the predetermined operation in each generative step. In the second stage, we follow the previous methods to rank the candidate queries. The experimental results show that our formal query building approach outperforms existing methods on complex questions while staying competitive on simple questions.
On Blockchain Architectures for Trust-Based Collaborative Intrusion Detection
Venue: arXiv cs.CR 2021
Year: 2021

This paper considers the use of novel technologies for mitigating attacks that aim at compromising intrusion detection systems (IDSs). Solutions based on collaborative intrusion detection networks (CIDNs) could increase the resilience against such attacks as they allow IDS nodes to gain knowledge from each other by sharing information. However, despite the vast research in this area, trust management issues still pose significant challenges and recent works investigate whether these could be addressed by relying on blockchain and related distributed ledger technologies. Towards that direction, the paper proposes the use of a trust-based blockchain in CIDNs, referred to as trust-chain, to protect the integrity of the information shared among the CIDN peers, enhance their accountability, and secure their collaboration by thwarting insider attacks. A consensus protocol is proposed for CIDNs, which is a combination of a proof-of-stake and proof-of-work protocols, to enable collaborative IDS nodes to maintain a reliable and tampered-resistant trust-chain.
Cost Problems for Parametric Time Petri Nets
Venue: arXiv cs.FL 2021
Year: 2021

We investigate the problem of parameter synthesis for time Petri nets with a cost variable that evolves both continuously with time, and discretely when firing transitions. More precisely, parameters are rational symbolic constants used for time constraints on the firing of transitions and we want to synthesise all their values such that some marking is reachable, with a cost that is either minimal or simply less than a given bound. We first prove that the mere existence of values for the parameters such that the latter property holds is undecidable. We nonetheless provide symbolic semi-algorithms for the two synthesis problems and we prove them both sound and complete when they terminate. We also show how to modify them for the case when parameter values are integers. Finally, we prove that these modified versions terminate if parameters are bounded. While this is to be expected since there are now only a finite number of possible parameter values, our algorithms are symbolic and thus avoid an explicit enumeration of all those values. Furthermore, the results are symbolic constraints representing finite unions of convex polyhedra that are easily amenable to further analysis through linear programming. We finally report on the implementation of the approach in Romeo, a software tool for the analysis of time Petri nets.