OpenRaiser/Intern-Atlas huggingface dataset:
A Practical Incremental Method to Train Deep CTR Models
Venue: arXiv cs.IR 2020
Year: 2020

Deep learning models in recommender systems are usually trained in the batch mode, namely iteratively trained on a fixed-size window of training data. Such batch mode training of deep learning models suffers from low training efficiency, which may lead to performance degradation when the model is not produced on time. To tackle this issue, incremental learning is proposed and has received much attention recently. Incremental learning has great potential in recommender systems, as two consecutive window of training data overlap most of the volume. It aims to update the model incrementally with only the newly incoming samples from the timestamp when the model is updated last time, which is much more efficient than the batch mode training. However, most of the incremental learning methods focus on the research area of image recognition where new tasks or classes are learned over time. In this work, we introduce a practical incremental method to train deep CTR models, which consists of three decoupled modules (namely, data, feature and model module). Our method can achieve comparable performance to the conventional batch mode training with much better training efficiency. We conduct extensive experiments on a public benchmark and a private dataset to demonstrate the effectiveness of our proposed method.
SketchPatch: Sketch Stylization via Seamless Patch-level Synthesis
Venue: arXiv cs.GR 2020
Year: 2020

The paradigm of image-to-image translation is leveraged for the benefit of sketch stylization via transfer of geometric textural details. Lacking the necessary volumes of data for standard training of translation systems, we advocate for operation at the patch level, where a handful of stylized sketches provide ample mining potential for patches featuring basic geometric primitives. Operating at the patch level necessitates special consideration of full sketch translation, as individual translation of patches with no regard to neighbors is likely to produce visible seams and artifacts at patch borders. Aligned pairs of styled and plain primitives are combined to form input hybrids containing styled elements around the border and plain elements within, and given as input to a seamless translation (ST) generator, whose output patches are expected to reconstruct the fully styled patch. An adversarial addition promotes generalization and robustness to diverse geometries at inference time, forming a simple and effective system for arbitrary sketch stylization, as demonstrated upon a variety of styles and sketches.
Hybrid DCOP Solvers: Boosting Performance of Local Search Algorithms
Venue: arXiv cs.MA 2020
Year: 2020

We propose a novel method for expediting both symmetric and asymmetric Distributed Constraint Optimization Problem (DCOP) solvers. The core idea is based on initializing DCOP solvers with greedy fast non-iterative DCOP solvers. This is contrary to existing methods where initialization is always achieved using a random value assignment. We empirically show that changing the starting conditions of existing DCOP solvers not only reduces the algorithm convergence time by up to 50\%, but also reduces the communication overhead and leads to a better solution quality. We show that this effect is due to structural improvements in the variable assignment, which is caused by the spreading pattern of DCOP algorithm activation.) /Subject (Hybrid DCOPs)
Zero-Bias Deep Learning for Accurate Identification of Internet of Things (IoT) Devices
Venue: arXiv cs.LG 2020
Year: 2020

The Internet of Things (IoT) provides applications and services that would otherwise not be possible. However, the open nature of IoT make it vulnerable to cybersecurity threats. Especially, identity spoofing attacks, where an adversary passively listens to existing radio communications and then mimic the identity of legitimate devices to conduct malicious activities. Existing solutions employ cryptographic signatures to verify the trustworthiness of received information. In prevalent IoT, secret keys for cryptography can potentially be disclosed and disable the verification mechanism. Non-cryptographic device verification is needed to ensure trustworthy IoT. In this paper, we propose an enhanced deep learning framework for IoT device identification using physical layer signals. Specifically, we enable our framework to report unseen IoT devices and introduce the zero-bias layer to deep neural networks to increase robustness and interpretability. We have evaluated the effectiveness of the proposed framework using real data from ADS-B (Automatic Dependent Surveillance-Broadcast), an application of IoT in aviation. The proposed framework has the potential to be applied to accurate identification of IoT devices in a variety of IoT applications and services. Codes and data are available in IEEE Dataport.
Vulnerability of Face Recognition Systems Against Composite Face Reconstruction Attack
Venue: arXiv cs.CV 2020
Year: 2020

Rounding confidence score is considered trivial but a simple and effective countermeasure to stop gradient descent based image reconstruction attacks. However, its capability in the face of more sophisticated reconstruction attacks is an uninvestigated research area. In this paper, we prove that, the face reconstruction attacks based on composite faces can reveal the inefficiency of rounding policy as countermeasure. We assume that, the attacker takes advantage of face composite parts which helps the attacker to get access to the most important features of the face or decompose it to the independent segments. Afterwards, decomposed segments are exploited as search parameters to create a search path to reconstruct optimal face. Face composition parts enable the attacker to violate the privacy of face recognition models even with a blind search. However, we assume that, the attacker may take advantage of random search to reconstruct the target face faster. The algorithm is started with random composition of face parts as initial face and confidence score is considered as fitness value. Our experiments show that, since the rounding policy as countermeasure can't stop the random search process, current face recognition systems are extremely vulnerable against such sophisticated attacks. To address this problem, we successfully test Face Detection Score Filtering (FDSF) as a countermeasure to protect the privacy of training data against proposed attack.
Table Scraps: An Actionable Framework for Multi-Table Data Wrangling From An Artifact Study of Computational Journalism
Venue: arXiv cs.HC 2020
Year: 2020

For the many journalists who use data and computation to report the news, data wrangling is an integral part of their work.Despite an abundance of literature on data wrangling in the context of enterprise data analysis, little is known about the specific operations, processes, and pain points journalists encounter while performing this tedious, time-consuming task. To better understand the needs of this user group, we conduct a technical observation study of 50 public repositories of data and analysis code authored by 33 professional journalists at 26 news organizations. We develop two detailed and cross-cutting taxonomies of data wrangling in computational journalism, for actions and for processes. We observe the extensive use of multiple tables, a notable gap in previous wrangling analyses. We develop a concise, actionable framework for general multi-table data wrangling that includes wrangling operations documented in our taxonomy that are without clear parallels in other work. This framework, the first to incorporate tablesas first-class objects, will support future interactive wrangling tools for both computational journalism and general-purpose use. We assess the generative and descriptive power of our framework through discussion of its relationship to our set of taxonomies.
2.5D Root of Trust: Secure System-Level Integration of Untrusted Chiplets
Venue: arXiv cs.CR 2020
Year: 2020

Dedicated, after acceptance and publication, in memory of the late Vassos Soteriou. For the first time, we leverage the 2.5D interposer technology to establish system-level security in the face of hardware- and software-centric adversaries. More specifically, we integrate chiplets (i.e., third-party hard intellectual property of complex functionality, like microprocessors) using a security-enforcing interposer. Such hardware organization provides a robust 2.5D root of trust for trustworthy, yet powerful and flexible, computation systems. The security paradigms for our scheme, employed firmly by design and construction, are: 1) stringent physical separation of trusted from untrusted components, and 2) runtime monitoring. The system-level activities of all untrusted commodity chiplets are checked continuously against security policies via physically separated security features. Aside from the security promises, the good economics of outsourced supply chains are still maintained; the system vendor is free to procure chiplets from the open market, while only producing the interposer and assembling the 2.5D system oneself. We showcase our scheme using the Cortex-M0 core and the AHB-Lite bus by ARM, building a secure 64-core system with shared memories. We evaluate our scheme through hardware simulation, considering different threat scenarios. Finally, we devise a physical-design flow for 2.5D systems, based on commercial-grade design tools, to demonstrate and evaluate our 2.5D root of trust.
Are the Old Days Gone? A Survey on Actual Software Engineering Processes in Video Game Industry
Venue: arXiv cs.SE 2020
Year: 2020

In the past 10 years, several researches studied video game development process who proposed approaches to improve the way how games are developed. These approaches usually adopt agile methodologies because of claims that traditional practices and the waterfall process are gone. However, are the "old days" really gone in the game industry? In this paper, we present a survey of software engineering processes in video game industry from postmortem project analyses. We analyzed 20 postmortems from Gamasutra Portal. We extracted their processes and modelled them through using the Business Process Model and Notation (BPMN). This work presents three main contributions. First, a postmortem analysis methodology to identify and extract project processes. Second, the study main result: \textbf{the "old days" are gone, but not completely}. \textbf{Iterative practices} are increasing and are applied to at least \textbf{65\% of projects} in which \textbf{45\% of this projects} explicitly adopted Agile practices. However, \textbf{waterfall} process is still applied at least \textbf{30\% of projects}. Finally, we discuss some implications, directions and opportunities for video game development community.
A Visual Analytics Approach for Exploratory Causal Analysis: Exploration, Validation, and Applications
Venue: arXiv cs.HC 2020
Year: 2020

Using causal relations to guide decision making has become an essential analytical task across various domains, from marketing and medicine to education and social science. While powerful statistical models have been developed for inferring causal relations from data, domain practitioners still lack effective visual interface for interpreting the causal relations and applying them in their decision-making process. Through interview studies with domain experts, we characterize their current decision-making workflows, challenges, and needs. Through an iterative design process, we developed a visualization tool that allows analysts to explore, validate, and apply causal relations in real-world decision-making scenarios. The tool provides an uncertainty-aware causal graph visualization for presenting a large set of causal relations inferred from high-dimensional data. On top of the causal graph, it supports a set of intuitive user controls for performing what-if analyses and making action plans. We report on two case studies in marketing and student advising to demonstrate that users can effectively explore causal relations and design action plans for reaching their goals.
Complementary Dynamics
Venue: arXiv cs.GR 2020
Year: 2020

We present a novel approach to enrich arbitrary rig animations with elastodynamic secondary effects. Unlike previous methods which pit rig displacements and physical forces as adversaries against each other, we advocate that physics should complement artists intentions. We propose optimizing for elastodynamic displacements in the subspace orthogonal to displacements that can be created by the rig. This ensures that the additional dynamic motions do not undo the rig animation. The complementary space is high dimensional, algebraically constructed without manual oversight, and capable of rich high-frequency dynamics. Unlike prior tracking methods, we do not require extra painted weights, segmentation into fixed and free regions or tracking clusters. Our method is agnostic to the physical model and plugs into non-linear FEM simulations, geometric as-rigid-as-possible energies, or mass-spring models. Our method does not require a particular type of rig and adds secondary effects to skeletal animations, cage-based deformations, wire deformers, motion capture data, and rigid-body simulations.
Generalization on the Enhancement of Layerwise Relevance Interpretability of Deep Neural Network
Venue: arXiv cs.CV 2020
Year: 2020

The practical application of deep neural networks are still limited by their lack of transparency. One of the efforts to provide explanation for decisions made by artificial intelligence (AI) is the use of saliency or heat maps highlighting relevant regions that contribute significantly to its prediction. A layer-wise amplitude filtering method was previously introduced to improve the quality of heatmaps, performing error corrections by noise-spike suppression. In this study, we generalize the layerwise error correction by considering any identifiable error and assuming there exists a groundtruth interpretable information. The forms of errors propagated through layerwise relevance methods are studied and we propose a filtering technique for interpretability signal rectification taylored to the trend of signal amplitude of the particular neural network used. Finally, we put forth arguments for the use of groundtruth interpretable information.
Optimal Deterministic Group Testing Algorithms to Estimate the Number of Defectives
Venue: arXiv cs.IT 2020
Year: 2020

We study the problem of estimating the number of defective items $d$ within a pile of $n$ elements up to a multiplicative factor of $Δ>1$, using deterministic group testing algorithms. We bring lower and upper bounds on the number of tests required in both the adaptive and the non-adaptive deterministic settings given an upper bound $D$ on the defectives number. For the adaptive deterministic settings, our results show that, any algorithm for estimating the defectives number up to a multiplicative factor of $Δ$ must make at least $Ω\left((D/Δ^2)\log (n/D) \right )$ tests. This extends the same lower bound achieved in \cite{ALA17} for non-adaptive algorithms. Moreover, we give a polynomial time adaptive algorithm that shows that our bound is tight up to a small additive term. For non-adaptive algorithms, an upper bound of $O((D/Δ^2)$ $(\log (n/D)+\log Δ) )$ is achieved by means of non-constructive proof. This improves the lower bound $O((\log D)/(\logΔ))D\log n)$ from \cite{ALA17} and matches the lower bound up to a small additive term. In addition, we study polynomial time constructive algorithms. We use existing polynomial time constructible \emph{expander regular bipartite graphs}, \emph{extractors} and \emph{condensers} to construct two polynomial time algorithms. The first algorithm makes $O((D^{1+o(1)}/Δ^2)\cdot \log n)$ tests, and the second makes $(D/Δ^2)\cdot quazipoly$ $(\log n)$ tests. This is the first explicit construction with an almost optimal test complexity.
Extended quasi-cyclic constructions of quantum codes and entanglement-assisted quantum codes
Venue: arXiv cs.IT 2020
Year: 2020

Construction of quantum codes and entanglement-assisted quantum codes with good parameters via classical codes is an important task for quantum computing and quantum information. In this paper, by a family of one-generator quasi-cyclic codes, we provide quasi-cyclic extended constructions that preserve the self-orthogonality to obtain stabilizer quantum codes. As for the computational results, some binary and ternary stabilizer codes with good parameters are constructed. Moreover, we present methods to construct maximal-entanglement entanglement-assisted quantum codes by means of the class of quasi-cyclic codes and their extended codes. As an application, some good maximal-entanglement entanglement-assisted quantum codes are obtained and their parameters are compared.
Visually Analyzing Contextualized Embeddings
Venue: arXiv cs.HC 2020
Year: 2020

In this paper we introduce a method for visually analyzing contextualized embeddings produced by deep neural network-based language models. Our approach is inspired by linguistic probes for natural language processing, where tasks are designed to probe language models for linguistic structure, such as parts-of-speech and named entities. These approaches are largely confirmatory, however, only enabling a user to test for information known a priori. In this work, we eschew supervised probing tasks, and advocate for unsupervised probes, coupled with visual exploration techniques, to assess what is learned by language models. Specifically, we cluster contextualized embeddings produced from a large text corpus, and introduce a visualization design based on this clustering and textual structure - cluster co-occurrences, cluster spans, and cluster-word membership - to help elicit the functionality of, and relationship between, individual clusters. User feedback highlights the benefits of our design in discovering different types of linguistic structures.
Particle Swarm Optimized Federated Learning For Industrial IoT and Smart City Services
Venue: arXiv cs.LG 2020
Year: 2020

Most of the research on Federated Learning (FL) has focused on analyzing global optimization, privacy, and communication, with limited attention focusing on analyzing the critical matter of performing efficient local training and inference at the edge devices. One of the main challenges for successful and efficient training and inference on edge devices is the careful selection of parameters to build local Machine Learning (ML) models. To this aim, we propose a Particle Swarm Optimization (PSO)-based technique to optimize the hyperparameter settings for the local ML models in an FL environment. We evaluate the performance of our proposed technique using two case studies. First, we consider smart city services and use an experimental transportation dataset for traffic prediction as a proxy for this setting. Second, we consider Industrial IoT (IIoT) services and use the real-time telemetry dataset to predict the probability that a machine will fail shortly due to component failures. Our experiments indicate that PSO provides an efficient approach for tuning the hyperparameters of deep Long short-term memory (LSTM) models when compared to the grid search method. Our experiments illustrate that the number of clients-server communication rounds to explore the landscape of configurations to find the near-optimal parameters are greatly reduced (roughly by two orders of magnitude needing only 2%--4% of the rounds compared to state of the art non-PSO-based approaches). We also demonstrate that utilizing the proposed PSO-based technique to find the near-optimal configurations for FL and centralized learning models does not adversely affect the accuracy of the models.
FlipOut: Uncovering Redundant Weights via Sign Flipping
Venue: arXiv cs.LG 2020
Year: 2020

Modern neural networks, although achieving state-of-the-art results on many tasks, tend to have a large number of parameters, which increases training time and resource usage. This problem can be alleviated by pruning. Existing methods, however, often require extensive parameter tuning or multiple cycles of pruning and retraining to convergence in order to obtain a favorable accuracy-sparsity trade-off. To address these issues, we propose a novel pruning method which uses the oscillations around $0$ (i.e. sign flips) that a weight has undergone during training in order to determine its saliency. Our method can perform pruning before the network has converged, requires little tuning effort due to having good default values for its hyperparameters, and can directly target the level of sparsity desired by the user. Our experiments, performed on a variety of object classification architectures, show that it is competitive with existing methods and achieves state-of-the-art performance for levels of sparsity of $99.6\%$ and above for most of the architectures tested. For reproducibility, we release our code publicly at https://github.com/AndreiXYZ/flipout.
Bluff: Interactively Deciphering Adversarial Attacks on Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Deep neural networks (DNNs) are now commonly used in many domains. However, they are vulnerable to adversarial attacks: carefully crafted perturbations on data inputs that can fool a model into making incorrect predictions. Despite significant research on developing DNN attack and defense techniques, people still lack an understanding of how such attacks penetrate a model's internals. We present Bluff, an interactive system for visualizing, characterizing, and deciphering adversarial attacks on vision-based neural networks. Bluff allows people to flexibly visualize and compare the activation pathways for benign and attacked images, revealing mechanisms that adversarial attacks employ to inflict harm on a model. Bluff is open-sourced and runs in modern web browsers.
Efficiency Improvements for Encrypt-to-Self
Venue: arXiv cs.CR 2020
Year: 2020

Recent work by Pijnenburg and Poettering (ESORICS'20) explores the novel cryptographic Encrypt-to-Self primitive that is dedicated to use cases of symmetric encryption where encryptor and decryptor coincide. The primitive is envisioned to be useful whenever a memory-bounded computing device is required to encrypt some data with the aim of temporarily depositing it on an untrusted storage device. While the new primitive protects the confidentiality of payloads as much as classic authenticated encryption primitives would do, it provides considerably better authenticity guarantees: Specifically, while classic solutions would completely fail in a context involving user corruptions, if an encrypt-to-self scheme is used to protect the data, all ciphertexts and messages fully remain unforgeable. To instantiate their encrypt-to-self primitive, Pijnenburg et al propose a mode of operation of the compression function of a hash function, with a carefully designed encoding function playing the central role in the serialization of the processed message and associated data. In the present work we revisit the design of this encoding function. Without questioning its adequacy for securely accomplishing the encrypt-to-self job, we improve on it from a technical/implementational perspective by proposing modifications that alleviate certain conditions that would inevitably require implementations to disrespect memory alignment restrictions imposed by the word-wise operation of modern CPUs, ultimately leading to performance penalties. Our main contributions are thus to propose an improved encoding function, to explain why it offers better performance, and to prove that it provides as much security as its predecessor. We finally report on our open-source implementation of the encrypt-to-self primitive based on the new encoding function.