OpenRaiser/Intern-Atlas huggingface dataset:
Security implications of digitalization: The dangers of data colonialism and the way towards sustainable and sovereign management of environmental data
Venue: arXiv cs.CY 2021
Year: 2021

Digitalization opens up new opportunities in the collection, analysis, and presentation of data which can contribute to the achievement of the 2030 Agenda and its Sustainable Development Goals (SDGs). In particular, the access to and control of environmental and geospatial data is fundamental to identify and understand global issues and trends. Also immediate crises such as the COVID-19 pandemic demonstrate the importance of accurate health data such as infection statistics and the relevance of digital tools like video conferencing platforms. However, today much of the data is collected and processed by private actors. Thus, governments and researchers depend on data platforms and proprietary systems of big tech companies such as Google or Microsoft. The market capitalization of the seven largest US and Chinese big tech companies has grown to 8.7tn USD in recent years, about twice the size of Germany's gross domestic product (GDP). Therefore, their market power is enormous, allowing them to dictate many rules of the digital space and even interfere with legislations. Based on a literature review and nine expert interviews this study presents a framework that identifies the risks and consequences along the workflow of collecting, processing, storing, using of data. It also includes solutions that governmental and multilateral actors can strive for to alleviate the risks. Fundamental to this framework is the novel concept of "data colonialism" which describes today's trend of private companies appropriating the digital sphere. Historically, colonial nations used to grab indigenous land and exploit the cheap labor of slave workers. In a similar way, today's big tech corporations use cheap data of their users to produce valuable services and thus create enormous market power.
Real vs Simulated Foveated Rendering to Reduce Visual Discomfort in Virtual Reality
Venue: arXiv cs.HC 2021
Year: 2021

In this paper, a study aimed at investigating the effects of real (using eye tracking to determine the fixation) and simulated foveated blurring in immersive Virtual Reality is presented. Techniques to reduce the optical flow perceived at the visual field margins are often employed in immersive Virtual Reality environments to alleviate discomfort experienced when the visual motion perception does not correspond to the body's acceleration. Although still preliminary, our results suggest that for participants with higher self-declared sensitivity to sickness, there might be an improvement for nausea when using blurring. The (perceived) difficulty of the task seems to improve when the real foveated method is used.
Hebbian learning with gradients: Hebbian convolutional neural networks with modern deep learning frameworks
Venue: arXiv cs.NE 2021
Year: 2021

Deep learning networks generally use non-biological learning methods. By contrast, networks based on more biologically plausible learning, such as Hebbian learning, show comparatively poor performance and difficulties of implementation. Here we show that Hebbian learning in hierarchical, convolutional neural networks can be implemented almost trivially with modern deep learning frameworks, by using specific losses whose gradients produce exactly the desired Hebbian updates. We provide expressions whose gradients exactly implement a plain Hebbian rule (dw ~= xy), Grossberg's instar rule (dw ~= y(x-w)), and Oja's rule (dw ~= y(x-yw)). As an application, we build Hebbian convolutional multi-layer networks for object recognition. We observe that higher layers of such networks tend to learn large, simple features (Gabor-like filters and blobs), explaining the previously reported decrease in decoding performance over successive layers. To combat this tendency, we introduce interventions (denser activations with sparse plasticity, pruning of connections between layers) which result in sparser learned features, massively increase performance, and allow information to increase over successive layers. We hypothesize that more advanced techniques (dynamic stimuli, trace learning, feedback connections, etc.), together with the massive computational boost offered by modern deep learning frameworks, could greatly improve the performance and biological relevance of multi-layer Hebbian networks.
KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Kronecker-factored Approximate Curvature (K-FAC) has recently been shown to converge faster in deep neural network (DNN) training than stochastic gradient descent (SGD); however, K-FAC's larger memory footprint hinders its applicability to large models. We present KAISA, a K-FAC-enabled, Adaptable, Improved, and ScAlable second-order optimizer framework that adapts the memory footprint, communication, and computation given specific models and hardware to improve performance and increase scalability. We quantify the tradeoffs between memory and communication cost and evaluate KAISA on large models, including ResNet-50, Mask R-CNN, U-Net, and BERT, on up to 128 NVIDIA A100 GPUs. Compared to the original optimizers, KAISA converges 18.1-36.3% faster across applications with the same global batch size. Under a fixed memory budget, KAISA converges 32.5% and 41.6% faster in ResNet-50 and BERT-Large, respectively. KAISA can balance memory and communication to achieve scaling efficiency equal to or better than the baseline optimizers. KAISA is open source and available at https://github.com/gpauloski/kfac_pytorch.
Popcorn: Paillier Meets Compression For Efficient Oblivious Neural Network Inference
Venue: arXiv cs.CR 2021
Year: 2021

Oblivious inference enables the cloud to provide neural network inference-as-a-service (NN-IaaS), whilst neither disclosing the client data nor revealing the server's model. However, the privacy guarantee under oblivious inference usually comes with a heavy cost of efficiency and accuracy. We propose Popcorn, a concise oblivious inference framework entirely built on the Paillier homomorphic encryption scheme. We design a suite of novel protocols to compute non-linear activation and max-pooling layers. We leverage neural network compression techniques (i.e., neural weights pruning and quantization) to accelerate the inference computation. To implement the Popcorn framework, we only need to replace algebraic operations of existing networks with their corresponding Paillier homomorphic operations, which is extremely friendly for engineering development. We first conduct the performance evaluation and comparison based on the MNIST and CIFAR-10 classification tasks. Compared with existing solutions, Popcorn brings a significant communication overhead deduction, with a moderate runtime increase. Then, we benchmark the performance of oblivious inference on ImageNet. To our best knowledge, this is the first report based on a commercial-level dataset, taking a step towards the deployment to production.
Automating Generative Deep Learning for Artistic Purposes: Challenges and Opportunities
Venue: arXiv cs.LG 2021
Year: 2021

We present a framework for automating generative deep learning with a specific focus on artistic applications. The framework provides opportunities to hand over creative responsibilities to a generative system as targets for automation. For the definition of targets, we adopt core concepts from automated machine learning and an analysis of generative deep learning pipelines, both in standard and artistic settings. To motivate the framework, we argue that automation aligns well with the goal of increasing the creative responsibility of a generative system, a central theme in computational creativity research. We understand automation as the challenge of granting a generative system more creative autonomy, by framing the interaction between the user and the system as a co-creative process. The development of the framework is informed by our analysis of the relationship between automation and creative autonomy. An illustrative example shows how the framework can give inspiration and guidance in the process of handing over creative responsibility.
Optimizing the Numbers of Queries and Replies in Federated Learning with Differential Privacy
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning (FL) empowers distributed clients to collaboratively train a shared machine learning model through exchanging parameter information. Despite the fact that FL can protect clients' raw data, malicious users can still crack original data with disclosed parameters. To amend this flaw, differential privacy (DP) is incorporated into FL clients to disturb original parameters, which however can significantly impair the accuracy of the trained model. In this work, we study a crucial question which has been vastly overlooked by existing works: what are the optimal numbers of queries and replies in FL with DP so that the final model accuracy is maximized. In FL, the parameter server (PS) needs to query participating clients for multiple global iterations to complete training. Each client responds a query from the PS by conducting a local iteration. Our work investigates how many times the PS should query clients and how many times each client should reply the PS. We investigate two most extensively used DP mechanisms (i.e., the Laplace mechanism and Gaussian mechanisms). Through conducting convergence rate analysis, we can determine the optimal numbers of queries and replies in FL with DP so that the final model accuracy can be maximized. Finally, extensive experiments are conducted with publicly available datasets: MNIST and FEMNIST, to verify our analysis and the results demonstrate that properly setting the numbers of queries and replies can significantly improve the final model accuracy in FL with DP.
Ray-ONet: Efficient 3D Reconstruction From A Single RGB Image
Venue: arXiv cs.CV 2021
Year: 2021

We propose Ray-ONet to reconstruct detailed 3D models from monocular images efficiently. By predicting a series of occupancy probabilities along a ray that is back-projected from a pixel in the camera coordinate, our method Ray-ONet improves the reconstruction accuracy in comparison with Occupancy Networks (ONet), while reducing the network inference complexity to O($N^2$). As a result, Ray-ONet achieves state-of-the-art performance on the ShapeNet benchmark with more than 20$\times$ speed-up at $128^3$ resolution and maintains a similar memory footprint during inference.
Ensemble and Auxiliary Tasks for Data-Efficient Deep Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Ensemble and auxiliary tasks are both well known to improve the performance of machine learning models when data is limited. However, the interaction between these two methods is not well studied, particularly in the context of deep reinforcement learning. In this paper, we study the effects of ensemble and auxiliary tasks when combined with the deep Q-learning algorithm. We perform a case study on ATARI games under limited data constraint. Moreover, we derive a refined bias-variance-covariance decomposition to analyze the different ways of learning ensembles and using auxiliary tasks, and use the analysis to help provide some understanding of the case study. Our code is open source and available at https://github.com/NUS-LID/RENAULT.
Analyzing a Knowledge Graph of Industry4.0 Standards
Venue: arXiv cs.DL 2021
Year: 2021

In this article, we tackle the problem of standard interoperability across different standardization frameworks, and devise a knowledge-driven approach that allows for the description of standards and standardization frameworks into an Industry 4.0 knowledge graph (I40KG). The STO ontology represents properties of standards and standardization frameworks, as well as relationships among them. The I40KG integrates more than 200 standards and four standardization frameworks. To populate the I40KG, the landscape of standards has been analyzed from a semantic perspective and the resulting I40KG represents knowledge expressed in more than 200 industrial related documents including technical reports, research articles, and white papers. Additionally, the I40KG has been linked to existing knowledge graphs and an automated reasoning has been implemented to reveal implicit relations between standards as well as mappings across standardization frameworks. We analyze both the number of discovered relations between standards and the accuracy of these relations. Observed results indicate that both reasoning and linking processes enable for increasing the connectivity in the knowledge graph by up to 80%, whilst up to 96% of the relations can be validated. These outcomes suggest that integrating standards and standardization frameworks into the I40KG enables the resolution of semantic interoperability conflicts, empowering the communication in smart factories.
Berserker: ASN.1-based Fuzzing of Radio Resource Control Protocol for 4G and 5G
Venue: arXiv cs.CR 2021
Year: 2021

Telecom networks together with mobile phones must be rigorously tested for robustness against vulnerabilities in order to guarantee availability. RRC protocol is responsible for the management of radio resources and is among the most important telecom protocols whose extensive testing is warranted. To that end, we present a novel RRC fuzzer, called Berserker, for 4G and 5G. Berserker's novelty comes from being backward and forward compatible to any version of 4G and 5G RRC technical specifications. It is based on RRC message format definitions in ASN.1 and additionally covers fuzz testing of another protocol, called NAS, tunneled in RRC. Berserker uses concrete implementations of telecom protocol stack and is unaffected by lower layer protocol handlings like encryption and segmentation. It is also capable of evading size and type constraints in RRC message format definitions. Berserker discovered two previously unknown serious vulnerabilities in srsLTE -- one of which also affects openLTE -- confirming its applicability to telecom robustness.
Machine Learning for Fraud Detection in E-Commerce: A Research Agenda
Venue: arXiv cs.LG 2021
Year: 2021

Fraud detection and prevention play an important part in ensuring the sustained operation of any e-commerce business. Machine learning (ML) often plays an important role in these anti-fraud operations, but the organizational context in which these ML models operate cannot be ignored. In this paper, we take an organization-centric view on the topic of fraud detection by formulating an operational model of the anti-fraud departments in e-commerce organizations. We derive 6 research topics and 12 practical challenges for fraud detection from this operational model. We summarize the state of the literature for each research topic, discuss potential solutions to the practical challenges, and identify 22 open research challenges.
Gaze Estimation with an Ensemble of Four Architectures
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents a method for gaze estimation according to face images. We train several gaze estimators adopting four different network architectures, including an architecture designed for gaze estimation (i.e.,iTracker-MHSA) and three originally designed for general computer vision tasks(i.e., BoTNet, HRNet, ResNeSt). Then, we select the best six estimators and ensemble their predictions through a linear combination. The method ranks the first on the leader-board of ETH-XGaze Competition, achieving an average angular error of $3.11^{\circ}$ on the ETH-XGaze test set.
Explainability via Interactivity? Supporting Nonexperts' Sensemaking of Pretrained CNN by Interacting with Their Daily Surroundings
Venue: arXiv cs.HC 2021
Year: 2021

Current research on Explainable AI (XAI) heavily targets on expert users (data scientists or AI developers). However, increasing importance has been argued for making AI more understandable to nonexperts, who are expected to leverage AI techniques, but have limited knowledge about AI. We present a mobile application to support nonexperts to interactively make sense of Convolutional Neural Networks (CNN); it allows users to play with a pretrained CNN by taking pictures of their surrounding objects. We use an up-to-date XAI technique (Class Activation Map) to intuitively visualize the model's decision (the most important image regions that lead to a certain result). Deployed in a university course, this playful learning tool was found to support design students to gain vivid understandings about the capabilities and limitations of pretrained CNNs in real-world environments. Concrete examples of students' playful explorations are reported to characterize their sensemaking processes reflecting different depths of thought.
Subset Privacy: Draw from an Obfuscated Urn
Venue: arXiv cs.CR 2021
Year: 2021

With the rapidly increasing ability to collect and analyze personal data, data privacy becomes an emerging concern. In this work, we develop a new statistical notion of local privacy to protect each categorical data that will be collected by untrusted entities. The proposed solution, named subset privacy, privatizes the original data value by replacing it with a random subset containing that value. We develop methods for the estimation of distribution functions and independence testing from subset-private data with theoretical guarantees. We also study different mechanisms to realize the subset privacy and evaluation metrics to quantify the amount of privacy in practice. Experimental results on both simulated and real-world datasets demonstrate the encouraging performance of the developed concepts and methods.
Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlated
Venue: arXiv cs.HC 2021
Year: 2021

The field explainable artificial intelligence (XAI) has brought about an arsenal of methods to render Machine Learning (ML) predictions more interpretable. But how useful explanations provided by transparent ML methods are for humans remains difficult to assess. Here we investigate the quality of interpretable computer vision algorithms using techniques from psychophysics. In crowdsourced annotation tasks we study the impact of different interpretability approaches on annotation accuracy and task time. We compare these quality metrics with classical XAI, automated quality metrics. Our results demonstrate that psychophysical experiments allow for robust quality assessment of transparency in machine learning. Interestingly the quality metrics computed without humans in the loop did not provide a consistent ranking of interpretability methods nor were they representative for how useful an explanation was for humans. These findings highlight the potential of methods from classical psychophysics for modern machine learning applications. We hope that our results provide convincing arguments for evaluating interpretability in its natural habitat, human-ML interaction, if the goal is to obtain an authentic assessment of interpretability.
A topological solution to object segmentation and tracking
Venue: arXiv cs.CV 2021
Year: 2021

The world is composed of objects, the ground, and the sky. Visual perception of objects requires solving two fundamental challenges: segmenting visual input into discrete units, and tracking identities of these units despite appearance changes due to object deformation, changing perspective, and dynamic occlusion. Current computer vision approaches to segmentation and tracking that approach human performance all require learning, raising the question: can objects be segmented and tracked without learning? Here, we show that the mathematical structure of light rays reflected from environment surfaces yields a natural representation of persistent surfaces, and this surface representation provides a solution to both the segmentation and tracking problems. We describe how to generate this surface representation from continuous visual input, and demonstrate that our approach can segment and invariantly track objects in cluttered synthetic video despite severe appearance changes, without requiring learning.
A Knowledge-based Approach for Answering Complex Questions in Persian
Venue: arXiv cs.CL 2021
Year: 2021

Research on open-domain question answering (QA) has a long tradition. A challenge in this domain is answering complex questions (CQA) that require complex inference methods and large amounts of knowledge. In low resource languages, such as Persian, there are not many datasets for open-domain complex questions and also the language processing toolkits are not very accurate. In this paper, we propose a knowledge-based approach for answering Persian complex questions using Farsbase; the Persian knowledge graph, exploiting PeCoQ; the newly created complex Persian question dataset. In this work, we handle multi-constraint and multi-hop questions by building their set of possible corresponding logical forms. Then Multilingual-BERT is used to select the logical form that best describes the input complex question syntactically and semantically. The answer to the question is built from the answer to the logical form, extracted from the knowledge graph. Experiments show that our approach outperforms other approaches in Persian CQA.
Recovering the Unbiased Scene Graphs from the Biased Ones
Venue: arXiv cs.CV 2021
Year: 2021

Given input images, scene graph generation (SGG) aims to produce comprehensive, graphical representations describing visual relationships among salient objects. Recently, more efforts have been paid to the long tail problem in SGG; however, the imbalance in the fraction of missing labels of different classes, or reporting bias, exacerbating the long tail is rarely considered and cannot be solved by the existing debiasing methods. In this paper we show that, due to the missing labels, SGG can be viewed as a "Learning from Positive and Unlabeled data" (PU learning) problem, where the reporting bias can be removed by recovering the unbiased probabilities from the biased ones by utilizing label frequencies, i.e., the per-class fraction of labeled, positive examples in all the positive examples. To obtain accurate label frequency estimates, we propose Dynamic Label Frequency Estimation (DLFE) to take advantage of training-time data augmentation and average over multiple training iterations to introduce more valid examples. Extensive experiments show that DLFE is more effective in estimating label frequencies than a naive variant of the traditional estimate, and DLFE significantly alleviates the long tail and achieves state-of-the-art debiasing performance on the VG dataset. We also show qualitatively that SGG models with DLFE produce prominently more balanced and unbiased scene graphs.
The Curious Case of the Diamond Network
Venue: arXiv cs.IT 2021
Year: 2021

This work considers the one-shot capacity of communication networks subject to adversarial noise affecting a subset of network edges. In particular, we examine previously-established upper bounds on one-shot capacity. We introduce the Diamond Network as a minimal example to show that known cut-set bounds are not sharp in general. We then give a capacity-achieving scheme for the Diamond Network that implements an adversary detection strategy. Finally, we give a sufficient condition for tightness of the Singleton Cut-Set Bound in a family of two-level networks.