OpenRaiser/Intern-Atlas huggingface dataset:
PEA: Improving the Performance of ReLU Networks for Free by Using Progressive Ensemble Activations
Venue: arXiv cs.CV 2022
Year: 2022

In recent years novel activation functions have been proposed to improve the performance of neural networks, and they show superior performance compared to the ReLU counterpart. However, there are environments, where the availability of complex activations is limited, and usually only the ReLU is supported. In this paper we propose methods that can be used to improve the performance of ReLU networks by using these efficient novel activations during model training. More specifically, we propose ensemble activations that are composed of the ReLU and one of these novel activations. Furthermore, the coefficients of the ensemble are neither fixed nor learned, but are progressively updated during the training process in a way that by the end of the training only the ReLU activations remain active in the network and the other activations can be removed. This means that in inference time the network contains ReLU activations only. We perform extensive evaluations on the ImageNet classification task using various compact network architectures and various novel activation functions. Results show 0.2-0.8% top-1 accuracy gain, which confirms the applicability of the proposed methods. Furthermore, we demonstrate the proposed methods on semantic segmentation and we boost the performance of a compact segmentation network by 0.34% mIOU on the Cityscapes dataset.
Ever heard of ethical AI? Investigating the salience of ethical AI issues among the German population
Venue: arXiv cs.CY 2022
Year: 2022

Building and implementing ethical AI systems that benefit the whole society is cost-intensive and a multi-faceted task fraught with potential problems. While computer science focuses mostly on the technical questions to mitigate social issues, social science addresses citizens' perceptions to elucidate social and political demands that influence the societal implementation of AI systems. Thus, in this study, we explore the salience of AI issues in the public with an emphasis on ethical criteria to investigate whether it is likely that ethical AI is actively requested by the population. Between May 2020 and April 2021, we conducted 15 surveys asking the German population about the most important AI-related issues (total of N=14,988 respondents). Our results show that the majority of respondents were not concerned with AI at all. However, it can be seen that general interest in AI and a higher educational level are predictive of some engagement with AI. Among those, who reported having thought about AI, specific applications (e.g., autonomous driving) were by far the most mentioned topics. Ethical issues are voiced only by a small subset of citizens with fairness, accountability, and transparency being the least mentioned ones. These have been identified in several ethical guidelines (including the EU Commission's proposal) as key elements for the development of ethical AI. The salience of ethical issues affects the behavioral intentions of citizens in the way that they 1) tend to avoid AI technology and 2) engage in public discussions about AI. We conclude that the low level of ethical implications may pose a serious problem for the actual implementation of ethical AI for the Common Good and emphasize that those who are presumably most affected by ethical issues of AI are especially unaware of ethical risks. Yet, once ethical AI is top of the mind, there is some potential for activism.
Entity Type Prediction Leveraging Graph Walks and Entity Descriptions
Venue: arXiv cs.CL 2022
Year: 2022

The entity type information in Knowledge Graphs (KGs) such as DBpedia, Freebase, etc. is often incomplete due to automated generation or human curation. Entity typing is the task of assigning or inferring the semantic type of an entity in a KG. This paper presents \textit{GRAND}, a novel approach for entity typing leveraging different graph walk strategies in RDF2vec together with textual entity descriptions. RDF2vec first generates graph walks and then uses a language model to obtain embeddings for each node in the graph. This study shows that the walk generation strategy and the embedding model have a significant effect on the performance of the entity typing task. The proposed approach outperforms the baseline approaches on the benchmark datasets DBpedia and FIGER for entity typing in KGs for both fine-grained and coarse-grained classes. The results show that the combination of order-aware RDF2vec variants together with the contextual embeddings of the textual entity descriptions achieve the best results.
Electricity Price Forecasting Model based on Gated Recurrent Units
Venue: arXiv cs.LG 2022
Year: 2022

The participation of consumers and producers in demand response programs has increased in smart grids, which reduces investment and operation costs of power systems. Also, with the advent of renewable energy sources, the electricity market is becoming more complex and unpredictable. To effectively implement demand response programs, forecasting the future price of electricity is very crucial for producers in the electricity market. Electricity prices are very volatile and change under the influence of various factors such as temperature, wind speed, rainfall, intensity of commercial and daily activities, etc. Therefore, considering the influencing factors as dependent variables can increase the accuracy of the forecast. In this paper, a model for electricity price forecasting is presented based on Gated Recurrent Units. The electrical load consumption is considered as an input variable in this model. Noise in electricity price seriously reduces the efficiency and effectiveness of analysis. Therefore, an adaptive noise reducer is integrated into the model for noise reduction. The SAEs are then used to extract features from the de-noised electricity price. Finally, the de-noised features are fed into the GRU to train predictor. Results on real dataset shows that the proposed methodology can perform effectively in prediction of electricity price.
FleetPy: A Modular Open-Source Simulation Tool for Mobility On-Demand Services
Venue: arXiv cs.MA 2022
Year: 2022

The market share of mobility on-demand (MoD) services strongly increased in recent years and is expected to rise even higher once vehicle automation is fully available. These services might reduce space consumption in cities as fewer parking spaces are required if private vehicle trips are replaced. If rides are shared additionally, occupancy related traffic efficiency is increased. Simulations help to identify the actual impact of MoD on a traffic system, evaluate new control algorithms for improved service efficiency and develop guidelines for regulatory measures. This paper presents the open-source agent-based simulation framework FleetPy. FleetPy (written in the programming language "Python") is explicitly developed to model MoD services in a high level of detail. It specially focuses on the modeling of interactions of users with operators while its flexibility allows the integration and embedding of multiple operators in the overall transportation system. Its modular structure ensures the transferabillity of previously developed elements and the selection of an appropriate level of modeling detail. This paper compares existing simulation frameworks for MoD services and highlights exclusive features of FleetPy. The upper level simulation flows are presented, followed by required input data for the simulation and the output data FleetPy produces. Additionally, the modules within FleetPy and high-level descriptions of current implementations are provided. Finally, an example showcase for Manhattan, NYC provides insights into the impacts of different modules for simulation flow, fleet optimization, traveler behavior and network representation.
Exploiting and Defending Against the Approximate Linearity of Apple's NeuralHash
Venue: arXiv cs.CR 2022
Year: 2022

Perceptual hashes map images with identical semantic content to the same $n$-bit hash value, while mapping semantically-different images to different hashes. These algorithms carry important applications in cybersecurity such as copyright infringement detection, content fingerprinting, and surveillance. Apple's NeuralHash is one such system that aims to detect the presence of illegal content on users' devices without compromising consumer privacy. We make the surprising discovery that NeuralHash is approximately linear, which inspires the development of novel black-box attacks that can (i) evade detection of "illegal" images, (ii) generate near-collisions, and (iii) leak information about hashed images, all without access to model parameters. These vulnerabilities pose serious threats to NeuralHash's security goals; to address them, we propose a simple fix using classical cryptographic standards.
CuDi: Curve Distillation for Efficient and Controllable Exposure Adjustment
Venue: arXiv cs.CV 2022
Year: 2022

We present Curve Distillation, CuDi, for efficient and controllable exposure adjustment without the requirement of paired or unpaired data during training. Our method inherits the zero-reference learning and curve-based framework from an effective low-light image enhancement method, Zero-DCE, with further speed up in its inference speed, reduction in its model size, and extension to controllable exposure adjustment. The improved inference speed and lightweight model are achieved through novel curve distillation that approximates the time-consuming iterative operation in the conventional curve-based framework by high-order curve's tangent line. The controllable exposure adjustment is made possible with a new self-supervised spatial exposure control loss that constrains the exposure levels of different spatial regions of the output to be close to the brightness distribution of an exposure map serving as an input condition. Different from most existing methods that can only correct either underexposed or overexposed photos, our approach corrects both underexposed and overexposed photos with a single model. Notably, our approach can additionally adjust the exposure levels of a photo globally or locally with the guidance of an input condition exposure map, which can be pre-defined or manually set in the inference stage. Through extensive experiments, we show that our method is appealing for its fast, robust, and flexible performance, outperforming state-of-the-art methods in real scenes. Project page: https://li-chongyi.github.io/CuDi_files/.
A parallel algorithm for unilateral contact problems
Venue: arXiv cs.CE 2022
Year: 2022

In this paper, we introduce a novel parallel contact algorithm designed to run efficiently in High-Performance Computing based supercomputers. Particular emphasis is put on its computational implementation in a multiphysics finite element code. The algorithm is based on the method of partial Dirichlet-Neumann boundary conditions and is capable to solve numerically a nonlinear contact problem between rigid and deformable bodies in a whole parallel framework. Its distinctive characteristic is that the contact is tackled as a coupled problem, in which the contacting bodies are treated separately, in a staggered way. Then, the coupling is performed through the exchange of boundary conditions at the contact interface following a Gauss-Seidel strategy. To validate this algorithm we conducted several benchmark tests by comparing the proposed solution against theoretical and other numerical solutions. Finally, we evaluated the parallel performance of the proposed algorithm in a real impact test to show its capabilities for large-scale applications.
Aztec curve: proposal for a new space-filling curve
Venue: arXiv cs.CV 2022
Year: 2022

Different space-filling curves (SFCs) are briefly reviewed in this paper, and a new one is proposed. A century has passed between the inception of this kind of curves, since then they have been found useful in computer science, particularly in data storage and indexing due to their clustering properties, being Hilbert curve the most well-known member of the family of fractals. The proposed Aztec curve, with similar characteristics to the Hilbert's curve, is introduced in this paper, accompanied by a grammatical description for its construction. It yields the possibility of creating bi-dimensional clusters, not available for Hilbert nor Peano curves. Additional to this, a case of application on the scope of Compressed Sensing is implemented, in which the use of Hilbert curve is contrasted with Aztec curve, having a similar performance, and positioning the Aztec curve as viable and a new alternative for future exploitation on applications that make use of SFC's.
Model selection with Gini indices under auto-calibration
Venue: arXiv cs.LG 2022
Year: 2022

The Gini index does not give a strictly consistent scoring rule in general. Therefore, maximizing the Gini index may lead to wrong decisions. The main issue is that the Gini index is a rank-based score that is not calibration-sensitive. We show that the Gini index allows for strictly consistent scoring if we restrict to the class of auto-calibrated regression models.
Expanding the class of global objective functions for dissimilarity-based hierarchical clustering
Venue: arXiv cs.LG 2022
Year: 2022

Recent work on dissimilarity-based hierarchical clustering has led to the introduction of global objective functions for this classical problem. Several standard approaches, such as average linkage, as well as some new heuristics have been shown to provide approximation guarantees. Here we introduce a broad new class of objective functions which satisfy desirable properties studied in prior work. Many common agglomerative and divisive clustering methods are shown to be greedy algorithms for these objectives, which are inspired by related concepts in phylogenetics.
Optimistic and Topological Value Iteration for Simple Stochastic Games
Venue: arXiv cs.GT 2022
Year: 2022

While value iteration (VI) is a standard solution approach to simple stochastic games (SSGs), it suffered from the lack of a stopping criterion. Recently, several solutions have appeared, among them also "optimistic" VI (OVI). However, OVI is applicable only to one-player SSGs with no end components. We lift these two assumptions, making it available to general SSGs. Further, we utilize the idea in the context of topological VI, where we provide an efficient precise solution. In order to compare the new algorithms with the state of the art, we use not only the standard benchmarks, but we also design a random generator of SSGs, which can be biased towards various types of models, aiding in understanding the advantages of different algorithms on SSGs.
Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions
Venue: arXiv cs.RO 2022
Year: 2022

Reinforcement Learning (RL) and continuous nonlinear control have been successfully deployed in multiple domains of complicated sequential decision-making tasks. However, given the exploration nature of the learning process and the presence of model uncertainty, it is challenging to apply them to safety-critical control tasks due to the lack of safety guarantee. On the other hand, while combining control-theoretical approaches with learning algorithms has shown promise in safe RL applications, the sample efficiency of safe data collection process for control is not well addressed. In this paper, we propose a \emph{provably} sample efficient episodic safe learning framework for online control tasks that leverages safe exploration and exploitation in an unknown, nonlinear dynamical system. In particular, the framework 1) extends control barrier functions (CBFs) in a stochastic setting to achieve provable high-probability safety under uncertainty during model learning and 2) integrates an optimism-based exploration strategy to efficiently guide the safe exploration process with learned dynamics for \emph{near optimal} control performance. We provide formal analysis on the episodic regret bound against the optimal controller and probabilistic safety with theoretical guarantees. Simulation results are provided to demonstrate the effectiveness and efficiency of the proposed algorithm.
Code Comment Inconsistency Detection with BERT and Longformer
Venue: arXiv cs.CL 2022
Year: 2022

Comments, or natural language descriptions of source code, are standard practice among software developers. By communicating important aspects of the code such as functionality and usage, comments help with software project maintenance. However, when the code is modified without an accompanying correction to the comment, an inconsistency between the comment and code can arise, which opens up the possibility for developer confusion and bugs. In this paper, we propose two models based on BERT (Devlin et al., 2019) and Longformer (Beltagy et al., 2020) to detect such inconsistencies in a natural language inference (NLI) context. Through an evaluation on a previously established corpus of comment-method pairs both during and after code changes, we demonstrate that our models outperform multiple baselines and yield comparable results to the state-of-the-art models that exclude linguistic and lexical features. We further discuss ideas for future research in using pretrained language models for both inconsistency detection and automatic comment updating.
Deep Learning-based Occluded Person Re-identification: A Survey
Venue: arXiv cs.CV 2022
Year: 2022

Occluded person re-identification (Re-ID) aims at addressing the occlusion problem when retrieving the person of interest across multiple cameras. With the promotion of deep learning technology and the increasing demand for intelligent video surveillance, the frequent occlusion in real-world applications has made occluded person Re-ID draw considerable interest from researchers. A large number of occluded person Re-ID methods have been proposed while there are few surveys that focus on occlusion. To fill this gap and help boost future research, this paper provides a systematic survey of occluded person Re-ID. Through an in-depth analysis of the occlusion in person Re-ID, most existing methods are found to only consider part of the problems brought by occlusion. Therefore, we review occlusion-related person Re-ID methods from the perspective of issues and solutions. We summarize four issues caused by occlusion in person Re-ID, i.e., position misalignment, scale misalignment, noisy information, and missing information. The occlusion-related methods addressing different issues are then categorized and introduced accordingly. After that, we summarize and compare the performance of recent occluded person Re-ID methods on four popular datasets: Partial-ReID, Partial-iLIDS, Occluded-ReID, and Occluded-DukeMTMC. Finally, we provide insights on promising future research directions.
GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation
Venue: arXiv cs.CL 2022
Year: 2022

Transformer structure, stacked by a sequence of encoder and decoder network layers, achieves significant development in neural machine translation. However, vanilla Transformer mainly exploits the top-layer representation, assuming the lower layers provide trivial or redundant information and thus ignoring the bottom-layer feature that is potentially valuable. In this work, we propose the Group-Transformer model (GTrans) that flexibly divides multi-layer representations of both encoder and decoder into different groups and then fuses these group features to generate target words. To corroborate the effectiveness of the proposed method, extensive experiments and analytic experiments are conducted on three bilingual translation benchmarks and two multilingual translation tasks, including the IWLST-14, IWLST-17, LDC, WMT-14 and OPUS-100 benchmark. Experimental and analytical results demonstrate that our model outperforms its Transformer counterparts by a consistent gain. Furthermore, it can be successfully scaled up to 60 encoder layers and 36 decoder layers.
Domain-Specific Quantum Architecture Optimization
Venue: arXiv cs.AR 2022
Year: 2022

With the steady progress in quantum computing over recent years, roadmaps for upscaling quantum processors have relied heavily on the targeted qubit architectures. So far, similarly to the early age of classical computing, these designs have been crafted by human experts. These general-purpose architectures, however, leave room for customization and optimization, especially when targeting popular near-term QC applications. In classical computing, customized architectures have demonstrated significant performance and energy efficiency gains over general-purpose counterparts. In this paper, we present a framework for optimizing quantum architectures, specifically through customizing qubit connectivity. It is the first work that (1) provides performance guarantees by integrating architecture optimization with an optimal compiler, (2) evaluates the impact of connectivity customization under a realistic crosstalk error model, and (3) benchmarks on realistic circuits of near-term interest, such as the quantum approximate optimization algorithm (QAOA) and quantum convolutional neural network (QCNN). We demonstrate up to 59% fidelity improvement in simulation by optimizing the heavy-hexagon architecture for QAOA circuits, and up to 14% improvement on the grid architecture. For the QCNN circuit, architecture optimization improves fidelity by 11% on the heavy-hexagon architecture and 605% on the grid architecture.
A Transfer Learning-Based Approach to Marine Vessel Re-Identification
Venue: arXiv cs.CV 2022
Year: 2022

Marine vessel re-identification technology is an important component of intelligent shipping systems and an important part of the visual perception tasks required for marine surveillance. However, unlike the situation on land, the maritime environment is complex and variable with fewer samples, and it is more difficult to perform vessel re-identification at sea. Therefore, this paper proposes a transfer dynamic alignment algorithm and simulates the swaying situation of vessels at sea, using a well-camouflaged and similar warship as the test target to improve the recognition difficulty and thus cope with the impact caused by complex sea conditions, and discusses the effect of different types of vessels as transfer objects. The experimental results show that the improved algorithm improves the mean average accuracy (mAP) by 10.2% and the first hit rate (Rank1) by 4.9% on average.
SERCNN: Stacked Embedding Recurrent Convolutional Neural Network in Detecting Depression on Twitter
Venue: arXiv cs.AI 2022
Year: 2022

Conventional approaches to identify depression are not scalable, and the public has limited awareness of mental health, especially in developing countries. As evident by recent studies, social media has the potential to complement mental health screening on a greater scale. The vast amount of first-person narrative posts in chronological order can provide insights into one's thoughts, feelings, behavior, or mood for some time, enabling a better understanding of depression symptoms reflected in the online space. In this paper, we propose SERCNN, which improves the user representation by (1) stacking two pretrained embeddings from different domains and (2) reintroducing the embedding context to the MLP classifier. Our SERCNN shows great performance over state-of-the-art and other baselines, achieving 93.7% accuracy in a 5-fold cross-validation setting. Since not all users share the same level of online activity, we introduced the concept of a fixed observation window that quantifies the observation period in a predefined number of posts. With as minimal as 10 posts per user, SERCNN performed exceptionally well with an 87% accuracy, which is on par with the BERT model, while having 98% less in the number of parameters. Our findings open up a promising direction for detecting depression on social media with a smaller number of posts for inference, towards creating solutions for a cost-effective and timely intervention. We hope that our work can bring this research area closer to real-world adoption in existing clinical practice.