OpenRaiser/Intern-Atlas huggingface dataset:
Tight Hardness Results for Training Depth-2 ReLU Networks
Venue: arXiv cs.LG 2020
Year: 2020

We prove several hardness results for training depth-2 neural networks with the ReLU activation function; these networks are simply weighted sums (that may include negative coefficients) of ReLUs. Our goal is to output a depth-2 neural network that minimizes the square loss with respect to a given training set. We prove that this problem is NP-hard already for a network with a single ReLU. We also prove NP-hardness for outputting a weighted sum of $k$ ReLUs minimizing the squared error (for $k>1$) even in the realizable setting (i.e., when the labels are consistent with an unknown depth-2 ReLU network). We are also able to obtain lower bounds on the running time in terms of the desired additive error $ε$. To obtain our lower bounds, we use the Gap Exponential Time Hypothesis (Gap-ETH) as well as a new hypothesis regarding the hardness of approximating the well known Densest $κ$-Subgraph problem in subexponential time (these hypotheses are used separately in proving different lower bounds). For example, we prove that under reasonable hardness assumptions, any proper learning algorithm for finding the best fitting ReLU must run in time exponential in $1/ε^2$. Together with a previous work regarding improperly learning a ReLU (Goel et al., COLT'17), this implies the first separation between proper and improper algorithms for learning a ReLU. We also study the problem of properly learning a depth-2 network of ReLUs with bounded weights giving new (worst-case) upper bounds on the running time needed to learn such networks both in the realizable and agnostic settings. Our upper bounds on the running time essentially matches our lower bounds in terms of the dependency on $ε$.
Eco-Routing Using Open Street Maps
Venue: arXiv cs.NI 2020
Year: 2020

A vehicle's fuel consumption depends on its type, the speed, the condition, and the gradients of the road on which it is moving. We developed a Routing Engine for finding an eco-route (one with low fuel consumption) between a source and a destination. Open Street Maps has data on road conditions. We used CGIAR-CSI road elevation data 16[4] to integrate the road gradients into the proposed route-finding algorithm that modifies Open Street Routing Machine (OSRM). It allowed us to dynamically predict a vehicle's velocity, considering both the conditions and the road segment's slope. Using Highway EneRgy Assessment (HERA) methodology, we calculated the fuel consumed by a vehicle given its type and velocity. We have created both web and mobile interfaces through which users can specify Geo coordinates or human-readable addresses of a source and a destination. The user interface graphically displays the route obtained from the proposed Routing Engine with a detailed travel itinerary.
SocialGuard: An Adversarial Example Based Privacy-Preserving Technique for Social Images
Venue: arXiv cs.CV 2020
Year: 2020

The popularity of various social platforms has prompted more people to share their routine photos online. However, undesirable privacy leakages occur due to such online photo sharing behaviors. Advanced deep neural network (DNN) based object detectors can easily steal users' personal information exposed in shared photos. In this paper, we propose a novel adversarial example based privacy-preserving technique for social images against object detectors based privacy stealing. Specifically, we develop an Object Disappearance Algorithm to craft two kinds of adversarial social images. One can hide all objects in the social images from being detected by an object detector, and the other can make the customized sensitive objects be incorrectly classified by the object detector. The Object Disappearance Algorithm constructs perturbation on a clean social image. After being injected with the perturbation, the social image can easily fool the object detector, while its visual quality will not be degraded. We use two metrics, privacy-preserving success rate and privacy leakage rate, to evaluate the effectiveness of the proposed method. Experimental results show that, the proposed method can effectively protect the privacy of social images. The privacy-preserving success rates of the proposed method on MS-COCO and PASCAL VOC 2007 datasets are high up to 96.1% and 99.3%, respectively, and the privacy leakage rates on these two datasets are as low as 0.57% and 0.07%, respectively. In addition, compared with existing image processing methods (low brightness, noise, blur, mosaic and JPEG compression), the proposed method can achieve much better performance in privacy protection and image visual quality maintenance.
Joint Extraction of Entity and Relation with Information Redundancy Elimination
Venue: arXiv cs.CL 2020
Year: 2020

To solve the problem of redundant information and overlapping relations of the entity and relation extraction model, we propose a joint extraction model. This model can directly extract multiple pairs of related entities without generating unrelated redundant information. We also propose a recurrent neural network named Encoder-LSTM that enhances the ability of recurrent units to model sentences. Specifically, the joint model includes three sub-modules: the Named Entity Recognition sub-module consisted of a pre-trained language model and an LSTM decoder layer, the Entity Pair Extraction sub-module which uses Encoder-LSTM network to model the order relationship between related entity pairs, and the Relation Classification sub-module including Attention mechanism. We conducted experiments on the public datasets ADE and CoNLL04 to evaluate the effectiveness of our model. The results show that the proposed model achieves good performance in the task of entity and relation extraction and can greatly reduce the amount of redundant information.
Beamforming Design for Multiuser uRLLC with Finite Blocklength Transmission
Venue: arXiv cs.IT 2020
Year: 2020

Driven by the explosive growth of Internet of Things (IoT) devices with stringent requirements on latency and reliability, ultra-reliability and low latency communication (uRLLC) has become one of the three key communication scenarios for the fifth generation (5G) and beyond 5G communication systems. In this paper, we focus on the beamforming design problem for the downlink multiuser uRLLC systems. Since the strict demand on the reliability and latency, in general, short packet transmission is a favorable form for uRLLC systems, which indicates the literature Shannon's capacity formula is no longer applicable. With the finite blocklength transmission, the achievable delivery rate is greatly influenced by the reliability and latency. Using the developed achievable delivery rate formula for finite blocklength transmission, we respectively formulate the problems of interest as the weighted sum rate maximization, energy efficiency maximization, and user fairness optimization by considering the maximum allowable transmission power and minimum rate requirement. It's worthy pointing out that this is the first work to design the beamforming vectors for the downlink multiuser uRLLC systems. To address these non-convex problems, some important insights have been discovered by analyzing the function of achievable delivery rate. For example, the minimum rate requirement can be realized by low bounded the signal-to-interference-plus-noise ratio. Based on the discovered results, we provide algorithms to optimize the beamforming vectors and power allocation, which are guaranteed to converge to a local optimum solution of the formulated problems with low computational complexity. Our simulation results reveal that our proposed beamforming algorithms outperform the zero-forcing beamforming algorithm with equal power allocation widely used in the existing literatures.
Use the Spear as a Shield: A Novel Adversarial Example based Privacy-Preserving Technique against Membership Inference Attacks
Venue: arXiv cs.CR 2020
Year: 2020

Recently, the membership inference attack poses a serious threat to the privacy of confidential training data of machine learning models. This paper proposes a novel adversarial example based privacy-preserving technique (AEPPT), which adds the crafted adversarial perturbations to the prediction of the target model to mislead the adversary's membership inference model. The added adversarial perturbations do not affect the accuracy of target model, but can prevent the adversary from inferring whether a specific data is in the training set of the target model. Since AEPPT only modifies the original output of the target model, the proposed method is general and does not require modifying or retraining the target model. Experimental results show that the proposed method can reduce the inference accuracy and precision of the membership inference model to 50%, which is close to a random guess. Further, for those adaptive attacks where the adversary knows the defense mechanism, the proposed AEPPT is also demonstrated to be effective. Compared with the state-of-the-art defense methods, the proposed defense can significantly degrade the accuracy and precision of membership inference attacks to 50% (i.e., the same as a random guess) while the performance and utility of the target model will not be affected.
3D Invisible Cloak
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we propose a novel physical stealth attack against the person detectors in real world. The proposed method generates an adversarial patch, and prints it on real clothes to make a three dimensional (3D) invisible cloak. Anyone wearing the cloak can evade the detection of person detectors and achieve stealth. We consider the impacts of those 3D physical constraints (i.e., radian, wrinkle, occlusion, angle, etc.) on person stealth attacks, and propose 3D transformations to generate 3D invisible cloak. We launch the person stealth attacks in 3D physical space instead of 2D plane by printing the adversarial patches on real clothes under challenging and complex 3D physical scenarios. The conventional and 3D transformations are performed on the patch during its optimization process. Further, we study how to generate the optimal 3D invisible cloak. Specifically, we explore how to choose input images with specific shapes and colors to generate the optimal 3D invisible cloak. Besides, after successfully making the object detector misjudge the person as other objects, we explore how to make a person completely disappeared, i.e., the person will not be detected as any objects. Finally, we present a systematic evaluation framework to methodically evaluate the performance of the proposed attack in digital domain and physical world. Experimental results in various indoor and outdoor physical scenarios show that, the proposed person stealth attack method is robust and effective even under those complex and challenging physical conditions, such as the cloak is wrinkled, obscured, curved, and from different angles. The attack success rate in digital domain (Inria data set) is 86.56%, while the static and dynamic stealth attack performance in physical world is 100% and 77%, respectively, which are significantly better than existing works.
Distributed joint dynamic maintenance and production scheduling in manufacturing systems: Framework based on model predictive control and Benders decomposition
Venue: arXiv cs.DC 2020
Year: 2020

Scheduling the maintenance based on the condition, respectively the degradation level of the system leads to improved system's reliability while minimizing the maintenance cost. Since the degradation level changes dynamically during the system's operation, we face a dynamic maintenance scheduling problem. In this paper, we address the dynamic maintenance scheduling of manufacturing systems based on their degradation level. The manufacturing system consists of several units with a defined capacity and an individual dynamic degradation model, seeking to optimize their reward. The units sell their production capacity, while maintaining the systems based on the degradation state to prevent failures. The manufacturing units are jointly responsible for fulfilling the demand of the system. This induces a coupling constraint among the agents. Hence, we face a large-scale mixed-integer dynamic maintenance scheduling problem. In order to handle the dynamic model of the system and large-scale optimization, we propose a distributed algorithm using model predictive control (MPC) and Benders decomposition method. In the proposed algorithm, first, the master problem obtains the maintenance scheduling for all the agents, and then based on this data, the agents obtain their optimal production using the distributed MPC method which employs the dual decomposition approach to tackle the coupling constraints among the agents. The effectiveness of the proposed method is investigated on a case study.
CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method
Venue: arXiv cs.LG 2020
Year: 2020

Drug discovery is a multi-stage process that comprises two costly major steps: pre-clinical research and clinical trials. Among its stages, lead optimization easily consumes more than half of the pre-clinical budget. We propose a combined machine learning and molecular modeling approach that automates lead optimization workflow \textit{in silico}. The initial data collection is achieved with physics-based molecular dynamics (MD) simulation. Contact matrices are calculated as the preliminary features extracted from the simulations. To take advantage of the temporal information from the simulations, we enhanced contact matrices data with temporal dynamism representation, which are then modeled with unsupervised convolutional variational autoencoder (CVAE). Finally, conventional clustering method and CVAE-based clustering method are compared with metrics to rank the submolecular structures and propose potential candidates for lead optimization. With no need for extensive structure-activity relationship database, our method provides new hints for drug modification hotspots which can be used to improve drug efficacy. Our workflow can potentially reduce the lead optimization turnaround time from months/years to days compared with the conventional labor-intensive process and thus can potentially become a valuable tool for medical researchers.
CybergeoNetworks, an interactive application for the geographical and semantic analysis of scientific publications
Venue: arXiv cs.DL 2020
Year: 2020

The increase in the number of publications has made more difficult for authors to situate their work within previous literature, especially on subjects studied from different disciplinary viewpoints. Besides, new data analysis techniques and new bibliometrics data sources provide an opportunity to map and navigate scientific landscapes. We introduce here an open-source and open-access web application designed for the multi-dimensional exploration of a journal content, including the mapping of geographical, semantic and citations networks. The application is profiled and implemented for the geography journal Cybergeo, a generalist geography journal which receives contributions from multiple sub-disciplines. We suggest that such initiatives are crucial to promote open science and reflexivity.
Investigation on Research Ethics and Building a Benchmark
Venue: arXiv cs.CR 2020
Year: 2020

When dealing with leading edge cyber security research, especially when operating from the perspective of an attacker or a red team, it becomes necessary for one to at times consider how ethics comes into play. There are currently no cyber security-specific ethics standards, which in particular is one reason more adversarial cyber security research lags behind in Japan. In this research, using machine learning and manual methods we extracted best practices for research ethics from past top conference papers. Using this knowledge we constructed an ethics knowledge base for cyber security research. Such a knowledge base can be used to properly distinguish grey-area research so that it is not wrongly forbidden. Using a decision tree-style user interface that we created for our knowledge base, researchers may be able to efficiently identify which aspects of their research require ethical consideration. In this work, as a preliminary step we focused on only a portion of the areas of research covered by cyber security conferences, but our results are applicable to any area of research.
Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach
Venue: arXiv cs.CV 2020
Year: 2020

Text segmentation is a prerequisite in many real-world text-related tasks, e.g., text style transfer, and scene text removal. However, facing the lack of high-quality datasets and dedicated investigations, this critical prerequisite has been left as an assumption in many works, and has been largely overlooked by current research. To bridge this gap, we proposed TextSeg, a large-scale fine-annotated text dataset with six types of annotations: word- and character-wise bounding polygons, masks and transcriptions. We also introduce Text Refinement Network (TexRNet), a novel text segmentation approach that adapts to the unique properties of text, e.g. non-convex boundary, diverse texture, etc., which often impose burdens on traditional segmentation models. In our TexRNet, we propose text specific network designs to address such challenges, including key features pooling and attention-based similarity checking. We also introduce trimap and discriminator losses that show significant improvement on text segmentation. Extensive experiments are carried out on both our TextSeg dataset and other existing datasets. We demonstrate that TexRNet consistently improves text segmentation performance by nearly 2% compared to other state-of-the-art segmentation methods. Our dataset and code will be made available at https://github.com/SHI-Labs/Rethinking-Text-Segmentation.
Discrete Math with Programming: A Principled Approach
Venue: arXiv cs.CY 2020
Year: 2020

Discrete mathematics is the foundation of computer science. It focuses on concepts and reasoning methods that are studied using math notations. It has long been argued that discrete math is better taught with programming, which takes concepts and computing methods and turns them into executable programs. What has been lacking is a principled approach that supports all central concepts of discrete math -- especially predicate logic -- and that directly and precisely connects math notations with executable programs. This paper introduces such an approach. It is based on the use of a powerful language that extends the Python programming language with proper logic quantification ("for all" and "exists some"), as well as declarative set comprehension (also known as set builder) and aggregation (e.g., sum and product). Math and logical statements can be expressed precisely at a high level and be executed directly on a computer, encouraging declarative programming together with algorithmic programming. We describe the approach, detailed examples, experience in using it, and the lessons learned.
Robotic grasp detection using a novel two-stage approach
Venue: arXiv cs.RO 2020
Year: 2020

Recently, deep learning has been successfully applied to robotic grasp detection. Based on convolutional neural networks (CNNs), there have been lots of end-to-end detection approaches. But end-to-end approaches have strict requirements for the dataset used for training the neural network models and it's hard to achieve in practical use. Therefore, we proposed a two-stage approach using particle swarm optimizer (PSO) candidate estimator and CNN to detect the most likely grasp. Our approach achieved an accuracy of 92.8% on the Cornell Grasp Dataset, which leaped into the front ranks of the existing approaches and is able to run at real-time speeds. After a small change of the approach, we can predict multiple grasps per object in the meantime so that an object can be grasped in a variety of ways.
Human-Agent Cooperation in Bridge Bidding
Venue: arXiv cs.AI 2020
Year: 2020

We introduce a human-compatible reinforcement-learning approach to a cooperative game, making use of a third-party hand-coded human-compatible bot to generate initial training data and to perform initial evaluation. Our learning approach consists of imitation learning, search, and policy iteration. Our trained agents achieve a new state-of-the-art for bridge bidding in three settings: an agent playing in partnership with a copy of itself; an agent partnering a pre-existing bot; and an agent partnering a human player.
Short-Term Load Forecasting using Bi-directional Sequential Models and Feature Engineering for Small Datasets
Venue: arXiv cs.LG 2020
Year: 2020

Electricity load forecasting enables the grid operators to optimally implement the smart grid's most essential features such as demand response and energy efficiency. Electricity demand profiles can vary drastically from one region to another on diurnal, seasonal and yearly scale. Hence to devise a load forecasting technique that can yield the best estimates on diverse datasets, specially when the training data is limited, is a big challenge. This paper presents a deep learning architecture for short-term load forecasting based on bidirectional sequential models in conjunction with feature engineering that extracts the hand-crafted derived features in order to aid the model for better learning and predictions. In the proposed architecture, named as Deep Derived Feature Fusion (DeepDeFF), the raw input and hand-crafted features are trained at separate levels and then their respective outputs are combined to make the final prediction. The efficacy of the proposed methodology is evaluated on datasets from five countries with completely different patterns. The results demonstrate that the proposed technique is superior to the existing state of the art.
LocKedge: Low-Complexity Cyberattack Detection in IoT Edge Computing
Venue: arXiv cs.CR 2020
Year: 2020

Internet of Things and its applications are becoming commonplace with more devices, but always at risk of network security. It is therefore crucial for an IoT network design to identify attackers accurately, quickly and promptly. Many solutions have been proposed, mainly concerning secure IoT architectures and classification algorithms, but none of them have paid enough attention to reducing the complexity. Our proposal in this paper is an edge cloud architecture that fulfills the detection task right at the edge layer, near the source of the attacks for quick response, versatility, as well as reducing the workload of the cloud. We also propose a multi attack detection mechanism called LocKedge Low Complexity Cyberattack Detection in IoT Edge Computing, which has low complexity for deployment at the edge zone while still maintaining high accuracy. LocKedge is implemented in two manners: centralized and federated learning manners in order to verify the performance of the architecture from different perspectives. The performance of our proposed mechanism is compared with that of other machine learning and deep learning methods using the most updated BoT IoT data set. The results show that LocKedge outperforms other algorithms such as NN, CNN, RNN, KNN, SVM, KNN, RF and Decision Tree in terms of accuracy and NN in terms of complexity.