OpenRaiser/Intern-Atlas huggingface dataset:
Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application
Venue: arXiv cs.AI 2021
Year: 2021

Benefiting from the strong ability of the pre-trained model, the research on Chinese Word Segmentation (CWS) has made great progress in recent years. However, due to massive computation, large and complex models are incapable of empowering their ability for industrial use. On the other hand, for low-resource scenarios, the prevalent decode method, such as Conditional Random Field (CRF), fails to exploit the full information of the training data. This work proposes a fast and accurate CWS framework that incorporates a light-weighted model and an upgraded decode method (PCRF) towards industrially low-resource CWS scenarios. First, we distill a Transformer-based student model as an encoder, which not only accelerates the inference speed but also combines open knowledge and domain-specific knowledge. Second, the perplexity score to evaluate the language model is fused into the CRF module to better identify the word boundaries. Experiments show that our work obtains relatively high performance on multiple datasets with as low as 14\% of time consumption compared with the original BERT-based model. Moreover, under the low-resource setting, we get superior results in comparison with the traditional decoding methods.
The Faulty GPS Problem: Shortest Time Paths in Networks with Unreliable Directions
Venue: arXiv cs.AI 2021
Year: 2021

This paper optimizes motion planning when there is a known risk that the road choice suggested by a Satnav (GPS) is not on a shortest path. At every branch node of a network Q, a Satnav (GPS) points to the arc leading to the destination, or home node, H - but only with a high known probability p. Always trusting the Satnav's suggestion may lead to an infinite cycle. If one wishes to reach H in least expected time, with what probability q=q(Q,p) should one trust the pointer (if not, one chooses randomly among the other arcs)? We call this the Faulty Satnav (GPS) Problem. We also consider versions where the trust probability q can depend on the degree of the current node and a `treasure hunt' where two searchers try to reach H first. The agent searching for H need not be a car, that is just a familiar example -- it could equally be a UAV receiving unreliable GPS information. This problem has its origin not in driver frustration but in the work of Fonio et al (2017) on ant navigation, where the pointers correspond to pheromone markers pointing to the nest. Neither the driver or ant will know the exact process by which a choice (arc) is suggested, which puts the problem into the domain of how much to trust an option suggested by AI.
Physics-guided Loss Functions Improve Deep Learning Performance in Inverse Scattering
Venue: arXiv cs.LG 2021
Year: 2021

Solving electromagnetic inverse scattering problems (ISPs) is challenging due to the intrinsic nonlinearity, ill-posedness, and expensive computational cost. Recently, deep neural network (DNN) techniques have been successfully applied on ISPs and shown potential of superior imaging over conventional methods. In this paper, we analyse the analogy between DNN solvers and traditional iterative algorithms and discuss how important physical phenomena cannot be effectively incorporated in the training process. We show the importance of including near-field priors in the learning process of DNNs. To this end, we propose new designs of loss functions which incorporate multiple-scattering based near-field quantities (such as scattered fields or induced currents within domain of interest). Effects of physics-guided loss functions are studied using a variety of numerical experiments. Pros and cons of the investigated ISP solvers with different loss functions are summarized.
Forecasting Crude Oil Price Using Event Extraction
Venue: arXiv cs.LG 2021
Year: 2021

Research on crude oil price forecasting has attracted tremendous attention from scholars and policymakers due to its significant effect on the global economy. Besides supply and demand, crude oil prices are largely influenced by various factors, such as economic development, financial markets, conflicts, wars, and political events. Most previous research treats crude oil price forecasting as a time series or econometric variable prediction problem. Although recently there have been researches considering the effects of real-time news events, most of these works mainly use raw news headlines or topic models to extract text features without profoundly exploring the event information. In this study, a novel crude oil price forecasting framework, AGESL, is proposed to deal with this problem. In our approach, an open domain event extraction algorithm is utilized to extract underlying related events, and a text sentiment analysis algorithm is used to extract sentiment from massive news. Then a deep neural network integrating the news event features, sentimental features, and historical price features is built to predict future crude oil prices. Empirical experiments are performed on West Texas Intermediate (WTI) crude oil price data, and the results show that our approach obtains superior performance compared with several benchmark methods.
Image-based monitoring of bolt loosening through deep-learning-based integrated detection and tracking
Venue: arXiv cs.CV 2021
Year: 2021

Structural bolts are critical components used in different structural elements, such as beam-column connections and friction damping devices. The clamping force in structural bolts is highly influenced by the bolt rotation. Much of the existing vision-based research about bolt rotation estimation relies on traditional computer vision algorithms such as Hough Transform to assess static images of bolts. This requires careful image preprocessing, and it may not perform well in the situation of complicated bolt assemblies, or in the presence of surrounding objects and background noise, thus hindering their real-world applications. In this study, an integrated real-time detect-track method, namely RTDT-Bolt, is proposed to monitor the bolt rotation angle. First, a real-time convolutional-neural-networks-based object detector, named YOLOv3-tiny, is established and trained to localize structural bolts. Then, the target-free object tracking algorithm based on optical flow is implemented, to continuously monitor and quantify the rotation of structural bolts. In order to enhance the tracking performance against background noise and potential illumination changes during tracking, the YOLOv3-tiny is integrated with the optical flow tracking algorithm to re-detect the bolts when the tracking gets lost. Extensive parameter studies were conducted to identify optimal tracking performance and examine the potential limitations. The results indicate the RTDT-Bolt method can greatly enhance the tracking performance of bolt rotation, which can achieve over 90% accuracy using the recommended range for the parameters.
Route Optimization via Environment-Aware Deep Network and Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Vehicle mobility optimization in urban areas is a long-standing problem in smart city and spatial data analysis. Given the complex urban scenario and unpredictable social events, our work focuses on developing a mobile sequential recommendation system to maximize the profitability of vehicle service providers (e.g., taxi drivers). In particular, we treat the dynamic route optimization problem as a long-term sequential decision-making task. A reinforcement-learning framework is proposed to tackle this problem, by integrating a self-check mechanism and a deep neural network for customer pick-up point monitoring. To account for unexpected situations (e.g., the COVID-19 outbreak), our method is designed to be capable of handling related environment changes with a self-adaptive parameter determination mechanism. Based on the yellow taxi data in New York City and vicinity before and after the COVID-19 outbreak, we have conducted comprehensive experiments to evaluate the effectiveness of our method. The results show consistently excellent performance, from hourly to weekly measures, to support the superiority of our method over the state-of-the-art methods (i.e., with more than 98% improvement in terms of the profitability for taxi drivers).
Smart Data Representations: Impact on the Accuracy of Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Deep Neural Networks are able to solve many complex tasks with less engineering effort and better performance. However, these networks often use data for training and evaluation without investigating its representation, i.e.~the form of the used data. In the present paper, we analyze the impact of data representations on the performance of Deep Neural Networks using energy time series forecasting. Based on an overview of exemplary data representations, we select four exemplary data representations and evaluate them using two different Deep Neural Network architectures and three forecasting horizons on real-world energy time series. The results show that, depending on the forecast horizon, the same data representations can have a positive or negative impact on the accuracy of Deep Neural Networks.
Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks
Venue: arXiv cs.MA 2021
Year: 2021

When an individual's behavior has rational characteristics, this may lead to irrational collective actions for the group. A wide range of organisms from animals to humans often evolve the social attribute of cooperation to meet this challenge. Therefore, cooperation among individuals is of great significance for allowing social organisms to adapt to changes in the natural environment. Based on multi-agent reinforcement learning, we propose a new learning strategy for achieving coordination by incorporating a learning rate that can balance exploration and exploitation. We demonstrate that agents that use the simple strategy improve a relatively collective return in a decision task called the intertemporal social dilemma, where the conflict between the individual and the group is particularly sharp. We also explore the effects of the diversity of learning rates on the population of reinforcement learning agents and show that agents trained in heterogeneous populations develop particularly coordinated policies relative to those trained in homogeneous populations.
Oil and Gas Pipeline Monitoring during COVID-19 Pandemic via Unmanned Aerial Vehicle
Venue: arXiv cs.CV 2021
Year: 2021

The vast network of oil and gas transmission pipelines requires periodic monitoring for maintenance and hazard inspection to avoid equipment failure and potential accidents. The severe COVID-19 pandemic situation forced the companies to shrink the size of their teams. One risk which is faced on-site is represented by the uncontrolled release of flammable oil and gas. Among many inspection methods, the unmanned aerial vehicle system contains flexibility and stability. Unmanned aerial vehicles can transfer data in real-time, while they are doing their monitoring tasks. The current article focuses on unmanned aerial vehicles equipped with optical sensing and artificial intelligence, especially image recognition with deep learning techniques for pipeline surveillance. Unmanned aerial vehicles can be used for regular patrolling duties to identify and capture images and videos of the area of interest. Places that are hard to reach will be accessed faster, cheaper and with less risk. The current paper is based on the idea of capturing video and images of drone-based inspections, which can discover several potential hazardous problems before they become dangerous. Damage can emerge as a weakening of the cladding on the external pipe insulation. There can also be the case when the thickness of piping through external corrosion can occur. The paper describes a survey completed by experts from the oil and gas industry done for finding the functional and non-functional requirements of the proposed system.
Estimation of the thermal properties of an historic building wall by combining Modal Identification Method and Optimal Experiment Design
Venue: arXiv cs.CE 2021
Year: 2021

The estimation of wall thermal properties by \emph{in situ} measurement enables to increase the reliability of the model predictions for building energy efficiency. Nevertheless, retrieving the unknown parameters has an important computational cost. Indeed, several computations of the heat transfer problem are required to identify these thermal properties. To handle this drawback, an innovative approach is investigated. The first step is to search the optimal experiment design among the sequence of observation of several months. A reduced sequence of observations of three days is identified which guarantees to estimate the parameter with the maximum accuracy. Moreover, the inverse problem is only solved for this short sequence. To decrease further the computational efforts, a reduced order model based on the modal identification method is employed. This \emph{a posteriori} model reduction method approximates the solution with a lower degree of freedom. The whole methodology is illustrated to estimate the thermal diffusivity of an historical building that has been monitored with temperature sensors for several months. The computational efforts is cut by five. The estimated parameter improves the reliability of the predictions of the wall thermal efficiency.
Accelerating JPEG Decompression on GPUs
Venue: arXiv cs.DC 2021
Year: 2021

The JPEG compression format has been the standard for lossy image compression for over multiple decades, offering high compression rates at minor perceptual loss in image quality. For GPU-accelerated computer vision and deep learning tasks, such as the training of image classification models, efficient JPEG decoding is essential due to limitations in memory bandwidth. As many decoder implementations are CPU-based, decoded image data has to be transferred to accelerators like GPUs via interconnects such as PCI-E, implying decreased throughput rates. JPEG decoding therefore represents a considerable bottleneck in these pipelines. In contrast, efficiency could be vastly increased by utilizing a GPU-accelerated decoder. In this case, only compressed data needs to be transferred, as decoding will be handled by the accelerators. In order to design such a GPU-based decoder, the respective algorithms must be parallelized on a fine-grained level. However, parallel decoding of individual JPEG files represents a complex task. In this paper, we present an efficient method for JPEG image decompression on GPUs, which implements an important subset of the JPEG standard. The proposed algorithm evaluates codeword locations at arbitrary positions in the bitstream, thereby enabling parallel decompression of independent chunks. Our performance evaluation shows that on an A100 (V100) GPU our implementation can outperform the state-of-the-art implementations libjpeg-turbo (CPU) and nvJPEG (GPU) by a factor of up to 51 (34) and 8.0 (5.7). Furthermore, it achieves a speedup of up to 3.4 over nvJPEG accelerated with the dedicated hardware JPEG decoder on an A100.
Sum-of-Squares Lower Bounds for Sparse Independent Set
Venue: arXiv cs.CC 2021
Year: 2021

The Sum-of-Squares (SoS) hierarchy of semidefinite programs is a powerful algorithmic paradigm which captures state-of-the-art algorithmic guarantees for a wide array of problems. In the average case setting, SoS lower bounds provide strong evidence of algorithmic hardness or information-computation gaps. Prior to this work, SoS lower bounds have been obtained for problems in the "dense" input regime, where the input is a collection of independent Rademacher or Gaussian random variables, while the sparse regime has remained out of reach. We make the first progress in this direction by obtaining strong SoS lower bounds for the problem of Independent Set on sparse random graphs. We prove that with high probability over an Erdos-Renyi random graph $G\sim G_{n,\frac{d}{n}}$ with average degree $d>\log^2 n$, degree-$D_{SoS}$ SoS fails to refute the existence of an independent set of size $k = Ω\left(\frac{n}{\sqrt{d}(\log n)(D_{SoS})^{c_0}} \right)$ in $G$ (where $c_0$ is an absolute constant), whereas the true size of the largest independent set in $G$ is $O\left(\frac{n\log d}{d}\right)$. Our proof involves several significant extensions of the techniques used for proving SoS lower bounds in the dense setting. Previous lower bounds are based on the pseudo-calibration heuristic of Barak et al [FOCS 2016] which produces a candidate SoS solution using a planted distribution indistinguishable from the input distribution via low-degree tests. In the sparse case the natural planted distribution does admit low-degree distinguishers, and we show how to adapt the pseudo-calibration heuristic to overcome this. Another notorious technical challenge for the sparse regime is the quest for matrix norm bounds. In this paper, we obtain new norm bounds for graph matrices in the sparse setting.
Facial Information Analysis Technology for Gender and Age Estimation
Venue: arXiv cs.CV 2021
Year: 2021

This is a study on facial information analysis technology for estimating gender and age, and poses are estimated using a transformation relationship matrix between the camera coordinate system and the world coordinate system for estimating the pose of a face image. Gender classification was relatively simple compared to age estimation, and age estimation was made possible using deep learning-based facial recognition technology. A comparative CNN was proposed to calculate the experimental results using the purchased database and the public database, and deep learning-based gender classification and age estimation performed at a significant level and was more robust to environmental changes compared to the existing machine learning techniques.
BigFoot: Exploiting and Mitigating Leakage in Encrypted Write-Ahead Logs
Venue: arXiv cs.CR 2021
Year: 2021

Modern databases and data-warehousing systems separate query processing and durable storage. Storage systems have idiosyncratic bugs and security vulnerabilities, thus attacks that compromise only storage are a realistic threat. In this paper, we show that encryption alone is not sufficient to protect databases from compromised storage. Using MongoDB WiredTiger as a concrete example, we demonstrate that sizes of encrypted writes to a durable write-ahead log can reveal sensitive information about the inputs and activities of MongoDB applications. We then design, implement, and evaluate BigFoot, a WAL modification that mitigates size leakage.
Automated PII Extraction from Social Media for Raising Privacy Awareness: A Deep Transfer Learning Approach
Venue: arXiv cs.SI 2021
Year: 2021

Internet users have been exposing an increasing amount of Personally Identifiable Information (PII) on social media. Such exposed PII can cause severe losses to the users, and informing users of their PII exposure is crucial to raise their privacy awareness and encourage them to take protective measures. To this end, advanced automatic techniques are needed. While Information Extraction (IE) techniques can be used to extract the PII automatically, Deep Learning (DL)-based IE models alleviate the need for feature engineering and further improve the efficiency. However, DL-based IE models often require large-scale labeled data for training, but PII-labeled social media posts are difficult to obtain due to privacy concerns. Also, these models rely heavily on pre-trained word embeddings, while PII in social media often varies in forms and thus has no fixed representations in pre-trained word embeddings. In this study, we propose the Deep Transfer Learning for PII Extraction (DTL-PIIE) framework to address these two limitations. DTL-PIIE transfers knowledge learned from publicly available PII data to social media to address the problem of rare PII-labeled data. Moreover, our framework leverages Graph Convolutional Networks (GCNs) to incorporate syntactic patterns to guide PIIE without relying on pre-trained word embeddings. Evaluation against benchmark IE models indicates that our approach outperforms state-of-the-art DL-based IE models. Our framework can facilitate various applications, such as PII misuse prediction and privacy risk assessment, protecting the privacy of internet users.
Low-to-Zero-Overhead IRS Reconfiguration: Decoupling Illumination and Channel Estimation
Venue: arXiv cs.NI 2021
Year: 2021

Most algorithms developed so far for the optimization of Intelligent Reflecting Surfaces (IRSs) require knowledge of full Channel State Information (CSI). However, the resulting acquisition overhead constitutes a major bottleneck for the realization of IRS-assisted wireless systems in practice. In contrast, in this paper, focusing on downlink transmissions from a Base Station (BS) to a Mobile User (MU) that is located in a blockage region, we propose to optimize the IRS for illumination of the area centered around the MU. Hence, the proposed design requires the estimation of the MU's position and not the full CSI. For a given IRS phase-shift configuration, the end-to-end BS-IRS-MU channel can then be estimated using conventional channel estimation techniques. The IRS reconfiguration overhead for the proposed scheme depends on the MU mobility as well as how wide the coverage of the IRS illumination is. Therefore, we develop a general IRS phase-shift design, which is valid for both the near- and far-field regimes and features a parameter for tuning the size of the illumination area. Moreover, we study a special case where the IRS illuminates the entire blockage area, which implies that the IRS phase shifts do not change over time leading to zero overhead for IRS reconfiguration.
Design of a Scalable 4G Portable Network Using Low Cost SDR And Raspberry Pi
Venue: arXiv cs.NI 2021
Year: 2021

Of late, Software Defined Radio (SDR) approach has become an effective means to design high data rate wireless systems for a range of applications. There are methods with which low cost SDR based 4th generation (4G) or long term evolution (LTE) systems can be designed. Using low cost Raspberry Pi systems, the SDR aided 4G systems can be designed for high data rate communication. The work is related to the design of a 4G wireless system using low cost SDR solutions and integrated to a programmable controller based on a Raspberry Pi. Experimental results show that the system is effective in a range of conditions.
L4-Norm Weight Adjustments for Converted Spiking Neural Networks
Venue: arXiv cs.NE 2021
Year: 2021

Spiking Neural Networks (SNNs) are being explored for their potential energy efficiency benefits due to sparse, event-driven computation. Non-spiking artificial neural networks are typically trained with stochastic gradient descent using backpropagation. The calculation of true gradients for backpropagation in spiking neural networks is impeded by the non-differentiable firing events of spiking neurons. On the other hand, using approximate gradients is effective, but computationally expensive over many time steps. One common technique, then, for training a spiking neural network is to train a topologically-equivalent non-spiking network, and then convert it to an spiking network, replacing real-valued inputs with proportionally rate-encoded Poisson spike trains. Converted SNNs function sufficiently well because the mean pre-firing membrane potential of a spiking neuron is proportional to the dot product of the input rate vector and the neuron weight vector, similar to the functionality of a non-spiking network. However, this conversion only considers the mean and not the temporal variance of the membrane potential. As the standard deviation of the pre-firing membrane potential is proportional to the L4-norm of the neuron weight vector, we propose a weight adjustment based on the L4-norm during the conversion process in order to improve classification accuracy of the converted network.