pwc-archive/papers-with-abstracts huggingface dataset:
Real-time Implementation of RMNv2 Classifier in NXP Bluebox 2.0 and NXP i.MX RT1060
Authors: Maneesh Ayi, Mohamed El-Sharkawy
Date: 2020-09-30 00:00:00
Tasks: Image Classification, object-detection, Object Detection

With regards to Advanced Driver Assistance Systems in vehicles, vision and image-based ADAS is profoundly well known since it utilizes Computer vision algorithms, for example, object detection, street sign identification, vehicle control, impact cautioning, and so on., to aid sheltered and smart driving. Deploying these algorithms directly in resource-constrained devices like mobile and embedded devices etc. is not possible. Reduced Mobilenet V2 (RMNv2) is one of those models which is specifically designed for deploying easily in embedded and mobile devices. In this paper, we implemented a real-time RMNv2 image classifier in NXP Bluebox 2.0 and NXP i.MX RT1060. Because of its low model size of 4.3MB, it is very successful to implement this model in those devices. The model is trained and tested with the CIFAR10 dataset.
Vicomtech at eHealth-KD Challenge 2020: Deep End-to-End Model for Entity and Relation Extraction in Medical Text
Authors: Aitor García-Pablos, Naiara Perez, Montse Cuadros and Elena Zotova
Date: 2020-09-20 00:00:00
Tasks: Medical Diagnosis, Medical Procedure, Multi-Label Classification Of Biomedical Texts, Relation Extraction

This paper describes the participation of the Vicomtech NLP team in the eHealth-KD 2020 shared task about
detecting and classifying entities and relations in health-related texts written in Spanish. The proposed system
consists of a single end-to-end deep neural network with pre-trained BERT models as the core for the semantic
representation of the input texts. We have experimented with two models: BERT-Base Multilingual Cased and
BETO, a BERT model pre-trained on Spanish text. Our system models all the output variables—entities and
relations—at the same time, modelling the whole problem jointly. Some of the outputs are fed back to latter
layers of the model, connecting the outcomes of the different subtasks in a pipeline fashion. Our system shows
robust results in all the scenarios of the task. It has achieved the first position in the main scenario of the
competition and top-3 in the rest of the scenarios.
hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices
Authors: Farah Fahim, Benjamin Hawks, Christian Herwig, James Hirschauer, Sergo Jindariani, Nhan Tran, Luca P. Carloni, Giuseppe Di Guglielmo, Philip Harris, Jeffrey Krupa, Dylan Rankin, Manuel Blanco Valentin, Josiah Hester, Yingyi Luo, John Mamish, Seda Orgrenci-Memik, Thea Aarrestad, Hamza Javed, Vladimir Loncar, Maurizio Pierini, Adrian Alan Pol, Sioni Summers, Javier Duarte, Scott Hauck, Shih-Chieh Hsu, Jennifer Ngadiuba, Mia Liu, Duc Hoang, Edward Kreinar, Zhenbin Wu
Date: 2021-03-09 00:00:00
Tasks: BIG-bench Machine Learning, Diagnostic, Experimental Design, Quantization

Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. To support domain scientists, we have developed hls4ml, an open-source software-hardware codesign workflow to interpret and translate machine learning algorithms for implementation with both FPGA and ASIC technologies. We expand on previous hls4ml work by extending capabilities and techniques towards low-power implementations and increased usability: new Python APIs, quantization-aware pruning, end-to-end FPGA workflows, long pipeline kernels for low power, and new device backends include an ASIC workflow. Taken together, these and continued efforts in hls4ml will arm a new generation of domain scientists with accessible, efficient, and powerful tools for machine-learning-accelerated discovery.
Hyperspectral Image Classification Method Based on 2D–3D CNN and Multibranch Feature Fusion
Authors: Zixian Ge, Guo Cao, Xuesong Li, Peng Fu
Date: 2020-09-18 00:00:00
Tasks: Classification, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

The emergence of a convolutional neural network (CNN) has greatly promoted the development of hyperspectral image (HSI) classification technology. However, the acquisition of HSI is difficult. The lack of training samples is the primary cause of low classification performance. The traditional CNN-based methods mainly use the 2-D CNN for feature extraction, which makes the interband correlations of HSIs underutilized. The 3-D CNN extracts the joint spectral-spatial information representation, but it depends on a more complex model. Also, too deep or too shallow network cannot extract the image features well. To tackle these issues, we propose an HSI classification method based on the 2D-3D CNN and multibranch feature fusion. We first combine 2-D CNN and 3-D CNN to extract image features. Then, by means of the multibranch neural network, three kinds of features from shallow to deep are extracted and fused in the spectral dimension. Finally, the fused features are passed into several fully connected layers and a softmax layer to obtain the classification results. In addition, our network model utilizes the state-of-the-art activation function Mish to further improve the classification performance. Our experimental results, conducted on four widely used HSI datasets, indicate that the proposed method achieves better performance than the existing alternatives.
Low-level cognitive skill transfer between two individuals' minds via computer game-based framework
Authors: Ahmet Orun
Date: 2021-03-03 00:00:00

The novel technique introduced here aims to accomplish the first stage of transferring low-level cognitive skills between two individuals (e.g. from expert to learner) to ease the consecutive higher level declarative learning process for the target "learner" individual in a game environment. Such low-level cognitive skill is associated with the procedural knowledge and established at low-level of mind which can be unveiled and transferred by only a novel technique (rather than by a traditional educational environment ) like a highly interactive computer game domain in which a user exposes his/her unconscious mind behaviors via the game-hero non-deliberately during the game sessions. The cognitive data exposed by the game-hero would be recorded, and then be modelled by the artificial intelligence technique like Bayesian networks for an early stage of cognitive skill transfer and the cognitive stimuli are also generated to be used as game agents to train the learner.
R-MnasNet: Reduced MnasNet for Computer Vision
Authors: Prasham Shah, Mohamed El-Sharkawy
Date: 2020-10-08 00:00:00
Tasks: Image Classification, object-detection, Object Detection

In Deep Learning, Convolutional Neural Networks (CNNs) are widely used for Computer Vision applications. With the advent of new technology, there is an inevitable necessity for CNNs to be computationally less expensive. It has become a key factor in determining its competence. CNN models must be compact in size and work efficiently when deployed on embedded systems. In order to achieve this goal, researchers have invented new algorithms which make CNNs lightweight yet accurate enough to be used for applications like object detection. In this paper, we have tried to do the same by modifying an architecture to make it compact with a fair trade-off between model size and accuracy. A new architecture, R-MnasNet (Reduced MnasNet), has been introduced which has a model size of 3 MB. It is trained on CIFAR-10 [4] and has a validation accuracy of 91.13%. Whereas the baseline architecture, MnasNet [1], has a model size of 12.7 MB with a validation accuracy of 80.8% when trained with CIFAR-10 dataset. R-MnasNet can be used on resource-constrained devices. It can be deployed on embedded systems for vision applications.
Doubly Contrastive Deep Clustering
Authors: Zhiyuan Dang, Cheng Deng, Xu Yang, Heng Huang
Date: 2021-03-09 00:00:00
Tasks: Clustering, Contrastive Learning, Data Augmentation, Deep Clustering

Deep clustering successfully provides more effective features than conventional ones and thus becomes an important technique in current unsupervised learning. However, most deep clustering methods ignore the vital positive and negative pairs introduced by data augmentation and further the significance of contrastive learning, which leads to suboptimal performance. In this paper, we present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views to obtain more discriminative features and competitive results. Specifically, for the sample view, we set the class distribution of the original sample and its augmented version as positive sample pairs and set one of the other augmented samples as negative sample pairs. After that, we can adopt the sample-wise contrastive loss to pull positive sample pairs together and push negative sample pairs apart. Similarly, for the class view, we build the positive and negative pairs from the sample distribution of the class. In this way, two contrastive losses successfully constrain the clustering results of mini-batch samples in both sample and class level. Extensive experimental results on six benchmark datasets demonstrate the superiority of our proposed model against state-of-the-art methods. Particularly in the challenging dataset Tiny-ImageNet, our method leads 5.6\% against the latest comparison method. Our code will be available at \url{https://github.com/ZhiyuanDang/DCDC}.
Satellite Image Small Target Application Based on Deep Segmented Residual Neural Network
Authors: Zikang Wei, Yunqing Liu
Date: 2020-10-26 00:00:00
Tasks: satellite image super-resolution, SSIM, Super-Resolution

This study employs a deep segmented residual neural network model to analyze the super-resolution of a single satellite image. A deep convolutional neural network model was analyzed, and its performance was improved. We proposed two residual layers to divide the deep network into two groups, the sum of the two residuals is the total residual, which can minimize the residual loss function and enhance the network performance. The experimental model achieved high peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) than networks without the proposed improvements when tested on satellite images. Considering these results, the application of this technology will be significant for further research on satellite images.
DeepWay: A Deep Learning Estimator for Unmanned Ground Vehicle Global Path Planning
Authors: Vittorio Mazzia, Francesco Salvetti, Diego Aghi, Marcello Chiaberge
Date: 2020-10-30 00:00:00

Agriculture 3.0 and 4.0 have gradually introduced service robotics and automation into several agricultural processes, mostly improving crops quality and seasonal yield. Row-based crops are the perfect settings to test and deploy smart machines capable of monitoring and manage the harvest. In this context, global path generation is essential either for ground or aerial vehicles, and it is the starting point for every type of mission plan. Nevertheless, little attention has been currently given to this problem by the research community and global path generation automation is still far to be solved. In order to generate a viable path for an autonomous machine, the presented research proposes a feature learning fully convolutional model capable of estimating waypoints given an occupancy grid map. In particular, we apply the proposed data-driven methodology to the specific case of row-based crops with the general objective to generate a global path able to cover the extension of the crop completely. Extensive experimentation with a custom made synthetic dataset and real satellite-derived images of different scenarios have proved the effectiveness of our methodology and demonstrated the feasibility of an end-to-end and completely autonomous global path planner.
Exploring Multimodal Features and Fusion Strategies for Analyzing Disaster Tweets
Authors: Raj Ratn Pranesh, Ambesh Shekhar, Anish Kumar
Date: 2020-11-06 00:00:00
Tasks: Multimodal Deep Learning, Transfer Learning

Social media platforms such as Twitter often provide firsthand news during the outbreak of a crisis. It is extremely
essential to process these facts quickly to plan the response
efforts for minimal loss. Processing this social media information poses multiple challenges such as parsing noisy messages containing both texts and images. Furthermore, these
messages are diverse, from personal achievements and opinions to situational crises.
Therefore, in this paper, we present an analysis of various
multimodal feature fusion techniques to analyze and classify the disaster tweets into multiple crisis events via transfer learning. In our study, we utilized three image modelsVGG19(Simonyan and Zisserman 2014), ResNet-50(He
et al. 2016) and AlexNet pre-trained on ImageNet(Deng
et al. 2009) dataset and three fine-tuned language modelsBERT(Devlin et al. 2018), ALBERT(Lan et al. 2019) and
RoBERTa(Liu et al. 2019) to learn the visual and textual
feature of the data and combine them to make predictions.
We have presented a systematic analysis of multiple intramodal as well as cross-modal fusion strategies and their effect over the performance of the multimodal disaster classification system. In our experiment, we used 8,242 disaster tweets each consisting of image and text data with five
disaster event classes. The results show that the multimodal
with transformer-attention mechanism and factorized bilinear pooling (FBP)(Zhang, Wang, and Du 2019) for intramodal and cross-modal feature fusion achieved the best performance.
TweetBLM: A Hate Speech Dataset and Analysis of Black Lives Matter-related Microblogs on Twitter:
Authors: Sumit Kumar, Raj Ratn Pranesh, Subhash Chandra Pandey
Date: 2020-06-25 00:00:00
Tasks: Hate Speech Detection

In the past few years, there has been a significant rise in toxic and hateful content on various social media platforms. Recently Black Lives Matter
movement came into the picture again causing an
avalanche of user-generated response on the internet. In this paper, we have proposed a Black Lives
Matter related tweet hate speech dataset- TweetBLM. Our dataset is consists of 9165 manually
annotated tweets that target the Black Lives Matter movement. The tweets were annotated into
two classes, i.e, ”HATE” and ”NON-HATE” on
the basis of their content related to racism erupted
from the movement. In this work, we also generated useful insights on our dataset and performed a
systematic analysis of various state-of-the-art models such as LSTM, Bi-LSTM, Fasttext, BERTbase
and BERTlarge for the classification task on our
dataset. Through our work, we aim at contributing
to the substantial efforts of the research community
for identification and mitigation of hate speech on
the internet.
Creating cloud-free satellite imagery from image time series with deep learning
Authors: Stefan Oehmcke, Tzu Hsin Karen Chen, Alexander V Prishchepov, Fabian Gieseke
Date: 2020-11-01 00:00:00
Tasks: Image Generation, Missing Values, Time Series, Time Series Analysis

Optical satellite images are important for environmental monitoring. Unfortunately, such images are often affected by distortions, such as clouds, shadows, or missing data. This work proposes a deep learning approach for cleaning and imputing satellite images, which could serve as a reliable preprocessing step for spatial and spatio-temporal analyzes. More specifically, a coherent and cloud-free image for a specific target date and region is created based on a sequence of images of that region obtained at previous dates. Our model first extracts information from the previous time steps via a special gating function and then resorts to a modified version of the well-known U-Net architecture to obtain the desired output image. The model uses supplementary data, namely the approximate cloud coverage of input images, the temporal distance to the target time, and a missing data mask for each input time step. During the training phase we condition our model with the targets cloud coverage and missing values (disabled in production), which allows us to use data afflicted by distortion during training and thus does not require pre-selection of distortion-free data. Our experimental evaluation, conducted on data of the Landsat missions, shows that our approach outperforms the commonly utilized approach that resorts to taking the median of cloud-free pixels for a given position. This is especially the case when the quality of the data for the considered period is poor (e.g., lack of cloud free-images during the winter/fall periods). Our deep learning approach allows to improve the utility of the entire Landsat archive, the only existing global medium-resolution free-access satellite archive dating back to the 1970s. It therefore holds scientific and societal potential for future analyses conducted on data from this and other satellite imagery repositories.
Aspect Based Sentiment Analysis with Self-Attention and Gated Convolutional Networks
Authors: Jian Yang, Juan Yang
Date: 2020-11-04 00:00:00
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Aspect Category Sentiment Analysis, Sentence, Sentiment Analysis

Aspect based sentiment analysis (ABSA) is a fine-grained sentiment analysis task, whose main goal is to identify the sentiment polarity of an aspect in a sentence. A sentence may contain many different aspects, each of which may have different sentiment polarities. Based on the current researches in this area, ABSA can be divided into two subtasks: aspect-category sentiment analysis (ACSA) and aspect-term sentiment analysis (ATSA). In the past, more commonly used method is to adopt the time serial algorithm such as Long Short-Term Memory (LSTM) or Recurrent Neural Network (RNN), which usually needs more training time and has complex structures. Moreover, many previous models lack the abilities to effectively learn the internal structure features of sentences. For ABSA, sometimes the sentence structure may significantly affect the final classification results. However, we found the excellent performance of self-attention algorithm and gating mechanism in some other related researches. Therefore, to solve the problems above, we build a new model based on gating mechanism, combined with convolutional neural networks (CNN) and self-attention mechanism. First, we use self-attention to extract the structural feature of the input, and integrate it with the features of the original sentence extracted by CNN. On such basis, we further combine the aspect-category or aspect-term of the input sentence to form the final sentiment feature. Experiments on SemEval datasets show the performance of our models and the effectiveness of the model is proved.
A Deep Convolutional Neural Network Applied to Ship Detection and Classification
Authors: Mikael Strauhs, Antonio Rafael Paulino de Lira, Ramiro Fernandes Ramos, João Vitor Marques de Oliveira Moita, Severino Virgínio Martins Neto, Jean-David Caprace
Date: 2020-10-29 00:00:00
Tasks: General Classification, Image Classification, object-detection, Object Detection

The task of detecting a ship is relevant for a variety of applications in both military and civilian fields, from maritime traffic surveillance to sea pollution monitoring. Despite the recent significant attention, deep learning-based object detection algorithms have received, they are still rarely applied in the detection of ships. Expectedly, we see even fewer applications when the goal is to detect specific ship classes, although those could deliver extra valuable information. In this work, we build a ship detection algorithm capable of distinguishing six common ship types: ore carrier, bulk cargo carrier, general cargo ship, container ship, fishing boat and passenger ship. To achieve this objective, we firstly built a deep convolutional neural network in python, taking advantage of the TensorFlow framework. Secondly, we analyzed our model’s ability to generalize on a new set of images, apart from the training and testing sets. The results showed that our proposed model is close to state-of-the-art performance since it was able to perform well on the test set - mAP = 97.62%. There is still room for improvement on the model robustness, associated primarily with possible training set limitations. In practice, this paper will contribute to the advance of research and applications on ship detection.
Multiple Trajectory Prediction with Deep Temporal and Spatial Convolutional Neural Networks
Authors: Jan Strohbeck, Vasileios Belagiannis, Johannes Muller, Marcel Schreiber, Martin Herrmann, Daniel Wolf and Michael Buchholz
Date: 2020-10-29 00:00:00
Tasks: Autonomous Driving, Motion Forecasting, Navigate, Object Tracking, Position, Trajectory Prediction

Automated vehicles need to not only perceive their
environment, but also predict the possible future behavior of
all detected traffic participants in order to safely navigate
in complex scenarios and avoid critical situations, ranging
from merging on highways to crossing urban intersections.
Due to the availability of datasets with large numbers of
recorded trajectories of traffic participants, deep learning based
approaches can be used to model the behavior of road users.
This paper proposes a convolutional network that operates on
rasterized actor-centric images which encode the static and
dynamic actor-environment. We predict multiple possible future
trajectories for each traffic actor, which include position, velocity, acceleration, orientation, yaw rate and position uncertainty
estimates. To make better use of the past movement of the
actor, we propose to employ temporal convolutional networks
(TCNs) and rely on uncertainties estimated from the previous
object tracking stage. We evaluate our approach on the public
“Argoverse Motion Forecasting” dataset, on which it won the
first prize at the Argoverse Motion Forecasting Challenge,
as presented on the NeurIPS 2019 workshop on “Machine
Learning for Autonomous Driving”.
A comparative approach on detecting multi-lingual and multi-oriented text in natural scene images
Authors: Aparna Yegnaraman, S. Valli
Date: 2020-11-17 00:00:00
Tasks: Binarization, Segmentation, Text Detection, Vocal Bursts Type Prediction

Text helps to convey the intended message to users very accurately. Detecting text from natural scene images for quadrilateral-type and polygon-type datasets is the primary scope of this work. A regression-based method using modified You Only Look Once YOLOv4 network is used for quadrilateral-type datasets. Hyperparameters for training the network are optimized using the Genetic Algorithm which proves to be a suitable candidate than traditional methods. The Pixels-IoU (PIoU) loss is introduced to derive an accurate bounding box and it seems to be productive under various challenging scenarios with high aspect ratios and complex background. This yielded quick results for quadrilateral-type datasets but did not scale for arbitrarily-shaped and curved scene text. So the approach is changed to segmentation based for enhancing the results. This introduces binarization operation in a segmentation network to boost its detection accuracy for polygon-type datasets. The introduction of a new module DiffBiSeg (Differentiable Binarization in Segmentation network) facilitates post-processing and text detection performance by setting the thresholds flexibly for binarization in the segmentation network. The efficacy of both approaches is clearly seen in their respective experimental results.