pwc-archive/papers-with-abstracts huggingface dataset:
A Masked Bounding-Box Selection Based ResNet Predictor for Text Rotation Prediction
Authors: Michael Yang, Yuan Lin, Chiuman Ho
Date: 2022-09-06 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

The existing Optical Character Recognition (OCR) systems are capable of recognizing images with horizontal texts. However, when the rotation of the texts increases, it becomes harder to recognizing these texts. The performance of the OCR systems decreases. Thus predicting the rotations of the texts and correcting the images are important. Previous work mainly uses traditional Computer Vision methods like Hough Transform and Deep Learning methods like Convolutional Neural Network. However, all of these methods are prone to background noises commonly existing in general images with texts. To tackle this problem, in this work, we introduce a new masked bounding-box selection method, that incorporating the bounding box information into the system. By training a ResNet predictor to focus on the bounding box as the region of interest (ROI), the predictor learns to overlook the background noises. Evaluations on the text rotation prediction tasks show that our method improves the performance by a large margin.
Generating detailed saliency maps using model-agnostic methods
Authors: Maciej Sakowicz
Date: 2022-09-04 00:00:00
Tasks: Decision Making, Explainable artificial intelligence, Informativeness

The emerging field of Explainable Artificial Intelligence focuses on researching methods of explaining the decision making processes of complex machine learning models. In the field of explainability for Computer Vision, explanations are provided as saliency maps, which visualize the importance of individual pixels of the input w.r.t. the model's prediction. In this work we focus on a perturbation-based, model-agnostic explainability method called RISE, elaborate on observed shortcomings of its grid-based approach and propose two modifications: replacement of square occlusions with convex polygonal occlusions based on cells of a Voronoi mesh and addition of an informativeness guarantee to the occlusion mask generator. These modifications, collectively called VRISE (Voronoi-RISE), are meant to, respectively, improve the accuracy of maps generated using large occlusions and accelerate convergence of saliency maps in cases where sampling density is either very low or very high. We perform a quantitative comparison of accuracy of saliency maps produced by VRISE and RISE on the validation split of ILSVRC2012, using a saliency-guided content insertion/deletion metric and a localization metric based on bounding boxes. Additionally, we explore the space of configurable occlusion pattern parameters to better understand their influence on saliency maps produced by RISE and VRISE. We also describe and demonstrate two effects observed over the course of experimentation, arising from the random sampling approach of RISE: "feature slicing" and "saliency misattribution". Our results show that convex polygonal occlusions yield more accurate maps for coarse occlusion meshes and multi-object images, but improvement is not guaranteed in other cases. The informativeness guarantee is shown to increase the convergence rate without incurring a significant computational overhead.
Table Detection in the Wild: A Novel Diverse Table Detection Dataset and Method
Authors: Mrinal Haloi, Shashank Shekhar, Nikhil Fande, Siddhant Swaroop Dash, Sanjay G
Date: 2022-08-31 00:00:00
Tasks: Deep Learning, Diversity, Table Detection

Recent deep learning approaches in table detection achieved outstanding performance and proved to be effective in identifying document layouts. Currently, available table detection benchmarks have many limitations, including the lack of samples diversity, simple table structure, the lack of training cases, and samples quality. In this paper, we introduce a diverse large-scale dataset for table detection with more than seven thousand samples containing a wide variety of table structures collected from many diverse sources. In addition to that, we also present baseline results using a convolutional neural network-based method to detect table structure in documents. Experimental results show the superiority of applying convolutional deep learning methods over classical computer vision-based methods. The introduction of this diverse table detection dataset will enable the community to develop high throughput deep learning methods for understanding document layout and tabular data processing. Dataset is available at: 1. https://www.kaggle.com/datasets/mrinalim/stdw-dataset 2. https://huggingface.co/datasets/n3011/STDW
Discriminative Sampling of Proposals in Self-Supervised Transformers for Weakly Supervised Object Localization
Authors: Shakeeb Murtaza, Soufiane Belharbi, Marco Pedersoli, Aydin Sarraf, Eric Granger
Date: 2022-09-09 00:00:00
Tasks: Object, Object Localization, Weakly-Supervised Object Localization

Drones are employed in a growing number of visual recognition applications. A recent development in cell tower inspection is drone-based asset surveillance, where the autonomous flight of a drone is guided by localizing objects of interest in successive aerial images. In this paper, we propose a method to train deep weakly-supervised object localization (WSOL) models based only on image-class labels to locate object with high confidence. To train our localizer, pseudo labels are efficiently harvested from a self-supervised vision transformers (SSTs). However, since SSTs decompose the scene into multiple maps containing various object parts, and do not rely on any explicit supervisory signal, they cannot distinguish between the object of interest and other objects, as required WSOL. To address this issue, we propose leveraging the multiple maps generated by the different transformer heads to acquire pseudo-labels for training a deep WSOL model. In particular, a new Discriminative Proposals Sampling (DiPS) method is introduced that relies on a CNN classifier to identify discriminative regions. Then, foreground and background pixels are sampled from these regions in order to train a WSOL model for generating activation maps that can accurately localize objects belonging to a specific class. Empirical results on the challenging TelDrone dataset indicate that our proposed approach can outperform state-of-art methods over a wide range of threshold values over produced maps. We also computed results on CUB dataset, showing that our method can be adapted for other tasks.
Use Classifier as Generator
Authors: Haoyang Li
Date: 2022-09-10 00:00:00
Tasks: Image Generation

Image recognition/classification is a widely studied problem, but its reverse problem, image generation, has drawn much less attention until recently. But the vast majority of current methods for image generation require training/retraining a classifier and/or a generator with certain constraints, which can be hard to achieve. In this paper, we propose a simple approach to directly use a normally trained classifier to generate images. We evaluate our method on MNIST and show that it produces recognizable results for human eyes with limited quality with experiments.
Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings
Authors: Yiren Jian, Chongyang Gao, Soroush Vosoughi
Date: 2022-09-20 00:00:00
Tasks: Clustering, Contrastive Learning, Representation Learning, Semantic Textual Similarity, Sentence, Sentence Embeddings

Semantic representation learning for sentences is an important and well-studied problem in NLP. The current trend for this task involves training a Transformer-based sentence encoder through a contrastive objective with text, i.e., clustering sentences with semantically similar meanings and scattering others. In this work, we find the performance of Transformer models as sentence encoders can be improved by training with multi-modal multi-task losses, using unpaired examples from another modality (e.g., sentences and unrelated image/audio data). In particular, besides learning by the contrastive loss on text, our model clusters examples from a non-linguistic domain (e.g., visual/audio) with a similar contrastive loss at the same time. The reliance of our framework on unpaired non-linguistic data makes it language-agnostic, enabling it to be widely applicable beyond English NLP. Experiments on 7 semantic textual similarity benchmarks reveal that models trained with the additional non-linguistic (images/audio) contrastive objective lead to higher quality sentence embeddings. This indicates that Transformer models are able to generalize better by doing a similar task (i.e., clustering) with unpaired examples from different modalities in a multi-task fashion.
Deep Spike Learning with Local Classifiers
Authors: Chenxiang Ma, Rui Yan, Zhaofei Yu, Qiang Yu
Date: 2022-07-22 00:00:00
Proceeding: IEEE Transactions on Cybernetics 2022 7

Backpropagation has been successfully generalized to optimize deep spiking neural networks (SNNs), where, nevertheless, gradients need to be propagated back through all layers, resulting in a massive consumption of computing resources and an obstacle to the parallelization of training. A biologically motivated scheme of local learning provides an alternative to efficiently train deep networks but often suffers a low performance of accuracy on practical tasks. Thus, how to train deep SNNs with the local learning scheme to achieve both efficient and accurate performance still remains an important challenge. In this study, we focus on a supervised local learning scheme where each layer is independently optimized with an auxiliary classifier. Accordingly, we first propose a spike-based efficient local learning rule by only considering the direct dependencies in the current time. We then propose two variants that additionally incorporate temporal dependencies through a backward and forward process, respectively. The effectiveness and performance of our proposed methods are extensively evaluated with six mainstream datasets. Experimental results show that our methods can successfully scale up to large networks and substantially outperform the spike-based local learning baselines on all studied benchmarks. Our results also reveal that gradients with temporal dependencies are essential for high performance on temporal tasks, while they have negligible effects on rate-based tasks. Our work is significant as it brings the performance of spike-based local learning to a new level with the computational benefits being retained.
A Real-time Integration of Semantics into Heterogeneous SensorStream Data with Context in the Internet of Things
Authors: Besmir Sejdiu1 a, Florije Ismaili1 b and Lule Ahmedi2 c
Date: 2020-01-01 00:00:00
Proceeding: 2020/01/01 2020 1
Tasks: Management

Recently, billions Internet of Things (IoT) devices, including sensors are producing sensed data continuously
in the stream data, and transmit these data to a centralized server. Due to the dramatically increase of streaming
data, their management and exploitation has become increasingly important and difficult to process and
integrate the semantic to sensor stream data in real-time. This research focuses on real-time integration of
semantics into heterogeneous sensor stream data with context in the IoT. In this context, an IoT real-time air
quality monitoring system and different semantic annotations are developed for sensor stream data in the
format of Sensor Observation Service (SOS).
MFAN: Multi-modal Feature-enhanced Attention Networks for Rumor Detection
Authors: Jiaqi Zheng, Xi Zhang, Sanchuan Guo, Quan Wang, Wenyu Zang, Yongdong Zhang
Date: 2022-07-26 00:00:00
Proceeding: 2022 2022 7

Rumor spreaders are increasingly taking advantage of multimedia content to attract and mislead news consumers on social media. Although recent multimedia rumor detection models have exploited both textual and visual features for classifcation, they do not integrate the social structure features simultaneously, which have shown promising performance for rumor identifcation. It is challenging to combine the heterogeneous multi-modal data in consideration of their complex relationships. In this work, we propose a novel Multi-modal Featureenhanced Attention Networks (MFAN) for rumor detection, which makes the frst attempt to integrate textual, visual, and social graph features in one unifed framework. Specifcally, it considers both the complement and alignment relationships between different modalities to achieve better fusion. Moreover, it takes into account the incomplete links in the social network data due to data collection constraints and proposes to infer hidden links to learn better social graph features. The experimental results show that MFAN can detect rumors effectively and outperform state-of-the-art methods.
A novel obstacle avoidance algorithm：“Follow the Gap Method"
Authors: Volkan Sezer ;Metin Gokasan
Date: 2012-12-01 00:00:00
Proceeding: journal 2012 12

In this paper, a novel obstacle avoidance method is designed and applied to an experimental autonomous ground vehicle system. The proposed method brings a new solution to the problem and has several advantages compared to previous methods. This novel algorithm is easy to tune and it takes into consideration the field of view and the nonholonomic constraints of the robot. Moreover the method does not have a local minimum problem and results in safer trajectories because of its inherent properties in the definition of the algorithm. The proposed algorithm is tested in simulations and after the observation of successful results, experimental tests are performed using static and dynamic obstacle scenarios. The experimental test platform is an autonomous ground vehicle with Ackermann steering geometry which brings nonholonomic constraints to the vehicle. Experimental results show that the task of obstacle avoidance can be achieved using the algorithm on the autonomous vehicle platform. The algorithm is very promising for application in mobile and industrial robotics where obstacle avoidance is a feature of the robotic system.
LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging
Authors: Andy Rosenbaum, Saleh Soltan, Wael Hamza, Yannick Versley, Markus Boese
Date: 2022-09-20 00:00:00
Proceeding: COLING 2022 10
Tasks: intent-classification, Intent Classification, Language Modeling, Language Modelling, Machine Translation, Translation

We present LINGUIST, a method for generating annotated data for Intent Classification and Slot Tagging (IC+ST), via fine-tuning AlexaTM 5B, a 5-billion-parameter multilingual sequence-to-sequence (seq2seq) model, on a flexible instruction prompt. In a 10-shot novel intent setting for the SNIPS dataset, LINGUIST surpasses state-of-the-art approaches (Back-Translation and Example Extrapolation) by a wide margin, showing absolute improvement for the target intents of +1.9 points on IC Recall and +2.5 points on ST F1 Score. In the zero-shot cross-lingual setting of the mATIS++ dataset, LINGUIST out-performs a strong baseline of Machine Translation with Slot Alignment by +4.14 points absolute on ST F1 Score across 6 languages, while matching performance on IC. Finally, we verify our results on an internal large-scale multilingual dataset for conversational agent IC+ST and show significant improvements over a baseline which uses Back-Translation, Paraphrasing and Slot Catalog Resampling. To our knowledge, we are the first to demonstrate instruction fine-tuning of a large-scale seq2seq model to control the outputs of multilingual intent- and slot-labeled data generation.
Soft Action Priors: Towards Robust Policy Transfer
Authors: Matheus Centa, Philippe Preux
Date: 2022-09-20 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Despite success in many challenging problems, reinforcement learning (RL) is still confronted with sample inefficiency, which can be mitigated by introducing prior knowledge to agents. However, many transfer techniques in reinforcement learning make the limiting assumption that the teacher is an expert. In this paper, we use the action prior from the Reinforcement Learning as Inference framework - that is, a distribution over actions at each state which resembles a teacher policy, rather than a Bayesian prior - to recover state-of-the-art policy distillation techniques. Then, we propose a class of adaptive methods that can robustly exploit action priors by combining reward shaping and auxiliary regularization losses. In contrast to prior work, we develop algorithms for leveraging suboptimal action priors that may nevertheless impart valuable knowledge - which we call soft action priors. The proposed algorithms adapt by adjusting the strength of teacher feedback according to an estimate of the teacher's usefulness in each state. We perform tabular experiments, which show that the proposed methods achieve state-of-the-art performance, surpassing it when learning from suboptimal priors. Finally, we demonstrate the robustness of the adaptive algorithms in continuous action deep RL problems, in which adaptive algorithms considerably improved stability when compared to existing policy distillation methods.
Open-vocabulary Queryable Scene Representations for Real World Planning
Authors: Boyuan Chen, Fei Xia, Brian Ichter, Kanishka Rao, Keerthana Gopalakrishnan, Michael S. Ryoo, Austin Stone, Daniel Kappler
Date: 2022-09-20 00:00:00
Tasks: Task Planning

Large language models (LLMs) have unlocked new capabilities of task planning from human instructions. However, prior attempts to apply LLMs to real-world robotic tasks are limited by the lack of grounding in the surrounding scene. In this paper, we develop NLMap, an open-vocabulary and queryable scene representation to address this problem. NLMap serves as a framework to gather and integrate contextual information into LLM planners, allowing them to see and query available objects in the scene before generating a context-conditioned plan. NLMap first establishes a natural language queryable scene representation with Visual Language models (VLMs). An LLM based object proposal module parses instructions and proposes involved objects to query the scene representation for object availability and location. An LLM planner then plans with such information about the scene. NLMap allows robots to operate without a fixed list of objects nor executable options, enabling real robot operation unachievable by previous methods. Project website: https://nlmap-saycan.github.io
On resolving conflicts between arguments
Authors: Nico Roos
Date: 2022-09-20 00:00:00
Tasks: Negation, valid

Argument systems are based on the idea that one can construct arguments for propositions; i.e., structured reasons justifying the belief in a proposition. Using defeasible rules, arguments need not be valid in all circumstances, therefore, it might be possible to construct an argument for a proposition as well as its negation. When arguments support conflicting propositions, one of the arguments must be defeated, which raises the question of \emph{which (sub-)arguments can be subject to defeat}? In legal argumentation, meta-rules determine the valid arguments by considering the last defeasible rule of each argument involved in a conflict. Since it is easier to evaluate arguments using their last rules, \emph{can a conflict be resolved by considering only the last defeasible rules of the arguments involved}? We propose a new argument system where, instead of deriving a defeat relation between arguments, \emph{undercutting-arguments} for the defeat of defeasible rules are constructed. This system allows us, (\textit{i}) to resolve conflicts (a generalization of rebutting arguments) using only the last rules of the arguments for inconsistencies, (\textit{ii}) to determine a set of valid (undefeated) arguments in linear time using an algorithm based on a JTMS, (\textit{iii}) to establish a relation with Default Logic, and (\textit{iv}) to prove closure properties such as \emph{cumulativity}. We also propose an extension of the argument system that enables \emph{reasoning by cases}.
Efficient Model Based Diagnosis
Authors: Nico Roos
Date: 2022-09-20 00:00:00
Tasks: Diagnostic, model

In this paper an efficient model based diagnostic process is described for systems whose components possess a causal relation between their inputs and their outputs. In this diagnostic process, firstly, a set of focuses on likely broken components is determined. Secondly, for each focus the most informative probing point within the focus can be determined. Both these steps of the diagnostic process have a worst case time complexity of ${\cal O}(n^2)$ where $n$ is the number of components. If the connectivity of the components is low, however, the diagnostic process shows a linear time complexity. It is also shown how the diagnostic process described can be applied in dynamic systems and systems containing loops. When diagnosing dynamic systems it is possible to choose between detecting intermitting faults or to improve the diagnostic precision by assuming non-intermittency.
Predictive Scale-Bridging Simulations through Active Learning
Authors: Satish Karra, Mohamed Mehana, Nicholas Lubbers, Yu Chen, Abdourahmane Diaw, Javier E. Santos, Aleksandra Pachalieva, Robert S. Pavel, Jeffrey R. Haack, Michael McKerns, Christoph Junghans, Qinjun Kang, Daniel Livescu, Timothy C. Germann, Hari S. Viswanathan
Date: 2022-09-20 00:00:00
Tasks: Active Learning

Throughout computational science, there is a growing need to utilize the continual improvements in raw computational horsepower to achieve greater physical fidelity through scale-bridging over brute-force increases in the number of mesh elements. For instance, quantitative predictions of transport in nanoporous media, critical to hydrocarbon extraction from tight shale formations, are impossible without accounting for molecular-level interactions. Similarly, inertial confinement fusion simulations rely on numerical diffusion to simulate molecular effects such as non-local transport and mixing without truly accounting for molecular interactions. With these two disparate applications in mind, we develop a novel capability which uses an active learning approach to optimize the use of local fine-scale simulations for informing coarse-scale hydrodynamics. Our approach addresses three challenges: forecasting continuum coarse-scale trajectory to speculatively execute new fine-scale molecular dynamics calculations, dynamically updating coarse-scale from fine-scale calculations, and quantifying uncertainty in neural network models.
An Outlier Exposure Approach to Improve Visual Anomaly Detection Performance for Mobile Robots
Authors: Dario Mantegazza, Alessandro Giusti, Luca Maria Gambardella, Jérôme Guzzi
Date: 2022-09-20 00:00:00
Tasks: Anomaly Detection

We consider the problem of building visual anomaly detection systems for mobile robots. Standard anomaly detection models are trained using large datasets composed only of non-anomalous data. However, in robotics applications, it is often the case that (potentially very few) examples of anomalies are available. We tackle the problem of exploiting these data to improve the performance of a Real-NVP anomaly detection model, by minimizing, jointly with the Real-NVP loss, an auxiliary outlier exposure margin loss. We perform quantitative experiments on a novel dataset (which we publish as supplementary material) designed for anomaly detection in an indoor patrolling scenario. On a disjoint test set, our approach outperforms alternatives and shows that exposing even a small number of anomalous frames yields significant performance improvements.