pwc-archive/papers-with-abstracts huggingface dataset:
Integrating Symbolic Reasoning into Neural Generative Models for Design Generation
Authors: Maxwell Joseph Jacobson, Yexiang Xue
Date: 2023-10-13 00:00:00
Tasks: Spatial Reasoning

Design generation requires tight integration of neural and symbolic reasoning, as good design must meet explicit user needs and honor implicit rules for aesthetics, utility, and convenience. Current automated design tools driven by neural networks produce appealing designs but cannot satisfy user specifications and utility requirements. Symbolic reasoning tools, such as constraint programming, cannot perceive low-level visual information in images or capture subtle aspects such as aesthetics. We introduce the Spatial Reasoning Integrated Generator (SPRING) for design generation. SPRING embeds a neural and symbolic integrated spatial reasoning module inside the deep generative network. The spatial reasoning module samples the set of locations of objects to be generated from a backtrack-free distribution. This distribution modifies the implicit preference distribution, which is learned by a recursive neural network to capture utility and aesthetics. Sampling from the backtrack-free distribution is accomplished by a symbolic reasoning approach, SampleSearch, which zeros out the probability of sampling spatial locations violating explicit user specifications. Embedding symbolic reasoning into neural generation guarantees that the output of SPRING satisfies user requirements. Furthermore, SPRING offers interpretability, allowing users to visualize and diagnose the generation process through the bounding boxes. SPRING is also adept at managing novel user specifications not encountered during its training, thanks to its proficiency in zero-shot constraint transfer. Quantitative evaluations and a human study reveal that SPRING outperforms baseline generative models, excelling in delivering high design quality and better meeting user specifications.
Recursively-Constrained Partially Observable Markov Decision Processes
Authors: Qi Heng Ho, Tyler Becker, Benjamin Kraske, Zakariya Laouar, Martin S. Feather, Federico Rossi, Morteza Lahijanian, Zachary N. Sunberg
Date: 2023-10-15 00:00:00

Many sequential decision problems involve optimizing one objective function while imposing constraints on other objectives. Constrained Partially Observable Markov Decision Processes (C-POMDP) model this case with transition uncertainty and partial observability. In this work, we first show that C-POMDPs violate the optimal substructure property over successive decision steps and thus may exhibit behaviors that are undesirable for some (e.g., safety critical) applications. Additionally, online re-planning in C-POMDPs is often ineffective due to the inconsistency resulting from this violation. To address these drawbacks, we introduce the Recursively-Constrained POMDP (RC-POMDP), which imposes additional history-dependent cost constraints on the C-POMDP. We show that, unlike C-POMDPs, RC-POMDPs always have deterministic optimal policies and that optimal policies obey Bellman's principle of optimality. We also present a point-based dynamic programming algorithm for RC-POMDPs. Evaluations on benchmark problems demonstrate the efficacy of our algorithm and show that policies for RC-POMDPs produce more desirable behaviors than policies for C-POMDPs.
Interactive Navigation in Environments with Traversable Obstacles Using Large Language and Vision-Language Models
Authors: Zhen Zhang, Anran Lin, Chun Wai Wong, Xiangyu Chu, Qi Dou, K. W. Samuel Au
Date: 2023-10-13 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Navigate

This paper proposes an interactive navigation framework by using large language and vision-language models, allowing robots to navigate in environments with traversable obstacles. We utilize the large language model (GPT-3.5) and the open-set Vision-language Model (Grounding DINO) to create an action-aware costmap to perform effective path planning without fine-tuning. With the large models, we can achieve an end-to-end system from textual instructions like "Can you pass through the curtains to deliver medicines to me?", to bounding boxes (e.g., curtains) with action-aware attributes. They can be used to segment LiDAR point clouds into two parts: traversable and untraversable parts, and then an action-aware costmap is constructed for generating a feasible path. The pre-trained large models have great generalization ability and do not require additional annotated data for training, allowing fast deployment in the interactive navigation tasks. We choose to use multiple traversable objects such as curtains and grasses for verification by instructing the robot to traverse them. Besides, traversing curtains in a medical scenario was tested. All experimental results demonstrated the proposed framework's effectiveness and adaptability to diverse environments.
HIO-SDF: Hierarchical Incremental Online Signed Distance Fields
Authors: Vasileios Vasilopoulos, Suveer Garg, Jinwook Huh, Bhoram Lee, Volkan Isler
Date: 2023-10-14 00:00:00

A good representation of a large, complex mobile robot workspace must be space-efficient yet capable of encoding relevant geometric details. When exploring unknown environments, it needs to be updatable incrementally in an online fashion. We introduce HIO-SDF, a new method that represents the environment as a Signed Distance Field (SDF). State of the art representations of SDFs are based on either neural networks or voxel grids. Neural networks are capable of representing the SDF continuously. However, they are hard to update incrementally as neural networks tend to forget previously observed parts of the environment unless an extensive sensor history is stored for training. Voxel-based representations do not have this problem but they are not space-efficient especially in large environments with fine details. HIO-SDF combines the advantages of these representations using a hierarchical approach which employs a coarse voxel grid that captures the observed parts of the environment together with high-resolution local information to train a neural network. HIO-SDF achieves a 46% lower mean global SDF error across all test scenes than a state of the art continuous representation, and a 30% lower error than a discrete representation at the same resolution as our coarse global SDF grid. Videos and code are available at: https://samsunglabs.github.io/HIO-SDF-project-page/
SelfVC: Voice Conversion With Iterative Refinement using Self Transformations
Authors: Paarth Neekhara, Shehzeen Hussain, Rafael Valle, Boris Ginsburg, Rishabh Ranjan, Shlomo Dubnov, Farinaz Koushanfar, Julian McAuley
Date: 2023-10-14 00:00:00
Tasks: Self-Supervised Learning, Speaker Verification, Speech Synthesis, Voice Conversion

We propose SelfVC, a training strategy to iteratively improve a voice conversion model with self-synthesized examples. Previous efforts on voice conversion focus on factorizing speech into explicitly disentangled representations that separately encode speaker characteristics and linguistic content. However, disentangling speech representations to capture such attributes using task-specific loss terms can lead to information loss. In this work, instead of explicitly disentangling attributes with loss terms, we present a framework to train a controllable voice conversion model on entangled speech representations derived from self-supervised learning (SSL) and speaker verification models. First, we develop techniques to derive prosodic information from the audio signal and SSL representations to train predictive submodules in the synthesis model. Next, we propose a training strategy to iteratively improve the synthesis model for voice conversion, by creating a challenging training objective using self-synthesized examples. We demonstrate that incorporating such self-synthesized examples during training improves the speaker similarity of generated speech as compared to a baseline voice conversion model trained solely on heuristically perturbed inputs. Our framework is trained without any text and achieves state-of-the-art results in zero-shot voice conversion on metrics evaluating naturalness, speaker similarity, and intelligibility of synthesized audio.
Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning
Authors: Jiachen Li, Qiaozi Gao, Michael Johnston, Xiaofeng Gao, Xuehai He, Suhaila Shakiah, Hangjie Shi, Reza Ghanadan, William Yang Wang
Date: 2023-10-14 00:00:00
Tasks: In-Context Learning, Instruction Following, Robot Manipulation, Task Planning

Prompt-based learning has been demonstrated as a compelling paradigm contributing to large language models' tremendous success (LLMs). Inspired by their success in language tasks, existing research has leveraged LLMs in embodied instruction following and task planning. In this work, we tackle the problem of training a robot to understand multimodal prompts, interleaving vision signals with text descriptions. This type of task poses a major challenge to robots' capability to understand the interconnection and complementarity between vision and language signals. In this work, we introduce an effective framework that learns a policy to perform robot manipulation with multimodal prompts from multi-task expert trajectories. Our methods consist of a two-stage training pipeline that performs inverse dynamics pretraining and multi-task finetuning. To facilitate multimodal understanding, we design our multimodal prompt encoder by augmenting a pretrained LM with a residual connection to the visual input and model the dependencies among action dimensions. Empirically, we evaluate the efficacy of our method on the VIMA-BENCH and establish a new state-of-the-art (10% improvement in success rate). Moreover, we demonstrate that our model exhibits remarkable in-context learning ability. Project page: \url{https://midas-icml.github.io/}.
A ML-LLM pairing for better code comment classification
Authors: Hanna Abi Akl
Date: 2023-10-13 00:00:00
Tasks: Classification, Information Retrieval, Language Modeling, Language Modelling, Large Language Model, Retrieval

The "Information Retrieval in Software Engineering (IRSE)" at FIRE 2023 shared task introduces code comment classification, a challenging task that pairs a code snippet with a comment that should be evaluated as either useful or not useful to the understanding of the relevant code. We answer the code comment classification shared task challenge by providing a two-fold evaluation: from an algorithmic perspective, we compare the performance of classical machine learning systems and complement our evaluations from a data-driven perspective by generating additional data with the help of large language model (LLM) prompting to measure the potential increase in performance. Our best model, which took second place in the shared task, is a Neural Network with a Macro-F1 score of 88.401% on the provided seed data and a 1.5% overall increase in performance on the data generated by the LLM.
Chimp: Efficient Lossless Floating Point Compression for Time Series Databases
Authors: Panagiotis Liakos, Katia Papakonstantinopoulou, Yannis Kotidis
Date: 2022-07-01 00:00:00
Proceeding: Proceedings of the VLDB Endowment 2022 7
Tasks: Astronomy, Time Series

Applications in diverse domains such as astronomy, economics
and industrial monitoring, increasingly press the need for analyzing massive collections of time series data. The sheer size of the
latter hinders our ability to efficiently store them and also yields
significant storage costs. Applying general purpose compression algorithms would effectively reduce the size of the data, at the expense
of introducing significant computational overhead. Time Series
Management Systems that have emerged to address the challenge
of handling this overwhelming amount of information, cannot suffer the ingestion rate restrictions that such compression algorithms
would cause. Data points are usually encoded using faster, streaming compression approaches. However, the techniques that contemporary systems use do not fully utilize the compression potential
of time series data, with implications in both storage requirements
and access times. In this work, we propose a novel streaming compression algorithm, suitable for floating point time series data. We
empirically establish properties exhibited by a diverse set of time
series and harness these features in our proposed encodings. Our
experimental evaluation demonstrates that our approach readily
outperforms competing techniques, attaining compression ratios
that are competitive with slower general purpose algorithms, and
on average around 50% of the space required by state-of-the-art
streaming approaches. Moreover, our algorithm outperforms all
earlier techniques with regards to both compression and access time,
offering a significantly improved trade-off between space and speed.
The aforementioned benefits of our approach –in terms of all space
requirements, compression time and read access– significantly improve the efficiency in which we can store and analyze time series
data.
An Effective WGAN-based Anomaly Detection Model for loT Multivariate Time Series Published on Pacific-Asia Conference on Knowledge Discovery and Data Mining
Authors: Sibo Qi, Juan Chen, Peng Chen, Peian Wen, Wenyu Shan, Ling Xiong
Date: 2023-05-27 00:00:00
Proceeding: Pacific-Asia Conference on Knowledge Discovery and Data Mining 2023 5
Tasks: Prediction, Time Series, Time Series Anomaly Detection

This paper studies an effective unsupervised deep learning model for multivariate time series anomaly detection. Since multivariate time series usually have problems of insufficient labeling and highly-complex temporal correlation, effectively detecting anomalies in multivariate time series data is particularly challenging. To solve this problem, we propose a model named Wasserstein-GAN with gradient Penalty and effective Scoring (WPS). In this model, Wasserstein Distance with Gradient Penalty helps to capture the data regularities between generator output and real data, thus improving the training stability. Meanwhile, an effective scoring function that consists of reconstruction error, discrimination error, and prediction error is designed to evaluate the accuracy of the abnormal prediction and recall. The experimental results show that compared with the suboptimal baseline model, our proposed WPS obtains 17.68% and 10.41% improvement in prediction precision and F1 score, respectively.
Pyramidal Transformer with Conv-Patchify for Person Re-identification
Authors: He Li, Mang Ye, Cong Wang, Bo Do
Date: 2022-10-10 00:00:00
Proceeding: Proceedings of the 30th ACM International Conference on Multimedia 2022 10
Tasks: Person Re-Identification, Re-Ranking, Translation

The robust and discriminative feature extraction is the key component in person re-identification (Re-ID). The major weakness ofconventional convolution neural network (CNN) based methods is that they cannot extract long-range information from diverse parts, which can be alleviated by recently developed Transformers. Existing vision Transformers show their power on various vision tasks. However, they (i) cannot address translation problems and different viewpoints; (ii) cannot capture detailed features to discriminate people with a similar appearance. In this paper, we propose a powerful Re-ID baseline built on top of the pyramidal transformer with conv-patchify operation, termed PTCR, which inherits the advantages of both CNN and Transformer. The pyramidal structure captures multi-scale fine-grained features, while the convpatchify enhances the robustness against translation. Moreover, we additionally design two novel modules to improve the robust feature learning. A Token Perception module augments the patch embeddings to enhance the robustness against perturbation and viewpoint changes, while the Auxiliary Embedding module integrates the auxiliary information (cam ID, pedestrian attributes, etc.)to reduce feature bias caused by non-visual factors. Our method is validated through extensive experiments to show its superior performance with abundant ablation studies. Notably, without re-ranking, we achieve 98.0% Rank-1 on Market-1501 and 88.6% Rank-1 on MSMT17, significantly outperforming the counterparts. The code is available at: https://github.com/lihe404/PTCR
A Novel Similarity Measure of Spatiotemporal Event Setting Sequences: Method Development and Case Study
Authors: Fuyu Xu, Kate Beard
Date: 2023-04-25 00:00:00
Proceeding: MDPI 2023 4

Examining the similarity of event environments or surroundings—more precisely, settings—provides additional insight in analyzing event sequences, as it provides information about the context and potential common factors that may have influenced them. This article proposes a new similarity measure for event setting sequences, which involve the space and time in which events occur.While similarity measures for spatiotemporal event sequences have been studied, the settings and setting sequences have not yet been studied. While modeling event setting sequences, we consider spatial and temporal scales to define the bounds of the setting and incorporate dynamic variables alongside static variables. Using a matrix-based representation and an extended Jaccard index, we developed new similarity measures that allow for the use of all variable data types. We successfully used these similarity measures coupled with other multivariate statistical analysis approaches in a case study involving setting sequences and pollution event sequences associated with the same monitoring stations, which validate the hypothesis that more similar spatial-temporal settings or setting sequences may generate more similar events or event sequences. In conclusion, the developed similarity measures have wide application beyond the case study to other disciplinary contexts and geographical settings. They offer researchers a powerful tool for understanding different factors and their dynamics corresponding to occurrences of spatiotemporal event sequences.
A class alignment method based on graph convolution neural network for bearing fault diagnosis in presence of missing data and changing working conditions
Authors: Mohammadreza Kavianpour, Amin Ramezani, Mohammad T.H. Beheshti
Date: 2022-07-01 00:00:00
Proceeding: Measurement 2022 7
Tasks: Domain Adaptation, Fault Diagnosis

Bearing fault diagnosis in real-world applications has challenges such as insufficient labeled data, changing working conditions of the rotary machinery, and missing data due to multi-rate sampling of sensors. Despite the numerous applications of conventional deep learning (DL) and domain adaptation methods in bearing fault diagnosis, these methods face challenges. Domain adaptation techniques neglect alignment across subdomains with the same class, and DL techniques do not consider data relationships and interdependencies. To tackle these challenges, this paper introduces a novel semi-supervised method based on ARMA graph convolution, adversarial adaptation, and multi-layer multi-kernel local maximum mean discrepancy (MK-LMMD). Structural information of data is extracted using ARMA graph convolution, adversarial adaptation is employed to decrease structural distribution discrepancy in the domains, and MK-LMMD is used to align the classes. Additionally, ARMA graph convolution and MK-LMMD can aid in reducing distribution discrepancy caused by missing data and changing working conditions.
An Intelligent Gearbox Fault Diagnosis under Different Operating Conditions using Adversarial Domain Adaptation
Authors: Mohammadreza Kavianpour, Mohammadreza Ghorvei, Parisa Kavianpour, Amin Ramezani, Mohammad TH Beheshti
Date: 2022-03-02 00:00:00
Proceeding: 8th International Conference on Control, Instrumentation and Automation (ICCIA) 2022 3
Tasks: Diagnostic, Domain Adaptation, Fault Diagnosis, Unsupervised Domain Adaptation

Effective gearbox diagnostic procedures can assist in rotary machinery's reliable and safe operation. On the other hand, the constant change in operational conditions, along with an absence of labeled data, have made fault diagnosis difficult. Changes in working conditions that cause discrepancies in data distribution and a lack of labeled data dramatically reduce fault diagnosis accuracy in deep learning algorithms. In recent years, unsupervised domain adaptation (UDA) has been utilized to overcome these challenges in various applications. This paper introduces a new approach based on the CNN model and hybrid domain adaptation called deep coral adversarial network (DCAN) to solve these issues. The CNN model is used to extract features, and the distribution discrepancy between domains is decreased by applying two modules of domain adversarial learning and deep coral adaptation. The proposed method's performance was evaluated using the SEU gearbox dataset. The results demonstrate the proposed method's proper performance in diagnosing gearbox faults under different operating conditions.
Synthetic to Real Framework based on Convolutional Multi-Head Attention and Hybrid Domain Alignment
Authors: Mohammadreza Ghorvei, Mohammadreza Kavianpour, Mohammad TH Beheshti, Amin Ramezani
Date: 2022-03-02 00:00:00
Proceeding: 8th International Conference on Control, Instrumentation and Automation (ICCIA) 2022 3
Tasks: Domain Adaptation, Fault Diagnosis

Domain adaptation (DA) has obtained remarkable results in unsupervised fault diagnosis methods. However, its performance is highly related to two significant factors: Firstly, proposed DA methods should alleviate the global and local distribution gap to precisely matching all distributions in the source and target domain. Most distance-based DA methods assume global DA or concentrate only on local aligning distributions on the country. Secondly, the generalization of most proposed unsupervised fault diagnosis methods relies on labeled faulty data collected from sensors. Contrarily, collected data in real-world scenarios are mostly unlabeled, which considerably declines the model's generalization. We proposed an unsupervised synthetic to the real framework to overcome two significant challenges. A convolution multi-head attention network based on hybrid multi-layer domain alignment (CMHA-HMLDA) is conducted to align global and local distributions. It also alleviates the gap between real and synthetic data more accurately to maintain a robust data-driven model for bearing fault diagnosis. Furthermore, the proposed method is reliable in real scenarios because of transferring employed knowledge of labeled synthetic data into unlabeled real data. To show the supervisory of our proposed method in diagnosing unlabeled real health states, we validated it with a synthetic dataset made from the benchmark bearing Case Western Reserve University (CWRU) dataset. We compared it with recently published unsupervised fault diagnosis methods. Consequently, we achieved state-of-art results that show our proposed method is capable of realizing unlabeled real bearing faults from synthetic data, and it is practical in real-world scenarios.