pwc-archive/papers-with-abstracts huggingface dataset:
Quantized Compressed Sensing with Score-based Generative Models
Authors: Xiangming Meng, Yoshiyuki Kabashima
Date: 2022-11-02 00:00:00
Tasks: compressed sensing, Quantization

We consider the general problem of recovering a high-dimensional signal from noisy quantized measurements. Quantization, especially coarse quantization such as 1-bit sign measurements, leads to severe information loss and thus a good prior knowledge of the unknown signal is helpful for accurate recovery. Motivated by the power of score-based generative models (SGM, also known as diffusion models) in capturing the rich structure of natural signals beyond simple sparsity, we propose an unsupervised data-driven approach called quantized compressed sensing with SGM (QCS-SGM), where the prior distribution is modeled by a pre-trained SGM. To perform posterior sampling, an annealed pseudo-likelihood score called noise perturbed pseudo-likelihood score is introduced and combined with the prior score of SGM. The proposed QCS-SGM applies to an arbitrary number of quantization bits. Experiments on a variety of baseline datasets demonstrate that the proposed QCS-SGM significantly outperforms existing state-of-the-art algorithms by a large margin for both in-distribution and out-of-distribution samples. Moreover, as a posterior sampling method, QCS-SGM can be easily used to obtain confidence intervals or uncertainty estimates of the reconstructed results. The code is available at https://github.com/mengxiangming/QCS-SGM.
Number Theory Meets Linguistics: Modelling Noun Pluralisation Across 1497 Languages Using 2-adic Metrics
Authors: Gregory Baker, Diego Molla-Aliod
Date: 2022-10-08 00:00:00
Tasks: regression

A simple machine learning model of pluralisation as a linear regression problem minimising a p-adic metric substantially outperforms even the most robust of Euclidean-space regressors on languages in the Indo-European, Austronesian, Trans New-Guinea, Sino-Tibetan, Nilo-Saharan, Oto-Meanguean and Atlantic-Congo language families. There is insufficient evidence to support modelling distinct noun declensions as a p-adic neighbourhood even in Indo-European languages.
DOANet: a deep dilated convolutional neural network approach for search and rescue with drone-embedded sound source localization
Authors: Alif Bin Abdul Qayyum, K. M. Naimul Hassan, Adrita Anika, Md. Farhan Shadiq, Md Mushfiqur Rahman, Md. Tariqul Islam, Sheikh Asif Imran, Shahruk Hossain, Mohammad Ariful Haque
Date: 2020-11-05 00:00:00
Proceeding: EURASIP Journal on Audio, Speech, and Music Processing 2020 11
Tasks: Sound Source Localization

Drone-embedded sound source localization (SSL) has interesting application perspective in challenging search and rescue scenarios due to bad lighting conditions or occlusions. However, the problem gets complicated by severe drone ego-noise that may result in negative signal-to-noise ratios in the recorded microphone signals. In this paper, we present our work on drone-embedded SSL using recordings from an 8-channel cube-shaped microphone array embedded in an unmanned aerial vehicle (UAV). We use angular spectrum-based TDOA (time difference of arrival) estimation methods such as generalized cross-correlation phase-transform (GCC-PHAT), minimum-variance-distortion-less-response (MVDR) as baseline, which are state-of-the-art techniques for SSL. Though we improve the baseline method by reducing ego-noise using speed correlated harmonics cancellation (SCHC) technique, our main focus is to utilize deep learning techniques to solve this challenging problem. Here, we propose an end-to-end deep learning model, called DOANet, for SSL. DOANet is based on a one-dimensional dilated convolutional neural network that computes the azimuth and elevation angles of the target sound source from the raw audio signal. The advantage of using DOANet is that it does not require any hand-crafted audio features or ego-noise reduction for DOA estimation. We then evaluate the SSL performance using the proposed and baseline methods and find that the DOANet shows promising results compared to both the angular spectrum methods with and without SCHC. To evaluate the different methods, we also introduce a well-known parameter—area under the curve (AUC) of cumulative histogram plots of angular deviations—as a performance indicator which, to our knowledge, has not been used as a performance indicator for this sort of problem before.
MFNet: Multi-Feature Fusion Network for Real-Time Semantic Segmentation in Road Scenes
Authors: Mengxu Lu; Zhenxue Chen; Chengyun Liu; Sile Ma; Lei Cai; Hao Qin
Date: 2022-11-01 00:00:00
Proceeding: IEEE Transactions on Intelligent Transportation Systems 2022 11
Tasks: Real-Time Semantic Segmentation, Semantic Segmentation

Although high-accuracy networks have been applied to semantic segmentation at present, their inference speeds remain slow. A trade-off between accuracy and speed is demanded for real-time applications. To approach this problem, we propose Multi-Feature Fusion Network (MFNet) with real-time efficient prediction capacity. MFNet adopts three branches (attention, semantic and spatial information) to capture low-level and high-level features. Additionally, MFNet exerts asymmetric factorized (AF) blocks to extract local and long-range features. As a result, without any pre-training or post-processing, MFNet using only 1.34 M parameters, achieves 72.1% mean intersection over union (mIoU) on the Cityscapes test set at a speed of 116 frames per second (FPS), with 512×1024 high resolution on a single Titan Xp graphics card. Our network’s performance stands out from other state-of-the-art networks on four datasets (Cityscapes, CamVid, KITTI, and Gatech).
FRNet: Factorized and Regular Blocks Network for Semantic Segmentation in Road Scene
Authors: Mengxu Lu; Zhenxue Chen; Q. M. Jonathan Wu; Nannan Wang; Xuewen Rong; Xinghe Yan
Date: 2022-04-01 00:00:00
Proceeding: IEEE Transactions on Intelligent Transportation Systems 2022 4
Tasks: Decoder, Segmentation, Semantic Segmentation

Nowadays, semantic segmentation methods for systems in road scene have a great demand. Most existing methods focus on high accuracy with low inference speed. And some approaches emphasize on speed, significantly sacrificing model accuracy. To make a trade-off between accuracy and inference speed, we propose a real-time network for semantic segmentation titled Factorized and Regular Network (FRNet), which employs an asymmetric encoder-decoder architecture with Factorized and Regular (FR) blocks. Our method achieves 70.4% mIoU on the Cityscapes test set with 1 million parameters at a speed of 127 frames per second (FPS) on a single Titan Xp at a resolution of 512×1024 . We evaluate FRNet on Cityscapes, Camvid, Kitti, and Gatech datasets to identify that our network stands out from other state-of-the-art networks.
Stock Price Prediction Based on Natural Language Processing
Authors: Xiaobin Tang, Nuo Lei, Manru Dong, Dan Ma
Date: 2022-05-06 00:00:00
Proceeding: Complexity 2022 5
Tasks: Prediction, Stock Price Prediction, Time Series, Time Series Analysis, Time Series Forecasting

The keywords used in traditional stock price prediction are mainly based on literature and experience. This paper designs a new text mining method for keywords augmentation based on natural language processing models including Bidirectional Encoder Representation from Transformers (BERT) and Neural Contextualized Representation for Chinese Language Understanding (NEZHA). The BERT vectorization and the NEZHA keyword discrimination model extend the seed keywords from two dimensions of similarity and importance respectively, thus constructing the keyword thesaurus for stock price prediction. Furthermore, the predictive ability of seed words and our generated words are compared by the LSTM model, taking the CSI 300 as an example. The result shows that, compared with seed keywords, the search indexes of extracted words have higher correlations with CSI 300 and can improve its forecasting performance. Therefore, the keywords augmentation model designed in this paper is helpful to provide references for other variable expansion in financial time series forecasting.
COVID-19 mRNA Vaccine Degradation Prediction using Regularized LSTM Model
Authors: Sheikh Asif Imran, Md. Tariqul Islam, Celia Shahnaz, Md. Tafhimul Islam, Omar Tawhid Imam, Moinul Haque
Date: 2020-12-27 00:00:00
Proceeding: IEEE International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE) 2020 12

Due to the advantages of mRNA vaccines such as potency, safety, and production feasibility, recent researches in vaccinology has seen strong focus in mRNA vaccines. As leading researches involving COVID-19 mRNA vaccine candidates are being carried out, the challenge of overcoming the stability tradeoff of mRNA vaccines stand between the production and effective mass distribution stages. With the help of the OpenVaccine RNA database with degradation rate measurements provided by Stanford researchers, we developed an artificial recurrent neural network model to help bioinformatics researchers identify whether and where mRNAs might be unstable and prone to degrade under certain incubation measures. For this purpose we’ve prepared a regularized LSTM model which minimizes mean columnwise root mean squared error for several degradation rates. We’ve found that recurrent algorithms perform better than tree-based algorithms.
Topic Classification from Text Using Decision Tree, K-NN and Multinomial Naïve Bayes
Authors: Md. Ataur Rahman, Yeasmin Ara Akter
Date: 2019-05-03 00:00:00
Proceeding: 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT) 2019 5
Tasks: Classification, Topic Classification

One of the central motivations behind Natural Language Processing is detecting patterns. Given a text document, the task of identifying the context is known to be as topic classification. This paper explores the performance of three different classifiers namely Decision Tree, K-Nearest Neighbors, and Multinomial Naive Bayes on a topic classification task (with six topic classes). The evaluation is done on the basis of accuracy, precision, recall, and f1-score based results. Among those three aforementioned classifiers, we have selected the Multinomial Naïve Bayes a sour best model using which we achieved 91.8% accuracy.
Structural topology optimization with smoothly varying fiber orientations
Authors: Martin-Pierre Schmidt, Laura Couret, Christian Gout & Claus B. W. Pedersen
Date: 2020-07-11 00:00:00
Proceeding: 2020 2020 7
Tasks: valid

In recent years, the field of additive manufacturing (AM), often referred to as 3D printing, has seen tremendous growth and radically changed the means by which we describe valid 3D models for production. In particular, it is now conceivable to produce composite structures consisting of smoothly varying oriented anisotropic constitutive materials. In the present work, we propose a sensitivity driven method for the generation of transverse isotropic fiber reinforced structures having smooth spatially varying orientations. Our approach builds upon finite element analysis (FEA) and density-based topology optimization (TO). The local material orientations are formulated as design variables in a stiffness maximization problem, and solved with a non-convex gradient-based optimization scheme. Length-scale control is achieved through the use of filters for regularization. We demonstrate the ability of the proposed approach to handle large-scale 3D problems with synchronous optimization of material densities and orientations yielding millions of design variables on multiple load case scenarios. The method is shown to be compatible with compliant mechanism optimization as well as local volume constraints. Finally, the approach is extended with an additional design variable dictating the ratio of anisotropy for each element, thereby delegating the choice of material type to the optimization scheme.
Prior Gradient Mask Guided Pruning-Aware Fine-Tuning
Authors: Linhang Cai, Zhulin An, Chuanguang Yang, Yangchun Yan, Yongjun Xu
Date: 2022-02-15 00:00:00
Proceeding: AAAI 2022 2
Tasks: image-classification, Image Classification

We proposed a Prior Gradient Mask Guided Pruning-aware Fine-Tuning (PGMPF) framework to accelerate deep Convolutional Neural Networks (CNNs). In detail, the proposed PGMPF selectively suppresses the gradient of those ”unimportant” parameters via a prior gradient mask generated by the pruning criterion during fine-tuning. PGMPF has three charming characteristics over previous works: (1) Pruning-aware network fine-tuning. A typical pruning pipeline consists of training, pruning and fine-tuning, which are relatively independent, while PGMPF utilizes a variant of the pruning mask as a prior gradient mask to guide fine-tuning, without complicated pruning criteria. (2) An excellent tradeoff between large model capacity during fine-tuning and stable convergence speed to obtain the final compact model. Previous works preserve more training information of pruned parameters during fine-tuning to pursue better performance, which would incur catastrophic non-convergence of the pruned model for relatively large pruning rates, while our PGMPF greatly stabilizes the fine-tuning phase by gradually constraining the learning rate of those ”unimportant” parameters. (3) Channel-wise random dropout of the prior gradient mask to impose some gradient noise to fine-tuning to further improve the robustness of final compact model. Experimental results on three image classification benchmarks CIFAR10/ 100 and ILSVRC-2012 demonstrate the effectiveness of our method for various CNN architectures, datasets and pruning rates. Notably, on ILSVRC-2012, PGMPF reduces 53.5% FLOPs on ResNet-50 with only 0.90% top-1 accuracy drop and 0.52% top-5 accuracy drop, which has advanced the state-of-the-art with negligible extra computational cost.
Adversarial Learning Semantic Volume for 2D/3D Face Shape Regression in the Wild
Authors: Hongwen Zhang, Qi Li, Zhenan Sun
Date: 2019-04-19 00:00:00
Proceeding: IEEE Transactions on Image Processing 2019 4
Tasks: 3D Face Alignment, 3D Facial Landmark Localization, Face Alignment, regression

Regression-based methods have revolutionized 2D landmark localization with the exploitation of deep neural networks and massive annotated datasets in the wild. However, it remains challenging for 3D landmark localization due to the lack of annotated datasets and the ambiguous nature of landmarks under the 3D perspective. This paper revisits regression-based methods and proposes an adversarial voxel and coordinate regression framework for 2D and 3D facial landmark localization in real-world scenarios. First, a semantic volumetric representation is introduced to encode the per-voxel likelihood of positions being the 3D landmarks. Then, an end-to-end pipeline is designed to jointly regress the proposed volumetric representation and the coordinate vector. Such a pipeline not only enhances the robustness and accuracy of the predictions but also unifies the 2D and 3D landmark localization so that the 2D and 3D datasets could be utilized simultaneously. Further, an adversarial learning strategy is exploited to distill 3D structure learned from synthetic datasets to real-world datasets under weakly supervised settings, where an auxiliary regression discriminator is proposed to encourage the network to produce plausible predictions for both the synthetic and real-world images. The effectiveness of our method is validated on benchmark datasets 3DFAW and AFLW2000-3D for both 2D and 3D facial landmark localization tasks. The experimental results show that the proposed method achieves significant improvements over the previous state-of-the-art methods.
MarianCG: a code generation transformer model inspired by machine translation
Authors: Ahmed S. Soliman, Mayada M. Hadhoud, Samir I. Shaheen
Date: 2022-11-22 00:00:00
Proceeding: Journal of Engineering and Applied Science 2022 11
Tasks: Code Generation, Code Translation, Language Modeling, Language Modelling, Machine Translation, NMT, Translation

The idea that computers can build their own programs is extremely significant, and many researchers are working on this challenge. Code generation is described as the process of generating executable code that can be run directly on the computer and fulfills the natural language requirements. It is an intriguing topic that might assist developers to learn a new software technology or programming language, or it could be a simple technique to help in coding through the description of the natural language code developer. In this paper, we present MarianCG, a code generation Transformer model used to tackle the code generation challenge of generating python code from natural language descriptions. Marian neural machine translation (NMT), which is the core model of the Microsoft Translator, is the basis for our NL-to-Code translation engine and is the heart of the teaching model. MarianMT is the teacher language model in our study, and it is one of the most successful machine translation transformers. In our approach, we use a sinusoidal positional embedding technique to represent the position of each token in the text, as well as no layer normalization embedding. Our code generation approach, MarianCG, is based on fine-tuning a machine translation pre-trained language model. This allows us to demonstrate that the pre-trained translation model can also operate and work as a code generation model. The proposed model outperforms recent state-of-the-art models in the problem of code generation when trained on the CoNaLa and DJANGO datasets. MarianCG model scores a BLEU score of 34.43 and an exact match accuracy of 10.2% on the CoNaLa dataset. Also, this model records a BLEU score of 90.41 and an exact match accuracy of 81.83% on the DJANGO dataset. The implementation of MarianCG model and relevant resources are available at https://www.github.com/AhmedSSoliman/MarianCG-NL-to-Code.
fseval: A Benchmarking Framework for Feature Selection and Feature Ranking Algorithms
Authors: Jeroen G. S. Overschie, Ahmad Alsahaf, George Azzopardi
Date: 2022-11-23 00:00:00
Proceeding: Journal of Open Source Software 2022 11
Tasks: Automated Feature Engineering, Benchmarking, Classification with Costly Features, Distributed Computing, Feature Engineering, Feature Importance, feature selection

The fseval Python package allows benchmarking Feature Selection and Feature Ranking algorithms on a large scale, and facilitates the comparison of multiple algorithms in a systematic way. In particular, fseval enables users to run experiments in parallel and distributed over multiple machines, and export the results to an SQL database. The execution of an experiment can be fully determined by a configuration file, which means the experiment results can be reproduced easily, given only the configuration file. fseval has high test coverage, continuous integration, and rich documentation. The package is open source and can be installed through PyPI. The source code is available at: https://github.com/dunnkers/fseval.
Hengam: An Adversarially Trained Transformer for Persian Temporal Tagging
Authors: Sajad Mirzababaei, Amir Hossein Kargaran, Hinrich Schütze, Ehsaneddin Asgari
Date: 2022-11-20 00:00:00
Proceeding: Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing 2022 11
Tasks: Information Retrieval, Named Entity Recognition (NER), Question Answering, Retrieval, Temporal Tagging, Text Anonymization, Text Summarization

Many NLP main tasks benefit from an accurate understanding of temporal expressions, e.g., text summarization, question answering, and information retrieval. This paper introduces Hengam, an adversarially trained transformer for Persian temporal tagging outperforming state-of-the-art approaches on a diverse and manually created dataset. We create Hengam in the following concrete steps: (1) we develop HengamTagger, an extensible rule-based tool that can extract temporal expressions from a set of diverse language-specific patterns for any language of interest. (2) We apply HengamTagger to annotate temporal tags in a large and diverse Persian text collection (covering both formal and informal contexts) to be used as weakly labeled data. (3) We introduce an adversarially trained transformer model on HengamCorpus that can generalize over the HengamTagger’s rules. We create HengamGold, the first high-quality gold standard for Persian temporal tagging. Our trained adversarial HengamTransformer not only achieves the best performance in terms of the F1-score (a type F1-Score of 95.42 and a partial F1-Score of 91.60) but also successfully deals with language ambiguities and incorrect spellings. Our code, data, and models are publicly available at https://github.com/kargaranamir/Hengam.
Reduction Algorithms for Persistence Diagrams of Networks: CoralTDA and PrunIT
Authors: Cuneyt Gurcan Akcora, Murat Kantarcioglu, Yulia R. Gel, Baris Coskunuzer
Date: 2022-11-24 00:00:00
Tasks: Topological Data Analysis

Topological data analysis (TDA) delivers invaluable and complementary information on the intrinsic properties of data inaccessible to conventional methods. However, high computational costs remain the primary roadblock hindering the successful application of TDA in real-world studies, particularly with machine learning on large complex networks. Indeed, most modern networks such as citation, blockchain, and online social networks often have hundreds of thousands of vertices, making the application of existing TDA methods infeasible. We develop two new, remarkably simple but effective algorithms to compute the exact persistence diagrams of large graphs to address this major TDA limitation. First, we prove that $(k+1)$-core of a graph $\mathcal{G}$ suffices to compute its $k^{th}$ persistence diagram, $PD_k(\mathcal{G})$. Second, we introduce a pruning algorithm for graphs to compute their persistence diagrams by removing the dominated vertices. Our experiments on large networks show that our novel approach can achieve computational gains up to 95%. The developed framework provides the first bridge between the graph theory and TDA, with applications in machine learning of large complex networks. Our implementation is available at https://github.com/cakcora/PersistentHomologyWithCoralPrunit
MUSTER: A Multi-scale Transformer-based Decoder for Semantic Segmentation
Authors: Jing Xu, Wentao Shi, Pan Gao, Zhengwei Wang, Qizhu Li
Date: 2022-11-25 00:00:00
Tasks: Decoder, Object Localization, Segmentation, Semantic Segmentation

In recent works on semantic segmentation, there has been a significant focus on designing and integrating transformer-based encoders. However, less attention has been given to transformer-based decoders. We emphasize that the decoder stage is equally vital as the encoder in achieving superior segmentation performance. It disentangles and refines high-level cues, enabling precise object boundary delineation at the pixel level. In this paper, we introduce a novel transformer-based decoder called MUSTER, which seamlessly integrates with hierarchical encoders and consistently delivers high-quality segmentation results, regardless of the encoder architecture. Furthermore, we present a variant of MUSTER that reduces FLOPS while maintaining performance. MUSTER incorporates carefully designed multi-head skip attention (MSKA) units and introduces innovative upsampling operations. The MSKA units enable the fusion of multi-scale features from the encoder and decoder, facilitating comprehensive information integration. The upsampling operation leverages encoder features to enhance object localization and surpasses traditional upsampling methods, improving mIoU (mean Intersection over Union) by 0.4% to 3.2%. On the challenging ADE20K dataset, our best model achieves a single-scale mIoU of 50.23 and a multi-scale mIoU of 51.88, which is on-par with the current state-of-the-art model. Remarkably, we achieve this while significantly reducing the number of FLOPs by 61.3%. Our source code and models are publicly available at: https://github.com/shiwt03/MUSTER.