OpenRaiser/Intern-Atlas huggingface dataset:
Bangla-Wave: Improving Bangla Automatic Speech Recognition Utilizing N-gram Language Models
Venue: arXiv cs.CL 2022
Year: 2022

Although over 300M around the world speak Bangla, scant work has been done in improving Bangla voice-to-text transcription due to Bangla being a low-resource language. However, with the introduction of the Bengali Common Voice 9.0 speech dataset, Automatic Speech Recognition (ASR) models can now be significantly improved. With 399hrs of speech recordings, Bengali Common Voice is the largest and most diversified open-source Bengali speech corpus in the world. In this paper, we outperform the SOTA pretrained Bengali ASR models by finetuning a pretrained wav2vec2 model on the common voice dataset. We also demonstrate how to significantly improve the performance of an ASR model by adding an n-gram language model as a post-processor. Finally, we do some experiments and hyperparameter tuning to generate a robust Bangla ASR model that is better than the existing ASR models.
Deontic Meta-Rules
Venue: arXiv cs.AI 2022
Year: 2022

The use of meta-rules in logic, i.e., rules whose content includes other rules, has recently gained attention in the setting of non-monotonic reasoning: a first logical formalisation and efficient algorithms to compute the (meta)-extensions of such theories were proposed in Olivieri et al (2021) This work extends such a logical framework by considering the deontic aspect. The resulting logic will not just be able to model policies but also tackle well-known aspects that occur in numerous legal systems. The use of Defeasible Logic (DL) to model meta-rules in the application area we just alluded to has been investigated. Within this line of research, the study mentioned above was not focusing on the general computational properties of meta-rules. This study fills this gap with two major contributions. First, we introduce and formalise two variants of Defeasible Deontic Logic with Meta-Rules to represent (1) defeasible meta-theories with deontic modalities, and (2) two different types of conflicts among rules: Simple Conflict Defeasible Deontic Logic, and Cautious Conflict Defeasible Deontic Logic. Second, we advance efficient algorithms to compute the extensions for both variants.
Flexible Job Classification with Zero-Shot Learning
Venue: arXiv cs.CL 2022
Year: 2022

Using a taxonomy to organize information requires classifying objects (documents, images, etc) with appropriate taxonomic classes. The flexible nature of zero-shot learning is appealing for this task because it allows classifiers to naturally adapt to taxonomy modifications. This work studies zero-shot multi-label document classification with fine-tuned language models under realistic taxonomy expansion scenarios in the human resource domain. Experiments show that zero-shot learning can be highly effective in this setting. When controlling for training data budget, zero-shot classifiers achieve a 12% relative increase in macro-AP when compared to a traditional multi-label classifier trained on all classes. Counterintuitively, these results suggest in some settings it would be preferable to adopt zero-shot techniques and spend resources annotating more documents with an incomplete set of classes, rather than spreading the labeling budget uniformly over all classes and using traditional classification techniques. Additional experiments demonstrate that adopting the well-known filter/re-rank decomposition from the recommender systems literature can significantly reduce the computational burden of high-performance zero-shot classifiers, empirically resulting in a 98% reduction in computational overhead for only a 2% relative decrease in performance. The evidence presented here demonstrates that zero-shot learning has the potential to significantly increase the flexibility of taxonomies and highlights directions for future research.
Multi-modal Video Chapter Generation
Venue: arXiv cs.CV 2022
Year: 2022

Chapter generation becomes practical technique for online videos nowadays. The chapter breakpoints enable users to quickly find the parts they want and get the summative annotations. However, there is no public method and dataset for this task. To facilitate the research along this direction, we introduce a new dataset called Chapter-Gen, which consists of approximately 10k user-generated videos with annotated chapter information. Our data collection procedure is fast, scalable and does not require any additional manual annotation. On top of this dataset, we design an effective baseline specificlly for video chapters generation task. which captures two aspects of a video,including visual dynamics and narration text. It disentangles local and global video features for localization and title generation respectively. To parse the long video efficiently, a skip sliding window mechanism is designed to localize potential chapters. And a cross attention multi-modal fusion module is developed to aggregate local features for title generation. Our experiments demonstrate that the proposed framework achieves superior results over existing methods which illustrate that the method design for similar task cannot be transfered directly even after fine-tuning. Code and dataset are available at https://github.com/czt117/MVCG.
Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts
Venue: arXiv cs.CL 2022
Year: 2022

Previous work has shown that there exists a scaling law between the size of Language Models (LMs) and their zero-shot performance on different downstream NLP tasks. In this work, we show that this phenomenon does not hold when evaluating large LMs on tasks with negated prompts, but instead shows an inverse scaling law. We evaluate 9 different tasks with negated prompts on (1) pretrained LMs (OPT & GPT-3) of varying sizes (125M - 175B), (2) LMs further pretrained to generalize to novel prompts (InstructGPT), (3) LMs provided with few-shot examples, and (4) LMs fine-tuned specifically on negated prompts; all LM types perform worse on negated prompts as they scale and show a huge performance gap between the human performance when comparing the average score on both original and negated prompts. By highlighting a critical limitation of existing LMs and methods, we urge the community to develop new approaches of developing LMs that actually follow the given instructions. We provide the code and the datasets to explore negated prompts at https://github.com/joeljang/negated-prompts-for-llms
A Simple Way to Learn Metrics Between Attributed Graphs
Venue: arXiv cs.LG 2022
Year: 2022

The choice of good distances and similarity measures between objects is important for many machine learning methods. Therefore, many metric learning algorithms have been developed in recent years, mainly for Euclidean data in order to improve performance of classification or clustering methods. However, due to difficulties in establishing computable, efficient and differentiable distances between attributed graphs, few metric learning algorithms adapted to graphs have been developed despite the strong interest of the community. In this paper, we address this issue by proposing a new Simple Graph Metric Learning - SGML - model with few trainable parameters based on Simple Graph Convolutional Neural Networks - SGCN - and elements of Optimal Transport theory. This model allows us to build an appropriate distance from a database of labeled (attributed) graphs to improve the performance of simple classification algorithms such as $k$-NN. This distance can be quickly trained while maintaining good performances as illustrated by the experimental study presented in this paper.
Outage Performance of Cross-Packet HARQ
Venue: arXiv cs.IT 2022
Year: 2022

As opposed to hybrid automatic repeat request with incremental redundancy (HARQ-IR) that all the resources are occupied to resend the redundant information, cross-packet HARQ (XP-HARQ) allows the introduction of new information into retransmissions to substantially exploit the remaining resources. This letter provides a profound investigation into the outage performance of XP-HARQ. In particular, the exact outage expression of XP-HARQ is derived if the maximum number of transmissions is two, and tight outage bounds are obtained for more than two transmissions. Moreover, the asymptotic outage analysis of XP-HARQ in the high signal-to-noise ratio (SNR) regime is carried out not only to simplify the outage expression, but also to show that full time diversity is achievable by XP-HARQ. The simulation results are eventually presented for verifications.
Do ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour
Venue: arXiv cs.CL 2022
Year: 2022

Language models (LMs) trained on raw texts have no direct access to the physical world. Gordon and Van Durme (2013) point out that LMs can thus suffer from reporting bias: texts rarely report on common facts, instead focusing on the unusual aspects of a situation. If LMs are only trained on text corpora and naively memorise local co-occurrence statistics, they thus naturally would learn a biased view of the physical world. While prior studies have repeatedly verified that LMs of smaller scales (e.g., RoBERTa, GPT-2) amplify reporting bias, it remains unknown whether such trends continue when models are scaled up. We investigate reporting bias from the perspective of colour in larger language models (LLMs) such as PaLM and GPT-3. Specifically, we query LLMs for the typical colour of objects, which is one simple type of perceptually grounded physical common sense. Surprisingly, we find that LLMs significantly outperform smaller LMs in determining an object's typical colour and more closely track human judgments, instead of overfitting to surface patterns stored in texts. This suggests that very large models of language alone are able to overcome certain types of reporting bias that are characterized by local co-occurrences.
Towards Data-driven GIM tools: Two Prototypes
Venue: arXiv cs.HC 2022
Year: 2022

Here we describe two approaches to improve group information management (GIM) and draw on the results of prior works to implement them in software prototypes. The first aids browsing and retrieving from large and unfamiliar collections like shared drives by dynamically reducing and re-organising them. The second supports the transfer and re-use of collections (e.g. to/by successors, descendants, or curators) by integrating novel sorting and annotation features. The prototypes' source code is shared online and screenshots are presented in the accompanying poster.
Feedback Exchange and Online Affinity: A Case Study of Online Fanfiction Writers
Venue: arXiv cs.SI 2022
Year: 2022

Feedback is a critical piece of the creative process. Prior CSCW research has invented peer-based and crowd-based systems that exchange feedback between online strangers at scale. However, creators run into socio-psychological challenges when engaging in online critique exchange with people they have never met. In this study, we step back and take a different approach to investigate online feedback by casting our attention to creators in online affinity spaces, where feedback exchange happens naturally as part of the interest-driven participatory culture. We present an interview study with 29 fanfiction writers that investigated how they seek feedback online, and how they identified and built connections with feedback providers. We identify four distinct feedback practices and unpack the social needs in feedback exchange. Our findings surface the importance of affinity and trust in online feedback exchange and illustrate how writers built relationships with feedback providers in public and private online spaces. Inspired by the powerful stories we heard about connection and feedback, we conclude with a series of design considerations for future feedback systems\textemdash namely addressing a range of social needs in feedback, helping feedback seekers signal interests and identity, supporting authentic relationships in feedback exchange, and building inclusive, safe community spaces for feedback.
Towards Direct Comparison of Community Structures in Social Networks
Venue: arXiv cs.SI 2022
Year: 2022

Community detection algorithms are in general evaluated by comparing evaluation metric values for the communities obtained with different algorithms. The evaluation metrics that are used for measuring quality of the communities incorporate the topological information of entities like connectivity of the nodes within or outside the communities. However, while comparing the metric values it loses direct involvement of topological information of the communities in the comparison process. In this paper, a direct comparison approach is proposed where topological information of the communities obtained with two algorithms are compared directly. A quality measure namely \emph{Topological Variance (TV)} is designed based on direct comparison of topological information of the communities. Considering the newly designed quality measure, two ranking schemes are developed. The efficacy of proposed quality metric as well as the ranking scheme is studied with eight widely used real-world datasets and six community detection algorithms.
Edge-assisted Collaborative Digital Twin for Safety-Critical Robotics in Industrial IoT
Venue: arXiv cs.RO 2022
Year: 2022

Digital Twin technology is playing a pivotal role in the modern industrial evolution. Especially, with the technological progress in the Internet-of-Things (IoT) and the increasing trend in autonomy, multi-sensor equipped robotics can create practical digital twin, which is particularly useful in the industrial applications for operations, maintenance and safety. Herein, we demonstrate a real-world digital twin of a safety-critical robotics applications with a Franka-Emika-Panda robotic arm. We develop and showcase an edge-assisted collaborative digital twin for dynamic obstacle avoidance which can be useful in real-time adaptation of the robots while operating in the uncertain and dynamic environments in industrial IoT.
Realistic Hair Synthesis with Generative Adversarial Networks
Venue: arXiv cs.GR 2022
Year: 2022

Recent successes in generative modeling have accelerated studies on this subject and attracted the attention of researchers. One of the most important methods used to achieve this success is Generative Adversarial Networks (GANs). It has many application areas such as; virtual reality (VR), augmented reality (AR), super resolution, image enhancement. Despite the recent advances in hair synthesis and style transfer using deep learning and generative modelling, due to the complex nature of hair still contains unsolved challenges. The methods proposed in the literature to solve this problem generally focus on making high-quality hair edits on images. In this thesis, a generative adversarial network method is proposed to solve the hair synthesis problem. While developing this method, it is aimed to achieve real-time hair synthesis while achieving visual outputs that compete with the best methods in the literature. The proposed method was trained with the FFHQ dataset and then its results in hair style transfer and hair reconstruction tasks were evaluated. The results obtained in these tasks and the operating time of the method were compared with MichiGAN, one of the best methods in the literature. The comparison was made at a resolution of 128x128. As a result of the comparison, it has been shown that the proposed method achieves competitive results with MichiGAN in terms of realistic hair synthesis, and performs better in terms of operating time.
Approximate Description Length, Covering Numbers, and VC Dimension
Venue: arXiv cs.LG 2022
Year: 2022

Recently, Daniely and Granot [arXiv:1910.05697] introduced a new notion of complexity called Approximate Description Length (ADL). They used it to derive novel generalization bounds for neural networks, that despite substantial work, were out of reach for more classical techniques such as discretization, Covering Numbers and Rademacher Complexity. In this paper we explore how ADL relates to classical notions of function complexity such as Covering Numbers and VC Dimension. We find that for functions whose range is the reals, ADL is essentially equivalent to these classical complexity measures. However, this equivalence breaks for functions with high dimensional range.
Optical Neural Ordinary Differential Equations
Venue: arXiv cs.LG 2022
Year: 2022

Increasing the layer number of on-chip photonic neural networks (PNNs) is essential to improve its model performance. However, the successively cascading of network hidden layers results in larger integrated photonic chip areas. To address this issue, we propose the optical neural ordinary differential equations (ON-ODE) architecture that parameterizes the continuous dynamics of hidden layers with optical ODE solvers. The ON-ODE comprises the PNNs followed by the photonic integrator and optical feedback loop, which can be configured to represent residual neural networks (ResNet) and recurrent neural networks with effectively reduced chip area occupancy. For the interference-based optoelectronic nonlinear hidden layer, the numerical experiments demonstrate that the single hidden layer ON-ODE can achieve approximately the same accuracy as the two-layer optical ResNet in image classification tasks. Besides, the ONODE improves the model classification accuracy for the diffraction-based all-optical linear hidden layer. The time-dependent dynamics property of ON-ODE is further applied for trajectory prediction with high accuracy.
Cross-lingual Dysarthria Severity Classification for English, Korean, and Tamil
Venue: arXiv cs.CL 2022
Year: 2022

This paper proposes a cross-lingual classification method for English, Korean, and Tamil, which employs both language-independent features and language-unique features. First, we extract thirty-nine features from diverse speech dimensions such as voice quality, pronunciation, and prosody. Second, feature selections are applied to identify the optimal feature set for each language. A set of shared features and a set of distinctive features are distinguished by comparing the feature selection results of the three languages. Lastly, automatic severity classification is performed, utilizing the two feature sets. Notably, the proposed method removes different features by languages to prevent the negative effect of unique features for other languages. Accordingly, eXtreme Gradient Boosting (XGBoost) algorithm is employed for classification, due to its strength in imputing missing data. In order to validate the effectiveness of our proposed method, two baseline experiments are conducted: experiments using the intersection set of mono-lingual feature sets (Intersection) and experiments using the union set of mono-lingual feature sets (Union). According to the experimental results, our method achieves better performance with a 67.14% F1 score, compared to 64.52% for the Intersection experiment and 66.74% for the Union experiment. Further, the proposed method attains better performances than mono-lingual classifications for all three languages, achieving 17.67%, 2.28%, 7.79% relative percentage increases for English, Korean, and Tamil, respectively. The result specifies that commonly shared features and language-specific features must be considered separately for cross-language dysarthria severity classification.
Towards Simple and Efficient Task-Adaptive Pre-training for Text Classification
Venue: arXiv cs.CL 2022
Year: 2022

Language models are pre-trained using large corpora of generic data like book corpus, common crawl and Wikipedia, which is essential for the model to understand the linguistic characteristics of the language. New studies suggest using Domain Adaptive Pre-training (DAPT) and Task-Adaptive Pre-training (TAPT) as an intermediate step before the final finetuning task. This step helps cover the target domain vocabulary and improves the model performance on the downstream task. In this work, we study the impact of training only the embedding layer on the model's performance during TAPT and task-specific finetuning. Based on our study, we propose a simple approach to make the intermediate step of TAPT for BERT-based models more efficient by performing selective pre-training of BERT layers. We show that training only the BERT embedding layer during TAPT is sufficient to adapt to the vocabulary of the target domain and achieve comparable performance. Our approach is computationally efficient, with 78\% fewer parameters trained during TAPT. The proposed embedding layer finetuning approach can also be an efficient domain adaptation technique.
Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 Tile
Venue: arXiv cs.AR 2022
Year: 2022

Most of today's computer vision pipelines are built around deep neural networks, where convolution operations require most of the generally high compute effort. The Winograd convolution algorithm computes convolutions with fewer MACs compared to the standard algorithm, reducing the operation count by a factor of 2.25x for 3x3 convolutions when using the version with 2x2-sized tiles $F_2$. Even though the gain is significant, the Winograd algorithm with larger tile sizes, i.e., $F_4$, offers even more potential in improving throughput and energy efficiency, as it reduces the required MACs by 4x. Unfortunately, the Winograd algorithm with larger tile sizes introduces numerical issues that prevent its use on integer domain-specific accelerators and higher computational overhead to transform input and output data between spatial and Winograd domains. To unlock the full potential of Winograd $F_4$, we propose a novel tap-wise quantization method that overcomes the numerical issues of using larger tiles, enabling integer-only inference. Moreover, we present custom hardware units that process the Winograd transformations in a power- and area-efficient way, and we show how to integrate such custom modules in an industrial-grade, programmable DSA. An extensive experimental evaluation on a large set of state-of-the-art computer vision benchmarks reveals that the tap-wise quantization algorithm makes the quantized Winograd $F_4$ network almost as accurate as the FP32 baseline. The Winograd-enhanced DSA achieves up to 1.85x gain in energy efficiency and up to 1.83x end-to-end speed-up for state-of-the-art segmentation and detection networks.
Efficient approach of using CNN based pretrained model in Bangla handwritten digit recognition
Venue: arXiv cs.CV 2022
Year: 2022

Due to digitalization in everyday life, the need for automatically recognizing handwritten digits is increasing. Handwritten digit recognition is essential for numerous applications in various industries. Bengali ranks the fifth largest language in the world with 265 million speakers (Native and non-native combined) and 4 percent of the world population speaks Bengali. Due to the complexity of Bengali writing in terms of variety in shape, size, and writing style, researchers did not get better accuracy using Supervised machine learning algorithms to date. Moreover, fewer studies have been done on Bangla handwritten digit recognition (BHwDR). In this paper, we proposed a novel CNN-based pre-trained handwritten digit recognition model which includes Resnet-50, Inception-v3, and EfficientNetB0 on NumtaDB dataset of 17 thousand instances with 10 classes.. The Result outperformed the performance of other models to date with 97% accuracy in the 10-digit classes. Furthermore, we have evaluated the result or our model with other research studies while suggesting future study