Reviewerly/exHarmony huggingface dataset:
Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-Based Multi-Modal Context Modeling

Comparing with traditional text-to-speech (TTS) systems, conversational TTS systems are required to synthesize speeches with proper speaking style confirming to the conversational context. However, state-of-the-art context modeling methods in conversational TTS only model the textual information in context with a recurrent neural network (RNN). Such methods have limited ability in modeling the inter-speaker influence in conversations, and also neglect the speaking styles and the intra-speaker inertia inside each speaker. Inspired by DialogueGCN and its superiority in modeling such conversational influences than RNN based approaches, we propose a graph-based multi-modal context modeling method and adopt it to conversational TTS to enhance the speaking styles of synthesized speeches. Both the textual and speaking style information in the context are extracted and processed by DialogueGCN to model the inter- and intra-speaker influence in conversations. The outputs of DialogueGCN are then summarized by attention mechanism, and converted to the enhanced speaking style for current utterance. An English conversation corpus is collected and annotated for our research and released to public. Experiment results on this corpus demonstrate the effectiveness of our proposed approach, which outperforms the state-of-the-art context modeling method in conversational TTS in both MOS and ABX preference rate.
ISOMETRIC MT: Neural Machine Translation for Automatic Dubbing

Automatic dubbing (AD) is among the machine translation (MT) use cases where translations should match a given length to allow for synchronicity between source and target speech. For neural MT, generating translations of length close to the source length (e.g. within ±10% in character count), while preserving quality is a challenging task. Controlling MT output length comes at a cost to translation quality, which is usually mitigated with a two step approach of generating N-best hypotheses and then re-ranking based on length and quality. This work introduces a self-learning approach that allows a transformer model to directly learn to generate outputs that closely match the source length, in short Isometric MT. In particular, our approach does not require to generate multiple hypotheses nor any auxiliary ranking function. We report results on four language pairs (English → French, Italian, German, Spanish) with a publicly available benchmark. Automatic and manual evaluations show that our method for Isometric MT outperforms more complex approaches proposed in the literature.
Prosodyspeech: Towards Advanced Prosody Model for Neural Text-to-Speech

This paper proposes ProsodySpeech, a novel prosody model to enhance encoder-decoder neural Text-To-Speech (TTS), to generate high expressive and personalized speech even with very limited training data. First, we use a Prosody Extractor built from a large speech corpus with various speakers to generate a set of prosody exemplars from multiple reference speeches, in which Mutual Information based Style content separation (MIST) is adopted to alleviate "content leakage" problem. Second, we use a Prosody Distributor to make a soft selection of appropriate prosody exemplars in phone-level with the help of an attention mechanism. The resulting prosody feature is then aggregated into the output of text encoder, together with additional phone-level pitch feature to enrich the prosody. We apply this method into two tasks: highly expressive multi style/emotion TTS and few-shot personalized TTS. The experiments show the proposed model outperforms baseline FastSpeech 2 + GST with significant improvements in terms of similarity and style expression.
Voice Filter: Few-Shot Text-to-Speech Speaker Adaptation Using Voice Conversion as a Post-Processing Module

State-of-the-art text-to-speech (TTS) systems require several hours of recorded speech data to generate high-quality synthetic speech. When using reduced amounts of training data, standard TTS models suffer from speech quality and intelligibility degradations, making training low-resource TTS systems problematic. In this paper, we propose a novel extremely low-resource TTS method called Voice Filter that uses as little as one minute of speech from a target speaker. It uses voice conversion (VC) as a post-processing module appended to a pre-existing high-quality TTS system and marks a conceptual shift in the existing TTS paradigm, framing the few-shot TTS problem as a VC task. Furthermore, we propose to use a duration-controllable TTS system to create a parallel speech corpus to facilitate the VC task. Results show that the Voice Filter outperforms state-of-the-art few-shot speech synthesis techniques in terms of objective and subjective metrics on one minute of speech on a diverse set of voices, while being competitive against a TTS model built on 30 times more data. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
A Low-Cost Embedded Car Counter System by using Jetson Nano Based on Computer Vision and Internet of Things

With the increasing volume of cars in traffic and the global traffic increasing exponentially, it has become critical to manage traffic as a challenge in the most developed countries. To address this issue, the intelligent traffic control system will use automatic vehicle counting as one of its core tasks to facilitate access, particularly in parking lots. The primary benefit of automatic vehicle counting is that it allows for managing and evaluating traffic conditions in the urban transportation system. The new era of technologies such as the Internet of Things and computer vision has transformed traditional systems into new smart city networks. Because of the proliferation of computer vision, traffic counting from low-cost control cameras may emerge as an appealing candidate for traffic flow control automation. This paper proposed a low-cost embedded car counter system using a Jetson nano card based on computer vision and IoT technologies to implement the offered system. In the proposed system, we apply a combination of background subtraction and counters, trackable objects, centroid tracking, and direction counting. Moreover, we implement the MoG foreground-background subtractor method. The proposed system is connected to the Internet using Telegram API to send notifications to smartphone hourly to analyze traffic congestion. In addition, we compared the performance of Jetson nano with the Raspberry Pi4 platform.
Post Covid-19 Sentiment Analysis of Success of Online Learning: A Case Study of India

Over the years, e-learning has seen a boost with the advancement in technology. The very pertinent problem of Covid-19 led to the adoption of online education in India. This study aims to understand the online education in India in terms of benefits, challenges, and strategies using sentiment analysis and the positive and negative instances faced by Indian students and teachers during the pandemic time. Responses from students and teachers have been garnered during the pandemic Covid-19 as the primary source of data. The nature of the research conducted is exploratory and qualitative. Further, a comparative analysis has been undertaken using data visualization. Primary data has been collected using questionnaire and interview methods while secondary data through Twitter. The data pertains to the sentiments of undergraduate and management graduate college students and teachers from February 2020 to January 2022. The secondary data corpus collected from Twitter (5000 tweets) was scraped using Application Program Interfaces (APIs) for further analysis. The Findings suggest that most of the opinions in the primary data collected have been negative while the secondary data analysis shows that polarity is more on neutral sentiments and subjectivity is more on positive sentiments. Relevant recommendations have been made on the conclusion drawn.
Incremental Context Aware Attentive Knowledge Tracing

Knowledge Tracing is the prediction of the future performance of a learner, given the past performance. The existing knowledge tracing models represent the training data and does not generalize when there is a drift in the data distribution. We first empirically demonstrate an evolving Knowledge Tracing (eKT) scenario with distinct distribution of learner performances and diversity of questions from similar concepts. Next, we empirically characterize drift in the data and propose a task agnostic incremental context aware attentive knowledge tracing (iAKT) approach to learn incrementally from the eKT. The iAKT regularizes representations to learn from diverse learner performance distributions. Finally, we evaluate the ability of the proposed iAKT for knowledge tracing and study the effect of various regularization strategies on ranking difficulty of questions, using the ASSISTments 2017 data set. Performance results show that the iAKT adapts its representations to drift in data characteristics, while iAKT with EWC regularizer is better at ranking the difficulty of questions.
Open-source tutorials benefit the field

Open research is increasingly required by journals and funders but involves many new skills. Creating open-source tutorials is useful to the field and personally rewarding, but these efforts must be credited accordingly. Open research is increasingly required by journals and funders but involves many new skills. Creating open-source tutorials is useful to the field and personally rewarding, but these efforts must be credited accordingly.
Selective Noise Based Power-Efficient and Effective Countermeasure against Thermal Covert Channel Attacks in Multi-Core Systems

With increasing interest in multi-core systems, such as any communication systems, infra-structures can become targets for information leakages via covert channel communication. Covert channel attacks lead to leaking secret information and data. To design countermeasures against these threats, we need to have good knowledge about classes of covert channel attacks along with their properties. Temperature–based covert communication channel, known as Thermal Covert Channel (TCC), can pose a threat to the security of critical information and data. In this paper, we present a novel scheme against such TCC attacks. The scheme adds selective noise to the thermal signal so that any possible TCC attack can be wiped out. The noise addition only happens at instances when there are chances of correct information exchange to increase the bit error rate (BER) and keep the power consumption low. Our experiments have illustrated that the BER of a TCC attack can increase to 94% while having similar power consumption as that of state-of-the-art.
An Ensemble Model for detecting Sarcasm on Social Media

Nowadays, sarcasm is extensively utilized in regular conversations over social media sites. There have been cases where sarcastic statements have significantly reduced the accuracy in automatically detecting sentiment analysis or fake news. The paper presents natural language processing based text embedding techniques utilizing the ensemble approach in this study. The framework proposes the generation of text embeddings from Word2Vec, Global vectors for word representation (GloVe), and long short-term memory (LSTM) models. Word2Vec and GloVe are word embeddings based generators, while LSTM considers full text. Thus, the proposed framework encapsulates small tweets and sentences scenarios to detect sarcasm. Various natural language processing approaches have been developed, but each has textual context and proximity restrictions. Thus, the proposed framework overcomes this restriction by using word and sentence embeddings and ensembling the output. Two social media datasets, the Headlines dataset and the Twitter dataset, are used to test the proposed model. Headlines datasets have long sentences, while Twitter datasets have short tweets. The accuracy of 81.4&#x0025; and 88.9&#x0025; are achieved, respectively. The accuracy metric values are better than previous state-of-art frameworks.
Subgraph Representation Learning with Hard Negative Samples for Inductive Link Prediction

The inductive link prediction in knowledge graphs (KGs) is often addressed to induce logical rules that capture entity-independent relational semantics. Recent studies suggest graph representation learning to encode these logical rules within the local subgraph structures. With this approach, the model can have the inductive ability to cope with unseen entities, which is practical for evolving nature of real-world KGs. However, despite the importance of a high-quality negative sample in link prediction, there is currently no method for selecting hard negatives for inductive link prediction. To overcome this limitation, we propose a new sampling method for selecting hard negative samples given a positive triplet. We also propose Subgraph Infomax (SGI), a novel inductive link prediction model, with a newly-proposed training objective that maximizes the mutual information (MI) between the target relation and the enclosing subgraph. We select hard negative samples by using the pretrained MI estimator of SGI. The model is then fine-tuned using the selected hard negative samples. Empirically, we demonstrate superior performances of our model on multiple datasets of the inductive KGC benchmark, showing the enhanced connectivity between the target relation embedding and the subgraph representation.
Adaptive Matching Strategy for Multi-Target Multi-Camera Tracking

Multi-Target Multi-Camera Tracking has a wide range of applications and is the basis for many high-level inference and prediction tasks. How to make the system perform efficiently on a large number of cameras is a crucial research issue. Previous works have proposed many matching strategies to reduce the matching range and improve the matching accuracy. However, these works require human participation when formulating matching strategies, which becomes infeasible as the scale of the camera system increases. To tackle this problem, we propose an adaptive matching strategy to replace manual rules when guiding the matching between cameras. Specifically, we use the Markov decision process to model the tracklets matching problem between cameras. Reinforcement learning and imitation learning are combined to predict a set of cameras where the tracking target might be located. The predicted candidate camera set can be used for intercamera matching and association between tracklets. Moreover, our method can be trained with or without ground truth inter-camera trajectories, making it more practical in real scenarios. We evaluate our method on the city-scale tracking dataset Cityflow, and the proposed method is sufficient to replace manual rules, and finally improve the performance of the overall MTMCT system.
OT Cleaner: Label Correction as Optimal Transport

Datasets with noisy labels present challenges for training Deep Neural Networks (DNNs) with high generalization ability. An direct idea is to correct the noisy labels for robust learning. However, existing label correction methods can not handle with heavy noise or datasets with samples of many categories so well. We explain the reasons and introduce a global label distribution regularization to remedy these deficiencies. With this regularization, we convert the label correction to the Optimal Transport (OT) formulation and propose to utilize a fast version of the Sinkhorn-Knopp algorithm for finding an approximate solution efficiently at scale. Experiments on benchmark datasets with both synthetic and real-world label noise show that the superiority of our OT Cleaner in terms of both training efficiency and classification accuracy. The code is available at: https://github.com/junxia97/OT-Cleaner.
DLD: An Optimized Chinese Speech Recognition Model Based on Deep Learning

Speech recognition technology has played an indispensable role in realizing human-computer intelligent interaction. However, most of the current Chinese speech recognition systems are provided online or offline models with low accuracy and poor performance. To improve the performance of offline Chinese speech recognition, we propose a hybrid acoustic model of deep convolutional neural network, long short-term memory, and deep neural network (DCNN-LSTM-DNN, DLD). This model utilizes DCNN to reduce frequency variation and adds a batch normalization (BN) layer after its convolutional layer to ensure the stability of data distribution, and then use LSTM to effectively solve the gradient vanishing problem. Finally, the fully connected structure of DNN is utilized to efficiently map the input features into a separable space, which is helpful for data classification. Therefore, leveraging the strengths of DCNN, LSTM, and DNN by combining them into a unified architecture can effectively improve speech recognition performance. Our model was tested on the open Chinese speech database THCHS-30 released by the Center for Speech and Language Technology (CSLT) of Tsinghua University, and it was concluded that the DLD model with 3 layers of LSTM and 3 layers of DNN had the best performance, reaching 13.49% of words error rate (WER).
Trajectory and Motion for Agricultural Robot

This paper provides a detailed analysis of the trajectory and motion planning for ground Agricultural-Robot. the approach is based on: i) scanning and analysis of the selected area; ii) obstacle avoidance behavior; and iii) position and image processing system of Agricultural-Robot. The purpose of this paper is to present the goal-directed robot path planning in the presence of uncertainties and obstacles. The proposed concept has been validated on a simulation test bed that is constructed upon a model of typical Agricultural-Robot and race results that clearly demonstrate the effectiveness of the planning system.
Blockchain and IoT in developing Fintech Ecosystem- An assistance to Insurance Industry

PURPOSE- The Covid Pandemic has coerced the insurers to determine how best to meet the demands of their customers, provide service with minimal effort, and achieve their cost efficiency objective. The cost-efficiency objective of the insurance industry should also be aimed at freeing up funds to invest in new technologies and not lose sight of the transformation imperative. The insurance business, despite its size, is underrepresented in the literature. As a result, this paper explains how blockchain technology and IoT might benefit the insurance business. We go over the fundamentals of blockchain and IoT, the most prominent platforms already in use, and a short theoretical description of the insurance sub-processes that both the technologies can positively alter. We also go over the roadblocks that must be overcome in order to properly utilize blockchain technology in the insurance industry.RESEARCH METHODOLOGY- This study provides a qualitative assessment and analysis of journals, articles, and white papers on the implementation of Blockchain and IoT in the Insurance industry, as well as research trends. In addition, the study attempts to identify potential opportunities in the insurance business. The systematic review aims to bring together findings from several fields of study. The goal of this review article is to analyze both the literature sources to comprehend the actual levels of implementation and use cases, as well as to determine the direction in which the insurance industry is now heading in terms of technological adoption.ORIGINALITY- It also covers a wide range of study topics, as well as the most significant articles from the best journals. This paper also covers book chapters, conference papers, journal articles, review papers, white papers, and reports from various organizations.IMPLICATION- The research can prove to be a useful beginning point for new researchers looking for interesting and relevant research on the application and implementation of blockchain and IoT in general insurance.
Improve Few-Shot Voice Cloning Using Multi-Modal Learning

Recently, few-shot voice cloning has achieved a significant improvement. However, most models for few-shot voice cloning are single-modal, and multi-modal few-shot voice cloning has been understudied. In this paper, we propose to use multi-modal learning to improve the few-shot voice cloning performance. Inspired by the recent works on un-supervised speech representation, the proposed multi-modal system is built by extending Tacotron2 with an unsupervised speech representation module. We evaluate our proposed system in two few-shot voice cloning scenarios, namely few-shot text-to-speech (TTS) and voice conversion (VC). Experimental results demonstrate that the proposed multi-modal learning can significantly improve the few-shot voice cloning performance over their counterpart single-modal systems.
Approaches to Digital Forensics in the Age of Big Data

For the time being, big data revaluation has resulted in the creation of new platforms in a variety of industries throughout the previous decade. Meanwhile, the availability of a large dataset may be beneficial in a variety of situations, with the same number of concerns and obstacles emerging in digital forensics research and analysis. In the era of information technology, compute and storage capacity have been reviewed and improved at an almost incomprehensible pace for digital data in a quickly increasing civilization. Volume, intricate interaction of information, and heterogeneity are seen as major concerns in big data research, owing to the large number of resources engaged in a single situation. The current common forensic tools provide amazing accessibility throughout the investigation and study procedure, but there is still a limitation in the fundamental problems of the large data examination and analysis process. The proposed paper will shed light on the current working large data forensics zone&#x0027;s strategy to turning inside out the challenges in this forensics standard to design the framework of the investigation process for criminal activities using best digital forensic instruments.
Cost Effective MLaaS Federation: A Combinatorial Reinforcement Learning Approach

With the advancement of deep learning techniques, major cloud providers and niche machine learning service providers start to offer their cloud-based machine learning tools, also known as machine learning as a service (MLaaS), to the public. According to our measurement, for the same task, these MLaaSes from different providers have varying performance due to the proprietary datasets, models, etc. Federating different MLaaSes together allows us to improve the analytic performance further. However, naively aggregating results from different MLaaSes not only incurs significant momentary cost but also may lead to sub-optimal performance gain due to the introduction of possible false-positive results. In this paper, we propose Armol, a framework to federate the right selection of MLaaS providers to achieve the best possible analytic performance. We first design a word grouping algorithm to unify the output labels across different providers. We then present a deep combinatorial reinforcement learning based-approach to maximize the accuracy while minimizing the cost. The predictions from the selected providers are then aggregated together using carefully chosen ensemble strategies. The real-world trace-driven evaluation further demonstrates that Armol is able to achieve the same accuracy results with 67% less inference cost.