Reviewerly/exHarmony huggingface dataset:
Integer-Only Zero-Shot Quantization for Efficient Speech Recognition

End-to-end neural network models achieve improved performance on various automatic speech recognition (ASR) tasks. However, these models perform poorly on edge hardware due to large memory and computation requirements. While quantizing model weights and/or activations to low-precision can be a promising solution, previous research on quantizing ASR models is limited. In particular, the previous approaches use floating-point arithmetic during inference and thus they cannot fully exploit efficient integer processing units. Moreover, they require training and/or validation data during quantization, which may not be available due to security or privacy concerns. To address these limitations, we propose an integer-only, zeroshot quantization scheme for ASR models. In particular, we generate synthetic data whose runtime statistics resemble the real data, and we use it to calibrate models during quantization. We apply our method to quantize QuartzNet, Jasper, and Conformer and show negligible WER degradation as compared to the full-precision baseline models, even without using any data. Moreover, we achieve up to 2.35× speedup on a T4 GPU and 4× compression rate, with a modest WER degradation of <1% with INT8 quantization.
Listen, Know and Spell: Knowledge-Infused Subword Modeling for Improving ASR Performance of OOV Named Entities

Automatic speech recognition (ASR) is increasingly being used in specialized domains such as medical ASR and news transcription. Owing to the lack of high quality annotated speech data in such domains, off-the-shelf models are commonly employed by fine-tuning on domain-specific data. This poses a significant challenge in transcribing long-tail expressions and out-of-vocabulary (OOV) named entities. On the other hand, readily available knowledge graphs (KGs) provide semantically structured knowledge for such domain-specific named entities. In this work, we propose the Knowledge-Infused Subword Model (KISM), a novel technique for incorporating semantic context from KGs into the ASR pipeline for improving the performance of OOV named entities. Our experiments show that KISM improves OOV recall of an ASR model by 4.58% (absolute) for named entities that were not seen during training.
An Efficient DP-SGD Mechanism for Large Scale NLU Models

Recent advances in deep learning have drastically improved performance on many Natural Language Understanding (NLU) tasks. However, the data used to train NLU models may contain private information such as addresses or phone numbers, particularly when drawn from human subjects. It is desirable that underlying models do not expose private information contained in the training data. Differentially Private Stochastic Gradient Descent (DP-SGD) has been proposed as a mechanism to build privacy-preserving models. However, DP-SGD can be prohibitively slow to train. In this work, we propose a more efficient DP-SGD for training using a GPU infrastructure and apply it to fine-tuning models based on LSTM and transformer architectures. We report faster training times, alongside accuracy, theoretical privacy guarantees and success of Membership inference attacks for our models and observe that fine-tuning with proposed variant of DP-SGD can yield competitive models without significant degradation in training time and improvement in privacy protection. We also make observations such as looser theoretical ϵ, δ can translate into significant practical privacy gains.
An efficient heuristic algorithm for software module clustering optimization

In the lifecycle of software projects, maintenance tasks usually entail 75% of the total costs, where most efforts are spent in understanding the program. To improve the maintainability of software projects, the code is often divided into components, which are then grouped in different modules following good design principles, lowering coupling and increasing cohesion. The Software Module Clustering Problem (SMCP) is an optimization problem that looks for maximizing the modularity of software projects in the context of Search-Based Software Engineering. In the SMCP, projects are often modeled as graphs. Therefore, the SMCP can be interpreted as a graph partitioning problem, which is proved to be NP-hard. In this work, we propose a new heuristic algorithm for software modularization, based on a Greedy Randomized Adaptive Search Procedure with Variable Neighborhood Descent. We present a three-fold categorization of neighborhoods for the SMCP and leverage domain-specific information to filter unpromising solutions. Our proposal has been successfully tested over a dataset of real software projects, outperforming the previous state-of-the-art approach in terms of Modularization Quality in very short computing times. Therefore, it could be integrated in software development tools to improve the quality of software projects in real time.
SoK: TEE-Assisted Confidential Smart Contract

The blockchain-based smart contract lacks privacy, since the contract state and instruction code are exposed to the public. Combining smart-contract execution with Trusted Execution Environments provides an efficient solution, called TEE-assisted smart contracts (TCSC), for protecting the confidentiality of contract states. However, the combination approaches are varied, and a systematic study is absent. Newly released systems may fail to draw upon the experience learned from existing protocols, such as repeating known design mistakes or applying TEE technology in insecure ways. In this paper, we first investigate and categorize existing systems into two types: the layer-one solution and the layer-two solution. Then, we establish an analysis framework to capture their common aspects, covering desired properties (for contract services), threat models, and security considerations (for underlying systems). Based on our taxonomy, we identify their ideal functionalities, and uncover fundamental flaws and challenges in each specification’s design. We believe that this work would provide a guide for the development of TEE-assisted smart contracts, as well as a framework to evaluate future TCSC systems.
Mobile Application Based Secured Smart Museum for COVID-19 Pandemic Situation

The spread of COVID-19 disease has reduced the visitors count to various public places like parks, libraries and museums. Even though Indian Government has relaxed the rules for the public to visit these places during the early 2021, people deny visiting those places due to the fear created by the impact of the disease. This leads to huge revenue loss due to lack of visitors. In order to solve this issue, a safer visiting procedure through a Mobile Application based Secured Smart Museum (MSSM) has been provided. This system strictly monitors the entry of the visitors through two way screening process which ensures the safety of the visitors at the museum. Two-way screening process involves the measuring the temperature of the visitors with the support of IR temperature sensor and monitoring the availability of the face mask with the help of Open Computer Vision. The system also facilitates the users to book the ticket through Mobile Application. This system also alerts the museum's housekeeping department to clean and sanitize the museum based upon the visitors count. In addition to this, our system facilitates the visitors to gain the knowledge about the contents available in the museum through mobile application itself in their own preferred language.
Improving Recognition-Synthesis Based any-to-one Voice Conversion with Cyclic Training

In recognition-synthesis based any-to-one voice conversion (VC), an automatic speech recognition (ASR) model is employed to extract content-related features and a synthesizer is built to predict the acoustic features of the target speaker from the content-related features of any source speakers at the conversion stage. Since source speakers are unknown at the training stage, we have to use the content-related features of the target speaker to estimate the parameters of the synthesizer. This inconsistency between conversion and training stages constrains the speaker similarity of converted speech. To address this issue, a cyclic training method is proposed in this paper. This method designs pseudo-source acoustic features, which are generated by converting the training data of the target speaker towards multiple speakers in a reference corpus. Then, these pseudo-source acoustic features are used as the input of the synthesizer at the training stage to predict the acoustic features of the target speaker and a cyclic reconstruction loss is derived. Experimental results show that our proposed method achieved more consistent accuracy of acoustic feature prediction for various source speakers than the baseline method. It also achieved better similarity of converted speech, especially for the pairs of source and target speakers with distant speaker characteristics.
Speaker Generation

This work explores the task of synthesizing speech in non-existent human-sounding voices. We call this task "speaker generation", and present TacoSpawn, a system that performs competitively at this task. TacoSpawn is a recurrent attention-based text-to-speech model that learns a distribution over a speaker embedding space, which enables sampling of novel and diverse speakers. Our method is easy to implement, and does not require transfer learning from speaker ID systems. We present objective and subjective metrics for evaluating performance on this task, and demonstrate that our proposed objective metrics correlate with human perception of speaker similarity. Audio samples are available on our demo page <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Multiple Kernel K-Means Clustering with Simultaneous Spectral Rotation

Multiple kernel k-means clustering (MKKM) and its variants have been thoroughly studied over the past decades. However, most existing models utilize a spectrum-based two-step approach to solve the clustering objective, which may deviate from the final cluster labels and lead to suboptimal performance. To address this issue, we elaborate a novel MKKMSR framework that simultaneously optimizes the discrete and continuous cluster labels by incorporating spectral rotation into MKKM. In addition, the proposed model can be easily integrated with other MKKM models to boost their performance. What’s more, an efficient alternative algorithm is proposed to solve the joint optimization problem. Extensive experiments on real-world datasets demonstrate the superiorities of the proposed framework.
Simultaneous 3D measurement for infrared chips with speckle interferometry

• A compact device that is able to simultaneously measure full-field three dimensional deformation of an object has been developed. • A real-time computing scheme enabling quantitative characterization of 3D deformation is proposed. • The 3D deformation of the infrared sensor due to variations of temperature is characterized. • The precision and robustness of the developed techniques have been validated via experiments and simulations. Temperature-induced deformation in the infrared detector chip is a key index for evaluating image aberration and stress concentration. Development of an evaluation means for simultaneous characterization of 3D deformation is of great interest to engineering societies. In this study, an integrated device based on speckle interferometry has been developed for evaluation of the three-dimensional (3D) deformation of the infrared detector chip with variations of temperature. With the use of a 3CCD color camera and a group of optical fibers, the 3D displacement components of the detector surface are measured simultaneously during the cooling process. A temporal phase shifting scheme based on GPU parallel computing has been proposed to enable quantitatively real-time 3D measurement in three image layers at a rate of 20 fps. The integrated device has been validated with a benchmark test. The experimental results show a good agreement with the theoretical solution validating the stability and accuracy of the experimental device. The developed speckle interferometry device provides an accurate means in simultaneously evaluating the 3D deformation of the infrared detector chip due to temperature variations.
Unsupervised Anomaly Detection for Container Cloud Via BILSTM-Based Variational Auto-Encoder

The appearance of container technology has profoundly changed the development and deployment of multi-tier distributed applications. However, the imperfect system resource isolation features and the kernel-sharing mechanism will introduce significant security risks to the container-based cloud. In this paper, we propose a real-time unsupervised anomaly detection system for monitoring system calls in container cloud via BiLSTM-based variational auto-encoder (VAE). Our proposed BiLSTM-based VAE network leverages the generative characteristics of VAE to learn the robust representations of normal patterns by reconstruction probabilities while being sensitive to long-term dependencies. Our evaluations using real-world datasets show that the BiLSTM-based VAE network achieves excellent detection performance without introducing significant running performance overhead to the container platform.
Groupwise Query Performance Prediction with BERT

While large-scale pre-trained language models like BERT have advanced the state-of-the-art in IR, its application in query performance prediction (QPP) is so far based on pointwise modeling of individual queries. Meanwhile, recent studies suggest that the cross-attention modeling of a group of documents can effectively boost performances for both learning-to-rank algorithms and BERT-based re-ranking. To this end, a BERT-based groupwise QPP model is proposed, in which the ranking contexts of a list of queries are jointly modeled to predict the relative performance of individual queries. Extensive experiments on three standard TREC collections showcase effectiveness of our approach. Our code is available at https://github.com/VerdureChen/Group-QPP .
Credit Card Fraud Detection using AdaBoost Algorithm in Comparison with Various Machine Learning Algorithms to Measure Accuracy, Sensitivity, Specificity, Precision and F-score

Credit card fraud detection is a critical problem for any credit card issuing banks. The AdaBoost classifier is used in this study to identify fraudulent transactions. By comparing the proposed algorithm with Naive Bayes, logistic regression, ANN and decision tree algorithms the efficiency of the algorithm is evaluated. A total of 2,84,807 transactions are divided into two subsets: a training dataset [n=2,27,845 (80%)] and a test dataset [n=56,962 (20%)] (0.8 g power). Out of 2,84,S07 transactions in the dataset, 492 transactions are fraud transactions. To detect the credit card frauds Adaboost algorithm is used and various machine learning algorithms are compared with it for performance evaluation. To determine the performance of algorithms, metrics such as accuracy, sensitivity, specificity, precision, and f-score are estimated. The detection accuracies of AdaBoost, Naive Bayes, logistic regression, ANN and decision tree algorithms are 99.43%, 90.93%, 95.35%, 94.81% and 94.81% respectively. The AdaBoost algorithm obtained an f-score of 99.48% with significance value p<0.05. From the qualitative analysis, it is observed that the proposed AdaBoost algorithm performed significantly better than the Naive Bayes, logistic regression, ANN and decision tree algorithms in detecting credit card frauds.
Cats: Complementary CNN and Transformer Encoders for Segmentation

Recently, deep learning methods have achieved state-of-the-art performance in many medical image segmentation tasks. Many of these are based on convolutional neural networks (CNNs). For such methods, the encoder is the key part for global and local information extraction from input images; the extracted features are then passed to the decoder for predicting the segmentations. In contrast, several recent works show a superior performance with the use of transformers, which can better model long-range spatial dependencies and capture low-level details. However, transformer as sole encoder underperforms for some tasks where it cannot efficiently replace the convolution based encoder. In this paper, we propose a model with double encoders for 3D biomedical image segmentation. Our model is a U-shaped CNN augmented with an independent transformer encoder. We fuse the information from the convolutional encoder and the transformer, and pass it to the decoder to obtain the results. We evaluate our methods on three public datasets from three different challenges: BTCV, MoDA and Decathlon. Compared to the state-of-theart models with and without transformers on each task, our proposed method obtains higher Dice scores across the board.
Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers

Automatic Speech Recognition (ASR) researchers are turning their attention towards supporting low-resource languages, such as isiXhosa or Marathi, with only limited training resources. We report and reflect on collaborative research across ASR & HCI to situate ASR-enabled technologies to suit the needs and functions of two communities of low-resource language speakers, on the outskirts of Cape Town, South Africa and in Mumbai, India. We build on longstanding community partnerships and draw on linguistics, media studies and HCI scholarship to guide our research. We demonstrate diverse design methods to: remotely engage participants; collect speech data to test ASR models; and ultimately field-test models with users. Reflecting on the research, we identify opportunities, challenges, and use-cases of ASR, in particular to support pervasive use of WhatsApp voice messaging. Finally, we uncover implications for collaborations across ASR & HCI that advance important discussions at CHI surrounding data, ethics, and AI.
Zoom Obscura: Counterfunctional Design for Video-Conferencing

This paper reports on Zoom Obscura – an artist-based design research project, responding to the ubiquity of video-conferencing as a technical and cultural phenomenon throughout the Covid-19 pandemic. As enterprise software, such as Zoom, rapidly came to mediate even the most personal and intimate interactions, we supported and collaborated with seven independent artists to explore technical and creative interventions in video-conferencing. Our call for participation sought critical interventions that would help users counter, and regain agency in regard to the various ways in which personal data is captured, transmitted and processed in video-conferencing tools. In this design study, we analyse post-hoc how each of the seven projects employed aspects of counterfunctional design to achieve these aims. Each project reveals different avenues and strategies for counterfunctionality in video-conferencing software, as well as opportunities to design critically towards interactions and experiences that challenge existing norms and expectations around these platforms.
Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis

UI designers often correct false affordances and improve the discoverability of features when users have trouble determining if elements are tappable. We contribute a novel system that models the perceived tappability of mobile UI elements with a vision-based deep neural network and helps provide design insights with dataset-level and instance-level explanations of model predictions. Our system retrieves designs from similar mobile UI examples from our dataset using the latent space of our model. We also contribute a novel use of an interpretability algorithm, XRAI, to generate a heatmap of UI elements that contribute to a given tappability prediction. Through several examples, we show how our system can help automate elements of UI usability analysis and provide insights for designers to iterate their designs. In addition, we share findings from an exploratory evaluation with professional designers to learn how AI-based tools can aid UI design and evaluation for tappability issues.
Heterogeneous online/offline signcryption for secure communication in Internet of Things

Internet of things (IoT) is a promising network that combines various information sensing devices with the Internet to realize the interconnection of pervasive things. Security is a pivotal issue to ensure widespread adoption of the IoT. To ensure the security of data transmission, in this paper, we construct a heterogeneous online and offline signcryption from a wireless sensor network (WSN) to an Internet server. In our construction, the WSN is in certificateless cryptosystem (CLC) which does not involve public key certificate management problem in public key infrastructure (PKI) and key escrow problem in identity based cryptosystem (IBC), while the server is in IBC which avoids public key certificate management problem. For signcryption process, the proposed scheme is split into two steps: offline step and online step. In the former step, a lot of time-consuming calculations are completed, but do not involve the message to be transmitted. In the latter step, only some lightweight calculations are involved, but message must be available. Confidentiality, authentication, non-repudiation, and integrity can be achieved in a logical single step. Meanwhile, a formal security proof is given in the random oracle model. In addition, an elaborate performance analysis demonstrates that our proposed scheme has advantages in terms of computational cost and private key size, offline storage, as well as communication overhead. Therefore, our construction can be well applied to the IoT.
Can’t Touch This: Rethinking Public Technology in a COVID-19 Era

What do pedestrian crossings, ATMs, elevators and ticket machines have in common? These are just a few of the ubiquitous yet essential elements of public-space infrastructure that rely on physical buttons or touchscreens; common interactions that, until recently, were considered perfectly safe to perform. This work investigates how we might integrate touchless technologies into public-space infrastructure in order to minimise physical interaction with shared devices in light of the ongoing COVID-19 pandemic. Drawing on an ethnographic exploration into how public utilities are being used, adapted or avoided, we developed and evaluated a suite of technology probes that can be either retrofitted into, or replace, these services. In-situ community deployments of our probes demonstrate strong uptake and provide insight into how hands-free technologies can be adapted and utilised for the public domain; and, in turn, used to inform the future of walk-up-and use public technologies.
An Improved Deep Learning Framework for MR-to-CT Image Synthesis with a New Hybrid Objective Function

There is an emerging interest in radiotherapy treatment planning that uses only magnetic resonance (MR) imaging. Cur-rent clinical workflows rely on computed tomography (CT) images for dose calculation and patient positioning, therefore synthetic CT images need to be derived from MR images. Re-cent efforts for MR-to-CT image synthesis have focused on unsupervised training for ease of data preparation. However, accuracy is more important than convenience. In this paper, we propose a deep learning framework for MR-to-CT image synthesis that is trained in a supervised manner. The pro-posed framework utilizes a new hybrid objective function to enforce visual realism, accurate electron density information, and structural consistency between the MR and CT image domains. Our experiments show that the proposed method (MAE of 68.22, PSNR of 22.28, and FID of 0.73) outperforms the existing unsupervised and supervised techniques in both quantitative and qualitative comparisons.