Reviewerly/exHarmony huggingface dataset:
Talking Head from Speech Audio using a Pre-trained Image Generator

We propose a novel method for generating high-resolution videos of talking-heads from speech audio and a single 'identity' image. Our method is based on a convolutional neural network model that incorporates a pre-trained StyleGAN generator. We model each frame as a point in the latent space of StyleGAN so that a video corresponds to a trajectory through the latent space. Training the network is in two stages. The first stage is to model trajectories in the latent space conditioned on speech utterances. To do this, we use an existing encoder to invert the generator, mapping from each video frame into the latent space. We train a recurrent neural network to map from speech utterances to displacements in the latent space of the image generator. These displacements are relative to the back-projection into the latent space of an identity image chosen from the individuals depicted in the training dataset. In the second stage, we improve the visual quality of the generated videos by tuning the image generator on a single image or a short video of any chosen identity. We evaluate our model on standard measures (PSNR, SSIM, FID and LMD) and show that it significantly outperforms recent state-of-the-art methods on one of two commonly used datasets and gives comparable performance on the other. Finally, we report on ablation experiments that validate the components of the model. The code and videos from experiments can be found at https://mohammedalghamdi.github.io/talking-heads-acm-mm
Comparing 1-dimensional and 2-dimensional spectral feature representations in voice pathology detection using machine learning and deep learning classifiers

The present study investigates the use of 1-dimensional (1-D) and 2-dimensional (2-D) spectral feature representations in voice pathology detection with several classical machine learning (ML) and recent deep learning (DL) classifiers.Four popularly used spectral feature representations (static mel-frequency cepstral coefficients (MFCCs), dynamic MFCCs, spectrogram and mel-spectrogram) are derived in both the 1-D and 2-D form from voice signals.Three widely used ML classifiers (support vector machine (SVM), random forest (RF) and Adaboost) and three DL classifiers (deep neural network (DNN), long shortterm memory (LSTM) network, and convolutional neural network (CNN)) are used with the 1-D feature representations.In addition, CNN classifiers are built using the 2-D feature representations.The popularly used HUPA database is considered in the pathology detection experiments.Experimental results revealed that using the CNN classifier with the 2-D feature representations yielded better accuracy compared to using the ML and DL classifiers with the 1-D feature representations.The best performance was achieved using the 2-D CNN classifier based on dynamic MFCCs that showed a detection accuracy of 81%.
Disentanglement Dynamics in Nonequilibrium Environments

We theoretically study the non-Markovian disentanglement dynamics of a two-qubit system coupled to nonequilibrium environments with nonstationary and non-Markovian random telegraph noise statistical properties. The reduced density matrix of the two-qubit system can be expressed as the Kraus representation in terms of the tensor products of the single qubit Kraus operators. We derive the relation between the entanglement and nonlocality of the two-qubit system which are both closely associated with the decoherence function. We identify the threshold values of the decoherence function to ensure the existences of the concurrence and nonlocal quantum correlations for an arbitrary evolution time when the two-qubit system is initially prepared in the composite Bell states and the Werner states, respectively. It is shown that the environmental nonequilibrium feature can suppress the disentanglement dynamics and reduce the entanglement revivals in non-Markovian dynamics regime. In addition, the environmental nonequilibrium feature can enhance the nonlocality of the two-qubit system. Moreover, the entanglement sudden death and rebirth phenomena and the transition between quantum and classical nonlocalities closely depend on the parameters of the initial states and the environmental parameters in nonequilibrium environments.
A conditional GAN-based approach for enhancing transfer learning performance in few-shot HCR tasks

Abstract Supervised learning with the restriction of a few existing training samples is called Few-Shot Learning. FSL is a subarea that puts deep learning performance in a gap, as building robust deep networks requires big training data. Using transfer learning in FSL tasks is an acceptable way to avoid the challenge of building new deep models from scratch. Transfer learning methodology considers borrowing the architecture and parameters of a previously trained model on a large-scale dataset and fine-tuning it for low-data target tasks. But practically, fine-tuning pretrained models in target FSL tasks suffers from overfitting. The few existing samples are not enough to correctly adjust the pretrained model’s parameters to provide the best fit for the target task. In this study, we consider mitigating the overfitting problem when applying transfer learning in few-shot Handwritten Character Recognition (HCR) tasks. A data augmentation approach based on Conditional Generative Adversarial Networks is introduced. CGAN is a generative model that can create artificial instances that appear more real and indistinguishable from the original samples. CGAN helps generate extra samples that hold the possible variations of human handwriting instead of applying traditional image transformations. These transformations are low-level, data-independent operations, and only produce augmented samples with limited diversity. The introduced approach was evaluated in fine-tuning the three pretrained models: AlexNet, VGG-16, and GoogleNet. The results show that the samples generated by CGAN can enhance transfer learning performance in few-shot HCR tasks. This is by achieving model fine-tuning with fewer epochs and by increasing the model’s $$F1-score$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>F</mml:mi> <mml:mn>1</mml:mn> <mml:mo>-</mml:mo> <mml:mi>s</mml:mi> <mml:mi>c</mml:mi> <mml:mi>o</mml:mi> <mml:mi>r</mml:mi> <mml:mi>e</mml:mi> </mml:mrow> </mml:math> and decreasing the Generalization Error $$(E_{test})$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mo>(</mml:mo> <mml:msub> <mml:mi>E</mml:mi> <mml:mrow> <mml:mi>test</mml:mi> </mml:mrow> </mml:msub> <mml:mo>)</mml:mo> </mml:mrow> </mml:math> .
From Battery Manufacturing to Smart Grids: Towards a Metaverse for the Energy Sciences**

Abstract We present two digital‐based educative games aiming to engage students and the general public with battery sciences. The first one is a multiscale simulator in Mixed Reality of a battery‐powered Electric Vehicle (EV) interacting with an Electrical Grid. One of the players drives the EV in a Virtual Reality (VR) environment where the EV can be recharged, and the other players control the electricity produced, distributed, consumed and stored by interacting with 3D‐printed devices. The second educative game is a digital twin of a lithium ion battery manufacturing pilot line, which can be played from an Internet Browser or by using VR hardware. The key steps of the manufacturing process of cylindrical cells are represented in an interactive way. In this concept, we reviewed the working principles of our games, their implications for motivation, engagement and learning, and why they pave the way towards new ways of collaborative R&amp;D in the battery field.
Explainable machine learning for breakdown prediction in high gradient rf cavities

The occurrence of vacuum arcs or radio frequency (rf) breakdowns is one of the most prevalent factors limiting the high-gradient performance of normal conducting rf cavities in particle accelerators. In this paper, we search for the existence of previously unrecognized features related to the incidence of rf breakdowns by applying a machine learning strategy to high-gradient cavity data from CERN's test stand for the Compact Linear Collider (CLIC). By interpreting the parameters of the learned models with explainable artificial intelligence (AI), we reverse-engineer physical properties for deriving fast, reliable, and simple rule--based models. Based on 6 months of historical data and dedicated experiments, our models show fractions of data with a high influence on the occurrence of breakdowns. Specifically, it is shown that the field emitted current following an initial breakdown is closely related to the probability of another breakdown occurring shortly thereafter. Results also indicate that the cavity pressure should be monitored with increased temporal resolution in future experiments, to further explore the vacuum activity associated with breakdowns.
Verifiable Delay Function and Its Blockchain-Related Application: A Survey

The concept of verifiable delay functions has received attention from researchers since it was first proposed in 2018. The applications of verifiable delay are also widespread in blockchain research, such as: computational timestamping, public random beacons, resource-efficient blockchains, and proofs of data replication. This paper introduces the concept of verifiable delay functions and systematically summarizes the types of verifiable delay functions. Firstly, the description and characteristics of verifiable delay functions are given, and weak verifiable delay functions, incremental verifiable delay functions, decodable verifiable delay functions, and trapdoor verifiable delay functions are introduced respectively. The construction of verifiable delay functions generally relies on two security assumptions: algebraic assumption or structural assumption. Then, the security assumptions of two different verifiable delay functions are described based on cryptography theory. Secondly, a post-quantum verifiable delay function based on super-singular isogeny is introduced. Finally, the paper summarizes the blockchain-related applications of verifiable delay functions.
AVQA: A Dataset for Audio-Visual Question Answering on Videos

Audio-visual question answering aims to answer questions regarding both audio and visual modalities in a given video, and has drawn increasing research interest in recent years. However, there have been no appropriate datasets for this challenging task on videos in real-life scenarios so far. They are either designed with questions containing only visual clues without taking any audio information into account, or considering audio with restrictions to specific scenarios, such as panoramic videos and videos about music performances. In this paper, to overcome the limitations of existing datasets, we introduce AVQA, a new audio-visual question answering dataset on videos in real-life scenarios. We collect 57,015 videos from daily audio-visual activities and 57,335 specially-designed question-answer pairs relying on clues from both modalities, where information contained in a single modality is insufficient or ambiguous. Furthermore, we propose a Hierarchical Audio-Visual Fusing module to model multiple semantic correlations among audio, visual, and text modalities and conduct ablation studies to analyze the role of different modalities on our datasets. Experimental results show that our proposed method significantly improves the audio-visual question answering performance over various question types. Therefore, AVQA can provide an adequate testbed for the generation of models with a deeper understanding of multimodal information on audio-visual question answering in real-life scenarios. (The dataset is available at https://mn.cs.tsinghua.edu.cn/avqa)
ASRS-CMFS vs. RoBERTa: Comparing Two Pre-Trained Language Models to Predict Anomalies in Aviation Occurrence Reports with a Low Volume of In-Domain Data Available

We consider the problem of solving Natural Language Understanding (NLU) tasks characterized by domain-specific data. An effective approach consists of pre-training Transformer-based language models from scratch using domain-specific data before fine-tuning them on the task at hand. A low domain-specific data volume is problematic in this context, given that the performance of language models relies heavily on the abundance of data during pre-training. To study this problem, we create a benchmark replicating realistic field use of language models to classify aviation occurrences extracted from the Aviation Safety Reporting System (ASRS) corpus. We compare two language models on this new benchmark: ASRS-CMFS, a compact model inspired from RoBERTa, pre-trained from scratch using only little domain-specific data, and the regular RoBERTa model, with no domain-specific pre-training. The RoBERTa model benefits from its size advantage, while the ASRS-CMFS benefits from the pre-training from scratch strategy. We find no compelling statistical evidence that RoBERTa outperforms ASRS-CMFS, but we show that ASRS-CMFS is more compute-efficient than RoBERTa. We suggest that pre-training a compact model from scratch is a good strategy for solving domain-specific NLU tasks using Transformer-based language models in the context of domain-specific data scarcity.
R <scp>e</scp> LAX: Reinforcement Learning Agent Explainer for Arbitrary Predictive Models

Counterfactual examples (CFs) are one of the most popular methods for attaching post-hoc explanations to machine learning (ML) models. However, existing CF generation methods either exploit the internals of specific models or depend on each sample's neighborhood, thus they are hard to generalize for complex models and inefficient for large datasets. This work aims to overcome these limitations and introduces ReLAX, a model-agnostic algorithm to generate optimal counterfactual explanations. Specifically, we formulate the problem of crafting CFs as a sequential decision-making task and then find the optimal CFs via deep reinforcement learning (DRL) with discrete-continuous hybrid action space. Extensive experiments conducted on several tabular datasets have shown that ReLAX outperforms existing CF generation baselines, as it produces sparser counterfactuals, is more scalable to complex target models to explain, and generalizes to both classification and regression tasks. Finally, to demonstrate the usefulness of our method in a real-world use case, we leverage CFs generated by ReLAX to suggest actions that a country should take to reduce the risk of mortality due to COVID-19. Interestingly enough, the actions recommended by our method correspond to the strategies that many countries have actually implemented to counter the COVID-19 pandemic.
Defining Digital Literacy: A Case Study of Australian Universities

&#x0D; &#x0D; &#x0D; The contemporary appearance of the term “digital literacy” on university websites suggests institutional interests on digital literacy that focus not only on the development of technology skills but also cognitive and attitudinal aspects in student development. This paper presents an exploration of institutional conceptions of digital literacy based on document analysis of university published information online. The investigation involved universities located in Australia as embedded case studies (n=42). Evidence suggests variations in defining this term and shows that universities have diverse goals as espoused in their corresponding definitions of digital literacy, from developing technical skills of using and understanding technology, to possessing a set of capabilities for living, learning and working in an increasingly digital world. For universities who enumerated a coherent account of digital literacy, the results indicate that their practices of promoting the development of digital literacy are entrenched in their espoused intent of graduate outcomes. The paper concludes with curricular and pedagogical implications in preparing and assisting students for the challenges of 21st century living, learning and working.&#x0D; &#x0D; &#x0D;
VBlock: A Blockchain-Based Tamper-Proofing Data Protection Model for Internet of Vehicle Networks

The rapid advancement of the Internet of Vehicles (IoV) has led to a massive growth in data received from IoV networks. The cloud storage has been a timely service that provides a vast range of data storage for IoV networks. However, existing data storage and access models used to manage and protect data in IoV networks have proven to be insufficient. They are centralized and usually accompanied by a lack of trust, transparency, security, immutability, and provenance. In this paper, we propose VBlock, a blockchain-based system that addresses the issues of illegal modification of outsourced vehicular data for smart city management and improvement. We introduce a novel collusion-resistant model for outsourcing data to cloud storage that ensures the network remains tamper-proof, has good data provenance and auditing, and solves the centralized problems prone to the single point of failure. We introduced a key revocation mechanism to secure the network from malicious nodes. We formally define the system model of VBlock in the setting of a consortium blockchain. Our simulation results and security analysis show that the proposed model provides a strong security guarantee with high efficiency and is practicable in the IoV environment.
Cross-domain structure learning for visual data recognition

Unsupervised domain adaptation methods are used to train an effect model by utilizing available knowledge from a labeled source domain for solving tasks in an unlabeled target domain. The most difficult challenge is determining methods to reduce distribution discrepancies and extract the largest number of domain-invariant features between the source and target domains to improve model performance. With the aim of minimizing the domain shift and maximizing domain-invariant feature extraction, we propose a cross-domain structure learning (CDSL) method for visual data recognition, which incorporates global distribution alignment and local discriminative structure preservation to capture the common, underlying features between domains. Specifically, we design a simple but effective classwise structure learning strategy with a specific compactness hierarchy to promote intraclass knowledge transfer and reduce the risk of negative transfer between domains. We also extend CDSL to different kinds of kernelization to address complex situations in the real world. Extensive experiments on several visual data benchmarks demonstrate the effectiveness of our proposed method.
A New Image Encryption Technique Based on Sine Map, Chaotic Tent Map, and Circulant Matrices

Image encryption is one of the sensitive areas used recently to save information over the Internet and confirms the security on a higher level. A new cryptosystem has been proposed for the encoding and decoding of images using sine map, chaotic tent map, and circulant matrices. The process involves three stages. The sine map is used for the permutation phase. In the substitution phase, the Hill cipher method together with prime circulant matrix are used and a chaotic tent map is used in the diffusion phase. The algorithm becomes impenetrable to resist differential- and statistical-type attacks. The algorithm's key space is sufficient in size to withstand brute-force attacks. This symmetric image encryption algorithm indicates good results for correlation analysis, key space analysis, entropy analysis, key sensitivity analysis, histogram analysis, number of pixel change rate (NPCR), unified average changing intensity (UACI), and peak signal-to-noise ratio (PSNR). From all these simulation results, we come to know that the constructed image encryption cryptosystem is systematic and vigorous, supplying better protection for image encryption.
Role of Artificial Intelligence and Machine Learning in Prediction, Diagnosis, and Prognosis of Cancer

Cancer is one of the most devastating, fatal, dangerous, and unpredictable ailments.To reduce the risk of fatality in this disease, we need some ways to predict the disease, diagnose it faster and precisely, and predict the prognosis accurately.The incorporation of artificial intelligence (AI), machine learning (ML), and deep learning (DL) algorithms into the healthcare system has already proven to work wonders for patients.Artificial intelligence is a simulation of intelligence that uses data, rules, and information programmed in it to make predictions.The science of machine learning (ML) uses data to enhance performance in a variety of activities and tasks.A bigger family of machine learning techniques built on artificial neural networks and representation learning is deep learning (DL).To clarify, we require AI, ML, and DL to predict cancer risk, survival chances, cancer recurrence, cancer diagnosis, and cancer prognosis.All of these are required to improve patient's quality of life, increase their survival rates, decrease anxiety and fear to some extent, and make a proper personalized treatment plan for the suffering patient.The survival rates of people with diffuse large B-cell lymphoma (DLBCL) can be forecasted.Both solid and non-solid tumors can be diagnosed precisely with the help of AI and ML algorithms.The prognosis of the disease can also be forecasted with AI and its approaches like deep learning.This improvement in cancer care is a turning point in advanced healthcare and will deeply impact patient's life for good.
An empirical study on how humans appreciate automated counterfactual explanations which embrace imprecise information

The explanatory capacity of interpretable fuzzy rule-based classifiers is usually limited to offering explanations for the predicted class only. A lack of potentially useful explanations for non-predicted alternatives can be overcome by designing methods for the so-called counterfactual reasoning. Nevertheless, state-of-the-art methods for counterfactual explanation generation require special attention to human evaluation aspects, as the final decision upon the classification under consideration is left for the end user. In this paper, we first introduce novel methods for qualitative and quantitative counterfactual explanation generation. Then, we carry out a comparative analysis of qualitative explanation generation methods operating on (combinations of) linguistic terms as well as a quantitative method suggesting precise changes in feature values. Then, we propose a new metric for assessing the perceived complexity of the generated explanations. Further, we design and carry out two human evaluation experiments to assess the explanatory power of the aforementioned methods. As a major result, we show that the estimated explanation complexity correlates well with the informativeness, relevance, and readability of explanations perceived by the targeted study participants. This fact opens the door to using the new automatic complexity metric for guiding multi-objective evolutionary explainable fuzzy modeling in the near future.