Reviewerly/exHarmony huggingface dataset:
Out-of-distribution Generalization and Its Applications for Multimedia

Out-of-distribution generalization is becoming a hot research topic in both academia and industry. This tutorial is to disseminate and promote the recent research achievements on out-of-distribution generalization as well as their applications on multimedia, which is an exciting and fast-growing research direction in the general field of machine learning and multimedia. We will advocate novel, high-quality research findings, as well as innovative solutions to the challenging problems in out-of-distribution generalization and its applications for multimedia. This topic is at the core of the scope of ACM Multimedia, and is attractive to MM audience from both academia and industry.
Local-Tetra-Patterns for Face Recognition Encoded on Spatial Pyramid Matching

Face recognition is a big challenge in the research field with a lot of problems like misalignment, illumination changes, pose variations, occlusion, and expressions. Providing a single solution to solve all these problems at a time is a challenging task. We have put some effort to provide a solution to solving all these issues by introducing a face recognition model based on local tetra patterns and spatial pyramid matching. The technique is based on a procedure where the input image is passed through an algorithm that extracts local features by using spatial pyramid matching and max-pooling. Finally, the input image is recognized using a robust kernel representation method using extracted features. The qualitative and quantitative analysis of the proposed method is carried on benchmark image datasets. Experimental results showed that the proposed method performs better in terms of standard performance evaluation parameters as compared to state-of-the-art methods on AR, ORL, LFW, and FERET face recognition datasets.
EDUCATION 4.0: NEW COMPETENCES FOR THE DIGITAL ECONOMY

Education 4.0 is a conceptually new approach to learning that meets the challenges of the digital revolution, aimed at transforming the future of education with the help of advanced technologies and automation. To integrate into the new digital space, it is necessary to revise the traditional educational paradigms with a futuristic approach. Students should acquire the skills necessary in the era of a dynamically changing technological environment. In this context, the article considers the main directions of digital transformation of education, namely, the modification and integration of traditional models of education. Also, within the framework of the study, the mechanisms of introducing digital technologies into the educational process were isolated, using the example of the Branch of the State budgetary educational institution of Higher Education «Stavropol State Pedagogical Institute»
Towards Lifelong Learning of Multilingual Text-to-Speech Synthesis

This work presents a lifelong learning approach to train a multilingual Text-To-Speech (TTS) system, where each language was seen as an individual task and was learned sequentially and continually. It does not require pooled data from all languages altogether, and thus alleviates the storage and computation burden. One of the challenges of lifelong learning methods is "catastrophic forgetting": in TTS scenario it means that model performance quickly degrades on previous languages when adapted to a new language. We approach this problem via a data-replay-based lifelong learning method. We formulate the replay process as a supervised learning problem, and propose a simple yet effective dual-sampler framework to tackle the heavily language-imbalanced training samples. Through objective and subjective evaluations, we show that this supervised learning formulation outperforms other gradient-based and regularization-based lifelong learning methods, achieving 43% Mel-Cepstral Distortion reduction compared to a fine-tuning baseline.
EventBERT: A Pre-Trained Model for Event Correlation Reasoning

Event correlation reasoning infers whether a natural language paragraph containing multiple events conforms to human common sense. For example, “Andrew was very drowsy, so he took a long nap, and now he is very alert” is sound and reasonable. In contrast, “Andrew was very drowsy, so he stayed up a long time, now he is very alert” does not comply with human common sense. Such reasoning capability is essential for many downstream tasks, such as script reasoning, abductive reasoning, narrative incoherence, story cloze test, etc. However, conducting event correlation reasoning is challenging due to a lack of large amounts of diverse event-based knowledge and difficulty in capturing correlation among multiple events. In this paper, we propose EventBERT, a pre-trained model to encapsulate eventuality knowledge from unlabeled text. Specifically, we collect a large volume of training examples by identifying natural language paragraphs that describe multiple correlated events and further extracting event spans in an unsupervised manner. We then propose three novel event- and correlation-based learning objectives to pre-train an event correlation model on our created training corpus. Experimental results show EventBERT outperforms strong baselines on four downstream tasks, and achieves state-of-the-art results on most of them. Moreover, it outperforms existing pre-trained models by a large margin, e.g., 6.5 ∼ 23%, in zero-shot learning of these tasks.
Deep-RBF Networks for Anomaly Detection in Automotive Cyber-Physical Systems

Deep Neural Networks (DNNs) are widely used in automotive Cyber-Physical Systems (CPSs) to implement autonomy related tasks. However, these networks have exhibited erroneous predictions to anomalous inputs that manifest either due to Out-of-Distribution (OOD) data or adversarial attacks. To detect these anomalies, a separate DNN called assurance monitor is used in parallel to the controller DNN, increasing the resource burden and latency. We hypothesize that a single network that can perform controller predictions and anomaly detection is necessary to reduce the resource requirements. Deep-Radial Basis Function (RBF) networks provide a rejection class alongside the class predictions, which can be used for anomaly detection. However, the use of RBF activation functions limits the applicability of these networks to only classification tasks. In this paper, we discuss the steps involved in detecting anomalies in CPS regression and classification tasks. Further, we design deep-RBF networks using popular DNNs such as NVIDIA DAVE-II and ResNet20 and then use the resulting rejection class for detecting physical and data poison adversarial attacks. We show that the deep-RBF network can effectively detect these attacks with limited resource requirements.
Differentially Private Medical Texts Generation Using Generative Neural Networks

Technological advancements in data science have offered us affordable storage and efficient algorithms to query a large volume of data. Our health records are a significant part of this data, which is pivotal for healthcare providers and can be utilized in our well-being. The clinical note in electronic health records is one such category that collects a patient’s complete medical information during different timesteps of patient care available in the form of free-texts. Thus, these unstructured textual notes contain events from a patient’s admission to discharge, which can prove to be significant for future medical decisions. However, since these texts also contain sensitive information about the patient and the attending medical professionals, such notes cannot be shared publicly. This privacy issue has thwarted timely discoveries on this plethora of untapped information. Therefore, in this work, we intend to generate synthetic medical texts from a private or sanitized (de-identified) clinical text corpus and analyze their utility rigorously in different metrics and levels. Experimental results promote the applicability of our generated data as it achieves more than 80\% accuracy in different pragmatic classification problems and matches (or outperforms) the original text data.
TaxoVec: Taxonomy Based Representation for Web User Profiling

Web users provide rich multi-modal and heterogeneous data. Taking use of the data for extracting the representation of web users is a vital prerequisite of further profiling tasks. In this paper, we study the web user profiling based on the user-entity interaction data integrating the taxonomy information of the entity. To overcome the challenges of the representation of a web user and the distance definition of two web users, we introduce TaxoVec, an interpretable method to represent a user’s interaction history by vectors with the help of the entities’ category information. The TaxoVec is employed in clustering-based web user profiling on their transaction data and geographical data, which provide the information on their interests and activity area. Experiments show that the TaxoVec performs well on representing characteristics of web users.
SI3DP: Source Identification Challenges and Benchmark for Consumer-Level 3D Printer Forensics

This paper lays the foundation for a new 3D content market by establishing a content security framework using databases and benchmarks for in-depth research on source identification of 3D printed objects. The proposed benchmark, SI3DP dataset, offers a more generalized multimedia forensic technique. Assuming that identifying the source of a 3D printing object can arise from various invisible traces occurring in the printing process, we obtain close-up images, full object images from 252 printed objects from 18 different printing setups. We then propose a benchmark with five challenging tasks such as device-level identification and scan-and-reprint detection using the provided dataset. Our baseline shows that the printer type and its attributes can be identified based on the microscopic difference of surface texture. Contrary to the conventional belief that only microscopic views such as close-up images are useful to identify printer model, we also achieved a certain level of performance even at a relatively macroscopic point of view. We then propose a multitask-multimodal architecture for device-level identification task to exploit rich knowledge from different image modality and task. The SI3DP dataset can promote future in-depth research studies related to digital forensics and intellectual property protection.
Validating Gaussian Process Models with Simulation-Based Calibration

Gaussian process priors are a popular choice for Bayesian analysis of regression problems. However, the implementation of these models can be complex, and ensuring that the implementation is correct can be challenging. In this paper we introduce Gaussian process simulation-based calibration, a procedure for validating the implementation of Gaussian process models and demonstrate the efficacy of this procedure in identifying a bug in existing code. We also present a novel application of this procedure to identify when marginalisation of the model hyperparameters is necessary.
Unsupervised Clustering Method to Discriminate Dense Deception Jamming for Surveillance Radar

With the development of digital radiofrequency memory devices, the deception jamming has become a major jamming type in radar electronic counter countermeasures, especially the dense deception jamming. Recent years, some methods using signal features are proposed for specific deception jamming excision scene. It verifies the feasibility that uses signal features to discriminate deception jamming. However, discriminator training and simulation are indeed hard to achievable since no jamming samples can be obtained in advance. In this letter, we focus on the online dense deception jamming discrimination problem without the offline data training. Since deception jamming signals generated by jammers are definitely different from physical backscattering target echoes on signal features and dense jamming signals generated by one jammer has similar signal fingerprint features, an unsupervised clustering method with heuristic feature extraction is proposed. All discrimination processes are data driven and can be executed online without <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information. To evaluate its effectiveness, the proposed method is applied to a group of experimental data collected from a trial <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</i> -band radar. Results corroborate the proposed method and indicate the direction of future research.
Biometric Template Protection for Neural-Network-Based Face Recognition Systems: A Survey of Methods and Evaluation Techniques

As automated face recognition applications tend towards ubiquity, there is a growing need to secure the sensitive face data used within these systems. This paper presents a survey of biometric template protection (BTP) methods proposed for securing face “templates” (images/features) in neural-network-based face recognition systems. The BTP methods are categorised into two types: Non-NN and NN-learned. Non-NN methods use a neural network (NN) as a feature extractor, but the BTP part is based on a non-NN algorithm applied at either image-level or feature-level. In contrast, NN-learned methods specifically employ a NN to learn a protected template from the unprotected face image/features. We present examples of Non-NN and NN-learned face BTP methods from the literature, along with a discussion of the two categories’ comparative strengths and weaknesses. We also investigate the techniques used to evaluate these BTP methods, in terms of the three most common BTP criteria: “recognition accuracy”, “irreversibility”, and “renewability/unlinkability”. As expected, the recognition accuracy of protected face recognition systems is generally evaluated using the same (empirical) techniques employed for evaluating standard (unprotected) biometric systems. On the contrary, most irreversibility and renewability/unlinkability evaluations are found to be based on theoretical assumptions/estimates or verbal implications, with a lack of empirical validation in a practical face recognition context. We recommend, therefore, a greater focus on empirical evaluation strategies, to provide more concrete insights into the irreversibility and renewability/unlinkability of face BTP methods in practice. Additionally, an exploration of the reproducibility of the studied BTP works, in terms of the public availability of their implementation code and evaluation datasets/procedures, suggests that it would currently be difficult for the BTP community to faithfully replicate (and thus validate) most of the reported findings. So, we advocate for a push towards reproducibility, in the hope of furthering our understanding of the face BTP research field.
Efficient Sequence Training of Attention Models Using Approximative Recombination

Sequence discriminative training is a great tool to improve the performance of an automatic speech recognition system. It does, however, necessitate a sum over all possible word sequences, which is intractable to compute in practice. Current state-of-the-art systems with unlimited label context circumvent this problem by limiting the summation to an n-best list of relevant competing hypotheses obtained from beam search.This work proposes to perform (approximative) recombinations of hypotheses during beam search, if they share a common local history. The error that is incurred by the approximation is analyzed and it is shown that using this technique the effective beam size can be increased by several orders of magnitude without significantly increasing the computational requirements. Lastly, it is shown that this technique can be used to effectively perform sequence discriminative training for attention-based encoder-decoder acoustic models on the LibriSpeech task.
Blazed grating pupil remapping and rotational synthesis with computational phase correction for a two-aperture system

Multi-aperture imaging systems can be used to decrease the size, weight, and cost of an imaging system while retaining a resolution related to the diameter of the aperture gain. One solution for phasing apertures in an array uses anamorphic pupil remapping and a computational phase correction algorithm. It is shown that using blazed gratings to remap the aperture fields reduces field dependent contrast and leads to piston corrections for partially coherent illumination. Additionally, rotational aperture synthesis is used to combine two piston corrected images and create an image with resolution increases in more than just the axis of aperture gain.
User-centric Distributed Route Planning in Smart Cities based on Multi-objective Optimization

The realization of edge-based cyber-physical systems (CPS) poses important challenges in terms of performance, robustness, security, etc. This paper examines a novel approach to providing a user-centric adaptive route planning service over a network of Road Side Units (RSUs) in smart cities. The key idea is to adaptively select routing task parameters such as privacy-cloaked area sizes and number of retained intersections to balance processing time, privacy protection level, and route accuracy for privacy-augmented distributed route search while also handling per-query user preferences. This is formulated as an optimization problem with a set of parameters giving the best result for a set of queries given system constraints. Processing Throughput, Privacy Protection, and Travel Time Accuracy were developed as the objective functions to be balanced. A Multi-Objective Genetic Algorithm based technique (NSGA-II) is applied to recover a feasible solution. The performance of this approach was then evaluated using traffic data from Osaka, Japan. Results show good performance of the approach in balancing the aforementioned objectives based on user preferences.
Attention-Based Probabilistic Planning with Active Perception

Attention control is a key cognitive ability for humans to select information relevant to the current task. This paper develops a computational model of attention and an algorithm for attention-based probabilistic planning in Markov decision processes. In attention-based planning, the robot decides to be in different attention modes. An attention mode corresponds to a subset of state variables monitored by the robot. By switching between different attention modes, the robot actively perceives task-relevant information to reduce the cost of information acquisition and processing, while achieving near-optimal task performance. Though planning with attention-based active perception inevitably introduces partial observations, a partially observable MDP formulation makes the problem computational expensive to solve. Instead, our proposed method employs a hierarchical planning framework in which the robot determines what to pay attention to and for how long the attention should be sustained before shifting to other information sources. During the attention sustaining phase, the robot carries out a sub-policy, computed from an abstraction of the original MDP given the current attention. We use an example where a robot is tasked to capture a set of intruders in a stochastic gridworld. The experimental results show that the proposed method enables information- and computation-efficient optimal planning in stochastic environments.
Motion planning of a steam generator mobile tube-inspection robot

Under the influence of nuclear radiation, the reliability of steam generators (SGs) is an important factor in the efficiency and safety of nuclear power plant (NPP) reactors. Motion planning that remotely manipulates an SG mobile tube-inspection robot to inspect SG heat transfer tubes is the mainstream trend of NPP robot development. To achieve motion planning, conditional traversal is usually used for base position optimization, and then the A∗ algorithm is used for path planning. However, the proposed approach requires considerable processing time and has a single expansion during path planning and plan paths with many turns, which decreases the working speed of the robot. Therefore, to reduce the calculation time and improve the efficiency of motion planning, modifications such as the matrix method, improved parent node, turning cost, and improved expanded node were proposed in this study. We also present a comprehensive evaluation index to evaluate the performance of the improved algorithm. We validated the efficiency of the proposed method by planning on a tube sheet with square-type tube arrays and experimenting with Model SG.
Direct Design of Biquad Filter Cascades with Deep Learning by Sampling Random Polynomials

Designing infinite impulse response filters to match an arbitrary magnitude response requires specialized techniques. Methods like modified Yule-Walker are relatively efficient, but may not be sufficiently accurate in matching high order responses. On the other hand, iterative optimization techniques often enable superior performance, but come at the cost of longer run-times and are sensitive to initial conditions, requiring manual tuning. In this work, we address some of these limitations by learning a direct mapping from the target magnitude response to the filter coefficient space with a neural network trained on millions of random filters. We demonstrate our approach enables both fast and accurate estimation of filter coefficients given a desired response. We investigate training with different families of random filters, and find training with a variety of filter families enables better generalization when estimating real-world filters, using head-related transfer functions and guitar cabinets as case studies. We compare our method against existing methods including modified Yule-Walker and gradient descent and show our approach is, on average, both faster and more accurate.
Comparative Analysis of Apriori and ECLAT Algorithm for Frequent Itemset Data Mining

Frequent itemset data mining is the generation of association rules from a transactional dataset. For example, in a supermarket if bread and butter is placed nearby then the customer who chooses bread will also take butter which will increase the sales. There are many algorithms that are used for frequent itemset. Among those algorithms, the two most commonly used algorithms are Apriori and ECLAT algorithm. In this research, our aim is to prove that ECLAT is far better than Apriori for the frequent set mining in experimental and execution time-wise. We evaluate these two algorithms through experiment and also execute the code to find the time. With the dataset that we obtained, we have done problem solving of the two algorithms and the result obtained is Apriori to provide 11 and ECLAT to provide 9 combinations. When the problem solving was done, only a slight difference is detected so this research work is focused on the code of these algorithms. So, the code of these two algorithms is executed with the same dataset and final result of ECLAT and taken 0.2 s faster than Apriori taken 0.4 s.