Reviewerly/exHarmony huggingface dataset:
HATS: An Open Data Set Integrating Human Perception Applied to the Evaluation of Automatic Speech Recognition Metrics

Conventionally, Automatic Speech Recognition (ASR) systems are evaluated on their ability to correctly recognize each word contained in a speech signal. In this context, the word error rate (WER) metric is the reference for evaluating speech transcripts. Several studies have shown that this measure is too limited to correctly evaluate an ASR system, which has led to the proposal of other variants of metrics (weighted WER, BERTscore, semantic distance, etc.). However, they remain system-oriented, even when transcripts are intended for humans. In this paper, we firstly present Human Assessed Transcription Side-by-side (HATS), an original French manually annotated data set in terms of human perception of transcription errors produced by various ASR systems. 143 humans were asked to choose the best automatic transcription out of two hypotheses. We investigated the relationship between human preferences and various ASR evaluation metrics, including lexical and embedding-based ones, the latter being those that correlate supposedly the most with human perception.
Online Speaker Diarization Using Optimized SE-ResNet Architecture

A new approach to speaker diarization (SD) suitable for real-time processing of streamed data is presented in this work. It utilizes a modified residual network with squeeze-and-excitation blocks (SE-ResNet-34) for extraction of speaker embeddings. These speaker embeddings are calculated in an optimized way by using cached buffers and are subsequently used for voice activity detection (VAD) as well as for block-online k-means clustering with a look-ahead mechanism. All these processing steps are first evaluated separately on a development set compiled from recordings of Czech broadcast programs. The whole scheme is then compared to an offline reference approach on various speech databases that are publicly available and include data in various languages. On this data, our method yields results similar to the reference system while operating on a CPU with a low real-time factor (RTF) below 0.1 and a latency of around 5.5 s.
Towards End-to-End Speech-to-Text Summarization

Speech-to-text (S2T) summarization is a time-saving technique for filtering and keeping up with the broadcast news uploaded online on a daily basis. The rise of large language models from deep learning with impressive text generation capabilities has placed the research focus on summarization systems that produce paraphrased compact versions of the document content, also known as abstractive summaries. End-to-end (E2E) modelling of S2T abstractive summarization is a promising approach that offers the possibility of generating rich latent representations that leverage non-verbal and acoustic information, as opposed to the use of only linguistic information from automatically generated transcripts in cascade systems. However, the few literature on E2E modelling of this task fails on exploring different domains, namely broadcast news, which is challenging domain where large and diversified volumes of data are presented to the user every day. We model S2T summarization both with a cascade and an E2E system for a corpus of broadcast news in French. Our novel E2E model leverages external data by resorting to transfer learning from a pre-trained T2T summarizer. Experiments show that both our cascade and E2E abstractive summarizers are stronger than an extractive baseline. However, the performance of the E2E model still lies behind the cascade one, which is object of an extensive analysis that includes future directions to close that gap.
Transfer Learning of Transformer-Based Speech Recognition Models from Czech to Slovak

In this paper, we are comparing several methods of training the Slovak speech recognition models based on the Transformers architecture. Specifically, we are exploring the approach of transfer learning from the existing Czech pre-trained Wav2Vec 2.0 model into Slovak. We are demonstrating the benefits of the proposed approach on three Slovak datasets. Our Slovak models scored the best results when initializing the weights from the Czech model at the beginning of the pre-training phase. Our results show that the knowledge stored in the Cezch pre-trained model can be successfully reused to solve tasks in Slovak while outperforming even much larger public multilingual models.
One Model to Rule Them All: Ranking Slovene Summarizers

Text summarization is an essential task in natural language processing, and researchers have developed various approaches over the years, ranging from rule-based systems to neural networks. However, there is no single model or approach that performs well on every type of text. We propose a system that recommends the most suitable summarization model for a given text. The proposed system employs a fully connected neural network that analyzes the input content and predicts which summarizer should score the best in terms of ROUGE score for a given input. The meta-model selects among four different summarization models, developed for the Slovene language, using different properties of the input, in particular its Doc2Vec document representation. The four Slovene summarization models deal with different challenges associated with text summarization in a less-resourced language. We evaluate the proposed SloMetaSum model performance automatically and parts of it manually. The results show that the system successfully automates the step of manually selecting the best model.
Digital Governance and Religious Populism in Pakistan

This chapter delves into the issue of digital authoritarianism in Pakistan, despite it being a democracy, having a vocal social media landscape. The chapter examines how Pakistan's internet governance institutions have been abused by the government and how laws such as the Prevention of Electronic Crimes Act (PECA) and the Fair Trial Act have legalized and intensified state surveillance programs. The chapter investigates the technical aspects of digital authoritarianism, including the full network governance of the internet in Pakistan. It also highlights the role of religion, particularly Sunni Islam, in justifying and legitimizing digital authoritarianism in the country. Appeals to Islamic morals and religious authority have helped generate support for content blocking, banning of users, and even social media platform bans, which has allowed for restrictive government legislation that curtails citizens' right to free expression, information, and privacy.
Agent modeling of the distributed dynamic systems on the earth’s surface with using the colored and nested Petri nets

The application of formalisms of multi-agent systems, color and nested Petri nets to digital modeling and control of dynamic processes on the Earth’s surface is considered. A GIS-based multi-agent system has been implemented in the form of a two-level color nested Petri net. The system is currently accepted for use as a training simulator.
Pedagogical Innovation Triggered by the Covid-19 Pandemic

Innovation, due to its multidisciplinary properties, favors its application in different areas of human activity. Pedagogy, in turn, with its role in the development of the next generations, takes the place of reference in this study. Thus, the image of pedagogical innovation goes back to an innovation in the educational field that intends, through the transformation and alteration of traditional teaching methodologies, to improve the quality of education through more relevant, engaging and efficient negotiation. In this way, the need to look at pedagogical innovation and its effects in the short and long term is highlighted, and this act attributes tangibility to the effects of innovation in this science, which allows validating its performance and importance in the vicissitudes of the present. Given that the Covid-19 pandemic came to launch the need for change and adaptation in the school environment, pedagogical innovation was driven, and the education system adjusted and fine-tuned its posture in the face of the new reality experienced with the use of distance learning apprenticeship. This application of new distance learning processes, on a scale never seen before, erected doubts regarding their effectiveness, thus raising the issue to be studied in this article. With this, it is sought to verify the effects of e-learning and increase the benefits of this shock on the education system and the very performance of teachers, and the change in these should not be seen as a disease of the pandemic. On the other hand, by understanding whether this pedagogical innovation had a positive impact on student learning, it is intended to open a window of investigation into this topic in the future.
Uncovering the effects of Urmia Lake desiccation on soil chemical ripening using advanced mapping techniques

Urmia Lake in Iran has been gradually drying up since the 2000 s, revealing soil that was once underwater. There is limited research on how this drying process affects the distribution of soil chemical ripening in the lakebed sediments. To investigate this, 131 surface soil samples were collected in July 2017 from six zones based on their distance from the lake's shoreline. Elemental concentrations in the samples were measured using a portable X-ray fluorescence (XRF) spectrometer. Five geochemical indices (GCIs) were calculated: Titanium/Zirconium, Barium/Strontium, Silicon/Aluminium, Silicon/R2O3, and the Darmody index. A remote sensing image (Landsat-8 OLI) from the same sampling period was used as environmental covariates to explain the spatial distribution of GCIs. Random forests (RF) and cubist tree models were employed to model and map these GCIs. The bootstrapping method was used to assess the uncertainty of the maps. Results indicated that the RF model performed better than the cubist model based on the concordance values in the validation dataset. Remotely sensed clay minerals ratio and normalized burn ratio were identified as the most significant covariates in making predictions. The study found that the distribution of all geochemical indices was directly related to their distance from the shoreline. Ba/Sr emerged as the best index for detecting soil chemical ripening in the study area. The index displayed the most significant differences between the drying zones. This research demonstrated that combining satellite imagery with GCIs using the digital soil mapping (DSM) technique can effectively detect soil ripening progression.
Immersive innovations: an examination of the efficacy and evolution of virtual reality in human movement training

Purpose This systematic review, following preferred reporting items for systematic reviews and meta-analysis guidelines, rigorously investigates the emergent role of virtual reality (VR) technology in human movement training. The purpose of this study is to explore the effectiveness and evolution of VR in enhancing movement training experiences. Design/methodology/approach Acknowledging its pivotal role in diverse applications, such as sports and rehabilitation, human movement training is currently experiencing accelerated evolution, facilitated by the proliferation of wearable devices and mobile applications. This review conducted an exhaustive search across five different electronic databases, such as Web of Science, PubMed and ProQuest, resulting in the selection of 69 eligible articles published within the past five years. It also integrates 40 studies into a narrative summary, categorized based on the level of immersion offered by respective VR systems. Findings Enhanced immersion in VR potentially augments the effectiveness of movement training by engendering more realistic and captivating experiences for users. The immersive and interactive environments provided by VR technology enable tailored training experiences accompanied by precise, objective feedback. This review highlights the benefits of VR in human movement training and its potential to revolutionize the way training is conducted. Originality/value This systematic review contributes significantly to the existing literature by providing a comprehensive examination of the efficacy and evolution of VR in human movement training. By organizing the findings based on the level of immersion offered by VR systems, it provides valuable insights into the importance of immersion in enhancing training outcomes. In addition, this study identifies the need for future research focusing on the impacts of VR on learning and performance, as well as strategies to optimize its effectiveness and improve accessibility.
Prediction of a Floating Photovoltaic Generation Utilizing KMA Weather Forecast

This paper proposes a method for predicting the hourly generation of floating photovoltaic by creating a model and inputting weather forecast data. In particular, the paper shows the process of selecting hyperparameter settings that minimize errors, which has not been presented in other papers. The prediction model uses the Long Short-Term Memory, a type of recurrent neural network in machine learning, to predict the hourly generation of floating photovoltaic in Chungju and Hapcheon. Instead of using all the 12 existing data, the model only inputs the irradiation and temperature data that are highly correlated with power generation. The prediction model is then used to forecast the hourly power generation for Chungju and Hapcheon. The results show that the predicted power generation meets the error rate of 5% or less, as set by the Korea Power Exchange, making the machine learning-based prediction of floating solar photovoltaic practical and applicable.
Blockchain-based digital chain of custody multimedia evidence preservation framework for internet-of-things

The Internet of Things (IoT) has made everyday tasks more efficient using smart devices, which are now integral parts of people's daily routines. As a result, there has been a surge in services supporting smart devices across various industries, such as home automation, healthcare, and agriculture, which necessitate users to share their personal information. The number of smart devices continues to grow, increasing the likelihood of digital crimes. Digital forensics can provide an advantage in investigating IoT-related crimes because everyone leaves digital traces. In this study, a blockchain-based evidence-preservation framework is developed for IoT devices called BEvPF-IoT to preserve digital artifacts. The aim is to create a secure environment that prevents third-party manipulation of digital evidence until it is presented in court. The feasibility of the proposal is evaluated through experiments conducted on a blockchain network, focusing on measuring latency, gas consumption, and throughput. The proposed framework also enhances transparency and accountability in the forensic examination of digital multimedia evidence, ultimately ensuring that investigation results can be trusted and relied upon.
Towards Unsupervised Object Detection from LiDAR Point Clouds

In this paper, we study the problem of unsupervised object detection from 3D point clouds in self-driving scenes. We present a simple yet effective method that exploits (i) point clustering in near-range areas where the point clouds are dense, (ii) temporal consistency to filter out noisy unsupervised detections, (iii) translation equivariance of CNNs to extend the auto-labels to long range, and (iv) self-supervision for improving on its own. Our approach, OYSTER (Object Discovery via Spatio-Temporal Refinement), does not impose constraints on data collection (such as repeated traversals of the same location), is able to detect objects in a zero-shot manner without supervised fine-tuning (even in sparse, distant regions), and continues to self-improve given more rounds of iterative self-training. To better measure model performance in self-driving scenarios, we propose a new planning-centric perception metric based on distance-to-collision. We demonstrate that our unsupervised object detector significantly outperforms unsupervised baselines on PandaSet and Argoverse 2 Sensor dataset, showing promise that self-supervision combined with object priors can enable object discovery in the wild. For more information, visit the project website: https://waabi.ai/research/oyster.
Anomaly-Based Intrusion on IoT Networks Using AIGAN-a Generative Adversarial Network

Adversarial attacks have threatened the credibility of machine learning models and cast doubts over the integrity of data. The attacks have created much harm in the fields of computer vision, and natural language processing. In this paper, we focus on the adversarial attack, in particular the poisoning attack, against the network intrusion detection system (NIDS), which is often viewed as the first line of defense against cyber threats. We develop an Anomaly-based intrusion on IoT networks Generative Adversarial Network (AIGAN), which uses deep learning techniques to generate adversarial data and to conduct an anomaly attack on IoT networks. To evaluate the effectiveness of our generator, we measure the similarities between real and fake data using the Jaccard similarity index, in addition comparing the F1-scores from four generic algorithms: multilayer perception, logistic regression, decision tree, random forest. We contrast the performance of ten machine learning classifiers experimented on two real IoT datasets and their fake adversarial samples. Our work highlights a vulnerable side of NIDS created by machine learning when attacked with adversarial perturbation.
Robust and Scalable Gaussian Process Regression and Its Applications

This paper introduces a robust and scalable Gaussian process regression (GPR) model via variational learning. This enables the application of Gaussian processes to a wide range of real data, which are often large-scale and contaminated by outliers. Towards this end, we employ a mixture likelihood model where outliers are assumed to be sampled from a uniform distribution. We next derive a variational formulation that jointly infers the mode of data, i.e., inlier or outlier, as well as hyperparameters by maximizing a lower bound of the true log marginal likelihood. Compared to previous robust GPR, our formulation approximates the exact posterior distribution. The inducing variable approximation and stochastic variational inference are further introduced to our variational framework, extending our model to large-scale data. We apply our model to two challenging real-world applications, namely feature matching and dense gene expression imputation. Extensive experiments demonstrate the superiority of our model in terms of robustness and speed. Notably, when matching 4k feature points, its inference is completed in milliseconds with almost no false matches. The code is at github.com/YifanLu2000/Robust-Scalable-GPR.
Unknown Sniffer for Object Detection: Don't Turn a Blind Eye to Unknown Objects

The recently proposed open-world object and open-set detection have achieved a breakthrough in finding never-seen-before objects and distinguishing them from known ones. However, their studies on knowledge transfer from known classes to unknown ones are not deep enough, resulting in the scanty capability for detecting unknowns hidden in the background. In this paper, we propose the unknown sniffer (UnSniffer) to find both unknown and known objects. Firstly, the generalized object confidence (GOC) score is introduced, which only uses known samples for supervision and avoids improper suppression of unknowns in the back-ground. Significantly, such confidence score learned from known objects can be generalized to unknown ones. Additionally, we propose a negative energy suppression loss to further suppress the non-object samples in the background. Next, the best box of each unknown is hard to obtain during inference due to lacking their semantic information in training. To solve this issue, we introduce a graph-based determination scheme to replace hand-designed non-maximum suppression (NMS) post-processing. Finally, we present the Unknown Object Detection Benchmark, the first publicly benchmark that encompasses precision evaluation for unknown detection to our knowledge. Experiments show that our method is far better than the existing state-of-the-art methods. Code is available at: https://github.com/Went-Liang/UnSniffer.
Bilateral Memory Consolidation for Continual Learning

Humans are proficient at continuously acquiring and integrating new knowledge. By contrast, deep models forget catastrophically, especially when tackling highly long task sequences. Inspired by the way our brains constantly rewrite and consolidate past recollections, we propose a novel Bilateral Memory Consolidation (BiMeCo) framework that focuses on enhancing memory interaction capabilities. Specifically, BiMeCo explicitly decouples model parameters into short-term memory module and long-term memory module, responsible for representation ability of the model and generalization over all learned tasks, respectively. BiMeCo encourages dynamic interactions between two memory modules by knowledge distillation and momentum-based updating for forming generic knowledge to prevent forgetting. The proposed BiMeCo is parameter-efficient and can be integrated into existing methods seamlessly. Extensive experiments on challenging benchmarks show that BiMeCo significantly improves the performance of existing continual learning methods. For example, combined with the state-of-the-art method CwD [55], BiMeCo brings in significant gains of around 2% to 6% while using 2x fewer parameters on CIFAR-100 under ResNet-18.
POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo

Enable neural networks to capture 3D geometrical-aware features is essential in multi-view based vision tasks. Previous methods usually encode the 3D information of multi-view stereo into the 2D features. In contrast, we present a novel method, named POEM, that directly operates on the 3D POints Embedded in the Multi-view stereo for reconstructing hand mesh in it. Point is a natural form of 3D information and an ideal medium for fusing features across views, as it has different projections on different views. Our method is thus in light of a simple yet effective idea, that a complex 3D hand mesh can be represented by a set of 3D points that 1) are embedded in the multi-view stereo, 2) carry features from the multi-view images, and 3) encircle the hand. To leverage the power of points, we design two operations: point-based feature fusion and cross-set point attention mechanism. Evaluation on three challenging multi-view datasets shows that POEM outperforms the state-of-the-art in hand mesh reconstruction. Code and models are available for research at github.com/lixiny/POEM