Reviewerly/exHarmony huggingface dataset:
Point CNN:3D Face Recognition with Local Feature Descriptor and Feature Enhancement Mechanism

Three-dimensional face recognition is an important part of the field of computer vision. Point clouds are widely used in the field of 3D vision due to the simple mathematical expression. However, the disorder of the points makes it difficult for them to have ordered indexes in convolutional neural networks. In addition, the point clouds lack detailed textures, which makes the facial features easily affected by expression or head pose changes. To solve the above problems, this paper constructs a new face recognition network, which mainly consists of two parts. The first part is a novel operator based on a local feature descriptor to realize the fine-grained features extraction and the permutation invariance of point clouds. The second part is a feature enhancement mechanism to enhance the discrimination of facial features. In order to verify the performance of our method, we conducted experiments on three public datasets: CASIA-3D, Bosphorus, and Lock3Dface. The results show that the accuracy of our method is improved by 0.7%, 0.4%, and 0.8% compared with the latest methods on these three datasets, respectively.
Deep‐sea image stitching: Using multi‐channel fusion and improved AKAZE

Abstract Deep‐sea image is of great significance for exploring seabed resources. However, the information of a single image is limited. Besides, deep‐sea image with low contrast and colour distortion further restricts useful feature extraction. To address the issues above, this paper presents a multi‐channel fusion and accelerated‐KAZE (AKAZE) feature detection algorithm for deep‐sea image stitching. First, the authors restore deep‐sea image in LAB colour space and RGB colour space, respectively; in LAB space, the authors use homomorphic filtering in L colour channel, and in RGB space, the authors adopt multi‐scale Retinex with chromaticity preservation algorithm to adjust the colour information. Then, the authors blend two pre‐processed images with dark channel prior weighted coefficient. After that, the authors detect feature points with the AKAZE algorithm and obtain feature descriptors with Boosted Efficient Binary Local Image Descriptor. Finally, the authors match the feature points and warp deep‐sea images to obtain the stitched image. Experimental results demonstrate that the authors’ method generates high‐quality stitched image with minimized seam. Compared with state‐of‐the‐art algorithms, the proposed method has better quantitative evaluation, visual stitching results, and robustness.
Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models

Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srinivasan Iyer. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.
DeepMaven: Deep Question Answering on Long-Distance Movie/TV Show Videos with Multimedia Knowledge Extraction and Synthesis

Yi Fung, Han Wang, Tong Wang, Ali Kebarighotbi, Mohit Bansal, Heng Ji, Prem Natarajan. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.
On the Consistency, Discriminative Power and Robustness of Sampled Metrics in Offline Top-N Recommender System Evaluation

Negative item sampling in offline top-n recommendation evaluation has become increasingly wide-spread, but remains controversial. While several studies have warned against using sampled evaluation metrics on the basis of being a poor approximation of the full ranking (i.e. using all negative items), others have highlighted their improved discriminative power and potential to make evaluation more robust. Unfortunately, empirical studies on negative item sampling are based on relatively few methods (between 3-12) and, therefore, lack the statistical power to assess the impact of negative item sampling in practice.
Natural-Language-Driven Multimodal Representation Learning for Audio-Visual Scene-Aware Dialog System

With the development of multimedia systems in wireless environments, the rising need for artificial intelligence is to design a system that can properly communicate with humans with a comprehensive understanding of various types of information in a human-like manner. Therefore, this paper addresses an audio-visual scene-aware dialog system that can communicate with users about audio-visual scenes. It is essential to understand not only visual and textual information but also audio information in a comprehensive way. Despite the substantial progress in multimodal representation learning with language and visual modalities, there are still two caveats: ineffective use of auditory information and the lack of interpretability of the deep learning systems' reasoning. To address these issues, we propose a novel audio-visual scene-aware dialog system that utilizes a set of explicit information from each modality as a form of natural language, which can be fused into a language model in a natural way. It leverages a transformer-based decoder to generate a coherent and correct response based on multimodal knowledge in a multitask learning setting. In addition, we also address the way of interpreting the model with a response-driven temporal moment localization method to verify how the system generates the response. The system itself provides the user with the evidence referred to in the system response process as a form of the timestamp of the scene. We show the superiority of the proposed model in all quantitative and qualitative measurements compared to the baseline. In particular, the proposed model achieved robust performance even in environments using all three modalities, including audio. We also conducted extensive experiments to investigate the proposed model. In addition, we obtained state-of-the-art performance in the system response reasoning task.
Problem Based Learning Models to Numeracy Literacy Skills : A Study in Elementary School

: low numeracy literacy ability of students is a particular concern considering that in this 21st-century learning, students should have mastered this ability. The problem-based learning models is a learning model that presents a contextual problems so that students can be directly involved in the problem-solving process. Thus, this study aims to determine the effect of the problem-based learning model on the numeracy literacy skills of elementary school students. The research method used by researchers is a pre-experiment with one group pretest and posttest design using a quantitative approach. The sample in this study was eleven students in fifth grade at Waru Timur 2 Pamekasan Madura state elementary school. Data collection was carried out by administering tests (pretest and posttest) regarding students’ numeracy literacy abilities. Data were analyzed using descriptive statistics, prerequisite tests, and hypothesis testing with paired samples test. Based on the results of data processing, it is known that there is a significant influence of the use of problem-based learning models on students’ numeracy literacy skills. That is because learning is student-centered so that students can be directly involved in problem-solving
Latent Transformer Models for out-of-distribution detection

Any clinically-deployed image-processing pipeline must be robust to the full range of inputs it may be presented with. One popular approach to this challenge is to develop predictive models that can provide a measure of their uncertainty. Another approach is to use generative modelling to quantify the likelihood of inputs. Inputs with a low enough likelihood are deemed to be out-of-distribution and are not presented to the downstream predictive model. In this work, we evaluate several approaches to segmentation with uncertainty for the task of segmenting bleeds in 3D CT of the head. We show that these models can fail catastrophically when operating in the far out-of-distribution domain, often providing predictions that are both highly confident and wrong. We propose to instead perform out-of-distribution detection using the Latent Transformer Model: a VQ-GAN is used to provide a highly compressed latent representation of the input volume, and a transformer is then used to estimate the likelihood of this compressed representation of the input. We demonstrate this approach can identify images that are both far- and near- out-of-distribution, as well as provide spatial maps that highlight the regions considered to be out-of-distribution. Furthermore, we find a strong relationship between an image's likelihood and the quality of a model's segmentation on it, demonstrating that this approach is viable for filtering out unsuitable images.
Single image denoising with a feature-enhanced network

Recent Transformer-based networks have shown impressive performance on single image denoising tasks. While the Transformer model promotes the interaction of long-range features, it generally involves high computational complexity. In this paper, we propose a feature-enhanced denoising network (FEDNet) by combining CNN architectures with Transformers. Specifically, we propose an effective cross-channel attention to boost the interaction of channel information and enhance channel features. In order to fully exploit image features, we incorporate Transformer blocks into minimum-scale layers of the network, which can not only capture the long-distance dependency of low-resolution features but also reduce the multiplier-accumulator operations (MACs). Meanwhile, a structure-preserving block is designed to enhance the structural feature extraction. Experimental results on both synthetic and real-world datasets demonstrate that our model can achieve the state-of-the-art denoising performance with low computational costs.
Kajian Ekranisasi Novel Rembulan Tenggelam di Wajahmu Karya Tere Liye dan Film Rembulan Tenggelam di Wajahmu Sutradara Danial Rifki

Ecranization is a study that discusses the transfer of the whitewashing of the novel into the film. The problems contained in this research, namely: how is the process of ecranization of the shrinking aspect in the novel Rembulan Tenggelam di Wajahmu? How is the process of ecranization of additional aspects in the novel and film Rembulan Tenggelam di Wajahmu? How is the process of ecranizing the various aspects of change in the novels and films of Rembulan Tenggelam di Facemu? This study aims to determine: the process of ecranization of the shrinking aspect in the novel and film Rembulan Tenggelam di Facemu; The process of ecranization of additional aspects in the novel and film Rembulan Tenggelam di Wajahmu; The process of ecranization of aspects of change varies from novels and films Rembulan Tenggelam di Wajahmu. This research includes library research (Library research). This study uses a qualitative method. Data collection techniques using hermeneutic techniques and observations. The theory used is the theory of Eneste (1991). The results of the study: (1) Aspects of shrinkage there are 97 data that occur in the elements of the plot, characters, and setting of the novel. Such as the absence of 3 characters Rinai, Beverage Trader, Qolu in the novel; (2) Aspects of addition there are 21 data that occur in the elements of plot, character, and film setting. As added scene 1, the characters of Sister and Doctor in the film; (3) Aspects of change vary, there are 34 data that occur in plot elements, characters, and novel settings. Like the changing setting of the atmosphere during the Eid prayer which depicts the caretaker of the orphanage and the orphanage children going to celebrate the takbiran night, while in the novel they go to the field for the Id prayer. The ecranization process causes shrinkage, addition, and various changes. This happens because of the differences in the media used in the making of novels and films, differences in the production process, and differences in the enjoyment process.
Encouraging Sparsity in Neural Topic Modeling with Non-Mean-Field Inference

Topic modeling is a popular method for discovering semantic information from textual data, with latent Dirichlet allocation (LDA) being a representative model. Recently, researchers have explored the use of variational autoencoders (VAE) to improve the performance of LDA. However, there remain two major limitations: (1) the Dirichlet prior is inadequate to extract precise semantic information in VAE-LDA models, as it introduces a trade-off between the topic quality and the sparsity of representations; (2) new variants of VAE-LDA models with auxiliary variables generally ignore the correlation between latent variables in the inference process due to the Mean-Field assumption. To address these issues, in this paper, we propose a Sparsity Reinforced and Non-Mean-Field Topic Model (SpareNTM) with a bank of auxiliary Bernoulli variables in the generative process of LDA to further model the sparsity of document representations. Thus individual documents are forced to focus on a subset of topics by a corresponding Bernoulli topic selector. Then, instead of applying the mean-field assumption for the posterior approximation, we take full advantage of VAE to realize a non-mean-field approximation, which succeeds in preserving the connection of latent variables. Experiment results on three datasets (20NewsGroup, Wikitext-103, and SearchSnippets) show that our model outperforms recent topic models in terms of both topic quality and sparsity.
Hyperbolic Uncertainty Aware Semantic Segmentation

Semantic segmentation (SS) aims to classify each pixel into one of the pre-defined classes. This task plays an important role in self-driving cars and autonomous drones. In SS, many works have shown that most misclassified pixels are commonly near object boundaries with high uncertainties. However, existing SS loss functions are not tailored to handle these uncertain pixels during training, as these pixels are usually treated equally as confidently classified pixels and cannot be embedded with arbitrary low distortion in Euclidean space, thereby degenerating the performance of SS. To overcome this problem, this paper designs a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Hyper</b> bolic <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</b> ncertainty <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> oss (HyperUL), which dynamically highlights the misclassified and high-uncertainty pixels in Hyperbolic space during training via the hyperbolic distances. The proposed HyperUL is model agnostic and can be easily applied to various neural architectures. After employing HyperUL to three recent SS models, the experimental results on Cityscapes, UAVid, and ACDC datasets reveal that the segmentation performance of existing SS models can be consistently improved. Additionally, reliable measurement of model uncertainty plays a key role in real-world applications such as autonomous controls of vehicles and drones. To meet this requirement, we propose the Hyperbolic Uncertainty Estimation method, which is easily implemented by only post-processing the generated Hyperbolic embeddings. By this approach, we can calculate the uncertainty values almost for free. Quantitative and qualitative results on Cityscapes, UAVid, and ACDC datasets verify that our proposed uncertainty estimation method usually outputs more meaningful results compared with popular MC-dropout and ensembling methods.
An efficient real-time marine object detection algorithm based on convolutional attention

Due to the influence of light and suspended particles in water, underwater image has many problems, such as details blur, color distortion and scale diversification, which are always difficult for underwater object detection. To solve the difficulty of low real-time performance and low recognition accuracy caused by low-quality images to traditional object detection algorithms. This paper proposes an improved YOLOv7 method. Its main idea is to use Ghost module to replace the traditional convolution block, and reduce computational complexity of this model through the idea of deep separation convolution. At the same time, SimAM is introduced in the feature fusion network to enhance the feature expression ability of the network. Finally, Mosaic data enhancement and transformer module are used in input part and Head part to comprehensively improve object recognition performance of the model. The results show that compared with the original YOLOv7, the precision of this algorithm is increased by 4.2%, mAP increase by 1.7%, fps is increased by 5.2 frames, and the number of parameters is effectively reduced, which can effectively meet the requirements of marine treasures detection.
SOC Analyst Performance Metrics: Towards an optimal performance model

Security Operation Center represents nowadays an indispensable component of the socio-technical system by supporting businesses to protect their security and ensure the confidentiality, integrity, and availability against cyberthreats and security attacks.The Security Operation Center provides various service levels and capabilities that need to be continuously assessed and tracked to ensure improvement of the main success factor of the SOC, which include technologies, processes, and SOC analysts. SOC analysts’ performance evaluation remains problematic due to the choice of the performance metrics and their inadequacy with the SOC socio-technical system. While we have some quantitative and qualitative measures to assess the performance of a SOC analyst, SOC capabilities, and SOC maturity levels, this evaluation is based on root cause analysis and independent evaluation of SOC elements, which is unrealistic, given the complex and evolving nature of SOC systems. However, the baselines of the performance metrics are the SOC challenges announced by the SOC analysts and their ability to face, reduce, and overcome them to provide and maintain a high detection rate of malicious and abnormal behaviors. We provide a comprehensive overview of the challenges faced by SOC analysts based on our previous study, and we provide a deep analysis of the challenges and the interconnexion of those challenges. Furthermore, we present the quantitative performance metrics and their weaknesses to assess the performance of the SOC analysts due to the SOC socio-technical system nature. Our study will enable SOC managers, analysts, and decision-makers to have clear visibility and details on the quantitative performance metrics and will provide a baseline for a new performance metrics model.
Real Time Command System for Power Network Scheduling Based on Blockchain Technology

Designing efficient and reasonable power grid dispatch methods and trading mechanisms is a key issue in achieving market-oriented reform of electricity and effective allocation of electricity resources. The current power grid dispatch models face significant challenges such as inaccurate predictions of renewable energy and load, limited information transparency and increased system security vulnerabilities. With its decentralized, transparent, credible, and traceable characteristics, blockchain technology is an emerging distributed database technology that is well-suited for the power grid. By providing a distributed, secure, and transparent information system for power dispatch and trading, it can complement the power grid. This article introduces blockchain technology into power grid operations, covering topics such as the current development status of power grid technology, the data structure, types, and operating principles of blockchain. On this basis, combined with the characteristics of microgrids, this article analyzes the compatibility between blockchain technology and grid scheduling. Secondly, this article constructs an energy blockchain network (EBN) architecture and proposes a multi time scale power grid scheduling method based on smart contracts, taking into account energy storage and user side response. By fully enhancing the utilization rate of renewable energy and bolstering information security for the power grid system, this method offers a comprehensive solution. Simulation experiments have confirmed the feasibility and effectiveness of the proposed scheduling strategy.
Research on Decentralized Federated Learning System for Vehicle Data Privacy Protection Based on Blockchain

This paper proposes an edge computing vehicle data privacy protection method based on blockchain and federated learning. Using blockchain to give edge computing tamper-proof and anti-single point of failure attacks and other characteristics. Integrate gradient verification and incentive mechanism into consensus protocol. By constructing master-slave chain architecture among vehicles, roadside units and base stations, distributed model security sharing is realized. An asynchronous federated learning algorithm based on incentive mechanism is proposed to motivate vehicles and roadside units to participate in the optimization process. The framework is verified on real locus data sets. LSTM self-coding is used as embedded learning network. The prototype of the proposed architecture is implemented in the experiment, and its feasibility, accuracy and performance are evaluated. The results show that this method maintains the integrity of data and has good prediction accuracy and performance.
RFRFlow: Recurrent Feature Refinement Network for Optical Flow Estimation

Optical flow represents vector motion of targets, and its estimation is a hot research in computer vision. The traditional prior-based methods build mathematical models using different regularizers on object characteristics. These methods are straightforward and easily understood, but they suffer from a high-computational burden. Deep learning-based approaches greatly improve their efficiency, but their filter sizes are fixed and receptive fields (RFs) are large. To solve these problems, this article proposes a novel recurrent feature refinement network based on short-term dense connection modules for optical flow estimation (RFRFlow), including the following key parts: first, we utilize short-term dense concatenate (STDC) modules as a backbone part for constructing a feature extracting network, which can preserve rich details and use global averaging pooling to extract global context information with a large RF. Second, a 4-D correlation volume is constructed using context features of image pairs to extract multiscale 4-D correlation information. Third, a motion feature encoder (MFE) is reported to transfer the multiscale 4-D correlation information into 2-D motion features. Finally, 2-D context features and 2-D motion features are input into a global motion aggregation (GMA) decoder for pursuing the final optical flow. In the experiments, we first present and analyze the results of ablation studies on the selection of batch size and image size. Then, both quantitative and qualitative results on Sintel and KITTI datasets are compared, showing that the proposed RFRFlow is effective and even performs better than the state-of-the-art optical flow estimation methods on performance improvement and objects’ shape preservation.
Data Note: The Digital Ludeme Project Database

This document outlines the types of data collected for the Digital Ludeme Project, an ERC-funded research project that aims to improve our understanding of the development of games throughout human history through computational analysis of the available (partial) historical data of games. This document outlines how this data is collected, formatted and stored, and how it can be accessed. It is the aim of the Digital Ludeme Project to provide a data resource of unprecedented depth and scope for the benefit of historical games researchers worldwide. Special attention is paid to the FAIR Guiding Principles for scientific data management and stewardship.