Reviewerly/exHarmony huggingface dataset:
FolketID: A Decentralized Blockchain-Based NemID Alternative Against DDoS Attacks

Electronic Identity (eID) is a solution toward solving the identity problem when deploying e-Government. Such solution should not only be secure, but also have to be robust enough under adversarial conditions. In Denmark, NemID is a digital eID used to log into public websites, for online banking and many digital services. However, it suffers from some security threats such as Distributed Denial of Service (DDoS) attack. In 2013, a tremendous DDoS attack over NemID caused chaos in Danish society, people found themselves out-of-reach to many services. Focused on this threat, we develop FolketID, a blockchain-based NemID alternative against DDoS attacks. The DDoS mitigation is achieved by not only utilizing its powerful data distribution ability, but also exploiting the economy system in blockchain. It particularly takes use of the transaction fees to effectively prevent DDoS attacks. In the evaluation, our experimental results demonstrate the viability and effectiveness of FolketID in defeating DDoS attacks.
Unbiased Multi-modality Guidance for Image Inpainting

Image inpainting is an ill-posed problem to recover missing or damaged image content based on incomplete images with masks. Previous works usually predict the auxiliary structures (e.g., edges, segmentation and contours) to help fill visually realistic patches in a multi-stage fashion. However, imprecise auxiliary priors may yield biased inpainted results. Besides, it is time-consuming for some methods to be implemented by multiple stages of complex neural networks. To solve this issue, we develop an end-to-end multi-modality guided transformer network, including one inpainting branch and two auxiliary branches for semantic segmentation and edge textures. Within each transformer block, the proposed multi-scale spatial-aware attention module can learn the multi-modal structural features efficiently via auxiliary denormalization. Different from previous methods relying on direct guidance from biased priors, our method enriches semantically consistent context in an image based on discriminative interplay information from multiple modalities. Comprehensive experiments on several challenging image inpainting datasets show that our method achieves state-of-the-art performance to deal with various regular/irregular masks efficiently. The code is available at https://github.com/yeates/MMT .
Towards Comprehensive Monocular Depth Estimation: Multiple Heads are Better Than One

Depth estimation attracts widespread attention in the computer vision community. However, it is still quite difficult to recover an accurate depth map using only one RGB image. We observe a phenomenon that existing methods tend to fail in different cases, caused by differences in network architecture, loss function and so on. In this work, we investigate into the phenomenon and propose to integrate the strengths of multiple weak depth predictor to build a comprehensive and accurate depth predictor, which is critical for many real-world applications, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , 3D reconstruction. Specifically, we construct multiple base (weak) depth predictors by utilizing different Transformer-based and convolutional neural network (CNN)-based architectures. Transformer establishes long-range correlation while CNN preserves local information ignored by Transformer due to the spatial inductive bias. Therefore, the coupling of Transformer and CNN contributes to the generation of complementary depth estimates, which are essential to achieve a comprehensive depth predictor. Then, we design mixers to learn from multiple weak predictions and adaptively fuse them into a strong depth estimate. The resultant model, which we refer to as Transformer-assisted depth ensembles (TEDepth). On the standard NYU-Depth-v2 and KITTI datasets, we thoroughly explore how the neural ensembles affect the depth estimation and demonstrate that our TEDepth achieves better results than previous state-of-the-art approaches. To validate the generalizability across cameras, we directly apply the models trained on NYU-Depth-v2 to the SUN RGB-D dataset without any fine-tuning, and the superior results emphasize its strong generalizability. The source code and trained models will be publicly available upon the acceptance.
Quantum Image Encryption Using a Self-Adaptive Hash Function-Controlled Chaotic Map (SAHF-CCM)

Recently, quantum image encryption algorithms are attracting more and more attention, due to the upcoming quantum threat problem to the current cryptographic encryption algorithms with the rapid progress toward the quantum computer production. The aim of this paper is to introduce a self-adaptive encryption scheme to protect quantum image efficiently with minimal storage requirements. The methodology is to use two rounds of encryption with two different pseudorandom number sequences which are obtained from a new designed pseudorandom number generator (PRNG). This PRNG consists of two parts. The first part is based on iterating a recently proposed chaotic-based parallel keyed hash function which is used as a controller for a second part. The second part is a multiplication of Tent and Chebyshev chaotic maps (TCM). This combination of hash function and chaotic maps provides high randomness and a dramatical increase in the control parameters and initial values number which achieves extremely large key space and so makes the scheme stronger against brute force attacks. The seed or initial value of PRNG depends on the input image itself which makes the scheme self-adaptive and so it is stronger against chosen plaintext attacks and known-plaintext attacks. In the first round of encryption, the value of each pixel qubits of the input image is changed to a new value by XORing it with the corresponding qubits of the first pseudorandom sequence using CNOT and Toffoli quantum gates then shifting to the next qubit with the Swap quantum gate. In this round, the pixel value is changed. In the second round, diffusion of the changed pixel value is extended to each pixel in the input image using the second pseudorandom sequence and Toffoli quantum gates. The time complexity of the proposed scheme is less than many recently published quantum image encryption schemes. The scheme security analysis is discussed, and the experimental results proved that the scheme is robust, secure and efficient.
Trustworthy Explanations for Knowledge Discovered from E-Health Records

In the current era of big data, very large amounts of data are generating at a rapid rate from a wide variety of rich data sources. Electronic health (e-health) records are examples of the big data. With the technological advancements, more healthcare practice has gradually been supported by electronic processes and communication. This enables health informatics, in which computer science meets the healthcare sector to address healthcare and medical problems. Embedded in the big data are valuable information and knowledge that can be discovered by data science, data mining and machine learning techniques. Many of these techniques apply "opaque box" approaches to make accurate predictions. However, these techniques may not be crystal clear to the users. As the users not necessarily be able to clearly view the entire knowledge discovery (e.g., prediction) process, they may not easily trust the discovered knowledge (e.g., predictions). Hence, in this paper, we present a system for providing trustworthy explanations for knowledge discovered from e-health records. Specifically, our system provides users with global explanations for the important features among the records. It also provides users with local explanations for a particular record. Evaluation results on real-life e-health records show the practicality of our system in providing trustworthy explanations to knowledge discovered (e.g., accurate predictions made).
Collision-Resistance from Multi-Collision-Resistance

Collision-resistant hash functions ( $$\textsf{CRH}$$ ) are a fundamental and ubiquitous cryptographic primitive. Several recent works have studied a relaxation of $$\textsf{CRH}$$ called t-way multi-collision-resistant hash functions ( $$t\text {-}\textsf{MCRH}$$ ). These are families of functions for which it is computationally hard to find a t-way collision, even though such collisions are abundant (and even $$(t-1)$$ -way collisions may be easy to find). The case of $$t=2$$ corresponds to standard $$\textsf{CRH}$$ , but it is natural to study t- $$\textsf{MCRH}$$ for larger values of t. Multi-collision-resistance seems to be a qualitatively weaker property than standard collision-resistance. Nevertheless, in this work we show a non-blackbox transformation of any moderately shrinking t- $$\textsf{MCRH}$$ , for $$t \in \{3,4\}$$ , into an (infinitely often secure) $$\textsf{CRH}$$ . This transformation is non-constructive – we can prove the existence of a $$\textsf{CRH}$$ but cannot explicitly point out a construction. Our result partially extends to larger values of t. In particular, we show that for suitable values of $$t>t'$$ , we can transform a t- $$\textsf{MCRH}$$ into a $$t'$$ - $$\textsf{MCRH}$$ , at the cost of reducing the shrinkage of the resulting hash function family and settling for infinitely often security. This result utilizes the list-decodability properties of Reed-Solomon codes.
Semi-Weakly-Supervised Learning of Complex Actions from Instructional Task Videos

We address the problem of action segmentation in instructional task videos with a small number of weakly-labeled training videos and a large number of unlabeled videos, which we refer to as Semi-Weakly-Supervised Learning (SWSL) of actions. We propose a general SWSL framework that can efficiently learn from both types of videos and can leverage any of the existing weakly-supervised action segmentation methods. Our key observation is that the distance between the transcript of an unlabeled video and those of the weakly-labeled videos from the same task is small yet often nonzero. Therefore, we develop a Soft Restricted Edit (SRE) loss to encourage small variations between the predicted transcripts of unlabeled videos and ground-truth transcripts of the weakly-labeled videos of the same task. To compute the SRE loss, we develop a flexible transcript prediction (FTP) method that uses the output of the action classifier to find both the length of the transcript and the sequence of actions occurring in an unlabeled video. We propose an efficient learning scheme in which we alternate between minimizing our proposed loss and generating pseudo-transcripts for unlabeled videos. By experiments on two benchmark datasets, we demonstrate that our approach can significantly improve the performance by using unlabeled videos, especially when the number of weakly-labeled videos is small. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code available at https://github.com/Yuhan-Shen/SWSL..
MSFANet: A Light Weight Object Detector Based on Context Aggregation and Attention Mechanism for Autonomous Mining Truck

Accurate and reliable object detection is a fundamental component of perception system for autonomous driving. Specially, in some circumstances like autonomous driving in surface mine, there is a fact that the particularity of scene brings tremendous challenges for object detection with a series of problems caused by the multi-scale and camouflaged objects. In this paper, a multi-scale feature fusion and attention based multi-branches framework was proposed to improve the performance of object detection for above problems called MSFANet. In the proposed MSFANet, a multi-scale feature fusion module, which was used to capture the rich context features for multi-scale high level feature maps, and a multi-scale attention module, which was used to enhance the feature saliency of objects with different scales, were designed. What's more, to improve the performance of multi-scale object detection, we build 4 different prediction branches for large, medium small and smaller scale objects respectively. At last, we built our own dataset for automatic driving in surface mine called SurMine and test the model at our own datasets and KITTI benchmark. It achieved 82.7 mAP(%) and 92.57 mAP(%) in 32 36 ms on a TITAN RTX, compared to 80.2 mAP(%) and 87.83 mAP(%) in 28 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\sim }$</tex-math></inline-formula> 34 ms by YOLOv7 on SurMine and KITTI benchmarks.
Multi-Scale Interactive Transformer for Remote Sensing Cross-Modal Image-Text Retrieval

Cross-modal Remote sensing (RS) image-text retrieval (CMR-SITR) plays a crucial role in the RS community. A common way for CMRSITR is to extract texts and RS images' feature representations separately and then measure their similarities in the specific or common feature space. Recently, along with the booming of deep convolutional neural networks (DCNNs), these kinds of methods are vivid and achieve successes in their own applications. However, they neglect the inherent relationships between different features, and they are always heavy. To overcome the limitations mentioned above, we propose a new model for CMRSITR in this paper, named multi-scale interactive transformer (MSIT). MSIT first adopts simple feature learning models for texts and RS images which could ensure the whole model is not heavy. Then, MSIT introduces transformer encoders to enhance features' usefulness by considering the potential relations between different representations. Also, a lightweight multi-scale feature learning module is proposed to mine more plentiful contents from RS images. Finally, instead of outputting the features, MSIT produces matching scores for texts and RS images, which can be used to decide the retrieval results directly. The experimental results on two RS datasets indicate our modal is effective for CMRSITR.
Slimmable Domain Adaptation

Vanilla unsupervised domain adaptation methods tend to optimize the model with fixed neural architecture, which is not very practical in real-world scenarios since the target data is usually processed by different resource-limited devices. It is therefore of great necessity to facilitate architecture adaptation across various devices. In this paper, we introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank, from which models of different capacities can be sampled to accommodate different accuracy-efficiency trade-offs. The main challenge in this frame-work lies in simultaneously boosting the adaptation performance of numerous models in the model bank. To tackle this problem, we develop a Stochastic EnsEmble Distillation method to fully exploit the complementary knowledge in the model bank for inter-model interaction. Nevertheless, considering the optimization conflict between inter-model interaction and intra-model adaptation, we augment the existing bi-classifier domain confusion architecture into an Optimization-Separated Tri-Classifier counterpart. After optimizing the model bank, architecture adaptation is leveraged via our proposed Unsupervised Performance Evaluation Metric. Under various resource constraints, our framework surpasses other competing approaches by a very large margin on multiple benchmarks. It is also worth emphasizing that our framework can preserve the performance improvement against the source-only model even when the computing complexity is reduced to 1/64. Code will be available at https://github.com/HIK-LAB/SlimDA.
Cross-View Contrastive Learning for Knowledge-Aware Session-Based Recommendation

Session-based Recommendation (SBR) methods play a crucial role in modern recommender systems owing to their ability to capture an anonymous user’s dynamic preference from his/her short-term historical interactions. Despite previous success, we argue that existing models generally suffer from the sparsity problem of user-item interactions, which may make their actual performance drop to some extent. To tackle this, inspired by the recent success of contrastive learning in mining supervised signals from data itself, we propose a novel Cross-view Contrastive learning mechanism for Knowledge-aware Session-based Recommendation, named CCKSR. Our model comprehensively considers two different graph views, i.e., subjectively structural view and objectively semantic view. Specifically, we take cross-session graph as the structural view, and the item-attribute hypergraph is regarded as the semantic view. Then, a query-aware graph attention network and a hypergraph convolutional network are designed to learn node representations from two views, so as to capture comprehensive sequential structure information and item-item attribute semantic relation in a contrastive self-supervised manner. Extensive experiments on three benchmark datasets show the superiority performance of our proposed method over the state-of-the-art baselines.
Graph Federated Learning for CIoT Devices in Smart Home Applications

This paper deals with the problem of statistical and system heterogeneity in a cross-silo Federated Learning (FL) framework where there exist a limited number of Consumer Internet of Things (CIoT) devices in a smart building.We propose a novel Graph Signal Processing (GSP)-inspired aggregation rule based on graph filtering dubbed "G-Fedfilt" 1 .The proposed aggregator enables a structured flow of information based on the graph's topology.This behavior allows capturing the interconnection of CIoT devices and training domain-specific models.The embedded graph filter is equipped with a tunable parameter which enables a continuous trade-off between domain-agnostic and domainspecific FL.In the case of domain-agnostic, it forces G-Fedfilt to act similar to the conventional Federated Averaging (FedAvg) aggregation rule.The proposed G-Fedfilt also enables an intrinsic smooth clustering based on the graph connectivity without explicitly specified which further boosts the personalization of the models in the framework.In addition, the proposed scheme enjoys a communication-efficient time-scheduling to alleviate the system heterogeneity.This is accomplished by adaptively adjusting the amount of training data samples and sparsity of the models' gradients to reduce communication desynchronization and latency.Simulation results show that the proposed G-Fedfilt achieves up to 3.99% better classification accuracy than the conventional FedAvg when concerning model personalization on the statistically heterogeneous local datasets, while it is capable of yielding up to 2.41% higher accuracy than FedAvg in the case of testing the generalization of the models.Furthermore, the proposed communication optimization scheme can boost the framework's efficiency by reducing the computation, communication desynchronization, and latency up to 70.21%, 99.65%, and 44.61%, respectively, at the cost of 0.36% accuracy and under the system heterogeneity 2 .
Design of Vocal Music Education System Based on VR Technology

With the popularization of conceptual teaching methods such as quality education and comprehensive education in schools, China has invested more energy in art education. Especially the vocal music education of art education, under this background, its music education management has entered a new stage. Through a series of means to make teaching more scientific and standardized. Music teachers are of high importance in the traditional meaning teaching mode, and most of them teach in person. The traditional teaching mode has a single way and the content is lack of change. In a very short time, teaching will lead to poor information transmission. As a product of the information age, network teaching platform has gradually become a new way for people to obtain knowledge. With the support of modern network technology, the development and Reform in the field of education have reached a qualitative leap. In view of the singleness of traditional vocal music education methods and learning situations and the limitations of learning resources, this paper deeply discusses the influence of constructivism theory on music education based on Moodle platform. Combining music education with the teaching platform based on network environment, taking constructivism theory as the main theoretical basis, using network technology, a music education system based on Moodle platform is designed and developed. It realizes the collection, playback and real-time display of audio segments, and also has the function of vocal music scoring. This design makes vocal education lessons intuitive, interactive, and interesting.
BoB: Bandwidth Prediction for Real-Time Communications Using Heuristic and Reinforcement Learning

Bandwidth prediction is critical in any Real-time Communication (RTC) service or application.This component decides how much media data can be sent in real time.Subsequently, the video and audio encoder dynamically adapts the bitrate to achieve the best quality without congesting the network and causing packets to be lost or delayed.To date, several RTC services have deployed the heuristic-based Google Congestion Control (GCC), which performs well under certain circumstances and falls short in some others.In this paper, we leverage the advancements in reinforcement learning and propose BoB (Bang-on-Bandwidth) -a hybrid bandwidth predictor for RTC.At the beginning of the RTC session, BoB uses a heuristic-based approach.It then switches to a learning-based approach.BoB predicts the available bandwidth accurately and improves bandwidth utilization under diverse network conditions compared to the two winning solutions of the ACM MMSys'21 grand challenge on bandwidth estimation in RTC.An opensource implementation of BoB is publicly available for further testing and research.
PROTOtypical Logic Tensor Networks (PROTO-LTN) for Zero Shot Learning

Semantic image interpretation can vastly benefit from approaches that combine sub-symbolic distributed representation learning with the capability to reason at a higher level of abstraction. Logic Tensor Networks (LTNs) are a class of neuro-symbolic systems based on a differentiable, first-order logic grounded into a deep neural network. LTNs replace the classical concept of training set with a knowledge base of fuzzy logical axioms. By defining a set of differentiable operators to approximate the role of connectives, predicates, functions and quantifiers, a loss function is automatically specified so that LTNs can learn to satisfy the knowledge base. We focus here on the subsumption or isOfClass predicate, which is fundamental to encode most semantic image interpretation tasks. Unlike conventional LTNs, which rely on a separate predicate for each class (e.g., dog, cat), each with its own set of learnable weights, we propose a common isOfClass predicate, whose level of truth is a function of the distance between an object embedding and the corresponding class prototype. The PROTOtypical Logic Tensor Networks (PROTO-LTN) extend the current formulation by grounding abstract concepts as parametrized class prototypes in a high-dimensional embedding space, while reducing the number of parameters required to ground the knowledge base.We show how this architecture can be effectively trained in the few and zero-shot learning scenarios. Experiments on Generalized Zero Shot Learning benchmarks validate the proposed implementation as a competitive alternative to traditional embedding-based approaches. The proposed formulation opens up new opportunities in zero shot learning settings, as the LTN formalism allows to integrate background knowledge in the form of logical axioms to compensate for the lack of labelled examples. PROTO-LTN was implemented in Tensorflow and is available at https://github.com/FrancescoManigrass/PROTO-LTN.git