Reviewerly/exHarmony huggingface dataset:
Network-Aware Optimization of Distributed Learning for Fog Computing

Fog computing promises to enable machine learning tasks to scale to large amounts of data by distributing processing across connected devices. Two key challenges to achieving this goal are (i) heterogeneity in devices’ compute resources and (ii) topology constraints on which devices communicate with each other. We address these challenges by developing a novel network-aware distributed learning methodology where devices optimally share local data processing and send their learnt parameters to a server for periodic aggregation. Unlike traditional federated learning, our method enables devices to offload their data processing tasks to each other, with these decisions optimized to trade off costs associated with data processing, offloading, and discarding. We analytically characterize the optimal data transfer solution under different assumptions on the fog network scenario, showing for example that the value of offloading is approximately linear in the range of computing costs in the network when the cost of discarding is modeled as decreasing linearly in the amount of data processed at each node. Our experiments on real-world data traces from our testbed confirm that our algorithms improve network resource utilization substantially without sacrificing the accuracy of the learned model, for varying distributions of data across devices. We also investigate the effect of network dynamics on model learning and resource costs.
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models

This paper studies zero-shot cross-lingual transfer of vision-language models.Specifically, we focus on multilingual text-tovideo search and propose a Transformer-based model that learns contextual multilingual multimodal embeddings.Under a zero-shot setting, we empirically demonstrate that performance degrades significantly when we query the multilingual text-video model with non-English sentences.To address this problem, we introduce a multilingual multimodal pre-training strategy, and collect a new multilingual instructional video dataset (Multi-HowTo100M) for pre-training.Experiments on VTT show that our method significantly improves video search in non-English languages without additional annotations.Furthermore, when multilingual annotations are available, our method outperforms recent baselines by a large margin in multilingual text-to-video search on VTT and VATEX; as well as in multilingual text-to-image search on Multi30K.Our model and Multi-HowTo100M is available at
Digestive neural networks: A novel defense strategy against inference attacks in federated learning

Federated Learning (FL) is an efficient and secure machine learning technique designed for decentralized computing systems such as fog and edge computing. Its learning process employs frequent communications as the participating local devices send updates, either gradients or parameters of their models, to a central server that aggregates them and redistributes new weights to the devices. In FL, private data does not leave the individual local devices, and thus, rendered as a robust solution in terms of privacy preservation. However, the recently introduced membership inference attacks pose a critical threat to the impeccability of FL mechanisms. By eavesdropping only on the updates transferring to the center server, these attacks can recover the private data of a local device. A prevalent solution against such attacks is the differential privacy scheme that augments a sufficient amount of noise to each update to hinder the recovering process. However, it suffers from a significant sacrifice in the classification accuracy of the FL. To effectively alleviate the problem, this paper proposes a Digestive Neural Network (DNN), an independent neural network attached to the FL. The private data owned by each device will pass through the DNN and then train the FL. The DNN modifies the input data, which results in distorting updates, in a way to maximize the classification accuracy of FL while the accuracy of inference attacks is minimized. Our simulation result shows that the proposed DNN shows significant performance on both gradient sharing- and weight sharing-based FL mechanisms. For the gradient sharing, the DNN achieved higher classification accuracy by 16.17% while 9% lower attack accuracy than the existing differential privacy schemes. For the weight sharing FL scheme, the DNN achieved at most 46.68% lower attack success rate with 3% higher classification accuracy.
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning

We present a plug-in replacement for batch normalization (BN) called exponential moving average normalization (EMAN), which improves the performance of existing student-teacher based self- and semi-supervised learning techniques. Unlike the standard BN, where the statistics are computed within each batch, EMAN, used in the teacher, updates its statistics by exponential moving average from the BN statistics of the student. This design reduces the intrinsic cross-sample dependency of BN and enhances the generalization of the teacher. EMAN improves strong baselines for self-supervised learning by 4-6/1-2 points and semi-supervised learning by about 7/2 points, when 1%/10% supervised labels are available on ImageNet. These improvements are consistent across methods, network architectures, training duration, and datasets, demonstrating the general effectiveness of this technique. The code will be made available online.
Deep Video Matting via Spatio-Temporal Alignment and Aggregation

Despite the significant progress made by deep learning in natural image matting, there has been so far no representative work on deep learning for video matting due to the inherent technical challenges in reasoning temporal domain and lack of large-scale video matting datasets. In this paper, we propose a deep learning-based video matting framework which employs a novel and effective spatio-temporal feature aggregation module (ST-FAM). As optical flow estimation can be very unreliable within matting regions, ST-FAM is designed to effectively align and aggregate information across different spatial scales and temporal frames within the network decoder. To eliminate frame-by-frame trimap annotations, a lightweight interactive trimap propagation network is also introduced. The other contribution consists of a large-scale video matting dataset with groundtruth alpha mattes for quantitative evaluation and real-world high-resolution videos with trimaps for qualitative evaluation. Quantitative and qualitative experimental results show that our framework significantly outperforms conventional video matting and deep image matting methods applied to video in presence of multi-frame temporal information. Our dataset is available at https://github.com/nowsyn/DVM.
Interoperable Blockchains for Highly-Integrated Supply Chains in Collaborative Manufacturing

Blockchain technology plays a pivotal role in the undergoing fourth industrial revolution or Industry 4.0. It is considered a tremendous boost to company digitalization; thus, considerable investments in blockchain are being made. However, there is no single blockchain technology, but various solutions exist, and they cannot interoperate with one each other. The ecosystem envisioned by the Industry 4.0 does not have centralized management or leading organization, so a single blockchain solution cannot be imposed. The various organizations hold their own blockchains, which must interoperate seamlessly. Despite some solutions for blockchain interoperability being proposed, the problem is still open. This paper aims to devise a secure solution for blockchain interoperability. The proposed approach consists of a relay scheme based on Trusted Execution Environment to provide higher security guarantees than the current literature. In particular, the proposed solution adopts an off-chain secure computation element invoked by a smart contract on a blockchain to securely communicate with its peered counterpart. A prototype has been implemented and used for the performance assessment, e.g., to measure the latency increase due to cross-blockchain interactions. The achieved and reported experimental results show that the proposed security solution introduces an additional latency that is entirely tolerable for transactions. At the same time, the usage of the Trusted Execution Environment imposes a negligible overhead.
CuCo: Graph Representation with Curriculum Contrastive Learning

Graph-level representation learning is to learn low-dimensional representation for the entire graph, which has shown a large impact on real-world applications. Recently, limited by expensive labeled data, contrastive learning based graph-level representation learning attracts considerable attention. However, these methods mainly focus on graph augmentation for positive samples, while the effect of negative samples is less explored. In this paper, we study the impact of negative samples on learning graph-level representations, and a novel curriculum contrastive learning framework for self-supervised graph-level representation, called CuCo, is proposed. Specifically, we introduce four graph augmentation techniques to obtain the positive and negative samples, and utilize graph neural networks to learn their representations. Then a scoring function is proposed to sort negative samples from easy to hard and a pacing function is to automatically select the negative samples in each training procedure. Extensive experiments on fifteen graph classification real-world datasets, as well as the parameter analysis, well demonstrate that our proposed CuCo yields truly encouraging results in terms of performance on classification and convergence.
Surrogate-Assisted Hybrid-Model Estimation of Distribution Algorithm for Mixed-Variable Hyperparameters Optimization in Convolutional Neural Networks

The performance of a convolutional neural network (CNN) heavily depends on its hyperparameters. However, finding a suitable hyperparameters configuration is difficult, challenging, and computationally expensive due to three issues, which are 1) the mixed-variable problem of different types of hyperparameters; 2) the large-scale search space of finding optimal hyperparameters; and 3) the expensive computational cost for evaluating candidate hyperparameters configuration. Therefore, this article focuses on these three issues and proposes a novel estimation of distribution algorithm (EDA) for efficient hyperparameters optimization, with three major contributions in the algorithm design. First, a hybrid-model EDA is proposed to efficiently deal with the mixed-variable difficulty. The proposed algorithm uses a mixed-variable encoding scheme to encode the mixed-variable hyperparameters and adopts an adaptive hybrid-model learning (AHL) strategy to efficiently optimize the mixed-variables. Second, an orthogonal initialization (OI) strategy is proposed to efficiently deal with the challenge of large-scale search space. Third, a surrogate-assisted multi-level evaluation (SME) method is proposed to reduce the expensive computational cost. Based on the above, the proposed algorithm is named s urrogate-assisted hybrid-model EDA (SHEDA). For experimental studies, the proposed SHEDA is verified on widely used classification benchmark problems, and is compared with various state-of-the-art methods. Moreover, a case study on aortic dissection (AD) diagnosis is carried out to evaluate its performance. Experimental results show that the proposed SHEDA is very effective and efficient for hyperparameters optimization, which can find a satisfactory hyperparameters configuration for the CIFAR10, CIFAR100, and AD diagnosis with only 0.58, 0.97, and 1.18 GPU days, respectively.
Fake Media Detection Based on Natural Language Processing and Blockchain Approaches

Social media network is one of the important parts of human life based on the recent technologies and developments in terms of computer science area. This environment has become a famous platform for sharing information and news on any topics and daily reports, which is the main era for collecting data and data transmission. There are various advantages of this environment, but in another point of view there are lots of fake news and information that mislead the reader and user for the information needed. Lack of trust-able information and real news of social media information is one of the huge problems of this system. To overcome this problem, we have proposed an integrated system for various aspects of blockchain and natural language processing (NLP) to apply machine learning techniques to detect fake news and better predict fake user accounts and posts. The Reinforcement Learning technique is applied for this process. To improve this platform in terms of security, the decentralized blockchain framework applied, which provides the outline of digital contents authority proof. More specifically, the concept of this system is developing a secure platform to predict and identify fake news in social media networks.
Engineering Graphical Captcha and AES Crypto Hash Functions for Secure Online Authentication

Password alone is currently not trusted for user online authentication and security as threats from hackers continue to grow, requiring highly efficient defense safeguard protection against unauthorized users. Therefore, CAPTCHA techniques came into the picture as an automated assistance to distinguish between humans and robots. The CAPTCHA has several applications in the online security domain requiring to be merged with encrypted hash function benefitting from the facility of the graphical password schemes. This paper proposes engineering an authentication technique using graphical CAPTCHA with an AES encrypted hash password to maintain applicable security accessing systems. We propose three layered security system that joins highly efficient security mechanisms to avoid users’ stress of entering password many times or different other hectic routines in order to save account accessing.
ProtGNN: Towards Self-Explaining Graph Neural Networks

Despite the recent progress in Graph Neural Networks (GNNs), it remains challenging to explain the predictions made by GNNs. Existing explanation methods mainly focus on post-hoc explanations where another explanatory model is employed to provide explanations for a trained GNN. The fact that post-hoc methods fail to reveal the original reasoning process of GNNs raises the need of building GNNs with built-in interpretability. In this work, we propose Prototype Graph Neural Network (ProtGNN), which combines prototype learning with GNNs and provides a new perspective on the explanations of GNNs. In ProtGNN, the explanations are naturally derived from the case-based reasoning process and are actually used during classification. The prediction of ProtGNN is obtained by comparing the inputs to a few learned prototypes in the latent space. Furthermore, for better interpretability and higher efficiency, a novel conditional subgraph sampling module is incorporated to indicate which part of the input graph is most similar to each prototype in ProtGNN+. Finally, we evaluate our method on a wide range of datasets and perform concrete case studies. Extensive results show that ProtGNN and ProtGNN+ can provide inherent interpretability while achieving accuracy on par with the non-interpretable counterparts.
Near-perfect fidelity polarization-encoded multilayer optical data storage based on aligned gold nanorods

Encoding information in light polarization is of great importance in facilitating optical data storage (ODS) for information security and data storage capacity escalation.However, despite recent advances in nanophotonic techniques vastly enhancing the feasibility of applying polarization channels, the data fidelity in reconstructed bits has been constrained by severe crosstalks occurring between varied polarization angles during data recording and reading process, which gravely hindered the utilization of this technique in practice.In this paper, we demonstrate an ultra-low crosstalk polarization-encoding multilayer ODS technique for high-fidelity data recording and retrieving by utilizing a nanofibre-based nanocomposite film involving highly aligned gold nanorods (GNRs).With parallelizing the gold nanorods in the recording medium, the information carrier configuration minimizes miswriting and misreading possibilities for information input and output, respectively, compared with its randomly self-assembled counterparts.The enhanced data accuracy has significantly improved the bit recall fidelity that is quantified by a correlation coefficient higher than 0.99.It is anticipated that the demonstrated technique can facilitate the development of multiplexing ODS for a greener future.
Artificial Intelligence Technology Assisted Music Teaching Design

With the continuous improvement of the global economic level and scientific level, information technology has penetrated into all fields of people’s life. Today, the strategy of vigorously promoting educational modernization has created conditions for intelligent music teaching and promoted the in-depth application of artificial intelligence technology in education. Intelligent instructional design supported by artificial intelligence technology is the deep integration of information technology and music teaching. Through the intelligent perception technology, learning analysis technology, and emotional computing technology of artificial intelligence, an intelligent music teaching model is established. The online learning and education platform based on big data intelligence provides teachers with rich teaching methods, provides personalized evaluation and adaptive learning services for students’ learning, and helps improve the efficiency of music teaching. The traditional teaching design model cannot effectively guide the intelligent music teaching and cannot meet the needs of students’ development. Therefore, on the basis of previous studies, the author studies the intelligent teaching design supported by artificial intelligence technology. This study uses new generation information technologies such as big data, Internet of things, mobile Internet, and artificial intelligence to build a complete set of scientific intelligent music teaching design model. Wisdom teaching provides a reference for the whole process before, during, and after class, helps guide teachers to better carry out wisdom teaching, helps students explore cooperative and autonomous learning, and promotes the wisdom transformation of teaching methods and learning methods to a certain extent. Music classroom teaching has become more targeted and effective. Therefore, it is of great significance to cultivate intelligent music talents.
Meta-features for meta-learning

Meta-learning is increasingly used to support the recommendation of machine learning algorithms and their configurations. These recommendations are made based on meta-data, consisting of performance evaluations of algorithms and characterizations on prior datasets. These characterizations, also called meta-features, describe properties of the data which are predictive for the performance of machine learning algorithms trained on them. Unfortunately, despite being used in many studies, meta-features are not uniformly described, organized and computed, making many empirical studies irreproducible and hard to compare. This paper aims to deal with this by systematizing and standardizing data characterization measures for classification datasets used in meta-learning. Moreover, it presents an extensive list of meta-features and characterization tools, which can be used as a guide for new practitioners. By identifying particularities and subtle issues related to the characterization measures, this survey points out possible future directions that the development of meta-features for meta-learning can assume.
Attachment Theory as a Framework to Understand Relationships with Social Chatbots: A Case Study of Replika

With increasing adoption of AI social chatbots, especially during the pandemic-related lockdowns, when people lack social companionship, there emerges a need for in-depth understanding and theorizing of relationship formation with digital conversational agents.Following the grounded theory approach, we analyzed in-depth interview transcripts obtained from 14 existing users of AI companion chatbot Replika.The emerging themes were interpreted through the lens of the attachment theory.Our results show that under conditions of distress and lack of human companionship, individuals can develop an attachment to social chatbots if they perceive the chatbots' responses to offer emotional support, encouragement, and psychological security.These findings suggest that social chatbots can be used for mental health and therapeutic purposes but have the potential to cause addiction and harm real-life intimate relationships.
Cooperative Multi-UAV Task Assignment in Cross-Regional Joint Operations Considering Ammunition Inventory

As combat missions become increasingly complex in both space and time, cross-regional joint operations (CRJO) is becoming an overwhelming trend in modern air warfare. How to allocate resources and missions prior to the operation becomes a central issue to improve the combat efficiency. In this paper, we focus on the cooperative mission planning of multiple heterogeneous unmanned aerial vehicles (UAVs) in a CRJO. A multi-objective optimization problem is presented with the aim of minimizing the makespan while maximizing the value expectation obtained. Moreover, it is not mandatory for each UAV to return exactly to the base which it takes off. Furthermore, in addition to the constraints commonly found in UAV mission assignment problems, the ammunition inventory at each base is also taken into account. To solve such a problem, we developed an improved genetic algorithm (IGA) with a novel chromosome encoding format. It can determine the number of attacks on a given target based on the expectations obtained, rather than being predetermined. Specifically, an efficient logic-based unlocking mechanism is designed for the crossover and mutation operations in the algorithm. Simulation results show that the developed IGA can efficiently solve the considered problem. Through numerical experimental comparisons, the algorithm proposed in this work is superior to other existing IGA-like algorithms in terms of computational efficiency.
Collaborative Normality Learning Framework for Weakly Supervised Video Anomaly Detection

Video anomaly detection (VAD) under weak supervision aims to temporally locate abnormal clips using the easy-to-obtain video-level labels. In this brief, we introduce the underlying thought of unsupervised VAD to the weakly supervised VAD and propose a collaborative normality learning framework to obtain more discriminative deep representations. Specifically, a deep auto-encoder is first trained in an unsupervised manner to learn the prototypical spatial-temporal patterns of normal videos. Then, both the normal and abnormal videos are used to train a regression module, where the objective is to make the average score of the abnormal videos higher than the maximum score of the normal videos. Finally, the clips in abnormal videos with an anomaly score lower than the average are regarded as normal and used to fine-tune the trained auto-encoder. The unsupervised auto-encoder collaborates with the weakly supervised regression model to extract prototypical features of normal clips, making the learned features of normal and abnormal events more distinguishable. Experimental results on three benchmark datasets show that the proposed framework achieves comparable performance to the state-of-the-art methods. Additionally, the results of ablation studies demonstrate the validity of collaborative normality learning.