Reviewerly/exHarmony huggingface dataset:
Semantic Segmentation of Fish and Underwater Environments Using Deep Convolutional Neural Networks and Learned Active Contours

The conservation of marine resources requires constant monitoring of the underwater environment by researchers. For this purpose, visual automated monitoring systems are of great interest, especially those that can describe the environment using semantic segmentation based on deep learning. Although they have been successfully used in several applications, such as biomedical ones, obtaining optimal results in underwater environments is still a challenge due to the heterogeneity of water and lighting conditions, and the scarcity of labeled datasets. Even more, the existing deep learning techniques oriented to semantic segmentation only provide low resolution results, lacking the enough spatial details for a high performance monitoring. To address these challenges, a combined loss function based on the active contour theory and level set methods is proposed to refine the spatial segmentation resolution and quality. To evaluate the method, a new underwater dataset with pixel annotations for three classes (fish, seafloor, and water) was created using images from publicly accessible datasets like SUIM, RockFish, and DeepFish. The performance of architectures of convolutional neural networks (CNNs), such as UNet and DeepLabV3+, trained with different loss functions (cross entropy, dice, and active contours) was compared, finding that the proposed combined loss function improved the segmentation results by around 3%, both in the metric Intercept Over Union (IoU) as in Hausdorff Distance (HD).
On the Construction of New Lightweight Involutory MDS Matrices in Generalized Subfield Form

Maximum Distance Separable (MDS) matrices are used as the main component of diffusion layers in block ciphers.MDS matrices have the optimal diffusion properties and the maximum branch number, which is a criterion to measure diffusion rate and security against linear and differential cryptanalysis.However, it is a challenging problem to construct hardware-friendly MDS matrices with optimal or close to optimal circuits, especially for involutory ones.In this paper, we consider the generalized subfield construction method given in (1) from the global optimization perspective and then give new 4×4 involutory MDS matrices over F 2 3 and F 2 5 .After that, we present 1,176 (= 28 × 42) new 4 × 4 involutory and MDS diffusion matrices by 33 XORs and depth 3.This new record also improves the previously best-known cost of 38 XOR gates.
Efficient Machine Learning on Edge Computing Through Data Compression Techniques

This paper discusses the increasing amount of data handled by companies and the need to use Big Data and Data Analytics to extract value from this data.However, due to the large amount of data collected, challenges related to the computational capacity of machines often arise when performing this analysis to acquire relevant information for the organization, especially when we are using edge computing.The paper aims to train machine learning models using compressed data, with two compression techniques applied to the original data.The results show that models trained with compressed data achieved similar accuracy to those trained with uncompressed data, and different compression techniques were compared.The research extended a previous study by analyzing the use of autoencoders for compression and reducing both instances and dimensionality of the dataset.The accuracy rate of the models when trained with compressed data instead of original data was maintained.
Dynamic Optimization Fabrics for Motion Generation

Optimization fabrics are a geometric approach to real-time local motion generation, where motions are designed by the composition of several differential equations that exhibit a desired motion behavior. We generalize this framework to dynamic scenarios and nonholonomic robots and prove that fundamental properties can be conserved. We show that convergence to desired trajectories and avoidance of moving obstacles can be guaranteed using simple construction rules of the components. In addition, we present the first quantitative comparisons between optimization fabrics and model predictive control and show that optimization fabrics can generate similar trajectories with better scalability, and thus, much higher replanning frequency (up to 500 Hz with a 7 degrees of freedom robotic arm). Finally, we present empirical results on several robots, including a nonholonomic mobile manipulator with 10 degrees of freedom and avoidance of a moving human, supporting the theoretical findings.
A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography

Video services are evolving from traditional two-dimensional video to virtual reality and holograms, which offer six degrees of freedom to users, enabling them to freely move around in a scene and change focus as desired. However, this increase in freedom translates into stringent requirements in terms of ultra-high bandwidth (in the order of Gigabits per second) and minimal latency (in the order of milliseconds). To realize such immersive services, the network transport, as well as the video representation and encoding, have to be fundamentally enhanced. The purpose of this tutorial article is to provide an elaborate introduction to the creation, streaming, and evaluation of immersive video. Moreover, it aims to provide lessons learned and to point at promising research paths to enable truly interactive immersive video applications toward holography.
UAV Visual Navigation System based on Digital Twin

In recent years, many scholars have carried out researchs on UAV digital twin from various aspects. However, the research is still in the preliminary stage, and there are still some problems, such as incomplete data and model fusion, poor migration of algorithm policy, poor relation between virtual and physical space, and lack of extensibility of application scenarios. In order to explore the application potential of digital twin technology in UAV fields, this paper introduces digital twin into UAV monocular visual navigation. Therefore, this paper proposes a digital twin(DT)-based framework integrating with deep neural network, which consists of physical space, virtual space, twin data layer and application layer. Next, the multi-modal decision model with decoupling methods in application layer consisting of perception model and control model is built to explore the global optimal solution and control the behaviors of UAV. Finally, the digital twin system and decision model are verified in virtual space and physical space respectively. The results shows that the UAV visual navigation system based on digital twin reduces the cost of application, algorithm development and deployment, and improves the migration ability of navigation policy. Compared with the baselines, the proposed decision model has the best navigation performance in both virtual space and physical space. Compared with the navigation policy without the decoupling method, the performance index is improved by about 8.6% in virtual space and 2.7 times in physical space.
The Status and Challenges of Image Data Augmentation Algorithms

Abstract Image data-augmentation algorithms effectively trump the problem of insufficient training samples for deep learning in some application fields, and it is typically for scholars to choose some of them for various computer vision tasks. But as the algorithms develop rapidly, the early proposed classification that the data-augmentation algorithms are sorted into classical ways and generating methods is no more suitable, because such classification misses some other meaningful strategies. Besides, it is frustrating for someone to decide which is the exact method to undertake, though there are too many optional algorithms to choose. Towards the goal of making some suggestions, the paper categorizes image data-augmentation algorithms into three kinds from the perspective of algorithm strategy, and they are matrix transformation algorithm, feature expansion algorithm and model generation based on neural network algorithm. The paper analyzes the typical algorithm principle, performance, application scenarios, research status and future challenges, and forecasts the development trend of data augmentation algorithms. The paper can provide academic reference for data augmentation algorithm in the fields of medicine and military.
Faceium–Face Tracking

Human faces are identified and localized via facial detection, which ignores any backdrop objects like curtains, windows, trees, etc. Each frame of the video is run through multiple stages of classifiers in OpenCV's Harr cascade, and if the frame passes through each level, the face is considered to be present; otherwise, the frame is dismissed from the classifier, which means that the face is not detected. When an image is detected, OpenCV also returns the height and breadth of the picture along with its cartesian coordinates. The center coordinates of the image can be determined from these coordinates. When the face is detected, these coordinates are sent via the pyserial library to the Arduino UNO. The camera is attached to one of the servos that are connected to the Arduino to create a pan/tilt mechanism. The servo will align to move the face toward the center of the screen when its coordinates are off-center.
Inter-Intra Modal Representation Augmentation With Trimodal Collaborative Disentanglement Network for Multimodal Sentiment Analysis

Recently, Multimodal Sentiment Analysis (MSA) is a challenging research area given its complex nature, and humans express emotional cues across various modalities such as language, facial expressions, and speech. Representation and fusion of features are the most crucial tasks in multimodal sentiment analysis research. However, in the current research, most methods ignore the importance of eliminating potential irrelevant features in the original features of each modality and cross-modal common feature. Moreover, the features extracted from all the modalities contain cluttered background noise and different occlusions noise, which negatively affects feature alignment. Different from these methods, we propose a novel Trimodal Collaborative Disentanglement Network (TCDN) to solve these problems in this paper. This work can obtain effective sentiment results on two aspects: i) Trimodal collaborative uses L1-norm to eliminate irrelevant features and unify the characteristics of the three modals (inter-modal). ii) Disentanglement network introduces an adversary noise by combining the original features of various single modalities and the common representation, alleviating the background noises within each modality (intramodal). This inter-intra modal feature augmentation method is the first work to obtain the common representation by implementing data augmentation as far as we know. Extensive experiments are completed on two benchmark datasets, including MOSI and MOSEI, demonstrating the superiority of the TCDN model over the state-of-the-art methods.
An Ontology for Social Media Data Analysis

Social media is one of the valuable information sources which present much data to the researchers. This information is mainly analyzed by machine learning and the deep learning methods, which is focused on feature engineering. We present a taxonomy of the different features. These features are learnt during the training for analyzing the textual information, specifically available on social platforms. This ontological view of the data will represent knowledge in a more understandable form besides interpreting the machine learning results for various tasks related to the social data analysis. We have taken Depression as the use-case purpose. The ontology is designed in the Protégé. The validation of the ontology is carried out with designed competency questions.
Visual Object Detection for Privacy-Preserving Federated Learning

Visual object detection is a computer vision technique based on deep learning.However, due to privacy issues, it is highly challenging to build an object detection model according to the current methods on the centrally stored training dataset.Federated learning is an effective approach to solving the challenge of training data collection by using distributed training.We propose FedVisionBC, a blockchainbased federated learning system for visual object detection that addresses the challenges of single point of failure, model poisoning attacks, and membership inference attacks in traditional federated learning.In the FedVisionBC system, we set up an aggregation node and a verification node instead of a central server to solve the single point of failure problem.We establish a security mechanism that uses encryption techniques, verification nodes, and smart contracts to resist model poisoning attacks.Experimental results show that FedVisionBC can accomplish the object detection task when the percentage of malicious clients is less than 60%.We also propose a new algorithm, ADPFedAvg, to prevent membership inference attacks, which relies on user-level differential privacy technology and the federal average algorithm.Experimental results indicate that ADPFedAvg can achieve a large-scale visual object detection model with differential privacy protection, while only a negligible cost in predictive accuracy.
Blockchain-Based Decentralized Authentication Model for IoT-Based E-Learning and Educational Environments

In recent times, technology has advanced significantly and is currently being integrated into educational environments to facilitate distance learning and interaction between learners. Integrating the Internet of Things (IoT) into education can facilitate the teaching and learning process and expand the context in which students learn. Nevertheless, learning data is very sensitive and must be protected when transmitted over the network or stored in data centers. Moreover, the identity and the authenticity of interacting students, instructors, and staff need to be verified to mitigate the impact of attacks. However, most of the current security and authentication schemes are centralized, relying on trusted third-party cloud servers, to facilitate continuous secure communication. In addition, most of these schemes are resource-intensive; thus, security and efficiency issues arise when heterogeneous and resource-limited IoT devices are being used. In this paper, we propose a blockchain-based architecture that accurately identifies and authenticates learners and their IoT devices in a decentralized manner and prevents the unauthorized modification of stored learning records in a distributed university network. It allows students and instructors to easily migrate to and join multiple universities within the network using their identity without the need for user re-authentication. The proposed architecture was tested using a simulation tool, and measured to evaluate its performance. The simulation results demonstrate the ability of the proposed architecture to significantly increase the throughput of learning transactions (40%), reduce the communication overhead and response time (26%), improve authentication efficiency (27%), and reduce the IoT power consumption (35%) compared to the centralized authentication mechanisms. In addition, the security analysis proves the effectiveness of the proposed architecture in resisting various attacks and ensuring the security requirements of learning data in the university network.
Visual Interpretation of Machine Learning: Genetical Classification of Apatite from Various Ore Sources

Machine learning provides solutions to a diverse range of problems in high-dimensional datasets in geosciences. However, machine learning is generally criticized for being an enigmatic black box as it focusses on results but ignores the processes. To address this issue, we used supervised decision boundary maps (SDBM) to visually illustrate and interpret the machine learning process. We constructed a SDBM to classify the ore genetics from 1551 trace element data of apatite in various types of deposits. Attribute-based visual explanation of multidimensional projections (A-MPs) was introduced to SDBM to further demonstrate the correlation between features and machine learning process. Our results show that SDBM explores the interpretability of machine learning process and the A-MPs approach reveals the role of trace elements in machine learning classification. Combining SDBM and A-MPs methods, we propose intuitive and accurate discrimination diagrams and the most indicative elements for ore genetic types. Our work provides novel insights for the visualization application of geo-machine learning, which is expected to be a powerful tool for high-dimensional geochemical data analysis and mineral deposit exploration.
Multimodal Named Entity Recognition with Bottleneck Fusion and Contrastive Learning

Multimodal named entity recognition (MNER) is the task of recognizing named entities in multimodal context. Existing methods focus on utilizing co-attention mechanism to discover the relationships between multiple modalities. However, they still have two deficiencies: First, current methods fail to fuse the multimodal representations in a fine-grained way, which may bring noise of visual modalities. Second, current methods ignore bridging the semantic gap between heterogeneous modalities. To solve the above issues, we propose a novel MNER method with bottleneck fusion and contrastive learning (BFCL). Specifically, we first incorporate the transformer-based bottleneck fusion mechanism, subsequently, information between different modalities can only be exchanged through several bottleneck tokens, thus reducing the noise propagation. Then we propose two decoupled image-text contrastive losses to align the unimodal representations, making the representations of semantically similar modalities closer, while the representations of semantically different modalities farther away. Experimental results demonstrate that our method is competitive to the state-of-the-art models, and achieves 74.54% and 85.70% F1-scores on Twitter-2015 and Twitter-2017 datasets, respectively.
The Lexicon of Emoji? Conventionality Modulates Processing of Emoji

Emoji have been ubiquitous in communication for over a decade, yet how they derive meaning remains underexplored. Here, we examine an aspect fundamental to linguistic meaning-making: the degree to which emoji have conventional lexicalized meanings and whether that conventionalization affects processing in real-time. Experiment 1 establishes a range of meaning agreement levels across emoji within a population; Experiment 2 measures accuracy and response times to word-emoji pairings in a match/mismatch task. In this experiment, we found that accuracy and response time both correlated significantly with the level of population-wide meaning agreement from Experiment 1, suggesting that lexical access of single emoji may be comparable to that of words, even out of context. This is consistent with theories of a multimodal lexicon that stores links between meaning, structure, and modality in long-term memory. Altogether, these findings suggest that emoji can allow a range of entrenched, lexicalized representations.
Dynamic Hunting Leadership optimization: Algorithm and applications

The Dynamic Hunting Leadership (DHL) algorithm is an innovative heuristic technique that draws inspiration from nature to find almost optimal solutions for various optimization problems. It consists of four variants, each highlighting distinct leadership strategies to guide the hunting process. The development of the algorithm was based on the realization that effective leadership during the hunting process can significantly improve its efficacy. The concept behind these methods is to dynamically modify the number of leaders, which can enhance the algorithm's performance. The stability of DHL variants in exploring the unknown area of the search space and exploitation phases is compared, and the advantages of exploration or exploration ability for the different variants of DHL are discussed. Moreover, the results are compared with more than twenty well-known algorithms. The efficacy of the proposed algorithms in discovering near-optimal solutions is tested across several real-world applications, and the outcomes demonstrate that the DHL algorithm outperforms other competing algorithms. The distinctiveness of the DHL algorithm is its ability to identify the global minimum on various benchmark problems and its superior performance in enhancing the objective value for the welded beam design problem and tension–compression spring problem, surpassing the values achieved by other algorithms. The algorithms' performance is also tested on an optimal allocation problem of distributed generation (DG) and energy storage system (ESS) for balanced electrical distribution systems. The results show that all four variants of DHL obtained the global minimum for the problem. For the optimal control strategy problem for voltage regulators in three-phase unbalanced power systems, the DHL algorithm improved the objective value by 35.7% compared to the best results found by other algorithms. Based on the analysis and comparison of the best objective values and convergence behavior for all benchmarks and problems, the DHL method proves to be an effective and reliable optimization method.
Data hiding during image processing using capsule networks

In daily life, some conventional image processing operations, e.g., histogram equalization, and filtering, are widely used to improve the visual quality of digital images. This paper designs an image processing network based on CapsNets (capsule networks), in which additional data can be carried in the processed image. Given an image to be processed, the proposed network is able to achieve some conventional image processing operations with satisfactory results. Meanwhile, additional data can be embedded into the processed image during the process of training, and the existence of additional data cannot be discovered. In this way, additional data can be transmitted secretly via the processed image, which looks normal. Compared with existing data hiding algorithms that embed data by modifying image content, the proposal to embed data during the process of training is more secure. Experimental results verify the effectiveness of the proposed network, including the quality of the processed image, embedding capacity, and security.
On the local convergence of GANs with differential Privacy: Gradient clipping and noise perturbation

Generative Adversarial Networks (GANs) are known to implicitly memorize details of sensitive data used to train them. To prevent privacy leakage, many approaches have been conducted. One of the most popular approaches is Differential Private Gradient Descent GANs (DPGD GANs), where the discriminator's gradients are clipped, and an appropriate random noise is added to the clipped gradients. In this article, a theoretical analysis of DPGD GAN convergence behavior is presented, and the effect of the clipping and noise perturbation operators on convergence properties is examined. It is proved that if the clipping bound is too small, it leads to instability in the training procedure. Then, assuming that the simultaneous/alternating gradient descent method is locally convergent to a fixed point and its operator is L-Lipschitz with L<1, the effect of noise perturbation on the last-iterate convergence rate is analyzed. Also, we show that parameters such as the privacy budget, the confidence parameter, the total number of training records, the clipping bound, the number of training iterations, and the learning rate, affect the convergence behavior of DPGD GANs. Furthermore, we confirm the effectiveness of these parameters on the convergence behavior of DPGD GANs through experimental evaluations.