Reviewerly/exHarmony huggingface dataset:
FedNorm: An Efficient Federated Learning Framework with Dual Heterogeneity Coexistence on Edge Intelligence Systems

Federated learning (FL) is an emerging distributed learning paradigm, which aims to train machine learning models on geo-decentralized edge devices while keeping the training data stored locally. However, due to the scattered and diverse properties of edge devices, FL is often accompanied by typical heterogeneous features. One of the key challenges is statistical heterogeneity (aka non-independent identically distributed data, Non-IID), which leads to severe client-drift problem and unstable convergence. Moreover, the computational heterogeneity of devices can result in large computation time variation and thus exacerbate client-drift through inconsistent local training steps. The previous studies either ignore the client-drift problem or ignore the scatter in local gradient information, causing limited optimization effect. This paper proposes FedNorm framework to enable training Non-IID data on heterogeneous devices efficiently. First, a local model consistency update method is introduced to mitigate client-drift by allowing heterogeneous edge devices to implement different local training steps. Next, a federated gradient normalization method is introduced to reduce gradient scattering and achieves stable convergence of the model by balancing the gradient information of each edge device. We conducted extensive ablation experiments on different training tasks and training platforms with dual heterogeneity. The experimental results show that FedNorm achieves 1.52 × -3.52× speedup on convergence ratio and 7.38%-13.90% improvement in accuracy, compared to the state-of-the-art frameworks on CIFAR10.
HyperDetector: Detecting, Isolating, and Mitigating Timing Attacks in Virtualized Environments

We present a generic approach, called HyperDetector, to detect, isolate, and prevent ongoing timing based side-channel attacks that operate by measuring the execution times of short-running operations in virtualized environments. HyperDetector, being implemented at the level of hypervisor, uses a hardware extension for virtualization to intercept the rdtsc instructions, such that the consecutive pairs of time readings that are close to each other in time can be detected. Once potentially malicious time measurements are detected, noise is introduced into the measurements to prevent the ongoing attacks and the sequence of such measurements is analyzed at runtime by using a sliding window-based approach to determine the processes involved in the attacks. In the experiments, HyperDetector detected all the malicious processes with a perfect accuracy after these processes made few time measurements, reduced the success rates of the attacks from between 98%–99% to between 0%–0.5%, and did so with a runtime overhead of 1.14%.
Visual Transformers with Primal Object Queries for Multi-Label Image Classification

Multi-label image classification is about predicting a set of class labels that can be considered as orderless sequential data. Transformers process the sequential data as a whole, therefore they are inherently good at set prediction. The first vision-based transformer model, which was proposed for the object detection task introduced the concept of object queries. Object queries are learnable positional encodings that are used by attention modules in decoder layers to decode the object classes or bounding boxes using the region of interests in an image. However, inputting the same set of object queries to different decoder layers hinders the training: it results in lower performance and delays convergence. In this paper, we propose the usage of primal object queries that are only provided at the start of the transformer decoder stack. In addition, we improve the mixup technique proposed for multi-label classification. The proposed transformer model with primal object queries improves the state-of-the-art class wise F1 metric by 2.1% and 1.8%; and speeds up the convergence by 79.0% and 38.6% on MS-COCO and NUS-WIDE datasets respectively.
AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Persons

Automatically generating videos in which synthesized speech is synchronized with lip movements in a talking head has great potential in many human-computer interaction scenarios. In this paper, we present an automatic method to generate synchronized speech and talking-head videos on the basis of text and a single face image of an arbitrary person as input. In contrast to previous text-driven talking head generation methods, which can only synthesize the voice of a specific person, the proposed method is capable of synthesizing speech for any person. Specifically, the proposed method decomposes the generation of synchronized speech and talking head videos into two stages, i.e., a text-to-speech (TTS) stage and a speech-driven talking head generation stage. The proposed TTS module is a face-conditioned multi-speaker TTS model that gets the speaker identity information from face images instead of speech, which allows us to synthesize a personalized voice on the basis of the input face image. To generate the talking head videos from the face images, a facial landmark-based method that can predict both lip movements and head rotations is proposed. Extensive experiments demonstrate that the proposed method is able to generate synchronized speech and talking head videos for arbitrary persons, in which the timbre of the synthesized voice is in harmony with the input face, and the proposed landmark-based talking head method outperforms the state-of-the-art landmark-based method on generating natural talking head videos.
PreSR: Neural-Enhanced Adaptive Streaming of VBR-Encoded Videos With Selective Prefetching

Variable Bitrate (VBR) video encoding can significantly improve the quality-of-experience (QoE) of viewing users due to its capability to provide much higher quality-to-bits ratio compared to Constant Bitrate (CBR) video encoding. However, the streaming of VBR-encoded videos suffers from large variance of video chunk size, which may directly result in frequent rebuffering if not properly handled. In this paper, we propose a novel neural-enhanced adaptive streaming framework for VBR-encoded videos called PreSR, which performs selective prefetching of video chunks to achieve a higher QoE for viewers. The design of PreSR is motivated by an important observation obtained from our measurement, namely, the video quality improvement and bandwidth savings brought by neural enhancement are more pronounced for low-resolution video chunks with complex scenes. By taking the above fact and the time required for neural enhancement into account, we formulate the problem into an optimization problem. Given that the problem is NP-hard, we design the PreSR framework, which is based on the model predictive control theory and also considers key features of VBR-encoded videos. PreSR parallelizes the download of video chunks and model inference processes to fully utilize the available compute resources. Finally, we conduct extensive experiments with real traces, and the results show that PreSR outperforms the state-of-the-art algorithms with an improvement up to 11.25% in terms of the average QoE.
RSMOT: Remote Sensing Multi-Object Tracking Network with Local Motion Prior for Objects in Satellite Videos

Multi-object tracking (MOT) in satellite videos is a new and challenging task. The difficulties stem from the extremely small objects and the low contrast between objects and background. To tackle the challenges of MOT in satellite videos, a multi-object tracking method is proposed in this paper to incorporate the local motion prior into the network. Specifically, we design a local cost volume construction module to obtain tracking offsets between adjacent frames. Based on the tracking offsets, features of previous frames can be propagated to the current frame to incorporate spatio-temporal information. We conduct extensive experiments on videos from Jilin-1 satellite, and the results demonstrate the effectiveness of the proposed method.
Genetic-based Joint Dynamic Pruning and Learning Algorithm to Boost DNN Performance

The learning process of a biological system is a continuous phenomenon with limited external interventions. As learning progress, the numbers of neurons and synapses are modified based on the circumstances, which will impact the learning rate (i.e., learning faster as learning progresses). However, different from the characteristics of biological systems, the current research on deep learning is focused on a fixed training process with a predefined architecture to obtain optimal accuracy. On the other hand, while model pruning techniques have been studied to eliminate redundant neurons or synapses, most of them are applied after training but before deployment to accelerate the inference. In this paper, we integrate pruning into training and propose a genetic-based joint pruning and learning algorithm that monitors the training process and prunes the redundant parameters while training. As a result, our method can accelerate both training and inference. The proposed genetic-based method is well-suited for both training from scratch and online learning tasks by considering both the importance and stability of the parameters in the pruning process. The effectiveness of the proposed algorithm is evaluated on different neural network architectures and datasets, which demonstrates significant improvements for the training under both batch learning and incremental learning scenarios.
Methodology for Creating a Community Corpus Using a Wikibase Knowledge Graph

The Wikibase environment is provided by the Wikimedia foundation. It provides a suite of tools and a collaborative environment and implements Semantic Web techniques to model and store data. It has been shown that the Wikibase environment is well adapted to store and manage documents with their meta-data and that it can be associated with a search mechanism that facilitates the accessibility to this set of information. Several implementations of such a web platform have been proposed for various communities, but no methodology has been yet proposed. A methodology would help to identify tasks and easily reproduce them for the new implementation of community Wikibases, together with the search function that provides access to the information stored. In this paper, we propose a methodology for domain-specific Wikibase instantiation, for documents and their meta-data, and we identify some tools related to this methodology.
Bitcoin P2P Network Measurements: A Testbed Study of the Effect of Peer Selection on Transaction Propagation and Confirmation Times

Bitcoin is the first and the most extensive decentralized electronic cryptocurrency system that uses blockchain technology. It uses a peer-to-peer (P2P) network to operate without a central authority and propagate system information such as transactions or blockchain updates. The communication between participating nodes is highly relying on the underlying network infrastructure to facilitate a platform. Understanding the impact of peer formation strategies, peer list, and delay is vital in understanding node to node communication and the system performance. Therefore, we performed an extensive study on the transaction characteristics of Bitcoin through a testbed. The analysis shows that peer selection strategies affect the transaction propagation and confirmation times. In particular, better performance, in terms of smaller transaction confirmation time and lower number of temporary forks, may be achieved by adjusting the default nearby-based peer selection strategy.
An open and private-by-design Active and Healthy Ageing smart home platform

The increasing trend in the percentage of older people among the European population, and in the number of older people living alone, poses a number of challenges, some of which can be addressed from digital solutions monitoring and supporting comfort, physical activity and health. Nowadays, research efforts seek to ensure, through technology-based solutions, the quality of life as people age and the needs that arise during the process. The home environment can host modern home-automation solutions supporting information gathering, monitoring and early intervention to prevent risks to lead to major consequences. Smart homes can thus make a positive contribution to the Active and Healthy Ageing paradigm. However, one of the biggest limitations preventing smart homes from being a reality is the lack of interoperability among different-vendor devices or the fact that most of the commer-tial solutions rely on cloud-based solutions. This work faces these limitations and proposes an architecture that, based on the use of commercial sensors, ensures that information is gathered and processed locally, independently of the commercial solution provided by the manufacturer of the device. The architecture has been experimentally validated by implementing a monitoring system based on environmental and security sensors interconnected by means of the ZigBee wireless communication protocol. The proposed system also supports caregivers caring for their older people who will be granted access to collected data. Thus, the system offers a remote monitoring system based on an information storage and visualisation solution. This system is intended to provide a solution that improves the quality of life and extends the time of independent living at home for older people.
A Reliable Online Method for Joint Estimation of Focal Length and Camera Rotation

Linear perspective cues deriving from regularities of the built environment can be used to recalibrate both intrinsic and extrinsic camera parameters online, but these estimates can be unreliable due to irregularities in the scene, uncertainties in line segment estimation and background clutter. Here we address this challenge through four initiatives. First, we use the PanoContext panoramic image dataset [27] to curate a novel and realistic dataset of planar projections over a broad range of scenes, focal lengths and camera poses. Second, we use this novel dataset and the YorkUrbanDB [4] to systematically evaluate the linear perspective deviation measures frequently found in the literature and show that the choice of deviation measure and likelihood model has a huge impact on reliability. Third, we use these findings to create a novel system for online camera calibration we call $$f\textbf{R}$$ , and show that it outperforms the prior state of the art, substantially reducing error in estimated camera rotation and focal length. Our fourth contribution is a novel and efficient approach to estimating uncertainty that can dramatically improve online reliability for performance-critical applications by strategically selecting which frames to use for recalibration.
These Aren’t The Caches You’re Looking For: Stochastic Channels on Randomized Caches

Many security attacks utilize cache side-channels to subvert memory security mechanisms. By exploiting co-resident processes that share the cache, attackers simply monitor and manipulate their own cache state to infer activities of victim processes, exfiltrate program secrets, or establish unauthorized communication channels. To counter these unwanted cache channels, defenses that randomize line address to cache set mappings have emerged. In this paper, we present a novel cache communication technique based on stochastic channels, which are probabilistically formed and evicted to establish communication. The approach is coupled with noise reduction techniques in order to tolerate system noise and the cases where probabilistic channels do not form. Since the approach is stochastic, it is agnostic to the cache-indexing mechanism; thus, cache randomization defenses, including those with re-mapping capabilities, have little effect on this channel. We demonstrate the covert communication channel working seamlessly on real-world caches, randomized cache designs proposed in literature, and futuristic randomized caches. We then demonstrate the applicability of our error-resilient covert channel with the Spectre attack on randomized caches. We close by noting that stochastic cache channels suggest that randomized caches are not a durable defense against covert cache communication, and we advocate for the community to focus on cache partitioning technologies and novel defenses such as in-cache encryption and side-channel resistant accelerators.
An OCR System : Towards Mobile Device

The OCR system has been widely used in many fields, such as office automation, file management, online education, etc. However, due to its high requirements on computing resources, the system is mostly runing on desktop or server platforms. In recent years, the performance of mobile devices has been improving, and they have been increasingly used in people's life and work. In this paper, we design an OCR system for mobile devices, which can better apply the performance of mobile devices, improve the stability of mobile OCR tasks, and reduce its dependence on network state by using various strategies to slimming and enhance the model applied by server, the total size of the final model is only 20M.
Distributed Meta-Learning for Context Based on Culture Heritage Collection Data in an Immersive Reality System

This article proposes a new workflow for Culture Heritage (CH) Collection Data in which distributed meta-learning with context-based fusion is required. The multidisciplinary research outlines a new paradigm for collaborative creation of a context- based data fusion CH workflow. Whose criterion restricts the search space to the cultural sector to include multiple other CH object types, texts and predominantly analyzes visualizations of CH object metadata. The approach promotes user-driven content creation and offsets economic models, thereby rewarding creators and performers. In response to these challenges, we propose a framework for bringing about massive and real-time presence and awareness on the Internet through an Internet of Things infrastructure to connect objects, context and intangible assets. We enable an online virtual world that incorporates proximity of pervasive information, objects, people, processes, data and places. Finally, we investigate some new ways to achieve immersive experiences via distributed meta-learning computing and point out the necessity to do more with regard to collaborative interaction creation.
A New Public Key Encryption Using Dickson Polynomials Over Finite Field with $$2^{m}$$

The Dickson polynomials are widely used in different context. Because of the permutation behaviors and semi-group property under composition, its application can also be noticed in cryptography. In this paper, we propose a new public key encryption scheme based on Dickson polynomial of first kind over a finite field with $$2^{m}$$ , where m is the product of two large primes p and q, with primes $$2^p-1$$ and $$2^q-1$$ . The security of the proposed scheme is equally dependent on the Integer Factorization Problem (IFP) and the Discrete Dickson Problem (DDP). We also compared the encryption-decryption performance of the proposed scheme with some other existing schemes and found that the performance of the proposed scheme is better compared to them.
Predicting Human-Object Interactions in Egocentric Videos

Egocentric videos provide a rich source of hand-object interactions that support action recognition. However, prior to action recognition, one may need to detect the presence of hands and objects in the scene. In this work, we propose an action estimation architecture based on the simultaneous detection of the hands and objects in the scene. For the hand and object detection, we have adapted well known YOLO architecture, leveraging its inference speed and accuracy. We experimentally determined the best performing architecture for our task. After obtaining the hand and object bounding boxes, we select the most likely objects to interact with, i.e., the closest objects to a hand. The rough estimation of the closest objects to a hand is a direct approach to determine hand-object interaction. After identifying the scene and alongside a set of per-object and global actions, we could determine the most suitable action we are performing in each context.
NFSP-PER: An efficient sampling NFSP-based method with prioritized experience replay

In two-player imperfect-information games (IIGs), Nash equilibrium strategy is a typical goal when solving these games. Neural fictitious self-play (NFSP) is a popular method to find Nash equilibrium in IIGs, which is the first end-to-end method to obtain the Nash equilibrium strategy. In NFSP, it computes best response with deep reinforcement learning method and fits its Nash equilibrium strategy by supervised learning. However, the training of nfsp requires a large number of sample data, and the interactive cost of obtaining such data is often very high. How to realize the efficient training of network under limited samples is an urgent problem to be solved. In this work, we propose a new NFSP-based variant, called NFSP-PRE, which combines NFSP and prioritized experience replay mechanism to improve the sample training efficiency. Extensive experimental results show that the proposed NFSP-PER effectively improves the sample learning efficiency compared with the comparison methds.
Multi-sector Risk Management Framework for Analysis Cybersecurity Challenges and Opportunities

This paper describes an approach to analyse transversal and inter-sectoral cybersecurity challenges and opportunities: dedicated risk assessment and management framework, which can be used to develop cybersecurity technology roadmaps. This multi-sector assessment framework is able to prioritise and evaluate cybersecurity risks in trans-sectoral and inter-sectoral contexts as well as supports proper resource allocations and mitigation actions. To achieve this goal, the analysis of known risk management and risk assessment frameworks was performed, and results are presented in this paper. Also, an overview on transversal, inter-sectoral and multi-sectoral technological challenges and opportunities is provided. The result of this analysis is an architecture of the ECHO Multi-sector Assessment Framework, which was described in detail, including identified and analysed transversal aspects, multi-sector dependencies, and technological challenges and opportunities determine the input data for the framework. This solution is applicable in many sectors, such as energy, healthcare, maritime transportation, or defence, however it can also be extended to others. The architecture of the framework proposed supports the design of cybersecurity technology roadmap and the definition of governance models.