Reviewerly/exHarmony huggingface dataset:
Low-Resolution Face Recognition Enhanced by High-Resolution Facial Images

Despite recent advances in high-resolution (HR) face recognition, recognizing identities from low-resolution (LR) facial images remains challenging due to the absence of facial shape and detail. Current research focuses solely on reducing the distribution discrepancy between the HR and LR embeddings from the output layer, rather than thoroughly investigating the superiority of HR facial images for improved performance. In this paper, we propose a novel low-resolution face recognition method enhanced by the guidance of high-resolution facial images in both feature map space and embedding space. Specifically, in feature map space, the similarity constraint across the multilayer feature maps is adopted to align the intermediate features of facial images. Then we introduce multiple generators to recover HR images from extracted feature maps and utilize the reconstructed loss to supplement the missing facial details in LR images. In embedding space, we propose a supervised auxiliary contrastive loss to encourage the paired HR and LR embedding from the same class to be pulled together, whereas those from different classes are pushed apart. The one-to-many matching strategy and the adaptive weight adjustment strategy are applied to make the network adapt to the inputs of different resolutions. Experiments on four benchmark datasets with both synthesized and realistic LR facial images demonstrate the superiority of the proposed method to state-of-the-art.
Video Inference for Human Mesh Recovery with Vision Transformer

Human Mesh Recovery (HMR) from an image is a challenging problem because of the inherent ambiguity of the task. Existing HMR methods utilized either temporal information or kinematic relationships to achieve higher accuracy, but there is no method using both. Hence, we propose “Video Inference for Human Mesh Recovery with Vision Transformer (HMR-ViT)” that can take into account both temporal and kinematic information. In HMR-ViT, a Temporal-kinematic Feature Image is constructed using feature vectors obtained from video frames by an image encoder. When generating the feature image, we use a Channel Rearranging Matrix (CRM) so that similar kinematic features could be located spatially close together. The feature image is then further encoded using Vision Transformer, and the SMPL pose and shape parameters are finally inferred using a regression network. Extensive evaluation on the 3DPW and Human3.6M datasets indicates that our method achieves a competitive performance in HMR.
Graph Transformer for Physical Rehabilitation Evaluation

Physical rehabilitation is a medical specialty that focuses on the restoration of body functions in the safest and most effective way possible. During a rehabilitation exercise session, patients' behavior reflects their health status and is an important indicator of the treatment outcome. Building an automatic system for the evaluation of human motion quality in an objective and reliable way can be used in medicine to establish a differential diagnosis, to choose the adequate treatment, or for patient monitoring. Deep learning has become the state-of-the-art in Human Action and Human Behavior Recognition from videos. Most of the state-of-the-art model architectures are CNNs based and often use RNNs to calculate temporal dependencies among frames, and ignore the topological structure of the human body. In this work, we propose to use a Graph Transformer network with spatial and temporal attention mechanisms for physical rehabilitation evaluation. First, we used a standard Transformer network with a selfattention mechanism, then we take advantage of graph skeletons as inputs for a two-stream spatio-temporal graph network with both spatial and temporal attention mechanisms. We used our model on UI-PRMD, a benchmark dataset that provides skeleton data using motion capture systems. Our results show that our attention-based based ST-GCN models outperform the state-of-the-art methods on quality score prediction and binary classification.
New Insights on Weight Estimation from Face Images

Weight is a soft biometric trait which estimation is useful in numerous health related applications such as remote estimation from a health professional or at-home daily monitoring. In scenarios when a scale is unavailable or the subject is unable to cooperate, i.e. road accidents, estimating a person's weight from face appearance allows for a contactless measurement. In this article, we define an optimal transfer learning protocol for a ResNet50 architecture obtaining better performances than the state-of-the-art thus moving one step forward in closing the gap between remote weight estimation and physical devices. We also demonstrate that gender-splitting, image cropping and hair occlusion play an important role in weight estimation which might not necessarily be the case in face recognition. We use up-to-date explainability tools to illustrate and validate our assumptions. We conduct extensive simulations on the most popular publicly available face dataset annotated by weight to ensure a fair comparison with other approaches and we aim to overcome its flaws by presenting our self-collected database composed of 400 new images.
Analyzing the Impact of Shape &amp; Context on the Face Recognition Performance of Deep Networks

In this article, we analyze how changing the underlying 3D shape of the base identity in face images can distort their overall appearance, especially from the perspective of deep face recognition. As done in popular training data augmentation schemes, we graphically render real and synthetic face images with randomly chosen or best-fitting 3D face models to generate novel views of the base identity. We compare deep features generated from these images to assess the perturbation these renderings introduce into the original identity. We perform this analysis at various degrees of facial yaw with the base identities varying in gender and ethnicity. Additionally, we investigate if adding some form of context and background pixels in these rendered images, when used as training data, further improves the downstream performance of a face recognition model. Our experiments demonstrate the significance of facial shape in accurate face matching and underpin the importance of contextual data for network training.
Hierarchically Organized Computer Vision in Support of Multi-Faceted Search for Missing Persons

Missing person searches are typically initiated with a description of a person that includes their age, race, clothing, and gender, possibly supported by a photo. Unmanned Aerial Systems (sUAS) imbued with Computer Vision (CV) capabilities, can be deployed to quickly search an area to find the missing person; however, the search task is far more difficult when a crowd of people is present, and only the person described in the missing person report must be identified. It is particularly challenging to perform this task on the potentially limited resources of an sUAS. We therefore propose AirSight, as a new model that hierarchically combines multiple CV models, exploits both onboard and off-board computing capabilities, and engages humans interactively in the search. For illustrative purposes, we use AirSight to show how a person's image, extracted from an aerial video can be matched to a basic description of the person. Finally, as a work-in-progress paper, we describe ongoing efforts in building an aerial dataset of partially occluded people and physically deploying AirSight on our sUAS.
Multicloud Deployment of AI Workflows Using FaaS and Storage Services

Multicloud deployments are getting traction for benefits such as agility, avoiding vendor lock-in, resilience, etc. Use of Function-as-a-Service (FaaS) platforms is emerging as a preferred choice for deploying AI workflows. These platforms from different cloud service providers (CSPs) have unique specifications and cost models. The onus is on a user to find a cost-performance optimal mapping of its application workflows to FaaS and its associated services in a multicloud deployment setting. In this work, we have presented an empirical study of multicloud deployment of AI inference-workflows using FaaS and cloud storage services. Our evaluation shows that the cost for executing AI workflow reduces by 83% using an optimal combination of CSPs over a naive deployment. Further, we also propose and evaluate analytical models for estimating the cost of deployment in multicloud using FaaS and storage services.
Delay-Aware Container Scheduling in Kubernetes

Kubernetes is a powerful tool to manage containerized applications, which is also regarded as one promising platform to support microservices in edge computing. The scheduler is a key component of Kubernetes. It allocates each pod (i.e., a set of running containers) to one worker node (i.e., a machine). The default scheduler in Kubernetes is designed for the cloud environment containing homogeneous nodes. However, IoT edge nodes usually have various computing power and network bandwidth. This article proposes a delay-aware container scheduling (DACS) algorithm to address the issue of node heterogeneity in edge computing. To efficiently assign pods to worker nodes, DACS takes account of not only residual resources of worker nodes but also potential delays caused by the pod assignment. We build a Kubernetes cluster by VMware to evaluate system performance. Experimental results reveal that DACS can significantly reduce both processing and network delays, thereby helping Kubernetes perform more efficiently in an edge environment.
TAICHI: Transform Your Secret Exploits Into Mine From a Victim's Perspective

Acquiring and analyzing exploits, which take advantage of vulnerabilities to conduct malicious actions, are crucial for victims (and defenders) when responding to system compromising incidents. However, exploits are sensitive and valuable assets that are not available to victims. The most common resource available for victims to investigate is network traffic, which covers the exploitation period. Thus reconstructing exploits from network traffic is demanded. In practice, the reconstruction process is performed manually, thus inefficient and non-scalable. In this article, we present an automated solution <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TAICHI</monospace> to reconstruct exploits from network traffic, able to generate replica exploits and facilitate timely incident analysis. By nature, a working exploit has to satisfy (1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">path constraints</i> which ensure the program path same as the original exploit's is explored and the same vulnerability is triggered, and (2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">exploit constraints</i> which ensure the same exploitation strategy is applied, e.g., to bypass deployed defenses or to stitch multiple gadgets together. We propose a hybrid solution to this problem by integrating techniques including multi-version execution (MVE), dynamic taint analysis (DTA), and concolic execution. We have implemented a prototype of <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TAICHI</monospace> on x86 and x86-64 Linux and tested it on the Cyber Grand Challenge (CGC) dataset, several Capture the Flag (CTF) challenges, and Metasploit exploit modules targeting real world applications. The evaluation results showed that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TAICHI</monospace> could reconstruct exploits efficiently with a high success rate. Moreover, it could be applied to production environments without disrupting running services, and could reconstruct exploits even if only one round of exploitation traffic is available.
Physical-Layer Secret Key Generation Based on Bidirectional Convergence Feature Learning Convolutional Network

Physical-layer secret key generation (PLKG) is a new research area that has emerged in recent years. It is aimed at scenarios where legitimate IoT devices communicate directly, interacting with confidential information for lower overhead and higher security by using wireless channel. When applying it to wireless feature extraction, noise removal is not taken into account in current deep learning networks. To address these problems, the PLKG scheme based on bidirectional convergence feature learning convolutional network (BCFL-based scheme) is proposed, which consists of neural network called BCFL and a new quantization method to achieve better secret key generation. Unlike existing PLKG schemes that enabling both parties to communicate for obtaining higher channel feature similarities, when training it, channel state information (CSI) obtained by channel estimation for two legitimate devices during coherent time is used as inputs; and mean square error (MSE) between two outputs is used as a result of loss function for iterative training. Thus, it can obtain better denoising ability with guaranteed low computational resource consumption, and two legitimate devices can obtain highly correlated channel features. Multiple quantization method is also proposed to address low secret key generation rate (KGR) and low-secret key randomness (KR). The results show that the proposed BCFL-based scheme has a lower MSE than other schemes in different scenarios, indicating that it has better capability to learn channel reciprocity; and secret key error rate (KER) and time consumption are only about 50% of other schemes, which is a significant performance improvement.
A Stochastic Second-Order Proximal Method for Distributed Optimization

We propose a distributed stochastic second-order proximal (St-SoPro) method that enables agents in a network to cooperatively minimize the sum of their local loss functions without any centralized coordination. St-SoPro incorporates a decentralized second-order approximation into an augmented Lagrangian function, and randomly samples the local gradients and Hessian matrices to update, so that it is efficient in solving large-scale problems. We show that for restricted strongly convex and smooth problems, the agents linearly converge in expectation to a neighborhood of the optimum, and the neighborhood can be arbitrarily small under proper parameter settings. Simulations over real machine learning datasets demonstrate that St-SoPro outperforms several state-of-the-art methods in terms of convergence speed as well as computation and communication costs.
Deep metric learning for computer vision: A brief overview

Objective functions that optimize deep neural networks play a vital role in creating an enhanced feature representation of the input data. Although cross-entropy-based loss formulations have been extensively used in a variety of supervised deep-learning applications, these methods tend to be less adequate when there is large intraclass variance and low interclass variance in input data distribution. Deep metric learning seeks to develop methods that aim to measure the similarity between data samples by learning a representation function that maps these data samples into a representative embedding space. It leverages carefully designed sampling strategies and loss functions that aid in optimizing the generation of a discriminative embedding space even for distributions having low interclass and high intraclass variances. In this chapter, we will provide an overview of recent progress in this area and discuss state-of-the-art deep metric learning approaches.
<i>Mjolnir</i>: Breaking the Glass in a Publicly Verifiable Yet Private Manner

This paper formally investigates the problem of unauthorized yet required access to electronically protected information, a.k.a. Break-the-Glass (BtG) access. Reflecting on the rising deployment of such protocols in the current digitized healthcare system, we present Mjolnir, a blockchain-based BtG framework that offers accountability of unauthorized accesses by healthcare practitioners, dependable right of notification to patients, and privacy of healthcare records accesses. Mjolnir is a smart contract-based protocol which provides undisputed public verifiability of the identity of BtG access entities while maintaining their anonymity except from concerned individual patients, hence protecting the patients’ privacy. We employ an application specific non-interactive cryptographic zero knowledge proof system which ensures that the signing entity (healthcare practitioner) belongs to a given authorized group and that the anonymity of their identity is only revocable by a given opening entity (patient). The security of our system relies on the hardness of the discrete logarithm and decisional Diffie–Hellman problems in elliptic curve groups, and the utilized proof system requires no trusted setup. We formally define and prove the security goals of Mjolnir, provide a proof of concept blockchain implementation on Ethereum, and report on performance experiments and comparisons with other generic zero knowledge proof systems.
Differential Fault Attack on Rasta and $\text{FiLIP}_{\text{DSM}}$

In this paper we propose Differential Fault Attack (DFA) on two Fully Homomorphic Encryption (FHE) friendly stream ciphers Rasta and FiLIP DSM .Design criteria of Rasta rely on affine layers and nonlinear layers, whereas FiLIP DSM relies on permutations and a nonlinear filter function.Here we show that the secret key of these two ciphers can be recovered by injecting only 1 bit fault in the initial state.Our DFA on full round (# rounds = 6) Rasta with 219 block size requires only one block (i.e., 219 bits) of normal and faulty keystream bits.In the case of our DFA on FiLIP-430 (one instance of FiLIP DSM ), we need 30000 normal and faulty keystream bits.
The Platypus of the Quantum Channel Zoo

Understanding quantum channels and the strange behavior of their capacities is a key objective of quantum information theory. Here we study a remarkably simple, low-dimensional, single-parameter family of quantum channels with exotic quantum information-theoretic features. As the simplest example from this family, we focus on a qutrit-to-qutrit channel that is intuitively obtained by hybridizing together a simple degradable channel and a completely useless qubit channel. Such hybridizing makes this channel’s capacities behave in a variety of interesting ways. For instance, the private and classical capacity of this channel coincide and can be explicitly calculated, even though the channel does not belong to any class for which the underlying information quantities are known to be additive. Moreover, the quantum capacity of the channel can be computed explicitly, given a clear and compelling conjecture is true. This “spin alignment conjecture,” which may be of independent interest, is proved in certain special cases and additional numerical evidence for its validity is provided. Finally, we generalize the qutrit channel in two ways, and the resulting channels and their capacities display similarly rich behavior. In the companion paper [1], we further show that the qutrit channel demonstrates superadditivity when transmitting quantum information jointly with a variety of assisting channels, in a manner unknown before.
Bayesian Inference for Stochastic Multipath Radio Channel Models

Stochastic radio channel models based on underlying point processes of multipath components (MPCs) have been studied intensively since the seminal papers of Turin and Saleh–Valenzuela (SV). Despite this, inference regarding parameters of these models has remained a major challenge. Current methods typically have a somewhat ad hoc flavor involving a multitude of steps requiring user specification of tuning parameters. In this article, we propose to instead adopt the principled framework of Bayesian inference to conduct inference for the SV model. The posterior distribution is not analytically tractable and we therefore compute approximations of the posterior using Markov chain Monte Carlo (MCMC) methods specific to point processes. To demonstrate the flexibility of our approach, we additionally propose a new multipath model and apply our inference method to it. The resulting inference methodology is computationally demanding and our successful implementation relies critically on our novel MPC updates within the MCMC sampler. We demonstrate the usefulness of our approach on simulated and real radio channel data.
RT-DLO: Real-Time Deformable Linear Objects Instance Segmentation

Deformable Linear Objects (DLOs) such as cables, wires, ropes, and elastic tubes are numerously present both in domestic and industrial environments. Unfortunately, robotic systems handling DLOs are rare and have limited capabilities due to the challenging nature of perceiving them. Hence, we propose a novel approach named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RT-DLO</i> for real-time instance segmentation of DLOs. First, the DLOs are semantically segmented from the background. Afterward, a novel method to separate the DLO instances is applied. It employs the generation of a graph representation of the scene given the semantic mask where the graph nodes are sampled from the DLOs center-lines whereas the graph edges are selected based on topological reasoning. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RT-DLO</i> is experimentally evaluated against both DLO-specific and general-purpose instance segmentation deep learning approaches, achieving overall better performances in terms of accuracy and inference time.
CoDi$: Randomized Caches Through Confusion and Diffusion

Cloud providers are increasingly exposed to malicious actors through transient attacks, such as Spectre and Meltdown. The cache hierarchy is the main target to build the required side-channels to leak data. Randomized caches can be employed to provide security but often rely on cryptographic primitives to deter side-channel attacks. These increase the access latency and deteriorates the system performance. This paper shows that randomized caches do not have to increase the cache access latency, and that their security does not have to rely on a cryptographic hash function or block-cipher. Herein, CoDi$ is proposed, a randomized last level cache that achieves security by tying the local and global states. Security is achieved through a higher miss energy consumption and occupied area, instead of penalizing performance. CoDi$ is able to evict any cached address by allowing addresses to be displaced in two levels of freedom, through hopscotch (local state) and cuckoo hashing (global state), without increasing the hit latency. Through these displacements, paths can be built to the eviction address. The security of CoDi$ relies on the hardness to control all possible displacement paths when a miss occurs, which requires control over the local and global states of the cache simultaneously. Confusion is generated when evicting an address, as there are many possible cache states that could result from this eviction. Also, for each executed eviction, multiple addresses in the cache are non-deterministically displaced, providing diffusion. Moreover, experimental analysis for a 48-bit secure CoDi$, using SPEC, NPB, and Polybench benchmarks, shows improvements in the number of instructions per cycle and up to 5% in misses, when compared to two state-of-the-art randomized caches.