Reviewerly/exHarmony huggingface dataset:
TFormer: A Transmission-Friendly ViT Model for IoT Devices

Deploying high-performance vision transformer (ViT) models on ubiquitous Internet of Things (IoT) devices to provide high-quality vision services will revolutionize the way we live, work, and interact with the world. Due to the contradiction between the limited resources of IoT devices and resource-intensive ViT models, the use of cloud servers to assist ViT model training has become mainstream. However, due to the larger number of parameters and floating-point operations (FLOPs) of the existing ViT models, the model parameters transmitted by cloud servers are large and difficult to run on resource-constrained IoT devices. To this end, this article proposes a transmission-friendly ViT model, TFormer, for deployment on resource-constrained IoT devices with the assistance of a cloud server. The high performance and small number of model parameters and FLOPs of TFormer are attributed to the proposed hybrid layer and the proposed partially connected feed-forward network (PCS-FFN). The hybrid layer consists of nonlearnable modules and a pointwise convolution, which can obtain multitype and multiscale features with only a few parameters and FLOPs to improve the TFormer performance. The PCS-FFN adopts group convolution to reduce the number of parameters. The key idea of this article is to propose TFormer with few model parameters and FLOPs to facilitate applications running on resource-constrained IoT devices to benefit from the high performance of the ViT models. Experimental results on the ImageNet-1K, MS COCO, and ADE20K datasets for image classification, object detection, and semantic segmentation tasks demonstrate that the proposed model outperforms other state-of-the-art models. Specifically, TFormer-S achieves 5% higher accuracy on ImageNet-1K than ResNet18 with 1.4× fewer parameters and FLOPs.
TIDE: A Novel Approach to Constructing Timed-Release Encryption

In ESORICS 2021, Chvojka et al. introduced the idea of taking a time-lock puzzle and using its solution to generate the keys of a public key encryption (PKE) scheme [12]. They use this to define a timed-release encryption (TRE) scheme, in which the secret key is encrypted ‘to the future’ using a time-lock puzzle, whilst the public key is published. This allows multiple parties to encrypt a message to the public key of the PKE scheme. Then, once a solver has spent a prescribed length of time evaluating the time-lock puzzle, they obtain the secret key and hence can decrypt all of the messages. In this work we introduce TIDE (TIme Delayed Encryption), a novel approach to constructing timed-release encryption based upon the RSA cryptosystem, where instead of directly encrypting the secret key to the future, we utilise number-theoretic techniques to allow the solver to factor the RSA modulus, and hence derive the decryption key. We implement TIDE on a desktop PC and on Raspberry Pi devices validating that TIDE is both efficient and practically implementable. We provide evidence of practicality with an extensive implementation study detailing the source code and practical performance of TIDE.
A Survey on Knowledge Graph Embedding

Knowledge graph (KG) is used to represent the relationships between different concepts in the real world. It is a special network in which nodes represent entities and edges represent relationships. KGs can intuitively model the connections between facts, but in many applications, there are certain limitations in directly using symbolic logic to represent knowledge in KGs and perform calculations, making it difficult to achieve expected results in downstream tasks. Meanwhile, with the explosive growth of Internet capacity, the traditional KG structure faces the problems of computational inefficiency and management difficulties. To alleviate these problems, Knowledge graph embedding (KGE) is proposed to improve the computational efficiency by embedding entities and relations in the KG into a low-dimensional, dense and continuous vector space, and thus the solution of some problems in the knowledge graph is transformed into vector operations. Moreover, KGE also can be used as a pre-trained model which is more beneficial to downstream applications, such as applications based on deep learning. In this paper, we classify KGE into three categories, namely translational distance models, semantic matching models and neural network based models. The advantages and disadvantages of different embedding methods are compared, while the main applications of KGE are summarized. Some current challenges of KGE are summarized, as well as some views on the future research directions of KGE.
Unsupervised Domain Adaptation with Implicit Pseudo Supervision for Semantic Segmentation

Pseudo-labelling is a popular technique in unsupervised domain adaptation for semantic segmentation. However, pseudo labels are noisy and inevitably have confirmation bias due to the discrepancy between source and target domains and training process. In this paper, we train the model by the pseudo labels which are implicitly produced by itself to learn new complementary knowledge about target domain. Specifically, we propose a tri-learning architecture, where every two branches produce the pseudo labels to train the third one. And we align the pseudo labels based on the similarity of the probability distributions for each two branches. To further implicitly utilize the pseudo labels, we maximize the distances of features for different classes and minimize the distances for the same classes by triplet loss. Extensive experiments on GTA5 to Cityscapes and SYNTHIA to Cityscapes tasks show that the proposed method has considerable improvements.
Post-Quantum Blockchain-Based Secure Service Orchestration in Multi-Cloud Networks

Existing network service provisioning and lifecycle management (LCM) workflows rely on heterogeneous devices from multiple vendors and collaboration between multiple network actors, which can lead to numerous trust management and interoperability issues. Blockchain networks (BCNs) are a revolutionary way to establish trust in untrusted environments. In this complex environment, BCNs can help ensure transparency and security for network service LCM. However, BCNs are also vulnerable to quantum attacks. Advances in quantum computing will challenge the security of existing blockchain technology based on Public Key Infrastructure (PKI) technologies. In this paper, we explore how network services can be managed in a multiple administrative scenario. Our approach uses BCNs to track the operational steps of network service instantiation metrics while benefiting from the security features of post-quantum cryptography (PQC). Together with the use of N-th degree Truncated polynomial Ring Units (NTRU) as an example of a PQC algorithm that relies on the parallelization power of the Toom-Cook computation method with different security levels, we have shown that Quorum can provide a lower average time-to-write value compared to other BCNs considered (Ethereum and Hyperledger). At the end of the paper, we discuss the evaluation results and future directions regarding the coexistence of PQC algorithms and BCNs for network service orchestration and service federation between multiple administrative domains.
Coordinated Multi-Agent Exploration, Rendezvous, &amp; Task Allocation in Unknown Environments with Limited Connectivity

The lack of communication between agents in a multi-robot system is often regarded as a limiting factor that can affect and delay cooperative exploration and exploitation of cluttered and uncertain environments. On the contrary, this paper proposes a complete planning framework to enable cooperative behavior without the need for constant communication between robots, demonstrating drastic improvements in task completion and coverage time as compared to both fully connected robotic networks and widely used frontier-based exploration methods. Specifically, the proposed scheme considers three behaviors: i) exploration, promoting separation and disconnection, ii) rendezvous to reconnect and share information gained during exploration, and iii) task allocation for prioritized objectives. Exploration is achieved via a Sobel edge detection frontier algorithm that enables navigation of unknown complex (both convex and non-convex) environments. Once a task is discovered, a multi-objective weighted sum optimization method is proposed for allocating tasks based on prioritization and expectation estimation. The utility, generality, and scalability of the proposed approach is demonstrated using extensive simulations and experiments with unmanned ground vehicles in various cluttered environments.
Learning with Free Object Segments for Long-Tailed Instance Segmentation

One fundamental challenge in building an instance segmentation model for a large number of classes in complex scenes is the lack of training examples, especially for rare objects. In this paper, we explore the possibility to increase the training examples without laborious data collection and annotation. We find that an abundance of instance segments can potentially be obtained freely from object-centric images, according to two insights: (i) an object-centric image usually contains one salient object in a simple background; (ii) objects from the same class often share similar appearances or similar contrasts to the background. Motivated by these insights, we propose a simple and scalable framework FreeSeg for extracting and leveraging these “free” object foreground segments to facilitate model training in long-tailed instance segmentation. Concretely, we investigate the similarity among object-centric images of the same class to propose candidate segments of foreground instances, followed by a novel ranking of segment quality. The resulting high-quality object segments can then be used to augment the existing long-tailed datasets, e.g., by copying and pasting the segments onto the original training images. Extensive experiments show that FreeSeg yields substantial improvements on top of strong baselines and achieves state-of-the-art accuracy for segmenting rare object categories. Our code is publicly available at https://github.com/czhang0528/FreeSeg .
Improving Adversarially Robust Few-shot Image Classification with Generalizable Representations

Few-Shot Image Classification (FSIC) aims to recognize novel image classes with limited data, which is significant in practice. In this paper, we consider the FSIC problem in the case of adversarial examples. This is an extremely challenging issue because current deep learning methods are still vulnerable when handling adversarial examples, even with massive labeled training samples. For this problem, existing works focus on training a network in the meta-learning fashion that depends on numerous sampled few-shot tasks. In comparison, we propose a simple but effective baseline through directly learning generalizable representations without tedious task sampling, which is robust to unforeseen adversarial FSIC tasks. Specifically, we introduce an adversarial-aware mechanism to establish auxiliary supervision via feature-level differences between legitimate and adversarial examples. Furthermore, we design a novel adversarial-reweighted training manner to alleviate the imbalance among adversarial examples. The feature purifier is also employed as post-processing for adversarial features. Moreover, our method can obtain generalizable representations to remain superior transferability, even facing cross-domain adversarial examples. Extensive experiments show that our method can significantly outperform state-of-the-art adversarially robust FSIC methods on two standard benchmarks.
Wild Horse Optimizer-Based Spiral Updating for Feature Selection

Feature selection (FS) is a vital and challenging process in many domains, including data mining, data clustering, text mining, education, biology, medicine, public health, machine learning, image processing, and others.The greedy and comprehensive algorithm methods cannot identify the best subset amid the rising number of features.Thus, swarm-based algorithms are becoming more popular for identifying the best group of features.This study relies on the spiral-updated position of the Whale Optimization Algorithm (WOA) to propose an improved version of the Wild Horse Optimizer (WHO).This improvement enhances the WHO's ability to update solutions and explore various possibilities in the search domain.The proposed method (WHOW) was assessed using two experiments to confirm the efficacy of the improved optimizer.The first experiment was global optimization using CEC 2019 benchmark functions, whereas the second was an FS by testing 20 benchmark datasets.The results obtained using the proposed WHOW method were compared with some popular algorithms over the benchmark datasets of global optimization and FS.The experimental results reflect the superiority of WHOW in the solutions to different optimization problems and its ability to select prominent features over most benchmark datasets.These results are due to the implementation of bubble nets in the WHO using spiral movement, which promotes flexibility and performance.
Learning Semantic Associations for Mirror Detection

Mirrors generally lack a consistent visual appearance, making mirror detection very challenging. Although recent works that are based on exploiting contextual contrasts and corresponding relations have achieved good results, heavily relying on contextual contrasts and corresponding relations to discover mirrors tend to fail in complex real-world scenes, where a lot of objects, e.g., doorways, may have similar features as mirrors. We observe that humans tend to place mirrors in relation to certain objects for specific functional purposes, e.g., a mirror above the sink. Inspired by this observation, we propose a model to exploit the semantic associations between the mirror and its surrounding objects for a reliable mirror localization. Our model first acquires class-specific knowledge of the surrounding objects via a semantic side-path. It then uses two novel modules to exploit semantic associations: 1) an Associations Exploration (AE) Module to extract the associations of the scene objects based on fully connected graph models, and 2) a Quadruple-Graph (QG) Module to facilitate the diffusion and aggregation of semantic association knowledge using graph convolutions. Extensive experiments show that our method outperforms the existing methods and sets the new state-of-the-art on both PMD dataset (f-measure: 0.844) and MSD dataset (f-measure: 0.889). Code is available at https://github.com/guanhuankang/Learning-Semantic-Associations-for-Mirror-Detection.
Modeling 3D Layout For Group Re-Identification

Group re-identification (GReID) attempts to correctly associate groups with the same members under different cameras. The main challenge is how to resist the membership and layout variations. Existing works attempt to incorporate layout modeling on the basis of appearance features to achieve robust group representations. However, layout ambiguity is introduced because these methods only consider the 2D layout on the imaging plane. In this paper, we overcome the above limitations by 3D layout modeling. Specifically, we propose a novel 3D transformer (3DT) that reconstructs the relative 3D layout relationship among members, then applies sampling and quantification to preset a series of layout tokens along three dimensions, and selects the corresponding tokens as layout features for each member. Furthermore, we build a synthetic GReID dataset, City1M, including 1.84M images, 45K persons and 11.5K groups with 3D annotations to alleviate data shortages and poor annotations. To the best of our knowledge, 3DT is the first work to address GReID with 3D perspective, and the City1M is the currently largest dataset. Several experiments show the superiority of our 3DT and City1M. Our project has been released on https://github.com/LinlyAC/City1M-dataset.
Efficient Federated Learning for AIoT Applications Using Knowledge Distillation

As a promising distributed machine learning paradigm, federated learning (FL) trains a central model with decentralized data without compromising user privacy, which makes it widely used by Artificial Intelligence Internet of Things (AIoT) applications. However, the traditional FL suffers from model inaccuracy, since it trains local models only using hard labels of data while useful information of incorrect predictions with small probabilities is ignored. Although various solutions try to tackle the bottleneck of the traditional FL, most of them introduce significant communication overhead, making the deployment of large-scale AIoT devices a great challenge. To address the above problem, this article presents a novel distillation-based FL (DFL) method that enables efficient and accurate FL for AIoT applications. By using knowledge distillation (KD), in each round of FL training, our approach uploads both the soft targets and local model gradients to the cloud server for aggregation, where the aggregation results are then dispatched to AIoT devices for the next round of local training. During the DFL local training, in addition to hard labels, the model predictions approximate soft targets, which can improve model accuracy by leveraging the knowledge of soft targets. To further improve our DFL model performance, we design a dynamic adjustment strategy of loss function weights for tuning the ratio of KD and FL, which can maximize the synergy between soft targets and hard labels. Comprehensive experimental results on well-known benchmarks show that our approach can significantly improve the model accuracy of FL without introducing significant communication overhead.
Blending On-Body and Mid-Air Interaction in Virtual Reality

On-body interfaces, which leverage the human body’s surface as an input or output platform, can provide new opportunities for designing VR interaction. However, it remains unclear how on-body interfaces can best support current VR systems that mainly rely on mid-air interaction. We propose BodyOn, a collection of six design patterns that leverage combined on-body and mid-air interfaces to achieve more effective 3D interaction. Specifically, a user may use thumb-on-finger gestures, finger-on-arm gestures, or on-body displays with mid-air input, including hand movement and orientation, to complete an interaction task. To test our design concepts, we implemented example interaction techniques based on BodyOn that can assist users in various 3D interaction tasks. We further conducted an expert evaluation using the techniques as probes to elicit immediate design issues that emerge from the novel combination of on-body and midair interaction. We provide insights that can inspire and inform the design of future 3D user interfaces.
Exploring the Impact of Visual Information on Intermittent Typing in Virtual Reality

For touch typists, using a physical keyboard ensures optimal text entry task performance in immersive virtual environments. However, successful typing depends on the user’s ability to accurately position their hands on the keyboard after performing other, non-keyboard tasks. Finding the correct hand position depends on sensory feedback, including visual information. We designed and conducted a user study where we investigated the impact of visual representations of the keyboard and users’ hands on the time required to place hands on the homing bars of a keyboard after performing other tasks. We found that this keyboard homing time decreased as the fidelity of visual representations of the keyboard and hands increased, with a video pass-through condition providing the best performance. We discuss additional impacts of visual representations of a user’s hands and the keyboard on typing performance and user experience in virtual reality.
FaaSt: Optimize makespan of serverless workflows in federated commercial FaaS

Nowadays, scientists migrate workflow applications on serverless Function-as-a-Service (FaaS) platforms in a form of so called function choreographies (FCs) to benefit from FaaS high elasticity and instantly spawning numerous functions. How-ever, the heterogeneous nature of federated FaaS overburdens decisions for the most appropriate configuration setup. Unfor-tunately, related work mainly support either (i) scheduling of serverful workflow applications that run on virtual machines or (ii) container-based algorithms to schedule individual functions on specific container (executor). Either approach is hard to implement for FCs in federated FaaS; the former due to specifics of the FaaS deployment model, while the latter because they are primarily focused on bag of functions and reducing startup latency down to microseconds. Such optimization is negligible for scientific FCs whose functions may run hundreds of seconds due to enormous compute and I/O operations to distributed cloud storage. Instead, scientific FCs would benefit from schedulers that select the appropriate FaaS provider, cloud region, and memory settings. To bridge this gap in scheduling scientific FCs, this paper introduces FaaSt, a novel list-based FC scheduler that optimizes makespan of an FC that runs functions in federated FaaS. The evaluation with three other schedulers showed that FaaSt overcomes limitations of a single FaaS region and generates speedup of up to 2.82× when running FCs across four cloud regions compared to a single region. Moreover, FaaSt achieves speedup of up to 1.74 × compared to the other state-of-the-art FC schedulers across the same four regions.
More Efficient Dishonest Majority Secure Computation over $$\mathbb {Z}_{2^k}$$ via Galois Rings

In this work we present a novel actively secure multiparty computation protocol in the dishonest majority setting, where the computation domain is a ring of the type $$\mathbb {Z}_{2^k}$$ . Instead of considering an "extension ring" of the form $$\mathbb {Z}_{2^{k+\kappa }}$$ as in SPD $$\mathbb {Z}_{2^k}$$ (Cramer et al., CRYPTO 2018) and its derivatives, we make use of an actual ring extension, or more precisely, a Galois ring extension $$\mathbb {Z}_{p^k}[\texttt{X}]/(h(\texttt{X}))$$ of large enough degree, in order to ensure that the adversary cannot cheat except with negligible probability. These techniques have been used already in the context of honest majority MPC over $$\mathbb {Z}_{p^k}$$ , and to the best of our knowledge, our work constitutes the first study of the benefits of these tools in the dishonest majority setting. Making use of Galois ring extensions requires great care in order to avoid paying an extra overhead due to the use of larger rings. To address this, reverse multiplication-friendly embeddings (RMFEs) have been used in the honest majority setting (e.g. Cascudo et al., CRYPTO 2018), and more recently in the dishonest majority setting for computation over $$\mathbb {Z}_2$$ (Cascudo and Gundersen, TCC 2020). We make use of the recent RMFEs over $$\mathbb {Z}_{p^k}$$ from (Cramer et al., CRYPTO 2021), together with adaptations of some RMFE optimizations introduced in (Abspoel et al., ASIACRYPT 2021) in the honest majority setting, to achieve an efficient protocol that only requires in its online phase $$12.4k(n-1)$$ bits of amortized communication complexity and one round of communication for each multiplication gate. We also instantiate the necessary offline phase using Oblivious Linear Evaluation (OLE) by generalizing the approach based on Oblivious Transfer (OT) proposed in MASCOT (Keller et al., CCS 2016). To this end, and as an additional contribution of potential independent interest, we present a novel technique using Multiplication-Friendly Embeddings (MFEs) to achieve OLE over Galois ring extensions using black-box access to an OLE protocol over the base ring $$\mathbb {Z}_{p^k}$$ without paying a quadratic cost in terms of the extension degree. This generalizes the approach in MASCOT based on Correlated OT Extension. Finally, along the way we also identify a bug in a central proof in MASCOT, and we implicitly present a fix in our generalized proof.