OpenRaiser/Intern-Atlas huggingface dataset:
CCR: Facial Image Editing with Continuity, Consistency and Reversibility
Venue: arXiv cs.CV 2022
Year: 2022

Three problems exist in sequential facial image editing: incontinuous editing, inconsistent editing, and irreversible editing. Incontinuous editing is that the current editing can not retain the previously edited attributes. Inconsistent editing is that swapping the attribute editing orders can not yield the same results. Irreversible editing means that operating on a facial image is irreversible, especially in sequential facial image editing. In this work, we put forward three concepts and corresponding definitions: editing continuity, consistency, and reversibility. Then, we propose a novel model to achieve the goal of editing continuity, consistency, and reversibility. A sufficient criterion is defined to determine whether a model is continuous, consistent, and reversible. Extensive qualitative and quantitative experimental results validate our proposed model and show that a continuous, consistent and reversible editing model has a more flexible editing function while preserving facial identity. Furthermore, we think that our proposed definitions and model will have wide and promising applications in multimedia processing. Code and data are available at https://github.com/mickoluan/CCR.
INFINITY: A Simple Yet Effective Unsupervised Framework for Graph-Text Mutual Conversion
Venue: arXiv cs.CL 2022
Year: 2022

Graph-to-text (G2T) generation and text-to-graph (T2G) triple extraction are two essential tasks for constructing and applying knowledge graphs. Existing unsupervised approaches turn out to be suitable candidates for jointly learning the two tasks due to their avoidance of using graph-text parallel data. However, they are composed of multiple modules and still require both entity information and relation type in the training process. To this end, we propose INFINITY, a simple yet effective unsupervised approach that does not require external annotation tools or additional parallel information. It achieves fully unsupervised graph-text mutual conversion for the first time. Specifically, INFINITY treats both G2T and T2G as a bidirectional sequence generation task by fine-tuning only one pretrained seq2seq model. A novel back-translation-based framework is then designed to automatically generate continuous synthetic parallel data. To obtain reasonable graph sequences with structural information from source texts, INFINITY employs reward-based training loss by leveraging the advantage of reward augmented maximum likelihood. As a fully unsupervised framework, INFINITY is empirically verified to outperform state-of-the-art baselines for G2T and T2G tasks.
Deep Learning Based Page Creation for Improving E-Commerce Organic Search Traffic
Venue: arXiv cs.CL 2022
Year: 2022

Organic search comprises a large portion of the total traffic for e-commerce companies. One approach to expand company's exposure on organic search channel lies on creating landing pages having broader coverage on customer intentions. In this paper, we present a transformer language model based organic channel page management system aiming at increasing prominence of the company's overall clicks on the channel. Our system successfully handles the creation and deployment process of millions of new landing pages. We show and discuss the real-world performances of state-of-the-art language representation learning method, and reveal how we find them as the production-optimal solutions.
Efficient CNN with uncorrelated Bag of Features pooling
Venue: arXiv cs.CV 2022
Year: 2022

Despite the superior performance of CNN, deploying them on low computational power devices is still limited as they are typically computationally expensive. One key cause of the high complexity is the connection between the convolution layers and the fully connected layers, which typically requires a high number of parameters. To alleviate this issue, Bag of Features (BoF) pooling has been recently proposed. BoF learns a dictionary, that is used to compile a histogram representation of the input. In this paper, we propose an approach that builds on top of BoF pooling to boost its efficiency by ensuring that the items of the learned dictionary are non-redundant. We propose an additional loss term, based on the pair-wise correlation of the items of the dictionary, which complements the standard loss to explicitly regularize the model to learn a more diverse and rich dictionary. The proposed strategy yields an efficient variant of BoF and further boosts its performance, without any additional parameters.
Angular-based Edge Bundled Parallel Coordinates Plot for the Visual Analysis of Large Ensemble Simulation Data
Venue: arXiv cs.HC 2022
Year: 2022

With the continuous increase in the computational power and resources of modern high-performance computing (HPC) systems, large-scale ensemble simulations have become widely used in various fields of science and engineering, and especially in meteorological and climate science. It is widely known that the simulation outputs are large time-varying, multivariate, and multivalued datasets which pose a particular challenge to the visualization and analysis tasks. In this work, we focused on the widely used Parallel Coordinates Plot (PCP) to analyze the interrelations between different parameters, such as variables, among the members. However, PCP may suffer from visual cluttering and drawing performance with the increase on the data size to be analyzed, that is, the number of polylines. To overcome this problem, we present an extension to the PCP by adding Bézier curves connecting the angular distribution plots representing the mean and variance of the inclination of the line segments between parallel axes. The proposed Angular-based Parallel Coordinates Plot (APCP) is capable of presenting a simplified overview of the entire ensemble data set while maintaining the correlation information between the adjacent variables. To verify its effectiveness, we developed a visual analytics prototype system and evaluated by using a meteorological ensemble simulation output from the supercomputer Fugaku.
Challenges in Visual Anomaly Detection for Mobile Robots
Venue: arXiv cs.CV 2022
Year: 2022

We consider the task of detecting anomalies for autonomous mobile robots based on vision. We categorize relevant types of visual anomalies and discuss how they can be detected by unsupervised deep learning methods. We propose a novel dataset built specifically for this task, on which we test a state-of-the-art approach; we finally discuss deployment in a real scenario.
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation
Venue: arXiv cs.CL 2022
Year: 2022

Large Language Models (LLMs) have in recent years demonstrated impressive prowess in natural language generation. A common practice to improve generation diversity is to sample multiple outputs from the model. However, there lacks a simple and robust way of selecting the best output from these stochastic samples. As a case study framed in the context of question generation, we propose two prompt-based approaches to selecting high-quality questions from a set of LLM-generated candidates. Our method works under the constraints of 1) a black-box (non-modifiable) question generation model and 2) lack of access to human-annotated references -- both of which are realistic limitations for real-world deployment of LLMs. With automatic as well as human evaluations, we empirically demonstrate that our approach can effectively select questions of higher qualities than greedy generation.
Zero Trust Federation: Sharing Context under User Control toward Zero Trust in Identity Federation
Venue: arXiv cs.CR 2022
Year: 2022

To securely control access to systems, the concept of Zero Trust has been proposed. Access Control based on Zero Trust concept removes implicit trust and instead focuses on evaluating trustworthiness at every access request by using contexts. Contexts are information about the entity making an access request like the user and the device status. Consider the scenario of Zero Trust in an identity federation where the entity (Relying Party; RP) enforces access control based on Zero Trust concept. RPs should continuously evaluate trustworthiness by using collected contexts by themselves, but RPs where users rarely access cannot collect enough contexts on their own. Therefore, we propose a new federation called Zero Trust Federation (ZTF). In ZTF, contexts as well as identity are shared so that RPs can enforce access control based on Zero Trust concept. Federated contexts are managed by a new entity called Context Attribute Provider, which is independent of Identity Providers. We design a mechanism sharing contexts among entities in a ZTF by using the two protocols; context transport protocol based on Continuous Access Evaluation Protocol and user consent protocol based on User Managed Access. We implemented the ZTF prototype and evaluated the capability of ZTF in 4 use-cases.
Metamorphic Testing in Autonomous System Simulations
Venue: arXiv cs.SE 2022
Year: 2022

Metamorphic testing has proven to be effective for test case generation and fault detection in many domains. It is a software testing strategy that uses certain relations between input-output pairs of a program, referred to as metamorphic relations. This approach is relevant in the autonomous systems domain since it helps in cases where the outcome of a given test input may be difficult to determine. In this paper therefore, we provide an overview of metamorphic testing as well as an implementation in the autonomous systems domain. We implement an obstacle detection and avoidance task in autonomous drones utilising the GNC API alongside a simulation in Gazebo. Particularly, we describe properties and best practices that are crucial for the development of effective metamorphic relations. We also demonstrate two metamorphic relations for metamorphic testing of single and more than one drones, respectively. Our relations reveal several properties and some weak spots of both the implementation and the avoidance algorithm in the light of metamorphic testing. The results indicate that metamorphic testing has great potential in the autonomous systems domain and should be considered for quality assurance in this field.
Exact Outage Probability Analysis of the Mixed RF/FSO System With Variable-Gain Relays
Venue: arXiv cs.IT 2022
Year: 2022

This paper presents a unified analysis of the mixed radio-frequency (RF)/free-space optics (FSO) relaying system, with multiple variable-gain amplify-and-forward relays. The partial relay selection (PRS) is employed to select the active relay for further re-transmission. Due to fast fading statistics of the first RF hop, it is assumed that the channel state information of the RF link is outdated, which is used for both the relay gain adjustment and the PRS procedure. The RF hops are subject to the Rayleigh fading, while the FSO hop is affected by the atmospheric turbulence and the pointing errors. The intensity fluctuations of the optical signal caused by atmospheric turbulence are modeled by the general Malaga (M) distribution, which takes into account the effect of multiple scattered components. An exact expression for the outage probability is derived. In addition, high-signal-to-noise-ratio approximations are provided, which can be used to efficiently determine the outage probability floor. Numerical results are validated by Monte Carlo simulations, which are used to examine the effects of the system and channel parameters on the RF/FSO system performance.
Decentralized Distributed Expert Assisted Learning (D2EAL) approach for cooperative target-tracking
Venue: arXiv cs.RO 2022
Year: 2022

This paper addresses the problem of cooperative target tracking using a heterogeneous multi-robot system, where the robots are communicating over a dynamic communication network, and heterogeneity is in terms of different types of sensors and prediction algorithms installed in the robots. The problem is cast into a distributed learning framework, where robots are considered as 'agents' connected over a dynamic communication network. Their prediction algorithms are considered as 'experts' giving their look-ahead predictions of the target's trajectory. In this paper, a novel Decentralized Distributed Expert-Assisted Learning (D2EAL) algorithm is proposed, which improves the overall tracking performance by enabling each robot to improve its look-ahead prediction of the target's trajectory by its information sharing, and running a weighted information fusion process combined with online learning of weights based on a prediction loss metric. Theoretical analysis of D2EAL is carried out, which involves the analysis of worst-case bounds on cumulative prediction loss, and weights convergence analysis. Simulation studies show that in adverse scenarios involving large dynamic bias or drift in the expert predictions, D2EAL outperforms well-known covariance-based estimate/prediction fusion methods, both in terms of prediction performance and scalability.
On the Heritage of Crypto Assets -- Tales From the Crypt Protocol
Venue: arXiv cs.CR 2022
Year: 2022

We discuss some issues to the inheritance of crypto assets. We propose a distributed, privacy preserving, protocol to establish a consensus on the death of the owner of crypto assets: the Tales From the Crypt Protocol. Until the actual death of the owner no link can be made between public information and the corresponding crypto assets. This protocol is generic and could be incorparated into any arbitrary crypto platform.
Optimization of FPGA-based CNN Accelerators Using Metaheuristics
Venue: arXiv cs.NE 2022
Year: 2022

In recent years, convolutional neural networks (CNNs) have demonstrated their ability to solve problems in many fields and with accuracy that was not possible before. However, this comes with extensive computational requirements, which made general CPUs unable to deliver the desired real-time performance. At the same time, FPGAs have seen a surge in interest for accelerating CNN inference. This is due to their ability to create custom designs with different levels of parallelism. Furthermore, FPGAs provide better performance per watt compared to GPUs. The current trend in FPGA-based CNN accelerators is to implement multiple convolutional layer processors (CLPs), each of which is tailored for a subset of layers. However, the growing complexity of CNN architectures makes optimizing the resources available on the target FPGA device to deliver optimal performance more challenging. In this paper, we present a CNN accelerator and an accompanying automated design methodology that employs metaheuristics for partitioning available FPGA resources to design a Multi-CLP accelerator. Specifically, the proposed design tool adopts simulated annealing (SA) and tabu search (TS) algorithms to find the number of CLPs required and their respective configurations to achieve optimal performance on a given target FPGA device. Here, the focus is on the key specifications and hardware resources, including digital signal processors, block RAMs, and off-chip memory bandwidth. Experimental results and comparisons using four well-known benchmark CNNs are presented demonstrating that the proposed acceleration framework is both encouraging and promising. The SA-/TS-based Multi-CLP achieves 1.31x - 2.37x higher throughput than the state-of-the-art Single-/Multi-CLP approaches in accelerating AlexNet, SqueezeNet 1.1, VGGNet, and GoogLeNet architectures on the Xilinx VC707 and VC709 FPGA boards.
FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification
Venue: arXiv cs.CV 2022
Year: 2022

Unmanned aerial vehicles (UAVs) are now widely applied to data acquisition due to its low cost and fast mobility. With the increasing volume of aerial videos, the demand for automatically parsing these videos is surging. To achieve this, current researches mainly focus on extracting a holistic feature with convolutions along both spatial and temporal dimensions. However, these methods are limited by small temporal receptive fields and cannot adequately capture long-term temporal dependencies which are important for describing complicated dynamics. In this paper, we propose a novel deep neural network, termed FuTH-Net, to model not only holistic features, but also temporal relations for aerial video classification. Furthermore, the holistic features are refined by the multi-scale temporal relations in a novel fusion module for yielding more discriminative video representations. More specially, FuTH-Net employs a two-pathway architecture: (1) a holistic representation pathway to learn a general feature of both frame appearances and shortterm temporal variations and (2) a temporal relation pathway to capture multi-scale temporal relations across arbitrary frames, providing long-term temporal dependencies. Afterwards, a novel fusion module is proposed to spatiotemporal integrate the two features learned from the two pathways. Our model is evaluated on two aerial video classification datasets, ERA and Drone-Action, and achieves the state-of-the-art results. This demonstrates its effectiveness and good generalization capacity across different recognition tasks (event classification and human action recognition). To facilitate further research, we release the code at https://gitlab.lrz.de/ai4eo/reasoning/futh-net.
Blockchain-Oriented Services Computing in Action: Insights from a User Study
Venue: arXiv cs.SE 2022
Year: 2022

Blockchain architectures promise disruptive innovation but factually they pose many architectural restrictions to classical service-based applications and show considerable design, implementation, and operations overhead. Furthermore, the relation between such overheads and user benefits is not clear yet. To shed light on the aforementioned relations, a service-based blockchain architecture was designed and deployed as part of a field study in real-life experimentation. An observational approach was then performed to elaborate on the technology-acceptance of the service-based blockchain architecture in question. Evidence shows that the resulting architecture is, in principle, not different than other less complex equivalents; furthermore, the architectural limitations posed by the blockchain-oriented design demand a significant additional effort to be put onto even the simplest of functionalities. We conclude that further research shall be invested in clarifying further the design principles we learned as part of this study as well as any trade-offs posed by blockchain-oriented service design and operation.
UNav: An Infrastructure-Independent Vision-Based Navigation System for People with Blindness and Low vision
Venue: arXiv cs.CV 2022
Year: 2022

Vision-based localization approaches now underpin newly emerging navigation pipelines for myriad use cases from robotics to assistive technologies. Compared to sensor-based solutions, vision-based localization does not require pre-installed sensor infrastructure, which is costly, time-consuming, and/or often infeasible at scale. Herein, we propose a novel vision-based localization pipeline for a specific use case: navigation support for end-users with blindness and low vision. Given a query image taken by an end-user on a mobile application, the pipeline leverages a visual place recognition (VPR) algorithm to find similar images in a reference image database of the target space. The geolocations of these similar images are utilized in downstream tasks that employ a weighted-average method to estimate the end-user's location and a perspective-n-point (PnP) algorithm to estimate the end-user's direction. Additionally, this system implements Dijkstra's algorithm to calculate a shortest path based on a navigable map that includes trip origin and destination. The topometric map used for localization and navigation is built using a customized graphical user interface that projects a 3D reconstructed sparse map, built from a sequence of images, to the corresponding a priori 2D floor plan. Sequential images used for map construction can be collected in a pre-mapping step or scavenged through public databases/citizen science. The end-to-end system can be installed on any internet-accessible device with a camera that hosts a custom mobile application. For evaluation purposes, mapping and localization were tested in a complex hospital environment. The evaluation results demonstrate that our system can achieve localization with an average error of less than 1 meter without knowledge of the camera's intrinsic parameters, such as focal length.
Exploring The Design of Prompts For Applying GPT-3 based Chatbots: A Mental Wellbeing Case Study on Mechanical Turk
Venue: arXiv cs.HC 2022
Year: 2022

Large-Language Models like GPT-3 have the potential to enable HCI designers and researchers to create more human-like and helpful chatbots for specific applications. But evaluating the feasibility of these chatbots and designing prompts that optimize GPT-3 for a specific task is challenging. We present a case study in tackling these questions, applying GPT-3 to a brief 5-minute chatbot that anyone can talk to better manage their mood. We report a randomized factorial experiment with 945 participants on Mechanical Turk that tests three dimensions of prompt design to initialize the chatbot (identity, intent, and behaviour), and present both quantitative and qualitative analyses of conversations and user perceptions of the chatbot. We hope other HCI designers and researchers can build on this case study, for other applications of GPT-3 based chatbots to specific tasks, and build on and extend the methods we use for prompt design, and evaluation of the prompt design.
Zero-Forcing Based Downlink Virtual MIMO-NOMA Communications in IoT Networks
Venue: arXiv cs.IT 2022
Year: 2022

To support massive connectivity and boost spectral efficiency for internet of things (IoT), a downlink scheme combining virtual multiple-input multiple-output (MIMO) and nonorthogonal multiple access (NOMA) is proposed. All the single-antenna IoT devices in each cluster cooperate with each other to establish a virtual MIMO entity, and multiple independent data streams are requested by each cluster. NOMA is employed to superimpose all the requested data streams, and each cluster leverages zero-forcing detection to de-multiplex the input data streams. Only statistical channel state information (CSI) is available at base station to avoid the waste of the energy and bandwidth on frequent CSI estimations. The outage probability and goodput of the virtual MIMO-NOMA system are thoroughly investigated by considering Kronecker model, which embraces both the transmit and receive correlations. Furthermore, the asymptotic results facilitate not only the exploration of physical insights but also the goodput maximization. In particular, the asymptotic outage expressions provide quantitative impacts of various system parameters and enable the investigation of diversity-multiplexing tradeoff (DMT). Moreover, power allocation coefficients and/or transmission rates can be properly chosen to achieve the maximal goodput. By favor of Karush-Kuhn-Tucker conditions, the goodput maximization problems can be solved in closed-form, with which the joint power and rate selection is realized by using alternately iterating optimization.Besides, the optimization algorithms tend to allocate more power to clusters under unfavorable channel conditions and support clusters with higher transmission rate under benign channel conditions.