OpenRaiser/Intern-Atlas huggingface dataset:
VToonify: Controllable High-Resolution Portrait Video Style Transfer
Venue: arXiv cs.CV 2022
Year: 2022

Generating high-quality artistic portrait videos is an important and desirable task in computer graphics and vision. Although a series of successful portrait image toonification models built upon the powerful StyleGAN have been proposed, these image-oriented methods have obvious limitations when applied to videos, such as the fixed frame size, the requirement of face alignment, missing non-facial details and temporal inconsistency. In this work, we investigate the challenging controllable high-resolution portrait video style transfer by introducing a novel VToonify framework. Specifically, VToonify leverages the mid- and high-resolution layers of StyleGAN to render high-quality artistic portraits based on the multi-scale content features extracted by an encoder to better preserve the frame details. The resulting fully convolutional architecture accepts non-aligned faces in videos of variable size as input, contributing to complete face regions with natural motions in the output. Our framework is compatible with existing StyleGAN-based image toonification models to extend them to video toonification, and inherits appealing features of these models for flexible style control on color and intensity. This work presents two instantiations of VToonify built upon Toonify and DualStyleGAN for collection-based and exemplar-based portrait video style transfer, respectively. Extensive experimental results demonstrate the effectiveness of our proposed VToonify framework over existing methods in generating high-quality and temporally-coherent artistic portrait videos with flexible style controls.
Trusted IP solution in multi-tenant cloud FPGA platform
Venue: arXiv cs.CR 2022
Year: 2022

Because FPGAs outperform traditional processing cores like CPUs and GPUs in terms of performance per watt and flexibility, they are being used more and more in cloud and data center applications. There are growing worries about the security risks posed by multi-tenant sharing as the demand for hardware acceleration increases and gradually gives way to FPGA multi-tenancy in the cloud. The confidentiality, integrity, and availability of FPGA-accelerated applications may be compromised if space-shared FPGAs are made available to many cloud tenants. We propose a root of trust-based trusted execution mechanism called \textbf{TrustToken} to prevent harmful software-level attackers from getting unauthorized access and jeopardizing security. With safe key creation and truly random sources, \textbf{TrustToken} creates a security block that serves as the foundation of trust-based IP security. By offering crucial security characteristics, such as secure, isolated execution and trusted user interaction, \textbf{TrustToken} only permits trustworthy connection between the non-trusted third-party IP and the rest of the SoC environment. The suggested approach does this by connecting the third-party IP interface to the \textbf{TrustToken} Controller and running run-time checks on the correctness of the IP authorization(Token) signals. With an emphasis on software-based assaults targeting unauthorized access and information leakage, we offer a noble hardware/software architecture for trusted execution in FPGA-accelerated clouds and data centers.
StyleTime: Style Transfer for Synthetic Time Series Generation
Venue: arXiv cs.LG 2022
Year: 2022

Neural style transfer is a powerful computer vision technique that can incorporate the artistic "style" of one image to the "content" of another. The underlying theory behind the approach relies on the assumption that the style of an image is represented by the Gram matrix of its features, which is typically extracted from pre-trained convolutional neural networks (e.g., VGG-19). This idea does not straightforwardly extend to time series stylization since notions of style for two-dimensional images are not analogous to notions of style for one-dimensional time series. In this work, a novel formulation of time series style transfer is proposed for the purpose of synthetic data generation and enhancement. We introduce the concept of stylized features for time series, which is directly related to the time series realism properties, and propose a novel stylization algorithm, called StyleTime, that uses explicit feature extraction techniques to combine the underlying content (trend) of one time series with the style (distributional properties) of another. Further, we discuss evaluation metrics, and compare our work to existing state-of-the-art time series generation and augmentation schemes. To validate the effectiveness of our methods, we use stylized synthetic data as a means for data augmentation to improve the performance of recurrent neural network models on several forecasting tasks.
Dynamic Control of Soft Robotic Arm: An Experimental Study
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, a reinforced soft robot prototype with a custom-designed actuator-space string encoder are created to investigate dynamic soft robotic trajectory tracking. The soft robot prototype embedded with the proposed adaptive passivity control and efficient dynamic model make the challenging trajectory tracking tasks possible. We focus on the exploration of tracking accuracy as well as the full potential of the proposed control strategy by performing experimental validations at different operation scenarios: various tracking speed and external disturbance. In all experimental scenarios, the proposed adaptive passivity control outperforms the conventional PD feedback linearization control. The experimental analysis details the advantage and shortcoming of the proposed approach, and points out the next steps for future soft robot dynamic control.
Fast Disparity Estimation from a Single Compressed Light Field Measurement
Venue: arXiv cs.CV 2022
Year: 2022

The abundant spatial and angular information from light fields has allowed the development of multiple disparity estimation approaches. However, the acquisition of light fields requires high storage and processing cost, limiting the use of this technology in practical applications. To overcome these drawbacks, the compressive sensing (CS) theory has allowed the development of optical architectures to acquire a single coded light field measurement. This measurement is decoded using an optimization algorithm or deep neural network that requires high computational costs. The traditional approach for disparity estimation from compressed light fields requires first recovering the entire light field and then a post-processing step, thus requiring long times. In contrast, this work proposes a fast disparity estimation from a single compressed measurement by omitting the recovery step required in traditional approaches. Specifically, we propose to jointly optimize an optical architecture for acquiring a single coded light field snapshot and a convolutional neural network (CNN) for estimating the disparity maps. Experimentally, the proposed method estimates disparity maps comparable with those obtained from light fields reconstructed using deep learning approaches. Furthermore, the proposed method is 20 times faster in training and inference than the best method that estimates the disparity from reconstructed light fields.
Overtwisting and Coiling Highly Enhances Strain Generation of Twisted String Actuators
Venue: arXiv cs.RO 2022
Year: 2022

Twisted string actuators (TSAs) have exhibited great promise in robotic applications by generating high translational force with low input torque. To further facilitate their robotic applications, it is strongly desirable but challenging to enhance their consistent strain generation while maintaining compliance. Existing studies predominantly considered overtwisting and coiling after the regular twisting stage to be undesirable non-uniform and unpredictable knots, entanglements, and coils formed to create an unstable and failure-prone structure. Overtwisting would work well for TSAs when uniform coils can be consistently formed. In this study, we realize uniform and consistent coil formation in overtwisted TSAs, which greatly increases their strain. Furthermore, we investigate methods for enabling uniform coil formation upon overtwisting the strings in a TSA and present a procedure to systematically "train" the strings. To the authors' best knowledge, this is the first study to experimentally investigate overtwisting for TSAs with different stiffnesses and realize consistent uniform coil formation. Ultra-high molecular-weight polyethylene (UHMWPE) strings form the stiff TSAs whereas compliant TSAs are realized with stretchable and conductive supercoiled polymer (SCP) strings. The strain, force, velocity, and torque of each overtwisted TSA was studied. Overtwisting and coiling resulted in approximately 70% strain in stiff TSAs and approximately 60% strain in compliant TSAs. This is more than twice the strain achieved through regular twisting. Lastly, the overtwisted TSA was successfully demonstrated in a robotic bicep.
LEADER: Learning Attention over Driving Behaviors for Planning under Uncertainty
Venue: arXiv cs.LG 2022
Year: 2022

Uncertainty on human behaviors poses a significant challenge to autonomous driving in crowded urban environments. The partially observable Markov decision processes (POMDPs) offer a principled framework for planning under uncertainty, often leveraging Monte Carlo sampling to achieve online performance for complex tasks. However, sampling also raises safety concerns by potentially missing critical events. To address this, we propose a new algorithm, LEarning Attention over Driving bEhavioRs (LEADER), that learns to attend to critical human behaviors during planning. LEADER learns a neural network generator to provide attention over human behaviors in real-time situations. It integrates the attention into a belief-space planner, using importance sampling to bias reasoning towards critical events. To train the algorithm, we let the attention generator and the planner form a min-max game. By solving the min-max game, LEADER learns to perform risk-aware planning without human labeling.
Reactive Anticipatory Robot Skills with Memory
Venue: arXiv cs.RO 2022
Year: 2022

Optimal control in robotics has been increasingly popular in recent years and has been applied in many applications involving complex dynamical systems. Closed-loop optimal control strategies include model predictive control (MPC) and time-varying linear controllers optimized through iLQR. However, such feedback controllers rely on the information of the current state, limiting the range of robotic applications where the robot needs to remember what it has done before to act and plan accordingly. The recently proposed system level synthesis (SLS) framework circumvents this limitation via a richer controller structure with memory. In this work, we propose to optimally design reactive anticipatory robot skills with memory by extending SLS to tracking problems involving nonlinear systems and nonquadratic cost functions. We showcase our method with two scenarios exploiting task precisions and object affordances in pick-and-place tasks in a simulated and a real environment with a 7-axis Franka Emika robot.
The path to 5G-Advanced and 6G Non-Terrestrial Network systems
Venue: arXiv cs.NI 2022
Year: 2022

Today, 5G networks are being worldwide rolled out, with significant benefits in our economy and society. However, 5G systems alone are not expected to be sufficient for the challenges that 2030 networks will experience, including, e.g., always-on networks, 1 Tbps peak data rate, <10 cm positioning, etc. Thus, the definition of evolutions of the 5G systems and their (r)evolutions are already being addressed by the scientific and industrial communities, targeting 5G-Advanced (5G-A) and 6G. In this framework, Non-Terrestrial Networks (NTN) have successfully been integrated in 3GPP Rel. 17 and it is expected that they will play an even more pivotal role for 5G-A (up to Rel. 20) and 6G systems (beyond Rel. 20). In this paper, we explore the path that will lead to 5G-A and 6G NTN communications, providing a clear perspective in terms of system architecture, services, technologies, and standardisation roadmap.
A Probabilistic Model of Activity Recognition with Loose Clothing
Venue: arXiv cs.RO 2022
Year: 2022

Human activity recognition has become an attractive research area with the development of on-body wearable sensing technology. With comfortable electronic-textiles, sensors can be embedded into clothing so that it is possible to record human movement outside the laboratory for long periods. However, a long-standing issue is how to deal with motion artefacts introduced by movement of clothing with respect to the body. Surprisingly, recent empirical findings suggest that cloth-attached sensor can actually achieve higher accuracy of activity recognition than rigid-attached sensor, particularly when predicting from short time-windows. In this work, a probabilistic model is introduced in which this improved accuracy and resposiveness is explained by the increased statistical distance between movements recorded via fabric sensing. The predictions of the model are verified in simulated and real human motion capture experiments, where it is evident that this counterintuitive effect is closely captured.
Formal Semantics of the CDL Language
Venue: arXiv cs.SE 2022
Year: 2022

We reverse-engineer a formal semantics of the Component Definition Language (CDL), which is part of the highly configurable, embedded operating system eCos. This work provides the basis for an analysis and comparison of the two variability-modeling languages Kconfig and CDL. The semantics given in this document are based on analyzing the CDL documentation, inspecting the source code of the toolchain, as well as testing the tools on particular examples.
An Algebraic-Geometry Approach to Prime Factorization
Venue: arXiv cs.CR 2022
Year: 2022

New algorithms for prime factorization that outperform the existing ones or take advantage of particular properties of the prime factors can have a practical impact on present implementations of cryptographic algorithms that rely on the complexity of factorization. Currently used keys are chosen on the basis of the present algorithmic knowledge and, thus, can potentially be subject to future breaches. For this reason, it is worth to investigate new approaches which have the potentiality of giving a computational advantage. The problem has also relevance in quantum computation, as an efficient quantum algorithm for prime factorization already exists. Thus, better classical asymptotic complexity can provide a better understanding of the advantages offered by quantum computers. In this paper, we reduce the factorization problem to the search of points of parametrizable varieties, in particular curves, over finite fields. The varieties are required to have an arbitrarily large number of intersection points with some hypersurface over the base field. For a subexponential or poly- nomial factoring complexity, the number of parameters have to scale sublinearly in the space dimension n and the complexity of computing a point given the parameters has to be subexponential or polynomial, respectively. We outline a procedure for building these varieties, which is illustrated with two constructions. In one case, we show that there are varieties whose points can be evaluated efficiently given a number of parameters not greater than n/2. In the other case, the bound is dropped to n/3. Incidentally, the first construction resembles a kind of retro-causal model. Retro-causality is considered one possible explanation of quantum weirdness.
An Overview of Violence Detection Techniques: Current Challenges and Future Directions
Venue: arXiv cs.CV 2022
Year: 2022

The Big Video Data generated in today's smart cities has raised concerns from its purposeful usage perspective, where surveillance cameras, among many others are the most prominent resources to contribute to the huge volumes of data, making its automated analysis a difficult task in terms of computation and preciseness. Violence Detection (VD), broadly plunging under Action and Activity recognition domain, is used to analyze Big Video data for anomalous actions incurred due to humans. The VD literature is traditionally based on manually engineered features, though advancements to deep learning based standalone models are developed for real-time VD analysis. This paper focuses on overview of deep sequence learning approaches along with localization strategies of the detected violence. This overview also dives into the initial image processing and machine learning-based VD literature and their possible advantages such as efficiency against the current complex models. Furthermore,the datasets are discussed, to provide an analysis of the current models, explaining their pros and cons with future directions in VD domain derived from an in-depth analysis of the previous methods.
Dynamic camera alignment optimization problem based on Fractal Decomposition based Algorithm
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we tackle the Dynamic Optimization Problem (DOP) of IA in a real-world application using a Dynamic Optimization Algorithm (DOA) called Fractal Decomposition Algorithm (FDA), introduced by recently. We used FDA to perform IA on CCTV camera feed from a tunnel. As the camera viewpoint can change by multiple reasons such as wind, maintenance, etc. the alignment is required to guarantee the correct functioning of video-based traffic security system.
Edge-oriented Implicit Neural Representation with Channel Tuning
Venue: arXiv cs.CV 2022
Year: 2022

Implicit neural representation, which expresses an image as a continuous function rather than a discrete grid form, is widely used for image processing. Despite its outperforming results, there are still remaining limitations on restoring clear shapes of a given signal such as the edges of an image. In this paper, we propose Gradient Magnitude Adjustment algorithm which calculates the gradient of an image for training the implicit representation. In addition, we propose Edge-oriented Representation Network (EoREN) that can reconstruct the image with clear edges by fitting gradient information (Edge-oriented module). Furthermore, we add Channel-tuning module to adjust the distribution of given signals so that it solves a chronic problem of fitting gradients. By separating backpropagation paths of the two modules, EoREN can learn true color of the image without hindering the role for gradients. We qualitatively show that our model can reconstruct complex signals and demonstrate general reconstruction ability of our model with quantitative results.
The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matrices
Venue: arXiv cs.CR 2022
Year: 2022

Deep Neural Networks (DNNs) are susceptible to backdoor attacks during training. The model corrupted in this way functions normally, but when triggered by certain patterns in the input, produces a predefined target label. Existing defenses usually rely on the assumption of the universal backdoor setting in which poisoned samples share the same uniform trigger. However, recent advanced backdoor attacks show that this assumption is no longer valid in dynamic backdoors where the triggers vary from input to input, thereby defeating the existing defenses. In this work, we propose a novel technique, Beatrix (backdoor detection via Gram matrix). Beatrix utilizes Gram matrix to capture not only the feature correlations but also the appropriately high-order information of the representations. By learning class-conditional statistics from activation patterns of normal samples, Beatrix can identify poisoned samples by capturing the anomalies in activation patterns. To further improve the performance in identifying target labels, Beatrix leverages kernel-based testing without making any prior assumptions on representation distribution. We demonstrate the effectiveness of our method through extensive evaluation and comparison with state-of-the-art defensive techniques. The experimental results show that our approach achieves an F1 score of 91.1% in detecting dynamic backdoors, while the state of the art can only reach 36.9%.
Evaluating Agent Interactions Through Episodic Knowledge Graphs
Venue: arXiv cs.AI 2022
Year: 2022

We present a new method based on episodic Knowledge Graphs (eKGs) for evaluating (multimodal) conversational agents in open domains. This graph is generated by interpreting raw signals during conversation and is able to capture the accumulation of knowledge over time. We apply structural and semantic analysis of the resulting graphs and translate the properties into qualitative measures. We compare these measures with existing automatic and manual evaluation metrics commonly used for conversational agents. Our results show that our Knowledge-Graph-based evaluation provides more qualitative insights into interaction and the agent's behavior.
Taking the Intentional Stance Seriously, or "Intending" to Improve Cognitive Systems
Venue: arXiv cs.AI 2022
Year: 2022

Finding claims that researchers have made considerable progress in artificial intelligence over the last several decades is easy. However, our everyday interactions with cognitive systems (e.g., Siri, Alexa, DALL-E) quickly move from intriguing to frustrating. One cause of those frustrations rests in a mismatch between the expectations we have due to our inherent, folk-psychological theories and the real limitations we experience with existing computer programs. The software does not understand that people have goals, beliefs about how to achieve those goals, and intentions to act accordingly. One way to align cognitive systems with our expectations is to imbue them with mental states that mirror those we use to predict and explain human behavior. This paper discusses these concerns and illustrates the challenge of following this route by analyzing the mental state 'intention.' That analysis is joined with high-level methodological suggestions that support progress in this endeavor.
Process Diagrams
Venue: arXiv cs.HC 2022
Year: 2022

This paper is simply a collection of process diagrams for further use and reference. These are diagrams about different approaches to research.
A direct time-of-flight image sensor with in-pixel surface detection and dynamic vision
Venue: arXiv cs.CV 2022
Year: 2022

3D flash LIDAR is an alternative to the traditional scanning LIDAR systems, promising precise depth imaging in a compact form factor, and free of moving parts, for applications such as self-driving cars, robotics and augmented reality (AR). Typically implemented using single-photon, direct time-of-flight (dToF) receivers in image sensor format, the operation of the devices can be hindered by the large number of photon events needing to be processed and compressed in outdoor scenarios, limiting frame rates and scalability to larger arrays. We here present a 64x32 pixel (256x128 SPAD) dToF imager that overcomes these limitations by using pixels with embedded histogramming, which lock onto and track the return signal. This reduces the size of output data frames considerably, enabling maximum frame rates in the 10 kFPS range or 100 kFPS for direct depth readings. The sensor offers selective readout of pixels detecting surfaces, or those sensing motion, leading to reduced power consumption and off-chip processing requirements. We demonstrate the application of the sensor in mid-range LIDAR.