OpenRaiser/Intern-Atlas huggingface dataset:
Forcing the Whole Video as Background: An Adversarial Learning Strategy for Weakly Temporal Action Localization
Venue: arXiv cs.CV 2022
Year: 2022

With video-level labels, weakly supervised temporal action localization (WTAL) applies a localization-by-classification paradigm to detect and classify the action in untrimmed videos. Due to the characteristic of classification, class-specific background snippets are inevitably mis-activated to improve the discriminability of the classifier in WTAL. To alleviate the disturbance of background, existing methods try to enlarge the discrepancy between action and background through modeling background snippets with pseudo-snippet-level annotations, which largely rely on artificial hypotheticals. Distinct from the previous works, we present an adversarial learning strategy to break the limitation of mining pseudo background snippets. Concretely, the background classification loss forces the whole video to be regarded as the background by a background gradient reinforcement strategy, confusing the recognition model. Reversely, the foreground(action) loss guides the model to focus on action snippets under such conditions. As a result, competition between the two classification losses drives the model to boost its ability for action modeling. Simultaneously, a novel temporal enhancement network is designed to facilitate the model to construct temporal relation of affinity snippets based on the proposed strategy, for further improving the performance of action localization. Finally, extensive experiments conducted on THUMOS14 and ActivityNet1.2 demonstrate the effectiveness of the proposed method.
Unified almost linear kernels for generalized covering and packing problems on nowhere dense classes
Venue: arXiv cs.DS 2022
Year: 2022

Let $\mathcal{F}$ be a family of graphs, and let $p,r$ be nonnegative integers. The \textsc{$(p,r,\mathcal{F})$-Covering} problem asks whether for a graph $G$ and an integer $k$, there exists a set $D$ of at most $k$ vertices in $G$ such that $G^p\setminus N_G^r[D]$ has no induced subgraph isomorphic to a graph in $\mathcal{F}$, where $G^p$ is the $p$-th power of $G$. The \textsc{$(p,r,\mathcal{F})$-Packing} problem asks whether for a graph $G$ and an integer $k$, $G^p$ has $k$ induced subgraphs $H_1,\ldots,H_k$ such that each $H_i$ is isomorphic to a graph in $\mathcal{F}$, and for distinct $i,j\in \{1, \ldots, k\}$, the distance between $V(H_i)$ and $V(H_j)$ in $G$ is larger than $r$. We show that for every fixed nonnegative integers $p,r$ and every fixed nonempty finite family $\mathcal{F}$ of connected graphs, the \textsc{$(p,r,\mathcal{F})$-Covering} problem with $p\leq2r+1$ and the \textsc{$(p,r,\mathcal{F})$-Packing} problem with $p\leq2\lfloor r/2\rfloor+1$ admit almost linear kernels on every nowhere dense class of graphs, and admit linear kernels on every class of graphs with bounded expansion, parameterized by the solution size $k$. We obtain the same kernels for their annotated variants. As corollaries, we prove that \textsc{Distance-$r$ Vertex Cover}, \textsc{Distance-$r$ Matching}, \textsc{$\mathcal{F}$-Free Vertex Deletion}, and \textsc{Induced-$\mathcal{F}$-Packing} for any fixed finite family $\mathcal{F}$ of connected graphs admit almost linear kernels on every nowhere dense class of graphs and linear kernels on every class of graphs with bounded expansion. Our results extend the results for \textsc{Distance-$r$ Dominating Set} by Drange et al. (STACS 2016) and Eickmeyer et al. (ICALP 2017), and the result for \textsc{Distance-$r$ Independent Set} by Pilipczuk and Siebertz (EJC 2021).
Deep Point-to-Plane Registration by Efficient Backpropagation for Error Minimizing Function
Venue: arXiv cs.CV 2022
Year: 2022

Traditional algorithms of point set registration minimizing point-to-plane distances often achieve a better estimation of rigid transformation than those minimizing point-to-point distances. Nevertheless, recent deep-learning-based methods minimize the point-to-point distances. In contrast to these methods, this paper proposes the first deep-learning-based approach to point-to-plane registration. A challenging part of this problem is that a typical solution for point-to-plane registration requires an iterative process of accumulating small transformations obtained by minimizing a linearized energy function. The iteration significantly increases the size of the computation graph needed for backpropagation and can slow down both forward and backward network evaluations. To solve this problem, we consider the estimated rigid transformation as a function of input point clouds and derive its analytic gradients using the implicit function theorem. The analytic gradient that we introduce is independent of how the error minimizing function (i.e., the rigid transformation) is obtained, thus allowing us to calculate both the rigid transformation and its gradient efficiently. We implement the proposed point-to-plane registration module over several previous methods that minimize point-to-point distances and demonstrate that the extensions outperform the base methods even with point clouds with noise and low-quality point normals estimated with local point distributions.
Iterative Soft-Input Soft-Output Decoding with Ordered Reliability Bits GRAND
Venue: arXiv cs.IT 2022
Year: 2022

Guessing Random Additive Noise Decoding (GRAND) is a universal decoding algorithm that can be used to perform maximum likelihood decoding. It attempts to find the errors introduced by the channel by generating a sequence of possible error vectors in order of likelihood of occurrence and applying them to the received vector. Ordered reliability bits GRAND (ORBGRAND) integrates soft information received from the channel to refine the error vector sequence. In this work, ORBGRAND is modified to produce a soft output, to enable its use as an iterative soft-input soft-output (SISO) decoder. Three techniques specific to iterative GRAND-based decoding are then proposed to improve the error-correction performance and decrease computational complexity and latency. Using the OFEC code as a case study, the proposed techniques are evaluated, yielding substantial performance gain and astounding complexity reduction of 48\% to 85\% with respect to the baseline SISO ORBGRAND.
Visualizing Gender Gap in Film Industry over the Past 100 Years
Venue: arXiv cs.HC 2022
Year: 2022

Visualizing big data can provide valuable insights into social science research. In this project, we focused on visualizing the potential gender gap in the global film industry over the past 100 years. We profiled the differences both for the actors/actresses and male/female movie audiences and analyzed the IMDb data of the most popular 10,000 movies (the composition and importance of casts of different genders, the cooperation network of the actors/actresses, the movie genres, the movie descriptions, etc.) and audience ratings (the differences between male's and female's ratings). Findings suggest that the gender gap has been distinct in many aspects, but a recent trend is that this gap narrows down and women are gaining discursive power in the film industry. Our study presented rich data, vivid illustrations, and novel perspectives that can serve as the foundation for further studies on related topics and their social implications.
Hardware-in-the-Loop Simulation for Evaluating Communication Impacts on the Wireless-Network-Controlled Robots
Venue: arXiv cs.RO 2022
Year: 2022

More and more robot automation applications have changed to wireless communication, and network performance has a growing impact on robotic systems. This study proposes a hardware-in-the-loop (HiL) simulation methodology for connecting the simulated robot platform to real network devices. This project seeks to provide robotic engineers and researchers with the capability to experiment without heavily modifying the original controller and get more realistic test results that correlate with actual network conditions. We deployed this HiL simulation system in two common cases for wireless-network-controlled robotic applications: (1) safe multi-robot coordination for mobile robots, and (2) human-motion-based teleoperation for manipulators. The HiL simulation system is deployed and tested under various network conditions in all circumstances. The experiment results are analyzed and compared with the previous simulation methods, demonstrating that the proposed HiL simulation methodology can identify a more reliable communication impact on robot systems.
Open Terminology Management and Sharing Toolkit for Federation of Terminology Databases
Venue: arXiv cs.CL 2022
Year: 2022

Consolidated access to current and reliable terms from different subject fields and languages is necessary for content creators and translators. Terminology is also needed in AI applications such as machine translation, speech recognition, information extraction, and other natural language processing tools. In this work, we facilitate standards-based sharing and management of terminology resources by providing an open terminology management solution - the EuroTermBank Toolkit. It allows organisations to manage and search their terms, create term collections, and share them within and outside the organisation by participating in the network of federated databases. The data curated in the federated databases are automatically shared with EuroTermBank, the largest multilingual terminology resource in Europe, allowing translators and language service providers as well as researchers and students to access terminology resources in their most current version.
Virtual reality (VR) as a testing bench for consumer optical solutions: A machine learning approach (GBR) to visual comfort under simulated progressive addition lenses (PALS) distortions
Venue: arXiv cs.HC 2022
Year: 2022

For decades, manufacturers have attempted to reduce or eliminate the optical aberrations that appear on the progressive addition lens' surfaces during manufacturing. Besides every effort made, some of these distortions are inevitable given how lenses are fabricated, where in fact, astigmatism appears on the surface and cannot be entirely removed or where non-uniform magnification becomes inherent to the power change across the lens. Some presbyopes may refer to certain discomfort when wearing these lenses for the first time, and a subset of them might never adapt. Developing, prototyping, testing and purveying those lenses into the market come at a cost, which is usually reflected in the retail price. This study aims to test the feasibility of virtual reality for testing customers' satisfaction with these lenses, even before getting them onto production. VR offers a controlled environment where different parameters affecting progressive lens comforts, such as distortions, image displacement or optical blurring, can be analysed separately. In this study, the focus was set on the distortions and image displacement, not taking blur into account. Behavioural changes (head and eye movements) were recorded using the built-in eye tracker. Participants were significantly more displeased in the presence of highly distorted lens simulations. In addition, a gradient boosting regressor was fitted to the data, so predictors of discomfort could be unveiled, and ratings could be predicted without performing additional measurements.
FFTc: An MLIR Dialect for Developing HPC Fast Fourier Transform Libraries
Venue: arXiv cs.MS 2022
Year: 2022

Discrete Fourier Transform (DFT) libraries are one of the most critical software components for scientific computing. Inspired by FFTW, a widely used library for DFT HPC calculations, we apply compiler technologies for the development of HPC Fourier transform libraries. In this work, we introduce FFTc, a domain-specific language, based on Multi-Level Intermediate Representation (MLIR), for expressing Fourier Transform algorithms. We present the initial design, implementation, and preliminary results of FFTc.
Challenges of SLAM in extremely unstructured environments: the DLR Planetary Stereo, Solid-State LiDAR, Inertial Dataset
Venue: arXiv cs.RO 2022
Year: 2022

We present the DLR Planetary Stereo, Solid-State LiDAR, Inertial (S3LI) dataset, recorded on Mt. Etna, Sicily, an environment analogous to the Moon and Mars, using a hand-held sensor suite with attributes suitable for implementation on a space-like mobile rover. The environment is characterized by challenging conditions regarding both the visual and structural appearance: severe visual aliasing poses significant limitations to the ability of visual SLAM systems to perform place recognition, while the absence of outstanding structural details, joined with the limited Field-of-View of the utilized Solid-State LiDAR sensor, challenges traditional LiDAR SLAM for the task of pose estimation using point clouds alone. With this data, that covers more than 4 kilometers of travel on soft volcanic slopes, we aim to: 1) provide a tool to expose limitations of state-of-the-art SLAM systems with respect to environments, which are not present in widely available datasets and 2) motivate the development of novel localization and mapping approaches, that rely efficiently on the complementary capabilities of the two sensors. The dataset is accessible at the following url: https://rmc.dlr.de/s3li_dataset
iColoriT: Towards Propagating Local Hint to the Right Region in Interactive Colorization by Leveraging Vision Transformer
Venue: arXiv cs.CV 2022
Year: 2022

Point-interactive image colorization aims to colorize grayscale images when a user provides the colors for specific locations. It is essential for point-interactive colorization methods to appropriately propagate user-provided colors (i.e., user hints) in the entire image to obtain a reasonably colorized image with minimal user effort. However, existing approaches often produce partially colorized results due to the inefficient design of stacking convolutional layers to propagate hints to distant relevant regions. To address this problem, we present iColoriT, a novel point-interactive colorization Vision Transformer capable of propagating user hints to relevant regions, leveraging the global receptive field of Transformers. The self-attention mechanism of Transformers enables iColoriT to selectively colorize relevant regions with only a few local hints. Our approach colorizes images in real-time by utilizing pixel shuffling, an efficient upsampling technique that replaces the decoder architecture. Also, in order to mitigate the artifacts caused by pixel shuffling with large upsampling ratios, we present the local stabilizing layer. Extensive quantitative and qualitative results demonstrate that our approach highly outperforms existing methods for point-interactive colorization, producing accurately colorized images with a user's minimal effort. Official codes are available at https://pmh9960.github.io/research/iColoriT
Revisiting Interest Indicators Derived from Web Reading Behavior for Implicit User Modeling
Venue: arXiv cs.HC 2022
Year: 2022

Today, intelligent user interfaces on the web often come in form of recommendation services tailoring content to individual users. Recommendation of web content such as news articles often requires a certain amount of explicit ratings to allow for satisfactory results, i.e., the selection of content actually relevant for the user. Yet, the collection of such explicit ratings is time-consuming and dependent on users' willingness to provide the required information on a regular basis. Thus, using implicit interest indicators can be a helpful complementation to relying on explicitly entered information only. Analysis of reading behavior on the web can be the basis for the derivation of such implicit indicators. Previous work has already identified several indicators and discussed how they can be used as a basis for user models. However, most earlier work is either of conceptual nature and does not involve studies to prove the suggested concepts or relies on meanwhile potentially outdated technology. All earlier discussions of the topic further have in common that they do not yet consider mobile contexts. This paper builds upon earlier work, however providing a major update regarding technology and web reading context, distinguishing between desktop and mobile settings. This update also allowed us to identify a set of new indicators that so far have not yet been discussed. This paper describes (i) our technical work, a framework for analyzing user interactions with the browser relying on latest web technologies, (ii) the implicit interest indicators we either revisited or newly identified, and (iii) the results of an online study on web reading behavior as a basis for derivation of interest we conducted with 96 participants.
Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation
Venue: arXiv cs.CV 2022
Year: 2022

Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications. Although photo extrapolation has been well studied, it is much more challenging to extrapolate a photo (i.e., selfie) from a narrow field of view to a wider one while maintaining a similar visual style. In this paper, we propose a factorized neural re-rendering model to produce photorealistic novel views from cluttered outdoor Internet photo collections, which enables the applications including controllable scene re-rendering, photo extrapolation and even extrapolated 3D photo generation. Specifically, we first develop a novel factorized re-rendering pipeline to handle the ambiguity in the decomposition of geometry, appearance and illumination. We also propose a composited training strategy to tackle the unexpected occlusion in Internet images. Moreover, to enhance photo-realism when extrapolating tourist photographs, we propose a novel realism augmentation process to complement appearance details, which automatically propagates the texture details from a narrow captured photo to the extrapolated neural rendered image. The experiments and photo editing examples on outdoor scenes demonstrate the superior performance of our proposed method in both photo-realism and downstream applications.
Real-time Streaming Video Denoising with Bidirectional Buffers
Venue: arXiv cs.CV 2022
Year: 2022

Video streams are delivered continuously to save the cost of storage and device memory. Real-time denoising algorithms are typically adopted on the user device to remove the noise involved during the shooting and transmission of video streams. However, sliding-window-based methods feed multiple input frames for a single output and lack computation efficiency. Recent multi-output inference works propagate the bidirectional temporal feature with a parallel or recurrent framework, which either suffers from performance drops on the temporal edges of clips or can not achieve online inference. In this paper, we propose a Bidirectional Streaming Video Denoising (BSVD) framework, to achieve high-fidelity real-time denoising for streaming videos with both past and future temporal receptive fields. The bidirectional temporal fusion for online inference is considered not applicable in the MoViNet. However, we introduce a novel Bidirectional Buffer Block as the core module of our BSVD, which makes it possible during our pipeline-style inference. In addition, our method is concise and flexible to be utilized in both non-blind and blind video denoising. We compare our model with various state-of-the-art video denoising models qualitatively and quantitatively on synthetic and real noise. Our method outperforms previous methods in terms of restoration fidelity and runtime. Our source code is publicly available at https://github.com/ChenyangQiQi/BSVD
Applications of Blockchain for the Governance of Integrated Project Delivery: A Crypto Commons Approach
Venue: arXiv cs.CR 2022
Year: 2022

This paper outlines why and how blockchain can digitally support and evolve the governance of collaborative project deliveries, such as integrated project deliveries (IPDs), to provide the foundation for novel and disruptive forms of organizational collaboration in the construction industry. Previous work has conceptualized IPDs as a common pool resource (CPR) scenario, where shared resources are collectively governed. Through the use of blockchain and smart contracts for trustworthy peer-to-peer transactions and execution logic, Ostrom's design principles can be digitally encoded to scale CPR scenarios. Building on the identified connections, the paper 1) synthesizes fourteen blockchain-based mechanisms to govern CPRs, 2) identifies twenty-two applications of these mechanisms to govern IPDs, and 3) introduces a conceptualization of the above relationships towards a holistic understanding of collaborative project deliveries on the crypto commons for novel collective organization of construction project delivery between both humans and machines.
Characterizing Nodes and Edges in Dynamic Attributed Networks: A Social-based Approach
Venue: arXiv cs.SI 2022
Year: 2022

How to characterize nodes and edges in dynamic attributed networks based on social aspects? We address this problem by exploring the strength of the ties between actors and their associated attributes over time, thus capturing the social roles of the actors and the meaning of their dynamic interactions in different social network scenarios. For this, we apply social concepts to promote a better understanding of the underlying complexity that involves actors and their social motivations. More specifically, we explore the notion of social capital given by the strategic positioning of a particular actor in a social structure by means of the concepts of brokerage, the ability of creating bridges with diversified patterns, and closure, the ability of aggregating nodes with similar patterns. As a result, we unveil the differences of social interactions in distinct academic coauthorship networks and questions \& answers communities. We also statistically validate our social definitions considering the importance of the nodes and edges in a social structure by means of network properties.
Convolutional Bypasses Are Better Vision Transformer Adapters
Venue: arXiv cs.CV 2022
Year: 2022

The pretrain-then-finetune paradigm has been widely adopted in computer vision. But as the size of Vision Transformer (ViT) grows exponentially, the full finetuning becomes prohibitive in view of the heavier storage overhead. Motivated by parameter-efficient transfer learning (PETL) on language transformers, recent studies attempt to insert lightweight adaptation modules (e.g., adapter layers or prompt tokens) to pretrained ViT and only finetune these modules while the pretrained weights are frozen. However, these modules were originally proposed to finetune language models and did not take into account the prior knowledge specifically for visual tasks. In this paper, we propose to construct Convolutional Bypasses (Convpass) in ViT as adaptation modules, introducing only a small amount (less than 0.5% of model parameters) of trainable parameters to adapt the large ViT. Different from other PETL methods, Convpass benefits from the hard-coded inductive bias of convolutional layers and thus is more suitable for visual tasks, especially in the low-data regime. Experimental results on VTAB-1K benchmark and few-shot learning datasets show that Convpass outperforms current language-oriented adaptation modules, demonstrating the necessity to tailor vision-oriented adaptation modules for adapting vision models.