OpenRaiser/Intern-Atlas huggingface dataset:
Conflict-free Collaborative Set Sharing for Distributed Systems
Venue: arXiv cs.DC 2021
Year: 2021

Collaborative Data Sharing is widely noticed to be essential for distributed systems. Among several proposed strategies, conflict-free techniques are considered useful for serverless concurrent systems. They aim at making shared data be consistent between peers in such a way that their local data do not become equal at once, but they arrive at the same data eventually when no updates occur in any peer. Although the Conflict-free Replicated Data Type (CRDT) approach could be used in data sharing as well, it puts restrictions on available operations so as to concurrent updates never cause conflicts. Even for sets, popular operations such as insertion and deletion are not freely used, for example. We propose a novel scheme for Conflict-free Collaborative Set Sharing that allows both insertion and deletion operations. It will provide a new synchronization method for data sharing and gives a fresh insight into designing conflict-free replicated data types. We might consider that this becomes a substitute for CRDTs.
Operation-based Collaborative Data Sharing for Distributed Systems
Venue: arXiv cs.DB 2021
Year: 2021

Collaborative Data Sharing raises a fundamental issue in distributed systems. Several strategies have been proposed for making shared data consistent between peers in such a way that the shared part of their local data become equal. Most of the proposals rely on state-based semantics. But this suffers from a lack of descriptiveness in conflict-free features of synchronization required for flexible network connections. Recent applications tend to use non-permanent connection with mobile devices or allow temporary breakaways from the system, for example. To settle ourselves in conflict-free data sharing, we propose a novel scheme "Operation-based Collaborative Data Sharing" that enables conflict-free strategies for synchronization based on operational semantics.
Graph Convolutional Module for Temporal Action Localization in Videos
Venue: arXiv cs.CV 2021
Year: 2021

Temporal action localization has long been researched in computer vision. Existing state-of-the-art action localization methods divide each video into multiple action units (i.e., proposals in two-stage methods and segments in one-stage methods) and then perform action recognition/regression on each of them individually, without explicitly exploiting their relations during learning. In this paper, we claim that the relations between action units play an important role in action localization, and a more powerful action detector should not only capture the local content of each action unit but also allow a wider field of view on the context related to it. To this end, we propose a general graph convolutional module (GCM) that can be easily plugged into existing action localization methods, including two-stage and one-stage paradigms. To be specific, we first construct a graph, where each action unit is represented as a node and their relations between two action units as an edge. Here, we use two types of relations, one for capturing the temporal connections between different action units, and the other one for characterizing their semantic relationship. Particularly for the temporal connections in two-stage methods, we further explore two different kinds of edges, one connecting the overlapping action units and the other one connecting surrounding but disjointed units. Upon the graph we built, we then apply graph convolutional networks (GCNs) to model the relations among different action units, which is able to learn more informative representations to enhance action localization. Experimental results show that our GCM consistently improves the performance of existing action localization methods, including two-stage methods (e.g., CBR and R-C3D) and one-stage methods (e.g., D-SSAD), verifying the generality and effectiveness of our GCM.
Unleashing the Potential of Unsupervised Pre-Training with Intra-Identity Regularization for Person Re-Identification
Venue: arXiv cs.CV 2021
Year: 2021

Existing person re-identification (ReID) methods typically directly load the pre-trained ImageNet weights for initialization. However, as a fine-grained classification task, ReID is more challenging and exists a large domain gap between ImageNet classification. Inspired by the great success of self-supervised representation learning with contrastive objectives, in this paper, we design an Unsupervised Pre-training framework for ReID based on the contrastive learning (CL) pipeline, dubbed UP-ReID. During the pre-training, we attempt to address two critical issues for learning fine-grained ReID features: (1) the augmentations in CL pipeline may distort the discriminative clues in person images. (2) the fine-grained local features of person images are not fully-explored. Therefore, we introduce an intra-identity (I$^2$-)regularization in the UP-ReID, which is instantiated as two constraints coming from global image aspect and local patch aspect: a global consistency is enforced between augmented and original person images to increase robustness to augmentation, while an intrinsic contrastive constraint among local patches of each image is employed to fully explore the local discriminative clues. Extensive experiments on multiple popular Re-ID datasets, including PersonX, Market1501, CUHK03, and MSMT17, demonstrate that our UP-ReID pre-trained model can significantly benefit the downstream ReID fine-tuning and achieve state-of-the-art performance. Codes and models will be released to https://github.com/Frost-Yang-99/UP-ReID.
Mutltimodal AI Companion for Interactive Fairytale Co-creation
Venue: arXiv cs.MM 2021
Year: 2021

AI fairy tale companions play an important role in early childhood education as an augmentation for parents' efforts to close the participation gap and boost kids' mental and language development. Existing systems are generally designed to provide vivid materials as unidirectional entertaining reading environments, e.g, visualizing inputting texts. However, due to the limited vocabulary of kids, these systems failed to afford effective interaction to motivate kids to write their own fairy tales. In this work, we propose AI.R Taletorium, an illustrative, immersive, and inclusive multimodal AI companion, for interactive fairy tale co-creation that actively involves kids to create fairy tales with both the AI agent and their normal peers. AI.R Taletorium consists a neural story generator and a doodler-based fairy tale visualizer. We design a character-centric bidirectional connection mechanism between the story generator and visualizer equipped with Contrastive Language Image Pretraining (CLIP), thus enabling kids to participant in the story generation process by simple sketching. Extensive experiments and user studies show that our system was able to generate and visualize meaningful and vivid fairy tales with limited training data and complete the full interaction cycle under various inputs (text, doodler) through the bidirectional connection.
Multi-View Stereo with Transformer
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes a network, referred to as MVSTR, for Multi-View Stereo (MVS). It is built upon Transformer and is capable of extracting dense features with global context and 3D consistency, which are crucial to achieving reliable matching for MVS. Specifically, to tackle the problem of the limited receptive field of existing CNN-based MVS methods, a global-context Transformer module is first proposed to explore intra-view global context. In addition, to further enable dense features to be 3D-consistent, a 3D-geometry Transformer module is built with a well-designed cross-view attention mechanism to facilitate inter-view information interaction. Experimental results show that the proposed MVSTR achieves the best overall performance on the DTU dataset and strong generalization on the Tanks & Temples benchmark dataset.
Rethink, Revisit, Revise: A Spiral Reinforced Self-Revised Network for Zero-Shot Learning
Venue: arXiv cs.CV 2021
Year: 2021

Current approaches to Zero-Shot Learning (ZSL) struggle to learn generalizable semantic knowledge capable of capturing complex correlations. Inspired by \emph{Spiral Curriculum}, which enhances learning processes by revisiting knowledge, we propose a form of spiral learning which revisits visual representations based on a sequence of attribute groups (e.g., a combined group of \emph{color} and \emph{shape}). Spiral learning aims to learn generalized local correlations, enabling models to gradually enhance global learning and thus understand complex correlations. Our implementation is based on a 2-stage \emph{Reinforced Self-Revised (RSR)} framework: \emph{preview} and \emph{review}. RSR first previews visual information to construct diverse attribute groups in a weakly-supervised manner. Then, it spirally learns refined localities based on attribute groups and uses localities to revise global semantic correlations. Our framework outperforms state-of-the-art algorithms on four benchmark datasets in both zero-shot and generalized zero-shot settings, which demonstrates the effectiveness of spiral learning in learning generalizable and complex correlations. We also conduct extensive analysis to show that attribute groups and reinforced decision processes can capture complementary semantic information to improve predictions and aid explainability.
Security issues of CFS-like digital signature algorithms
Venue: arXiv cs.CR 2021
Year: 2021

We analyse the security of some variants of the CFS code-based digital signature scheme. We show how the adoption of some code-based hash-functions to improve the efficiency of CFS leads to the ability of an attacker to produce a forgery compatible to the rightful user's public key.
Seeking Sinhala Sentiment: Predicting Facebook Reactions of Sinhala Posts
Venue: arXiv cs.LG 2021
Year: 2021

The Facebook network allows its users to record their reactions to text via a typology of emotions. This network, taken at scale, is therefore a prime data set of annotated sentiment data. This paper uses millions of such reactions, derived from a decade worth of Facebook post data centred around a Sri Lankan context, to model an eye of the beholder approach to sentiment detection for online Sinhala textual content. Three different sentiment analysis models are built, taking into account a limited subset of reactions, all reactions, and another that derives a positive/negative star rating value. The efficacy of these models in capturing the reactions of the observers are then computed and discussed. The analysis reveals that binary classification of reactions, for Sinhala content, is significantly more accurate than the other approaches. Furthermore, the inclusion of the like reaction hinders the capability of accurately predicting other reactions.
FaceTuneGAN: Face Autoencoder for Convolutional Expression Transfer Using Neural Generative Adversarial Networks
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we present FaceTuneGAN, a new 3D face model representation decomposing and encoding separately facial identity and facial expression. We propose a first adaptation of image-to-image translation networks, that have successfully been used in the 2D domain, to 3D face geometry. Leveraging recently released large face scan databases, a neural network has been trained to decouple factors of variations with a better knowledge of the face, enabling facial expressions transfer and neutralization of expressive faces. Specifically, we design an adversarial architecture adapting the base architecture of FUNIT and using SpiralNet++ for our convolutional and sampling operations. Using two publicly available datasets (FaceScape and CoMA), FaceTuneGAN has a better identity decomposition and face neutralization than state-of-the-art techniques. It also outperforms classical deformation transfer approach by predicting blendshapes closer to ground-truth data and with less of undesired artifacts due to too different facial morphologies between source and target.
Quoting is not Citing: Disentangling Affiliation and Interaction on Twitter
Venue: arXiv cs.SI 2021
Year: 2021

Interaction networks are generally much less homophilic than affiliation networks, accommodating for many more cross-cutting links. By statistically assigning a political valence to users from their network-level affiliation patterns, and by further contrasting interaction and affiliation (quotes and retweets) within specific discursive events, namely quote trees, we describe a variety of cross-cutting patterns which significantly nuance the traditional "echo chamber" narrative.
Semi-Supervised Surface Anomaly Detection of Composite Wind Turbine Blades From Drone Imagery
Venue: arXiv cs.CV 2021
Year: 2021

Within commercial wind energy generation, the monitoring and predictive maintenance of wind turbine blades in-situ is a crucial task, for which remote monitoring via aerial survey from an Unmanned Aerial Vehicle (UAV) is commonplace. Turbine blades are susceptible to both operational and weather-based damage over time, reducing the energy efficiency output of turbines. In this study, we address automating the otherwise time-consuming task of both blade detection and extraction, together with fault detection within UAV-captured turbine blade inspection imagery. We propose BladeNet, an application-based, robust dual architecture to perform both unsupervised turbine blade detection and extraction, followed by super-pixel generation using the Simple Linear Iterative Clustering (SLIC) method to produce regional clusters. These clusters are then processed by a suite of semi-supervised detection methods. Our dual architecture detects surface faults of glass fibre composite material blades with high aptitude while requiring minimal prior manual image annotation. BladeNet produces an Average Precision (AP) of 0.995 across our Ørsted blade inspection dataset for offshore wind turbines and 0.223 across the Danish Technical University (DTU) NordTank turbine blade inspection dataset. BladeNet also obtains an AUC of 0.639 for surface anomaly detection across the Ørsted blade inspection dataset.
3D Reconstruction Using a Linear Laser Scanner and a Camera
Venue: arXiv cs.CV 2021
Year: 2021

With the rapid development of computer graphics and vision, several three-dimensional (3D) reconstruction techniques have been proposed and used to obtain the 3D representation of objects in the form of point cloud models, mesh models, and geometric models. The cost of 3D reconstruction is declining due to the maturing of this technology, however, the inexpensive 3D reconstruction scanners on the market may not be able to generate a clear point cloud model as expected. This study systematically reviews some basic types of 3D reconstruction technology and introduces an easy implementation using a linear laser scanner, a camera, and a turntable. The implementation is based on the monovision with laser and has tested several objects like wiki and mug. The accuracy and resolution of the point cloud result are quite satisfying. It turns out everyone can build such a 3D reconstruction system with appropriate procedures.
A Two-Level Approximate Logic Synthesis Combining Cube Insertion and Removal
Venue: arXiv cs.OH 2021
Year: 2021

Approximate computing is an attractive paradigm for reducing the design complexity of error-resilient systems, therefore improving performance and saving power consumption. In this work, we propose a new two-level approximate logic synthesis method based on cube insertion and removal procedures. Experimental results have shown significant literal count and runtime reduction compared to the state-of-the-art approach. The method scalability is illustrated for a high error threshold over large benchmark circuits. The obtained solutions have presented a literal number reduction up to 38%, 56% and 93% with respect to an error rate of 1%, 3% and 5%, respectively.
The Effect of People Recommenders on Echo Chambers and Polarization
Venue: arXiv cs.SI 2021
Year: 2021

The effects of social media on critical issues, such as polarization and misinformation, are under scrutiny due to the disruptive consequences that these phenomena can have on our societies. Among the algorithms routinely used by social media platforms, people-recommender systems are of special interest, as they directly contribute to the evolution of the social network structure, affecting the information and the opinions users are exposed to. In this paper, we propose a framework to assess the effect of people recommenders on the evolution of opinions. Our proposal is based on Monte Carlo simulations combining link recommendation and opinion-dynamics models. In order to control initial conditions, we define a random network model to generate graphs with opinions, with tunable amounts of modularity and homophily. We join these elements into a methodology to study the effects of the recommender system on echo chambers and polarization. We also show how to use our framework to measure, by means of simulations, the impact of different intervention strategies. Our thorough experimentation shows that people recommenders can in fact lead to a significant increase in echo chambers. However, this happens only if there is considerable initial homophily in the network. Also, we find that if the network already contains echo chambers, the effect of the recommendation algorithm is negligible. Such findings are robust to two very different opinion dynamics models, a bounded confidence model and an epistemological model.
DeepSportLab: a Unified Framework for Ball Detection, Player Instance Segmentation and Pose Estimation in Team Sports Scenes
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents a unified framework to (i) locate the ball, (ii) predict the pose, and (iii) segment the instance mask of players in team sports scenes. Those problems are of high interest in automated sports analytics, production, and broadcast. A common practice is to individually solve each problem by exploiting universal state-of-the-art models, \eg, Panoptic-DeepLab for player segmentation. In addition to the increased complexity resulting from the multiplication of single-task models, the use of the off-the-shelf models also impedes the performance due to the complexity and specificity of the team sports scenes, such as strong occlusion and motion blur. To circumvent those limitations, our paper proposes to train a single model that simultaneously predicts the ball and the player mask and pose by combining the part intensity fields and the spatial embeddings principles. Part intensity fields provide the ball and player location, as well as player joints location. Spatial embeddings are then exploited to associate player instance pixels to their respective player center, but also to group player joints into skeletons. We demonstrate the effectiveness of the proposed model on the DeepSport basketball dataset, achieving comparable performance to the SoA models addressing each individual task separately.
Graph Conditioned Sparse-Attention for Improved Source Code Understanding
Venue: arXiv cs.LG 2021
Year: 2021

Transformer architectures have been successfully used in learning source code representations. The fusion between a graph representation like Abstract Syntax Tree (AST) and a source code sequence makes the use of current approaches computationally intractable for large input sequence lengths. Source code can have long-range dependencies that require larger sequence lengths to model effectively. Current approaches have a quadratic growth in computational and memory costs with respect to the sequence length. Using such models in practical scenarios is difficult. In this work, we propose the conditioning of a source code snippet with its graph modality by using the graph adjacency matrix as an attention mask for a sparse self-attention mechanism and the use of a graph diffusion mechanism to model longer-range token dependencies. Our model reaches state-of-the-art results in BLEU, METEOR, and ROUGE-L metrics for the code summarization task and near state-of-the-art accuracy in the variable misuse task. The memory use and inference time of our model have linear growth with respect to the input sequence length as compared to the quadratic growth of previous works.
MDFM: Multi-Decision Fusing Model for Few-Shot Learning
Venue: arXiv cs.CV 2021
Year: 2021

In recent years, researchers pay growing attention to the few-shot learning (FSL) task to address the data-scarce problem. A standard FSL framework is composed of two components: i) Pre-train. Employ the base data to generate a CNN-based feature extraction model (FEM). ii) Meta-test. Apply the trained FEM to the novel data (category is different from base data) to acquire the feature embeddings and recognize them. Although researchers have made remarkable breakthroughs in FSL, there still exists a fundamental problem. Since the trained FEM with base data usually cannot adapt to the novel class flawlessly, the novel data's feature may lead to the distribution shift problem. To address this challenge, we hypothesize that even if most of the decisions based on different FEMs are viewed as weak decisions, which are not available for all classes, they still perform decently in some specific categories. Inspired by this assumption, we propose a novel method Multi-Decision Fusing Model (MDFM), which comprehensively considers the decisions based on multiple FEMs to enhance the efficacy and robustness of the model. MDFM is a simple, flexible, non-parametric method that can directly apply to the existing FEMs. Besides, we extend the proposed MDFM to two FSL settings (i.e., supervised and semi-supervised settings). We evaluate the proposed method on five benchmark datasets and achieve significant improvements of 3.4%-7.3% compared with state-of-the-arts.
Label-Free Model Evaluation with Semi-Structured Dataset Representations
Venue: arXiv cs.CV 2021
Year: 2021

Label-free model evaluation, or AutoEval, estimates model accuracy on unlabeled test sets, and is critical for understanding model behaviors in various unseen environments. In the absence of image labels, based on dataset representations, we estimate model performance for AutoEval with regression. On the one hand, image feature is a straightforward choice for such representations, but it hampers regression learning due to being unstructured (\ie no specific meanings for component at certain location) and of large-scale. On the other hand, previous methods adopt simple structured representations (like average confidence or average feature), but insufficient to capture the data characteristics given their limited dimensions. In this work, we take the best of both worlds and propose a new semi-structured dataset representation that is manageable for regression learning while containing rich information for AutoEval. Based on image features, we integrate distribution shapes, clusters, and representative samples for a semi-structured dataset representation. Besides the structured overall description with distribution shapes, the unstructured description with clusters and representative samples include additional fine-grained information facilitating the AutoEval task. On three existing datasets and 25 newly introduced ones, we experimentally show that the proposed representation achieves competitive results. Code and dataset are available at https://github.com/sxzrt/Semi-Structured-Dataset-Representations.