OpenRaiser/Intern-Atlas huggingface dataset:
Reconfigurable Intelligent Surfaces Empowered Cooperative Rate Splitting with User Relaying
Venue: arXiv cs.IT 2022
Year: 2022

Cooperative rate splitting (CRS), built upon rate splitting multiple access (RSMA) and opportunistic user relaying, has been recognized as a promising transmission strategy to enhance the user fairness and spectral efficiency in multiantenna broadcast channels. To further boost its performance, the interplay of CRS and reconfigurable intelligent surface (RIS) is investigated in this work. Specifically, a novel RIS-aided CRS transmission framework is proposed and the corresponding resource allocation problem to maximize the minimum rate among users is investigated. An alternative optimization algorithm is then proposed to optimize the transmit beamforming, common rate allocation, and RIS phases, iteratively. Numerical results show that the proposed RIS-aided CRS transmission framework significantly improves the spectral efficiency compared with its non-cooperative counterpart and other schemes without RIS.
Detecting User Exits from Online Behavior: A Duration-Dependent Latent State Model
Venue: arXiv cs.LG 2022
Year: 2022

In order to steer e-commerce users towards making a purchase, marketers rely upon predictions of when users exit without purchasing. Previously, such predictions were based upon hidden Markov models (HMMs) due to their ability of modeling latent shopping phases with different user intents. In this work, we develop a duration-dependent hidden Markov model. In contrast to traditional HMMs, it explicitly models the duration of latent states and thereby allows states to become "sticky". The proposed model is superior to prior HMMs in detecting user exits: out of 100 user exits without purchase, it correctly identifies an additional 18. This helps marketers in better managing the online behavior of e-commerce customers. The reason for the superior performance of our model is the duration dependence, which allows our model to recover latent states that are characterized by a distorted sense of time. We finally provide a theoretical explanation for this, which builds upon the concept of "flow".
Generating Coherent Narratives by Learning Dynamic and Discrete Entity States with a Contrastive Framework
Venue: arXiv cs.CL 2022
Year: 2022

Despite advances in generating fluent texts, existing pretraining models tend to attach incoherent event sequences to involved entities when generating narratives such as stories and news. We conjecture that such issues result from representing entities as static embeddings of superficial words, while neglecting to model their ever-changing states, i.e., the information they carry, as the text unfolds. Therefore, we extend the Transformer model to dynamically conduct entity state updates and sentence realization for narrative generation. We propose a contrastive framework to learn the state representations in a discrete space, and insert additional attention layers into the decoder to better exploit these states. Experiments on two narrative datasets show that our model can generate more coherent and diverse narratives than strong baselines with the guidance of meaningful entity states.
Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model
Venue: arXiv cs.CV 2022
Year: 2022

Large-scale vision foundation models have made significant progress in visual tasks on natural images, with vision transformers being the primary choice due to their good scalability and representation ability. However, large-scale models in remote sensing (RS) have not yet been sufficiently explored. In this paper, we resort to plain vision transformers with about 100 million parameters and make the first attempt to propose large vision models tailored to RS tasks and investigate how such large models perform. To handle the large sizes and objects of arbitrary orientations in RS images, we propose a new rotated varied-size window attention to replace the original full attention in transformers, which can significantly reduce the computational cost and memory footprint while learning better object representation by extracting rich context from the generated diverse windows. Experiments on detection tasks show the superiority of our model over all state-of-the-art models, achieving 81.24% mAP on the DOTA-V1.0 dataset. The results of our models on downstream classification and segmentation tasks also show competitive performance compared to existing advanced methods. Further experiments show the advantages of our models in terms of computational complexity and data efficiency in transferring.
On the Benefits And Problems Related to Using Definition of Done -- A Survey Study
Venue: arXiv cs.SE 2022
Year: 2022

Context: Definition of Done (DoD) is one of the fundamental concepts of Scrum. It expresses a shared view of a Scrum Team on what makes an increment of their product complete. DoDs are often defined as checklists with items being requirements towards software (e.g., quality requirements) or towards activities performed to make the increment shippable (e.g., code reviews, testing). Unfortunately, the knowledge about the usefulness of DoD is still very limited. Objective: The goal is to study what benefits using the DoD practice can bring to an agile project, what problems it may trigger, and how it is created and maintained. Method: In the survey among members of agile software development projects, 137 practitioners from all over the globe shared their experience with us. Results: 93% of the respondents perceive DoD as at least valuable for their ventures. It helps them to make work items complete, assure product quality, and ensure the needed activities are executed. However, they indicated that every second project struggles with infeasible, incorrect, unavailable, or creeping DoD. Conclusions: It follows from the study that DoD is important but not easy to use and more empirical studies are needed to identify best practices in this area.
Two-Stream Networks for Object Segmentation in Videos
Venue: arXiv cs.CV 2022
Year: 2022

Existing matching-based approaches perform video object segmentation (VOS) via retrieving support features from a pixel-level memory, while some pixels may suffer from lack of correspondence in the memory (i.e., unseen), which inevitably limits their segmentation performance. In this paper, we present a Two-Stream Network (TSN). Our TSN includes (i) a pixel stream with a conventional pixel-level memory, to segment the seen pixels based on their pixellevel memory retrieval. (ii) an instance stream for the unseen pixels, where a holistic understanding of the instance is obtained with dynamic segmentation heads conditioned on the features of the target instance. (iii) a pixel division module generating a routing map, with which output embeddings of the two streams are fused together. The compact instance stream effectively improves the segmentation accuracy of the unseen pixels, while fusing two streams with the adaptive routing map leads to an overall performance boost. Through extensive experiments, we demonstrate the effectiveness of our proposed TSN, and we also report state-of-the-art performance of 86.1% on YouTube-VOS 2018 and 87.5% on the DAVIS-2017 validation split.
Relevance Judgment Convergence Degree -- A Measure of Inconsistency among Assessors for Information Retrieval
Venue: arXiv cs.IR 2022
Year: 2022

Relevance judgment of human assessors is inherently subjective and dynamic when evaluation datasets are created for Information Retrieval (IR) systems. However, a small group of experts' relevance judgment results are usually taken as ground truth to "objectively" evaluate the performance of the IR systems. Recent trends intend to employ a group of judges, such as outsourcing, to alleviate the potentially biased judgment results stemmed from using only a single expert's judgment. Nevertheless, different judges may have different opinions and may not agree with each other, and the inconsistency in human relevance judgment may affect the IR system evaluation results. In this research, we introduce a Relevance Judgment Convergence Degree (RJCD) to measure the quality of queries in the evaluation datasets. Experimental results reveal a strong correlation coefficient between the proposed RJCD score and the performance differences between the two IR systems.
Efficient Description of some Classes of Codes using Group Algebras
Venue: arXiv cs.IT 2022
Year: 2022

Circulant matrices are an important tool widely used in coding theory and cryptography. A circulant matrix is a square matrix whose rows are the cyclic shifts of the first row. Such a matrix can be efficiently stored in memory because it is fully specified by its first row. The ring of $n \times n$ circulant matrices can be identified with the quotient ring $\mathbb{F}[x]/(x^n-1)$. In consequence, the strong algebraic structure of the ring $\mathbb{F}[x]/(x^n-1)$ can be used to study properties of the collection of all $n\times n$ circulant matrices. The ring $\mathbb{F}[x]/(x^n-1)$ is a special case of a group algebra and elements of any finite dimensional group algebra can be represented with square matrices which are specified by a single column. In this paper we study this representation and prove that it is an injective Hamming weight preserving homomorphism of $\mathbb{F}$-algebras and classify it in the case where the underlying group is abelian. Our work is motivated by the desire to generalize the BIKE cryptosystem (a contender in the NIST competition to get a new post-quantum standard for asymmetric cryptography). Group algebras can be used to design similar cryptosystems or, more generally, to construct low density or moderate density parity-check matrices for linear codes.
Bug-Fix Variants: Visualizing Unique Source Code Changes across GitHub Forks
Venue: arXiv cs.SE 2022
Year: 2022

Forking is a common practice for developers when building upon on already existing projects. These forks create variants, which have a common code base but then evolve the code in different directions, which is specific to that forked project requirements. An interesting side-effect of having multiple forks is the ability to select between different evolution directions of the code which is based on developers fixing bugs in the code base. However, the key issue that this decentralized form of information is difficult to analyze. In this study, we propose a visualization to analyze active changes in fork repositories that have not been merged back to the original project. Our visualization shows code commit activities in multiple forks with highlight on bug fix commits in the history of forks. While the commit activity of each repository is visualized similarly to the code frequency view of GitHub, our view shows only commits unique to fork repositories. To illustrate the effectiveness of our visualization, we have applied our view to two use cases: identifying forks from a repository no longer maintained, and identifying a bug fix among forks. In the first case, we identify a fork of a suspended project named Obfuscator-LLVM. Our view shows the original repository and its most active fork that continue the development on the top. In the second case, we identify a bug fix in a fork of Clipy project. Our view shows that the most active fork has its own bug fixes; we could easily identify a patch for the bug highlighted in the view. As a new ideas paper, we then present our outline of three research questions to spark real world use-cases and goals for our visualization has the potential to uncover. A prototype of our visualization is available at \textcolor{blue}{\url{https://naist-se.github.io/vissoft2022/}
Where Are You Looking?: A Large-Scale Dataset of Head and Gaze Behavior for 360-Degree Videos and a Pilot Study
Venue: arXiv cs.MM 2022
Year: 2022

360° videos in recent years have experienced booming development. Compared to traditional videos, 360° videos are featured with uncertain user behaviors, bringing opportunities as well as challenges. Datasets are necessary for researchers and developers to explore new ideas and conduct reproducible analyses for fair comparisons among different solutions. However, existing related datasets mostly focused on users' field of view (FoV), ignoring the more important eye gaze information, not to mention the integrated extraction and analysis of both FoV and eye gaze. Besides, users' behavior patterns are highly related to videos, yet most existing datasets only contained videos with subjective and qualitative classification from video genres, which lack quantitative analysis and fail to characterize the intrinsic properties of a video scene. To this end, we first propose a quantitative taxonomy for 360° videos that contains three objective technical metrics. Based on this taxonomy, we collect a dataset containing users' head and gaze behaviors simultaneously, which outperforms existing datasets with rich dimensions, large scale, strong diversity, and high frequency. Then we conduct a pilot study on user's behaviors and get some interesting findings such as user's head direction will follow his/her gaze direction with the most possible time interval. A case of application in tile-based 360° video streaming based on our dataset is later conducted, demonstrating a great performance improvement of existing works by leveraging our provided gaze information. Our dataset is available at https://cuhksz-inml.github.io/head_gaze_dataset/
Style Transfer of Black and White Silhouette Images using CycleGAN and a Randomly Generated Dataset
Venue: arXiv cs.LG 2022
Year: 2022

CycleGAN can be used to transfer an artistic style to an image. It does not require pairs of source and stylized images to train a model. Taking this advantage, we propose using randomly generated data to train a machine learning model that can transfer traditional art style to a black and white silhouette image. The result is noticeably better than the previous neural style transfer methods. However, there are some areas for improvement, such as removing artifacts and spikes from the transformed image.
An Urban Population Health Observatory for Disease Causal Pathway Analysis and Decision Support: Underlying Explainable Artificial Intelligence Model
Venue: arXiv cs.AI 2022
Year: 2022

This study sought to (1) expand our existing Urban Population Health Observatory (UPHO) system by incorporating a semantics layer; (2) cohesively employ machine learning and semantic/logical inference to provide measurable evidence and detect pathways leading to undesirable health outcomes; (3) provide clinical use case scenarios and design case studies to identify socioenvironmental determinants of health associated with the prevalence of obesity, and (4) design a dashboard that demonstrates the use of UPHO in the context of obesity surveillance using the provided scenarios. The system design includes a knowledge graph generation component that provides contextual knowledge from relevant domains of interest. This system leverages semantics using concepts, properties, and axioms from existing ontologies. In addition, we used the publicly available US Centers for Disease Control and Prevention 500 Cities data set to perform multivariate analysis. A cohesive approach that employs machine learning and semantic/logical inference reveals pathways leading to diseases. In this study, we present 2 clinical case scenarios and a proof-of-concept prototype design of a dashboard that provides warnings, recommendations, and explanations and demonstrates the use of UPHO in the context of obesity surveillance, treatment, and prevention. While exploring the case scenarios using a support vector regression machine learning model, we found that poverty, lack of physical activity, education, and unemployment were the most important predictive variables that contribute to obesity in Memphis, TN. The application of UPHO could help reduce health disparities and improve urban population health. The expanded UPHO feature incorporates an additional level of interpretable knowledge to enhance physicians, researchers, and health officials' informed decision-making at both patient and community levels.
Efficient Neural Net Approaches in Metal Casting Defect Detection
Venue: arXiv cs.CV 2022
Year: 2022

One of the most pressing challenges prevalent in the steel manufacturing industry is the identification of surface defects. Early identification of casting defects can help boost performance, including streamlining production processes. Though, deep learning models have helped bridge this gap and automate most of these processes, there is a dire need to come up with lightweight models that can be deployed easily with faster inference times. This research proposes a lightweight architecture that is efficient in terms of accuracy and inference time compared with sophisticated pre-trained CNN architectures like MobileNet, Inception, and ResNet, including vision transformers. Methodologies to minimize computational requirements such as depth-wise separable convolution and global average pooling (GAP) layer, including techniques that improve architectural efficiencies and augmentations, have been experimented. Our results indicate that a custom model of 590K parameters with depth-wise separable convolutions outperformed pretrained architectures such as Resnet and Vision transformers in terms of accuracy (81.87%) and comfortably outdid architectures such as Resnet, Inception, and Vision transformers in terms of faster inference times (12 ms). Blurpool fared outperformed other techniques, with an accuracy of 83.98%. Augmentations had a paradoxical effect on the model performance. No direct correlation between depth-wise and 3x3 convolutions on inference time, they, however, they played a direct role in improving model efficiency by enabling the networks to go deeper and by decreasing the number of trainable parameters. Our work sheds light on the fact that custom networks with efficient architectures and faster inference times can be built without the need of relying on pre-trained architectures.
Neural Message Passing for Visual Relationship Detection
Venue: arXiv cs.CV 2022
Year: 2022

Visual relationship detection aims to detect the interactions between objects in an image; however, this task suffers from combinatorial explosion due to the variety of objects and interactions. Since the interactions associated with the same object are dependent, we explore the dependency of interactions to reduce the search space. We explicitly model objects and interactions by an interaction graph and then propose a message-passing-style algorithm to propagate the contextual information. We thus call the proposed method neural message passing (NMP). We further integrate language priors and spatial cues to rule out unrealistic interactions and capture spatial interactions. Experimental results on two benchmark datasets demonstrate the superiority of our proposed method. Our code is available at https://github.com/PhyllisH/NMP.
Object Detection Using Sim2Real Domain Randomization for Robotic Applications
Venue: arXiv cs.RO 2022
Year: 2022

Robots working in unstructured environments must be capable of sensing and interpreting their surroundings. One of the main obstacles of deep-learning-based models in the field of robotics is the lack of domain-specific labeled data for different industrial applications. In this article, we propose a sim2real transfer learning method based on domain randomization for object detection with which labeled synthetic datasets of arbitrary size and object types can be automatically generated. Subsequently, a state-of-the-art convolutional neural network, YOLOv4, is trained to detect the different types of industrial objects. With the proposed domain randomization method, we could shrink the reality gap to a satisfactory level, achieving 86.32% and 97.38% mAP50 scores, respectively, in the case of zero-shot and one-shot transfers, on our manually annotated dataset containing 190 real images. Our solution fits for industrial use as the data generation process takes less than 0.5 s per image and the training lasts only around 12 h, on a GeForce RTX 2080 Ti GPU. Furthermore, it can reliably differentiate similar classes of objects by having access to only one real image for training. To our best knowledge, this is the only work thus far satisfying these constraints.
Content-Based Landmark Retrieval Combining Global and Local Features using Siamese Neural Networks
Venue: arXiv cs.IR 2022
Year: 2022

In this work, we present a method for landmark retrieval that utilizes global and local features. A Siamese network is used for global feature extraction and metric learning, which gives an initial ranking of the landmark search. We utilize the extracted feature maps from the Siamese architecture as local descriptors, the search results are then further refined using a cosine similarity between local descriptors. We conduct a deeper analysis of the Google Landmark Dataset, which is used for evaluation, and augment the dataset to handle various intra-class variances. Furthermore, we conduct several experiments to compare the effects of transfer learning and metric learning, as well as experiments using other local descriptors. We show that a re-ranking using local features can improve the search results. We believe that the proposed local feature extraction using cosine similarity is a simple approach that can be extended to many other retrieval tasks.
A Design Space for Surfacing Content Recommendations in Visual Analytic Platforms
Venue: arXiv cs.HC 2022
Year: 2022

Recommendation algorithms have been leveraged in various ways within visualization systems to assist users as they perform of a range of information tasks. One common focus for these techniques has been the recommendation of content, rather than visual form, as a means to assist users in the identification of information that is relevant to their task context. A wide variety of techniques have been proposed to address this general problem, with a range of design choices in how these solutions surface relevant information to users. This paper reviews the state-of-the-art in how visualization systems surface recommended content to users during users' visual analysis; introduces a four-dimensional design space for visual content recommendation based on a characterization of prior work; and discusses key observations regarding common patterns and future research opportunities.