OpenRaiser/Intern-Atlas huggingface dataset:
Diffusion-HPC: Synthetic Data Generation for Human Mesh Recovery in Challenging Domains
Venue: arXiv cs.CV 2023
Year: 2023

Recent text-to-image generative models have exhibited remarkable abilities in generating high-fidelity and photo-realistic images. However, despite the visually impressive results, these models often struggle to preserve plausible human structure in the generations. Due to this reason, while generative models have shown promising results in aiding downstream image recognition tasks by generating large volumes of synthetic data, they are not suitable for improving downstream human pose perception and understanding. In this work, we propose a Diffusion model with Human Pose Correction (Diffusion-HPC), a text-conditioned method that generates photo-realistic images with plausible posed humans by injecting prior knowledge about human body structure. Our generated images are accompanied by 3D meshes that serve as ground truths for improving Human Mesh Recovery tasks, where a shortage of 3D training data has long been an issue. Furthermore, we show that Diffusion-HPC effectively improves the realism of human generations under varying conditioning strategies.
PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision
Venue: arXiv cs.CV 2023
Year: 2023

Impressive progress in generative models and implicit representations gave rise to methods that can generate 3D shapes of high quality. However, being able to locally control and edit shapes is another essential property that can unlock several content creation applications. Local control can be achieved with part-aware models, but existing methods require 3D supervision and cannot produce textures. In this work, we devise PartNeRF, a novel part-aware generative model for editable 3D shape synthesis that does not require any explicit 3D supervision. Our model generates objects as a set of locally defined NeRFs, augmented with an affine transformation. This enables several editing operations such as applying transformations on parts, mixing parts from different objects etc. To ensure distinct, manipulable parts we enforce a hard assignment of rays to parts that makes sure that the color of each ray is only determined by a single NeRF. As a result, altering one part does not affect the appearance of the others. Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.
User Scheduling and Passive Beamforming for FDMA/OFDMA in Intelligent Reflection Surface
Venue: arXiv cs.IT 2023
Year: 2023

Most prior works on intelligent reflecting surface (IRS) merely consider point-to-point communications, including a single user, for ease of analysis. Nevertheless, a practical wireless system needs to accommodate multiple users simultaneously. Due to the lack of frequency-selective reflection, namely the set of phase shifts cannot be different across frequency subchannels, the integration of IRS imposes a fundamental challenge to frequency-multiplexing approaches such as frequency-division multiple access (FDMA) and the widely adopted technique called orthogonal FDMA (OFDMA). It motivates us to study (O)FDMA-based multi-user IRS communications to clarify which user scheduling and passive beamforming are favorable under this non-frequency-selective reflection environment. Theoretical analysis and numerical evaluation reveal that (O)FDMA does not need user scheduling when there are a few users. If the number of users becomes large, neither user scheduling nor IRS reflection optimization is necessary. These findings help substantially simplify the design of (O)FDMA-based IRS communications.
A Novel Scholar Embedding Model for Interdisciplinary Collaboration
Venue: arXiv cs.DL 2023
Year: 2023

Interdisciplinary collaboration has become a driving force for scientific breakthroughs, and evaluating scholars' performance in interdisciplinary researches is essential for promoting such collaborations. However, traditional scholar evaluation methods based solely on individual achievements do not consider interdisciplinary cooperation, creating a challenge for interdisciplinary scholar evaluation and recommendation. To address this issue, we propose a scholar embedding model that quantifies and represents scholars based on global semantic information and social influence, enabling real-time tracking of scholars' research trends. Our model incorporates semantic information and social influence for interdisciplinary scholar evaluation, laying the foundation for future interdisciplinary collaboration discovery and recommendation projects. We demonstrate the effectiveness of our model on a sample of scholars from the Beijing University of Posts and Telecommunications.
Vision Based Docking of Multiple Satellites with an Uncooperative Target
Venue: arXiv cs.RO 2023
Year: 2023

With the ever growing number of space debris in orbit, the need to prevent further space population is becoming more and more apparent. Refueling, servicing, inspection and deorbiting of spacecraft are some example missions that require precise navigation and docking in space. Having multiple, collaborating robots handling these tasks can greatly increase the efficiency of the mission in terms of time and cost. This article will introduce a modern and efficient control architecture for satellites on collaborative docking missions. The proposed architecture uses a centralized scheme that combines state-of-the-art, ad-hoc implementations of algorithms and techniques to maximize robustness and flexibility. It is based on a Model Predictive Controller (MPC) for which efficient cost function and constraint sets are designed to ensure a safe and accurate docking. A simulation environment is also presented to validate and test the proposed control scheme.
Wasmizer: Curating WebAssembly-driven Projects on GitHub
Venue: arXiv cs.SE 2023
Year: 2023

WebAssembly has attracted great attention as a portable compilation target for programming languages. To facilitate in-depth studies about this technology, we have deployed Wasmizer, a tool that regularly mines GitHub projects and makes an up-to-date dataset of WebAssembly sources and their binaries publicly available. Presently, we have collected 2 540 C and C++ projects that are highly-related to WebAssembly, and built a dataset of 8 915 binaries that are linked to their source projects. To demonstrate an application of this dataset, we have investigated the presence of eight WebAssembly compilation smells in the wild.
Online Reinforcement Learning in Periodic MDP
Venue: arXiv cs.LG 2023
Year: 2023

We study learning in periodic Markov Decision Process (MDP), a special type of non-stationary MDP where both the state transition probabilities and reward functions vary periodically, under the average reward maximization setting. We formulate the problem as a stationary MDP by augmenting the state space with the period index, and propose a periodic upper confidence bound reinforcement learning-2 (PUCRL2) algorithm. We show that the regret of PUCRL2 varies linearly with the period $N$ and as $\mathcal{O}(\sqrt{Tlog T})$ with the horizon length $T$. Utilizing the information about the sparsity of transition matrix of augmented MDP, we propose another algorithm PUCRLB which enhances upon PUCRL2, both in terms of regret ($O(\sqrt{N})$ dependency on period) and empirical performance. Finally, we propose two other algorithms U-PUCRL2 and U-PUCRLB for extended uncertainty in the environment in which the period is unknown but a set of candidate periods are known. Numerical results demonstrate the efficacy of all the algorithms.
Tribe or Not? Critical Inspection of Group Differences Using TribalGram
Venue: arXiv cs.LG 2023
Year: 2023

With the rise of AI and data mining techniques, group profiling and group-level analysis have been increasingly used in many domains including policy making and direct marketing. In some cases, the statistics extracted from data may provide insights to a group's shared characteristics; in others, the group-level analysis can lead to problems including stereotyping and systematic oppression. How can analytic tools facilitate a more conscientious process in group analysis? In this work, we identify a set of accountable group analytics design guidelines to explicate the needs for group differentiation and preventing overgeneralization of a group. Following the design guidelines, we develop TribalGram, a visual analytic suite that leverages interpretable machine learning algorithms and visualization to offer inference assessment, model explanation, data corroboration, and sense-making. Through the interviews with domain experts, we showcase how our design and tools can bring a richer understanding of "groups" mined from the data.
Delayed and Indirect Impacts of Link Recommendations
Venue: arXiv cs.SI 2023
Year: 2023

The impacts of link recommendations on social networks are challenging to evaluate, and so far they have been studied in limited settings. Observational studies are restricted in the kinds of causal questions they can answer and naive A/B tests often lead to biased evaluations due to unaccounted network interference. Furthermore, evaluations in simulation settings are often limited to static network models that do not take into account the potential feedback loops between link recommendation and organic network evolution. To this end, we study the impacts of recommendations on social networks in dynamic settings. Adopting a simulation-based approach, we consider an explicit dynamic formation model -- an extension of the celebrated Jackson-Rogers model -- and investigate how link recommendations affect network evolution over time. Empirically, we find that link recommendations have surprising delayed and indirect effects on the structural properties of networks. Specifically, we find that link recommendations can exhibit considerably different impacts in the immediate term and in the long term. For instance, we observe that friend-of-friend recommendations can have an immediate effect in decreasing degree inequality, but in the long term, they can make the degree distribution substantially more unequal. Moreover, we show that the effects of recommendations can persist in networks, in part due to their indirect impacts on natural dynamics even after recommendations are turned off. We show that, in counterfactual simulations, removing the indirect effects of link recommendations can make the network trend faster toward what it would have been under natural growth dynamics.
Understanding why shooters shoot -- An AI-powered engine for basketball performance profiling
Venue: arXiv cs.LG 2023
Year: 2023

Understanding player shooting profiles is an essential part of basketball analysis: knowing where certain opposing players like to shoot from can help coaches neutralize offensive gameplans from their opponents; understanding where their players are most comfortable can lead them to developing more effective offensive strategies. An automatic tool that can provide these performance profiles in a timely manner can become invaluable for coaches to maximize both the effectiveness of their game plan as well as the time dedicated to practice and other related activities. Additionally, basketball is dictated by many variables, such as playstyle and game dynamics, that can change the flow of the game and, by extension, player performance profiles. It is crucial that the performance profiles can reflect the diverse playstyles, as well as the fast-changing dynamics of the game. We present a tool that can visualize player performance profiles in a timely manner while taking into account factors such as play-style and game dynamics. Our approach generates interpretable heatmaps that allow us to identify and analyze how non-spatial factors, such as game dynamics or playstyle, affect player performance profiles.
Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services
Venue: arXiv cs.HC 2023
Year: 2023

Recent years have seen growing adoption of AI-based decision-support systems (ADS) in homeless services, yet we know little about stakeholder desires and concerns surrounding their use. In this work, we aim to understand impacted stakeholders' perspectives on a deployed ADS that prioritizes scarce housing resources. We employed AI lifecycle comicboarding, an adapted version of the comicboarding method, to elicit stakeholder feedback and design ideas across various components of an AI system's design. We elicited feedback from county workers who operate the ADS daily, service providers whose work is directly impacted by the ADS, and unhoused individuals in the region. Our participants shared concerns and design suggestions around the AI system's overall objective, specific model design choices, dataset selection, and use in deployment. Our findings demonstrate that stakeholders, even without AI knowledge, can provide specific and critical feedback on an AI system's design and deployment, if empowered to do so.
Goldfish: Peer selection using Matrix completion in unstructured P2P network
Venue: arXiv cs.NI 2023
Year: 2023

Peer-to-peer (P2P) networks underlie a variety of decentralized paradigms including blockchains, distributed file storage and decentralized domain name systems. A central primitive in P2P networks is the peer selection algorithm, which decides how a node should select a fixed number of neighbors to connect with. In this paper, we consider the design of a peer-selection algorithm for unstructured P2P networks with the goal of minimizing the broadcast latency. We propose Goldfish, a novel solution that dynamically decides the neighbor set by exploiting the past experiences as well as exploring new neighbors. The key technical contributions come from bringing ideas of matrix completion for estimating message delivery times for every possible message for every peer ever connected, and a streaming algorithm to efficiently perform the estimation while achieving good performance. The matrix completion interpolates the delivery times to all virtual connections in order to select the best combination of neighbors. Goldfish employs a streaming algorithm that only uses a short recent memory to finish matrix interpolation. When the number of publishing source is equal to a node's maximal number of connections, Goldfish found the global optimal solution with 92.7% probability by exploring every node only once. In more complex situations where nodes are publishing based on exponential distribution and adjusting connection in real time, we compare Goldfish with a baseline peer selection system, and show Goldfish saves approximately 14.5% less time under real world geolocation and propagation latency.
S/C: Speeding up Data Materialization with Bounded Memory
Venue: arXiv cs.DB 2023
Year: 2023

With data pipeline tools and the expressiveness of SQL, managing interdependent materialized views (MVs) are becoming increasingly easy. These MVs are updated repeatedly upon new data ingestion (e.g., daily), from which database admins can observe performance metrics (e.g., refresh time of each MV, size on disk) in a consistent way for different types of updates (full vs. incremental) and for different systems (single node, distributed, cloud-hosted). One missed opportunity is that existing data systems treat those MV updates as independent SQL statements without fully exploiting their dependency information and performance metrics. However, if we know that the result of a SQL statement will be consumed immediately after for subsequent operations, those subsequent operations do not have to wait until the early results are fully materialized on storage because the results are already readily available in memory. Of course, this may come at a cost because keeping results in memory (even temporarily) will reduce the amount of available memory; thus, our decision should be careful. In this paper, we introduce a new system, called S/C, which tackles this problem through efficient creation and update of a set of MVs with acyclic dependencies among them. S/C judiciously uses bounded memory to reduce end-to-end MV refresh time by short-circuiting expensive reads and writes; S/C's objective function accurately estimates time savings from keeping intermediate data in memory for particular periods. Our solution jointly optimizes an MV refresh order, what data to keep in memory, and when to release data from memory. At a high level, S/C still materializes all data exactly as defined in MV definitions; thus, it doesn't impact any service-level agreements. In our experiments with TPC-DS datasets (up to 1TB), we show S/C's optimization can speedup end-to-end runtime by 1.04x-5.08x with 1.6GB memory.
GOOD: General Optimization-based Fusion for 3D Object Detection via LiDAR-Camera Object Candidates
Venue: arXiv cs.CV 2023
Year: 2023

3D object detection serves as the core basis of the perception tasks in autonomous driving. Recent years have seen the rapid progress of multi-modal fusion strategies for more robust and accurate 3D object detection. However, current researches for robust fusion are all learning-based frameworks, which demand a large amount of training data and are inconvenient to implement in new scenes. In this paper, we propose GOOD, a general optimization-based fusion framework that can achieve satisfying detection without training additional models and is available for any combinations of 2D and 3D detectors to improve the accuracy and robustness of 3D detection. First we apply the mutual-sided nearest-neighbor probability model to achieve the 3D-2D data association. Then we design an optimization pipeline that can optimize different kinds of instances separately based on the matching result. Apart from this, the 3D MOT method is also introduced to enhance the performance aided by previous frames. To the best of our knowledge, this is the first optimization-based late fusion framework for multi-modal 3D object detection which can be served as a baseline for subsequent research. Experiments on both nuScenes and KITTI datasets are carried out and the results show that GOOD outperforms by 9.1\% on mAP score compared with PointPillars and achieves competitive results with the learning-based late fusion CLOCs.
DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Deep neural networks lack interpretability and tend to be overconfident, which poses a serious problem in safety-critical applications like autonomous driving, medical imaging, or machine vision tasks with high demands on reliability. Quantifying the predictive uncertainty is a promising endeavour to open up the use of deep neural networks for such applications. Unfortunately, current available methods are computationally expensive. In this work, we present a novel approach for efficient and reliable uncertainty estimation which we call Deep Uncertainty Distillation using Ensembles for Segmentation (DUDES). DUDES applies student-teacher distillation with a Deep Ensemble to accurately approximate predictive uncertainties with a single forward pass while maintaining simplicity and adaptability. Experimentally, DUDES accurately captures predictive uncertainties without sacrificing performance on the segmentation task and indicates impressive capabilities of identifying wrongly classified pixels and out-of-domain samples on the Cityscapes dataset. With DUDES, we manage to simultaneously simplify and outperform previous work on Deep Ensemble-based Uncertainty Distillation.
Trained on 100 million words and still in shape: BERT meets British National Corpus
Venue: arXiv cs.CL 2023
Year: 2023

While modern masked language models (LMs) are trained on ever larger corpora, we here explore the effects of down-scaling training to a modestly-sized but representative, well-balanced, and publicly available English text source -- the British National Corpus. We show that pre-training on this carefully curated corpus can reach better performance than the original BERT model. We argue that this type of corpora has great potential as a language modeling benchmark. To showcase this potential, we present fair, reproducible and data-efficient comparative studies of LMs, in which we evaluate several training objectives and model architectures and replicate previous empirical results in a systematic way. We propose an optimized LM architecture called LTG-BERT.
Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memes
Venue: arXiv cs.CL 2023
Year: 2023

Memes are the new-age conveyance mechanism for humor on social media sites. Memes often include an image and some text. Memes can be used to promote disinformation or hatred, thus it is crucial to investigate in details. We introduce Memotion 3, a new dataset with 10,000 annotated memes. Unlike other prevalent datasets in the domain, including prior iterations of Memotion, Memotion 3 introduces Hindi-English Codemixed memes while prior works in the area were limited to only the English memes. We describe the Memotion task, the data collection and the dataset creation methodologies. We also provide a baseline for the task. The baseline code and dataset will be made available at https://github.com/Shreyashm16/Memotion-3.0
A Benchmark of PDF Information Extraction Tools using a Multi-Task and Multi-Domain Evaluation Framework for Academic Documents
Venue: arXiv cs.IR 2023
Year: 2023

Extracting information from academic PDF documents is crucial for numerous indexing, retrieval, and analysis use cases. Choosing the best tool to extract specific content elements is difficult because many, technically diverse tools are available, but recent performance benchmarks are rare. Moreover, such benchmarks typically cover only a few content elements like header metadata or bibliographic references and use smaller datasets from specific academic disciplines. We provide a large and diverse evaluation framework that supports more extraction tasks than most related datasets. Our framework builds upon DocBank, a multi-domain dataset of 1.5M annotated content elements extracted from 500K pages of research papers on arXiv. Using the new framework, we benchmark ten freely available tools in extracting document metadata, bibliographic references, tables, and other content elements from academic PDF documents. GROBID achieves the best metadata and reference extraction results, followed by CERMINE and Science Parse. For table extraction, Adobe Extract outperforms other tools, even though the performance is much lower than for other content elements. All tools struggle to extract lists, footers, and equations. We conclude that more research on improving and combining tools is necessary to achieve satisfactory extraction quality for most content elements. Evaluation datasets and frameworks like the one we present support this line of research. We make our data and code publicly available to contribute toward this goal.