OpenRaiser/Intern-Atlas huggingface dataset:
DyTed: Disentangled Representation Learning for Discrete-time Dynamic Graph
Venue: arXiv cs.SI 2023
Year: 2023

Unsupervised representation learning for dynamic graphs has attracted a lot of research attention in recent years. Compared with static graph, the dynamic graph is a comprehensive embodiment of both the intrinsic stable characteristics of nodes and the time-related dynamic preference. However, existing methods generally mix these two types of information into a single representation space, which may lead to poor explanation, less robustness, and a limited ability when applied to different downstream tasks. To solve the above problems, in this paper, we propose a novel disenTangled representation learning framework for discrete-time Dynamic graphs, namely DyTed. We specially design a temporal-clips contrastive learning task together with a structure contrastive learning to effectively identify the time-invariant and time-varying representations respectively. To further enhance the disentanglement of these two types of representation, we propose a disentanglement-aware discriminator under an adversarial learning framework from the perspective of information theory. Extensive experiments on Tencent and five commonly used public datasets demonstrate that DyTed, as a general framework that can be applied to existing methods, achieves state-of-the-art performance on various downstream tasks, as well as be more robust against noise.
An Automated Market Maker Minimizing Loss-Versus-Rebalancing
Venue: arXiv cs.GT 2023
Year: 2023

The always-available liquidity of automated market makers (AMMs) has been one of the most important catalysts in early cryptocurrency adoption. However, it has become increasingly evident that AMMs in their current form are not viable investment options for passive liquidity providers. This is large part due to the cost incurred by AMMs providing stale prices to arbitrageurs against external market prices, formalized as loss-versus-rebalancing (LVR) [Milionis et al., 2022]. In this paper, we present Diamond, an automated market making protocol that aligns the incentives of liquidity providers and block producers in the protocol-level retention of LVR. In Diamond, block producers effectively auction the right to capture any arbitrage that exists between the external market price of a Diamond pool, and the price of the pool itself. The proceeds of these auctions are shared by the Diamond pool and block producer in a way that is proven to remain incentive compatible for the block producer. Given the participation of competing arbitrageurs to capture LVR, LVR is minimized in Diamond. We formally prove this result, and detail an implementation of Diamond. We also provide comparative simulations of Diamond to relevant benchmarks, further evidencing the LVR-protection capabilities of Diamond. With this new protection, passive liquidity provision on blockchains can become rationally viable, beckoning a new age for decentralized finance.
Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems
Venue: arXiv cs.IR 2023
Year: 2023

Existing benchmark datasets for recommender systems (RS) either are created at a small scale or involve very limited forms of user feedback. RS models evaluated on such datasets often lack practical values for large-scale real-world applications. In this paper, we describe Tenrec, a novel and publicly available data collection for RS that records various user feedback from four different recommendation scenarios. To be specific, Tenrec has the following five characteristics: (1) it is large-scale, containing around 5 million users and 140 million interactions; (2) it has not only positive user feedback, but also true negative feedback (vs. one-class recommendation); (3) it contains overlapped users and items across four different scenarios; (4) it contains various types of user positive feedback, in forms of clicks, likes, shares, and follows, etc; (5) it contains additional features beyond the user IDs and item IDs. We verify Tenrec on ten diverse recommendation tasks by running several classical baseline models per task. Tenrec has the potential to become a useful benchmark dataset for a majority of popular recommendation tasks.
Weakly Supervised Learning for Analyzing Political Campaigns on Facebook
Venue: arXiv cs.CL 2023
Year: 2023

Social media platforms are currently the main channel for political messaging, allowing politicians to target specific demographics and adapt based on their reactions. However, making this communication transparent is challenging, as the messaging is tightly coupled with its intended audience and often echoed by multiple stakeholders interested in advancing specific policies. Our goal in this paper is to take a first step towards understanding these highly decentralized settings. We propose a weakly supervised approach to identify the stance and issue of political ads on Facebook and analyze how political campaigns use some kind of demographic targeting by location, gender, or age. Furthermore, we analyze the temporal dynamics of the political ads on election polls.
CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion
Venue: arXiv cs.CV 2023
Year: 2023

Masked Image Modeling (MIM) has recently been established as a potent pre-training paradigm. A pretext task is constructed by masking patches in an input image, and this masked content is then predicted by a neural network using visible patches as sole input. This pre-training leads to state-of-the-art performance when finetuned for high-level semantic tasks, e.g. image classification and object detection. In this paper we instead seek to learn representations that transfer well to a wide variety of 3D vision and lower-level geometric downstream tasks, such as depth prediction or optical flow estimation. Inspired by MIM, we propose an unsupervised representation learning task trained from pairs of images showing the same scene from different viewpoints. More precisely, we propose the pretext task of cross-view completion where the first input image is partially masked, and this masked content has to be reconstructed from the visible content and the second image. In single-view MIM, the masked content often cannot be inferred precisely from the visible portion only, so the model learns to act as a prior influenced by high-level semantics. In contrast, this ambiguity can be resolved with cross-view completion from the second unmasked image, on the condition that the model is able to understand the spatial relationship between the two images. Our experiments show that our pretext task leads to significantly improved performance for monocular 3D vision downstream tasks such as depth estimation. In addition, our model can be directly applied to binocular downstream tasks like optical flow or relative camera pose estimation, for which we obtain competitive results without bells and whistles, i.e., using a generic architecture without any task-specific design.
TabLLM: Few-shot Classification of Tabular Data with Large Language Models
Venue: arXiv cs.CL 2023
Year: 2023

We study the application of large language models to zero-shot and few-shot classification of tabular data. We prompt the large language model with a serialization of the tabular data to a natural-language string, together with a short description of the classification problem. In the few-shot setting, we fine-tune the large language model using some labeled examples. We evaluate several serialization methods including templates, table-to-text models, and large language models. Despite its simplicity, we find that this technique outperforms prior deep-learning-based tabular classification methods on several benchmark datasets. In most cases, even zero-shot classification obtains non-trivial performance, illustrating the method's ability to exploit prior knowledge encoded in large language models. Unlike many deep learning methods for tabular datasets, this approach is also competitive with strong traditional baselines like gradient-boosted trees, especially in the very-few-shot setting.
Cluster and Aggregate: Face Recognition with Large Probe Set
Venue: arXiv cs.CV 2023
Year: 2023

Feature fusion plays a crucial role in unconstrained face recognition where inputs (probes) comprise of a set of $N$ low quality images whose individual qualities vary. Advances in attention and recurrent modules have led to feature fusion that can model the relationship among the images in the input set. However, attention mechanisms cannot scale to large $N$ due to their quadratic complexity and recurrent modules suffer from input order sensitivity. We propose a two-stage feature fusion paradigm, Cluster and Aggregate, that can both scale to large $N$ and maintain the ability to perform sequential inference with order invariance. Specifically, Cluster stage is a linear assignment of $N$ inputs to $M$ global cluster centers, and Aggregation stage is a fusion over $M$ clustered features. The clustered features play an integral role when the inputs are sequential as they can serve as a summarization of past features. By leveraging the order-invariance of incremental averaging operation, we design an update rule that achieves batch-order invariance, which guarantees that the contributions of early image in the sequence do not diminish as time steps increase. Experiments on IJB-B and IJB-S benchmark datasets show the superiority of the proposed two-stage paradigm in unconstrained face recognition. Code and pretrained models are available in https://github.com/mk-minchul/caface
A Referable NFT Scheme
Venue: arXiv cs.CR 2023
Year: 2023

Existing NFTs confront restrictions of \textit{one-time incentive} and \textit{product isolation}. Creators cannot obtain benefits once having sold their NFT products due to the lack of relationships across different NFTs, which results in controversial possible profit sharing. This work proposes a referable NFT scheme to extend the incentive sustainability of NFTs. We construct the referable NFT (rNFT) network to increase exposure and enhance the referring relationship of inclusive items. We introduce the DAG topology to generate directed edges between each pair of NFTs with corresponding weights and labels for advanced usage. We accordingly implement and propose the scheme under Ethereum Improvement Proposal (EIP) standards, indexed in EIP-1155. Further, we provide the mathematical formation to analyze the utility for each rNFT participant. The discussion gives general guidance among multi-dimensional parameters. To our knowledge, this is the first study to build the referable NFT network, explicitly showing the virtual connections among NFTs.
Entire Space Counterfactual Learning: Tuning, Analytical Properties and Industrial Applications
Venue: arXiv cs.LG 2023
Year: 2023

As a basic research problem for building effective recommender systems, post-click conversion rate (CVR) estimation has long been plagued by sample selection bias and data sparsity issues. To address the data sparsity issue, prevalent methods based on entire space multi-task model leverage the sequential pattern of user actions, i.e. exposure $\rightarrow$ click $\rightarrow$ conversion to construct auxiliary learning tasks. However, they still fall short of guaranteeing the unbiasedness of CVR estimates. This paper theoretically demonstrates two defects of these entire space multi-task models: (1) inherent estimation bias (IEB) for CVR estimation, where the CVR estimate is inherently higher than the ground truth; (2) potential independence priority (PIP) for CTCVR estimation, where the causality from click to conversion might be overlooked. This paper further proposes a principled method named entire space counterfactual multi-task model (ESCM$^2$), which employs a counterfactual risk minimizer to handle both IEB and PIP issues at once. To demonstrate the effectiveness of the proposed method, this paper explores its parameter tuning in practice, derives its analytic properties, and showcases its effectiveness in industrial CVR estimation, where ESCM$^2$ can effectively alleviate the intrinsic IEB and PIP issues and outperform baseline models.
Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering
Venue: arXiv cs.IR 2023
Year: 2023

Collaborative filtering (CF) models easily suffer from popularity bias, which makes recommendation deviate from users' actual preferences. However, most current debiasing strategies are prone to playing a trade-off game between head and tail performance, thus inevitably degrading the overall recommendation accuracy. To reduce the negative impact of popularity bias on CF models, we incorporate Bias-aware margins into Contrastive loss and propose a simple yet effective BC Loss, where the margin tailors quantitatively to the bias degree of each user-item interaction. We investigate the geometric interpretation of BC loss, then further visualize and theoretically prove that it simultaneously learns better head and tail representations by encouraging the compactness of similar users/items and enlarging the dispersion of dissimilar users/items. Over eight benchmark datasets, we use BC loss to optimize two high-performing CF models. On various evaluation settings (i.e., imbalanced/balanced, temporal split, fully-observed unbiased, tail/head test evaluations), BC loss outperforms the state-of-the-art debiasing and non-debiasing methods with remarkable improvements. Considering the theoretical guarantee and empirical success of BC loss, we advocate using it not just as a debiasing strategy, but also as a standard loss in recommender models.
Voter Coalitions and democracy in Decentralized Finance: Evidence from MakerDAO
Venue: arXiv cs.SI 2023
Year: 2023

Decentralized Autonomous Organization (DAO) provides a decentralized governance solution through blockchain, where decision-making process relies on on-chain voting and follows majority rule. This paper focuses on MakerDAO, and we find three voter coalitions after applying clustering algorithm to voting history. The emergence of a dominant voter coalition is a signal of governance centralization in DAO, and voter coalitions have complicated influence on Maker protocol, which is governed by MakerDAO. This paper presents empirical evidence of multicoalition democracy in DAO and further contributes to the contemporary debate on whether decentralized governance is possible.
Development of a hardware-In-the-Loop (HIL) testbed for cyber-physical security in smart buildings
Venue: arXiv cs.CR 2023
Year: 2023

As smart buildings move towards open communication technologies, providing access to the Building Automation System (BAS) through the intranet, or even remotely through the Internet, has become a common practice. However, BAS was historically developed as a closed environment and designed with limited cyber-security considerations. Thus, smart buildings are vulnerable to cyber-attacks with the increased accessibility. This study introduces the development and capability of a Hardware-in-the-Loop (HIL) testbed for testing and evaluating the cyber-physical security of typical BASs in smart buildings. The testbed consists of three subsystems: (1) a real-time HIL emulator simulating the behavior of a virtual building as well as the Heating, Ventilation, and Air Conditioning (HVAC) equipment via a dynamic simulation in Modelica; (2) a set of real HVAC controllers monitoring the virtual building operation and providing local control signals to control HVAC equipment in the HIL emulator; and (3) a BAS server along with a web-based service for users to fully access the schedule, setpoints, trends, alarms, and other control functions of the HVAC controllers remotely through the BACnet network. The server generates rule-based setpoints to local HVAC controllers. Based on these three subsystems, the HIL testbed supports attack/fault-free and attack/fault-injection experiments at various levels of the building system. The resulting test data can be used to inform the building community and support the cyber-physical security technology transfer to the building industry.
Communication breakdown: On the low mutual intelligibility between human and neural captioning
Venue: arXiv cs.CL 2023
Year: 2023

We compare the 0-shot performance of a neural caption-based image retriever when given as input either human-produced captions or captions generated by a neural captioner. We conduct this comparison on the recently introduced ImageCoDe data-set (Krojer et al., 2022) which contains hard distractors nearly identical to the images to be retrieved. We find that the neural retriever has much higher performance when fed neural rather than human captions, despite the fact that the former, unlike the latter, were generated without awareness of the distractors that make the task hard. Even more remarkably, when the same neural captions are given to human subjects, their retrieval performance is almost at chance level. Our results thus add to the growing body of evidence that, even when the ``language'' of neural models resembles English, this superficial resemblance might be deeply misleading.
3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows
Venue: arXiv cs.HC 2023
Year: 2023

Text-to-image AI are capable of generating novel images for inspiration, but their applications for 3D design workflows and how designers can build 3D models using AI-provided inspiration have not yet been explored. To investigate this, we integrated DALL-E, GPT-3, and CLIP within a CAD software in 3DALL-E, a plugin that generates 2D image inspiration for 3D design. 3DALL-E allows users to construct text and image prompts based on what they are modeling. In a study with 13 designers, we found that designers saw great potential in 3DALL-E within their workflows and could use text-to-image AI to produce reference images, prevent design fixation, and inspire design considerations. We elaborate on prompting patterns observed across 3D modeling tasks and provide measures of prompt complexity observed across participants. From our findings, we discuss how 3DALL-E can merge with existing generative design workflows and propose prompt bibliographies as a form of human-AI design history.
Competing Bandits in Time Varying Matching Markets
Venue: arXiv cs.LG 2023
Year: 2023

We study the problem of online learning in two-sided non-stationary matching markets, where the objective is to converge to a stable match. In particular, we consider the setting where one side of the market, the arms, has fixed known set of preferences over the other side, the players. While this problem has been studied when the players have fixed but unknown preferences, in this work we study the problem of how to learn when the preferences of the players are time varying and unknown. Our contribution is a methodology that can handle any type of preference structure and variation scenario. We show that, with the proposed algorithm, each player receives a uniform sub-linear regret of {$\widetilde{\mathcal{O}}(L^{1/2}_TT^{1/2})$} up to the number of changes in the underlying preferences of the agents, $L_T$. Therefore, we show that the optimal rates for single-agent learning can be achieved in spite of the competition up to a difference of a constant factor. We also discuss extensions of this algorithm to the case where the number of changes need not be known a priori.
LittleBird: Efficient Faster & Longer Transformer for Question Answering
Venue: arXiv cs.CL 2023
Year: 2023

BERT has shown a lot of sucess in a wide variety of NLP tasks. But it has a limitation dealing with long inputs due to its attention mechanism. Longformer, ETC and BigBird addressed this issue and effectively solved the quadratic dependency problem. However we find that these models are not sufficient, and propose LittleBird, a novel model based on BigBird with improved speed and memory footprint while maintaining accuracy. In particular, we devise a more flexible and efficient position representation method based on Attention with Linear Biases (ALiBi). We also show that replacing the method of global information represented in the BigBird with pack and unpack attention is more effective. The proposed model can work on long inputs even after being pre-trained on short inputs, and can be trained efficiently reusing existing pre-trained language model for short inputs. This is a significant benefit for low-resource languages where large amounts of long text data are difficult to obtain. As a result, our experiments show that LittleBird works very well in a variety of languages, achieving high performance in question answering tasks, particularly in KorQuAD2.0, Korean Question Answering Dataset for long paragraphs.
Splay Top Trees
Venue: arXiv cs.DS 2023
Year: 2023

The top tree data structure is an important and fundamental tool in dynamic graph algorithms. Top trees have existed for decades, and today serve as an ingredient in many state-of-the-art algorithms for dynamic graphs. In this work, we give a new direct proof of the existence of top trees, facilitating simpler and more direct implementations of top trees, based on ideas from splay trees. This result hinges on new insights into the structure of top trees, and in particular the structure of each root path in a top tree.
Rational Ponzi Games in Algorithmic Stablecoin
Venue: arXiv cs.GT 2023
Year: 2023

Algorithmic stablecoins (AS) are one special type of stablecoins that are not backed by any asset (equiv. without collateral). They stand to revolutionize the way a sovereign fiat operates. As implemented, these coins are poorly stabilized in most cases, easily deviating from the price target or even falling into a catastrophic collapse (a.k.a. Death spiral), and are as a result dismissed as a Ponzi scheme. However, is this the whole picture? In this paper, we try to reveal the truth and clarify such a deceptive concept. We find that Ponzi is basically a financial protocol that pays existing investors with funds collected from new ones. Running a Ponzi, however, does not necessarily imply that any participant is in any sense losing out, as long as the game can be perpetually rolled over. Economists call such realization as a \textit{rational Ponzi game}. We thereby propose a rational model in the context of AS and draw its holding conditions. We apply the model to examine: \textit{whether or not the algorithmic stablecoin is a rational Ponzi game.} Accordingly, we discuss two types of algorithmic stablecoins (\text{Rebase} \& \text{Seigniorage shares}) and dig into the historical market performance of two impactful projects (\text{Ampleforth} \& \text{TerraUSD}, respectively) to demonstrate the effectiveness of our model.
An Analysis of Fusion Functions for Hybrid Retrieval
Venue: arXiv cs.IR 2023
Year: 2023

We study hybrid search in text retrieval where lexical and semantic search are fused together with the intuition that the two are complementary in how they model relevance. In particular, we examine fusion by a convex combination (CC) of lexical and semantic scores, as well as the Reciprocal Rank Fusion (RRF) method, and identify their advantages and potential pitfalls. Contrary to existing studies, we find RRF to be sensitive to its parameters; that the learning of a CC fusion is generally agnostic to the choice of score normalization; that CC outperforms RRF in in-domain and out-of-domain settings; and finally, that CC is sample efficient, requiring only a small set of training examples to tune its only parameter to a target domain.