OpenRaiser/Intern-Atlas huggingface dataset:
Universal Multi-modal Multi-domain Pre-trained Recommendation
Venue: arXiv cs.IR 2023
Year: 2023

There is a rapidly-growing research interest in modeling user preferences via pre-training multi-domain interactions for recommender systems. However, Existing pre-trained multi-domain recommendations mostly select the item texts to be bridges across domains, and simply explore the user behaviors in target domains. Hence, they ignore other informative multi-modal item contents (e.g., visual information), and also lack of thorough consideration of user behaviors from all interactive domains. To address these issues, in this paper, we propose to pre-train universal multi-modal item content presentation for multi-domain recommendation, called UniM^2Rec, which could smoothly learn the multi-modal item content presentations and the multi-modal user preferences from all domains. With the pre-trained multi-domain recommendation model, UniM^2Rec could be efficiently and effectively transferred to new target domains in practice. Extensive experiments conducted on five real-world datasets in target domains demonstrate the superiority of the proposed method over existing competitive methods, especially for the real-world recommendation scenarios that usually struggle with seriously missing or noisy item contents.
Spectral Clustering of Attributed Multi-relational Graphs
Venue: arXiv cs.LG 2023
Year: 2023

Graph clustering aims at discovering a natural grouping of the nodes such that similar nodes are assigned to a common cluster. Many different algorithms have been proposed in the literature: for simple graphs, for graphs with attributes associated to nodes, and for graphs where edges represent different types of relations among nodes. However, complex data in many domains can be represented as both attributed and multi-relational networks. In this paper, we propose SpectralMix, a joint dimensionality reduction technique for multi-relational graphs with categorical node attributes. SpectralMix integrates all information available from the attributes, the different types of relations, and the graph structure to enable a sound interpretation of the clustering results. Moreover, it generalizes existing techniques: it reduces to spectral embedding and clustering when only applied to a single graph and to homogeneity analysis when applied to categorical data. Experiments conducted on several real-world datasets enable us to detect dependencies between graph structure and categorical attributes, moreover, they exhibit the superiority of SpectralMix over existing methods.
Leveraging Mobile Learning Platforms for Flexible Education Delivery: Bridging Educational Gaps in Afghanistan
Venue: arXiv cs.CY 2023
Year: 2023

The educational landscape of Afghanistan, besieged by infrastructural inadequacies and socio-political tribulations, presents a compelling case for the integration of mobile learning platforms. This article embarks on an exploratory voyage into the realms of mobile learning as a potential harbinger of educational transformation in Afghanistan. It delineates the pervasive educational challenges, underscores the technological innovations powering mobile learning platforms, and illuminates the pathways through which mobile learning can transcend the extant barriers to education. Enriched by real-world case studies, the narrative unravels the pragmatic lessons that can be harnessed to tailor mobile learning solutions to Afghanistan's unique context. The discussion further traverses the collaborative horizon, elucidating the synergistic interplay among academia, government, the private sector, and international bodies essential for the successful implementation of mobile learning platforms. The article also furnishes pragmatic recommendations, emphasizing the triad of policy formulation, infrastructure enhancement, and capacity building as cornerstone imperatives. The envisioned integration of mobile learning platforms augurs a paradigmatic shift towards a more accessible, inclusive, and resilient educational framework in Afghanistan, with far-reaching implications for socio-economic development. Through a meticulous amalgamation of technology, policy, and collaborative endeavors, this article posits that Afghanistan stands on the cusp of an educational renaissance, with mobile learning platforms serving as a pivotal conduit toward this envisioned horizon.
Enhancing search engine precision and user experience through sentiment-based polysemy resolution
Venue: arXiv cs.IR 2023
Year: 2023

With the proliferation of digital content and the need for efficient information retrieval, this study's insights can be applied to various domains, including news services, e-commerce, and digital marketing, to provide users with more meaningful and tailored experiences. The study addresses the common problem of polysemy in search engines, where the same keyword may have multiple meanings. It proposes a solution to this issue by embedding a smart search function into the search engine, which can differentiate between different meanings based on sentiment. The study leverages sentiment analysis, a powerful natural language processing (NLP) technique, to classify and categorize news articles based on their emotional tone. This can provide more insightful and nuanced search results. The article reports an impressive accuracy rate of 85% for the proposed smart search function, which outperforms conventional search engines. This indicates the effectiveness of the sentiment-based approach. The research explores multiple sentiment analysis models, including Sentistrength and Valence Aware Dictionary for Sentiment Reasoning (VADER), to determine the best-performing approach. The findings can be applied to enhance search engines, making them more capable of understanding the context and intent behind users 'queries. This can lead to better search results that are more aligned with what users are looking for. The proposed smart search function can improve the user experience by reducing the need to sift through irrelevant search results. This is particularly important in an age where information overload is common.
From Chaos to Calibration: A Geometric Mutual Information Approach to Target-Free Camera LiDAR Extrinsic Calibration
Venue: arXiv cs.CV 2023
Year: 2023

Sensor fusion is vital for the safe and robust operation of autonomous vehicles. Accurate extrinsic sensor to sensor calibration is necessary to accurately fuse multiple sensor's data in a common spatial reference frame. In this paper, we propose a target free extrinsic calibration algorithm that requires no ground truth training data, artificially constrained motion trajectories, hand engineered features or offline optimization and that is accurate, precise and extremely robust to initialization error. Most current research on online camera-LiDAR extrinsic calibration requires ground truth training data which is impossible to capture at scale. We revisit analytical mutual information based methods first proposed in 2012 and demonstrate that geometric features provide a robust information metric for camera-LiDAR extrinsic calibration. We demonstrate our proposed improvement using the KITTI and KITTI-360 fisheye data set.
Dynamic Redundancy-aware Blockchain-based Partial Computation Offloading for the Metaverse in In-network Computing
Venue: arXiv cs.DC 2023
Year: 2023

The computing in the network (COIN) paradigm has emerged as a potential solution for computation-intensive applications like the metaverse by utilizing unused network resources. The blockchain (BC) guarantees task-offloading privacy, but cost reduction, queueing delays, and redundancy elimination remain open problems. This paper presents a redundancy-aware BC-based approach for the metaverse's partial computation offloading (PCO). Specifically, we formulate a joint BC redundancy factor (BRF) and PCO problem to minimize computation costs, maximize incentives, and meet delay and BC offloading constraints. We proved this problem is NP-hard and transformed it into two subproblems based on their temporal correlation: real-time PCO and Markov decision process-based BRF. We formulated the PCO problem as a multiuser game, proposed a decentralized algorithm for Nash equilibrium under any BC redundancy state, and designed a double deep Q-network-based algorithm for the optimal BRF policy. The BRF strategy is updated periodically based on user computation demand and network status to assist the PCO algorithm. The experimental results suggest that the proposed approach outperforms existing schemes, resulting in a remarkable 47% reduction in cost overhead, delivering approximately 64% higher rewards, and achieving convergence in just a few training episodes.
Constructing Temporal Dynamic Knowledge Graphs from Interactive Text-based Games
Venue: arXiv cs.CL 2023
Year: 2023

In natural language processing, interactive text-based games serve as a test bed for interactive AI systems. Prior work has proposed to play text-based games by acting based on discrete knowledge graphs constructed by the Discrete Graph Updater (DGU) to represent the game state from the natural language description. While DGU has shown promising results with high interpretability, it suffers from lower knowledge graph accuracy due to its lack of temporality and limited generalizability to complex environments with objects with the same label. In order to address DGU's weaknesses while preserving its high interpretability, we propose the Temporal Discrete Graph Updater (TDGU), a novel neural network model that represents dynamic knowledge graphs as a sequence of timestamped graph events and models them using a temporal point based graph neural network. Through experiments on the dataset collected from a text-based game TextWorld, we show that TDGU outperforms the baseline DGU. We further show the importance of temporal information for TDGU's performance through an ablation study and demonstrate that TDGU has the ability to generalize to more complex environments with objects with the same label. All the relevant code can be found at \url{https://github.com/yukw777/temporal-discrete-graph-updater}.
Swarm Performance Indicators: Metrics for Robustness, Fault Tolerance, Scalability and Adaptability
Venue: arXiv cs.RO 2023
Year: 2023

Swarms have distributed control and so are assumed to inherently have superior robustness, scalability and adaptability compared to centralised multi-agent systems. However, these features have generally only been defined qualitatively and there is a lack of quantitative metrics and experimental measures for the claimed parameters. Swarm Performance Indicators are defined here as Key Performance Indicators for swarm features but can be applied to multi-agent systems with centralised control as well. These swarm features are Robustness, Fault Tolerance, Adaptability and Scalability. Swarm Performance Indicators can be used to highlight the benefits of swarms beyond solely considering task-based performance metrics (e.g. time taken)
Don't Make Your LLM an Evaluation Benchmark Cheater
Venue: arXiv cs.CL 2023
Year: 2023

Large language models~(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity. To assess the model performance, a typical approach is to construct evaluation benchmarks for measuring the ability level of LLMs in different aspects. Despite that a number of high-quality benchmarks have been released, the concerns about the appropriate use of these benchmarks and the fair comparison of different models are increasingly growing. Considering these concerns, in this paper, we discuss the potential risk and impact of inappropriately using evaluation benchmarks and misleadingly interpreting the evaluation results. Specially, we focus on a special issue that would lead to inappropriate evaluation, \ie \emph{benchmark leakage}, referring that the data related to evaluation sets is occasionally used for model training. This phenomenon now becomes more common since pre-training data is often prepared ahead of model test. We conduct extensive experiments to study the effect of benchmark leverage, and find that it can dramatically boost the evaluation results, which would finally lead to an unreliable assessment of model performance. To improve the use of existing evaluation benchmarks, we finally present several guidelines for both LLM developers and benchmark maintainers. We hope this work can draw attention to appropriate training and evaluation of LLMs.
ProSG: Using Prompt Synthetic Gradients to Alleviate Prompt Forgetting of RNN-like Language Models
Venue: arXiv cs.CL 2023
Year: 2023

RNN-like language models are getting renewed attention from NLP researchers in recent years and several models have made significant progress, which demonstrates performance comparable to traditional transformers. However, due to the recurrent nature of RNNs, this kind of language model can only store information in a set of fixed-length state vectors. As a consequence, they still suffer from forgetfulness though after a lot of improvements and optimizations, when given complex instructions or prompts. As the prompted generation is the main and most concerned function of LMs, solving the problem of forgetting in the process of generation is no wonder of vital importance. In this paper, focusing on easing the prompt forgetting during generation, we proposed an architecture to teach the model memorizing prompt during generation by synthetic gradient. To force the model to memorize the prompt, we derive the states that encode the prompt, then transform it into model parameter modification using low-rank gradient approximation, which hard-codes the prompt into model parameters temporarily. We construct a dataset for experiments, and the results have demonstrated the effectiveness of our method in solving the problem of forgetfulness in the process of prompted generation. We will release all the code upon acceptance.
NSF Integrated Circuit Research, Education and Workforce Development Workshop Final Report
Venue: arXiv cs.AR 2023
Year: 2023

As the pace of progress that has followed Moore's law continues to diminish, it is critical that the US support Integrated Circuit (IC or chip) education and research to maintain technological innovation. Furthermore, US economic independence, security, and future international standing rely on having on-shore IC design capabilities. New devices with disparate technologies, improved design software toolchains and methodologies, and technologies to integrate heterogeneous systems will be needed to advance IC design capabilities. This will require rethinking both how we teach design to address the new complexity and how we inspire student interest in a hardware systems career path. The main recommendation of this workshop is that accessibility is the key issue. To this end, a National Chip Design Center (NCDC) should be established to further research and education by partnering academics and industry to train our future workforce. This should not be limited to R1 universities, but should also include R2, community college, minority serving institutions (MSI), and K-12 institutions to have the broadest effect. The NCDC should support the access, development, and maintenance of open design tools, tool flows, design kits, design components, and educational materials. Open-source options should be emphasized wherever possible to maximize accessibility. The NCDC should also provide access and support for chip fabrication, packaging and testing for both research and educational purposes.
MaRU: A Manga Retrieval and Understanding System Connecting Vision and Language
Venue: arXiv cs.IR 2023
Year: 2023

Manga, a widely celebrated Japanese comic art form, is renowned for its diverse narratives and distinct artistic styles. However, the inherently visual and intricate structure of Manga, which comprises images housing multiple panels, poses significant challenges for content retrieval. To address this, we present MaRU (Manga Retrieval and Understanding), a multi-staged system that connects vision and language to facilitate efficient search of both dialogues and scenes within Manga frames. The architecture of MaRU integrates an object detection model for identifying text and frame bounding boxes, a Vision Encoder-Decoder model for text recognition, a text encoder for embedding text, and a vision-text encoder that merges textual and visual information into a unified embedding space for scene retrieval. Rigorous evaluations reveal that MaRU excels in end-to-end dialogue retrieval and exhibits promising results for scene retrieval.
Solving MaxSAT with Matrix Multiplication
Venue: arXiv cs.AI 2023
Year: 2023

We propose an incomplete algorithm for Maximum Satisfiability (MaxSAT) specifically designed to run on neural network accelerators such as GPUs and TPUs. Given a MaxSAT problem instance in conjunctive normal form, our procedure constructs a Restricted Boltzmann Machine (RBM) with an equilibrium distribution wherein the probability of a Boolean assignment is exponential in the number of clauses it satisfies. Block Gibbs sampling is used to stochastically search the space of assignments with parallel Markov chains. Since matrix multiplication is the main computational primitive for block Gibbs sampling in an RBM, our approach leads to an elegantly simple algorithm (40 lines of JAX) well-suited for neural network accelerators. Theoretical results about RBMs guarantee that the required number of visible and hidden units of the RBM scale only linearly with the number of variables and constant-sized clauses in the MaxSAT instance, ensuring that the computational cost of a Gibbs step scales reasonably with the instance size. Search throughput can be increased by batching parallel chains within a single accelerator as well as by distributing them across multiple accelerators. As a further enhancement, a heuristic based on unit propagation running on CPU is periodically applied to the sampled assignments. Our approach, which we term RbmSAT, is a new design point in the algorithm-hardware co-design space for MaxSAT. We present timed results on a subset of problem instances from the annual MaxSAT Evaluation's Incomplete Unweighted Track for the years 2018 to 2021. When allotted the same running time and CPU compute budget (but no TPUs), RbmSAT outperforms other participating solvers on problems drawn from three out of the four years' competitions. Given the same running time on a TPU cluster for which RbmSAT is uniquely designed, it outperforms all solvers on problems drawn from all four years.
A Virtual Reality Training System for Automotive Engines Assembly and Disassembly
Venue: arXiv cs.HC 2023
Year: 2023

Automotive engine assembly and disassembly are common and crucial programs in the automotive industry. Traditional education trains students to learn automotive engine assembly and disassembly in lecture courses and then to operate with physical engines, which are generally low effectiveness and high cost. In this work, we developed a multi-layer structured Virtual Reality (VR) system to provide students with training in automotive engine (Buick Verano) assembly and disassembly. We designed the VR training system with The VR training system is designed to have several major features, including replaceable engine parts and reusable tools, friendly user interfaces and guidance, and bottom-up designed multi-layer architecture, which can be extended to various engine models. The VR system is evaluated with controlled experiments of two groups of students. The results demonstrate that our VR training system provides remarkable usability in terms of effectiveness and efficiency. Currently, our VR system has been demonstrated and employed in the courses of Chinese colleges to train students in automotive engine assembly and disassembly. A free-to-use executable file (Microsoft Windows) and open-source code are available at https://github.com/LadissonLai/SUSTech_VREngine for facilitating the development of VR systems in the automotive industry. Finally, a video describing the operations in our VR training system is available at https://www.youtube.com/watch?v=yZe4YTwwAC4
Enhancing Monocular Height Estimation from Aerial Images with Street-view Images
Venue: arXiv cs.CV 2023
Year: 2023

Accurate height estimation from monocular aerial imagery presents a significant challenge due to its inherently ill-posed nature. This limitation is rooted in the absence of adequate geometric constraints available to the model when training with monocular imagery. Without additional geometric information to supplement the monocular image data, the model's ability to provide reliable estimations is compromised. In this paper, we propose a method that enhances monocular height estimation by incorporating street-view images. Our insight is that street-view images provide a distinct viewing perspective and rich structural details of the scene, serving as geometric constraints to enhance the performance of monocular height estimation. Specifically, we aim to optimize an implicit 3D scene representation, density field, with geometry constraints from street-view images, thereby improving the accuracy and robustness of height estimation. Our experimental results demonstrate the effectiveness of our proposed method, outperforming the baseline and offering significant improvements in terms of accuracy and structural consistency.
Hierarchical Reinforcement Learning for Power Network Topology Control
Venue: arXiv cs.LG 2023
Year: 2023

Learning in high-dimensional action spaces is a key challenge in applying reinforcement learning (RL) to real-world systems. In this paper, we study the possibility of controlling power networks using RL methods. Power networks are critical infrastructures that are complex to control. In particular, the combinatorial nature of the action space poses a challenge to both conventional optimizers and learned controllers. Hierarchical reinforcement learning (HRL) represents one approach to address this challenge. More precisely, a HRL framework for power network topology control is proposed. The HRL framework consists of three levels of action abstraction. At the highest level, there is the overall long-term task of power network operation, namely, keeping the power grid state within security constraints at all times, which is decomposed into two temporally extended actions: 'do nothing' versus 'propose a topology change'. At the intermediate level, the action space consists of all controllable substations. Finally, at the lowest level, the action space consists of all configurations of the chosen substation. By employing this HRL framework, several hierarchical power network agents are trained for the IEEE 14-bus network. Whereas at the highest level a purely rule-based policy is still chosen for all agents in this study, at the intermediate level the policy is trained using different state-of-the-art RL algorithms. At the lowest level, either an RL algorithm or a greedy algorithm is used. The performance of the different 3-level agents is compared with standard baseline (RL or greedy) approaches. A key finding is that the 3-level agent that employs RL both at the intermediate and the lowest level outperforms all other agents on the most difficult task. Our code is publicly available.