OpenRaiser/Intern-Atlas huggingface dataset:
MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) have shown great success in various Natural Language Processing (NLP) tasks, whist they still need updates after deployment to fix errors or keep pace with the changing knowledge in the world. Researchers formulate such problem as Model Editing and have developed various editors focusing on different axes of editing properties. However, current editors can hardly support all properties and rely on heavy computational resources. In this paper, we propose a plug-in Model Editing method based on neuron-indexed dynamic LoRA (MELO), which alters the behavior of language models by dynamically activating certain LoRA blocks according to the index built in an inner vector database. Our method satisfies various editing properties with high efficiency and can be easily integrated into multiple LLM backbones. Experimental results show that our proposed MELO achieves state-of-the-art editing performance on three sequential editing tasks (document classification, question answering and hallucination correction), while requires the least trainable parameters and computational cost.
Users Approach on Providing Feedback for Smart Home Devices Phase I
Venue: arXiv cs.HC 2023
Year: 2023

Smart home technology is part of our everyday lives, and this technology is fast-evolving compared to other technologies. The user's feedback is gathered in this paper by conducting expert interviews on how collecting the feedback from the smart home devices will be helpful to improve the devices. We are yet to know about the feedback system of the smart home devices and how provided feedback will support increasing the devices' requirements. Today, we present our analysis from our exploratory interview method with the student of a certain group, and we try to study the attitude of providing feedback. The results suggested that the users are ready to give their feedback very actively to better their usage as every user has their own needs to fulfill.
Regulating Intermediate 3D Features for Vision-Centric Autonomous Driving
Venue: arXiv cs.CV 2023
Year: 2023

Multi-camera perception tasks have gained significant attention in the field of autonomous driving. However, existing frameworks based on Lift-Splat-Shoot (LSS) in the multi-camera setting cannot produce suitable dense 3D features due to the projection nature and uncontrollable densification process. To resolve this problem, we propose to regulate intermediate dense 3D features with the help of volume rendering. Specifically, we employ volume rendering to process the dense 3D features to obtain corresponding 2D features (e.g., depth maps, semantic maps), which are supervised by associated labels in the training. This manner regulates the generation of dense 3D features on the feature level, providing appropriate dense and unified features for multiple perception tasks. Therefore, our approach is termed Vampire, stands for "Volume rendering As Multi-camera Perception Intermediate feature REgulator". Experimental results on the Occ3D and nuScenes datasets demonstrate that Vampire facilitates fine-grained and appropriate extraction of dense 3D features, and is competitive with existing SOTA methods across diverse downstream perception tasks like 3D occupancy prediction, LiDAR segmentation and 3D objection detection, while utilizing moderate GPU resources. We provide a video demonstration in the supplementary materials and Codes are available at github.com/cskkxjk/Vampire.
Predicting Human Translation Difficulty with Neural Machine Translation
Venue: arXiv cs.CL 2023
Year: 2023

Human translators linger on some words and phrases more than others, and predicting this variation is a step towards explaining the underlying cognitive processes. Using data from the CRITT Translation Process Research Database, we evaluate the extent to which surprisal and attentional features derived from a Neural Machine Translation (NMT) model account for reading and production times of human translators. We find that surprisal and attention are complementary predictors of translation difficulty, and that surprisal derived from a NMT model is the single most successful predictor of production duration. Our analyses draw on data from hundreds of translators operating across 13 language pairs, and represent the most comprehensive investigation of human translation difficulty to date.
Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Network
Venue: arXiv cs.CV 2023
Year: 2023

3D-aware Generative Adversarial Networks (3D-GANs) currently exhibit artifacts in their 3D geometrical modeling, such as mesh imperfections and holes. These shortcomings are primarily attributed to the limited availability of annotated 3D data, leading to a constrained "valid latent area" for satisfactory modeling. To address this, we present a Self-Supervised Learning (SSL) technique tailored as an auxiliary loss for any 3D-GAN, designed to improve its 3D geometrical modeling capabilities. Our approach pioneers an inversion technique for 3D-GANs, integrating an encoder that performs adaptive spatially-varying range operations. Utilizing this inversion, we introduce the Cyclic Generative Constraint (CGC), aiming to densify the valid latent space. The CGC operates via augmented local latent vectors that maintain the same geometric form, and it imposes constraints on the cycle path outputs, specifically the generator-encoder-generator sequence. This SSL methodology seamlessly integrates with the inherent GAN loss, ensuring the integrity of pre-existing 3D-GAN architectures without necessitating alterations. We validate our approach with comprehensive experiments across various datasets and architectures, underscoring its efficacy. Our project website: https://3dgan-ssl.github.io
Hierarchical and Incremental Structural Entropy Minimization for Unsupervised Social Event Detection
Venue: arXiv cs.SI 2023
Year: 2023

As a trending approach for social event detection, graph neural network (GNN)-based methods enable a fusion of natural language semantics and the complex social network structural information, thus showing SOTA performance. However, GNN-based methods can miss useful message correlations. Moreover, they require manual labeling for training and predetermining the number of events for prediction. In this work, we address social event detection via graph structural entropy (SE) minimization. While keeping the merits of the GNN-based methods, the proposed framework, HISEvent, constructs more informative message graphs, is unsupervised, and does not require the number of events given a priori. Specifically, we incrementally explore the graph neighborhoods using 1-dimensional (1D) SE minimization to supplement the existing message graph with edges between semantically related messages. We then detect events from the message graph by hierarchically minimizing 2-dimensional (2D) SE. Our proposed 1D and 2D SE minimization algorithms are customized for social event detection and effectively tackle the efficiency problem of the existing SE minimization algorithms. Extensive experiments show that HISEvent consistently outperforms GNN-based methods and achieves the new SOTA for social event detection under both closed- and open-set settings while being efficient and robust.
Transformer Network for Multi-Person Tracking and Re-Identification in Unconstrained Environment
Venue: arXiv cs.CV 2023
Year: 2023

Multi-object tracking (MOT) has profound applications in a variety of fields, including surveillance, sports analytics, self-driving, and cooperative robotics. Despite considerable advancements, existing MOT methodologies tend to falter when faced with non-uniform movements, occlusions, and appearance-reappearance scenarios of the objects. Recognizing this inadequacy, we put forward an integrated MOT method that not only marries object detection and identity linkage within a singular, end-to-end trainable framework but also equips the model with the ability to maintain object identity links over long periods of time. Our proposed model, named STMMOT, is built around four key modules: 1) candidate proposal generation, which generates object proposals via a vision-transformer encoder-decoder architecture that detects the object from each frame in the video; 2) scale variant pyramid, a progressive pyramid structure to learn the self-scale and cross-scale similarities in multi-scale feature maps; 3) spatio-temporal memory encoder, extracting the essential information from the memory associated with each object under tracking; and 4) spatio-temporal memory decoder, simultaneously resolving the tasks of object detection and identity association for MOT. Our system leverages a robust spatio-temporal memory module that retains extensive historical observations and effectively encodes them using an attention-based aggregator. The uniqueness of STMMOT lies in representing objects as dynamic query embeddings that are updated continuously, which enables the prediction of object states with attention mechanisms and eradicates the need for post-processing.
Parameterized Decision-making with Multi-modal Perception for Autonomous Driving
Venue: arXiv cs.AI 2023
Year: 2023

Autonomous driving is an emerging technology that has advanced rapidly over the last decade. Modern transportation is expected to benefit greatly from a wise decision-making framework of autonomous vehicles, including the improvement of mobility and the minimization of risks and travel time. However, existing methods either ignore the complexity of environments only fitting straight roads, or ignore the impact on surrounding vehicles during optimization phases, leading to weak environmental adaptability and incomplete optimization objectives. To address these limitations, we propose a parameterized decision-making framework with multi-modal perception based on deep reinforcement learning, called AUTO. We conduct a comprehensive perception to capture the state features of various traffic participants around the autonomous vehicle, based on which we design a graph-based model to learn a state representation of the multi-modal semantic features. To distinguish between lane-following and lane-changing, we decompose an action of the autonomous vehicle into a parameterized action structure that first decides whether to change lanes and then computes an exact action to execute. A hybrid reward function takes into account aspects of safety, traffic efficiency, passenger comfort, and impact to guide the framework to generate optimal actions. In addition, we design a regularization term and a multi-worker paradigm to enhance the training. Extensive experiments offer evidence that AUTO can advance state-of-the-art in terms of both macroscopic and microscopic effectiveness.
Competition among Pairwise Lottery Contests
Venue: arXiv cs.GT 2023
Year: 2023

We investigate a two-stage competitive model involving multiple contests. In this model, each contest designer chooses two participants from a pool of candidate contestants and determines the biases. Contestants strategically distribute their efforts across various contests within their budget. We first show the existence of a pure strategy Nash equilibrium (PNE) for the contestants, and propose a polynomial-time algorithm to compute an $ε$-approximate PNE. In the scenario where designers simultaneously decide the participants and biases, the subgame perfect equilibrium (SPE) may not exist. Nonetheless, when designers' decisions are made in two substages, the existence of SPE is established. In the scenario where designers can hold multiple contests, we show that the SPE exists under mild conditions and can be computed efficiently.
GroupMixNorm Layer for Learning Fair Models
Venue: arXiv cs.LG 2023
Year: 2023

Recent research has identified discriminatory behavior of automated prediction algorithms towards groups identified on specific protected attributes (e.g., gender, ethnicity, age group, etc.). When deployed in real-world scenarios, such techniques may demonstrate biased predictions resulting in unfair outcomes. Recent literature has witnessed algorithms for mitigating such biased behavior mostly by adding convex surrogates of fairness metrics such as demographic parity or equalized odds in the loss function, which are often not easy to estimate. This research proposes a novel in-processing based GroupMixNorm layer for mitigating bias from deep learning models. The GroupMixNorm layer probabilistically mixes group-level feature statistics of samples across different groups based on the protected attribute. The proposed method improves upon several fairness metrics with minimal impact on overall accuracy. Analysis on benchmark tabular and image datasets demonstrates the efficacy of the proposed method in achieving state-of-the-art performance. Further, the experimental analysis also suggests the robustness of the GroupMixNorm layer against new protected attributes during inference and its utility in eliminating bias from a pre-trained network.
Diffusing More Objects for Semi-Supervised Domain Adaptation with Less Labeling
Venue: arXiv cs.CV 2023
Year: 2023

For object detection, it is possible to view the prediction of bounding boxes as a reverse diffusion process. Using a diffusion model, the random bounding boxes are iteratively refined in a denoising step, conditioned on the image. We propose a stochastic accumulator function that starts each run with random bounding boxes and combines the slightly different predictions. We empirically verify that this improves detection performance. The improved detections are leveraged on unlabelled images as weighted pseudo-labels for semi-supervised learning. We evaluate the method on a challenging out-of-domain test set. Our method brings significant improvements and is on par with human-selected pseudo-labels, while not requiring any human involvement.
Students' and Professionals' Perceived Creativity In Software Engineering: A Comparative Study
Venue: arXiv cs.SE 2023
Year: 2023

Creativity is a critical skill that professional software engineers leverage to tackle difficult problems. In higher education, multiple efforts have been made to spark creative skills of engineering students. However, creativity is a vague concept that is open to interpretation. Furthermore, studies have shown that there is a gap in perception and implementation of creativity between industry and academia. To better understand the role of creativity in software engineering (SE), we interviewed 33 professionals via four focus groups and 10 SE students. Our results reveal 45 underlying topics related to creativity. When comparing the perception of students versus professionals, we discovered fundamental differences, grouped into five themes: the creative environment, application of techniques, creative collaboration, nature vs nurture, and the perceived value of creativity. As our aim is to use these findings to install and further encourage creative problem solving in higher education, we have included a list of implications for educational practice.
Founder-GPT: Self-play to evaluate the Founder-Idea fit
Venue: arXiv cs.CL 2023
Year: 2023

This research introduces an innovative evaluation method for the "founder-idea" fit in early-stage startups, utilizing advanced large language model techniques to assess founders' profiles against their startup ideas to enhance decision-making. Embeddings, self-play, tree-of-thought, and critique-based refinement techniques show early promising results that each idea's success patterns are unique and they should be evaluated based on the context of the founder's background.
Monitoring Auditable Claims in the Cloud
Venue: arXiv cs.SE 2023
Year: 2023

When deploying mission-critical systems in the cloud, where deviations may have severe consequences, the assurance of critical decisions becomes essential. Typical cloud systems are operated by third parties and are built on complex software stacks consisting of e.g., Kubernetes, Istio, or Kafka, which due to their size are difficult to be verified. Nevertheless, one needs to make sure that mission-critical choices are made correctly. We propose a flexible runtime monitoring approach that is independent of the implementation of the observed system that allows to monitor safety and data-related properties. Our approach is based on combining distributed Datalog-based programs with tamper-proof storage based on Trillian to verify the premises of safety-critical actions. The approach can be seen as a generalization of the Certificate Transparency project. We apply our approach to an industrial use case that uses a cloud infrastructure for orchestrating unmanned air vehicles.
MPI Planar Correction of Pulse Based ToF Cameras
Venue: arXiv cs.CV 2023
Year: 2023

Time-of-Flight (ToF) cameras are becoming popular in a wide span of areas ranging from consumer-grade electronic devices to safety-critical industrial robots. This is mainly due to their high frame rate, relative good precision and the lowered costs. Although ToF cameras are in continuous development, especially pulse-based variants, they still face different problems, including spurious noise over the points or multipath inference (MPI). The latter can cause deformed surfaces to manifest themselves on curved surfaces instead of planar ones, making standard spatial data preprocessing, such as plane extraction, difficult. In this paper, we focus on the MPI reduction problem using Feature Pyramid Networks (FPN) which allow the mitigation of this type of artifact for pulse-based ToF cameras. With our end-to-end network, we managed to attenuate the MPI effect on planar surfaces using a learning-based method on real ToF data. Both the custom dataset used for our model training as well as the code is available on the author's Github homepage.
A Survey on Property-Preserving Database Encryption Techniques in the Cloud
Venue: arXiv cs.CR 2023
Year: 2023

Outsourcing a relational database to the cloud offers several benefits, including scalability, availability, and cost-effectiveness. However, there are concerns about the security and confidentiality of the outsourced data. A general approach here would be to encrypt the data with a standardized encryption algorithm and then store the data only encrypted in the cloud. The problem with this approach, however, is that with encryption, important properties of the data such as sorting, format or comparability, which are essential for the functioning of database queries, are lost. One solution to this problem is the use of encryption algorithms, which also preserve these properties in the encrypted data, thus enabling queries to encrypted data. These algorithms range from simple algorithms like Caesar encryption to secure algorithms like mOPE. The report at hand presents a survey on common encryption techniques used for storing data in relation Cloud database services. It presents the applied methods and identifies their characteristics.
Towards Learning-Based Gyrocompassing
Venue: arXiv cs.RO 2023
Year: 2023

Inertial navigation systems (INS) are widely used in both manned and autonomous platforms. One of the most critical tasks prior to their operation is to accurately determine their initial alignment while stationary, as it forms the cornerstone for the entire INS operational trajectory. While low-performance accelerometers can easily determine roll and pitch angles (leveling), establishing the heading angle (gyrocompassing) with low-performance gyros proves to be a challenging task without additional sensors. This arises from the limited signal strength of Earth's rotation rate, often overridden by gyro noise itself. To circumvent this deficiency, in this study we present a practical deep learning framework to effectively compensate for the inherent errors in low-performance gyroscopes. The resulting capability enables gyrocompassing, thereby eliminating the need for subsequent prolonged filtering phase (fine alignment). Through the development of theory and experimental validation, we demonstrate that the improved initial conditions establish a new lower error bound, bringing affordable gyros one step closer to being utilized in high-end tactical tasks.
Object-Aware Domain Generalization for Object Detection
Venue: arXiv cs.CV 2023
Year: 2023

Single-domain generalization (S-DG) aims to generalize a model to unseen environments with a single-source domain. However, most S-DG approaches have been conducted in the field of classification. When these approaches are applied to object detection, the semantic features of some objects can be damaged, which can lead to imprecise object localization and misclassification. To address these problems, we propose an object-aware domain generalization (OA-DG) method for single-domain generalization in object detection. Our method consists of data augmentation and training strategy, which are called OA-Mix and OA-Loss, respectively. OA-Mix generates multi-domain data with multi-level transformation and object-aware mixing strategy. OA-Loss enables models to learn domain-invariant representations for objects and backgrounds from the original and OA-Mixed images. Our proposed method outperforms state-of-the-art works on standard benchmarks. Our code is available at https://github.com/WoojuLee24/OA-DG.
LoKit (revisited): A Toolkit for Building Distributed Collaborative Applications
Venue: arXiv cs.PL 2023
Year: 2023

LoKit is a toolkit based on the coordination language LO. It allows to build distributed collaborative applications by providing a set of generic tools. This paper briefly introduces the concept of the toolkit, presents a subset of the LoKit tools, and finally demonstrates its power by discussing a sample application built with the toolkit.
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment
Venue: arXiv cs.CL 2023
Year: 2023

With the continuous growth in the number of parameters of transformer-based pretrained language models (PLMs), particularly the emergence of large language models (LLMs) with billions of parameters, many natural language processing (NLP) tasks have demonstrated remarkable success. However, the enormous size and computational demands of these models pose significant challenges for adapting them to specific downstream tasks, especially in environments with limited computational resources. Parameter Efficient Fine-Tuning (PEFT) offers an effective solution by reducing the number of fine-tuning parameters and memory usage while achieving comparable performance to full fine-tuning. The demands for fine-tuning PLMs, especially LLMs, have led to a surge in the development of PEFT methods, as depicted in Fig. 1. In this paper, we present a comprehensive and systematic review of PEFT methods for PLMs. We summarize these PEFT methods, discuss their applications, and outline future directions. Furthermore, we conduct experiments using several representative PEFT methods to better understand their effectiveness in parameter efficiency and memory efficiency. By offering insights into the latest advancements and practical applications, this survey serves as an invaluable resource for researchers and practitioners seeking to navigate the challenges and opportunities presented by PEFT in the context of PLMs.