OpenRaiser/Intern-Atlas huggingface dataset:
AI-Assisted SQL Authoring at Industry Scale
Venue: arXiv cs.SE 2024
Year: 2024

SqlCompose brings generative AI into the data analytics domain. SQL is declarative, has formal table schemas, and is often written in a non-linear manner. We address each of these challenges and develop a set of models that shows the importance of each problem. We first develop an internal SQL benchmark to perform offline tests at Meta. We evaluate how well the Public Llama model performs. We attain a BLEU score of 53% and 24% for single- and multi-line predictions, respectively. This performance is consistent with prior works on imperative languages. We then fine-tune Llama on our internal data and database schemas. SqlComposeSA substantially outperforms Llama by 16 percentage points on BLEU score. SQL is often written with multiple sub queries and in a non-sequential manner. We develop SqlComposeFIM which is aware of the context before and after the line(s) that need to be completed. This fill-in-the-middle model outperform SqlComposeFIM by 35 percentage points. We also measure how often the models get the correct table names, and SqlComposeFIM is able to do this 75% of the time. Aside from our scientific research, we also roll out SqlComposeFIM at Meta. SqlCompose is used on a weekly basis by over 10k users including data scientists and software engineers, less than 1% of users have disabled SqlCompose. We use the feedback from users to improve SqlCompose. Interesting positive themes include completing tedious or repetitive SQL clauses, suggesting boilerplate coding, and help in eliminate the need to remember difficult SQL syntax. The most significant negative themes was table and column name hallucinations, which has been reduced with the release of SqlComposeFIM. The SqlCompose models consistently outperform public and internal LLMs, despite being smaller (7 bn and 13 bn), which provides early indications that smaller specialist models can outperform larger general purpose models.
Griffin: Fast Transactional Database Index with Hash and B+-Tree
Venue: arXiv cs.DB 2024
Year: 2024

Index access is one of the dominant performance factors in transactional database systems. Many systems use a B+-tree or one of its variants to handle point and range operations. This access pattern has room for performance improvement. Firstly, point operations can potentially be processed in $O(1)$ with a hash table. Secondly, to ensure serializability of transactions, range operations incur overhead from phantom avoidance techniques that involve additional processing or synchronization, such as an extra traversal of the B+-tree. To address these issues, we propose a hybrid index architecture, Griffin. For point operations, Griffin has a hash table that provides access paths in $O(1)$ time, along with a B+-tree. For phantom avoidance, Griffin employs a precision locking method, which does not involve additional traversal of the B+-tree. Despite its hybrid architecture, Griffin transparently provides linearizable operations and an interface of a single database index. We built a Griffin index combining a hash table and BwTree. Compared to a baseline index that is composed of a BwTree only, it achieves up to 3.1x higher throughput in a point operation dominant workload, and up to 5.4x higher throughput in a range operation dominant workload.
Exposure Completing for Temporally Consistent Neural High Dynamic Range Video Rendering
Venue: arXiv cs.CV 2024
Year: 2024

High dynamic range (HDR) video rendering from low dynamic range (LDR) videos where frames are of alternate exposure encounters significant challenges, due to the exposure change and absence at each time stamp. The exposure change and absence make existing methods generate flickering HDR results. In this paper, we propose a novel paradigm to render HDR frames via completing the absent exposure information, hence the exposure information is complete and consistent. Our approach involves interpolating neighbor LDR frames in the time dimension to reconstruct LDR frames for the absent exposures. Combining the interpolated and given LDR frames, the complete set of exposure information is available at each time stamp. This benefits the fusing process for HDR results, reducing noise and ghosting artifacts therefore improving temporal consistency. Extensive experimental evaluations on standard benchmarks demonstrate that our method achieves state-of-the-art performance, highlighting the importance of absent exposure completing in HDR video rendering. The code is available at https://github.com/cuijiahao666/NECHDR.
Fully Test-Time rPPG Estimation via Synthetic Signal-Guided Feature Learning
Venue: arXiv cs.CV 2024
Year: 2024

Many remote photoplethysmography (rPPG) estimation models have achieved promising performance in the training domain but often fail to accurately estimate physiological signals or heart rates (HR) in the target domains. Domain generalization (DG) or domain adaptation (DA) techniques are therefore adopted during the offline training stage to adapt the model to either unobserved or observed target domains by utilizing all available source domain data. However, in rPPG estimation problems, the adapted model usually encounters challenges in estimating target data with significant domain variation. In contrast, Test-Time Adaptation (TTA) enables the model to adaptively estimate rPPG signals in various unseen domains by online adapting to unlabeled target data without referring to any source data. In this paper, we first establish a new TTA-rPPG benchmark that encompasses various domain information and HR distributions to simulate the challenges encountered in real-world rPPG estimation. Next, we propose a novel synthetic signal-guided rPPG estimation framework to address the forgetting issue during the TTA stage and to enhance the adaptation capability of the pre-trained rPPG model. To this end, we develop a synthetic signal-guided feature learning method by synthesizing pseudo rPPG signals as pseudo ground truths to guide a conditional generator in generating latent rPPG features. In addition, we design an effective spectral-based entropy minimization technique to encourage the rPPG model to learn new target domain information. Both the generated rPPG features and synthesized rPPG signals prevent the rPPG model from overfitting to target data and forgetting previously acquired knowledge, while also broadly covering various heart rate (HR) distributions. Our extensive experiments on the TTA-rPPG benchmark show that the proposed method achieves superior performance.
Hybrid Deep Learning-Based for Enhanced Occlusion Segmentation in PICU Patient Monitoring
Venue: arXiv cs.CV 2024
Year: 2024

Remote patient monitoring has emerged as a prominent non-invasive method, using digital technologies and computer vision (CV) to replace traditional invasive monitoring. While neonatal and pediatric departments embrace this approach, Pediatric Intensive Care Units (PICUs) face the challenge of occlusions hindering accurate image analysis and interpretation. \textit{Objective}: In this study, we propose a hybrid approach to effectively segment common occlusions encountered in remote monitoring applications within PICUs. Our approach centers on creating a deep-learning pipeline for limited training data scenarios. \textit{Methods}: First, a combination of the well-established Google DeepLabV3+ segmentation model with the transformer-based Segment Anything Model (SAM) is devised for occlusion segmentation mask proposal and refinement. We then train and validate this pipeline using a small dataset acquired from real-world PICU settings with a Microsoft Kinect camera, achieving an Intersection-over-Union (IoU) metric of 85\%. \textit{Results}: Both quantitative and qualitative analyses underscore the effectiveness of our proposed method. The proposed framework yields an overall classification performance with 92.5\% accuracy, 93.8\% recall, 90.3\% precision, and 92.0\% F1-score. Consequently, the proposed method consistently improves the predictions across all metrics, with an average of 2.75\% gain in performance compared to the baseline CNN-based framework. \textit{Conclusions}: Our proposed hybrid approach significantly enhances the segmentation of occlusions in remote patient monitoring within PICU settings. This advancement contributes to improving the quality of care for pediatric patients, addressing a critical need in clinical practice by ensuring more accurate and reliable remote monitoring.
Learning-From-Mistakes Prompting for Indigenous Language Translation
Venue: arXiv cs.CL 2024
Year: 2024

Using large language models, this paper presents techniques to improve extremely low-resourced indigenous language translations. Our approaches are grounded in the use of (1) the presence of a datastore consisting of a limited number of parallel translation examples, (2) the inherent capabilities of LLMs like GPT-3.5, and (3) a word-level translation dictionary. We harness the potential of LLMs and in-context learning techniques in such a setting for using LLMs as universal translators for extremely low-resourced languages. Our methodology hinges on utilizing LLMs as language compilers for selected language pairs, hypothesizing that they could internalize syntactic structures to facilitate accurate translation. We introduce three techniques: KNNPrompting with Retrieved Prompting Context, Chain-of-Thought Prompting and Learningfrom-Mistakes Prompting, with the last method addressing past errors. The evaluation results suggest that, even with limited corpora, LLMs can effectively translate extremely low-resource languages when paired with proper prompting.
Affordance Perception by a Knowledge-Guided Vision-Language Model with Efficient Error Correction
Venue: arXiv cs.CV 2024
Year: 2024

Mobile robot platforms will increasingly be tasked with activities that involve grasping and manipulating objects in open world environments. Affordance understanding provides a robot with means to realise its goals and execute its tasks, e.g. to achieve autonomous navigation in unknown buildings where it has to find doors and ways to open these. In order to get actionable suggestions, robots need to be able to distinguish subtle differences between objects, as they may result in different action sequences: doorknobs require grasp and twist, while handlebars require grasp and push. In this paper, we improve affordance perception for a robot in an open-world setting. Our contribution is threefold: (1) We provide an affordance representation with precise, actionable affordances; (2) We connect this knowledge base to a foundational vision-language models (VLM) and prompt the VLM for a wider variety of new and unseen objects; (3) We apply a human-in-the-loop for corrections on the output of the VLM. The mix of affordance representation, image detection and a human-in-the-loop is effective for a robot to search for objects to achieve its goals. We have demonstrated this in a scenario of finding various doors and the many different ways to open them.
Time Synchronization of TESLA-enabled GNSS Receivers
Venue: arXiv cs.CR 2024
Year: 2024

As TESLA-enabled GNSS for authenticated positioning reaches ubiquity, receivers must use an onboard, GNSS-independent clock and carefully constructed time synchronization algorithms to assert the authenticity afforded. This work provides the necessary checks and synchronization protocols needed in the broadcast-only GNSS context. We provide proof of security for each of our algorithms under a delay-capable adversary. The algorithms included herein enable a GNSS receiver to use its onboard, GNSS-independent clock to determine whether a message arrived at the correct time, to determine whether its onboard, GNSS-independent clock is safe to use and when the clock will no longer be safe in the future due to predicted clock drift, and to resynchronize its onboard, GNSS-independent clock. Each algorithm is safe to use even when an adversary induces delays within the protocol. Moreover, we discuss the implications of GNSS authentication schemes that use two simultaneous TESLA instances of different authentication cadences. To a receiver implementer or standards author, this work provides the necessary implementation algorithms to assert security and provides a comprehensive guide on why these methods are required.
WiNet: Wavelet-based Incremental Learning for Efficient Medical Image Registration
Venue: arXiv cs.CV 2024
Year: 2024

Deep image registration has demonstrated exceptional accuracy and fast inference. Recent advances have adopted either multiple cascades or pyramid architectures to estimate dense deformation fields in a coarse-to-fine manner. However, due to the cascaded nature and repeated composition/warping operations on feature maps, these methods negatively increase memory usage during training and testing. Moreover, such approaches lack explicit constraints on the learning process of small deformations at different scales, thus lacking explainability. In this study, we introduce a model-driven WiNet that incrementally estimates scale-wise wavelet coefficients for the displacement/velocity field across various scales, utilizing the wavelet coefficients derived from the original input image pair. By exploiting the properties of the wavelet transform, these estimated coefficients facilitate the seamless reconstruction of a full-resolution displacement/velocity field via our devised inverse discrete wavelet transform (IDWT) layer. This approach avoids the complexities of cascading networks or composition operations, making our WiNet an explainable and efficient competitor with other coarse-to-fine methods. Extensive experimental results from two 3D datasets show that our WiNet is accurate and GPU efficient. The code is available at https://github.com/x-xc/WiNet .
End-To-End Clinical Trial Matching with Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Matching cancer patients to clinical trials is essential for advancing treatment and patient care. However, the inconsistent format of medical free text documents and complex trial eligibility criteria make this process extremely challenging and time-consuming for physicians. We investigated whether the entire trial matching process - from identifying relevant trials among 105,600 oncology-related clinical trials on clinicaltrials.gov to generating criterion-level eligibility matches - could be automated using Large Language Models (LLMs). Using GPT-4o and a set of 51 synthetic Electronic Health Records (EHRs), we demonstrate that our approach identifies relevant candidate trials in 93.3% of cases and achieves a preliminary accuracy of 88.0% when matching patient-level information at the criterion level against a baseline defined by human experts. Utilizing LLM feedback reveals that 39.3% criteria that were initially considered incorrect are either ambiguous or inaccurately annotated, leading to a total model accuracy of 92.7% after refining our human baseline. In summary, we present an end-to-end pipeline for clinical trial matching using LLMs, demonstrating high precision in screening and matching trials to individual patients, even outperforming the performance of qualified medical doctors. Our fully end-to-end pipeline can operate autonomously or with human supervision and is not restricted to oncology, offering a scalable solution for enhancing patient-trial matching in real-world settings.
Attention Overflow: Language Model Input Blur during Long-Context Missing Items Recommendation
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) can suggest missing elements from items listed in a prompt, which can be used for list completion or recommendations based on users' history. However, their performance degrades when presented with too many items, as they start to suggest items already included in the input list. This occurs at around 100 items for mid-2024 flagship LLMs. We evaluate this phenomenon on both synthetic problems (e.g., finding missing numbers in a given range of shuffled integers) and realistic movie recommendation scenarios. We refer to this issue as \textit{attention overflow}, as preventing repetition requires attending to all items simultaneously. Although iterative loops can mitigate this problem, their costs increase with the repetition rate, affecting the language models' ability to derive novelty from lengthy inputs.
Identifying Research Hotspots and Future Development Trends in Current Psychology: A Bibliometric Analysis of the Past Decade's Publications
Venue: arXiv cs.DL 2024
Year: 2024

By conducting a bibliometric analysis on 4,869 publications in Current Psychology from 2013 to 2022, this paper examined the annual publications and annual citations, as well as the leading institutions, countries, and keywords. CiteSpace, VOSviewer and SCImago Graphica were utilized for visualization analysis. On one hand, this paper analyzed the academic influence of Current Psychology over the past decade. On the other hand, it explored the research hotspots and future development trends within the field of international psychology. The results revealed that the three main research areas covered in the publications of Current Psychology were: the psychological well-being of young people, the negative emotions of adults, and self-awareness and management. The latest research hotspots highlighted in the journal include negative emotions, personality, and mental health. The three main development trends of Current Psychology are: 1) exploring the personality psychology of both adolescents and adults, 2) promoting the interdisciplinary research to study social psychological issues through the use of diversified research methods, and 3) emphasizing the emotional psychology of individuals and their interaction with social reality, from a people-oriented perspective.
Provably Robust and Secure Steganography in Asymmetric Resource Scenario
Venue: arXiv cs.CR 2024
Year: 2024

To circumvent the unbridled and ever-encroaching surveillance and censorship in cyberspace, steganography has garnered attention for its ability to hide private information in innocent-looking carriers. Current provably secure steganography approaches require a pair of encoder and decoder to hide and extract private messages, both of which must run the same model with the same input to obtain identical distributions. These requirements pose significant challenges to the practical implementation of steganography, including limited access to powerful hardware and the intolerance of any changes to the shared input. To relax the limitation of hardware and solve the challenge of vulnerable shared input, a novel and practically significant scenario with asymmetric resource should be considered, where only the encoder is high-resource and accessible to powerful models while the decoder can only read the steganographic carriers without any other model's input. This paper proposes a novel provably robust and secure steganography framework for the asymmetric resource setting. Specifically, the encoder uses various permutations of distribution to hide secret bits, while the decoder relies on a sampling function to extract the hidden bits by guessing the permutation used. Further, the sampling function only takes the steganographic carrier as input, which makes the decoder independent of model's input and model itself. A comprehensive assessment of applying our framework to generative models substantiates its effectiveness. Our implementation demonstrates robustness when transmitting over binary symmetric channels with errors.
Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation
Venue: arXiv cs.RO 2024
Year: 2024

Large Language Models (LLMs) have been recently used in robot applications for grounding LLM common-sense reasoning with the robot's perception and physical abilities. In humanoid robots, memory also plays a critical role in fostering real-world embodiment and facilitating long-term interactive capabilities, especially in multi-task setups where the robot must remember previous task states, environment states, and executed actions. In this paper, we address incorporating memory processes with LLMs for generating cross-task robot actions, while the robot effectively switches between tasks. Our proposed dual-layered architecture features two LLMs, utilizing their complementary skills of reasoning and following instructions, combined with a memory model inspired by human cognition. Our results show a significant improvement in performance over a baseline of five robotic tasks, demonstrating the potential of integrating memory with LLMs for combining the robot's action and perception for adaptive task execution.
A Security Assessment tool for Quantum Threat Analysis
Venue: arXiv cs.CR 2024
Year: 2024

The rapid advancement of quantum computing poses a significant threat to many current security algorithms used for secure communication, digital authentication, and information encryption. A sufficiently powerful quantum computer could potentially exploit vulnerabilities in these algorithms, rendering data in transit insecure. This threat is expected to materialize within the next 20 years. Immediate transition to quantum-resilient cryptographic schemes is crucial, primarily to mitigate store-now-decrypt-later attacks and to ensure the security of products with decade-long operational lives. This transition requires a systematic approach to identifying and upgrading vulnerable cryptographic implementations. This work developed a quantum assessment tool for organizations, providing tailored recommendations for transitioning their security protocols into a post-quantum world. The work included a systematic evaluation of the proposed solution using qualitative feedback from network administrators and cybersecurity experts. This feedback was used to refine the accuracy and usability of the assessment process. The results demonstrate its effectiveness and usefulness in helping organizations prepare for quantum computing threats. The assessment tool is publicly available at (https://quantum-watch.soton.ac.uk).
PriPL-Tree: Accurate Range Query for Arbitrary Distribution under Local Differential Privacy
Venue: arXiv cs.CR 2024
Year: 2024

Answering range queries in the context of Local Differential Privacy (LDP) is a widely studied problem in Online Analytical Processing (OLAP). Existing LDP solutions all assume a uniform data distribution within each domain partition, which may not align with real-world scenarios where data distribution is varied, resulting in inaccurate estimates. To address this problem, we introduce PriPL-Tree, a novel data structure that combines hierarchical tree structures with piecewise linear (PL) functions to answer range queries for arbitrary distributions. PriPL-Tree precisely models the underlying data distribution with a few line segments, leading to more accurate results for range queries. Furthermore, we extend it to multi-dimensional cases with novel data-aware adaptive grids. These grids leverage the insights from marginal distributions obtained through PriPL-Trees to partition the grids adaptively, adapting the density of underlying distributions. Our extensive experiments on both real and synthetic datasets demonstrate the effectiveness and superiority of PriPL-Tree over state-of-the-art solutions in answering range queries across arbitrary data distributions.