OpenRaiser/Intern-Atlas huggingface dataset:
XHand: Real-time Expressive Hand Avatar
Venue: arXiv cs.CV 2024
Year: 2024

Hand avatars play a pivotal role in a wide array of digital interfaces, enhancing user immersion and facilitating natural interaction within virtual environments. While previous studies have focused on photo-realistic hand rendering, little attention has been paid to reconstruct the hand geometry with fine details, which is essential to rendering quality. In the realms of extended reality and gaming, on-the-fly rendering becomes imperative. To this end, we introduce an expressive hand avatar, named XHand, that is designed to comprehensively generate hand shape, appearance, and deformations in real-time. To obtain fine-grained hand meshes, we make use of three feature embedding modules to predict hand deformation displacements, albedo, and linear blending skinning weights, respectively. To achieve photo-realistic hand rendering on fine-grained meshes, our method employs a mesh-based neural renderer by leveraging mesh topological consistency and latent codes from embedding modules. During training, a part-aware Laplace smoothing strategy is proposed by incorporating the distinct levels of regularization to effectively maintain the necessary details and eliminate the undesired artifacts. The experimental evaluations on InterHand2.6M and DeepHandMesh datasets demonstrate the efficacy of XHand, which is able to recover high-fidelity geometry and texture for hand animations across diverse poses in real-time. To reproduce our results, we will make the full implementation publicly available at https://github.com/agnJason/XHand.
Human-Data Interaction Framework: A Comprehensive Model for a Future Driven by Data and Humans
Venue: arXiv cs.CY 2024
Year: 2024

In an age defined by rapid data expansion, the connection between individuals and their digital footprints has become more intricate. The Human-Data Interaction (HDI) framework has become an essential approach to tackling the challenges and ethical issues associated with data governance and utilization in the modern digital world. This paper outlines the fundamental steps required for organizations to seamlessly integrate HDI principles, emphasizing auditing, aligning, formulating considerations, and the need for continuous monitoring and adaptation. Through a thorough audit, organizations can critically assess their current data management practices, trace the data lifecycle from collection to disposal, and evaluate the effectiveness of existing policies, security protocols, and user interfaces. The next step involves aligning these practices with the main HDI principles, such as informed consent, data transparency, user control, algorithm transparency, and ethical data use, to identify gaps that need strategic action. Formulating preliminary considerations includes developing policies and technical solutions to close identified gaps, ensuring that these practices not only meet legal standards, but also promote fairness and accountability in data interactions. The final step, monitoring and adaptation, highlights the need for setting up continuous evaluation mechanisms and being responsive to technological, regulatory, and societal developments, ensuring HDI practices stay up-to-date and effective. Successful implementation of the HDI framework requires multi-disciplinary collaboration, incorporating insights from technology, law, ethics, and user experience design. The paper posits that this comprehensive approach is vital for building trust and legitimacy in digital environments, ultimately leading to more ethical, transparent, and user-centric data interactions.
E-Commerce Product Recommendation System based on ML Algorithms
Venue: arXiv cs.IR 2024
Year: 2024

Algorithms are used in eCommerce product recommendation systems. These systems just recently began utilizing machine learning algorithms due to the development and growth of the artificial intelligence research community. This project aspires to transform how eCommerce platforms communicate with their users. We have created a model that can customize product recommendations and offers for each unique customer using cutting-edge machine learning techniques, we used PCA to reduce features and four machine learning algorithms like Gaussian Naive Bayes (GNB), Random Forest (RF), Logistic Regression (LR), Decision Tree (DT), the Random Forest algorithms achieve the highest accuracy of 99.6% with a 96.99 r square score, 1.92% MSE score, and 0.087 MAE score. The outcome is advantageous for both the client and the business. In this research, we will examine the model's development and training in detail and show how well it performs using actual data. Learning from machines can change of eCommerce world.
Success Probability in Multi-View Imaging
Venue: arXiv cs.CV 2024
Year: 2024

Platforms such as robots, security cameras, drones and satellites are used in multi-view imaging for three-dimensional (3D) recovery by stereoscopy or tomography. Each camera in the setup has a field of view (FOV). Multi-view analysis requires overlap of the FOVs of all cameras, or a significant subset of them. However, the success of such methods is not guaranteed, because the FOVs may not sufficiently overlap. The reason is that pointing of a camera from a mount or platform has some randomness (noise), due to imprecise platform control, typical to mechanical systems, and particularly moving systems such as satellites. So, success is probabilistic. This paper creates a framework to analyze this aspect. This is critical for setting limitations on the capabilities of imaging systems, such as resolution (pixel footprint), FOV, the size of domains that can be captured, and efficiency. The framework uses the fact that imprecise pointing can be mitigated by self-calibration - provided that there is sufficient overlap between pairs of views and sufficient visual similarity of views. We show an example considering the design of a formation of nanosatellites that seek 3D reconstruction of clouds.
An Application of Large Language Models to Coding Negotiation Transcripts
Venue: arXiv cs.CL 2024
Year: 2024

In recent years, Large Language Models (LLM) have demonstrated impressive capabilities in the field of natural language processing (NLP). This paper explores the application of LLMs in negotiation transcript analysis by the Vanderbilt AI Negotiation Lab. Starting in September 2022, we applied multiple strategies using LLMs from zero shot learning to fine tuning models to in-context learning). The final strategy we developed is explained, along with how to access and use the model. This study provides a sense of both the opportunities and roadblocks for the implementation of LLMs in real life applications and offers a model for how LLMs can be applied to coding in other fields.
Unlocking the Potential: Benchmarking Large Language Models in Water Engineering and Research
Venue: arXiv cs.CL 2024
Year: 2024

Recent advancements in Large Language Models (LLMs) have sparked interest in their potential applications across various fields. This paper embarked on a pivotal inquiry: Can existing LLMs effectively serve as "water expert models" for water engineering and research tasks? This study was the first to evaluate LLMs' contributions across various water engineering and research tasks by establishing a domain-specific benchmark suite, namely, WaterER. Herein, we prepared 983 tasks related to water engineering and research, categorized into "wastewater treatment", "environmental restoration", "drinking water treatment and distribution", "sanitation", "anaerobic digestion" and "contaminants assessment". We evaluated the performance of seven LLMs (i.e., GPT-4, GPT-3.5, Gemini, GLM-4, ERNIE, QWEN and Llama3) on these tasks. We highlighted the strengths of GPT-4 in handling diverse and complex tasks of water engineering and water research, the specialized capabilities of Gemini in academic contexts, Llama3's strongest capacity to answer Chinese water engineering questions and the competitive performance of Chinese-oriented models like GLM-4, ERNIE and QWEN in some water engineering tasks. More specifically, current LLMs excelled particularly in generating precise research gaps for papers on "contaminants and related water quality monitoring and assessment". Additionally, they were more adept at creating appropriate titles for research papers on "treatment processes for wastewaters", "environmental restoration", and "drinking water treatment". Overall, this study pioneered evaluating LLMs in water engineering and research by introducing the WaterER benchmark to assess the trustworthiness of their predictions. This standardized evaluation framework would also drive future advancements in LLM technology by using targeting datasets, propelling these models towards becoming true "water expert".
An Active Inference Strategy for Prompting Reliable Responses from Large Language Models in Medical Practice
Venue: arXiv cs.CL 2024
Year: 2024

Continuing advances in Large Language Models (LLMs) in artificial intelligence offer important capacities in intuitively accessing and using medical knowledge in many contexts, including education and training as well as assessment and treatment. Most of the initial literature on LLMs in medicine has emphasized that LLMs are unsuitable for medical use because they are non-deterministic, may provide incorrect or harmful responses, and cannot be regulated to assure quality control. If these issues could be corrected, optimizing LLM technology could benefit patients and physicians by providing affordable, point-of-care medical knowledge. Our proposed framework refines LLM responses by restricting their primary knowledge base to domain-specific datasets containing validated medical information. Additionally, we introduce an actor-critic LLM prompting protocol based on active inference principles of human cognition, where a Therapist agent initially responds to patient queries, and a Supervisor agent evaluates and adjusts responses to ensure accuracy and reliability. We conducted a validation study where expert cognitive behaviour therapy for insomnia (CBT-I) therapists evaluated responses from the LLM in a blind format. Experienced human CBT-I therapists assessed responses to 100 patient queries, comparing LLM-generated responses with appropriate and inappropriate responses crafted by experienced CBT-I therapists. Results showed that LLM responses received high ratings from the CBT-I therapists, often exceeding those of therapist-generated appropriate responses. This structured approach aims to integrate advanced LLM technology into medical applications, meeting regulatory requirements for establishing the safe and effective use of special purpose validated LLMs in medicine.
Occam's Razor and Bender and Koller's Octopus
Venue: arXiv cs.CL 2024
Year: 2024

We discuss the teaching of the discussion surrounding Bender and Koller's prominent ACL 2020 paper, "Climbing toward NLU: on meaning form, and understanding in the age of data" \cite{bender2020climbing}. We present what we understand to be the main contentions of the paper, and then recommend that the students engage with the natural counter-arguments to the claims in the paper. We attach teaching materials that we use to facilitate teaching this topic to undergraduate students.
Physical Modelling and Cancellation of External Passive Intermodulation in FDD MIMO
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, the physical approach to model external (air-induced) passive intermodulation (PIM) is presented in a frequency-division duplexing (FDD) multiple-input multiple-output (MIMO) system with an arbitrary number of transceiver chains. The external PIM is a special case of intermodulation distortion (IMD), mainly generated by metallic objects possessing nonlinear properties ("rusty bolt" effect). Typically, such sources are located in the near-field or transition region of the antenna array. PIM products may fall into the receiver band of the FDD system, negatively affecting the uplink signal. In contrast to other works, this one directly simulates the physical external PIM. The system includes models of a point-source external PIM, a finite-length dipole antenna, a MIMO antenna array, and a baseband multicarrier 5G NR OFDM signal. The Channel coefficients method for multi-PIM-source compensation is replicated to verify the proposed external PIM modelling approach. Simulation results of artificially generated PIM cancellation show similar performance as real-life experiments. Therefore, the proposed approach allows testing PIM compensation algorithms on large systems with many antennas and arbitrary array structures. This eliminates the need for experiments with real hardware at the development stage of the PIM cancellation algorithm.
Multi-task Photonic Reservoir Computing: Wavelength Division Multiplexing for Parallel Computing with a Silicon Microring Resonator
Venue: arXiv cs.NE 2024
Year: 2024

Nowadays, as the ever-increasing demand for more powerful computing resources continues, alternative advanced computing paradigms are under extensive investigation. Significant effort has been made to deviate from conventional Von Neumann architectures. In-memory computing has emerged in the field of electronics as a possible solution to the infamous bottleneck between memory and computing processors, which reduces the effective throughput of data. In photonics, novel schemes attempt to collocate the computing processor and memory in a single device. Photonics offers the flexibility of multiplexing streams of data not only spatially and in time, but also in frequency or, equivalently, in wavelength, which makes it highly suitable for parallel computing. Here, we numerically show the use of time and wavelength division multiplexing (WDM) to solve four independent tasks at the same time in a single photonic chip, serving as a proof of concept for our proposal. The system is a time-delay reservoir computing (TDRC) based on a microring resonator (MRR). The addressed tasks cover different applications: Time-series prediction, waveform signal classification, wireless channel equalization, and radar signal prediction. The system is also tested for simultaneous computing of up to 10 instances of the same task, exhibiting excellent performance. The footprint of the system is reduced by using time-division multiplexing of the nodes that act as the neurons of the studied neural network scheme. WDM is used for the parallelization of wavelength channels, each addressing a single task. By adjusting the input power and frequency of each optical channel, we can achieve levels of performance for each of the tasks that are comparable to those quoted in state-of-the-art reports focusing on single-task operation...
LoRaWAN Based Dynamic Noise Mapping with Machine Learning for Urban Noise Enforcement
Venue: arXiv cs.AI 2024
Year: 2024

Static noise maps depicting long-term noise levels over wide areas are valuable urban planning assets for municipalities in decreasing noise exposure of residents. However, non-traffic noise sources with transient behavior, which people complain frequently, are usually ignored by static maps. We propose here a dynamic noise mapping approach using the data collected via low-power wide-area network (LPWAN, specifically LoRaWAN) based internet of things (IoT) infrastructure, which is one of the most common communication backbones for smart cities. Noise mapping based on LPWAN is challenging due to the low data rates of these protocols. The proposed dynamic noise mapping approach diminishes the negative implications of data rate limitations using machine learning (ML) for event and location prediction of non-traffic sources based on the scarce data. The strength of these models lies in their consideration of the spatial variance in acoustic behavior caused by the buildings in urban settings. The effectiveness of the proposed method and the accuracy of the resulting dynamic maps are evaluated in field tests. The results show that the proposed system can decrease the map error caused by non-traffic sources up to 51% and can stay effective under significant packet losses.
NeuroSEM: A hybrid framework for simulating multiphysics problems by coupling PINNs and spectral elements
Venue: arXiv cs.LG 2024
Year: 2024

Multiphysics problems that are characterized by complex interactions among fluid dynamics, heat transfer, structural mechanics, and electromagnetics, are inherently challenging due to their coupled nature. While experimental data on certain state variables may be available, integrating these data with numerical solvers remains a significant challenge. Physics-informed neural networks (PINNs) have shown promising results in various engineering disciplines, particularly in handling noisy data and solving inverse problems in partial differential equations (PDEs). However, their effectiveness in forecasting nonlinear phenomena in multiphysics regimes, particularly involving turbulence, is yet to be fully established. This study introduces NeuroSEM, a hybrid framework integrating PINNs with the high-fidelity Spectral Element Method (SEM) solver, Nektar++. NeuroSEM leverages the strengths of both PINNs and SEM, providing robust solutions for multiphysics problems. PINNs are trained to assimilate data and model physical phenomena in specific subdomains, which are then integrated into the Nektar++ solver. We demonstrate the efficiency and accuracy of NeuroSEM for thermal convection in cavity flow and flow past a cylinder. We applied NeuroSEM to the Rayleigh-Bénard convection system, including cases with missing thermal boundary conditions and noisy datasets, and to real particle image velocimetry (PIV) data to capture flow patterns characterized by horseshoe vortical structures. The framework's plug-and-play nature facilitates its extension to other multiphysics or multiscale problems. Furthermore, NeuroSEM is optimized for efficient execution on emerging integrated GPU-CPU architectures. This hybrid approach enhances the accuracy and efficiency of simulations, making it a powerful tool for tackling complex engineering challenges in various scientific domains.
Advancing Vietnamese Visual Question Answering with Transformer and Convolutional Integration
Venue: arXiv cs.CV 2024
Year: 2024

Visual Question Answering (VQA) has recently emerged as a potential research domain, captivating the interest of many in the field of artificial intelligence and computer vision. Despite the prevalence of approaches in English, there is a notable lack of systems specifically developed for certain languages, particularly Vietnamese. This study aims to bridge this gap by conducting comprehensive experiments on the Vietnamese Visual Question Answering (ViVQA) dataset, demonstrating the effectiveness of our proposed model. In response to community interest, we have developed a model that enhances image representation capabilities, thereby improving overall performance in the ViVQA system. Specifically, our model integrates the Bootstrapping Language-Image Pre-training with frozen unimodal models (BLIP-2) and the convolutional neural network EfficientNet to extract and process both local and global features from images. This integration leverages the strengths of transformer-based architectures for capturing comprehensive contextual information and convolutional networks for detailed local features. By freezing the parameters of these pre-trained models, we significantly reduce the computational cost and training time, while maintaining high performance. This approach significantly improves image representation and enhances the performance of existing VQA systems. We then leverage a multi-modal fusion module based on a general-purpose multi-modal foundation model (BEiT-3) to fuse the information between visual and textual features. Our experimental findings demonstrate that our model surpasses competing baselines, achieving promising performance. This is particularly evident in its accuracy of $71.04\%$ on the test set of the ViVQA dataset, marking a significant advancement in our research area. The code is available at https://github.com/nngocson2002/ViVQA.
Bug Analysis Towards Bug Resolution Time Prediction
Venue: arXiv cs.SE 2024
Year: 2024

Bugs are inevitable in software development, and their reporting in open repositories can enhance software transparency and reliability assessment. This study aims to extract information from the issue tracking system Jira and proposes a methodology to estimate resolution time for new bugs. The methodology is applied to network project ONAP, addressing concerns of network operators and manufacturers. This research provides insights into bug resolution times and related aspects in network softwarization projects.
Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens
Venue: arXiv cs.CL 2024
Year: 2024

While large language models (LLMs) are extensively used, there are raising concerns regarding privacy, security, and copyright due to their opaque training data, which brings the problem of detecting pre-training data on the table. Current solutions to this problem leverage techniques explored in machine learning privacy such as Membership Inference Attacks (MIAs), which heavily depend on LLMs' capability of verbatim memorization. However, this reliance presents challenges, especially given the vast amount of training data and the restricted number of effective training epochs. In this paper, we propose an adaptive pre-training data detection method which alleviates this reliance and effectively amplify the identification. Our method adaptively locates \textit{surprising tokens} of the input. A token is surprising to a LLM if the prediction on the token is "certain but wrong", which refers to low Shannon entropy of the probability distribution and low probability of the ground truth token at the same time. By using the prediction probability of surprising tokens to measure \textit{surprising}, the detection method is achieved based on the simple hypothesis that seeing seen data is less surprising for the model compared with seeing unseen data. The method can be applied without any access to the the pre-training data corpus or additional training like reference models. Our approach exhibits a consistent enhancement compared to existing methods in diverse experiments conducted on various benchmarks and models, achieving a maximum improvement of 29.5\%. We also introduce a new benchmark Dolma-Book developed upon a novel framework, which employs book data collected both before and after model training to provide further evaluation.