OpenRaiser/Intern-Atlas huggingface dataset:
Sorting Out Quantum Monte Carlo
Venue: arXiv cs.LG 2023
Year: 2023

Molecular modeling at the quantum level requires choosing a parameterization of the wavefunction that both respects the required particle symmetries, and is scalable to systems of many particles. For the simulation of fermions, valid parameterizations must be antisymmetric with respect to the exchange of particles. Typically, antisymmetry is enforced by leveraging the anti-symmetry of determinants with respect to the exchange of matrix rows, but this involves computing a full determinant each time the wavefunction is evaluated. Instead, we introduce a new antisymmetrization layer derived from sorting, the $\textit{sortlet}$, which scales as $O(N \log N)$ with regards to the number of particles -- in contrast to $O(N^3)$ for the determinant. We show numerically that applying this anti-symmeterization layer on top of an attention based neural-network backbone yields a flexible wavefunction parameterization capable of reaching chemical accuracy when approximating the ground state of first-row atoms and small molecules.
3D-QAE: Fully Quantum Auto-Encoding of 3D Point Clouds
Venue: arXiv cs.CV 2023
Year: 2023

Existing methods for learning 3D representations are deep neural networks trained and tested on classical hardware. Quantum machine learning architectures, despite their theoretically predicted advantages in terms of speed and the representational capacity, have so far not been considered for this problem nor for tasks involving 3D data in general. This paper thus introduces the first quantum auto-encoder for 3D point clouds. Our 3D-QAE approach is fully quantum, i.e. all its data processing components are designed for quantum hardware. It is trained on collections of 3D point clouds to produce their compressed representations. Along with finding a suitable architecture, the core challenges in designing such a fully quantum model include 3D data normalisation and parameter optimisation, and we propose solutions for both these tasks. Experiments on simulated gate-based quantum hardware demonstrate that our method outperforms simple classical baselines, paving the way for a new research direction in 3D computer vision. The source code is available at https://4dqv.mpi-inf.mpg.de/QAE3D/.
Understanding the Difference between Office Presence and Co-presence in Team Member Interactions
Venue: arXiv cs.CY 2023
Year: 2023

Although the public health emergency related to the coronavirus disease 2019 (COVID-19) pandemic has officially ended, many software developers still work partly from home. Agile teams that coordinate their office time foster a sense of unity, collaboration, and cohesion among team members. In contrast, teams with limited co-presence may experience challenges in establishing psychological safety and developing a cohesive and inclusive team culture, potentially hindering effective communication, knowledge sharing, and trust building. Therefore, the effect of agile team members not being co-located daily must be investigated. We explore the co-presence patterns of 17 agile teams in a large agile telecommunications company whose employees work partly from home. Based on office access card data, we found significant variation in co-presence practices. Some teams exhibited a coordinated approach, ensuring team members are simultaneously present at the office. However, other teams demonstrated fragmented co-presence, with only small subgroups of members meeting in person and the remainder rarely interacting with their team members face-to-face. Thus, high average office presence in the team does not necessarily imply that team members meet often in person at the office. In contrast, non-coordinated teams may have both high average office presence and low frequency of in-person interactions among the members. Our results suggest that the promotion of mere office presence without coordinated co-presence is based on a false assumption that good average attendance levels guarantee frequent personal interactions. These findings carry important implications for research on long-term team dynamics and practice.
A Design and Development of Rubrics System for Android Applications
Venue: arXiv cs.CY 2023
Year: 2023

Online grading systems have become extremely prevalent as majority of academic materials are in the process of being digitized, if not already done. In this paper, we present the concept of design and implementation of a mobile application for "Student Evaluation System", envisaged with the purpose of making the task of evaluation of students performance by faculty and graders facile. This application aims to provide an user-friendly interface for viewing the students performance and has several functions which extends the Rubrics with graphical analysis of students assignments. Rubrics evaluation system is the widespread practice in both the software industry and the educational institutes. Our application promises to make the grading system easier and to enhance the effectiveness in terms of time and resources. This application also allows the user/grader to keep track of submissions and the evaluated data in a form that can be easily accessed and statistically analysed in a consistent manner.
Fundamentally New Coupled Approach to Contact Mechanics via the Dirichlet-Neumann Schwarz Alternating Method
Venue: arXiv cs.CE 2023
Year: 2023

Contact phenomena are essential in understanding the behavior of mechanical systems. Existing computational approaches for simulating mechanical contact often encounter numerical issues, such as inaccurate physical predictions, energy conservation errors, and unwanted oscillations. We introduce an alternative technique, rooted in the non-overlapping Schwarz alternating method, originally developed for domain decomposition. In multi-body contact scenarios, this method treats each body as a separate, non-overlapping domain and prevents interpenetration using an alternating Dirichlet-Neumann iterative process. This approach has a strong theoretical foundation, eliminates the need for contact constraints, and offers flexibility, making it well-suited for multiscale and multi-physics applications. We conducted a numerical comparison between the Schwarz method and traditional methods like Lagrange multiplier and penalty methods, focusing on a benchmark impact problem. Our results indicate that the Schwarz alternating method surpasses traditional methods in several key areas: it provides more accurate predictions for various measurable quantities and demonstrates exceptional energy conservation capabilities. To address the issue of unwanted oscillations in contact velocities and forces, we explored various algorithms and stabilization techniques, ultimately opting for the naive-stabilized Newmark scheme for its simplicity and effectiveness. Furthermore, we validated the efficiency of the Schwarz method in a three-dimensional impact problem, highlighting its innate capacity to accommodate different mesh topologies, time integration schemes, and time steps for each interacting body.
The Paradox of Noise: An Empirical Study of Noise-Infusion Mechanisms to Improve Generalization, Stability, and Privacy in Federated Learning
Venue: arXiv cs.LG 2023
Year: 2023

In a data-centric era, concerns regarding privacy and ethical data handling grow as machine learning relies more on personal information. This empirical study investigates the privacy, generalization, and stability of deep learning models in the presence of additive noise in federated learning frameworks. Our main objective is to provide strategies to measure the generalization, stability, and privacy-preserving capabilities of these models and further improve them. To this end, five noise infusion mechanisms at varying noise levels within centralized and federated learning settings are explored. As model complexity is a key component of the generalization and stability of deep learning models during training and evaluation, a comparative analysis of three Convolutional Neural Network (CNN) architectures is provided. The paper introduces Signal-to-Noise Ratio (SNR) as a quantitative measure of the trade-off between privacy and training accuracy of noise-infused models, aiming to find the noise level that yields optimal privacy and accuracy. Moreover, the Price of Stability and Price of Anarchy are defined in the context of privacy-preserving deep learning, contributing to the systematic investigation of the noise infusion strategies to enhance privacy without compromising performance. Our research sheds light on the delicate balance between these critical factors, fostering a deeper understanding of the implications of noise-based regularization in machine learning. By leveraging noise as a tool for regularization and privacy enhancement, we aim to contribute to the development of robust, privacy-aware algorithms, ensuring that AI-driven solutions prioritize both utility and privacy.
Robust Constant-Time Cryptography
Venue: arXiv cs.CR 2023
Year: 2023

The constant-time property is considered the security standard for cryptographic code. Code following the constant-time discipline is free from secret-dependent branches and memory accesses, and thus avoids leaking secrets through cache and timing side-channels. The constant-time property makes a number of implicit assumptions that are fundamentally at odds with the reality of cryptographic code. Constant-time is not robust. The first issue with constant-time is that it is a whole-program property: It relies on the entirety of the code base being constant-time. But, cryptographic developers do not generally write whole programs; rather, they provide libraries and specific algorithms for other application developers to use. As such, developers of security libraries must maintain their security guarantees even when their code is operating within (potentially untrusted) application contexts. Constant-time requires memory safety. The whole-program nature of constant-time also leads to a second issue: constant-time requires memory safety of all the running code. Any memory safety bugs, whether in the library or the application, will wend their way back to side-channel leaks of secrets if not direct disclosure. And although cryptographic libraries should (and are) written to be memory-safe, it is unfortunately unrealistic to expect the same from every application that uses each library. We formalize robust constant-time and build a RobustIsoCrypt compiler that transforms the library code and protects the secrets even when they are linked with untrusted code. Our evaluation with SUPERCOP benchmarking framework shows that the performance overhead is less than five percent on average.
Towards Interpretable Motion-level Skill Assessment in Robotic Surgery
Venue: arXiv cs.RO 2023
Year: 2023

Purpose: We study the relationship between surgical gestures and motion primitives in dry-lab surgical exercises towards a deeper understanding of surgical activity at fine-grained levels and interpretable feedback in skill assessment. Methods: We analyze the motion primitive sequences of gestures in the JIGSAWS dataset and identify inverse motion primitives in those sequences. Inverse motion primitives are defined as sequential actions on the same object by the same tool that effectively negate each other. We also examine the correlation between surgical skills (measured by GRS scores) and the number and total durations of inverse motion primitives in the dry-lab trials of Suturing, Needle Passing, and Knot Tying tasks. Results: We find that the sequence of motion primitives used to perform gestures can help detect labeling errors in surgical gestures. Inverse motion primitives are often used as recovery actions to correct the position or orientation of objects or may be indicative of other issues such as with depth perception. The number and total durations of inverse motion primitives in trials are also strongly correlated with lower GRS scores in the Suturing and Knot Tying tasks. Conclusion: The sequence and pattern of motion primitives could be used to provide interpretable feedback in surgical skill assessment. Combined with an action recognition model, the explainability of automated skill assessment can be improved by showing video clips of the inverse motion primitives of inefficient or problematic movements.
Tamil-Llama: A New Tamil Language Model Based on Llama 2
Venue: arXiv cs.CL 2023
Year: 2023

Language modeling has witnessed remarkable advancements in recent years, with Large Language Models (LLMs) like ChatGPT setting unparalleled benchmarks in human-like text generation. However, a prevailing limitation is the underrepresentation of languages like Tamil in these cutting-edge models, leading to suboptimal performance in diverse linguistic contexts. This paper addresses this lacuna, enhancing the open-source LLaMA model with an addition of 16,000 Tamil tokens, aiming to achieve superior text generation and comprehension in the Tamil language. We strategically employ the LoRA methodology for efficient model training on a comprehensive Tamil corpus, ensuring computational feasibility and model robustness. Moreover, we introduce a Tamil-translated version of the Alpaca dataset and a subset of the OpenOrca dataset tailored for instruction fine-tuning. Our results showcase significant performance improvements in Tamil text generation, with potential implications for the broader landscape of LLMs in Indian languages. We further underscore our commitment to open research by making our models, datasets, and code publicly accessible, fostering further innovations in language modeling.
Central Angle Optimization for 360-degree Holographic 3D Content
Venue: arXiv cs.CV 2023
Year: 2023

In this study, we propose a method to find an optimal central angle in deep learning-based depth map estimation used to produce realistic holographic content. The acquisition of RGB-depth map images as detailed as possible must be performed to generate holograms of high quality, despite the high computational cost. Therefore, we introduce a novel pipeline designed to analyze various values of central angles between adjacent camera viewpoints equidistant from the origin of an object-centered environment. Then we propose the optimal central angle to generate high-quality holographic content. The proposed pipeline comprises key steps such as comparing estimated depth maps and comparing reconstructed CGHs (Computer-Generated Holograms) from RGB images and estimated depth maps. We experimentally demonstrate and discuss the relationship between the central angle and the quality of digital holographic content.
A Practical Guide to Implementing Off-Axis Stereo Projection Using Existing Ray Tracing Libraries
Venue: arXiv cs.GR 2023
Year: 2023

Virtual reality (VR) renderers driving CAVEs and similar immersive environments use the off-axis stereo camera model so that a tracked user can move freely in front of the projection plane. Geometrically, off-axis projection results in asymmetric viewing frusta and generalizes the ubiquitous perspective camera model to support positioning off the center of the projection plane. VR renderers often integrate with larger visualization systems that rely on libraries for position tracking and pose estimates, for ray tracing-based rendering, and for user interaction. We demonstrate different strategies to implement off-axis stereo projection within the constraints of given VR applications and ray tracing libraries. We aim for minimal to no adjustments required to the internal camera representation of such libraries. We include host and shader code with the article that can be directly integrated in custom applications.
Low-Multi-Rank High-Order Bayesian Robust Tensor Factorization
Venue: arXiv cs.LG 2023
Year: 2023

The recently proposed tensor robust principal component analysis (TRPCA) methods based on tensor singular value decomposition (t-SVD) have achieved numerous successes in many fields. However, most of these methods are only applicable to third-order tensors, whereas the data obtained in practice are often of higher order, such as fourth-order color videos, fourth-order hyperspectral videos, and fifth-order light-field images. Additionally, in the t-SVD framework, the multi-rank of a tensor can describe more fine-grained low-rank structure in the tensor compared with the tubal rank. However, determining the multi-rank of a tensor is a much more difficult problem than determining the tubal rank. Moreover, most of the existing TRPCA methods do not explicitly model the noises except the sparse noise, which may compromise the accuracy of estimating the low-rank tensor. In this work, we propose a novel high-order TRPCA method, named as Low-Multi-rank High-order Bayesian Robust Tensor Factorization (LMH-BRTF), within the Bayesian framework. Specifically, we decompose the observed corrupted tensor into three parts, i.e., the low-rank component, the sparse component, and the noise component. By constructing a low-rank model for the low-rank component based on the order-$d$ t-SVD and introducing a proper prior for the model, LMH-BRTF can automatically determine the tensor multi-rank. Meanwhile, benefiting from the explicit modeling of both the sparse and noise components, the proposed method can leverage information from the noises more effectivly, leading to an improved performance of TRPCA. Then, an efficient variational inference algorithm is established for parameters estimation. Empirical studies on synthetic and real-world datasets demonstrate the effectiveness of the proposed method in terms of both qualitative and quantitative results.
BPCoach: Exploring Hero Drafting in Professional MOBA Tournaments via Visual Analytics
Venue: arXiv cs.HC 2023
Year: 2023

Hero drafting for multiplayer online arena (MOBA) games is crucial because drafting directly affects the outcome of a match. Both sides take turns to "ban"/"pick" a hero from a roster of approximately 100 heroes to assemble their drafting. In professional tournaments, the process becomes more complex as teams are not allowed to pick heroes used in the previous rounds with the "best-of-N" rule. Additionally, human factors including the team's familiarity with drafting and play styles are overlooked by previous studies. Meanwhile, the huge impact of patch iteration on drafting strengths in the professional tournament is of concern. To this end, we propose a visual analytics system, BPCoach, to facilitate hero drafting planning by comparing various drafting through recommendations and predictions and distilling relevant human and in-game factors. Two case studies, expert feedback, and a user study suggest that BPCoach helps determine hero drafting in a rounded and efficient manner.
Aggregation Weighting of Federated Learning via Generalization Bound Estimation
Venue: arXiv cs.LG 2023
Year: 2023

Federated Learning (FL) typically aggregates client model parameters using a weighting approach determined by sample proportions. However, this naive weighting method may lead to unfairness and degradation in model performance due to statistical heterogeneity and the inclusion of noisy data among clients. Theoretically, distributional robustness analysis has shown that the generalization performance of a learning model with respect to any shifted distribution is bounded. This motivates us to reconsider the weighting approach in federated learning. In this paper, we replace the aforementioned weighting method with a new strategy that considers the generalization bounds of each local model. Specifically, we estimate the upper and lower bounds of the second-order origin moment of the shifted distribution for the current local model, and then use these bounds disagreements as the aggregation proportions for weightings in each communication round. Experiments demonstrate that the proposed weighting strategy significantly improves the performance of several representative FL algorithms on benchmark datasets.
KRATT: QBF-Assisted Removal and Structural Analysis Attack Against Logic Locking
Venue: arXiv cs.CR 2023
Year: 2023

This paper introduces KRATT, a removal and structural analysis attack against state-of-the-art logic locking techniques, such as single and double flip locking techniques (SFLTs and DFLTs). KRATT utilizes powerful quantified Boolean formulas (QBFs), which have not found widespread use in hardware security, to find the secret key of SFLTs for the first time. It can handle locked circuits under both oracle-less (OL) and oracle-guided (OG) threat models. It modifies the locked circuit and uses a prominent OL attack to make a strong guess under the OL threat model. It uses a structural analysis technique to identify promising protected input patterns and explores them using the oracle under the OG model. Experimental results on ISCAS'85, ITC'99, and HeLLO: CTF'22 benchmarks show that KRATT can break SFLTs using a QBF formulation in less than a minute, can decipher a large number of key inputs of SFLTs and DFLTs with high accuracy under the OL threat model, and can easily find the secret key of DFLTs under the OG threat model. It is shown that KRATT outperforms publicly available OL and OG attacks in terms of solution quality and run-time.
JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models
Venue: arXiv cs.AI 2023
Year: 2023

Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. JARVIS-1 is the existing most general agent in Minecraft, capable of completing over 200 different tasks using control and observation space similar to humans. These tasks range from short-horizon tasks, e.g., "chopping trees" to long-horizon tasks, e.g., "obtaining a diamond pickaxe". JARVIS-1 performs exceptionally well in short-horizon tasks, achieving nearly perfect performance. In the classic long-term task of $\texttt{ObtainDiamondPickaxe}$, JARVIS-1 surpasses the reliability of current state-of-the-art agents by 5 times and can successfully complete longer-horizon and more challenging tasks. The project page is available at https://craftjarvis.org/JARVIS-1
2D Image head pose estimation via latent space regression under occlusion settings
Venue: arXiv cs.CV 2023
Year: 2023

Head orientation is a challenging Computer Vision problem that has been extensively researched having a wide variety of applications. However, current state-of-the-art systems still underperform in the presence of occlusions and are unreliable for many task applications in such scenarios. This work proposes a novel deep learning approach for the problem of head pose estimation under occlusions. The strategy is based on latent space regression as a fundamental key to better structure the problem for occluded scenarios. Our model surpasses several state-of-the-art methodologies for occluded HPE, and achieves similar accuracy for non-occluded scenarios. We demonstrate the usefulness of the proposed approach with: (i) two synthetically occluded versions of the BIWI and AFLW2000 datasets, (ii) real-life occlusions of the Pandora dataset, and (iii) a real-life application to human-robot interaction scenarios where face occlusions often occur. Specifically, the autonomous feeding from a robotic arm.