OpenRaiser/Intern-Atlas huggingface dataset:
PrIeD-KIE: Towards Privacy Preserved Document Key Information Extraction
Venue: arXiv cs.CL 2023
Year: 2023

In this paper, we introduce strategies for developing private Key Information Extraction (KIE) systems by leveraging large pretrained document foundation models in conjunction with differential privacy (DP), federated learning (FL), and Differentially Private Federated Learning (DP-FL). Through extensive experimentation on six benchmark datasets (FUNSD, CORD, SROIE, WildReceipts, XFUND, and DOCILE), we demonstrate that large document foundation models can be effectively fine-tuned for the KIE task under private settings to achieve adequate performance while maintaining strong privacy guarantees. Moreover, by thoroughly analyzing the impact of various training and model parameters on model performance, we propose simple yet effective guidelines for achieving an optimal privacy-utility trade-off for the KIE task under global DP. Finally, we introduce FeAm-DP, a novel DP-FL algorithm that enables efficiently upscaling global DP from a standalone context to a multi-client federated environment. We conduct a comprehensive evaluation of the algorithm across various client and privacy settings, and demonstrate its capability to achieve comparable performance and privacy guarantees to standalone DP, even when accommodating an increasing number of participating clients. Overall, our study offers valuable insights into the development of private KIE systems, and highlights the potential of document foundation models for privacy-preserved Document AI applications. To the best of authors' knowledge, this is the first work that explores privacy preserved document KIE using document foundation models.
HandMeThat: Human-Robot Communication in Physical and Social Environments
Venue: arXiv cs.AI 2023
Year: 2023

We introduce HandMeThat, a benchmark for a holistic evaluation of instruction understanding and following in physical and social environments. While previous datasets primarily focused on language grounding and planning, HandMeThat considers the resolution of human instructions with ambiguities based on the physical (object states and relations) and social (human actions and goals) information. HandMeThat contains 10,000 episodes of human-robot interactions. In each episode, the robot first observes a trajectory of human actions towards her internal goal. Next, the robot receives a human instruction and should take actions to accomplish the subgoal set through the instruction. In this paper, we present a textual interface for our benchmark, where the robot interacts with a virtual environment through textual commands. We evaluate several baseline models on HandMeThat, and show that both offline and online reinforcement learning algorithms perform poorly on HandMeThat, suggesting significant room for future work on physical and social human-robot communications and interactions.
Variational Barycentric Coordinates
Venue: arXiv cs.GR 2023
Year: 2023

We propose a variational technique to optimize for generalized barycentric coordinates that offers additional control compared to existing models. Prior work represents barycentric coordinates using meshes or closed-form formulae, in practice limiting the choice of objective function. In contrast, we directly parameterize the continuous function that maps any coordinate in a polytope's interior to its barycentric coordinates using a neural field. This formulation is enabled by our theoretical characterization of barycentric coordinates, which allows us to construct neural fields that parameterize the entire function class of valid coordinates. We demonstrate the flexibility of our model using a variety of objective functions, including multiple smoothness and deformation-aware energies; as a side contribution, we also present mathematically-justified means of measuring and minimizing objectives like total variation on discontinuous neural fields. We offer a practical acceleration strategy, present a thorough validation of our algorithm, and demonstrate several applications.
Frequency Domain Analysis of Nonlinear Series Elastic Actuator via Describing Function
Venue: arXiv cs.RO 2023
Year: 2023

Nonlinear stiffness SEAs (NSEAs) inspired by biological muscles offer promise in achieving adaptable stiffness for assistive robots. While assistive robots are often designed and compared based on torque capability and control bandwidth, NSEAs have not been systematically designed in the frequency domain due to their nonlinearity. The describing function, an analytical concept for nonlinear systems, offers a means to understand their behavior in the frequency domain. This paper introduces a frequency domain analysis of nonlinear series elastic actuators using the describing function method. This framework aims to equip researchers and engineers with tools for improved design and control in assistive robotics.
CrysFormer: Protein Structure Prediction via 3d Patterson Maps and Partial Structure Attention
Venue: arXiv cs.LG 2023
Year: 2023

Determining the structure of a protein has been a decades-long open question. A protein's three-dimensional structure often poses nontrivial computation costs, when classical simulation algorithms are utilized. Advances in the transformer neural network architecture -- such as AlphaFold2 -- achieve significant improvements for this problem, by learning from a large dataset of sequence information and corresponding protein structures. Yet, such methods only focus on sequence information; other available prior knowledge, such as protein crystallography and partial structure of amino acids, could be potentially utilized. To the best of our knowledge, we propose the first transformer-based model that directly utilizes protein crystallography and partial structure information to predict the electron density maps of proteins. Via two new datasets of peptide fragments (2-residue and 15-residue) , we demonstrate our method, dubbed \texttt{CrysFormer}, can achieve accurate predictions, based on a much smaller dataset size and with reduced computation costs.
Coloring Deep CNN Layers with Activation Hue Loss
Venue: arXiv cs.CV 2023
Year: 2023

This paper proposes a novel hue-like angular parameter to model the structure of deep convolutional neural network (CNN) activation space, referred to as the {\em activation hue}, for the purpose of regularizing models for more effective learning. The activation hue generalizes the notion of color hue angle in standard 3-channel RGB intensity space to $N$-channel activation space. A series of observations based on nearest neighbor indexing of activation vectors with pre-trained networks indicate that class-informative activations are concentrated about an angle $θ$ in both the $(x,y)$ image plane and in multi-channel activation space. A regularization term in the form of hue-like angular $θ$ labels is proposed to complement standard one-hot loss. Training from scratch using combined one-hot + activation hue loss improves classification performance modestly for a wide variety of classification tasks, including ImageNet.
Unsupervised SFQ-Based Spiking Neural Network
Venue: arXiv cs.ET 2023
Year: 2023

Single Flux Quantum (SFQ) technology represents a groundbreaking advancement in computational efficiency and ultra-high-speed neuromorphic processing. The key features of SFQ technology, particularly data representation, transmission, and processing through SFQ pulses, closely mirror fundamental aspects of biological neural structures. Consequently, SFQ-based circuits emerge as an ideal candidate for realizing Spiking Neural Networks (SNNs). This study presents a proof-of-concept demonstration of an SFQ-based SNN architecture, showcasing its capacity for ultra-fast switching at remarkably low energy consumption per output activity. Notably, our work introduces innovative approaches: (i) We introduce a novel spike-timing-dependent plasticity mechanism to update synapses and to trace spike-activity by incorporating a leaky non-destructive readout circuit. (ii) We propose a novel method to dynamically regulate the threshold behavior of leaky integrate and fire superconductor neurons, enhancing the adaptability of our SNN architecture. (iii) Our research incorporates a novel winner-take-all mechanism, aligning with practical strategies for SNN development and enabling effective decision-making processes. The effectiveness of these proposed structural enhancements is evaluated by integrating high-level models into the BindsNET framework. By leveraging BindsNET, we model the online training of an SNN, integrating the novel structures into the learning process. To ensure the robustness and functionality of our circuits, we employ JoSIM for circuit parameter extraction and functional verification through simulation.
Improving classifier decision boundaries using nearest neighbors
Venue: arXiv cs.LG 2023
Year: 2023

Neural networks are not learning optimal decision boundaries. We show that decision boundaries are situated in areas of low training data density. They are impacted by few training samples which can easily lead to overfitting. We provide a simple algorithm performing a weighted average of the prediction of a sample and its nearest neighbors' (computed in latent space) leading to a minor favorable outcomes for a variety of important measures for neural networks. In our evaluation, we employ various self-trained and pre-trained convolutional neural networks to show that our approach improves (i) resistance to label noise, (ii) robustness against adversarial attacks, (iii) classification accuracy, and to some degree even (iv) interpretability. While improvements are not necessarily large in all four areas, our approach is conceptually simple, i.e., improvements come without any modification to network architecture, training procedure or dataset. Furthermore, they are in stark contrast to prior works that often require trade-offs among the four objectives or provide valuable, but non-actionable insights.
GroundLink: A Dataset Unifying Human Body Movement and Ground Reaction Dynamics
Venue: arXiv cs.GR 2023
Year: 2023

The physical plausibility of human motions is vital to various applications in fields including but not limited to graphics, animation, robotics, vision, biomechanics, and sports science. While fully simulating human motions with physics is an extreme challenge, we hypothesize that we can treat this complexity as a black box in a data-driven manner if we focus on the ground contact, and have sufficient observations of physics and human activities in the real world. To prove our hypothesis, we present GroundLink, a unified dataset comprised of captured ground reaction force (GRF) and center of pressure (CoP) synchronized to standard kinematic motion captures. GRF and CoP of GroundLink are not simulated but captured at high temporal resolution using force platforms embedded in the ground for uncompromising measurement accuracy. This dataset contains 368 processed motion trials (~1.59M recorded frames) with 19 different movements including locomotion and weight-shifting actions such as tennis swings to signify the importance of capturing physics paired with kinematics. GroundLinkNet, our benchmark neural network model trained with GroundLink, supports our hypothesis by predicting GRFs and CoPs accurately and plausibly on unseen motions from various sources. The dataset, code, and benchmark models are made public for further research on various downstream tasks leveraging the rich physics information at https://csr.bu.edu/groundlink/.
Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinations
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) can generate fluent natural language texts when given relevant documents as background context. This ability has attracted considerable interest in developing industry applications of LLMs. However, LLMs are prone to generate hallucinations that are not supported by the provided sources. In this paper, we propose a hierarchical framework to detect and mitigate such ungrounded hallucination. Our framework uses Chain of Natural Language Inference (CoNLI) for hallucination detection and hallucination reduction via post-editing. Our approach achieves state-of-the-art performance on hallucination detection and enhances text quality through rewrite, using LLMs without any fine-tuning or domain-specific prompt engineering. We show that this simple plug-and-play framework can serve as an effective choice for hallucination detection and reduction, achieving competitive performance across various contexts.
Quantized Transformer Language Model Implementations on Edge Devices
Venue: arXiv cs.CL 2023
Year: 2023

Large-scale transformer-based models like the Bidirectional Encoder Representations from Transformers (BERT) are widely used for Natural Language Processing (NLP) applications, wherein these models are initially pre-trained with a large corpus with millions of parameters and then fine-tuned for a downstream NLP task. One of the major limitations of these large-scale models is that they cannot be deployed on resource-constrained devices due to their large model size and increased inference latency. In order to overcome these limitations, such large-scale models can be converted to an optimized FlatBuffer format, tailored for deployment on resource-constrained edge devices. Herein, we evaluate the performance of such FlatBuffer transformed MobileBERT models on three different edge devices, fine-tuned for Reputation analysis of English language tweets in the RepLab 2013 dataset. In addition, this study encompassed an evaluation of the deployed models, wherein their latency, performance, and resource efficiency were meticulously assessed. Our experiment results show that, compared to the original BERT large model, the converted and quantized MobileBERT models have 160$\times$ smaller footprints for a 4.1% drop in accuracy while analyzing at least one tweet per second on edge devices. Furthermore, our study highlights the privacy-preserving aspect of TinyML systems as all data is processed locally within a serverless environment.
Testing Higher-order Clusterability on graphs
Venue: arXiv cs.DS 2023
Year: 2023

Analysis of higher-order organizations, usually small connected subgraphs called motifs, is a fundamental task on complex networks. This paper studies a new problem of testing higher-order clusterability: given query access to an undirected graph, can we judge whether this graph can be partitioned into a few clusters of highly-connected motifs? This problem is an extension of the former work proposed by Czumaj et al. (STOC' 15), who recognized cluster structure on graphs using the framework of property testing. In this paper, a good graph cluster on high dimensions is first defined for higher-order clustering. Then, query lower bound is given for testing whether this kind of good cluster exists. Finally, an optimal sublinear-time algorithm is developed for testing clusterability based on triangles.
Bivariate Estimation-of-Distribution Algorithms Can Find an Exponential Number of Optima
Venue: arXiv cs.NE 2023
Year: 2023

Finding a large set of optima in a multimodal optimization landscape is a challenging task. Classical population-based evolutionary algorithms typically converge only to a single solution. While this can be counteracted by applying niching strategies, the number of optima is nonetheless trivially bounded by the population size. Estimation-of-distribution algorithms (EDAs) are an alternative, maintaining a probabilistic model of the solution space instead of a population. Such a model is able to implicitly represent a solution set far larger than any realistic population size. To support the study of how optimization algorithms handle large sets of optima, we propose the test function EqualBlocksOneMax (EBOM). It has an easy fitness landscape with exponentially many optima. We show that the bivariate EDA mutual-information-maximizing input clustering, without any problem-specific modification, quickly generates a model that behaves very similarly to a theoretically ideal model for EBOM, which samples each of the exponentially many optima with the same maximal probability. We also prove via mathematical means that no univariate model can come close to having this property: If the probability to sample an optimum is at least inverse-polynomial, there is a Hamming ball of logarithmic radius such that, with high probability, each sample is in this ball.
Game-Theoretic Analysis of (Non-)Refundable Fees in the Lightning Network
Venue: arXiv cs.GT 2023
Year: 2023

In PCNs, nodes that forward payments between a source and a receiver are paid a small fee if the payment is successful. The fee is a compensation for temporarily committing funds to the payment. However, payments may fail due to insufficient funds or attacks, often after considerable delays of up to several days, leaving a node without compensation. Furthermore, attackers can intentionally cause failed payments, e.g., to infer private information (like channel balances), without any cost in fees. In this paper, we first use extensive form games to formally characterize the conditions that lead to rational intermediaries refusing (or agreeing) to forward payments. A decision made by an intermediary to forward or not depends on the probability of failure, which they approximate based on past experience. We then propose and analyze an alternative fee model that allows the sender to determine and pay a fraction of the fee to intermediaries in a non refundable manner. A rational sender chooses the fraction such that the intermediaries' utility for forwarding the payment exceeds their utility for not forwarding. Our simulation study, based on real world Lightning snapshots, confirms that our novel mechanism can increase the probability of successful payments by 12 percent and decrease routing fees for senders by about 6 percent if all nodes behave rationally. Furthermore, previously cost free probing attacks now require that the attacker pays 1500 satoshis for every 1 million satoshis inferred. Finally, we propose a modification to the Hash Time Locked Contract to enable secure payments of the non refundable fees.
Deterministic Clustering in High Dimensional Spaces: Sketches and Approximation
Venue: arXiv cs.DS 2023
Year: 2023

In all state-of-the-art sketching and coreset techniques for clustering, as well as in the best known fixed-parameter tractable approximation algorithms, randomness plays a key role. For the classic $k$-median and $k$-means problems, there are no known deterministic dimensionality reduction procedure or coreset construction that avoid an exponential dependency on the input dimension $d$, the precision parameter $\varepsilon^{-1}$ or $k$. Furthermore, there is no coreset construction that succeeds with probability $1-1/n$ and whose size does not depend on the number of input points, $n$. This has led researchers in the area to ask what is the power of randomness for clustering sketches [Feldman, WIREs Data Mining Knowl. Discov'20]. Similarly, the best approximation ratio achievable deterministically without a complexity exponential in the dimension are $Ω(1)$ for both $k$-median and $k$-means, even when allowing a complexity FPT in the number of clusters $k$. This stands in sharp contrast with the $(1+\varepsilon)$-approximation achievable in that case, when allowing randomization. In this paper, we provide deterministic sketches constructions for clustering, whose size bounds are close to the best-known randomized ones. We also construct a deterministic algorithm for computing $(1+\varepsilon)$-approximation to $k$-median and $k$-means in high dimensional Euclidean spaces in time $2^{k^2/\varepsilon^{O(1)}} poly(nd)$, close to the best randomized complexity. Furthermore, our new insights on sketches also yield a randomized coreset construction that uses uniform sampling, that immediately improves over the recent results of [Braverman et al. FOCS '22] by a factor $k$.
Impact of Gender on the Evaluation of Security Decisions
Venue: arXiv cs.CY 2023
Year: 2023

Security decisions are made by human analysts under uncertain conditions which leaves room for bias judgement. However, little is known about how demographics like gender and education impact these judgments. We conducted an empirical study to investigate their influence on security decision evaluations, addressing this knowledge gap.
Maximizing Performance with Minimal Resources for Real-Time Transition Detection
Venue: arXiv cs.RO 2023
Year: 2023

Assistive devices, such as exoskeletons and prostheses, have revolutionized the field of rehabilitation and mobility assistance. Efficiently detecting transitions between different activities, such as walking, stair ascending and descending, and sitting, is crucial for ensuring adaptive control and enhancing user experience. We here present an approach for real-time transition detection, aimed at optimizing the processing-time performance. By establishing activity-specific threshold values through trained machine learning models, we effectively distinguish motion patterns and we identify transition moments between locomotion modes. This threshold-based method improves real-time embedded processing time performance by up to 11 times compared to machine learning approaches. The efficacy of the developed finite-state machine is validated using data collected from three different measurement systems. Moreover, experiments with healthy participants were conducted on an active pelvis orthosis to validate the robustness and reliability of our approach. The proposed algorithm achieved high accuracy in detecting transitions between activities. These promising results show the robustness and reliability of the method, reinforcing its potential for integration into practical applications.
Errata to: "Faster Deterministic Exponential Time Algorithm for Energy Games and Mean Payoff Games"
Venue: arXiv cs.DS 2023
Year: 2023

An improved exponential time algorithm for Energy Games and Mean Payoff Games has been recently proposed in ICALP 19. The new algorithm prevents some of the repetitive operations performed by the classic value iteration algorithm of Brim et al., leading to an approach with time complexity $O(\min(mnW,mn2^{n/2}\log W))$. Unfortunately, the pseudo-code of the algorithm includes inaccuracies that violate two Lemmata used in the correctness and complexity proofs. In this technical report, we describe the problems, propose a fixed version of the algorithm, and correct the proofs for the Lemmata.
Introducing the Attribution Stability Indicator: a Measure for Time Series XAI Attributions
Venue: arXiv cs.LG 2023
Year: 2023

Given the increasing amount and general complexity of time series data in domains such as finance, weather forecasting, and healthcare, there is a growing need for state-of-the-art performance models that can provide interpretable insights into underlying patterns and relationships. Attribution techniques enable the extraction of explanations from time series models to gain insights but are hard to evaluate for their robustness and trustworthiness. We propose the Attribution Stability Indicator (ASI), a measure to incorporate robustness and trustworthiness as properties of attribution techniques for time series into account. We extend a perturbation analysis with correlations of the original time series to the perturbed instance and the attributions to include wanted properties in the measure. We demonstrate the wanted properties based on an analysis of the attributions in a dimension-reduced space and the ASI scores distribution over three whole time series classification datasets.