OpenRaiser/Intern-Atlas huggingface dataset:
Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputs
Venue: arXiv cs.LG 2023
Year: 2023

Deep learning often faces the challenge of efficiently processing dynamic inputs, such as sensor data or user inputs. For example, an AI writing assistant is required to update its suggestions in real time as a document is edited. Re-running the model each time is expensive, even with compression techniques like knowledge distillation, pruning, or quantization. Instead, we take an incremental computing approach, looking to reuse calculations as the inputs change. However, the dense connectivity of conventional architectures poses a major obstacle to incremental computation, as even minor input changes cascade through the network and restrict information reuse. To address this, we use vector quantization to discretize intermediate values in the network, which filters out noisy and unnecessary modifications to hidden neurons, facilitating the reuse of their values. We apply this approach to the transformers architecture, creating an efficient incremental inference algorithm with complexity proportional to the fraction of the modified inputs. Our experiments with adapting the OPT-125M pre-trained language model demonstrate comparable accuracy on document classification while requiring 12.1X (median) fewer operations for processing sequences of atomic edits.
FLiCR: A Fast and Lightweight LiDAR Point Cloud Compression Based on Lossy RI
Venue: arXiv cs.MM 2023
Year: 2023

Light detection and ranging (LiDAR) sensors are becoming available on modern mobile devices and provide a 3D sensing capability. This new capability is beneficial for perceptions in various use cases, but it is challenging for resource-constrained mobile devices to use the perceptions in real-time because of their high computational complexity. In this context, edge computing can be used to enable LiDAR online perceptions, but offloading the perceptions on the edge server requires a low-latency, lightweight, and efficient compression due to the large volume of LiDAR point clouds data. This paper presents FLiCR, a fast and lightweight LiDAR point cloud compression method for enabling edge-assisted online perceptions. FLiCR is based on range images (RI) as an intermediate representation (IR), and dictionary coding for compressing RIs. FLiCR achieves its benefits by leveraging lossy RIs, and we show the efficiency of bytestream compression is largely improved with quantization and subsampling. In addition, we identify the limitation of current quality metrics for presenting the entropy of a point cloud, and introduce a new metric that reflects both point-wise and entropy-wise qualities for lossy IRs. The evaluation results show FLiCR is more suitable for edge-assisted real-time perceptions than the existing LiDAR compressions, and we demonstrate the effectiveness of our compression and metric with the evaluations on 3D object detection and LiDAR SLAM.
Adaptive Segmentation Network for Scene Text Detection
Venue: arXiv cs.CV 2023
Year: 2023

Inspired by deep convolution segmentation algorithms, scene text detectors break the performance ceiling of datasets steadily. However, these methods often encounter threshold selection bottlenecks and have poor performance on text instances with extreme aspect ratios. In this paper, we propose to automatically learn the discriminate segmentation threshold, which distinguishes text pixels from background pixels for segmentation-based scene text detectors and then further reduces the time-consuming manual parameter adjustment. Besides, we design a Global-information Enhanced Feature Pyramid Network (GE-FPN) for capturing text instances with macro size and extreme aspect ratios. Following the GE-FPN, we introduce a cascade optimization structure to further refine the text instances. Finally, together with the proposed threshold learning strategy and text detection structure, we design an Adaptive Segmentation Network (ASNet) for scene text detection. Extensive experiments are carried out to demonstrate that the proposed ASNet can achieve the state-of-the-art performance on four text detection benchmarks, i.e., ICDAR 2015, MSRA-TD500, ICDAR 2017 MLT and CTW1500. The ablation experiments also verify the effectiveness of our contributions.
3-Coloring $C_4$ or $C_3$-free Diameter Two Graphs
Venue: arXiv cs.DS 2023
Year: 2023

The question of whether 3-Coloring can be solved in polynomial-time for the diameter two graphs is a well-known open problem in the area of algorithmic graph theory. We study the problem restricted to graph classes that avoid cycles of given lengths as induced subgraphs. Martin et. al. [CIAC 2021] showed that the problem is polynomial-time solvable for $C_5$-free or $C_6$-free graphs, and, $(C_4,C_s)$-free graphs where $s \in \{3,7,8,9\}$. We extend their result proving that it is polynomial-time solvable for $(C_4,C_s)$-free graphs, for any constant $s$, and for $(C_3,C_7)$-free graphs. Our results also hold for the more general problem List 3-Colouring.
Writer adaptation for offline text recognition: An exploration of neural network-based methods
Venue: arXiv cs.CV 2023
Year: 2023

Handwriting recognition has seen significant success with the use of deep learning. However, a persistent shortcoming of neural networks is that they are not well-equipped to deal with shifting data distributions. In the field of handwritten text recognition (HTR), this shows itself in poor recognition accuracy for writers that are not similar to those seen during training. An ideal HTR model should be adaptive to new writing styles in order to handle the vast amount of possible writing styles. In this paper, we explore how HTR models can be made writer adaptive by using only a handful of examples from a new writer (e.g., 16 examples) for adaptation. Two HTR architectures are used as base models, using a ResNet backbone along with either an LSTM or Transformer sequence decoder. Using these base models, two methods are considered to make them writer adaptive: 1) model-agnostic meta-learning (MAML), an algorithm commonly used for tasks such as few-shot classification, and 2) writer codes, an idea originating from automatic speech recognition. Results show that an HTR-specific version of MAML known as MetaHTR improves performance compared to the baseline with a 1.4 to 2.0 improvement in word error rate (WER). The improvement due to writer adaptation is between 0.2 and 0.7 WER, where a deeper model seems to lend itself better to adaptation using MetaHTR than a shallower model. However, applying MetaHTR to larger HTR models or sentence-level HTR may become prohibitive due to its high computational and memory requirements. Lastly, writer codes based on learned features or Hinge statistical features did not lead to improved recognition performance.
End-to-end Remote Sensing Change Detection of Unregistered Bi-temporal Images for Natural Disasters
Venue: arXiv cs.CV 2023
Year: 2023

Change detection based on remote sensing images has been a prominent area of interest in the field of remote sensing. Deep networks have demonstrated significant success in detecting changes in bi-temporal remote sensing images and have found applications in various fields. Given the degradation of natural environments and the frequent occurrence of natural disasters, accurately and swiftly identifying damaged buildings in disaster-stricken areas through remote sensing images holds immense significance. This paper aims to investigate change detection specifically for natural disasters. Considering that existing public datasets used in change detection research are registered, which does not align with the practical scenario where bi-temporal images are not matched, this paper introduces an unregistered end-to-end change detection synthetic dataset called xBD-E2ECD. Furthermore, we propose an end-to-end change detection network named E2ECDNet, which takes an unregistered bi-temporal image pair as input and simultaneously generates the flow field prediction result and the change detection prediction result. It is worth noting that our E2ECDNet also supports change detection for registered image pairs, as registration can be seen as a special case of non-registration. Additionally, this paper redefines the criteria for correctly predicting a positive case and introduces neighborhood-based change detection evaluation metrics. The experimental results have demonstrated significant improvements.
Taken By Surprise? Evaluating how Bayesian Weighting Influences Peoples' Takeaways in Map Visualizations
Venue: arXiv cs.HC 2023
Year: 2023

Choropleth maps have been studied and extended in many ways to counteract the many biases that can occur when using them. Two recent techniques, Surprise metrics and Value Suppressing Uncertainty Palettes (VSUPs), offer promising solutions but have yet to be tested empirically with users of visualizations. In this paper, we explore how well people can make use of these techniques in map exploration tasks. We report a crowdsourced experiment where n = 300 participants are assigned to one of Choropleth, Surprise (only), and VSUP conditions (depicting rates and Surprise in a suppressed palette). Results show clear differences in map analysis outcomes, e.g. with Surprise maps leading people to significantly higher areas of population, or VSUPs performing similar or better than Choropleths for rate selection. Qualitative analysis suggests that many participants may only consider a subset of the metrics presented to them during exploration and decision-making. We discuss how these results generally support the use of Surprise and VSUP techniques in practice, and opportunities for further technique development. The material for the study (data, study results and code) is publicly available on https://osf.io/exb95/.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Venue: arXiv cs.AI 2023
Year: 2023

Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals. RLHF has emerged as the central method used to finetune state-of-the-art large language models (LLMs). Despite this popularity, there has been relatively little public work systematizing its flaws. In this paper, we (1) survey open problems and fundamental limitations of RLHF and related methods; (2) overview techniques to understand, improve, and complement RLHF in practice; and (3) propose auditing and disclosure standards to improve societal oversight of RLHF systems. Our work emphasizes the limitations of RLHF and highlights the importance of a multi-faceted approach to the development of safer AI systems.
A deep transfer learning network for structural condition identification with limited real-world training data
Venue: arXiv cs.CE 2023
Year: 2023

Structural condition identification based on monitoring data is important for automatic civil infrastructure asset management. Nevertheless, the monitoring data is almost always insufficient, because the real-time monitoring data of a structure only reflects a limited number of structural conditions, while the number of possible structural conditions is infinite. With insufficient monitoring data, the identification performance may significantly degrade. This study aims to tackle this challenge by proposing a deep transfer learning (TL) approach for structural condition identification. It effectively integrates physics-based and data-driven methods, by generating various training data based on the calibrated finite element (FE) model, pretraining a deep learning (DL) network, and transferring its embedded knowledge to the real monitoring/testing domain. Its performance is demonstrated in a challenging case, vibration-based condition identification of steel frame structures with bolted connection damage. The results show that even though the training data are from a different domain and with different types of labels, intrinsic physics can be learned through the pretraining process, and the TL results can be clearly improved, with the identification accuracy increasing from 81.8% to 89.1%. The comparative studies show that SHMnet with three convolutional layers stands out as the pretraining DL architecture, with 21.8% and 25.5% higher identification accuracy values over the other two networks, VGGnet-16 and ResNet-18. The findings of this study advance the potential application of the proposed approach towards expert-level condition identification based on limited real-world training data.
Reducing Latency of DAG-based Consensus in the Asynchronous Setting via the UTXO Model
Venue: arXiv cs.DC 2023
Year: 2023

DAG-based consensus has attracted significant interest due to its high throughput in asynchronous network settings. However, existing protocols such as DAG-rider (Keidar et al., PODC 2021) and ``Narwhal and Tusk'' (Danezis et al., Eurosys 2022) face two undesired practical issues: (1) high transaction latency and (2) high cost to verify transaction outcomes. To address (1), this work introduces a novel commit rule based on the Unspent Transaction Output (UTXO) Data Model, which allows a node to predict the transaction results before triggering the commitment. We propose a new consensus algorithm named ``Board and Clerk'', which reduces the transaction latency by half for roughly 50% of transactions. As the tolerance for faults escalates, more transactions can partake in this latency reduction. In addition, we also propose the Hyper-Block Model with two flexible proposing strategies to tackle (2): blocking and non-blocking. Using our proposed strategies, each node first predicts the transaction results if its proposal is committed and packs this result as a commitment in its proposal. The hyper-block packs the signature of the proposal and the outputs of the consensus layer together in order to prove the transaction results.
AC-Norm: Effective Tuning for Medical Image Analysis via Affine Collaborative Normalization
Venue: arXiv cs.CV 2023
Year: 2023

Driven by the latest trend towards self-supervised learning (SSL), the paradigm of "pretraining-then-finetuning" has been extensively explored to enhance the performance of clinical applications with limited annotations. Previous literature on model finetuning has mainly focused on regularization terms and specific policy models, while the misalignment of channels between source and target models has not received sufficient attention. In this work, we revisited the dynamics of batch normalization (BN) layers and observed that the trainable affine parameters of BN serve as sensitive indicators of domain information. Therefore, Affine Collaborative Normalization (AC-Norm) is proposed for finetuning, which dynamically recalibrates the channels in the target model according to the cross-domain channel-wise correlations without adding extra parameters. Based on a single-step backpropagation, AC-Norm can also be utilized to measure the transferability of pretrained models. We evaluated AC-Norm against the vanilla finetuning and state-of-the-art fine-tuning methods on transferring diverse pretrained models to the diabetic retinopathy grade classification, retinal vessel segmentation, CT lung nodule segmentation/classification, CT liver-tumor segmentation and MRI cardiac segmentation tasks. Extensive experiments demonstrate that AC-Norm unanimously outperforms the vanilla finetuning by up to 4% improvement, even under significant domain shifts where the state-of-the-art methods bring no gains. We also prove the capability of AC-Norm in fast transferability estimation. Our code is available at https://github.com/EndoluminalSurgicalVision-IMR/ACNorm.
ChatHome: Development and Evaluation of a Domain-Specific Language Model for Home Renovation
Venue: arXiv cs.CL 2023
Year: 2023

This paper presents the development and evaluation of ChatHome, a domain-specific language model (DSLM) designed for the intricate field of home renovation. Considering the proven competencies of large language models (LLMs) like GPT-4 and the escalating fascination with home renovation, this study endeavors to reconcile these aspects by generating a dedicated model that can yield high-fidelity, precise outputs relevant to the home renovation arena. ChatHome's novelty rests on its methodology, fusing domain-adaptive pretraining and instruction-tuning over an extensive dataset. This dataset includes professional articles, standard documents, and web content pertinent to home renovation. This dual-pronged strategy is designed to ensure that our model can assimilate comprehensive domain knowledge and effectively address user inquiries. Via thorough experimentation on diverse datasets, both universal and domain-specific, including the freshly introduced "EvalHome" domain dataset, we substantiate that ChatHome not only amplifies domain-specific functionalities but also preserves its versatility.
Mixbiotic society measures: Comparison of organizational structures based on communication simulation
Venue: arXiv cs.SI 2023
Year: 2023

The philosophical world has proposed the concept of "mixbiotic society," in which individuals with freedom and diverse values mix and mingle to recognize their respective "fundamental incapability" each other and sublimate into solidarity, toward solving the issues of social isolation and fragmentation. Based on this concept, the mixbiotic society measures have been proposed to evaluate dynamic communication patterns with reference to classification in cellular automata and particle reaction-diffusion that simulate living phenomena. In this paper, we applied these measures to five typologies of organizational structure (Red: impulsive, Amber: adaptive, Orange: achievement, Green: pluralistic, and Teal: evolutionary) and evaluated their features. Specifically, we formed star, tree, tree+jumpers, tree+more jumpers, and small-world type networks corresponding to each of five typologies, conducted communication simulations on these networks, and calculated values for mixbiotic society measures. The results showed that Teal organization has the highest value of the mixism measure among mixbiotic society measures, i.e., it balances similarity (mixing) and dissimilarity (mingling) in communication, and is living and mixbiotic between order and chaos. Measures other than mixism showed that in Teal organization, information is not concentrated in a central leader and that communication takes place among various members. This evaluation of organizational structures shows that the mixbiotic society measures is also useful for assessing organizational change. In the future, these measures will be used not only in business organizations, but also in digital democratic organizations and platform cooperatives in conjunction with information technology.
Toward Transparent Sequence Models with Model-Based Tree Markov Model
Venue: arXiv cs.LG 2023
Year: 2023

In this study, we address the interpretability issue in complex, black-box Machine Learning models applied to sequence data. We introduce the Model-Based tree Hidden Semi-Markov Model (MOB-HSMM), an inherently interpretable model aimed at detecting high mortality risk events and discovering hidden patterns associated with the mortality risk in Intensive Care Units (ICU). This model leverages knowledge distilled from Deep Neural Networks (DNN) to enhance predictive performance while offering clear explanations. Our experimental results indicate the improved performance of Model-Based trees (MOB trees) via employing LSTM for learning sequential patterns, which are then transferred to MOB trees. Integrating MOB trees with the Hidden Semi-Markov Model (HSMM) in the MOB-HSMM enables uncovering potential and explainable sequences using available information.
AffineGlue: Joint Matching and Robust Estimation
Venue: arXiv cs.CV 2023
Year: 2023

We propose AffineGlue, a method for joint two-view feature matching and robust estimation that reduces the combinatorial complexity of the problem by employing single-point minimal solvers. AffineGlue selects potential matches from one-to-many correspondences to estimate minimal models. Guided matching is then used to find matches consistent with the model, suffering less from the ambiguities of one-to-one matches. Moreover, we derive a new minimal solver for homography estimation, requiring only a single affine correspondence (AC) and a gravity prior. Furthermore, we train a neural network to reject ACs that are unlikely to lead to a good model. AffineGlue is superior to the SOTA on real-world datasets, even when assuming that the gravity direction points downwards. On PhotoTourism, the AUC@10° score is improved by 6.6 points compared to the SOTA. On ScanNet, AffineGlue makes SuperPoint and SuperGlue achieve similar accuracy as the detector-free LoFTR.
Optimising Highly-Parallel Simulation-Based Verification of Cyber-Physical Systems
Venue: arXiv cs.SE 2023
Year: 2023

Cyber-Physical Systems (CPSs), comprising both software and physical components, arise in many industry-relevant domains and are often mission- or safety-critical. System-Level Verification (SLV) of CPSs aims at certifying that given (e.g., safety or liveness) specifications are met, or at estimating the value of some KPIs, when the system runs in its operational environment, i.e., in presence of inputs (from users or other systems) and/or of additional, uncontrolled disturbances. To enable SLV of complex systems from the early design phases, the currently most adopted approach envisions the simulation of a system model under the (time bounded) operational scenarios of interest. Simulation-based SLV can be computationally prohibitive (years of sequential simulation), since model simulation is computationally intensive and the set of scenarios of interest can huge. We present a technique that, given a collection of scenarios of interest (extracted from mass-storage databases or from symbolic structures, e.g., constraint-based scenario generators), computes parallel shortest simulation campaigns, which drive a possibly large number of system model simulators running in parallel in a HPC infrastructure through all (and only) those scenarios in the user-defined (possibly random) order, by wisely avoiding multiple simulations of repeated trajectories, thus minimising the overall completion time, compatibly with the available simulator memory capacity. Our experiments on Modelica/FMU and Simulink case study models with up to ~200 million scenarios show that our optimisation yields speedups as high as 8x. This, together with the enabled massive parallelisation, makes practically viable (a few weeks in a HPC infrastructure) verification tasks (both statistical and exhaustive, with respect to the given set of scenarios) which would otherwise take inconceivably long time.
Towards a Fully Unsupervised Framework for Intent Induction in Customer Support Dialogues
Venue: arXiv cs.CL 2023
Year: 2023

State of the art models in intent induction require annotated datasets. However, annotating dialogues is time-consuming, laborious and expensive. In this work, we propose a completely unsupervised framework for intent induction within a dialogue. In addition, we show how pre-processing the dialogue corpora can improve results. Finally, we show how to extract the dialogue flows of intentions by investigating the most common sequences. Although we test our work in the MultiWOZ dataset, the fact that this framework requires no prior knowledge make it applicable to any possible use case, making it very relevant to real world customer support applications across industry.