OpenRaiser/Intern-Atlas huggingface dataset:
AutoML-based Almond Yield Prediction and Projection in California
Venue: arXiv cs.LG 2022
Year: 2022

Almonds are one of the most lucrative products of California, but are also among the most sensitive to climate change. In order to better understand the relationship between climatic factors and almond yield, an automated machine learning framework is used to build a collection of machine learning models. The prediction skill is assessed using historical records. Future projections are derived using 17 downscaled climate outputs. The ensemble mean projection displays almond yield changes under two different climate scenarios, along with two technology development scenarios, where the role of technology development is highlighted. The mean projections and distributions provide insightful results to stakeholders and can be utilized by policymakers for climate adaptation.
Understanding the Role of Mixup in Knowledge Distillation: An Empirical Study
Venue: arXiv cs.CV 2022
Year: 2022

Mixup is a popular data augmentation technique based on creating new samples by linear interpolation between two given data samples, to improve both the generalization and robustness of the trained model. Knowledge distillation (KD), on the other hand, is widely used for model compression and transfer learning, which involves using a larger network's implicit knowledge to guide the learning of a smaller network. At first glance, these two techniques seem very different, however, we found that "smoothness" is the connecting link between the two and is also a crucial attribute in understanding KD's interplay with mixup. Although many mixup variants and distillation methods have been proposed, much remains to be understood regarding the role of a mixup in knowledge distillation. In this paper, we present a detailed empirical study on various important dimensions of compatibility between mixup and knowledge distillation. We also scrutinize the behavior of the networks trained with a mixup in the light of knowledge distillation through extensive analysis, visualizations, and comprehensive experiments on image classification. Finally, based on our findings, we suggest improved strategies to guide the student network to enhance its effectiveness. Additionally, the findings of this study provide insightful suggestions to researchers and practitioners that commonly use techniques from KD. Our code is available at https://github.com/hchoi71/MIX-KD.
Significance-Based Categorical Data Clustering
Venue: arXiv cs.LG 2022
Year: 2022

Although numerous algorithms have been proposed to solve the categorical data clustering problem, how to access the statistical significance of a set of categorical clusters remains unaddressed. To fulfill this void, we employ the likelihood ratio test to derive a test statistic that can serve as a significance-based objective function in categorical data clustering. Consequently, a new clustering algorithm is proposed in which the significance-based objective function is optimized via a Monte Carlo search procedure. As a by-product, we can further calculate an empirical $p$-value to assess the statistical significance of a set of clusters and develop an improved gap statistic for estimating the cluster number. Extensive experimental studies suggest that our method is able to achieve comparable performance to state-of-the-art categorical data clustering algorithms. Moreover, the effectiveness of such a significance-based formulation on statistical cluster validation and cluster number estimation is demonstrated through comprehensive empirical results.
Parameter and Data Efficient Continual Pre-training for Robustness to Dialectal Variance in Arabic
Venue: arXiv cs.CL 2022
Year: 2022

The use of multilingual language models for tasks in low and high-resource languages has been a success story in deep learning. In recent times, Arabic has been receiving widespread attention on account of its dialectal variance. While prior research studies have tried to adapt these multilingual models for dialectal variants of Arabic, it still remains a challenging problem owing to the lack of sufficient monolingual dialectal data and parallel translation data of such dialectal variants. It remains an open problem on whether the limited dialectical data can be used to improve the models trained in Arabic on its dialectal variants. First, we show that multilingual-BERT (mBERT) incrementally pretrained on Arabic monolingual data takes less training time and yields comparable accuracy when compared to our custom monolingual Arabic model and beat existing models (by an avg metric of +$6.41$). We then explore two continual pre-training methods -- (1) using small amounts of dialectical data for continual finetuning and (2) parallel Arabic to English data and a Translation Language Modeling loss function. We show that both approaches help improve performance on dialectal classification tasks ($+4.64$ avg. gain) when used on monolingual models.
Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gaps
Venue: arXiv cs.CL 2022
Year: 2022

IR models using a pretrained language model significantly outperform lexical approaches like BM25. In particular, SPLADE, which encodes texts to sparse vectors, is an effective model for practical use because it shows robustness to out-of-domain datasets. However, SPLADE still struggles with exact matching of low-frequency words in training data. In addition, domain shifts in vocabulary and word frequencies deteriorate the IR performance of SPLADE. Because supervision data are scarce in the target domain, addressing the domain shifts without supervision data is necessary. This paper proposes an unsupervised domain adaptation method by filling vocabulary and word-frequency gaps. First, we expand a vocabulary and execute continual pretraining with a masked language model on a corpus of the target domain. Then, we multiply SPLADE-encoded sparse vectors by inverse document frequency weights to consider the importance of documents with lowfrequency words. We conducted experiments using our method on datasets with a large vocabulary gap from a source domain. We show that our method outperforms the present stateof-the-art domain adaptation method. In addition, our method achieves state-of-the-art results, combined with BM25.
Computing palindromes on a trie in linear time
Venue: arXiv cs.DS 2022
Year: 2022

A trie $\mathcal{T}$ is a rooted tree such that each edge is labeled by a single character from the alphabet, and the labels of out-going edges from the same node are mutually distinct. Given a trie $\mathcal{T}$ with $n$ edges, we show how to compute all distinct palindromes and all maximal palindromes on $\mathcal{T}$ in $O(n)$ time, in the case of integer alphabets of size polynomial in $n$. This improves the state-of-the-art $O(n \log h)$-time algorithms by Funakoshi et al. [PCS 2019], where $h$ is the height of $\mathcal{T}$. Using our new algorithms, the eertree with suffix links for a given trie $\mathcal{T}$ can readily be obtained in $O(n)$ time. Further, our trie-based $O(n)$-space data structure allows us to report all distinct palindromes and maximal palindromes in a query string represented in the trie $\mathcal{T}$, in output optimal time. This is an improvement over an existing (naïve) solution that precomputes and stores all distinct palindromes and maximal palindromes for each and every string in the trie $\mathcal{T}$ separately, using a total $O(n^2)$ preprocessing time and space, and reports them in output optimal time upon query.
Non-existence of a short algorithm for multiplication of $3\times3$ matrices with group $S_4\times S_3$, II
Venue: arXiv cs.CC 2022
Year: 2022

It is proved that there is no an algorithm for multiplication of $3\times3$ matrices of multiplicative length $\leq23$ that is invariant under a certain group isomorphic to $S_4\times S_3$. The proof makes use of description of the orbits of this group on decomposable tensors in the tensor cube $(M_3({\mathbb C}))^{\otimes3}$ which was obtained earlier.
Fast GPU-Based Two-Way Continuous Collision Handling
Venue: arXiv cs.GR 2022
Year: 2022

Step-and-project is a popular way to simulate non-penetrated deformable bodies in physically-based animation. First integrating the system in time regardless of contacts and post resolving potential intersections practically strike a good balance between plausibility and efficiency. However, existing methods could be defective and unsafe when the time step is large, taking risks of failures or demands of repetitive collision testing and resolving that severely degrade performance. In this paper, we propose a novel two-way method for fast and reliable continuous collision handling. Our method launches the optimization at both ends of the intermediate time-integrated state and the previous intersection-free state, progressively generating a piecewise-linear path and finally reaching a feasible solution for the next time step. Technically, our method interleaves between a forward step and a backward step at a low cost, until the result is conditionally converged. Due to a set of unified volume-based contact constraints, our method can flexibly and reliably handle a variety of codimensional deformable bodies, including volumetric bodies, cloth, hair and sand. The experiments show that our method is safe, robust, physically faithful and numerically efficient, especially suitable for large deformations or large time steps.
Many Destinations, Many Pathways: A Quantitative Analysis of Legitimate Peripheral Participation in Scratch
Venue: arXiv cs.HC 2022
Year: 2022

Although informal online learning communities have proliferated over the last two decades, a fundamental question remains: What are the users of these communities expected to learn? Guided by the work of Etienne Wenger on communities of practice, we identify three distinct types of learning goals common to online informal learning communities: the development of domain skills, the development of identity as a community member, and the development of community-specific values and practices. Given these goals, what is the best way to support learning? Drawing from previous research in social computing, we ask how different types of legitimate peripheral participation by newcomers-contribution to core tasks, engagement with practice proxies, social bonding, and feedback exchange-may be associated with these three learning goals. Using data from the Scratch online community, we conduct a quantitative analysis to explore these questions. Our study contributes both theoretical insights and empirical evidence on how different types of learning occur in informal online environments.
Factors that Contribute to the Success of a Software Organisation's DevOps Environment: A Systematic Review
Venue: arXiv cs.SE 2022
Year: 2022

This research assesses the aspects of software organizations' DevOps environments and identifies the factors contributing to these environments' success. DevOps is a recent concept, and many organizations are moving from old-style software development methods to agile approaches such as DevOps. However, there is no comprehensive information on what factors impact the success of the DevOps environment once organizations adopt it. This research focused on addressing this gap through a systematic literature review. The systematic review consisted of 33 articles from five selected search systems and databases from 2015 to 2021. Based on the included articles, 15 factors were identified and grouped into four categories: Collaborative Culture, Organizational Aspects, Tooling and Technology, and Continuous Practices. In addition, this research proposes a DevOps environment success factors model to potentially contribute to DevOps research and practice. Recommendations are made for additional research on the effectiveness of the proposed model and its success factors.
Spiking sampling network for image sparse representation and dynamic vision sensor data compression
Venue: arXiv cs.NE 2022
Year: 2022

Sparse representation has attracted great attention because it can greatly save storage resources and find representative features of data in a low-dimensional space. As a result, it may be widely applied in engineering domains including feature extraction, compressed sensing, signal denoising, picture clustering, and dictionary learning, just to name a few. In this paper, we propose a spiking sampling network. This network is composed of spiking neurons, and it can dynamically decide which pixel points should be retained and which ones need to be masked according to the input. Our experiments demonstrate that this approach enables better sparse representation of the original image and facilitates image reconstruction compared to random sampling. We thus use this approach for compressing massive data from the dynamic vision sensor, which greatly reduces the storage requirements for event data.
Coupled Modeling and Fusion Control for a Multi-modal Deformable Land-air Robot
Venue: arXiv cs.RO 2022
Year: 2022

A deformable land-air robot is introduced with excellent driving and flying capabilities, offering a smooth switching mechanism between the two modes. An elaborate coupled dynamics model is established for the robot, including rotors, chassis, suspension, and the deformable structure. In addition, a model-based controller is designed for landing and mode switching in various unstructured conditions, such as slopes and curved surface. And considering locomotion and complex near-ground situations to achieve cooperation between the two fused modalities. This system was simulated in ADAMS/Simulink and a tested with hardware-in-the-loop system was constructed for testing in various slopes. With a designed controller, the results showed the robot is capable of fast and smooth land-air switching, with a 24.6 % faster landing on slopes. The controller can also reduce landing offset and impact force more effectively than the normal control method at 32.7 % and 34.3 %, respectively.
Third-Party Aligner for Neural Word Alignments
Venue: arXiv cs.CL 2022
Year: 2022

Word alignment is to find translationally equivalent words between source and target sentences. Previous work has demonstrated that self-training can achieve competitive word alignment results. In this paper, we propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Specifically, source word and target word of each word pair aligned by the third-party aligner are trained to be close neighbors to each other in the contextualized embedding space when fine-tuning a pre-trained cross-lingual language model. Experiments on the benchmarks of various language pairs show that our approach can surprisingly do self-correction over the third-party supervision by finding more accurate word alignments and deleting wrong word alignments, leading to better performance than various third-party word aligners, including the currently best one. When we integrate all supervisions from various third-party aligners, we achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner. We released our code at https://github.com/sdongchuanqi/Third-Party-Supervised-Aligner.
Generative Adversarial Networks for anonymous Acneic face dataset generation
Venue: arXiv cs.CV 2022
Year: 2022

It is well known that the performance of any classification model is effective if the dataset used for the training process and the test process satisfy some specific requirements. In other words, the more the dataset size is large, balanced, and representative, the more one can trust the proposed model's effectiveness and, consequently, the obtained results. Unfortunately, large-size anonymous datasets are generally not publicly available in biomedical applications, especially those dealing with pathological human face images. This concern makes using deep-learning-based approaches challenging to deploy and difficult to reproduce or verify some published results. In this paper, we suggest an efficient method to generate a realistic anonymous synthetic dataset of human faces with the attributes of acne disorders corresponding to three levels of severity (i.e. Mild, Moderate and Severe). Therefore, a specific hierarchy StyleGAN-based algorithm trained at distinct levels is considered. To evaluate the performance of the proposed scheme, we consider a CNN-based classification system, trained using the generated synthetic acneic face images and tested using authentic face images. Consequently, we show that an accuracy of 97,6\% is achieved using InceptionResNetv2. As a result, this work allows the scientific community to employ the generated synthetic dataset for any data processing application without restrictions on legal or ethical concerns. Moreover, this approach can also be extended to other applications requiring the generation of synthetic medical images. We can make the code and the generated dataset accessible for the scientific community.
Nimbus: Toward Speed Up Function Signature Recovery via Input Resizing and Multi-Task Learning
Venue: arXiv cs.CR 2022
Year: 2022

Function signature recovery is important for many binary analysis tasks such as control-flow integrity enforcement, clone detection, and bug finding. Existing works try to substitute learning-based methods with rule-based methods to reduce human effort.They made considerable efforts to enhance the system's performance, which also bring the side effect of higher resource consumption. However, recovering the function signature is more about providing information for subsequent tasks, and both efficiency and performance are significant. In this paper, we first propose a method called Nimbus for efficient function signature recovery that furthest reduces the whole-process resource consumption without performance loss. Thanks to information bias and task relation (i.e., the relation between parameter count and parameter type recovery), we utilize selective inputs and introduce multi-task learning (MTL) structure for function signature recovery to reduce computational resource consumption, and fully leverage mutual information. Our experimental results show that, with only about the one-eighth processing time of the state-of-the-art method, we even achieve about 1% more prediction accuracy over all function signature recovery tasks.
On Multi-Robot Path Planning Based on Petri Net Models and LTL specifications
Venue: arXiv cs.RO 2022
Year: 2022

This work considers the path planning problem for a team of identical robots evolving in a known environment. The robots should satisfy a global specification given as a Linear Temporal Logic (LTL) formula over a set of regions of interest. The proposed method exploits the advantages of Petri net models for the team of robots and Büchi automata modeling the specification. The approach in this paper consists in combining the two models into one, denoted Composed Petri net and use it to find a sequence of action movements for the mobile robots, providing collision free trajectories to fulfill the specification. The solution results from a set of Mixed Integer Linear Programming (MILP) problems. The main advantage of the proposed solution is the completeness of the algorithm, meaning that a solution is found when exists, this representing the key difference with our previous work in [1]. The simulations illustrate comparison results between current and previous approaches, focusing on the computational complexity.
Self-conditioned Embedding Diffusion for Text Generation
Venue: arXiv cs.CL 2022
Year: 2022

Can continuous diffusion models bring the same performance breakthrough on natural language they did for image generation? To circumvent the discrete nature of text data, we can simply project tokens in a continuous space of embeddings, as is standard in language modeling. We propose Self-conditioned Embedding Diffusion, a continuous diffusion mechanism that operates on token embeddings and allows to learn flexible and scalable diffusion models for both conditional and unconditional text generation. Through qualitative and quantitative evaluation, we show that our text diffusion models generate samples comparable with those produced by standard autoregressive language models - while being in theory more efficient on accelerator hardware at inference time. Our work paves the way for scaling up diffusion models for text, similarly to autoregressive models, and for improving performance with recent refinements to continuous diffusion.
Two-stream Multi-dimensional Convolutional Network for Real-time Violence Detection
Venue: arXiv cs.CV 2022
Year: 2022

The increasing number of surveillance cameras and security concerns have made automatic violent activity detection from surveillance footage an active area for research. Modern deep learning methods have achieved good accuracy in violence detection and proved to be successful because of their applicability in intelligent surveillance systems. However, the models are computationally expensive and large in size because of their inefficient methods for feature extraction. This work presents a novel architecture for violence detection called Two-stream Multi-dimensional Convolutional Network (2s-MDCN), which uses RGB frames and optical flow to detect violence. Our proposed method extracts temporal and spatial information independently by 1D, 2D, and 3D convolutions. Despite combining multi-dimensional convolutional networks, our models are lightweight and efficient due to reduced channel capacity, yet they learn to extract meaningful spatial and temporal information. Additionally, combining RGB frames and optical flow yields 2.2% more accuracy than a single RGB stream. Regardless of having less complexity, our models obtained state-of-the-art accuracy of 89.7% on the largest violence detection benchmark dataset.
Note on generalized group testing
Venue: arXiv cs.IT 2022
Year: 2022

In this note, we present a new adaptive algorithm for generalized group testing, which is asymptotically optimal if $d=o(\log_2|E|)$, $E$ is a set of potentially contaminated sets, $d$ is a maximal size of elements of $E$. Also, we design a 3-stage algorithm, which is asymptotically optimal for $d=2$.
BER: Balanced Error Rate For Speaker Diarization
Venue: arXiv cs.SD 2022
Year: 2022

DER is the primary metric to evaluate diarization performance while facing a dilemma: the errors in short utterances or segments tend to be overwhelmed by longer ones. Short segments, e.g., `yes' or `no,' still have semantic information. Besides, DER overlooks errors in less-talked speakers. Although JER balances speaker errors, it still suffers from the same dilemma. Considering all those aspects, duration error, segment error, and speaker-weighted error constituting a complete diarization evaluation, we propose a Balanced Error Rate (BER) to evaluate speaker diarization. First, we propose a segment-level error rate (SER) via connected sub-graphs and adaptive IoU threshold to get accurate segment matching. Second, to evaluate diarization in a unified way, we adopt a speaker-specific harmonic mean between duration and segment, followed by a speaker-weighted average. Third, we analyze our metric via the modularized system, EEND, and the multi-modal method on real datasets. SER and BER are publicly available at https://github.com/X-LANCE/BER.
An Ensemble-based approach for assigning text to correct Harmonized system code
Venue: arXiv cs.AI 2022
Year: 2022

Industries must follow government rules and regulations around the world to classify products when assessing duties and taxes for international shipment. Harmonized System (HS) is the most standardized numerical method of classifying traded products among industry classification systems. A hierarchical ensemble model comprising of Bert-transformer, NER, distance-based approaches, and knowledge-graphs have been developed to address scalability, coverage, ability to capture nuances, automation and auditing requirements when classifying unknown text-descriptions as per HS method.