OpenRaiser/Intern-Atlas huggingface dataset:
Edge-aware Consistent Stereo Video Depth Estimation
Venue: arXiv cs.CV 2023
Year: 2023

Video depth estimation is crucial in various applications, such as scene reconstruction and augmented reality. In contrast to the naive method of estimating depths from images, a more sophisticated approach uses temporal information, thereby eliminating flickering and geometrical inconsistencies. We propose a consistent method for dense video depth estimation; however, unlike the existing monocular methods, ours relates to stereo videos. This technique overcomes the limitations arising from the monocular input. As a benefit of using stereo inputs, a left-right consistency loss is introduced to improve the performance. Besides, we use SLAM-based camera pose estimation in the process. To address the problem of depth blurriness during test-time training (TTT), we present an edge-preserving loss function that improves the visibility of fine details while preserving geometrical consistency. We show that our edge-aware stereo video model can accurately estimate the dense depth maps.
LatentAugment: Dynamically Optimized Latent Probabilities of Data Augmentation
Venue: arXiv cs.CV 2023
Year: 2023

Although data augmentation is a powerful technique for improving the performance of image classification tasks, it is difficult to identify the best augmentation policy. The optimal augmentation policy, which is the latent variable, cannot be directly observed. To address this problem, this study proposes $\textit{LatentAugment}$, which estimates the latent probability of optimal augmentation. The proposed method is appealing in that it can dynamically optimize the augmentation strategies for each input and model parameter in learning iterations. Theoretical analysis shows that LatentAugment is a general model that includes other augmentation methods as special cases, and it is simple and computationally efficient in comparison with existing augmentation methods. Experimental results show that the proposed LatentAugment has higher test accuracy than previous augmentation methods on the CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets.
Caption Anything: Interactive Image Description with Diverse Multimodal Controls
Venue: arXiv cs.CV 2023
Year: 2023

Controllable image captioning is an emerging multimodal topic that aims to describe the image with natural language following human purpose, $\textit{e.g.}$, looking at the specified regions or telling in a particular text style. State-of-the-art methods are trained on annotated pairs of input controls and output captions. However, the scarcity of such well-annotated multimodal data largely limits their usability and scalability for interactive AI systems. Leveraging unimodal instruction-following foundation models is a promising alternative that benefits from broader sources of data. In this paper, we present Caption AnyThing (CAT), a foundation model augmented image captioning framework supporting a wide range of multimodel controls: 1) visual controls, including points, boxes, and trajectories; 2) language controls, such as sentiment, length, language, and factuality. Powered by Segment Anything Model (SAM) and ChatGPT, we unify the visual and language prompts into a modularized framework, enabling the flexible combination between different controls. Extensive case studies demonstrate the user intention alignment capabilities of our framework, shedding light on effective user interaction modeling in vision-language applications. Our code is publicly available at https://github.com/ttengwang/Caption-Anything.
In-situ Anomaly Detection in Additive Manufacturing with Graph Neural Networks
Venue: arXiv cs.CV 2023
Year: 2023

Transforming a design into a high-quality product is a challenge in metal additive manufacturing due to rare events which can cause defects to form. Detecting these events in-situ could, however, reduce inspection costs, enable corrective action, and is the first step towards a future of tailored material properties. In this study a model is trained on laser input information to predict nominal laser melting conditions. An anomaly score is then calculated by taking the difference between the predictions and new observations. The model is evaluated on a dataset with known defects achieving an F1 score of 0.821. This study shows that anomaly detection methods are an important tool in developing robust defect detection methods.
DECICE: Device-Edge-Cloud Intelligent Collaboration Framework
Venue: arXiv cs.DC 2023
Year: 2023

DECICE is a Horizon Europe project that is developing an AI-enabled open and portable management framework for automatic and adaptive optimization and deployment of applications in computing continuum encompassing from IoT sensors on the Edge to large-scale Cloud / HPC computing infrastructures. In this paper, we describe the DECICE framework and architecture. Furthermore, we highlight use-cases for framework evaluation: intelligent traffic intersection, magnetic resonance imaging, and emergency response.
Avatar Knowledge Distillation: Self-ensemble Teacher Paradigm with Uncertainty
Venue: arXiv cs.CV 2023
Year: 2023

Knowledge distillation is an effective paradigm for boosting the performance of pocket-size model, especially when multiple teacher models are available, the student would break the upper limit again. However, it is not economical to train diverse teacher models for the disposable distillation. In this paper, we introduce a new concept dubbed Avatars for distillation, which are the inference ensemble models derived from the teacher. Concretely, (1) For each iteration of distillation training, various Avatars are generated by a perturbation transformation. We validate that Avatars own higher upper limit of working capacity and teaching ability, aiding the student model in learning diverse and receptive knowledge perspectives from the teacher model. (2) During the distillation, we propose an uncertainty-aware factor from the variance of statistical differences between the vanilla teacher and Avatars, to adjust Avatars' contribution on knowledge transfer adaptively. Avatar Knowledge Distillation AKD is fundamentally different from existing methods and refines with the innovative view of unequal training. Comprehensive experiments demonstrate the effectiveness of our Avatars mechanism, which polishes up the state-of-the-art distillation methods for dense prediction without more extra computational cost. The AKD brings at most 0.7 AP gains on COCO 2017 for Object Detection and 1.83 mIoU gains on Cityscapes for Semantic Segmentation, respectively. Code is available at https://github.com/Gumpest/AvatarKD.
Family Theories in Child-Robot Interactions: Understanding Families as a Whole for Child-Robot Interaction Design
Venue: arXiv cs.HC 2023
Year: 2023

In this work, we discuss a theoretically motivated family-centered design approach for child-robot interactions, adapted by Family Systems Theory (FST) and Family Ecological Model (FEM). Long-term engagement and acceptance of robots in the home is influenced by factors that surround the child and the family, such as child-sibling-parent relationships and family routines, rituals, and values. A family-centered approach to interaction design is essential when developing in-home technology for children, especially for social agents like robots with which they can form connections and relationships. We review related literature in family theories and connect it with child-robot interaction and child-computer interaction research. We present two case studies that exemplify how family theories, FST and FEM, can inform the integration of robots into homes, particularly research into child-robot and family-robot interaction. Finally, we pose five overarching recommendations for a family-centered design approach in child-robot interactions.
Cellular blood flow modelling with HemoCell
Venue: arXiv cs.CE 2023
Year: 2023

Many of the intriguing properties of blood originate from its cellular nature. Bulk effects, such as viscosity, depend on the local shear rates and on the size of the vessels. While empirical descriptions of bulk rheology are available for decades, their validity is limited to the experimental conditions they were observed under. These are typically artificial scenarios (e.g., perfectly straight glass tube or in pure shear with no gradients). Such conditions make experimental measurements simpler, however, they do not exist in real systems (i.e., in a real human circulatory system). Therefore, as we strive to increase our understanding on the cardiovascular system and improve the accuracy of our computational predictions, we need to incorporate a more comprehensive description of the cellular nature of blood. This, however, presents several computational challenges that can only be addressed by high performance computing. In this chapter we describe HemoCell , an open-source high performance cellular blood flow simulation, which implements validated mechanical models for red blood cells and is capable of reproducing the emergent transport characteristics of such a complex cellular system. We discuss the accuracy, the range of validity, and demonstrate applications on a series of human diseases.
BranchNorm: Robustly Scaling Extremely Deep Transformers
Venue: arXiv cs.LG 2023
Year: 2023

Recently, DeepNorm scales Transformers into extremely deep (i.e., 1000 layers) and reveals the promising potential of deep scaling. To stabilize the training of deep models, DeepNorm (Wang et al., 2022) attempts to constrain the model update to a constant value. Although applying such a constraint can benefit the early stage of model training, it may lead to undertrained models during the whole training procedure. In this paper, we propose BranchNorm, which dynamically rescales the non-residual branch of Transformer in accordance with the training period. BranchNorm not only theoretically stabilizes the training with smooth gradient norms at the early stage, but also encourages better convergence in the subsequent training stage. Experiment results on multiple translation tasks demonstrate that BranchNorm achieves a better trade-off between training stability and converge performance.
Local Computation Algorithms for Hypergraph Coloring -- following Beck's approach (full version)
Venue: arXiv cs.DS 2023
Year: 2023

We investigate local computation algorithms (LCA) for two-coloring of $k$-uniform hypergraphs. We focus on hypergraph instances that satisfy strengthened assumption of the Lovász Local Lemma of the form $2^{1-αk} (Δ+1) \mathrm{e} < 1$, where $Δ$ is the bound on the maximum edge degree. The main question which arises here is for how large $α$ there exists an LCA that is able to properly color such hypergraphs in polylogarithmic time per query. We describe briefly how upgrading the classical sequential procedure of Beck from 1991 with Moser and Tardos' RESAMPLE yields polylogarithmic LCA that works for $α$ up to $1/4$. Then, we present an improved procedure that solves wider range of instances by allowing $α$ up to $1/3$.
The AI generation gap: Are Gen Z students more interested in adopting generative AI such as ChatGPT in teaching and learning than their Gen X and Millennial Generation teachers?
Venue: arXiv cs.CY 2023
Year: 2023

This study aimed to explore the experiences, perceptions, knowledge, concerns, and intentions of Gen Z students with Gen X and Gen Y teachers regarding the use of generative AI (GenAI) in higher education. A sample of students and teachers were recruited to investigate the above using a survey consisting of both open and closed questions. The findings showed that Gen Z participants were generally optimistic about the potential benefits of GenAI, including enhanced productivity, efficiency, and personalized learning, and expressed intentions to use GenAI for various educational purposes. Gen X and Gen Y teachers acknowledged the potential benefits of GenAI but expressed heightened concerns about overreliance, ethical and pedagogical implications, emphasizing the need for proper guidelines and policies to ensure responsible use of the technology. The study highlighted the importance of combining technology with traditional teaching methods to provide a more effective learning experience. Implications of the findings include the need to develop evidence-based guidelines and policies for GenAI integration, foster critical thinking and digital literacy skills among students, and promote responsible use of GenAI technologies in higher education.
Simple Noisy Environment Augmentation for Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

Data augmentation is a widely used technique for improving model performance in machine learning, particularly in computer vision and natural language processing. Recently, there has been increasing interest in applying augmentation techniques to reinforcement learning (RL) problems, with a focus on image-based augmentation. In this paper, we explore a set of generic wrappers designed to augment RL environments with noise and encourage agent exploration and improve training data diversity which are applicable to a broad spectrum of RL algorithms and environments. Specifically, we concentrate on augmentations concerning states, rewards, and transition dynamics and introduce two novel augmentation techniques. In addition, we introduce a noise rate hyperparameter for control over the frequency of noise injection. We present experimental results on the impact of these wrappers on return using three popular RL algorithms, Soft Actor-Critic (SAC), Twin Delayed DDPG (TD3), and Proximal Policy Optimization (PPO), across five MuJoCo environments. To support the choice of augmentation technique in practice, we also present analysis that explores the performance these techniques across environments. Lastly, we publish the wrappers in our noisyenv repository for use with gym environments.
SlipCover: Near Zero-Overhead Code Coverage for Python
Venue: arXiv cs.SE 2023
Year: 2023

Coverage analysis is widely used but can suffer from high overhead. This overhead is especially acute in the context of Python, which is already notoriously slow (a recent study observes a roughly 30x slowdown vs. native code). We find that the state-of-the-art coverage tool for Python, coverage$.$py, introduces a median overhead of 180% with the standard Python interpreter. Slowdowns are even more extreme when using PyPy, a JIT-compiled Python implementation, with coverage$.$py imposing a median overhead of 1,300%. This performance degradation reduces the utility of coverage analysis in most use cases, including testing and fuzzing, and precludes its use in deployment. This paper presents SlipCover, a novel, near-zero overhead coverage analyzer for Python. SlipCover works without modifications to either the Python interpreter or PyPy. It first processes a program's AST to accurately identify all branches and lines. SlipCover then dynamically rewrites Python bytecodes to add lightweight instrumentation to each identified branch and line. At run time, SlipCover periodically de-instruments already-covered lines and branches. The result is extremely low overheads -- a median of just 5% -- making SlipCover suitable for use in deployment. We show its efficiency can translate to significant increases in the speed of coverage-based clients. As a proof of concept, we integrate SlipCover into TPBT, a targeted property-based testing system, and observe a 22x speedup.
What Else Can Voronoi Diagrams Do For Diameter In Planar Graphs?
Venue: arXiv cs.DS 2023
Year: 2023

The Voronoi diagrams technique was introduced by Cabello to compute the diameter of planar graphs in subquadratic time. We present novel applications of this technique in static, fault-tolerant, and partially-dynamic undirected unweighted planar graphs, as well as some new limitations. 1. In the static case, we give $n^{3+o(1)}/D^2$ and $\tilde{O}(n\cdot D^2)$ time algorithms for computing the diameter of a planar graph $G$ with diameter $D$. These are faster than the state of the art $\tilde{O}(n^{5/3})$ when $D<n^{1/3}$ or $D>n^{2/3}$. 2. In the fault-tolerant setting, we give an $n^{7/3+o(1)}$ time algorithm for computing the diameter of $G\setminus \{e\}$ for every edge $e$ in $G$ the replacement diameter problem. Compared to the naive $\tilde{O}(n^{8/3})$ time algorithm that runs the static algorithm for every edge. 3. In the incremental setting, where we wish to maintain the diameter while while adding edges, we present an algorithm with total running time $n^{7/3+o(1)}$. Compared to the naive $\tilde{O}(n^{8/3})$ time algorithm that runs the static algorithm after every update. 4. We give a lower bound (conditioned on the SETH) ruling out an amortized $O(n^{1-\varepsilon})$ update time for maintaining the diameter in *weighted* planar graph. The lower bound holds even for incremental or decremental updates. Our upper bounds are obtained by novel uses and manipulations of Voronoi diagrams. These include maintaining the Voronoi diagram when edges of the graph are deleted, allowing the sites of the Voronoi diagram to lie on a BFS tree level (rather than on boundaries of $r$-division), and a new reduction from incremental diameter to incremental distance oracles that could be of interest beyond planar graphs. Our lower bound is the first lower bound for a dynamic planar graph problem that is conditioned on the SETH.
Majorizing Measures, Codes, and Information
Venue: arXiv cs.IT 2023
Year: 2023

The majorizing measure theorem of Fernique and Talagrand is a fundamental result in the theory of random processes. It relates the boundedness of random processes indexed by elements of a metric space to complexity measures arising from certain multiscale combinatorial structures, such as packing and covering trees. This paper builds on the ideas first outlined in a little-noticed preprint of Andreas Maurer to present an information-theoretic perspective on the majorizing measure theorem, according to which the boundedness of random processes is phrased in terms of the existence of efficient variable-length codes for the elements of the indexing metric space.
Preliminary results of a therapeutic lab for promoting autonomies in autistic children
Venue: arXiv cs.RO 2023
Year: 2023

This extended abtract describes the preliminary qualitative results coming from a therapeutic laboratory focused on the use of the Pepper robot to promote autonomies and functional acquisitions in highly functioning (Asperger) children with autism. The field lab, ideated and led by a multidisciplinary team, involved 4 children, aged 11-13, who attended the laboratory sessions once a week for four months.
Notes on Refactoring Exponential Macros in Common Lisp
Venue: arXiv cs.PL 2023
Year: 2023

I recently consulted for a very big Common Lisp project having more than one million lines of code (including comments). Let's call it "System X" in the following. System X suffered from extremely long compilation times; i.e., a full recompile took about 33:17 minutes on a 3.1 GHz MacBook Pro Intel Core i7 with SSD and 16 GBs of RAM, using ACL 10.1. It turns out that a number of macros were causing an exponential code blowup. With these macros refactored, the system then recompiled in 5:30 minutes - a speedup by a factor of ~ 6. In this experience report, I will first illuminate the problem, and then demonstrate two potential solutions in terms of macro refactoring techniques. These techniques can be applied in related scenarios.
Ranking and unranking bordered and unbordered words
Venue: arXiv cs.DS 2023
Year: 2023

A \emph{border} of a word $w$ is a word that is both a non-empty proper prefix and suffix of $w$. If $w$ has a border, then it is said to be \emph{bordered}; otherwise, it is said to be \emph{unbordered}. The main results of this paper are the first algorithms to rank and unrank length-$n$ bordered and unbordered words over a $k$-letter alphabet. We show that, under the unit-cost RAM model, ranking bordered and unbordered words can be done in $O(kn^3)$ time using $O(n)$ space, and unranking them can be done in $O(n^4k\log k)$ time using $O(n)$ space.
Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large Language Models
Venue: arXiv cs.CL 2023
Year: 2023

This project focuses on enhancing open-source large language models through instruction-tuning and providing comprehensive evaluations of their performance. We explore how various training data factors, such as quantity, quality, and linguistic distribution, influence the performance of instruction-tuned models trained on publicly accessible high-quality instruction datasets for both English and Chinese languages. Our goal is to supplement evaluation with quantitative analyses, providing valuable insights for the continued advancement of open-source chat models. Our model, data, and code are publicly available for others to use and build upon.
Are VAEs Bad at Reconstructing Molecular Graphs?
Venue: arXiv cs.LG 2023
Year: 2023

Many contemporary generative models of molecules are variational auto-encoders of molecular graphs. One term in their training loss pertains to reconstructing the input, yet reconstruction capabilities of state-of-the-art models have not yet been thoroughly compared on a large and chemically diverse dataset. In this work, we show that when several state-of-the-art generative models are evaluated under the same conditions, their reconstruction accuracy is surprisingly low, worse than what was previously reported on seemingly harder datasets. However, we show that improving reconstruction does not directly lead to better sampling or optimization performance. Failed reconstructions from the MoLeR model are usually similar to the inputs, assembling the same motifs in a different way, and possess similar chemical properties such as solubility. Finally, we show that the input molecule and its failed reconstruction are usually mapped by the different encoders to statistically distinguishable posterior distributions, hinting that posterior collapse may not fully explain why VAEs are bad at reconstructing molecular graphs.
Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization
Venue: arXiv cs.CV 2023
Year: 2023

There is a growing demand for the accessible creation of high-quality 3D avatars that are animatable and customizable. Although 3D morphable models provide intuitive control for editing and animation, and robustness for single-view face reconstruction, they cannot easily capture geometric and appearance details. Methods based on neural implicit representations, such as signed distance functions (SDF) or neural radiance fields, approach photo-realism, but are difficult to animate and do not generalize well to unseen data. To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing. Trained from a collection of high-quality 3D scans, our face model is parameterized by geometry, expression, and texture latent codes with a learned SDF and explicit UV texture parameterization. Once trained, we can reconstruct an avatar from a single in-the-wild image by leveraging the learned prior to project the image into the latent space of our model. Our implicit morphable face models can be used to render an avatar from novel views, animate facial expressions by modifying expression codes, and edit textures by directly painting on the learned UV-texture maps. We demonstrate quantitatively and qualitatively that our method improves upon photo-realism, geometry, and expression accuracy compared to state-of-the-art methods.