OpenRaiser/Intern-Atlas huggingface dataset:
Generating Finite Element Codes combining Adaptive Octrees with Complex Geometries
Venue: arXiv cs.CE 2023
Year: 2023

We present a high-level domain-specific language (DSL) interface to drive an adaptive incomplete $k$-d tree-based framework for finite element (FEM) solutions to PDEs. This DSL provides three key advances: (a) it abstracts out the complexity of implementing non-trivial FEM formulations, (b) it simplifies deploying these formulations on arbitrarily complicated and adaptively refined meshes, and (c) it exhibits good parallel performance. Taken together, the DSL interface allows end-users to rapidly and efficiently prototype new mathematical approaches, and deploy them on large clusters for solving practical problems. We illustrate this DSL by implementing a workflow for solving PDEs using the recently developed shifted boundary method (SBM). The SBM requires approximating relatively complicated integrals over boundary surfaces. Using a high-level DSL greatly simplifies this process and allows rapid exploration of variations. We demonstrate these tools on a variety of 2-D and 3-D configurations. With fewer than 20 lines of input, we can produce a parallel code that scales well to thousands of processes. This generated code is made accessible and readable to be easily modified and hand-tuned, making this tool useful even to experts with the target software.
PaintSeg: Training-free Segmentation via Painting
Venue: arXiv cs.CV 2023
Year: 2023

The paper introduces PaintSeg, a new unsupervised method for segmenting objects without any training. We propose an adversarial masked contrastive painting (AMCP) process, which creates a contrast between the original image and a painted image in which a masked area is painted using off-the-shelf generative models. During the painting process, inpainting and outpainting are alternated, with the former masking the foreground and filling in the background, and the latter masking the background while recovering the missing part of the foreground object. Inpainting and outpainting, also referred to as I-step and O-step, allow our method to gradually advance the target segmentation mask toward the ground truth without supervision or training. PaintSeg can be configured to work with a variety of prompts, e.g. coarse masks, boxes, scribbles, and points. Our experimental results demonstrate that PaintSeg outperforms existing approaches in coarse mask-prompt, box-prompt, and point-prompt segmentation tasks, providing a training-free solution suitable for unsupervised segmentation.
Hierarchical Multi-Instance Multi-Label Learning for Detecting Propaganda Techniques
Venue: arXiv cs.CL 2023
Year: 2023

Since the introduction of the SemEval 2020 Task 11 (Martino et al., 2020a), several approaches have been proposed in the literature for classifying propaganda based on the rhetorical techniques used to influence readers. These methods, however, classify one span at a time, ignoring dependencies from the labels of other spans within the same context. In this paper, we approach propaganda technique classification as a Multi-Instance Multi-Label (MIML) learning problem (Zhou et al., 2012) and propose a simple RoBERTa-based model (Zhuang et al., 2021) for classifying all spans in an article simultaneously. Further, we note that, due to the annotation process where annotators classified the spans by following a decision tree, there is an inherent hierarchical relationship among the different techniques, which existing approaches ignore. We incorporate these hierarchical label dependencies by adding an auxiliary classifier for each node in the decision tree to the training objective and ensembling the predictions from the original and auxiliary classifiers at test time. Overall, our model leads to an absolute improvement of 2.47% micro-F1 over the model from the shared task winning team in a cross-validation setup and is the best performing non-ensemble model on the shared task leaderboard.
Pixelated Interactions: Exploring Pixel Art for Graphical Primitives on a Tactile Display
Venue: arXiv cs.HC 2023
Year: 2023

Two-dimensional pin array tactile displays enable access to tactile graphics that are important for the education of students with visual impairments. Due to their prohibitive cost, limited access, and limited research within HCI, the rules to design graphical primitives on these low-resolution tactile displays are unclear. In this paper, eight tactile readers with visual impairments qualitatively evaluate the implementation of Pixel Art to create tactile graphical primitives on a pin array display. Every pin of the pin array is assumed to be a pixel on a pixel grid. Our findings suggest that Pixel Art tactile graphics on a pin array are clear and comprehensible to tactile readers, positively confirming its use to design basic tactile shapes and line segments. The guidelines provide a framework to create tactile media which implies that the guidelines can be used to downsize basic shapes for refreshable pin-array displays.
CVSNet: A Computer Implementation for Central Visual System of The Brain
Venue: arXiv cs.CV 2023
Year: 2023

In computer vision, different basic blocks are created around different matrix operations, and models based on different basic blocks have achieved good results. Good results achieved in vision tasks grants them rationality. However, these experimental-based models also make deep learning long criticized for principle and interpretability. Deep learning originated from the concept of neurons in neuroscience, but recent designs detached natural neural networks except for some simple concepts. In this paper, we build an artificial neural network, CVSNet, which can be seen as a computer implementation for central visual system of the brain. Each block in CVSNet represents the same vision information as that in brains. In CVSNet, blocks differs from each other and visual information flows through three independent pathways and five different blocks. Thus CVSNet is completely different from the design of all previous models, in which basic blocks are repeated to build model and information between channels is mixed at the outset. In ablation experiment, we show the information extracted by blocks in CVSNet and compare with previous networks, proving effectiveness and rationality of blocks in CVSNet from experiment side. And in the experiment of object recognition, CVSNet achieves comparable results to ConvNets, Vision Transformers and MLPs.
M3ICRO: Machine Learning-Enabled Compact Photonic Tensor Core based on PRogrammable Multi-Operand Multimode Interference
Venue: arXiv cs.ET 2023
Year: 2023

Photonic computing shows promise for transformative advancements in machine learning (ML) acceleration, offering ultra-fast speed, massive parallelism, and high energy efficiency. However, current photonic tensor core (PTC) designs based on standard optical components hinder scalability and compute density due to their large spatial footprint. To address this, we propose an ultra-compact PTC using customized programmable multi-operand multimode interference (MOMMI) devices, named M3ICRO. The programmable MOMMI leverages the intrinsic light propagation principle, providing a single-device programmable matrix unit beyond the conventional computing paradigm of one multiply-accumulate (MAC) operation per device. To overcome the optimization difficulty of customized devices that often requires time-consuming simulation, we apply ML for optics to predict the device behavior and enable a differentiable optimization flow. We thoroughly investigate the reconfigurability and matrix expressivity of our customized PTC, and introduce a novel block unfolding method to fully exploit the computing capabilities of a complex-valued PTC for near-universal real-valued linear transformations. Extensive evaluations demonstrate that M3ICRO achieves a 3.4-9.6x smaller footprint, 1.6-4.4x higher speed, 10.6-42x higher compute density, 3.7-12x higher system throughput, and superior noise robustness compared to state-of-the-art coherent PTC designs, while maintaining close-to-digital task accuracy across various ML benchmarks. Our code is open-sourced at https://github.com/JeremieMelo/M3ICRO-MOMMI.
Signal Is Harder To Learn Than Bias: Debiasing with Focal Loss
Venue: arXiv cs.LG 2023
Year: 2023

Spurious correlations are everywhere. While humans often do not perceive them, neural networks are notorious for learning unwanted associations, also known as biases, instead of the underlying decision rule. As a result, practitioners are often unaware of the biased decision-making of their classifiers. Such a biased model based on spurious correlations might not generalize to unobserved data, leading to unintended, adverse consequences. We propose Signal is Harder (SiH), a variational-autoencoder-based method that simultaneously trains a biased and unbiased classifier using a novel, disentangling reweighting scheme inspired by the focal loss. Using the unbiased classifier, SiH matches or improves upon the performance of state-of-the-art debiasing methods. To improve the interpretability of our technique, we propose a perturbation scheme in the latent space for visualizing the bias that helps practitioners become aware of the sources of spurious correlations.
VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning Challenges
Venue: arXiv cs.CV 2023
Year: 2023

The third edition of the "VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning" workshop featured four data-impaired challenges, focusing on addressing the limitations of data availability in training deep learning models for computer vision tasks. The challenges comprised of four distinct data-impaired tasks, where participants were required to train models from scratch using a reduced number of training samples. The primary objective was to encourage novel approaches that incorporate relevant inductive biases to enhance the data efficiency of deep learning models. To foster creativity and exploration, participants were strictly prohibited from utilizing pre-trained checkpoints and other transfer learning techniques. Significant advancements were made compared to the provided baselines, where winning solutions surpassed the baselines by a considerable margin in all four tasks. These achievements were primarily attributed to the effective utilization of extensive data augmentation policies, model ensembling techniques, and the implementation of data-efficient training methods, including self-supervised representation learning. This report highlights the key aspects of the challenges and their outcomes.
Assessing Word Importance Using Models Trained for Semantic Tasks
Venue: arXiv cs.CL 2023
Year: 2023

Many NLP tasks require to automatically identify the most significant words in a text. In this work, we derive word significance from models trained to solve semantic task: Natural Language Inference and Paraphrase Identification. Using an attribution method aimed to explain the predictions of these models, we derive importance scores for each input token. We evaluate their relevance using a so-called cross-task evaluation: Analyzing the performance of one model on an input masked according to the other model's weight, we show that our method is robust with respect to the choice of the initial task. Additionally, we investigate the scores from the syntax point of view and observe interesting patterns, e.g. words closer to the root of a syntactic tree receive higher importance scores. Altogether, these observations suggest that our method can be used to identify important words in sentences without any explicit word importance labeling in training.
Building Extractive Question Answering System to Support Human-AI Health Coaching Model for Sleep Domain
Venue: arXiv cs.CL 2023
Year: 2023

Non-communicable diseases (NCDs) are a leading cause of global deaths, necessitating a focus on primary prevention and lifestyle behavior change. Health coaching, coupled with Question Answering (QA) systems, has the potential to transform preventive healthcare. This paper presents a human-Artificial Intelligence (AI) health coaching model incorporating a domain-specific extractive QA system. A sleep-focused dataset, SleepQA, was manually assembled and used to fine-tune domain-specific BERT models. The QA system was evaluated using automatic and human methods. A data-centric framework enhanced the system's performance by improving passage retrieval and question reformulation. Although the system did not outperform the baseline in automatic evaluation, it excelled in the human evaluation of real-world questions. Integration into a Human-AI health coaching model was tested in a pilot Randomized Controlled Trial (RCT).
Direct Learning-Based Deep Spiking Neural Networks: A Review
Venue: arXiv cs.CV 2023
Year: 2023

The spiking neural network (SNN), as a promising brain-inspired computational model with binary spike information transmission mechanism, rich spatially-temporal dynamics, and event-driven characteristics, has received extensive attention. However, its intricately discontinuous spike mechanism brings difficulty to the optimization of the deep SNN. Since the surrogate gradient method can greatly mitigate the optimization difficulty and shows great potential in directly training deep SNNs, a variety of direct learning-based deep SNN works have been proposed and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these direct learning-based deep SNN works, mainly categorized into accuracy improvement methods, efficiency improvement methods, and temporal dynamics utilization methods. In addition, we also divide these categorizations into finer granularities further to better organize and introduce them. Finally, the challenges and trends that may be faced in future research are prospected.
Data Representations' Study of Latent Image Manifolds
Venue: arXiv cs.LG 2023
Year: 2023

Deep neural networks have been demonstrated to achieve phenomenal success in many domains, and yet their inner mechanisms are not well understood. In this paper, we investigate the curvature of image manifolds, i.e., the manifold deviation from being flat in its principal directions. We find that state-of-the-art trained convolutional neural networks for image classification have a characteristic curvature profile along layers: an initial steep increase, followed by a long phase of a plateau, and followed by another increase. In contrast, this behavior does not appear in untrained networks in which the curvature flattens. We also show that the curvature gap between the last two layers has a strong correlation with the generalization capability of the network. Moreover, we find that the intrinsic dimension of latent codes is not necessarily indicative of curvature. Finally, we observe that common regularization methods such as mixup yield flatter representations when compared to other methods. Our experiments show consistent results over a variety of deep learning architectures and multiple data sets. Our code is publicly available at https://github.com/azencot-group/CRLM
Concentrated Geo-Privacy
Venue: arXiv cs.CR 2023
Year: 2023

This paper proposes concentrated geo-privacy (CGP), a privacy notion that can be considered as the counterpart of concentrated differential privacy (CDP) for geometric data. Compared with the previous notion of geo-privacy [ABCP13, CABP13], which is the counterpart of standard differential privacy, CGP offers many benefits including simplicity of the mechanism, lower noise scale in high dimensions, and better composability known as advanced composition. The last one is the most important, as it allows us to design complex mechanisms using smaller building blocks while achieving better utilities. To complement this result, we show that the previous notion of geo-privacy inherently does not admit advanced composition even using its approximate version. Next, we study three problems on private geometric data: the identity query, k nearest neighbors, and convex hulls. While the first problem has been previously studied, we give the first mechanisms for the latter two under geo-privacy. For all three problems, composability is essential in obtaining good utility guarantees on the privatized query answer.
Recursive Metropolis-Hastings Naming Game: Symbol Emergence in a Multi-agent System based on Probabilistic Generative Models
Venue: arXiv cs.CL 2023
Year: 2023

In the studies on symbol emergence and emergent communication in a population of agents, a computational model was employed in which agents participate in various language games. Among these, the Metropolis-Hastings naming game (MHNG) possesses a notable mathematical property: symbol emergence through MHNG is proven to be a decentralized Bayesian inference of representations shared by the agents. However, the previously proposed MHNG is limited to a two-agent scenario. This paper extends MHNG to an N-agent scenario. The main contributions of this paper are twofold: (1) we propose the recursive Metropolis-Hastings naming game (RMHNG) as an N-agent version of MHNG and demonstrate that RMHNG is an approximate Bayesian inference method for the posterior distribution over a latent variable shared by agents, similar to MHNG; and (2) we empirically evaluate the performance of RMHNG on synthetic and real image data, enabling multiple agents to develop and share a symbol system. Furthermore, we introduce two types of approximations -- one-sample and limited-length -- to reduce computational complexity while maintaining the ability to explain communication in a population of agents. The experimental findings showcased the efficacy of RMHNG as a decentralized Bayesian inference for approximating the posterior distribution concerning latent variables, which are jointly shared among agents, akin to MHNG. Moreover, the utilization of RMHNG elucidated the agents' capacity to exchange symbols. Furthermore, the study discovered that even the computationally simplified version of RMHNG could enable symbols to emerge among the agents.
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting
Venue: arXiv cs.CL 2023
Year: 2023

Multilingual image captioning has recently been tackled by training with large-scale machine translated data, which is an expensive, noisy, and time-consuming process. Without requiring any multilingual caption data, we propose LMCap, an image-blind few-shot multilingual captioning model that works by prompting a language model with retrieved captions. Specifically, instead of following the standard encoder-decoder paradigm, given an image, LMCap first retrieves the captions of similar images using a multilingual CLIP encoder. These captions are then combined into a prompt for an XGLM decoder, in order to generate captions in the desired language. In other words, the generation model does not directly process the image, instead processing retrieved captions. Experiments on the XM3600 dataset of geographically diverse images show that our model is competitive with fully-supervised multilingual captioning models, without requiring any supervised training on any captioning data.
Inverse-design of nonlinear mechanical metamaterials via video denoising diffusion models
Venue: arXiv cs.CE 2023
Year: 2023

The accelerated inverse design of complex material properties - such as identifying a material with a given stress-strain response over a nonlinear deformation path - holds great potential for addressing challenges from soft robotics to biomedical implants and impact mitigation. While machine learning models have provided such inverse mappings, they are typically restricted to linear target properties such as stiffness. To tailor the nonlinear response, we here show that video diffusion generative models trained on full-field data of periodic stochastic cellular structures can successfully predict and tune their nonlinear deformation and stress response under compression in the large-strain regime, including buckling and contact. Unlike commonly encountered black-box models, our framework intrinsically provides an estimate of the expected deformation path, including the full-field internal stress distribution closely agreeing with finite element simulations. This work has thus the potential to simplify and accelerate the identification of materials with complex target performance.
Biography-based Robot Games for Older Adults
Venue: arXiv cs.RO 2023
Year: 2023

One issue in aging is how to stimulate the cognitive skills of older adults. One way to address it is the use of serious games delivered through humanoid robots, to provide engaging ways to perform exercises to train memory, attention, processing, and planning activities. We present an approach in which a humanoid robot, by using various modalities, propose the games in a way personalised to specific individuals' experiences using their personal memories associated with facts and events that occurred in older adults' life. This personalization can increase their interest and engagement, and thus potentially reduce the cognitive training drop-out.
Exploring the Journey to Drug Overdose: Applying the Journey to Crime Framework to Drug Sales Locations and Overdose Death Locations
Venue: arXiv cs.SI 2023
Year: 2023

Drug overdose is a pressing public health concern in the United States, resulting in a significant number of fatalities each year. In this study, we employ the Journey to Crime (JTC) framework borrowed from the field of environmental criminology to examine the association between drug sales locations and overdose death locations. In this research, our objective is to elucidate the trajectory of overdose victims to overdose locations, aiming to enhance the distribution of overdose services and interventions. To the best of our knowledge, no previous studies have applied the JTC framework to investigate drug overdose deaths. By scrutinizing data obtained from the Hamilton County, OH Coroners, and the Cincinnati Police Department, we endeavor to explore the plausible correlation between overdose deaths and drug sales locations. Our findings underscore the necessity of implementing a comprehensive strategy to curtail overdose deaths. This strategy should encompass various facets, including targeted efforts to reduce the accessibility of illicit drugs, the enhancement of responses to overdose incidents through a collaborative multidisciplinary approach, and the availability of data to inform evidence-based strategies and facilitate outcome evaluation. By shedding light on the relationship between drug sales locations and overdose death locations through the utilization of the JTC framework, this study contributes valuable insights to the field of drug overdose prevention. It emphasizes the significance of adopting multifaceted approaches to address this public health crisis effectively. Ultimately, our research aims to inform the development of evidence-based interventions and policies that can mitigate the occurrence and impact of drug overdoses in our communities.