pwc-archive/papers-with-abstracts huggingface dataset:
Transferability analysis of data-driven additive manufacturing knowledge: a case study between powder bed fusion and directed energy deposition
Authors: Mutahar Safdar, Jiarui Xie, Hyunwoong Ko, Yan Lu, Guy Lamouche, Yaoyao Fiona Zhao
Date: 2023-09-12 00:00:00
Tasks: Transfer Learning

Data-driven research in Additive Manufacturing (AM) has gained significant success in recent years. This has led to a plethora of scientific literature to emerge. The knowledge in these works consists of AM and Artificial Intelligence (AI) contexts that have not been mined and formalized in an integrated way. Moreover, no tools or guidelines exist to support data-driven knowledge transfer from one context to another. As a result, data-driven solutions using specific AI techniques are being developed and validated only for specific AM process technologies. There is a potential to exploit the inherent similarities across various AM technologies and adapt the existing solutions from one process or problem to another using AI, such as Transfer Learning. We propose a three-step knowledge transferability analysis framework in AM to support data-driven AM knowledge transfer. As a prerequisite to transferability analysis, AM knowledge is featurized into identified knowledge components. The framework consists of pre-transfer, transfer, and post-transfer steps to accomplish knowledge transfer. A case study is conducted between flagship metal AM processes. Laser Powder Bed Fusion (LPBF) is the source of knowledge motivated by its relative matureness in applying AI over Directed Energy Deposition (DED), which drives the need for knowledge transfer as the less explored target process. We show successful transfer at different levels of the data-driven solution, including data representation, model architecture, and model parameters. The pipeline of AM knowledge transfer can be automated in the future to allow efficient cross-context or cross-process knowledge exchange.
Jersey Number Recognition using Keyframe Identification from Low-Resolution Broadcast Videos
Authors: Bavesh Balaji, Jerrin Bright, Harish Prakash, Yuhao Chen, David A Clausi, John Zelek
Date: 2023-09-12 00:00:00
Tasks: Jersey Number Recognition

Player identification is a crucial component in vision-driven soccer analytics, enabling various downstream tasks such as player assessment, in-game analysis, and broadcast production. However, automatically detecting jersey numbers from player tracklets in videos presents challenges due to motion blur, low resolution, distortions, and occlusions. Existing methods, utilizing Spatial Transformer Networks, CNNs, and Vision Transformers, have shown success in image data but struggle with real-world video data, where jersey numbers are not visible in most of the frames. Hence, identifying frames that contain the jersey number is a key sub-problem to tackle. To address these issues, we propose a robust keyframe identification module that extracts frames containing essential high-level information about the jersey number. A spatio-temporal network is then employed to model spatial and temporal context and predict the probabilities of jersey numbers in the video. Additionally, we adopt a multi-task loss function to predict the probability distribution of each digit separately. Extensive evaluations on the SoccerNet dataset demonstrate that incorporating our proposed keyframe identification module results in a significant 37.81% and 37.70% increase in the accuracies of 2 different test sets with domain gaps. These results highlight the effectiveness and importance of our approach in tackling the challenges of automatic jersey number detection in sports videos.
Enhancing multimodal cooperation via sample-level modality valuation
Authors: Yake Wei, Ruoxuan Feng, Zihe Wang, Di Hu
Date: 2023-09-12 00:00:00
Proceeding: CVPR 2024 1

One primary topic of multimodal learning is to jointly incorporate heterogeneous information from different modalities. However most models often suffer from unsatisfactory multimodal cooperation which cannot jointly utilize all modalities well. Some methods are proposed to identify and enhance the worse learnt modality but they are often hard to provide the fine-grained observation of multimodal cooperation at sample-level with theoretical support. Hence it is essential to reasonably observe and improve the fine-grained cooperation between modalities especially when facing realistic scenarios where the modality discrepancy could vary across different samples. To this end we introduce a sample-level modality valuation metric to evaluate the contribution of each modality for each sample. Via modality valuation we observe that modality discrepancy indeed could be different at sample-level beyond the global contribution discrepancy at dataset-level. We further analyze this issue and improve cooperation between modalities at sample-level by enhancing the discriminative ability of low-contributing modalities in a targeted manner. Overall our methods reasonably observe the fine-grained uni-modal contribution and achieve considerable improvement. The source code and dataset are available at https://github.com/GeWu-Lab/Valuate-and-Enhance-Multimodal-Cooperation.
On the Injunction of XAIxArt
Authors: Cheshta Arora, Debarun Sarkar
Date: 2023-09-12 00:00:00
Tasks: Explainable artificial intelligence, Position

The position paper highlights the range of concerns that are engulfed in the injunction of explainable artificial intelligence in art (XAIxArt). Through a series of quick sub-questions, it points towards the ambiguities concerning 'explanation' and the postpositivist tradition of 'relevant explanation'. Rejecting both 'explanation' and 'relevant explanation', the paper takes a stance that XAIxArt is a symptom of insecurity of the anthropocentric notion of art and a nostalgic desire to return to outmoded notions of authorship and human agency. To justify this stance, the paper makes a distinction between an ornamentation model of explanation to a model of explanation as sense-making.
Expressive dynamics models with nonlinear injective readouts enable reliable recovery of latent features from neural activity
Authors: Christopher Versteeg, Andrew R. Sedler, Jonathan D. McCart, Chethan Pandarinath
Date: 2023-09-12 00:00:00

The advent of large-scale neural recordings has enabled new methods to discover the computational mechanisms of neural circuits by understanding the rules that govern how their state evolves over time. While these \textit{neural dynamics} cannot be directly measured, they can typically be approximated by low-dimensional models in a latent space. How these models represent the mapping from latent space to neural space can affect the interpretability of the latent representation. We show that typical choices for this mapping (e.g., linear or MLP) often lack the property of injectivity, meaning that changes in latent state are not obligated to affect activity in the neural space. During training, non-injective readouts incentivize the invention of dynamics that misrepresent the underlying system and the computation it performs. Combining our injective Flow readout with prior work on interpretable latent dynamics models, we created the Ordinary Differential equations autoencoder with Injective Nonlinear readout (ODIN), which captures latent dynamical systems that are nonlinearly embedded into observed neural activity via an approximately injective nonlinear mapping. We show that ODIN can recover nonlinearly embedded systems from simulated neural activity, even when the nature of the system and embedding are unknown. Additionally, ODIN enables the unsupervised recovery of underlying dynamical features (e.g., fixed points) and embedding geometry. When applied to biological neural recordings, ODIN can reconstruct neural activity with comparable accuracy to previous state-of-the-art methods while using substantially fewer latent dimensions. Overall, ODIN's accuracy in recovering ground-truth latent features and ability to accurately reconstruct neural activity with low dimensionality make it a promising method for distilling interpretable dynamics that can help explain neural computation.
Compiled Models, Built-In Exploits: Uncovering Pervasive Bit-Flip Attack Surfaces in DNN Executables
Authors: Yanzuo Chen, Zhibo Liu, Yuanyuan Yuan, Sihang Hu, Tianxiang Li, Shuai Wang
Date: 2023-09-12 00:00:00

Bit-flip attacks (BFAs) can manipulate deep neural networks (DNNs). For high-level DNN models running on deep learning (DL) frameworks like PyTorch, extensive BFAs have been used to flip bits in model weights and shown effective. Defenses have also been proposed to guard model weights. However, DNNs are increasingly compiled into DNN executables by DL compilers to leverage hardware primitives. These executables manifest distinct computation paradigms; existing research fails to accurately capture and expose the BFA surfaces on DNN executables. To this end, we launch the first systematic study of BFAs on DNN executables. Prior BFAs are limited to attacking model weights and assume a strong whitebox attacker with full knowledge of victim model weights, which is unrealistic as weights are often confidential. In contrast, we find that BFAs on DNN executables can achieve high effectiveness by exploiting the model structure (usually stored in the executable code), which only requires knowing the (often public) model structure. Importantly, such structure-based BFAs are pervasive, transferable, and more severe in DNN executables. They also slip past existing defenses. To demonstrate the new attack surfaces, we assume a weak and more realistic attacker with no knowledge of victim model weights. We design an automated tool to identify vulnerable bits in victim executables with high confidence (70% vs. baseline 2%). We show on DDR4 DRAM that only 1.4 flips on average are needed to fully downgrade the accuracy of victim models, including quantized ones which could require 23x more flips previously, to random guesses. We comprehensively evaluate 16 DNN executables, covering large-scale models trained on commonly-used datasets compiled by the two most popular DL compilers. Our finding calls for incorporating security mechanisms in future DNN compilation toolchains.
Unveiling the Complexity of Neural Populations: Evaluating the Validity and Limitations of the Wilson-Cowan Model
Authors: Maryam Saadati, Saba Sadat Khodaei, Yousef Jamali
Date: 2023-09-12 00:00:00
Tasks: Diversity

The population model of Wilson-Cowan is perhaps the most popular in the history of computational neuroscience. It embraces the nonlinear mean field dynamics of excitatory and inhibitory neuronal populations provided via a temporal coarse-graining technique. The traditional Wilson-Cowan equations exhibit either steady-state regimes or else limit cycle competitions for an appropriate range of parameters. As these equations lower the resolution of the neural system and obscure vital information, we assess the validity of mass-type model approximations for complex neural behaviors. Using a large-scale network of Hodgkin-Huxley style neurons, we derive implicit average population dynamics based on mean field assumptions. Our comparison of the microscopic neural activity with the macroscopic temporal profiles reveals dependency on the binary state of interacting subpopulations and the random property of the structural network at the Hopf bifurcation points when different synaptic weights are considered. For substantial configurations of stimulus intensity, our model provides further estimates of the neural population's dynamics official, ranging from simple periodic to quasi-periodic and aperiodic patterns, as well as phase transition regimes. While this shows its great potential for studying the collective behavior of individual neurons particularly concentrating on the occurrence of bifurcation phenomena, we must accept a quite limited accuracy of the Wilson-Cowan approximations-at least in some parameter regimes. Additionally, we report that the complexity and temporal diversity of neural dynamics, especially in terms of limit cycle trajectory, and synchronization can be induced by either small heterogeneity in the degree of various types of local excitatory connectivity or considerable diversity in the external drive to the excitatory pool.
SCP: Scene Completion Pre-training for 3D Object Detection
Authors: Yiming Shan, Yan Xia, Yuhong Chen, Daniel Cremers
Date: 2023-09-12 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Object, object-detection, Object Detection

3D object detection using LiDAR point clouds is a fundamental task in the fields of computer vision, robotics, and autonomous driving. However, existing 3D detectors heavily rely on annotated datasets, which are both time-consuming and prone to errors during the process of labeling 3D bounding boxes. In this paper, we propose a Scene Completion Pre-training (SCP) method to enhance the performance of 3D object detectors with less labeled data. SCP offers three key advantages: (1) Improved initialization of the point cloud model. By completing the scene point clouds, SCP effectively captures the spatial and semantic relationships among objects within urban environments. (2) Elimination of the need for additional datasets. SCP serves as a valuable auxiliary network that does not impose any additional efforts or data requirements on the 3D detectors. (3) Reduction of the amount of labeled data for detection. With the help of SCP, the existing state-of-the-art 3D detectors can achieve comparable performance while only relying on 20% labeled data.
Spontaneous brain activity emerges from pairwise interactions in the larval zebrafish brain
Authors: Richard E. Rosch, Dominic R. W. Burrows, Christopher W. Lynn, Arian Ashourvan
Date: 2023-09-12 00:00:00

Brain activity is characterized by brain-wide spatiotemporal patterns which emerge from synapse-mediated interactions between individual neurons. Calcium imaging provides access to in vivo recordings of whole-brain activity at single-neuron resolution and, therefore, allows the study of how large-scale brain dynamics emerge from local activity. In this study, we used a statistical mechanics approach - the pairwise maximum entropy model (MEM) - to infer microscopic network features from collective patterns of activity in the larval zebrafish brain, and relate these features to the emergence of observed whole-brain dynamics. Our findings indicate that the pairwise interactions between neural populations and their intrinsic activity states are sufficient to explain observed whole-brain dynamics. In fact, the pairwise relationships between neuronal populations estimated with the MEM strongly correspond to observed structural connectivity patterns. Model simulations also demonstrated how tuning pairwise neuronal interactions drives transitions between critical and pathologically hyper-excitable whole-brain regimes. Finally, we use virtual resection to identify the brain structures that are important for maintaining the brain in a critical regime. Together, our results indicate that whole-brain activity emerges out of a complex dynamical system that transitions between basins of attraction whose strength and topology depend on the connectivity between brain areas.
360$^\circ$ from a Single Camera: A Few-Shot Approach for LiDAR Segmentation
Authors: Laurenz Reichardt, Nikolas Ebert, Oliver Wasenmüller
Date: 2023-09-12 00:00:00
Tasks: Segmentation, Semi-Supervised Semantic Segmentation

Deep learning applications on LiDAR data suffer from a strong domain gap when applied to different sensors or tasks. In order for these methods to obtain similar accuracy on different data in comparison to values reported on public benchmarks, a large scale annotated dataset is necessary. However, in practical applications labeled data is costly and time consuming to obtain. Such factors have triggered various research in label-efficient methods, but a large gap remains to their fully-supervised counterparts. Thus, we propose ImageTo360, an effective and streamlined few-shot approach to label-efficient LiDAR segmentation. Our method utilizes an image teacher network to generate semantic predictions for LiDAR data within a single camera view. The teacher is used to pretrain the LiDAR segmentation student network, prior to optional fine-tuning on 360$^\circ$ data. Our method is implemented in a modular manner on the point level and as such is generalizable to different architectures. We improve over the current state-of-the-art results for label-efficient methods and even surpass some traditional fully-supervised segmentation networks.
A 3M-Hybrid Model for the Restoration of Unique Giant Murals: A Case Study on the Murals of Yongle Palace
Authors: Jing Yang, Nur Intan Raihana Ruhaiyem, ChiChun Zhou
Date: 2023-09-12 00:00:00
Tasks: SSIM, Transfer Learning

The Yongle Palace murals, as valuable cultural heritage, have suffered varying degrees of damage, making their restoration of significant importance. However, the giant size and unique data of Yongle Palace murals present challenges for existing deep-learning based restoration methods: 1) The distinctive style introduces domain bias in traditional transfer learning-based restoration methods, while the scarcity of mural data further limits the applicability of these methods. 2) Additionally, the giant size of these murals results in a wider range of defect types and sizes, necessitating models with greater adaptability. Consequently, there is a lack of focus on deep learning-based restoration methods for the unique giant murals of Yongle Palace. Here, a 3M-Hybrid model is proposed to address these challenges. Firstly, based on the characteristic that the mural data frequency is prominent in the distribution of low and high frequency features, high and low frequency features are separately abstracted for complementary learning. Furthermore, we integrate a pre-trained Vision Transformer model (VIT) into the CNN module, allowing us to leverage the benefits of a large model while mitigating domain bias. Secondly, we mitigate seam and structural distortion issues resulting from the restoration of large defects by employing a multi-scale and multi-perspective strategy, including data segmentation and fusion. Experimental results demonstrate the efficacy of our proposed model. In regular-sized mural restoration, it improves SSIM and PSNR by 14.61% and 4.73%, respectively, compared to the best model among four representative CNN models. Additionally, it achieves favorable results in the final restoration of giant murals.
The bionic neural network for external simulation of human locomotor system
Authors: Yue Shi, Shuhao Ma, Yihui Zhao
Date: 2023-09-11 00:00:00

Muscle forces and joint kinematics estimated with musculoskeletal (MSK) modeling techniques offer useful metrics describing movement quality. Model-based computational MSK models can interpret the dynamic interaction between the neural drive to muscles, muscle dynamics, body and joint kinematics, and kinetics. Still, such a set of solutions suffers from high computational time and muscle recruitment problems, especially in complex modeling. In recent years, data-driven methods have emerged as a promising alternative due to the benefits of flexibility and adaptability. However, a large amount of labeled training data is not easy to be acquired. This paper proposes a physics-informed deep learning method based on MSK modeling to predict joint motion and muscle forces. The MSK model is embedded into the neural network as an ordinary differential equation (ODE) loss function with physiological parameters of muscle activation dynamics and muscle contraction dynamics to be identified. These parameters are automatically estimated during the training process which guides the prediction of muscle forces combined with the MSK forward dynamics model. Experimental validations on two groups of data, including one benchmark dataset and one self-collected dataset from six healthy subjects, are performed. The results demonstrate that the proposed deep learning method can effectively identify subject-specific MSK physiological parameters and the trained physics-informed forward-dynamics surrogate yields accurate motion and muscle forces predictions.
Glancing Future for Simultaneous Machine Translation
Authors: Shoutao Guo, Shaolei Zhang, Yang Feng
Date: 2023-09-12 00:00:00
Tasks: Machine Translation, Sentence, Translation

Simultaneous machine translation (SiMT) outputs translation while reading the source sentence. Unlike conventional sequence-to-sequence (seq2seq) training, existing SiMT methods adopt the prefix-to-prefix (prefix2prefix) training, where the model predicts target tokens based on partial source tokens. However, the prefix2prefix training diminishes the ability of the model to capture global information and introduces forced predictions due to the absence of essential source information. Consequently, it is crucial to bridge the gap between the prefix2prefix training and seq2seq training to enhance the translation capability of the SiMT model. In this paper, we propose a novel method that glances future in curriculum learning to achieve the transition from the seq2seq training to prefix2prefix training. Specifically, we gradually reduce the available source information from the whole sentence to the prefix corresponding to that latency. Our method is applicable to a wide range of SiMT methods and experiments demonstrate that our method outperforms strong baselines.
An ancestral axial twist explains the contralateral forebrain and the optic chiasm in vertebrates
Authors: Marc H. E. de Lussanet, Jan W. M. Osse
Date: 2010-03-09 00:00:00

Among the best-known facts of the brain are the contralateral visual, auditory, sensational, and motor mappings in the forebrain. How and why did these evolve? The few theories to this question provide functional answers, such as better networks for visuomotor control. However, these theories contradict the data, as discussed here. Instead we propose that a 90-deg left-turn around the body-axis evolved in a common ancestor of all vertebrates. Compensatory migrations of the tissues during development restore body symmetry. Eyes, nostrils and forebrain compensate in the direction of the turn, whereas more caudal structures migrate in the opposite direction. As a result of these opposite migrations the forebrain becomes crossed and inverted with respect to the rest of the nervous system. We show that these compensatory migratory movements can indeed be observed in the zebrafish (Danio rerio) and the chick (Gallus gallus). With a model we show how the axial twist hypothesis predicts that an optic chiasm should develop on the ventral side of the brain, whereas the olfactory tract should be uncrossed. In addition, the hypothesis explains the decussation of the trochlear nerve, why olfaction is non-crossed, why the cerebellar hemispheres represent the ipsilateral bodyside, why in sharks the forebrain halves each represent the ipsilateral eye, why the heart and other inner organs are asymmetric in the body. Due to the poor fossil record, the possible evolutionary scenarios remain speculative. Molecular evidence does support the hypothesis. The findings may throw new insight on the problematic structure of the forebrain.