pwc-archive/papers-with-abstracts huggingface dataset:
AnchorInv: Few-Shot Class-Incremental Learning of Physiological Signals via Representation Space Guided Inversion
Authors: Chenqi Li, Boyan Gao, Gabriel Jones, Timothy Denison, Tingting Zhu
Date: 2024-12-18 00:00:00
Tasks: class-incremental learning, Class Incremental Learning, Few-Shot Class-Incremental Learning, Incremental Learning

Deep learning models have demonstrated exceptional performance in a variety of real-world applications. These successes are often attributed to strong base models that can generalize to novel tasks with limited supporting data while keeping prior knowledge intact. However, these impressive results are based on the availability of a large amount of high-quality data, which is often lacking in specialized biomedical applications. In such fields, models are usually developed with limited data that arrive incrementally with novel categories. This requires the model to adapt to new information while preserving existing knowledge. Few-Shot Class-Incremental Learning (FSCIL) methods offer a promising approach to addressing these challenges, but they also depend on strong base models that face the same aforementioned limitations. To overcome these constraints, we propose AnchorInv following the straightforward and efficient buffer-replay strategy. Instead of selecting and storing raw data, AnchorInv generates synthetic samples guided by anchor points in the feature space. This approach protects privacy and regularizes the model for adaptation. When evaluated on three public physiological time series datasets, AnchorInv exhibits efficient knowledge forgetting prevention and improved adaptation to novel classes, surpassing state-of-the-art baselines.
SSE-SAM: Balancing Head and Tail Classes Gradually through Stage-Wise SAM
Authors: Xingyu Lyu, Qianqian Xu, Zhiyong Yang, Shaojie Lyu, Qingming Huang
Date: 2024-12-18 00:00:00

Real-world datasets often exhibit a long-tailed distribution, where vast majority of classes known as tail classes have only few samples. Traditional methods tend to overfit on these tail classes. Recently, a new approach called Imbalanced SAM (ImbSAM) is proposed to leverage the generalization benefits of Sharpness-Aware Minimization (SAM) for long-tailed distributions. The main strategy is to merely enhance the smoothness of the loss function for tail classes. However, we argue that improving generalization in long-tail scenarios requires a careful balance between head and tail classes. We show that neither SAM nor ImbSAM alone can fully achieve this balance. For SAM, we prove that although it enhances the model's generalization ability by escaping saddle point in the overall loss landscape, it does not effectively address this for tail-class losses. Conversely, while ImbSAM is more effective at avoiding saddle points in tail classes, the head classes are trained insufficiently, resulting in significant performance drops. Based on these insights, we propose Stage-wise Saddle Escaping SAM (SSE-SAM), which uses complementary strengths of ImbSAM and SAM in a phased approach. Initially, SSE-SAM follows the majority sample to avoid saddle points of the head-class loss. During the later phase, it focuses on tail-classes to help them escape saddle points. Our experiments confirm that SSE-SAM has better ability in escaping saddles both on head and tail classes, and shows performance improvements.
Federated Source-free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Data
Authors: Junki Mori, Kosuke Kihara, Taiki Miyagawa, Akinori F. Ebihara, Isamu Teranishi, Hisashi Kashima
Date: 2024-12-18 00:00:00
Tasks: Domain Adaptation, Federated Learning, Semantic Segmentation, Source-Free Domain Adaptation

Federated learning (FL) commonly assumes that the server or some clients have labeled data, which is often impractical due to annotation costs and privacy concerns. Addressing this problem, we focus on a source-free domain adaptation task, where (1) the server holds a pre-trained model on labeled source domain data, (2) clients possess only unlabeled data from various target domains, and (3) the server and clients cannot access the source data in the adaptation phase. This task is known as Federated source-Free Domain Adaptation (FFREEDA). Specifically, we focus on classification tasks, while the previous work solely studies semantic segmentation. Our contribution is the novel Federated learning with Weighted Cluster Aggregation (FedWCA) method, designed to mitigate both domain shifts and privacy concerns with only unlabeled data. FedWCA comprises three phases: private and parameter-free clustering of clients to obtain domain-specific global models on the server, weighted aggregation of the global models for the clustered clients, and local domain adaptation with pseudo-labeling. Experimental results show that FedWCA surpasses several existing methods and baselines in FFREEDA, establishing its effectiveness and practicality.
Harnessing Transfer Learning from Swahili: Advancing Solutions for Comorian Dialects
Authors: Naira Abdou Mohamed, Zakarya Erraji, Abdessalam Bahafid, Imade Benelallam
Date: 2024-12-09 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, speech-recognition, Speech Recognition, Transfer Learning

If today some African languages like Swahili have enough resources to develop high-performing Natural Language Processing (NLP) systems, many other languages spoken on the continent are still lacking such support. For these languages, still in their infancy, several possibilities exist to address this critical lack of data. Among them is Transfer Learning, which allows low-resource languages to benefit from the good representation of other languages that are similar to them. In this work, we adopt a similar approach, aiming to pioneer NLP technologies for Comorian, a group of four languages or dialects belonging to the Bantu family. Our approach is initially motivated by the hypothesis that if a human can understand a different language from their native language with little or no effort, it would be entirely possible to model this process on a machine. To achieve this, we consider ways to construct Comorian datasets mixed with Swahili. One thing to note here is that in terms of Swahili data, we only focus on elements that are closest to Comorian by calculating lexical distances between candidate and source data. We empirically test this hypothesis in two use cases: Automatic Speech Recognition (ASR) and Machine Translation (MT). Our MT model achieved ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.6826, 0.42, and 0.6532, respectively, while our ASR system recorded a WER of 39.50\% and a CER of 13.76\%. This research is crucial for advancing NLP in underrepresented languages, with potential to preserve and promote Comorian linguistic heritage in the digital age.
Forward and Inverse Simulation of Pseudo-Two-Dimensional Model of Lithium-Ion Batteries Using Neural Networks
Authors: Myeong-Su Lee, Jaemin Oh, Dong-Chan Lee, Kangwook Lee, Sooncheol Park, Youngjoon Hong
Date: 2024-12-02 00:00:00
Tasks: parameter estimation

In this work, we address the challenges posed by the high nonlinearity of the Butler-Volmer (BV) equation in forward and inverse simulations of the pseudo-two-dimensional (P2D) model using the physics-informed neural network (PINN) framework. The BV equation presents significant challenges for PINNs, primarily due to the hyperbolic sine term, which renders the Hessian of the PINN loss function highly ill-conditioned. To address this issue, we introduce a bypassing term that improves numerical stability by substantially reducing the condition number of the Hessian matrix. Furthermore, the small magnitude of the ionic flux \( j \) often leads to a common failure mode where PINNs converge to incorrect solutions. We demonstrate that incorporating a secondary conservation law for the solid-phase potential \( \psi \) effectively prevents such convergence issues and ensures solution accuracy. The proposed methods prove effective for solving both forward and inverse problems involving the BV equation. Specifically, we achieve precise parameter estimation in inverse scenarios and reliable solution predictions for forward simulations.
Model Decides How to Tokenize: Adaptive DNA Sequence Tokenization with MxDNA
Authors: Lifeng Qiao, Peng Ye, Yuchen Ren, Weiqiang Bai, Chaoqi Liang, Xinzhu Ma, Nanqing Dong, Wanli Ouyang
Date: 2024-12-18 00:00:00

Foundation models have made significant strides in understanding the genomic language of DNA sequences. However, previous models typically adopt the tokenization methods designed for natural language, which are unsuitable for DNA sequences due to their unique characteristics. In addition, the optimal approach to tokenize DNA remains largely under-explored, and may not be intuitively understood by humans even if discovered. To address these challenges, we introduce MxDNA, a novel framework where the model autonomously learns an effective DNA tokenization strategy through gradient decent. MxDNA employs a sparse Mixture of Convolution Experts coupled with a deformable convolution to model the tokenization process, with the discontinuous, overlapping, and ambiguous nature of meaningful genomic segments explicitly considered. On Nucleotide Transformer Benchmarks and Genomic Benchmarks, MxDNA demonstrates superior performance to existing methods with less pretraining data and time, highlighting its effectiveness. Finally, we show that MxDNA learns unique tokenization strategy distinct to those of previous methods and captures genomic functionalities at a token level during self-supervised pretraining. Our MxDNA aims to provide a new perspective on DNA tokenization, potentially offering broad applications in various domains and yielding profound insights.
Data-driven Discovery of Biophysical T Cell Receptor Co-specificity Rules
Authors: Andrew G. T. Pyo, Yuta Nagano, Martina Milighetti, James Henderson, Curtis G. Callan Jr., Benny Chain, Ned S. Wingreen, Andreas Tiffeau-Mayer
Date: 2024-12-18 00:00:00
Tasks: Diversity, Specificity

The biophysical interactions between the T cell receptor (TCR) and its ligands determine the specificity of the cellular immune response. However, the immense diversity of receptors and ligands has made it challenging to discover generalizable rules across the distinct binding affinity landscapes created by different ligands. Here, we present an optimization framework for discovering biophysical rules that predict whether TCRs share specificity to a ligand. Applying this framework to TCRs associated with a collection of SARS-CoV-2 peptides we systematically characterize how co-specificity depends on the type and position of amino-acid differences between receptors. We also demonstrate that the inferred rules generalize to ligands highly dissimilar to any seen during training. Our analysis reveals that matching of steric properties between substituted amino acids is more important for receptor co-specificity than the hydrophobic properties that prominently determine evolutionary substitutability. Our analysis also quantifies the substantial importance of positions not in direct contact with the peptide for specificity. These findings highlight the potential for data-driven approaches to uncover the molecular mechanisms underpinning the specificity of adaptive immune responses.
BlendPCR: Seamless and Efficient Rendering of Dynamic Point Clouds captured by Multiple RGB-D Cameras
Authors: Andre Mühlenbrock, Rene Weller, Gabriel Zachmann
Date: 2024-12-01 00:00:00
Proceeding: ICAT-EGVE 2024 12
Tasks: GPU, NeRF

Traditional techniques for rendering continuous surfaces from dynamic, noisy point clouds using multi-camera setups often suffer from disruptive artifacts in overlapping areas, similar to z-fighting. We introduce BlendPCR, an advanced rendering technique that effectively addresses these artifacts through a dual approach of point cloud processing and screen space blending. Additionally, we present a UV coordinate encoding scheme to enable high-resolution texture mapping via standard camera SDKs. We demonstrate that our approach offers superior visual rendering quality over traditional splat and mesh-based methods and exhibits no artifacts in those overlapping areas, which still occur in leading-edge NeRF and Gaussian Splat based approaches like Pointersect and P2ENet. In practical tests with seven Microsoft Azure Kinects, processing, including uploading the point clouds to GPU, requires only 13.8 ms (when using one color per point) or 29.2 ms (using high-resolution color textures), and rendering at a resolution of 3580 x 2066 takes just 3.2 ms, proving its suitability for real-time VR applications.
A Deep Convolutional Neural Network for Time Series Classification with Intermediate Targets
Authors: Aboozar Taherkhani, Georgina Cosma, T. M. McGinnity
Date: 2023-10-28 00:00:00
Proceeding: SN Computer Science 2023 10
Tasks: Time Series, Time Series Classification

Deep Convolutional Neural Networks (CNNs) have been successfully used in different applications, including image recognition. Time series data, which are generated in many applications, such as tasks using sensor data, have different characteristics compared to image data, and accordingly, there is a need for specific CNN structures to address their processing. This paper proposes a new CNN for classifying time series data. It is proposed to have new intermediate outputs extracted from different hidden layers instead of having a single output to control weight adjustment in the hidden layers during training. Intermediate targets are used to act as labels for the intermediate outputs to improve the performance of the method. The intermediate targets are different from the main target. Additionally, the proposed method artificially increases the number of training instances using the original training samples and the intermediate targets. The proposed approach converts a classification task with original training samples to a new (but equivalent) classification task that contains two classes with a high number of training instances. The proposed CNN for Time Series classification, called CNN-TS, extracts features depending the distance of two time series. CNN-TS was evaluated on various benchmark time series datasets. The proposed CNN-TS achieved 5.1% higher overall accuracy compared to the CNN base method (without an intermediate layer). Additionally, CNN-TS achieved 21.1% higher average accuracy compared to classical machine-learning methods, i.e., linear SVM, RBF SVM, and RF. Moreover, CNN-TS was on average 8.43 times faster in training time compared to the ResNet method.
The ROSAT All-Sky Survey Bright Source Catalogue
Authors: Wolfgang Voges, Bernd Aschenbach, Thomas Boller, Heinrich Braeuninger, Ulrich Briel, Wolfgang Burkert, Konrad Dennerl, Jakob Englhauser, Rainer Gruber, Frank Haberl, Gisela Hartner, Guenther Hasinger, Elmar Pfeffermann, Wolfgang Pietsch, Peter Predehl, Cristina Rosso, Juergen H. M. M. Schmitt, Joachim Truemper, Hans-Ulrich Zimmermann
Date: 1999-09-17 00:00:00
Tasks: All, Survey

We present the ROSAT All-Sky Survey Bright Source Catalogue (RASS-BSC, revision 1RXS) derived from the all-sky survey performed during the first half year (1990/91) of the ROSAT mission. 18,811 sources are catalogued (i) down to a limiting ROSAT PSPC count-rate of 0.05 cts/s in the 0.1-2.4 keV energy band, (ii) with a detection likelihood of at least 15 and (iii) at least 15 source counts. The 18,811 sources underwent both an automatic validation and an interactive visual verification process in which for 94% of the sources the results of the standard processing were confirmed. The remaining 6% have been analyzed using interactive methods and these sources have been flagged. Flags are given for (i) nearby sources; (ii) sources with positional errors; (iii) extended sources; (iv) sources showing complex emission structures; and (v) sources which are missed by the standard analysis software. Broad band (0.1-2.4 keV) images are available for sources flagged by (ii), (iii) and (iv). For each source the ROSAT name, position in equatorial coordinates, positional error, source count-rate and error, background count-rate, exposure time, two hardness-ratios and errors, extent and likelihood of extent, likelihood of detection, and the source extraction radius are provided. At a brightness limit of 0.1 cts/s (8,547 sources) the catalogue represents a sky coverage of 92%. The RASS-BSC, the table of possible identification candidates, and the broad band images are available in electronic form (Voges et al. 1996a) via http://wave.xray.mpe.mpg.de/rosat/catalogues/rass-bsc .
Revisiting Weight Averaging for Model Merging
Authors: Jiho Choi, Donggyun Kim, Chanhyuk Lee, Seunghoon Hong
Date: 2024-12-11 00:00:00
Tasks: image-classification, Image Classification, model, Multi-Task Learning

Model merging aims to build a multi-task learner by combining the parameters of individually fine-tuned models without additional training. While a straightforward approach is to average model parameters across tasks, this often results in suboptimal performance due to interference among parameters across tasks. In this paper, we present intriguing results that weight averaging implicitly induces task vectors centered around the weight averaging itself and that applying a low-rank approximation to these centered task vectors significantly improves merging performance. Our analysis shows that centering the task vectors effectively separates core task-specific knowledge and nuisance noise within the fine-tuned parameters into the top and lower singular vectors, respectively, allowing us to reduce inter-task interference through its low-rank approximation. We evaluate our method on eight image classification tasks, demonstrating that it outperforms prior methods by a significant margin, narrowing the performance gap with traditional multi-task learning to within 1-3%
The IBEX Imaging Knowledge-Base: A Community Resource Enabling Adoption and Development of Immunofluoresence Imaging Methods
Authors: Ziv Yaniv, Ifeanyichukwu U. Anidi, Leanne Arakkal, Armando J. Arroyo-Mejías, Rebecca T. Beuschel, Katy Börner, Colin J. Chu, Beatrice Clark, Menna R. Clatworthy, Jake Colautti, Fabian Coscia, Joshua Croteau, Saven Denha, Rose Dever, Walderez O. Dutra, Sonja Fritzsche, Spencer Fullam, Michael Y. Gerner, Anita Gola, Kenneth J. Gollob, Jonathan M. Hernandez, Jyh Liang Hor, Hiroshi Ichise, Zhixin Jing, Danny Jonigk, Evelyn Kandov, Wolfgang Kastenmüller, Joshua F. E. Koenig, Aanandita Kothurkar, Rosa K. Kortekaas, Alexandra Y. Kreins, Ian T. Lamborn, Yuri Lin, Katia Luciano Pereira Morais, Aleksandra Lunich, Jean C. S. Luz, Ryan B. MacDonald, Chen Makranz, Vivien I. Maltez, John E. McDonough, Ryan V. Moriarty, Juan M. Ocampo-Godinez, Vitoria M. Olyntho, Annette Oxenius, Kartika Padhan, Kirsten Remmert, Nathan Richoz, Edward C. Schrom, Wanjing Shang, Lihong Shi, Rochelle M. Shih, Emily Speranza, Salome Stierli, Sarah A. Teichmann, Tibor Z. Veres, Megan Vierhout, Brianna T. Wachter, Margaret Williams, Nathan Zangger, Ronald N. Germain, Andrea J. Radtke
Date: 2024-12-17 00:00:00
Tasks: scientific discovery

The iterative bleaching extends multiplexity (IBEX) Knowledge-Base is a central portal for researchers adopting IBEX and related 2D and 3D immunofluorescence imaging methods. The design of the Knowledge-Base is modeled after efforts in the open-source software community and includes three facets: a development platform (GitHub), static website, and service for data archiving. The Knowledge-Base facilitates the practice of open science throughout the research life cycle by providing validation data for recommended and non-recommended reagents, e.g., primary and secondary antibodies. In addition to reporting negative data, the Knowledge-Base empowers method adoption and evolution by providing a venue for sharing protocols, videos, datasets, software, and publications. A dedicated discussion forum fosters a sense of community among researchers while addressing questions not covered in published manuscripts. Together, scientists from around the world are advancing scientific discovery at a faster pace, reducing wasted time and effort, and instilling greater confidence in the resulting data.
Towards a (meta-)mathematical theory of consciousness: universal (mapping) properties of experience
Authors: Steven Phillips, Naotsugu Tsuchiya
Date: 2024-12-13 00:00:00

Conscious (subjective) experience permeates our daily lives, yet general consensus on a theory of consciousness remains elusive. Integrated Information Theory (IIT) is a prominent approach that asserts the existence of subjective experience (0th axiom), from an intrinsic system of causally related units, and five essential properties (axioms 1-5): intrinsicality, information, integration, exclusion and composition. However, despite empirical support for some aspects of IIT, the supposed necessity of these axioms is unclear given their informal presentation and operationalized dependence on a specific mathematical instantiation as the so-called postulates. The category theory approach presented here attempts to redress this situation. Category theory is a kind of meta-mathematics invented to make relations between formal structures formally precise and so facilitate doing "ordinary" mathematics. In this way, the five essential properties for consciousness are organized around a smaller number of meta-mathematical principles for comparison with IIT. In particular, category theory characterizes mathematical structures by their "universal mapping properties" -- a unique-existence condition for all instances of the structure. Accordingly, axioms 1-5 pertain to universal mapping properties for experience, whence the slogan, "Consciousness is a universal property."
Organizational culture and the usage of Industry 4.0 technologies: evidence from Swiss businesses
Authors: Simon Alexander Wiese, Johannes Lehmann, Michael Beckmann
Date: 2024-12-17 00:00:00
Tasks: Descriptive

Using novel establishment-level observational data from Switzerland, we empirically examine whether the usage of key technologies of Industry 4.0 distinguishes across firms with different types of organizational culture. Based on the Technology-Organization-Environment and the Competing Values framework, we hypothesize that the developmental culture has the greatest potential to promote the usage of Industry 4.0 technologies. We also hypothesize that companies with a hierarchical or rational culture are especially likely to make use of automation technologies, such as AI and robotics. By means of descriptive statistics and multiple regression analysis, we find empirical support for our first hypothesis, while we cannot con-firm our second hypothesis. Our empirical results provide important implications for managerial decision-makers. Specifically, the link between organizational culture and the implementation of Industry 4.0 technologies is relevant for managers, as this knowledge helps them to cope with digital transformation in turbulent times and keep their businesses competitive.