OpenRaiser/Intern-Atlas huggingface dataset:
Konzeption und Umsetzung einer mobilen Applikation zur Validierung von fälschungssicheren Produktlabeln
Venue: arXiv cs.CR 2023
Year: 2023

Due to increasing numbers of product piracy worldwide, a cost-effective method for verifying the origin of a product is to be developed. For this purpose, a certificate of authenticity can be created using precisely measurable, unique properties of special physical objects that are difficult to reconstruct. In the context of the present work, this is a counterfeit-proof label composed of randomly distributed gold nanospheres or rods in a semi-transparent material. The characteristic positioning of the label's elements can be precisely measured using a smartphone's camera and additional technologies. This can create an offline usable verification method for the general public without the need for an existing network connection. The present work provides a first part of the proof of concept that such a system and especially the associated algorithmic computation method can be implemented and efficiently used in a mobile application. In addition, a method suitable in practice for transmitting and securing the required information is determined in each case. Furthermore, the results of the validation of counterfeit-proof product labels are analyzed in detail and existing weaknesses are pointed out. -- Auf Grund weltweit steigender Zahlen der Produktpiraterie soll ein kostengünstiges Verfahren zur Verifizierung der Herkunft eines Produktes entwickelt werden. Dafür lässt sich durch exakt messbare, einzigartige, jedoch schwer rekonstruierbare Eigenschaften spezieller physischer Objekte ein Echtheitszertifikat kreieren. Dieses ist im Kontext der vorliegenden Arbeit ein fälschungssicheres Label, das sich in einem semi-transparenten Material aus zufällig verteilten Goldnanokügelchen oder -stäbchen zusammensetzt. Die charakteristischen Positionierungen der Elemente des Labels lassen sich mit der Kamera eines Smartphones und zusätzlichen Technologien präzise messen. Dadurch kann für die breite Bevölkerung ohne die Notwendigkeit einer bestehenden Netzwerkverbindung ein offline verwendbares Verifikationsverfahren erschaffen werden. Die vorliegende Arbeit liefert einen ersten Teil des Machbarkeitsnachweises, dass ein derartiges System und insbesondere das damit einhergehende algorithmische Berechnungsverfahren in einer mobilen Applikation implementier -- und effizient einsetzbar ist. Zudem wird je eine in der Praxis geeignete Methode zur Übermittlung und Sicherung der benötigten Informationen eruiert. Des Weiteren werden die Resultate der Validierung von fälschungssicheren Produktlabeln ausführlich analysiert und vorhandene Schwächen aufgezeigt.
ElegansNet: a brief scientific report and initial experiments
Venue: arXiv cs.NE 2023
Year: 2023

This research report introduces ElegansNet, a neural network that mimics real-world neuronal network circuitry, with the goal of better understanding the interplay between connectome topology and deep learning systems. The proposed approach utilizes the powerful representational capabilities of living beings' neuronal circuitry to design and generate improved deep learning systems with a topology similar to natural networks. The Caenorhabditis elegans connectome is used as a reference due to its completeness, reasonable size, and functional neuron classes annotations. It is demonstrated that the connectome of simple organisms exhibits specific functional relationships between neurons, and once transformed into learnable tensor networks and integrated into modern architectures, it offers bio-plausible structures that efficiently solve complex tasks. The performance of the models is demonstrated against randomly wired networks and compared to artificial networks ranked on global benchmarks. In the first case, ElegansNet outperforms randomly wired networks. Interestingly, ElegansNet models show slightly similar performance with only those based on the Watts-Strogatz small-world property. When compared to state-of-the-art artificial neural networks, such as transformers or attention-based autoencoders, ElegansNet outperforms well-known deep learning and traditional models in both supervised image classification tasks and unsupervised hand-written digits reconstruction, achieving top-1 accuracy of 99.99% on Cifar10 and 99.84% on MNIST Unsup on the validation sets.
Byzantine-Resilient Learning Beyond Gradients: Distributing Evolutionary Search
Venue: arXiv cs.DC 2023
Year: 2023

Modern machine learning (ML) models are capable of impressive performances. However, their prowess is not due only to the improvements in their architecture and training algorithms but also to a drastic increase in computational power used to train them. Such a drastic increase led to a growing interest in distributed ML, which in turn made worker failures and adversarial attacks an increasingly pressing concern. While distributed byzantine resilient algorithms have been proposed in a differentiable setting, none exist in a gradient-free setting. The goal of this work is to address this shortcoming. For that, we introduce a more general definition of byzantine-resilience in ML - the \textit{model-consensus}, that extends the definition of the classical distributed consensus. We then leverage this definition to show that a general class of gradient-free ML algorithms - ($1,λ$)-Evolutionary Search - can be combined with classical distributed consensus algorithms to generate gradient-free byzantine-resilient distributed learning algorithms. We provide proofs and pseudo-code for two specific cases - the Total Order Broadcast and proof-of-work leader election.
Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning
Venue: arXiv cs.LG 2023
Year: 2023

Communication efficiency and privacy protection are two critical issues in distributed machine learning. Existing methods tackle these two issues separately and may have a high implementation complexity that constrains their application in a resource-limited environment. We propose a comprehensive quantization-based solution that could simultaneously achieve communication efficiency and privacy protection, providing new insights into the correlated nature of communication and privacy. Specifically, we demonstrate the effectiveness of our proposed solutions in the distributed stochastic gradient descent (SGD) framework by adding binomial noise to the uniformly quantized gradients to reach the desired differential privacy level but with a minor sacrifice in communication efficiency. We theoretically capture the new trade-offs between communication, privacy, and learning performance.
Turning block-sequential automata networks into smaller parallel networks with isomorphic limit dynamics
Venue: arXiv cs.DM 2023
Year: 2023

We state an algorithm that, given an automata network and a block-sequential update schedule, produces an automata network of the same size or smaller with the same limit dynamics under the parallel update schedule. Then, we focus on the family of automata cycles which share a unique path of automata, called tangential cycles, and show that a restriction of our algorithm allows to reduce any instance of these networks under a block-sequential update schedule into a smaller parallel network of the family and to characterize the number of reductions operated while conserving their limit dynamics. We also show that any tangential cycles reduced by our main algorithm are transformed into a network whose size is that of the largest cycle of the initial network. We end by showing that the restricted algorithm allows the direct characterization of block-sequential double cycles as parallel ones.
On the Order of Power Series and the Sum of Square Roots Problem
Venue: arXiv cs.CC 2023
Year: 2023

This paper focuses on the study of the order of power series that are linear combinations of a given finite set of power series. The order of a formal power series, known as $\textrm{ord}(f)$, is defined as the minimum exponent of $x$ that has a non-zero coefficient in $f(x)$. Our first result is that the order of the Wronskian of these power series is equivalent up to a polynomial factor, to the maximum order which occurs in the linear combination of these power series. This implies that the Wronskian approach used in (Kayal and Saha, TOCT'2012) to upper bound the order of sum of square roots is optimal up to a polynomial blowup. We also demonstrate similar upper bounds, similar to those of (Kayal and Saha, TOCT'2012), for the order of power series in a variety of other scenarios. We also solve a special case of the inequality testing problem outlined in (Etessami et al., TOCT'2014). In the second part of the paper, we study the equality variant of the sum of square roots problem, which is decidable in polynomial time due to (Blömer, FOCS'1991). We investigate a natural generalization of this problem when the input integers are given as straight line programs. Under the assumption of the Generalized Riemann Hypothesis (GRH), we show that this problem can be reduced to the so-called ``one dimensional'' variant. We identify the key mathematical challenges for solving this ``one dimensional'' variant.
ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summaries
Venue: arXiv cs.CL 2023
Year: 2023

Automatic chart to text summarization is an effective tool for the visually impaired people along with providing precise insights of tabular data in natural language to the user. A large and well-structured dataset is always a key part for data driven models. In this paper, we propose ChartSumm: a large-scale benchmark dataset consisting of a total of 84,363 charts along with their metadata and descriptions covering a wide range of topics and chart types to generate short and long summaries. Extensive experiments with strong baseline models show that even though these models generate fluent and informative summaries by achieving decent scores in various automatic evaluation metrics, they often face issues like suffering from hallucination, missing out important data points, in addition to incorrect explanation of complex trends in the charts. We also investigated the potential of expanding ChartSumm to other languages using automated translation tools. These make our dataset a challenging benchmark for future research.
Using Implicit Feedback to Improve Question Generation
Venue: arXiv cs.CL 2023
Year: 2023

Question Generation (QG) is a task of Natural Language Processing (NLP) that aims at automatically generating questions from text. Many applications can benefit from automatically generated questions, but often it is necessary to curate those questions, either by selecting or editing them. This task is informative on its own, but it is typically done post-generation, and, thus, the effort is wasted. In addition, most existing systems cannot incorporate this feedback back into them easily. In this work, we present a system, GEN, that learns from such (implicit) feedback. Following a pattern-based approach, it takes as input a small set of sentence/question pairs and creates patterns which are then applied to new unseen sentences. Each generated question, after being corrected by the user, is used as a new seed in the next iteration, so more patterns are created each time. We also take advantage of the corrections made by the user to score the patterns and therefore rank the generated questions. Results show that GEN is able to improve by learning from both levels of implicit feedback when compared to the version with no learning, considering the top 5, 10, and 20 questions. Improvements go up from 10%, depending on the metric and strategy used.
UniNeXt: Exploring A Unified Architecture for Vision Recognition
Venue: arXiv cs.CV 2023
Year: 2023

Vision Transformers have shown great potential in computer vision tasks. Most recent works have focused on elaborating the spatial token mixer for performance gains. However, we observe that a well-designed general architecture can significantly improve the performance of the entire backbone, regardless of which spatial token mixer is equipped. In this paper, we propose UniNeXt, an improved general architecture for the vision backbone. To verify its effectiveness, we instantiate the spatial token mixer with various typical and modern designs, including both convolution and attention modules. Compared with the architecture in which they are first proposed, our UniNeXt architecture can steadily boost the performance of all the spatial token mixers, and narrows the performance gap among them. Surprisingly, our UniNeXt equipped with naive local window attention even outperforms the previous state-of-the-art. Interestingly, the ranking of these spatial token mixers also changes under our UniNeXt, suggesting that an excellent spatial token mixer may be stifled due to a suboptimal general architecture, which further shows the importance of the study on the general architecture of vision backbone.
Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery
Venue: arXiv cs.AI 2023
Year: 2023

Despite growing interest in using large language models (LLMs) in healthcare, current explorations do not assess the real-world utility and safety of LLMs in clinical settings. Our objective was to determine whether two LLMs can serve information needs submitted by physicians as questions to an informatics consultation service in a safe and concordant manner. Sixty six questions from an informatics consult service were submitted to GPT-3.5 and GPT-4 via simple prompts. 12 physicians assessed the LLM responses' possibility of patient harm and concordance with existing reports from an informatics consultation service. Physician assessments were summarized based on majority vote. For no questions did a majority of physicians deem either LLM response as harmful. For GPT-3.5, responses to 8 questions were concordant with the informatics consult report, 20 discordant, and 9 were unable to be assessed. There were 29 responses with no majority on "Agree", "Disagree", and "Unable to assess". For GPT-4, responses to 13 questions were concordant, 15 discordant, and 3 were unable to be assessed. There were 35 responses with no majority. Responses from both LLMs were largely devoid of overt harm, but less than 20% of the responses agreed with an answer from an informatics consultation service, responses contained hallucinated references, and physicians were divided on what constitutes harm. These results suggest that while general purpose LLMs are able to provide safe and credible responses, they often do not meet the specific information need of a given question. A definitive evaluation of the usefulness of LLMs in healthcare settings will likely require additional research on prompt engineering, calibration, and custom-tailoring of general purpose models.
Association Rules Mining with Auto-Encoders
Venue: arXiv cs.LG 2023
Year: 2023

Association rule mining is one of the most studied research fields of data mining, with applications ranging from grocery basket problems to explainable classification systems. Classical association rule mining algorithms have several limitations, especially with regards to their high execution times and number of rules produced. Over the past decade, neural network solutions have been used to solve various optimization problems, such as classification, regression or clustering. However there are still no efficient way association rules using neural networks. In this paper, we present an auto-encoder solution to mine association rule called ARM-AE. We compare our algorithm to FP-Growth and NSGAII on three categorical datasets, and show that our algorithm discovers high support and confidence rule set and has a better execution time than classical methods while preserving the quality of the rule set produced.
GPU accelerated matrix factorization of large scale data using block based approach
Venue: arXiv cs.LG 2023
Year: 2023

Matrix Factorization (MF) on large scale data takes substantial time on a Central Processing Unit (CPU). While Graphical Processing Unit (GPU)s could expedite the computation of MF, the available memory on a GPU is finite. Leveraging GPUs require alternative techniques that allow not only parallelism but also address memory limitations. Synchronization between computation units, isolation of data related to a computational unit, sharing of data between computational units and identification of independent tasks among computational units are some of the challenges while leveraging GPUs for MF. We propose a block based approach to matrix factorization using Stochastic Gradient Descent (SGD) that is aimed at accelerating MF on GPUs. The primary motivation for the approach is to make it viable to factorize extremely large data sets on limited hardware without having to compromise on results. The approach addresses factorization of large scale data by identifying independent blocks, each of which are factorized in parallel using multiple computational units. The approach can be extended to one or more GPUs and even to distributed systems. The RMSE results of the block based approach are with in acceptable delta in comparison to the results of CPU based variant and multi-threaded CPU variant of similar SGD kernel implementation. The advantage, of the block based variant, in-terms of speed are significant in comparison to other variants.
Lane Change Intention Recognition and Vehicle Status Prediction for Autonomous Vehicles
Venue: arXiv cs.LG 2023
Year: 2023

Accurately detecting and predicting lane change (LC)processes of human-driven vehicles can help autonomous vehicles better understand their surrounding environment, recognize potential safety hazards, and improve traffic safety. This paper focuses on LC processes, first developing a temporal convolutional network with an attention mechanism (TCN-ATM) model to recognize LC intention. Considering the intrinsic relationship among output variables, the Multi-task Learning (MTL)framework is employed to simultaneously predict multiple LC vehicle status indicators. Furthermore, a unified modeling framework for LC intention recognition and driving status prediction (LC-IR-SP) is developed. The results indicate that the classification accuracy of LC intention was improved from 96.14% to 98.20% when incorporating the attention mechanism into the TCN model. For LC vehicle status prediction issues, three multi-tasking learning models are constructed based on MTL framework. The results indicate that the MTL-LSTM model outperforms the MTL-TCN and MTL-TCN-ATM models. Compared to the corresponding single-task model, the MTL-LSTM model demonstrates an average decrease of 26.04% in MAE and 25.19% in RMSE.
Scalable, Distributed AI Frameworks: Leveraging Cloud Computing for Enhanced Deep Learning Performance and Efficiency
Venue: arXiv cs.LG 2023
Year: 2023

In recent years, the integration of artificial intelligence (AI) and cloud computing has emerged as a promising avenue for addressing the growing computational demands of AI applications. This paper presents a comprehensive study of scalable, distributed AI frameworks leveraging cloud computing for enhanced deep learning performance and efficiency. We first provide an overview of popular AI frameworks and cloud services, highlighting their respective strengths and weaknesses. Next, we delve into the critical aspects of data storage and management in cloud-based AI systems, discussing data preprocessing, feature engineering, privacy, and security. We then explore parallel and distributed training techniques for AI models, focusing on model partitioning, communication strategies, and cloud-based training architectures. In subsequent chapters, we discuss optimization strategies for AI workloads in the cloud, covering load balancing, resource allocation, auto-scaling, and performance benchmarking. We also examine AI model deployment and serving in the cloud, outlining containerization, serverless deployment options, and monitoring best practices. To ensure the cost-effectiveness of cloud-based AI solutions, we present a thorough analysis of costs, optimization strategies, and case studies showcasing successful deployments. Finally, we summarize the key findings of this study, discuss the challenges and limitations of cloud-based AI, and identify emerging trends and future research opportunities in the field.
Fine Tuning with Abnormal Examples
Venue: arXiv cs.CL 2023
Year: 2023

Given the prevalence of crowd sourced labor in creating Natural Language processing datasets, these aforementioned sets have become increasingly large. For instance, the SQUAD dataset currently sits at over 80,000 records. However, because the English language is rather repetitive in structure, the distribution of word frequencies in the SQUAD dataset's contexts are relatively unchanged. By measuring each sentences distance from the co-variate distance of frequencies of all sentences in the dataset, we identify 10,500 examples that create a more uniform distribution for training. While fine-tuning ELECTRA [4] on this subset of examples reaches better performance to a model trained on all 87,000 examples. Herein we introduce a methodology for systematically pruning datasets for fine tuning reaching better out of sample performance.
Physics-informed neural networks for predicting gas flow dynamics and unknown parameters in diesel engines
Venue: arXiv cs.LG 2023
Year: 2023

This paper presents a physics-informed neural network (PINN) approach for monitoring the health of diesel engines. The aim is to evaluate the engine dynamics, identify unknown parameters in a "mean value" model, and anticipate maintenance requirements. The PINN model is applied to diesel engines with a variable-geometry turbocharger and exhaust gas recirculation, using measurement data of selected state variables. The results demonstrate the ability of the PINN model to predict simultaneously both unknown parameters and dynamics accurately with both clean and noisy data, and the importance of the self-adaptive weight in the loss function for faster convergence. The input data for these simulations are derived from actual engine running conditions, while the outputs are simulated data, making this a practical case study of PINN's ability to predict real-world dynamical systems. The mean value model of the diesel engine incorporates empirical formulae to represent certain states, but these formulae may not be generalizable to other engines. To address this, the study considers the use of deep neural networks (DNNs) in addition to the PINN model. The DNNs are trained using laboratory test data and are used to model the engine-specific empirical formulae in the mean value model, allowing for a more flexible and adaptive representation of the engine's states. In other words, the mean value model uses both the PINN model and the DNNs to represent the engine's states, with the PINN providing a physics-based understanding of the engine's overall dynamics and the DNNs offering a more engine-specific and adaptive representation of the empirical formulae. By combining these two approaches, the study aims to offer a comprehensive and versatile approach to monitoring the health and performance of diesel engines.