pwc-archive/papers-with-abstracts huggingface dataset:
Emergent predictability in microbial ecosystems
Authors: Jacob Moran, Mikhail Tikhonov
Date: 2024-03-28 00:00:00

Microbial ecosystems carry out essential functions for global climate, human health, and industry. These complex communities exhibit a surprising amount of functionally relevant diversity at all levels of taxonomic resolution, presenting a significant challenge for most modeling frameworks. A long-standing hope of theoretical ecology is that some patterns might persist despite community complexity -- or perhaps even emerge because of it. A deeper understanding of such "emergent simplicity" could enable new approaches for predicting the behaviors of the complex ecosystems in nature. However, most examples described so far afford limited predictive power, as they focused on reproducibility rather than prediction. Here, we propose an information-theoretic framework for defining, nuancing and quantifying emergent simplicity in empirical data based on the ability of simple models to predict community-level functional properties. Applying this framework to two published datasets, we demonstrate that the majority of properties measured across both experiments exhibit robust evidence of emergent predictability: surprisingly, as community richness increases, simple compositional descriptions become more predictive. We show that this behavior is not typical within the standard modeling frameworks of theoretical ecology, and argue that improving our ability to predict and control natural microbial communities will require a shift of focus: away from complexity of _ecosystems_, and towards prediction complexity of _properties_ of ecosystems.
Real-time Geoinformation Systems to Improve the Quality, Scalability, and Cost of Internet of Things for Agri-environment Research
Authors: Bryan C. Runck, Bobby Schulz, Jeff Bishop, Nathan Carlson, Bryan Chantigian, Gary Deters, Jesse Erdmann, Patrick M. Ewing, Michael Felzan, Xiao Fu, Jan Greyling, Christopher J. Hogan, Andrew Hollman, Ali Joglekar, Kris Junker, Michael Kantar, Lumbani Kaunda, Mohana Krishna, Benjamin Lynch, Peter Marchetto, Megan Marsolek, Troy McKay, Brad Morris, Ali Rashid Niaghi, Keerthi Pamulaparthy, Philip Pardey, Ann Piotrowski, Christina Poudyal, Tom Prather, Barath Raghavan, Maggie Reiter, Lucas Rosen, Benjamin Salazar, Andrew Scobbie, Vasudha Sharma, Kevin A. T. Silverstein, Gurparteet Singh, Jeff Strock, Samikshya Subedi, Evan Tang, Gianna Turturillo, Eric Watkins, Blake Webster, James Wilgenbusch
Date: 2024-03-28 00:00:00

With the increasing emphasis on machine learning and artificial intelligence to drive knowledge discovery in the agricultural sciences, spatial internet of things (IoT) technologies have become increasingly important for collecting real-time, high resolution data for these models. However, managing large fleets of devices while maintaining high data quality remains an ongoing challenge as scientists iterate from prototype to mature end-to-end applications. Here, we provide a set of case studies using the framework of technology readiness levels for an open source spatial IoT system. The spatial IoT systems underwent 3 major and 14 minor system versions, had over 2,727 devices manufactured both in academic and commercial contexts, and are either in active or planned deployment across four continents. Our results show the evolution of a generalizable, open source spatial IoT system designed for agricultural scientists, and provide a model for academic researchers to overcome the challenges that exist in going from one-off prototypes to thousands of internet-connected devices.
Artificial Intelligence-based Analysis of Change in Public Finance between US and International Markets
Authors: Kapil Panda
Date: 2023-12-10 00:00:00

Public finances are one of the fundamental mechanisms of economic governance that refer to the financial activities and decisions made by government entities to fund public services, projects, and operations through assets. In today's globalized landscape, even subtle shifts in one nation's public debt landscape can have significant impacts on that of international finances, necessitating a nuanced understanding of the correlations between international and national markets to help investors make informed investment decisions. Therefore, by leveraging the capabilities of artificial intelligence, this study utilizes neural networks to depict the correlations between US and International Public Finances and predict the changes in international public finances based on the changes in US public finances. With the neural network model achieving a commendable Mean Squared Error (MSE) value of 2.79, it is able to affirm a discernible correlation and also plot the effect of US market volatility on international markets. To further test the accuracy and significance of the model, an economic analysis was conducted that aimed to correlate the changes seen by the results of the model with historical stock market changes. This model demonstrates significant potential for investors to predict changes in international public finances based on signals from US markets, marking a significant stride in comprehending the intricacies of global public finances and the role of artificial intelligence in decoding its multifaceted patterns for practical forecasting.
Long Short-Term Memory Pattern Recognition in Currency Trading
Authors: Jai Pal
Date: 2024-02-23 00:00:00
Tasks: Financial Analysis

This study delves into the analysis of financial markets through the lens of Wyckoff Phases, a framework devised by Richard D. Wyckoff in the early 20th century. Focusing on the accumulation pattern within the Wyckoff framework, the research explores the phases of trading range and secondary test, elucidating their significance in understanding market dynamics and identifying potential trading opportunities. By dissecting the intricacies of these phases, the study sheds light on the creation of liquidity through market structure, offering insights into how traders can leverage this knowledge to anticipate price movements and make informed decisions. The effective detection and analysis of Wyckoff patterns necessitate robust computational models capable of processing complex market data, with spatial data best analyzed using Convolutional Neural Networks (CNNs) and temporal data through Long Short-Term Memory (LSTM) models. The creation of training data involves the generation of swing points, representing significant market movements, and filler points, introducing noise and enhancing model generalization. Activation functions, such as the sigmoid function, play a crucial role in determining the output behavior of neural network models. The results of the study demonstrate the remarkable efficacy of deep learning models in detecting Wyckoff patterns within financial data, underscoring their potential for enhancing pattern recognition and analysis in financial markets. In conclusion, the study highlights the transformative potential of AI-driven approaches in financial analysis and trading strategies, with the integration of AI technologies shaping the future of trading and investment practices.
DeepTraderX: Challenging Conventional Trading Strategies with Deep Learning in Multi-Threaded Market Simulations
Authors: Armand Mihai Cismaru
Date: 2024-02-06 00:00:00

In this paper, we introduce DeepTraderX (DTX), a simple Deep Learning-based trader, and present results that demonstrate its performance in a multi-threaded market simulation. In a total of about 500 simulated market days, DTX has learned solely by watching the prices that other strategies produce. By doing this, it has successfully created a mapping from market data to quotes, either bid or ask orders, to place for an asset. Trained on historical Level-2 market data, i.e., the Limit Order Book (LOB) for specific tradable assets, DTX processes the market state $S$ at each timestep $T$ to determine a price $P$ for market orders. The market data used in both training and testing was generated from unique market schedules based on real historic stock market data. DTX was tested extensively against the best strategies in the literature, with its results validated by statistical analysis. Our findings underscore DTX's capability to rival, and in many instances, surpass, the performance of public-domain traders, including those that outclass human traders, emphasising the efficiency of simple models, as this is required to succeed in intricate multi-threaded simulations. This highlights the potential of leveraging "black-box" Deep Learning systems to create more efficient financial markets.
Enhancing Financial Data Visualization for Investment Decision-Making
Authors: Nisarg Patel, Harmit Shah, Kishan Mewada
Date: 2023-12-09 00:00:00
Tasks: Data Visualization, Decision Making, feature selection, Stock Market Prediction, Time Series Forecasting

Navigating the intricate landscape of financial markets requires adept forecasting of stock price movements. This paper delves into the potential of Long Short-Term Memory (LSTM) networks for predicting stock dynamics, with a focus on discerning nuanced rise and fall patterns. Leveraging a dataset from the New York Stock Exchange (NYSE), the study incorporates multiple features to enhance LSTM's capacity in capturing complex patterns. Visualization of key attributes, such as opening, closing, low, and high prices, aids in unraveling subtle distinctions crucial for comprehensive market understanding. The meticulously crafted LSTM input structure, inspired by established guidelines, incorporates both price and volume attributes over a 25-day time step, enabling the model to capture temporal intricacies. A comprehensive methodology, including hyperparameter tuning with Grid Search, Early Stopping, and Callback mechanisms, leads to a remarkable 53% improvement in predictive accuracy. The study concludes with insights into model robustness, contributions to financial forecasting literature, and a roadmap for real-time stock market prediction. The amalgamation of LSTM networks, strategic hyperparameter tuning, and informed feature selection presents a potent framework for advancing the accuracy of stock price predictions, contributing substantively to financial time series forecasting discourse.
JEP-KD: Joint-Embedding Predictive Architecture Based Knowledge Distillation for Visual Speech Recognition
Authors: Chang Sun, Hong Yang, Bo Qin
Date: 2024-03-04 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Knowledge Distillation, speech-recognition, Speech Recognition, Visual Speech Recognition

Visual Speech Recognition (VSR) tasks are generally recognized to have a lower theoretical performance ceiling than Automatic Speech Recognition (ASR), owing to the inherent limitations of conveying semantic information visually. To mitigate this challenge, this paper introduces an advanced knowledge distillation approach using a Joint-Embedding Predictive Architecture (JEPA), named JEP-KD, designed to more effectively utilize audio features during model training. Central to JEP-KD is the inclusion of a generative network within the embedding layer, which enhances the video encoder's capacity for semantic feature extraction and brings it into closer alignment with the audio features from a pre-trained ASR model's encoder. This approach aims to progressively reduce the performance gap between VSR and ASR. Moreover, a comprehensive multimodal, multistage training regimen for the JEP-KD framework is established, bolstering the robustness and efficacy of the training process. Experiment results demonstrate that JEP-KD significantly improves the performance of VSR models and demonstrates versatility across different VSR platforms, indicating its potential for broader application within other multimodal tasks.
XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection
Authors: Yuanhang Yang, shiyi qi, Wenchao Gu, Chaozheng Wang, Cuiyun Gao, Zenglin Xu
Date: 2024-02-27 00:00:00
Tasks: Language Modeling, Language Modelling, Machine Translation, Mixture-of-Experts

Sparse models, including sparse Mixture-of-Experts (MoE) models, have emerged as an effective approach for scaling Transformer models. However, they often suffer from computational inefficiency since a significant number of parameters are unnecessarily involved in computations via multiplying values by zero or low activation values. To address this issue, we present \tool, a novel MoE designed to enhance both the efficacy and efficiency of sparse MoE models. \tool leverages small experts and a threshold-based router to enable tokens to selectively engage only essential parameters. Our extensive experiments on language modeling and machine translation tasks demonstrate that \tool can enhance model performance while decreasing the computation load at MoE layers by over 50\% without sacrificing performance. Furthermore, we present the versatility of \tool by applying it to dense models, enabling sparse computation during inference. We provide a comprehensive analysis and make our code available at https://github.com/ysngki/XMoE.
Frog-Snake prey-predation Relationship Optimization (FSRO) : A novel nature-inspired metaheuristic algorithm for feature selection
Authors: Hayata Saitou, Harumi Haraguchi
Date: 2024-02-13 00:00:00
Tasks: feature selection

Swarm intelligence algorithms have traditionally been designed for continuous optimization problems, and these algorithms have been modified and extended for application to discrete optimization problems. Notably, their application in feature selection for machine learning has demonstrated improvements in model accuracy, reduction of unnecessary data, and decreased computational time. This study proposes the Frog-Snake prey-predation Relationship Optimization (FSRO) algorithm, inspired by the prey-predation relationship between frogs and snakes for application to discrete optimization problems. The algorithm models three stages of a snake's foraging behavior "search", "approach", and "capture" as well as the frog's characteristic behavior of staying still to attract and then escaping. Furthermore, the introduction of the concept of evolutionary game theory enables dynamic control of the search process. The proposed algorithm conducts computational experiments on feature selection using 26 types of machine learning datasets to analyze its performance and identify improvements. In computer experiments, the proposed algorithm showed better performance than the comparison algorithms in terms of the best and standard deviation of fitness value and Accuracy. It was also proved that dynamic search control by evolutionary game theory is an effective method, and the proposed algorithm has the ability of a well-balanced search, achieving the two objectives of improving accuracy and reducing data.
Evolution-based Feature Selection for Predicting Dissolved Oxygen Concentrations in Lakes
Authors: Runlong Yu, Robert Ladwig, Xiang Xu, Peijun Zhu, Paul C. Hanson, Yiqun Xie, Xiaowei Jia
Date: 2024-02-15 00:00:00
Tasks: feature selection

Accurate prediction of dissolved oxygen (DO) concentrations in lakes requires a comprehensive study of phenological patterns across ecosystems, highlighting the need for precise selection of interactions amongst external factors and internal physical-chemical-biological variables. This paper presents the Multi-population Cognitive Evolutionary Search (MCES), a novel evolutionary algorithm for complex feature interaction selection problems. MCES allows models within every population to evolve adaptively, selecting relevant feature interactions for different lake types and tasks. Evaluated on diverse lakes in the Midwestern USA, MCES not only consistently produces accurate predictions with few observed labels but also, through gene maps of models, reveals sophisticated phenological patterns of different lake types, embodying the innovative concept of "AI from nature, for nature".
A Review of Neuroscience-Inspired Machine Learning
Authors: Alexander Ororbia, Ankur Mali, Adam Kohan, Beren Millidge, Tommaso Salvatori
Date: 2024-02-16 00:00:00

One major criticism of deep learning centers around the biological implausibility of the credit assignment schema used for learning -- backpropagation of errors. This implausibility translates into practical limitations, spanning scientific fields, including incompatibility with hardware and non-differentiable implementations, thus leading to expensive energy requirements. In contrast, biologically plausible credit assignment is compatible with practically any learning condition and is energy-efficient. As a result, it accommodates hardware and scientific modeling, e.g. learning with physical systems and non-differentiable behavior. Furthermore, it can lead to the development of real-time, adaptive neuromorphic processing systems. In addressing this problem, an interdisciplinary branch of artificial intelligence research that lies at the intersection of neuroscience, cognitive science, and machine learning has emerged. In this paper, we survey several vital algorithms that model bio-plausible rules of credit assignment in artificial neural networks, discussing the solutions they provide for different scientific fields as well as their advantages on CPUs, GPUs, and novel implementations of neuromorphic hardware. We conclude by discussing the future challenges that will need to be addressed in order to make such algorithms more useful in practical applications.
Bridging Generative Networks with the Common Model of Cognition
Authors: Robert L. West, Spencer Eckler, Brendan Conway-Smith, Nico Turcas, Eilene Tomkins-Flanagan, Mary Alexandria Kelly
Date: 2024-01-25 00:00:00

This article presents a theoretical framework for adapting the Common Model of Cognition to large generative network models within the field of artificial intelligence. This can be accomplished by restructuring modules within the Common Model into shadow production systems that are peripheral to a central production system, which handles higher-level reasoning based on the shadow productions' output. Implementing this novel structure within the Common Model allows for a seamless connection between cognitive architectures and generative neural networks.
Structurally Prune Anything: Any Architecture, Any Framework, Any Time
Authors: Xun Wang, John Rachwan, Stephan Günnemann, Bertrand Charpentier
Date: 2024-03-03 00:00:00
Tasks: Network Pruning

Neural network pruning serves as a critical technique for enhancing the efficiency of deep learning models. Unlike unstructured pruning, which only sets specific parameters to zero, structured pruning eliminates entire channels, thus yielding direct computational and storage benefits. However, the diverse patterns for coupling parameters, such as residual connections and group convolutions, the diverse deep learning frameworks, and the various time stages at which pruning can be performed make existing pruning methods less adaptable to different architectures, frameworks, and pruning criteria. To address this, we introduce Structurally Prune Anything (SPA), a versatile structured pruning framework that can prune neural networks with any architecture, from any framework, and at any stage of training. SPA leverages a standardized computational graph and ONNX representation to prune diverse neural network architectures without the need for manual intervention. SPA employs a group-level importance estimation method, which groups dependent computational operators, estimates their importance, and prunes unimportant coupled channels. This enables the transfer of various existing pruning criteria into a structured group style. As a result, SPA supports pruning at any time, either before training, after training with fine-tuning, or after training without fine-tuning. In the context of the latter, we introduce Optimal Brain SPA (OBSPA), an algorithm that achieves state-of-the-art pruning results needing neither fine-tuning nor calibration data. In extensive experiments, SPA shows competitive to state-of-the-art pruning performance across various architectures, from popular frameworks, at different pruning times.
The Blind Normalized Stein Variational Gradient Descent-Based Detection for Intelligent Random Access in Cellular IoT
Authors: Xin Zhu, Ahmet Enis Cetin
Date: 2024-03-08 00:00:00
Tasks: Denoising

The lack of an efficient preamble detection algorithm remains a challenge for solving preamble collision problems in intelligent random access (RA) in the cellular Internet of Things (IoT). To address this problem, we present an early preamble detection scheme based on a maximum likelihood estimation (MLE) model at the first step of the grant-based RA procedure. A novel blind normalized Stein variational gradient descent (SVGD)-based detector is proposed to obtain an approximate solution to the MLE model. First, by exploring the relationship between the Hadamard transform and wavelet packet transform, a new modified Hadamard transform (MHT) is developed to separate high-frequency components from signals using the second-order derivative filter. Next, to eliminate noise and mitigate the vanishing gradients problem in the SVGD-based detectors, the block MHT layer is designed based on the MHT, scaling layer, soft-thresholding layer, inverse MHT and sparsity penalty. Then, the blind normalized SVGD algorithm is derived to perform preamble detection without prior knowledge of noise power and the number of active IoT devices. The experimental results show the proposed block MHT layer outperforms other transform-based methods in terms of computation costs and denoising performance. Furthermore, with the assistance of the block MHT layer, the proposed blind normalized SVGD algorithm achieves a higher preamble detection accuracy and throughput than other state-of-the-art detection methods.
Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks
Authors: Chunmei Xu, Shengheng Liu, Yongming Huang, Bjorn Ottersten, Dusit Niyato
Date: 2024-02-20 00:00:00
Tasks: Federated Learning

At present, there is a trend to deploy ubiquitous artificial intelligence (AI) applications at the edge of the network. As a promising framework that enables secure edge intelligence, federated learning (FL) has received widespread attention, and over-the-air computing (AirComp) has been integrated to further improve the communication efficiency. In this paper, we consider a joint device selection and aggregate beamforming design with the objectives of minimizing the aggregate error and maximizing the number of selected devices. This yields a combinatorial problem, which is difficult to solve especially in large-scale networks. To tackle the problems in a cost-effective manner, we propose a random aggregate beamforming-based scheme, which generates the aggregator beamforming vector via random sampling rather than optimization. The implementation of the proposed scheme does not require the channel estimation. We additionally use asymptotic analysis to study the obtained aggregate error and the number of the selected devices when the number of devices becomes large. Furthermore, a refined method that runs with multiple randomizations is also proposed for performance improvement. Extensive simulation results are presented to demonstrate the effectiveness of the proposed random aggregate beamforming-based scheme as well as the refined method.