OpenRaiser/Intern-Atlas huggingface dataset:
Differential Convolutional Fuzzy Time Series Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

Fuzzy time series forecasting (FTSF) is a typical forecasting method with wide application. Traditional FTSF is regarded as an expert system which leads to loss of the ability to recognize undefined features. The mentioned is the main reason for poor forecasting with FTSF. To solve the problem, the proposed model Differential Fuzzy Convolutional Neural Network (DFCNN) utilizes a convolution neural network to re-implement FTSF with learnable ability. DFCNN is capable of recognizing potential information and improving forecasting accuracy. Thanks to the learnable ability of the neural network, the length of fuzzy rules established in FTSF is expended to an arbitrary length that the expert is not able to handle by the expert system. At the same time, FTSF usually cannot achieve satisfactory performance of non-stationary time series due to the trend of non-stationary time series. The trend of non-stationary time series causes the fuzzy set established by FTSF to be invalid and causes the forecasting to fail. DFCNN utilizes the Difference algorithm to weaken the non-stationary of time series so that DFCNN can forecast the non-stationary time series with a low error that FTSF cannot forecast in satisfactory performance. After the mass of experiments, DFCNN has an excellent prediction effect, which is ahead of the existing FTSF and common time series forecasting algorithms. Finally, DFCNN provides further ideas for improving FTSF and holds continued research value.
Deep Photonic Reservoir Computer Based on Frequency Multiplexing with Fully Analog Connection Between Layers
Venue: arXiv cs.ET 2023
Year: 2023

Reservoir computers (RC) are randomized recurrent neural networks well adapted to process time series, performing tasks such as nonlinear distortion compensation or prediction of chaotic dynamics. Deep reservoir computers (deep-RC), in which the output of one reservoir is used as the input for another one, can lead to improved performance because, as in other deep artificial neural networks, the successive layers represent the data in more and more abstract ways. We present a fiber-based photonic implementation of a two-layer deep-RC based on frequency multiplexing. The two RC layers are encoded in two frequency combs propagating in the same experimental setup. The connection between the layers is fully analog and does not require any digital processing. We find that the deep-RC outperforms a traditional RC by up to two orders of magnitude on two benchmark tasks. This work paves the way towards using fully analog photonic neuromorphic computing for complex processing of time series, while avoiding costly analog-to-digital and digital-to-analog conversions.
Influential Billboard Slot Selection using Spatial Clustering and Pruned Submodularity Graph
Venue: arXiv cs.DB 2023
Year: 2023

Billboard advertising is a popular out-of-home advertising technique adopted by commercial houses. Companies own billboards and offer them to commercial houses on a payment basis. Given a database of billboards with slot information, we want to determine which k slots to choose to maximize influence. We call this the INFLUENTIAL BILLBOARD SLOT SELECTION (IBSS) Problem and pose it as a combinatorial optimization problem. We show that the influence function considered in this paper is non-negative, monotone, and submodular. The incremental greedy approach based on the marginal gain computation leads to a constant factor approximation guarantee. However, this method scales very poorly when the size of the problem instance is very large. To address this, we propose a spatial partitioning and pruned submodularity graph-based approach that is divided into the following three steps: preprocessing, pruning, and selection. We analyze the proposed solution approaches to understand their time, space requirement, and performance guarantee. We conduct extensive set of experiments with real-world datasets and compare the performance of the proposed solution approaches with the available baseline methods. We observe that the proposed approaches lead to more influence than all the baseline methods within reasonable computational time.
SuSana Distancia is all you need: Enforcing class separability in metric learning via two novel distance-based loss functions for few-shot image classification
Venue: arXiv cs.CV 2023
Year: 2023

Few-shot learning is a challenging area of research that aims to learn new concepts with only a few labeled samples of data. Recent works based on metric-learning approaches leverage the meta-learning approach, which is encompassed by episodic tasks that make use a support (training) and query set (test) with the objective of learning a similarity comparison metric between those sets. Due to the lack of data, the learning process of the embedding network becomes an important part of the few-shot task. Previous works have addressed this problem using metric learning approaches, but the properties of the underlying latent space and the separability of the difference classes on it was not entirely enforced. In this work, we propose two different loss functions which consider the importance of the embedding vectors by looking at the intra-class and inter-class distance between the few data. The first loss function is the Proto-Triplet Loss, which is based on the original triplet loss with the modifications needed to better work on few-shot scenarios. The second loss function, which we dub ICNN loss is based on an inter and intra class nearest neighbors score, which help us to assess the quality of embeddings obtained from the trained network. Our results, obtained from a extensive experimental setup show a significant improvement in accuracy in the miniImagenNet benchmark compared to other metric-based few-shot learning methods by a margin of 2%, demonstrating the capability of these loss functions to allow the network to generalize better to previously unseen classes. In our experiments, we demonstrate competitive generalization capabilities to other domains, such as the Caltech CUB, Dogs and Cars datasets compared with the state of the art.
An Offline Time-aware Apprenticeship Learning Framework for Evolving Reward Functions
Venue: arXiv cs.LG 2023
Year: 2023

Apprenticeship learning (AL) is a process of inducing effective decision-making policies via observing and imitating experts' demonstrations. Most existing AL approaches, however, are not designed to cope with the evolving reward functions commonly found in human-centric tasks such as healthcare, where offline learning is required. In this paper, we propose an offline Time-aware Hierarchical EM Energy-based Sub-trajectory (THEMES) AL framework to tackle the evolving reward functions in such tasks. The effectiveness of THEMES is evaluated via a challenging task -- sepsis treatment. The experimental results demonstrate that THEMES can significantly outperform competitive state-of-the-art baselines.
AAAI 2022 Fall Symposium: System-1 and System-2 realized within the Common Model of Cognition
Venue: arXiv cs.AI 2023
Year: 2023

Attempts to import dual-system descriptions of System-1 and System-2 into AI have been hindered by a lack of clarity over their distinction. We address this and other issues by situating System-1 and System-2 within the Common Model of Cognition. Results show that what are thought to be distinctive characteristics of System-1 and 2 instead form a spectrum of cognitive properties. The Common Model provides a comprehensive vision of the computational units involved in System-1 and System-2, their underlying mechanisms, and the implications for learning, metacognition, and emotion.
Multi-view MERA Subspace Clustering
Venue: arXiv cs.CV 2023
Year: 2023

Tensor-based multi-view subspace clustering (MSC) can capture high-order correlation in the self-representation tensor. Current tensor decompositions for MSC suffer from highly unbalanced unfolding matrices or rotation sensitivity, failing to fully explore inter/intra-view information. Using the advanced tensor network, namely, multi-scale entanglement renormalization ansatz (MERA), we propose a low-rank MERA based MSC (MERA-MSC) algorithm, where MERA factorizes a tensor into contractions of one top core factor and the rest orthogonal/semi-orthogonal factors. Benefiting from multiple interactions among orthogonal/semi-orthogonal (low-rank) factors, the low-rank MERA has a strong representation power to capture the complex inter/intra-view information in the self-representation tensor. The alternating direction method of multipliers is adopted to solve the optimization model. Experimental results on five multi-view datasets demonstrate MERA-MSC has superiority against the compared algorithms on six evaluation metrics. Furthermore, we extend MERA-MSC by incorporating anchor learning to develop a scalable low-rank MERA based multi-view clustering method (sMREA-MVC). The effectiveness and efficiency of sMERA-MVC have been validated on three large-scale multi-view datasets. To our knowledge, this is the first work to introduce MERA to the multi-view clustering topic. The codes of MERA-MSC and sMERA-MVC are publicly available at https://github.com/longzhen520/MERA-MSC.
Security Evaluation of Thermal Covert-channels on SmartSSDs
Venue: arXiv cs.CR 2023
Year: 2023

Continued expansion of cloud computing offerings now includes SmartSSDs. A SmartSSD is a solid-state disk (SSD) augmented with an FPGA. Through public cloud providers, it is now possible to rent on-demand virtual machines enabled with SmartSSDs. Because of the FPGA component of the SmartSSD, cloud users who access the SmartSSD can instantiate custom circuits within the FPGA. This includes possibly malicious circuits for measurement of power and temperature. Normally, cloud users have no remote access to power and temperature data, but with SmartSSDs they could abuse the FPGA component to learn this information. This paper shows for the first time that heat generated by a cloud user accessing the SSD component of the SmartSSD and the resulting temperature increase, can be measured by a different cloud user accessing the FPGA component of the same SmartSSD by using the ring oscillators circuits to measure temperature. The thermal state remains elevated for a few minutes after the SSD is heated up and can be measured from the FPGA side by a subsequent user for up to a few minutes after the SSD heating is done. Further, in a future multi-tenant SmartSSD setting, the thermal changes can be measured in parallel if one user controls the SSD and the other the FPGA. Based on this temporal thermal state of the SmartSSD, a novel thermal communication channel is demonstrated for the first time.
Adversarial Speaker Disentanglement Using Unannotated External Data for Self-supervised Representation Based Voice Conversion
Venue: arXiv cs.SD 2023
Year: 2023

Nowadays, recognition-synthesis-based methods have been quite popular with voice conversion (VC). By introducing linguistics features with good disentangling characters extracted from an automatic speech recognition (ASR) model, the VC performance achieved considerable breakthroughs. Recently, self-supervised learning (SSL) methods trained with a large-scale unannotated speech corpus have been applied to downstream tasks focusing on the content information, which is suitable for VC tasks. However, a huge amount of speaker information in SSL representations degrades timbre similarity and the quality of converted speech significantly. To address this problem, we proposed a high-similarity any-to-one voice conversion method with the input of SSL representations. We incorporated adversarial training mechanisms in the synthesis module using external unannotated corpora. Two auxiliary discriminators were trained to distinguish whether a sequence of mel-spectrograms has been converted by the acoustic model and whether a sequence of content embeddings contains speaker information from external corpora. Experimental results show that our proposed method achieves comparable similarity and higher naturalness than the supervised method, which needs a huge amount of annotated corpora for training and is applicable to improve similarity for VC methods with other SSL representations as input.
Stochastic Porous Microstructures
Venue: arXiv cs.GR 2023
Year: 2023

Stochastic porous structures are ubiquitous in natural phenomena and have gained considerable traction across diverse domains owing to their exceptional physical properties. The recent surge in interest in microstructures can be attributed to their impressive attributes, such as a high strength-to-weight ratio, isotropic elasticity, and bio-inspired design principles. Notwithstanding, extant stochastic structures are predominantly generated via procedural modeling techniques, which present notable difficulties in representing geometric microstructures with periodic boundaries, thereby leading to intricate simulations and computational overhead. In this manuscript, we introduce an innovative method for designing stochastic microstructures that guarantees the periodicity of each microstructure unit to facilitate homogenization. We conceptualize each pore and the interconnecting tunnel between proximate pores as Gaussian kernels and leverage a modified version of the minimum spanning tree technique to assure pore connectivity. We harness the dart-throwing strategy to stochastically produce pore locations, tailoring the distribution law to enforce boundary periodicity. We subsequently employ the level-set technique to extract the stochastic microstructures. Conclusively, we adopt Wang tile rules to amplify the stochasticity at the boundary of the microstructure unit, concurrently preserving periodicity constraints among units. Our methodology offers facile parametric control of the designed stochastic microstructures. Experimental outcomes on 3D models manifest the superior isotropy and energy absorption performance of the stochastic porous microstructures. We further corroborate the efficacy of our modeling strategy through simulations of mechanical properties and empirical experiments.
Ortho-ODE: Enhancing Robustness and of Neural ODEs against Adversarial Attacks
Venue: arXiv cs.LG 2023
Year: 2023

Neural Ordinary Differential Equations (NODEs) probed the usage of numerical solvers to solve the differential equation characterized by a Neural Network (NN), therefore initiating a new paradigm of deep learning models with infinite depth. NODEs were designed to tackle the irregular time series problem. However, NODEs have demonstrated robustness against various noises and adversarial attacks. This paper is about the natural robustness of NODEs and examines the cause behind such surprising behaviour. We show that by controlling the Lipschitz constant of the ODE dynamics the robustness can be significantly improved. We derive our approach from Grownwall's inequality. Further, we draw parallels between contractivity theory and Grownwall's inequality. Experimentally we corroborate the enhanced robustness on numerous datasets - MNIST, CIFAR-10, and CIFAR 100. We also present the impact of adaptive and non-adaptive solvers on the robustness of NODEs.
Information Energy Ratio of XOR Logic Gate at Mesoscopic Scale
Venue: arXiv cs.IT 2023
Year: 2023

As the size of transistors approaches the mesoscopic scale, existing energy consumption analysis methods exhibit various limits, especially when being applied to describe the non-equilibrium information processing of transistors at ultra-low voltages. The stochastic thermodynamics offers a theoretic tool to analyze the energy consumption of transistor during the non-equilibrium information processing. Based on this theory, an information energy ratio of XOR gate composed of single-electron transistors is proposed at the mesoscopic scale, which can be used to quantify the exchange between the information and energy at XOR gates. Furthermore, the energy efficiency of the parity check circuit is proposed to analyze the energy consumption of digital signal processing systems. Compared with the energy efficiency of parity check circuit adopting the 7 nm semiconductor process supply voltage, simulation results show that the energy efficiency of the parity check circuit is improved by 266% when the supply voltage is chosen at a specified value.
Counterfactual Outcome Prediction using Structured State Space Model
Venue: arXiv cs.LG 2023
Year: 2023

Counterfactual outcome prediction in longitudinal data has recently gained attention due to its potential applications in healthcare and social sciences. In this paper, we explore the use of the state space model, a popular sequence model, for this task. Specifically, we compare the performance of two models: Treatment Effect Neural Controlled Differential Equation (TE-CDE) and structured state space model (S4Model). While TE-CDE uses controlled differential equations to address time-dependent confounding, it suffers from optimization issues and slow training. In contrast, S4Model is more efficient at modeling long-range dependencies and easier to train. We evaluate the models on a simulated lung tumor growth dataset and find that S4Model outperforms TE-CDE with 1.63x reduction in per epoch training time and 10x better normalized mean squared error. Additionally, S4Model is more stable during training and less sensitive to weight initialization than TE-CDE. Our results suggest that the state space model may be a promising approach for counterfactual outcome prediction in longitudinal data, with S4Model offering a more efficient and effective alternative to TE-CDE.
Component Training of Turbo Autoencoders
Venue: arXiv cs.IT 2023
Year: 2023

Isolated training with Gaussian priors (TGP) of the component autoencoders of turbo-autoencoder architectures enables faster, more consistent training and better generalization to arbitrary decoding iterations than training based on deep unfolding. We propose fitting the components via extrinsic information transfer (EXIT) charts to a desired behavior which enables scaling to larger message lengths ($k \approx 1000$) while retaining competitive performance. To the best of our knowledge, this is the first autoencoder that performs close to classical codes in this regime. Although the binary cross-entropy (BCE) loss function optimizes the bit error rate (BER) of the components, the design via EXIT charts enables to focus on the block error rate (BLER). In serially concatenated systems the component-wise TGP approach is well known for inner components with a fixed outer binary interface, e.g., a learned inner code or equalizer, with an outer binary error correcting code. In this paper we extend the component training to structures with an inner and outer autoencoder, where we propose a new 1-bit quantization strategy for the encoder outputs based on the underlying communication problem. Finally, we discuss the model complexity of the learned components during design time (training) and inference and show that the number of weights in the encoder can be reduced by 99.96 %.
A Multi-Client Searchable Encryption Scheme for IoT Environment
Venue: arXiv cs.CR 2023
Year: 2023

The proliferation of connected devices through Internet connectivity presents both opportunities for smart applications and risks to security and privacy. It is vital to proactively address these concerns to fully leverage the potential of the Internet of Things. IoT services where one data owner serves multiple clients, like smart city transportation, smart building management and healthcare can offer benefits but also bring cybersecurity and data privacy risks. For example, in healthcare, a hospital may collect data from medical devices and make it available to multiple clients such as researchers and pharmaceutical companies. This data can be used to improve medical treatments and research but if not protected, it can also put patients' personal information at risk. To ensure the benefits of these services, it is important to implement proper security and privacy measures. In this paper, we propose a symmetric searchable encryption scheme with dynamic updates on a database that has a single owner and multiple clients for IoT environments. Our proposed scheme supports both forward and backward privacy. Additionally, our scheme supports a decentralized storage environment in which data owners can outsource data across multiple servers or even across multiple service providers to improve security and privacy. Further, it takes a minimum amount of effort and costs to revoke a client's access to our system at any time. The performance and formal security analyses of the proposed scheme show that our scheme provides better functionality, and security and is more efficient in terms of computation and storage than the closely related works.
Privacy-Preserving Ensemble Infused Enhanced Deep Neural Network Framework for Edge Cloud Convergence
Venue: arXiv cs.CR 2023
Year: 2023

We propose a privacy-preserving ensemble infused enhanced Deep Neural Network (DNN) based learning framework in this paper for Internet-of-Things (IoT), edge, and cloud convergence in the context of healthcare. In the convergence, edge server is used for both storing IoT produced bioimage and hosting DNN algorithm for local model training. The cloud is used for ensembling local models. The DNN-based training process of a model with a local dataset suffers from low accuracy, which can be improved by the aforementioned convergence and Ensemble Learning. The ensemble learning allows multiple participants to outsource their local model for producing a generalized final model with high accuracy. Nevertheless, Ensemble Learning elevates the risk of leaking sensitive private data from the final model. The proposed framework presents a Differential Privacy-based privacy-preserving DNN with Transfer Learning for a local model generation to ensure minimal loss and higher efficiency at edge server. We conduct several experiments to evaluate the performance of our proposed framework.
A new node-shift encoding representation for the travelling salesman problem
Venue: arXiv cs.NE 2023
Year: 2023

This paper presents a new genetic algorithm encoding representation to solve the travelling salesman problem. To assess the performance of the proposed chromosome structure, we compare it with state-of-the-art encoding representations. For that purpose, we use 14 benchmarks of different sizes taken from TSPLIB. Finally, after conducting the experimental study, we report the obtained results and draw our conclusion.
Hybrid and Collaborative Passage Reranking
Venue: arXiv cs.IR 2023
Year: 2023

In passage retrieval system, the initial passage retrieval results may be unsatisfactory, which can be refined by a reranking scheme. Existing solutions to passage reranking focus on enriching the interaction between query and each passage separately, neglecting the context among the top-ranked passages in the initial retrieval list. To tackle this problem, we propose a Hybrid and Collaborative Passage Reranking (HybRank) method, which leverages the substantial similarity measurements of upstream retrievers for passage collaboration and incorporates the lexical and semantic properties of sparse and dense retrievers for reranking. Besides, built on off-the-shelf retriever features, HybRank is a plug-in reranker capable of enhancing arbitrary passage lists including previously reranked ones. Extensive experiments demonstrate the stable improvements of performance over prevalent retrieval and reranking methods, and verify the effectiveness of the core components of HybRank.