Miracle-dz/UniFAR-FTUs huggingface dataset:
Knowledge Distillation of Convolutional Neural Networks through Feature Map Transformation using Decision Trees

The interpretation of reasoning by Deep Neural Networks (DNN) is still challenging due to their perceived black-box nature. Therefore, deploying DNNs in several real-world tasks is restricted by the lack of transparency of these models. We propose a distillation approach by extracting features from the final layer of the convolutional neural network (CNN) to address insights to its reasoning. The feature maps in the final layer of a CNN are transformed into a one-dimensional feature vector using a fully connected layer. Subsequently, the extracted features are used to train a decision tree to achieve the best accuracy under constraints of depth and nodes. We use the medical images of dermaMNIST, octMNIST, and pneumoniaMNIST from the medical MNIST datasets to demonstrate our proposed work. We observed that performance of the decision tree is as good as a CNN with minimum complexity. The results encourage interpreting decisions made by the CNNs using decision trees.
Remote Sensing for Management of Invasive Plants in Great Lakes Coastal Wetlands

vi LIST OF TABLES . xi LIST OF FIGURES . xii LIST OF ABBREVIATIONS . xiv CHAPTER ONE: INTEGRATION OF RADAR AND OPTICAL SATELLITE DATA FOR FINE SCALE DETECTION OF AN INVASIVE PLANT IN GREAT LAKES COASTAL WETLANDS . 1 Abstract . 1.
Towards Unsupervised And Incremental Semantic Mapping

Robots are slowly entering our houses. But for them to perform more difficult and interesting tasks, they need to have a richer knowledge of the different objects and places present in a particular household. Building and using such a complex knowledge is called semantic mapping and navigation, which requires many different capabilities such as mapping, localization and object recognition. Most of them have been thoroughly studied in the past but often separately and in a different context. We propose a complete solution to perform incremental and unsupervised semantic mapping working on a real robot. Each aspect of the solution is discussed from the software architecture to all the functionalities needed by the robot. Some already existing techniques have been derived in the effort of integration, but more importantly three different parts of the solution are original contributions. The first one is the use of 2D laser range finder to perform object recognition using multiple views of the objects. Numerous semantic mapping solutions are using this sensor for mapping and obstacle avoidance, but we fully leveraged it by developing techniques to use it in the entire process of semantic mapping, showing that it is effective at recognizing up to 20 different objects in an indoor scenario. Secondly we introduce the graph-of-views object modeling method. In our effort to extend bag-of-views techniques to perform multi-modal modeling of objects we derived an original formulation which is more adapted to partial perception of objects. We applied this approach to object recognition using both a laser range finder and a RGB-D camera, showing that it is able to take advantage of both sensor strengths. Finally, while most existing semantic mapping techniques rely on supervised learning of objects, we present an original incremental and unsupervised learning algorithm. Our technique was applied to the learning of multi-modal models of dynamic objects that are encountered as the robot navigate an indoor environment for an extended period of time, showing that it is able to produce consistent models of these objects without human intervention.
Predicting Depression and Anxiety: A Multi-Layer Perceptron for Analyzing the Mental Health Impact of COVID-19 (preprint)

We introduce a multi-layer perceptron (MLP) called the COVID-19 Depression and Anxiety Predictor (CoDAP) to predict mental health trends, particularly anxiety and depression, during the COVID-19 pandemic. Our method utilizes a comprehensive dataset, which tracked mental health symptoms weekly over ten weeks during the initial COVID-19 wave (April to June 2020) in a diverse cohort of U.S. adults. This period, characterized by a surge in mental health symptoms and conditions, offers a critical context for our analysis. Our focus was to extract and analyze patterns of anxiety and depression through a unique lens of qualitative individual attributes using CoDAP. This model not only predicts patterns of anxiety and depression during the pandemic but also unveils key insights into the interplay of demographic factors, behavioral changes, and social determinants of mental health. These findings contribute to a more nuanced understanding of the complexity of mental health issues in times of global health crises, potentially guiding future early interventions.
TivNe-SLAM: Dynamic Mapping and Tracking via Time-Varying Neural Radiance Fields

Previous attempts to integrate Neural Radiance Fields (NeRF) into the Simultaneous Localization and Mapping (SLAM) framework either rely on the assumption of static scenes or require the ground truth camera poses, which impedes their application in real-world scenarios. In this paper, we propose a time-varying representation to track and reconstruct the dynamic scenes. Firstly, two processes, tracking process and mapping process, are simultaneously maintained in our framework. For the tracking process, all input images are uniformly sampled, then progressively trained in a self-supervised paradigm. For the mapping process, we leverage motion masks to distinguish dynamic objects from static background, and sample more pixels from dynamic areas. Secondly, the parameter optimization for both processes consists of two stages: the first stage associates time with 3D positions to convert the deformation field to the canonical field. And the second stage associates time with the embeddings of canonical field to obtain colors and Signed Distance Function (SDF). Lastly, we propose a novel keyframe selection strategy based on the overlapping rate. We evaluate our approach on two synthetic datasets and one real-world dataset. And the experiments validate that our method achieves competitive results in both tracking and mapping when compared to existing state-of-the-art NeRF-based methods.
Global warming contributes to reduction in the intensity of Artemisia pollen seasons in Lublin, central-eastern Poland

Introduction and Objective. Species of the genus Artemisia (Asteraceae) are weeds and ruderal plants growing in northern temperate regions of the world. Many of them are used in medicine and the cosmetic industry and for culinary purposes. Pollen grains of plants of this genus contain the most important aeroallergens. Materials and Method. An aerobiological study conducted with the volumetric method in Lublin in 2001–2022. Trend lines for the season parameters were established. Spearman’s correlation and stepwise regression analyses were carried out to determine relationships between various parameters of the pollen season and meteorological factors. PCA analysis was also carried out to visually compare the pollen seasons. Results. In Lublin, central-eastern Poland, the Artemisia pollen season lasted on average from the second ten days of July to the end of August, with its beginning depending on the temperature in April and May. The highest pollen concentrations were mainly recorded in the first half of August and were largely dependent on the mean temperature in June and July. The second peak in the pollen season recorded in September was associated with the presence of Artemisia annua pollen. Intense sunshine in June and the higher temperatures in June and July resulted in significant reduction in the Artemisia annual pollen sum (by 65%) over 22 years. Artemisia vulgaris is abundant in the Lublin region and contributes substantially to the amount of Artemisia pollen in the aeroplankton. Conclusions. The downward trend in the amount of Artemisia pollen was a result of the increase in temperatures observed in the summer months, and the declining rainfall rates. The global warming effect is extremely unfavourable for plants of Artemisia vulgaris, as they require moist soil substrates for growth.
A new comprehensive index for drought monitoring with TM data

Drought is one of the most important and frequent natural hazards to agriculture production in North China Plain. To improve agriculture water management, accurate drought monitoring information is needed. This study proposed a method for comprehensive drought monitoring by combining a meteorological index and three satellite drought indices of TM data together. SPI (Standard Precipitation Index), the meteorological drought index, is used to measure precipitation deficiency. Three satellite drought indices (Temperature Vegetation Drought Index, Land Surface Water Index, Modified Perpendicular Drought Index) are used to evaluate agricultural drought risk by exploring data from various channels (VIS, NIR, SWIR, TIR). Considering disparities in data ranges of different drought indices, normalization is implemented before combination. First, SPI is normalized to 0 — 100 given that its normal range is -4 - +4. Then, the three satellite drought indices are normalized to 0 - 100 according to the maximum and minimum values in the image, and aggregated using weighted average method (the result is denoted as ADI, Aggregated drought index). Finally, weighed geometric mean of SPI and ADI are calculated (the result is denoted as DIcombined). A case study in North China plain using three TM images acquired during April-May 2007 show that the method proposed in this study is effective. In spatial domain, DIcombined demonstrates dramatically more details than SPI; in temporal domain, DIcombined shows more reasonable drought development trajectory than satellite indices that are derived from independent TM images.
Chemical characterization of wet depositions in urban and rural area

The aim of this study was to determine the chemical composition of wet depositions in two locations from urban and rural area near Bucharest from January to May 2017. For all the samples, pH, conductivity, anions, cations and heavy metals were determined. For anions (Cl, NO3 and SO4) and cations (Ca, K, Mg, Na) determination an ion chromatographic method was used. Heavy metals (Co, Cr, Cu, Mn, Ni, Pb and Zn) were determined using AAS with flame and graphite chamber. Results showed the presence of the interested compounds in wet depositions samples with SO4 and Ca as the dominant anion and cation. Heavy metals concentration followed the trend: Zn > Mn > Cu > Pb > Ni > Cr > Co. Analysing the results obtained the potential sources of ambient air pollution were also suggested.
Bayesian Downscaling Methods for Aggregated Count Data

Policy-critical, micro-level statistical data are often unavailable at the desired level of disaggregation. We present a Bayesian methodology for “downscaling” aggregated count data to the micro level, using an outside statistical sample. Our procedure combines numerical simulation with exact calculation of combinatorial probabilities. We motivate our approach with an application estimating the number of farms in a region, using count totals at higher levels of aggregation. In a simulation analysis over varying population sizes, we demonstrate both robustness to sampling variability and outperformance relative to maximum likelihood. Spatial considerations, implementation of “informative” priors, non-spatial classification problems, and best practices are discussed.
A survey of degree-boundedness

If a graph has no large balanced bicliques, but has large minimum degree, then what can we say about its induced subgraphs? This question motivates the study of degree-boundedness, which is like $\chi$-boundedness but for minimum degree instead of chromatic number. We survey this area with an eye towards open problems.
SecureRights: A Blockchain-Powered Trusted DRM Framework for Robust Protection and Asserting Digital Rights

In the dynamic realm of digital content, safeguarding intellectual property rights poses critical challenges. This paper presents"SecureRights,"an innovative Blockchain-based Trusted Digital Rights Management (DRM) framework. It strengthens the defence against unauthorized use and streamlines the claim of digital rights. Utilizing blockchain, digital watermarking, perceptual hashing, Quick Response (QR) codes, and the Interplanetary File System (IPFS), SecureRights securely stores watermark information on the blockchain with timestamp authentication. Incorporating perceptual hashing generates robust hash tokens based on image structure. The addition of QR codes enhances the watermarking, offering a comprehensive solution for resilient intellectual property rights protection. Rigorous evaluations affirm SecureRights' resilience against various attacks, establishing its efficacy in safeguarding digital content and simplifying rightful ownership assertion.
Nonparametric Automatic Differentiation Variational Inference with Spline Approximation

Automatic Differentiation Variational Inference (ADVI) is efficient in learning probabilistic models. Classic ADVI relies on the parametric approach to approximate the posterior. In this paper, we develop a spline-based nonparametric approximation approach that enables flexible posterior approximation for distributions with complicated structures, such as skewness, multimodality, and bounded support. Compared with widely-used nonparametric variational inference methods, the proposed method is easy to implement and adaptive to various data structures. By adopting the spline approximation, we derive a lower bound of the importance weighted autoencoder and establish the asymptotic consistency. Experiments demonstrate the efficiency of the proposed method in approximating complex posterior distributions and improving the performance of generative models with incomplete data.
Channel Estimation Considerate Precoder Design for Multi-user Massive MIMO-OFDM Systems: The Concept and Fast Algorithms

The sixth-generation (6G) communication networks target peak data rates exceeding 1Tbps, necessitating base stations (BS) to support up to 100 simultaneous data streams. However, sparse pilot allocation to accommodate such streams poses challenges for users' channel estimation. This paper presents Channel Estimation Considerate Precoding (CECP), where BS precoders prioritize facilitating channel estimation alongside maximizing transmission rate. To address the computational complexity of 6G large-scale multi-input multi-output (MIMO) systems, we propose a computationally-efficient space-time block diagonal channel shortening (ST-BDCS) precoding scheme. By leveraging the sparse Toeplitz property of orthogonal frequency division multiplexing (OFDM) channels, this time-domain precoding design effectively mitigates multi-user interference in the downlink and shortens the effective channel's temporal length. Consequently, users can estimate the channels using sparse pilots. To enable fast implementation, we develop a generalized complex-valued Toeplitz matrix QR decomposition algorithm applicable to various space-time signal processing problems. Simulation results demonstrate that the ST-BDCS precoding method approximates the rate performance of conventional subcarrier-by-subcarrier precoding schemes. However, it offers the advantages of easier channel estimation for users and significantly reduced computational complexity for the BS.
Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU

Recent advances in large language models have brought immense value to the world, with their superior capabilities stemming from the massive number of parameters they utilize. However, even the GPUs with the highest memory capacities, currently peaking at 80GB, are far from sufficient to accommodate these vast parameters and their associated optimizer states when conducting stochastic gradient descent-based optimization. One approach to hosting such huge models is to aggregate device memory from many GPUs. However, this approach introduces prohibitive costs for most academic researchers, who always have a limited budget for many high-end GPU servers. In this paper, we focus on huge model fine-tuning on a single, even low-end, GPU in a commodity server, which is accessible to most AI researchers. In such a scenario, the state-of-the-art work ZeRO-Infinity suffers from two severe issues when running in a commodity server: 1) low GPU utilization due to inefficient swapping, and 2) limited trainable model size due to CPU memory capacity. The underlying reason is that ZeRO-Infinity is optimized for running on high-end GPU servers. To this end, we present Fuyou, a low-cost training framework that enables efficient 100B huge model fine-tuning on a low-end server with a low-end GPU and limited CPU memory capacity. The key idea is to add the SSD-CPU communication as an optimization dimension and thus carefully co-optimize computation and data swapping from a systematic approach to maximize GPU utilization. The experimental results show that 1) Fuyou is able to fine-tune 175B GPT-3 on a consumer GPU RTX 4090 with high GPU utilization, while ZeRO-Infinity fails to fine-tune; and 2) when training a small GPT-3 13B model, Fuyou achieves 156 TFLOPS on an RTX 4090 GPU while ZeRO-Infinity only achieves 45 TFLOPS.
RIS-Enabled Joint Near-Field 3D Localization and Synchronization in SISO Multipath Environments

Reconfigurable Intelligent Surfaces (RIS) show great promise in the realm of 6th generation (6G) wireless systems, particularly in the areas of localization and communication. Their cost-effectiveness and energy efficiency enable the integration of numerous passive and reflective elements, enabling near-field propagation. In this paper, we tackle the challenges of RIS-aided 3D localization and synchronization in multipath environments, focusing on the near-field of mmWave systems. Specifically, our approach involves formulating a maximum likelihood (ML) estimation problem for the channel parameters. To initiate this process, we leverage a combination of canonical polyadic decomposition (CPD) and orthogonal matching pursuit (OMP) to obtain coarse estimates of the time of arrival (ToA) and angle of departure (AoD) under the far-field approximation. Subsequently, distances are estimated using $l_{1}$-regularization based on a near-field model. Additionally, we introduce a refinement phase employing the spatial alternating generalized expectation maximization (SAGE) algorithm. Finally, a weighted least squares approach is applied to convert channel parameters into position and clock offset estimates. To extend the estimation algorithm to ultra-large (UL) RIS-assisted localization scenarios, it is further enhanced to reduce errors associated with far-field approximations, especially in the presence of significant near-field effects, achieved by narrowing the RIS aperture. Moreover, the Cram\'er-Rao Bound (CRB) is derived and the RIS phase shifts are optimized to improve the positioning accuracy. Numerical results affirm the efficacy of the proposed estimation algorithm.
Fluent: Round-efficient Secure Aggregation for Private Federated Learning

Federated learning (FL) facilitates collaborative training of machine learning models among a large number of clients while safeguarding the privacy of their local datasets. However, FL remains susceptible to vulnerabilities such as privacy inference and inversion attacks. Single-server secure aggregation schemes were proposed to address these threats. Nonetheless, they encounter practical constraints due to their round and communication complexities. This work introduces Fluent, a round and communication-efficient secure aggregation scheme for private FL. Fluent has several improvements compared to state-of-the-art solutions like Bell et al. (CCS 2020) and Ma et al. (SP 2023): (1) it eliminates frequent handshakes and secret sharing operations by efficiently reusing the shares across multiple training iterations without leaking any private information; (2) it accomplishes both the consistency check and gradient unmasking in one logical step, thereby reducing another round of communication. With these innovations, Fluent achieves the fewest communication rounds (i.e., two in the collection phase) in the malicious server setting, in contrast to at least three rounds in existing schemes. This significantly minimizes the latency for geographically distributed clients; (3) Fluent also introduces Fluent-Dynamic with a participant selection algorithm and an alternative secret sharing scheme. This can facilitate dynamic client joining and enhance the system flexibility and scalability. We implemented Fluent and compared it with existing solutions. Experimental results show that Fluent improves the computational cost by at least 75% and communication overhead by at least 25% for normal clients. Fluent also reduces the communication overhead for the server at the expense of a marginal increase in computational cost.
DeepTextMark: A Deep Learning-Driven Text Watermarking Approach for Identifying Large Language Model Generated Text

The rapid advancement of Large Language Models (LLMs) has significantly enhanced the capabilities of text generators. With the potential for misuse escalating, the importance of discerning whether texts are human-authored or generated by LLMs has become paramount. Several preceding studies have ventured to address this challenge by employing binary classifiers to differentiate between human-written and LLM-generated text. Nevertheless, the reliability of these classifiers has been subject to question. Given that consequential decisions may hinge on the outcome of such classification, it is imperative that text source detection is of high caliber. In light of this, the present paper introduces DeepTextMark, a deep learning-driven text watermarking methodology devised for text source identification. By leveraging Word2Vec and Sentence Encoding for watermark insertion, alongside a transformer-based classifier for watermark detection, DeepTextMark epitomizes a blend of blindness, robustness, imperceptibility, and reliability. As elaborated within the paper, these attributes are crucial for universal text source detection, with a particular emphasis in this paper on text produced by LLMs. DeepTextMark offers a viable “add-on” solution to prevailing text generation frameworks, requiring no direct access or alterations to the underlying text generation mechanism. Experimental evaluations underscore the high imperceptibility, elevated detection accuracy, augmented robustness, reliability, and swift execution of DeepTextMark.
Can Large Language Models Automatically Score Proficiency of Written Essays?

Although several methods were proposed to address the problem of automated essay scoring (AES) in the last 50 years, there is still much to desire in terms of effectiveness. Large Language Models (LLMs) are transformer-based models that demonstrate extraordinary capabilities on various tasks. In this paper, we test the ability of LLMs, given their powerful linguistic knowledge, to analyze and effectively score written essays. We experimented with two popular LLMs, namely ChatGPT and Llama. We aim to check if these models can do this task and, if so, how their performance is positioned among the state-of-the-art (SOTA) models across two levels, holistically and per individual writing trait. We utilized prompt-engineering tactics in designing four different prompts to bring their maximum potential to this task. Our experiments conducted on the ASAP dataset revealed several interesting observations. First, choosing the right prompt depends highly on the model and nature of the task. Second, the two LLMs exhibited comparable average performance in AES, with a slight advantage for ChatGPT. Finally, despite the performance gap between the two LLMs and SOTA models in terms of predictions, they provide feedback to enhance the quality of the essays, which can potentially help both teachers and students.