Miracle-dz/UniFAR-FTUs huggingface dataset:
RRR-Net: Reusing, Reducing, and Recycling a Deep Backbone Network

It has become mainstream in computer vision and other machine learning domains to reuse backbone networks pretrained on large datasets as preprocessors. Typically, the last layer is replaced by a shallow learning machine of sorts; the newly-added classification head and (optionally) deeper layers are fine-tuned on a new task. Due to its strong performance and simplicity, a common pre-trained backbone network is ResNet152. However, ResNet152 is relatively large and induces inference latency. In many cases, a compact and efficient backbone with similar performance would be preferable over a larger, slower one. This paper investigates techniques to reuse a pre-trained backbone with the objective of creating a smaller and faster model. Starting from a large ResNet152 backbone pre-trained on ImageNet, we first reduce it from 51 blocks to 5 blocks, reducing its number of parameters and FLOPs by more than 6 times, without significant performance degradation. Then, we split the model after 3 blocks into several branches, while preserving the same number of parameters and FLOPs, to create an ensemble of sub-networks to improve performance. Our experiments on a large benchmark of 40 image classification datasets from various domains suggest that our techniques match the performance (if not better) of “classical backbone fine-tuning” while achieving a smaller model size and faster inference speed.
Attention-Based Convolutional Neural Network for CT Scan COVID-19 Detection

The accurate detection of Covid-19 from chest Computed Tomography (CT) images can assist in early diagnosis and management of the disease. This paper presents a solution for Covid-19 detection, presented in the challenge of 3rd Covid-19 competition, inside the “AI-enabled Medical Image Analysis Workshop” organized by IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) 2023. In this work, the application of deep learning models for chest CT image analysis was investigated, focusing on the use of a ResNet as a backbone network augmented with attention mechanisms. The ResNet provides an effective feature extractor for the classification task, while the attention mechanisms improve the model’s ability to focus on important regions of interest within the images. We conducted extensive experiments on a provided dataset and achieved a macro F1 score of 0.78 on the test set, demonstrating the potential to assist the diagnosis of Covid-19. Our proposed approach leverages the power of deep learning with attention mechanisms to address the challenges of Covid-19 detection in the early detection and management of the disease. In both test and validation set, the proposed method outperformed the baseline of the challenge, ranking fifth in the competition.
A Bio-Inspired Computational Astrocyte Model for Spiking Neural Networks

The mammalian brain is the most capable and complex computing entity known today, and for years there has been research focused on reproducing the brain's capabilities. An early example of this endeavor was the perceptron which has become a core building block of neural network models in the deep learning era. Despite many successes achieved through deep learning, these networks behave much differently than their biological counterparts. In a search for improvements to things like training time and dataset size, power consumption, noise, and adversarial input tolerance, research is looking towards the brain, and bio-inspired computing models. Spiking neural networks (SNNs) take a step closer to biology in their operation and are the focus of much research, geared towards reproducing some of these novel features of the brain. As of yet, SNNs have not reached their full potential. This work explores the advancement of SNNs though introduction of a novel astrocyte model. Astrocytes, initially thought to be passive support cells in the brain are now known to actively participate in neural processing. The proposed astrocyte model is geared towards synaptic plasticity and is shown to generalize and extend spike-timing dependent plasticity (STDP). In addition, the model supports plasticity-focused multi-synapse integration. Logical AND was used as a test function for multi-synapse astrocyte plasticity, and convergence for a single leaky integrate and fire (LIF) neuron with 2, 3, and 4 synapses was demonstrated. The plasticity model is general and many other functions could presumably be used in-place of AND, taking advantage of a multi-synapse view. This leaves a solid path forward for future work.
Performance Evaluation of Transformer-based NLP Models on Fake News Detection Datasets

Fake news has become a major concern due to its spread on social media. To combat this, various machine learning (ML) techniques have been proposed. However, there is a lack of research on the performance of transformer models using datasets from a wide range of domains. This paper investigates the performance of ML algorithms on three fake news datasets: LIAR, FNC-1 and Balanced Dataset for Fake News Analysis. Pretrained transformer language models such as BERT, RoBERTa, ALBERT and DistilBERT were chosen for this paper. The performance of the models was consistent across all datasets. RoBERTa obtained an accuracy of 69% when trained on the LIAR dataset, an 11% improvement over the existing traditional and deep learning ML model implementations, and an accuracy of 97% when trained on the FNC-1 dataset, proving to be the best-performing model across all the fake news detection datasets utilized in the experiments. DistilBERT trains at a significantly faster rate than the other three variants. The experimental results from the paper can help the research community to continue investigating and gain insights into fake news detection.
What We Do in the Shadows: How does Experiencing Cybercrime Affect Response Actions & Protective Practices?

Despite the increasing prevalence of cybercrimes, there is limited understanding of the response strategies and protective (privacy, security) practices of individuals who have experienced at least one attempted or successful cybercrime incident. To address this gap, we conduct an online survey across an N=605 sample from the UK, eliciting firsthand accounts of cybercrime experiences, actions taken in response to such incidents, as well as their level of engagement with security and privacy (SP) methods and technologies. We identify the occurrence of 8 distinct types of cybercrime and 9 corresponding response strategies amongst 319 cybercrime reports, encompassing IT and protective themes. We depict the association between the experience of cybercrime and the utilisation of response strategies through visual representation. In parallel with IT-related or protective SP theme response strategies, we find that the type of cybercrime experienced (e.g., malware) and the frequency of experiences (e.g., multiple cybercrimes or repeated experiences of the same cybercrime type) influence the engagement with SP technologies. This paper provides empirical insights into experiences of different types of cybercrimes, including their multiple occurrences, and (protective) actions. We also provide recommendations for research and practical implementation, emphasising the importance of addressing the specific needs of individuals who have experienced multiple or repeat cybercrimes. Our paper suggests the adoption of a de-responsibiling cyber-strategy that complements individual guardianship — while promoting infrastructure that caters to the diverse range of users.
Gas Sensor-Based Nondestructive Testing for Jackfruit Ripeness Level

Jackfruit is a fruit native to South and Southeast Asia that is now cultivated all over the world due to its adaptability, as it can be eaten both ripe and unripe. This study was conducted to accurately identify the Jackfruit’s ripeness. Four MQ families have been identified as the most effective for classifying the ripeness of Jackfruit based on the sensitivity of the sensors. MQ Gas Sensors such as MQ3, MQ5, MQ7, and MQ135 are used to determine the ripeness of the Jackfruit without cutting it open. Jackfruit emits an aroma, which can be used to determine the ripeness of the fruit. The data is collected using 36 Jackfruits, of which 12 Jackfruits each classification. This study evaluates the performance of a device designed to classify Jackfruits into three categories based solely on the gas concentration in their aroma: unripe, ripe, and forcibly ripe. As a result, the system can determine the level of ripeness of the jackfruit. Unripe jackfruit received a score of 1 for precision, recall, F1-Score, and accuracy. Meanwhile, ripe jackfruit received precision, recall, F1-score, and accuracy scores of 0.83333, 1, 0.9091, and 0.9444, respectively. Precision was given a score of 1 for forcibly ripe, while recall and F1-score were 0.8571 and 0.9231, respectively. Similarly, the precision score for forcibly ripe was 0.9444. The device’s overall accuracy rate was 96%, indicating that it distinguished the ripeness level of the jackfruit with high precision. The results of the study provide producers and consumers with a good course of action, allowing them to produce and consume fruit more efficiently and effectively. This not only minimizes food waste, but also ensures that the fruit is consumed at its optimal ripeness, leading to greater consumer satisfaction.
ST-CA YOLOv5: Improved YOLOv5 Based on Swin Transformer and Coordinate Attention for Surface Defect Detection

Surface defect detection plays a crucial role in industrial equipment to ensure industrial safety. With the development of deep learning, a series of deep learning-based surface defect detection algorithms are proposed and achieved significant success. However, the application of the algorithms in real-world scenarios is restricted by computing power and memory resource, resulting in either deployment issues or performance degradation. In order to balance memory consumption and detection accuracy, we propose a novel method named ST-CA YOLOv5 for surface defect detection. Based on YOLOv5, The Swin Transformer Block (ST) is introduced to design the C3STR module, enhancing the ability to capture long-range semantic information. In the prediction head, we present the CAHead structure by utilizing the lightweight attention module, Coordinate Attention (CA), to fuse feature information. With the benefit of the ST and CA module, the model improves the detection ability for small objects and yields better overall detection performance. Extensive experiments conducted on several real-world datasets demonstrate the effectiveness and superiority of the proposed method, compared with the state-of-the-art methods in terms of detection performance.
Analyzing the benefits of communication channels between deep learning models

As artificial intelligence systems spread to more diverse and larger tasks in many domains, the machine learning algorithms, and in particular the deep learning models and the databases required to train them are getting bigger themselves. Some algorithms do allow for some scaling of large computations by leveraging data parallelism. However, they often require a large amount of data to be exchanged in order to ensure the shared knowledge throughout the compute nodes is accurate. In this work, the effect of different levels of communications between deep learning models is studied, in particular how it affects performance. The first approach studied looks at decentralizing the numerous computations that are done in parallel in training procedures such as synchronous and asynchronous stochastic gradient descent. In this setting, a simplified communication that consists of exchanging low bandwidth outputs between compute nodes can be beneficial. In the following chapter, the communication protocol is slightly modified to further include training instructions. Indeed, this is studied in a simplified setup where a pre-trained model, analogous to a teacher, can customize a randomly initialized model's training procedure to accelerate learning. Finally, a communication channel where two deep learning models can exchange a purposefully crafted language is explored while allowing for different ways of optimizing that language.
Bayesian evaluation of the performance of three diagnostic tests for Chlamydophila psittaci in humans

Chlamydophila psittaci (C. psittaci) remains a significant threat to the health of farming communities in close contact with psittacine birds yet its infection burden remains poorly understood owing to the low accuracy of available diagnostic tests. This study aimed to evaluate the accuracy of chicken embryo (CEI) and mice inoculation (MI) tests and a PCR assay for the detection of C. psittaci in humans. Sputum specimens from 70 Egyptian individuals in contact with psittacine birds were screened for the presence of the pathogen using the three tests. A Bayesian latent class model was used to estimate the Se and Sp of the three tests. The PCR assay had a higher Se (85%; PCI 42.4% - 99.4%) than CEI (68.5%; PCI 24.6% - 95.6%) and MI (47.0%; PCI 12.3% - 85.1%) tests together with a higher Sp (98.9%; PCI 94.1% - 100%) than CEI (98.6%; PCI 93.8% - 99.9%) and MI (98.6%; PCI 93.8% - 99.9%) tests. To our knowledge, this is the first attempt at evaluating the accuracy of these tests for the detection of C. psittaci in humans. The PCR assay clearly outperforms the inoculation tests and hence holds better promise for routine use in surveillance programs for psittacosis.
Reducing the Length Divergence Bias for Textual Matching Models via Alternating Adversarial Training

Although deep learning has made remarkable achievements in natural language processing tasks, many researchers have recently indicated that models achieve high performance by exploiting statistical bias in datasets. However, once such models obtained on statistically biased datasets are applied in scenarios where statistical bias does not exist, they show a significant decrease in accuracy. In this work, we focus on the length divergence bias, which makes language models tend to classify samples with high length divergence as negative and vice versa. We propose a solution to make the model pay more attention to semantics and not be affected by bias. First, we propose constructing an adversarial test set to magnify the effect of bias on models. Then, we introduce some novel techniques to demote length divergence bias. Finally, we conduct our experiments on two textual matching corpora, and the results show that our approach effectively improves the generalization and robustness of the model, although the degree of bias of the two corpora is not the same.
GLSI Texture Descriptor Based on Complex Networks for Music Genre Classification

The texture classification of an image is related to an important musical attribute, the music genre. This relationship is depicted in the visual representation of the audio signal, called as spectrogram. In this paper, we propose a new Music Genre Classification (MGC) system that processes the spectrogram texture using the Gray Level and Structural Information (GLSI) descriptor, and represents the interconnection between the descriptor codes through complex networks. The GLSI descriptor is an improvement of the CLBP (Completed Local Binary Pattern) descriptor, which quantifies the texture of an image with three codes: signal (CLBP-S), magnitude (CLBP-M), and central (CLBP-C). By transforming the CLBP-C code, GLSI adds macro-structural information. The network nodes represent the descriptor codes, and the respective edges, the relationship according to the horizontal and vertical consecutive condition. We defined two representations for the nodes: 1) individual code node, obtaining the $G_{s},\ G_{m}$ and $G_{g}$ networks, and 2) triple code node, obtaining the $G_{smg}$ network. For the experimental stage, we used the GTZAN dataset, three types of spectrograms: conventional, mel-spectrogram and gammatonegram; and mining with network topological measures. For each type of spectrogram, we performed three experiments according to feature vector combinations, such as measures of: 1) $G_{s},\ G_{m}$ and $G_{g}$, 2) $G_{smg}$, and 3) all networks. In the machine learning stage, we used the ensemble classifier Bagging with Random Forest, and 10-fold cross-validation repeated 100 times. The experiment using all measures and all spectrograms revealed a satisfactory result, indicating that the MGC proposed is promising. We also propose a new equation to calculate the GLSI code, which proved to be much faster and with more intuitive encoding.
Reinforcement Learning for Weighted p-median Problem

This paper presents and explores the possibility of using reinforcement learning to solve the weighted p-median problem. The weighted p-median problem is formulated as a gradual build-up of centers generating admissible solutions. The agent is based on the Proximal Policy Optimization method. Several designs of environment state representations, reward functions, and artificial neural network architectures are tested and compared. The proposed approach is applied to solve the weighted p-median problem at the level of regions and the whole Slovak Republic. For the largest problem with the number of vertices 2916 and parameter $p=273$, the proposed method found solution differing from the optimal solution by 5.7% which shows the potential of using reinforcement learning to solve the weighted p-median problem.
Topological semimetal phases manifested in transition metal dichalcogenides intercalated with 
3d
 metals

In the search for stable topological semimetals with clean band profiles, we have screened all the $3d$ metal-intercalated transition metal dichalcogenides (3dI-TMDCs) by performing hybrid-functional-based ab initio calculations. Two classes of topological materials featuring 12 Weyl points (WPs) in the ${k}_{z}=0$ plane (without spin-orbit interactions) are identified: (a) ferromagnetic Weyl semimetals $\mathrm{V}{T}_{3}{X}_{6}$ and (b) spinless Weyl semimetals $\mathrm{Mn}{T}_{3}{X}_{6}$ (nonmagnetic), where $T=\text{Nb}$, Ta and $X=\text{S}$, Se. ${\mathrm{VNb}}_{3}{\mathrm{S}}_{6}$, prototypical of class (a), is half metallic with only two bands crossing at the Fermi level to form WPs. If spin-orbit interactions are included, ${\mathrm{VNb}}_{3}{\mathrm{S}}_{6}$ is a Weyl semimetal with only two WPs for magnetization $\mathbf{m}$ in the basal plane, whereas it is a ferromagnetic topological insulator for $\mathbf{m}$ normal to the plane. ${\mathrm{MnNb}}_{3}{\mathrm{S}}_{6}$ in the nonmagnetic phase is essentially a spinless version of ${\mathrm{VNb}}_{3}{\mathrm{S}}_{6}$ featuring an equally clean band profile. Interestingly, both materials exhibit a quasinodal line connecting the WPs, which we explain in terms of the quasimirror symmetry and the orbital nature of the bands. 3dI-TMDCs are chemically and thermally stable stoichiometric compounds containing no toxic elements and are a viable platform for the study of topological condensed-matter physics.
A Novel Sidewinding Snake Robot with Non-zero Slope in Granular Terrains Modeled by DRFM

Exploring granular terrains like loose sand is usually not easy for robots, which must solve the sinking and low-moving efficiency problems. Relying on a unique sidewinding gait, snakes can move efficiently in granular terrains, which brings design insight for continuum robots. However, most previous studies of snake-like sidewinding robots focus on the lateral undulation motion like a sine wave, but few generate the body’s non-zero slope like real snakes. In this article, we analyze the effect of nonzero slope on locomotion efficiency using a Dynamic Resistive Force Model (DRFM) and a material point method (MPM). This work develops a non-wheeled 3D printed snake robot, and the body’s slope can be changed. In addition, the unique structure with soft helix rod makes it have more than 35 degrees of freedom in 35 centimeters in length. In order to reduce weight and complexity, the robot needs only a single motor to achieve the sidewinding gait. Experiments not only confirm that the robot can move more efficiently by changing the slope, which shows the importance of the non-zero slope of the snake robot but demonstrates the designed robot’s ability to explore in granular terrains.
A Brain-inspired Fully Hardware Hopfield Neural Network based on Memristive Arrays

In this work, we propose a fully hardware implementation of Hopfield neural network (HNN) based on memristive arrays, which adopts a bottom-up brain-inspired hardware framework. Memristors are used as key computing units to design cell modules in the low layer to play roles similar to neurons and synapses, so as to perform information integration, filtering, and plasticity of weights in a simple circuit structure and in-memory computing way. Cell modules are cascaded by the mode of encoding and mapping based on HNN in a structured regular circuit way to construct functional modules with braininspired parallel analog computing capacity in middle layers. Different functional modules perform information interaction according to the system's requirements and goals, thus realizing the overall system in the top layer. Our proposed HNN system is then used to solve combinatorial optimization problems. Different from other similar work, our system starts from a memristor-based brain-inspired framework and is implemented fully in hardware. The experimental results show that our work can not only improve the convergence speed, but also can be conveniently used to solve problems of different scales because of its good scalability. In addition, with hardware overhead and power consumption analysis, our system has been shown to be very hardware-friendly. Our work represents an advance towards a memristor-based hardware system with brain-inspired structure and high performance.
Zero-Shot Cross-Lingual Opinion Target Extraction

Aspect-based sentiment analysis involves the recognition of so called opinion target expressions (OTEs). To automatically extract OTEs, supervised learning algorithms are usually employed which are trained on manually annotated corpora. The creation of these corpora is labor-intensive and sufficiently large datasets are therefore usually only available for a very narrow selection of languages and domains. In this work, we address the lack of available annotated data for specific languages by proposing a zero-shot cross-lingual approach for the extraction of opinion target expressions. We leverage multilingual word embeddings that share a common vector space across various languages and incorporate these into a convolutional neural network architecture for OTE extraction. Our experiments with 5 languages give promising results: We can successfully train a model on annotated data of a source language and perform accurate prediction on a target language without ever using any annotated samples in that target language. Depending on the source and target language pairs, we reach performances in a zero-shot regime of up to 77% of a model trained on target language data. Furthermore, we can increase this performance up to 87% of a baseline model trained on target language data by performing cross-lingual learning from multiple source languages.