Reviewerly/exHarmony huggingface dataset:
ORSI Salient Object Detection via Cross-Scale Interaction and Enlarged Receptive Field

Due to the diversity of scales and shapes, the uncertainty of object position, and the complexity of edge details, the recent merging problem of salient object detection in optical remote sensing image (RSI-SOD) is a considerably challenging topic. To cope with the challenges, we propose a new cross-scale interaction network (CIFNet) equipped with the enlarged receptive field, which mainly contains three modules in an encoder–decoder architecture, including a furcate skip connection module (FSCM), a global leading attention module, and an expansion–integration module (EIM). First, the FSCM uses dilated convolutions to enlarge the receptive field and furcate skip connections to capture more multiscale contextual information, both of which facilitate the adaptability of the model to different sizes, shapes, and quantities of the target objects. Second, on the low-resolution branch, a global leading attention module (GLM) locates the potentially significant object positions in the feature map from a global semantic perspective. Finally, through an attention-guided cascade structure, the EIM seeks more delicate characteristics by refining the features in a coarse-to-fine fashion. Extensive experiments are conducted on two RSI-SOD datasets, from which superior results can be achieved by our CIFNet, outperforming the other state-of-the-art methods. Compared with the second-best method, the performance gain of our method reaches 3.45% on mean absolute error (MAE) and 1.38% on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F_{\beta} ^{\mathrm {adp}}$ </tex-math></inline-formula> . Notably, the proposed CIF-Net runs with 40.40-M parameters, 14.8-GFLOPs computational complexity, and 58-frames/s inference speed, which guarantees high efficiency.
Theme-Aware Visual Attribute Reasoning for Image Aesthetics Assessment

People usually assess image aesthetics according to visual attributes, e.g., interesting content, good lighting and vivid color, etc. Further, the perception of visual attributes depends on the image theme. Therefore, the inherent relationship between visual attributes and image theme is crucial for image aesthetics assessment (IAA), which has not been comprehensively investigated. With this motivation, this paper presents a new IAA model based on Theme-Aware Visual Attribute Reasoning (TAVAR). The underlying idea is to simulate the process of human perception in image aesthetics by performing bilevel reasoning. Specifically, a visual attribute analysis network and a theme understanding network are first pre-trained to extract aesthetic attribute features and theme features, respectively. Then, the first level Attribute-Theme Graph (ATG) is built to investigate the coupling relationship between visual attributes and image theme. Further, a flexible aesthetics network is introduced to extract general aesthetic features, based on which we built the second level Attribute-Aesthetics Graph (AAG) to mine the relationship between theme-aware visual attributes and aesthetic features, producing the final aesthetic prediction. Extensive experiments on four public IAA databases demonstrate the superiority of the proposed TAVAR model over the state-of-the-arts. Furthermore, TAVAR features better explainability due to the use of visual attributes.
Less is Better: Exponential Loss for Cross-Modal Matching

Deep metric learning has become a key component of cross-modal retrieval. By learning to pull the features of matched instances closer while pushing the features of mismatched instances farther away, one can learn highly robust multi-modal representations. Most existing cross-modal retrieval methods leverage vanilla triplet loss to train the network, which cannot adaptively penalize pairs with different hardness. Although various weighting strategies have been designed for unimodal matching tasks, few weighting strategies have been applied to cross-modal tasks due to the specificity of cross-modal tasks. While few weighting strategies are designed for cross-modal scenarios, they usually involve a lot of hyper-parameters, which require a lot of computational resources to fine-tune. In this paper, we introduce a new exponential loss, which can assign appropriate weights to individual positive and negative pairs according to their similarity so that it can adaptively penalize pairs with different hardness. Furthermore, the exponential loss has only two hyper-parameters, making it easier to find the optimal parameters to suit various data distributions in practice. Exponential loss can be universally applied to well-established cross-modal models and further boost their retrieval performance. We exhaustively ablate our method on Image-Text matching, Video-Text matching, as well as unimodal Image matching. Experimental results show that a standard model trained with exponential loss can achieve noticeable performance gains.
Underwater Depth Estimation via Stereo Adaptation Networks

With the fast development and wide application of stereo depth estimation, adequate high-quality stereo training data with groundtruth depth information plays an important role, but is not easily acquired in underwater environments. Therefore, satisfactory performance of depth estimation is difficult to achieve in underwater environments. In addition, the domain gap also leads to the failure of directly applying existing models of terrestrial scene to underwater scene. Therefore, this paper proposes a novel underwater depth estimation network which can infer depth maps from real underwater stereo images in an adaptation manner. The proposed learning pipeline mainly contains three different adaptation modules, i.e., style adaptation, semantic adaptation and disparity range adaptation, to progressively adapt a terrestrial depth estimation model to the underwater domain. Specifically, due to the lack of underwater training data, we first propose a depth-aware stereo image translation network to synthesize stylized underwater stereo images from terrestrial dataset, thus benefiting the effective training of depth estimation network. Then, considering the weak generalization to the real underwater data when only trained on the above synthetic data, we present a self-ensembling semantic adaptation for depth estimation network to minimize the semantic domain discrepancy between synthetic and real underwater data. Meanwhile, we design a disparity range adaptation module to address the problem of disparity range miss-match between both data, thus obtaining more accurate depth predictions for large-disparity-span underwater images. Experimental results show that by integrating the proposed adaptation modules into the off-the-shelf depth estimation backbones, our method successfully achieves superior performance of underwater depth estimation compared to other state-of-the-art methods.
UIESC: An Underwater Image Enhancement Framework via Self-Attention and Contrastive Learning

Low contrast, color distortion, and blurred details are common problems that perplex vision-guided underwater robots. To this end, we propose an underwater image enhancement framework via self-attention and contrastive learning (UIESC) to solve these problems. In this article, local features and global dependencies are constructed through space and channel dual attention, and criss-cross attention is used to solve the high computational complexity of self-attention. Moreover, contrastive learning is introduced into network training as a loss function, and contrastive regularization ensures that the enhanced images are closer to clear positive samples and away from source negative samples. Finally, smoothed-histogram equalization is adopted for further optimization to accommodate complex and variable underwater scenes. Extensive experiments have shown that our framework outperforms state-of-the-art methods in underwater image enhancement tasks.
An Efficient Model For Student Behavior Recognition in Classroom

AI and big data analysis for student classroom behavior recognition can be used as auxiliary means to improve teaching quality. Recognition in classroom scenarios suffers from issues such as tiny targets and complex environmental interference. To tackle these problems, an efficient model based on YOLOv4-tiny is proposed in this paper. Specifically, we design a new module named ResBlock-S to reduce the floating point operations (FLOPs) of the model to improve the speed. Then, the introduction of the Convolutional Block Attention Module (CBAM) mechanism to obtain extra local information of images during the training process, which can ensure the recognition accuracy. As most available public datasets are not applicable to this work, we construct a classroom behavior dataset. Experiments were conducted on the public dataset and our self-built dataset to verify the performance of our model in general scenarios and classroom scenarios, respectively. Compared with YOLOv4-tiny and other lightweight CNN models such as MobileNetv2, MobileNetv3 and ShuffleNetv2, the mean Average Precision (mAP) of our approach on the self-built dataset is higher and up to 89.9%. Additionally, the detection speed of our approach is faster than the aforementioned methods, which is up to 167 fps.
Automatic Calcification Morphology and Distribution Classification for Breast Mammograms With Multi-Task Graph Convolutional Neural Network

The morphology and distribution of microcalcifications are the most important descriptors for radiologists to diagnose breast cancer based on mammograms. However, it is very challenging and time-consuming for radiologists to characterize these descriptors manually, and there also lacks of effective and automatic solutions for this problem. We observed that the distribution and morphology descriptors are determined by the radiologists based on the spatial and visual relationships among calcifications. Thus, we hypothesize that this information can be effectively modelled by learning a relationship-aware representation using graph convolutional networks (GCNs). In this study, we propose a multi-task deep GCN method for automatic characterization of both the morphology and distribution of microcalcifications in mammograms. Our proposed method transforms morphology and distribution characterization into node and graph classification problem and learns the representations concurrently. We trained and validated the proposed method in an in-house dataset and public DDSM dataset with 195 and 583 cases,respectively. The proposed method reaches good and stable results with distribution AUC at 0.812 ± 0.043 and 0.873 ± 0.019, morphology AUC at 0.663 ± 0.016 and 0.700 ± 0.044 for both in-house and public datasets. In both datasets, our proposed method demonstrates statistically significant improvements compared to the baseline models. The performance improvements brought by our proposed multi-task mechanism can be attributed to the association between the distribution and morphology of calcifications in mammograms, which is interpretable using graphical visualizations and consistent with the definitions of descriptors in the standard BI-RADS guideline. In short, we explore, for the first time, the application of GCNs in microcalcification characterization that suggests the potential of using graph learning for more robust understanding of medical images.
Student Action Recognition Based on Fuzzy Broad Learning System

Automatic recognition of student action is an important means to evaluate students' learning status in the class. It also provides a technique for measuring the effectiveness of teaching. However, the complexity of student action poses a challenge to automatic recognition. In this paper, a student action recognition method based on the fuzzy broad learning system (fuzzy BLS) is proposed. Fuzzy BLS is designed by merging the Takagi-Sugeno (TS) fuzzy system into BLS. As a neuro-fuzzy model, fuzzy BLS overcomes some problems, such as suffering from a time-consuming training stage and a large number of fuzzy rules. To get more abundant local features from student action images, we use the Scale-Invariant Feature Transform (SIFT) descriptor combined with the Local LogEuclidean Multivariate Gaussian $(\mathrm{L}^{2}\mathrm{E}\mathrm{M}\mathrm{G})$ descriptor to extract image features. Then, the extracted features are fed into fuzzy BLS after dimension reduction. The experimental results on the self-built dataset have shown that the proposed student action recognition method achieves better performance than other benchmarking methods.
Intelligent Movement Tracking Robot

Due to their numerous uses in manufacturing and modern society, Human Following robots have been studied and actively developed for many years. The robot needs to use a variety of methods, including robot control algorithm, human target detection, and obstacle avoidance. This project showcases a four-wheeled robot that follows people while equipped with an ultrasonic sensor to prevent collisions. The user is located by the robot using feedback from ultrasonic sensors. This robot can assist humans in carrying a variety of accessories, luggage, and highly sensitive chemicals in a number of different applications. Using ultrasonic sensors, an algorithm was created to track and position a human user while maintaining a constant distance from the robot. In the presence of numerous such obstacles, the IR sensors are used to uniquely identify the user. The most crucial element for an autonomous robot is communication between the robot and the user. There has been a noticeable increase in awareness of the use of these technologies. The creation of such robots is tangentially related to this research. Fully autonomous robots ought to be able to connect with the person controlling them in some way in addition to performing the tasks that are expected of them. Two IR sensors and an ultrasonic sensor are employed. Ultrasonic sensors are used to move the robot backward and IR sensors are used to follow a person or object.
Human Activity Recognition Using LSTM Network with Dropout Technique

Human Activity Recognition (HAR) is one of the fields where the research is going on in the field of deep learning and machine learning. As the everyday work that we do is ubiquitous (present everywhere), it would be helpful if a model can predict the activities performed. Many unsupervised techniques were used before but with the advent of deep learning techniques and neural networks, it has become easy for us to train and test the datasets with more accuracy. Recurrent Neural Networks (RNN) will be used as we will be working on time series data. RNNs use Backpropagation through time (BPTT) algorithm. The Long Short Term Memory (LSTM) model combined with the dropout mechanism deal with issues of vanishing gradients and overfitting of the model. LSTM consists of a cell state for preserving memory and forgetting it, whereas dropout nullifies some active states, allowing other states to participate and train the data. In this project, a better strategy has been considered that is easier to implement than the existing deep learning models that are contemplated here. This method achieved a high accuracy of 92.67% on the experimented dataset.
Emoji Anticipation and Prediction Using Deep Neural Network Model

Emojis are tiny pictorial depictions of feelings or objects that are frequently used in text messaging to enhance intercultural dialogue. Emojis are increasingly being used by users of social media sites like Twitter and Instagram in their text messages to express broader emotions that are frequently challenging to express in words alone. Combining text and emojis to express emotion has become a crucial part of how people communicate in the twenty-first century. In order to ascertain the connection between these text messages and the emojis used in them, a problem statement has been developed. In this study, we offer a technique for anticipating several emojis for a given text-based tweet message. Three modules make up the suggested model. The given text data is preprocessed in the first module, the model on which the data is trained is in the second, and a multi-class classifier is used in the third module to predict the emojis that the given text would evoke. This proposed deep neural network model aims to understand the underlying semantics of a text sentence using natural language processing techniques in order to predict appropriate emojis. Experiments performed utilizing a sizable data collection of Twitter-sourced real-world emoji usage.
Automated Road Monitoring System Using Machine Learning

Automated Road Monitoring System (ARMS) is an integrated system for reducing road accidents through surveillance. The main focus of this is on the signal mechanism on hill roads and the second one is the surveillance and alerting system that is helpful for the people in hilly regions and travelers. In developing countries accidents are the cause for death for the majority reasons. “Speed Kills”, but still people are lacking to drive safely on the road. Nowadays these people are the main reason for this road traffic accidents that are responsible for death as well as these also contribute a huge loss in the country's economy. About 2.3 million people die in India per year according to the Million Death Study (MDS). Out of these 137000 are because of road traffic accidents. That is about 377 people per day. We can see that many of them are curved roads. Thousands of people lose their lives each year because of this situation. As far as concerned, mountain road accidents are not considered to a level and these mountain accidents must be looked upon seriously. One of the solutions for this problem is alerting the driver about the vehicles coming from opposite sides in hair pin road sections. In India there are around 13.6% of accidents due to hilly curves. This system is one of the solutions for reducing these kinds of accidents by installing Automated Road surveillance systems in accident-prone regions. This integrated system also has an accident detection system that uses machine learning and image processing techniques for surveillance.
Deep Learning and Machine Learning Model Comparison for Diagnosis Detection from Medical Records

Structured data is needed in hospitals as a means of exchanging information between doctors, nurses, pharmacy department, coder/medical record section, and administration section. Structured data improves interoperability and uniformity of interpretation between entities working in the hospital. One of the stages of the process to generate structured data such as ICD codes is to detect diagnoses from medical records written by doctors. The entity in charge of interpreting medical records and determining the relevant ICD code according to the doctor's diagnosis written in the medical record is called a coder. In determining the ICD code, the coder looks for the patient's diagnosis in the medical record. However, coders with minimal experience may find it challenging to find a patient's diagnosis. This will cause inaccuracy in determining the ICD code to diagnose the patient's disease. This research constructed a predictor in the diagnostic recommendation system. We developed a supervised deep learning model, which is an LSTM model, and a Stochastic Gradient Descent model as a baseline in this study. Compared to the Stochastic Gradient Descent model, it was discovered that the proposed LSTM model produced the best results, reaching up to 98% accuracy in 14 epochs.
A Systematic Survey for Differential Privacy Techniques in Federated Learning

Federated learning is a distributed machine learning technique that trains a global model by exchanging model parameters or intermediate results among multiple data sources.Although federated learning achieves physical isolation of data, the local data of federated learning clients are still at risk of leakage under the attack of malicious individuals.For this reason, combining data protection techniques (e.g., differential privacy techniques) with federated learning is a sure way to further improve the data security of federated learning models.In this survey, we review recent advances in the research of differentially-private federated learning models.First, we introduce the workflow of federated learning and the theoretical basis of differential privacy.Then, we review three differentially-private federated learning paradigms: central differential privacy, local differential privacy, and distributed differential privacy.After this, we review the algorithmic optimization and communication cost optimization of federated learning models with differential privacy.Finally, we review the applications of federated learning models with differential privacy in various domains.By systematically summarizing the existing research, we propose future research opportunities.
Simulation of the irradiance and yield calculation of bifacial PV systems in the USA and Germany by combining ray tracing and view factor model

Compared to conventional photovoltaic (PV), there are more influencing factors in bifacial photovoltaics to be considered to calculate incoming irradiance and energy yield. Accurate models to investigate the influences of the elevation, the albedo of the ground, the shading conditions between the PV rows and many other rear-side related factors are required. This paper combines the ray tracing (RT) and view factor (VF) models to calculate the irradiance with a subsequent electrical yield calculation using the one-diode model. To verify the results of the developed open-source simulation program BifacialSimu, accurate data from a plant in Golden, USA (single-axis tracked) and a commercially operated plant in Germany (fixed-tilt) are used. Through comparisons to the actual data, it can be concluded that a combination of RT and VF models seems to be valid for longer simulation periods with several months since the relative errors balance out. The RT-only simulation accurately reproduces the precise hourly radiation and electrical yield pattern. Still, a continuous positive deviation was found, which does not even out over long periods and is thus less accurate than the VF/RT combination. A simulation for a single month with RT can take several hours. Thus, the best simulation mode results according to user requirements.