Reviewerly/exHarmony huggingface dataset:
Dense context distillation network for semantic parsing of oblique UAV images

Semantic segmentation of oblique unmanned aerial vehicle (UAV) images serves as a foundation for many modern urban applications, such as road scene monitoring and semantic 3D modeling. However, objects in UAV images can vary intensely in size and undergo severe perspective distortion because of the oblique viewing style. Existing general segmentation models designed for ground and remote sensing images rarely considered these challenges specific to UAV images. Therefore, they have large difficulties in learning discriminative representation for simultaneously reasoning the extremely large and small objects in UAV images. In this paper, we propose a dense context distillation network (DCDNet) to learn distortion-robust feature representation for semantic segmentation of UAV images. The basic DCDNet is deployed as an dual-branch encoder–decoder architecture. To accomplish the goal of dense context distillation, DCDNet is first equipped with several cross-scale context selectors at different encoding stages to densely and selectively gather the useful context from low- to high-level dual-scale feature maps. A joint supervision is then applied to reinforce the learning of shallower features for distilling more low-level contexts that are vital to the reasoning of small or thin structures. A multi-scale feature aggregator is incorporated to adaptively fuse the long-range context during decoding, which absorbs the complementary merits of the dense context captured from feature maps of different levels. With the dense context distillation, DCDNet is more capable of offering the differently scaled objects with the required context for better learning and prediction. Extensive experiments on the challenging UAVid dataset demonstrate that our DCDNet can well adapt to the oblique UAV images, achieving a state-of-the-art segmentation performance with a mIoU score of 72.38%.
End-to-End Deep Multi-Score Model for No-Reference Stereoscopic Image Quality Assessment

Deep learning-based quality metrics have recently given significant improvement in Image Quality Assessment (IQA).In the field of stereoscopic vision, information is evenly distributed with slight disparity to the left and right eyes.However, due to asymmetric distortion, the objective quality ratings for the left and right images would differ, necessitating the learning of unique quality indicators for each view.Unlike existing stereoscopic IQA measures which focus mainly on estimating a global human score, we suggest incorporating left, right, and stereoscopic objective scores to extract the corresponding properties of each view, and so forth estimating stereoscopic image quality without reference.Therefore, we use a deep multi-score Convolutional Neural Network (CNN).Our model has been trained to perform four tasks: First, predict the left view's quality.Second, predict the quality of the left view.Third and fourth, predict the quality of the stereo view and global quality, respectively, with the global score serving as the ultimate quality.Experiments are conducted on Waterloo IVC 3D Phase 1 and Phase 2 databases.The results obtained show the superiority of our method when comparing with those of the state-of-the-art.The implementation code can be found at: https://github.com/o-messai/multiscore-SIQA
Mirror Segmentation via Semantic-aware Contextual Contrasted Feature Learning

Mirrors are everywhere in our daily lives. Existing computer vision systems do not consider mirrors, and hence may get confused by the reflected content inside a mirror, resulting in a severe performance degradation. However, separating the real content outside a mirror from the reflected content inside it is non-trivial. The key challenge is that mirrors typically reflect contents similar to their surroundings, making it very difficult to differentiate the two. In this article, we present a novel method to segment mirrors from a single RGB image. To the best of our knowledge, this is the first work to address the mirror segmentation problem with a computational approach. We make the following contributions: First, we propose a novel network, called MirrorNet+, for mirror segmentation, by modeling both contextual contrasts and semantic associations. Second, we construct the first large-scale mirror segmentation dataset, which consists of 4,018 pairs of images containing mirrors and their corresponding manually annotated mirror masks, covering a variety of daily-life scenes. Third, we conduct extensive experiments to evaluate the proposed method and show that it outperforms the related state-of-the-art detection and segmentation methods. Fourth, we further validate the effectiveness and generalization capability of the proposed semantic awareness contextual contrasted feature learning by applying MirrorNet+ to other vision tasks, i.e., salient object detection and shadow detection. Finally, we provide some applications of mirror segmentation and analyze possible future research directions. Project homepage: https://mhaiyang.github.io/TOMM2022-MirrorNet+/index.html .
PACMem

Memory safety is a key security property that stops memory corruption vulnerabilities. Different types of memory safety enforcement solutions have been proposed and adopted by sanitizers or mitigations to catch and stop such bugs, at the development or deployment phase. However, existing solutions either provide partial memory safety or have overwhelmingly high performance overheads.
Modified EDA and Backtranslation Augmentation in Deep Learning Models for Indonesian Aspect-Based Sentiment Analysis

In the process of developing a business, aspect-based sentiment analysis (ABSA) could help extract customers' opinions on different aspects of the business from online reviews. Researchers have found great prospective in deep learning approaches to solving ABSA tasks. Furthermore, studies have also explored the implementation of text augmentation, such as Easy Data Augmentation (EDA), to improve the deep learning models’ performance using only simple operations. However, when implementing EDA to ABSA, there will be high chances that the augmented sentences could lose important aspects or sentiment-related words (target words) critical for training. Corresponding to that, another study has made adjustments to EDA for English aspect-based sentiment data provided with the target words tag. However, the solution still needs additional modifications in the case of non-tagged data. Hence, in this work, we will focus on modifying EDA that integrates POS tagging and word similarity to not only understand the context of the words but also extract the target words directly from non-tagged sentences. Additionally, the modified EDA is combined with the backtranslation method, as the latter has also shown quite a significant contribution to the model’s performance in several research studies. The proposed method is then evaluated on a small Indonesian ABSA dataset using baseline deep learning models. Results show that the augmentation method could increase the model’s performance on a limited dataset problem. In general, the best performance for aspect classification is achieved by implementing the proposed method, which increases the macro-accuracy and F1, respectively, on Long Short-Term Memory (LSTM) and Bidirectional LSTM models compared to the original EDA. The proposed method also obtained the best performance for sentiment classification using a convolutional neural network, increasing the overall accuracy by 2.2% and F1 by 3.2%. Doi: 10.28991/ESJ-2023-07-01-018 Full Text: PDF
Scattering Inversion Study for Suspended Label-Free Lymphocytes with Complex Fine Structures

Objective and Impact Statement . Distinguishing malignant lymphocytes from normal ones is vital in pathological examination. We proposed an inverse light scattering (ILS) method for label-free suspended lymphocytes with complex fine structures to identify their volumes for pathological state. Introduction . Light scattering as cell’s “fingerprint” provides valuable morphology information closely related to its biophysical states. However, the detail relationships between the morphology with complex fine structures and its scattering characters are not fully understood. Methods . To quantitatively inverse the volumes of membrane and nucleus as the main scatterers, clinical lymphocyte morphologies were modeled combining the Gaussian random sphere geometry algorithm by 750 reconstructed results after confocal scanning, which allowed the accurate simulation to solve ILS problem. For complex fine structures, the specificity for ILS study was firstly discussed (to our knowledge) considering the differences of not only surface roughness, posture, but also the ratio of nucleus to the cytoplasm and refractive index. Results . The volumes of membrane and nucleus were proved theoretically to have good linear relationship with the effective area and entropy of forward scattering images. Their specificity deviations were less than 3.5%. Then, our experimental results for microsphere and clinical leukocytes showed the Pearson product-moment correlation coefficients (PPMCC) of this linear relationship were up to 0.9830~0.9926. Conclusion . Our scattering inversion method could be effectively applied to identify suspended label-free lymphocytes without destructive sample pretreatments and complex experimental systems.
Automated learning of interpretable models with quantified uncertainty

Interpretability and uncertainty quantification in machine learning can provide justification for decisions, promote scientific discovery and lead to a better understanding of model behavior. Symbolic regression provides inherently interpretable machine learning, but relatively little work has focused on the use of symbolic regression on noisy data and the accompanying necessity to quantify uncertainty. A new Bayesian framework for genetic-programming-based symbolic regression (GPSR) is introduced that uses model evidence (i.e., marginal likelihood) to formulate replacement probability during the selection phase of evolution. Model parameter uncertainty is automatically quantified, enabling probabilistic predictions with each equation produced by the GPSR algorithm. Model evidence is also quantified in this process, and its use is shown to increase interpretability, improve robustness to noise, and reduce overfitting when compared to a conventional GPSR implementation on both numerical and physical experiments.
Generalized Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data

The vanilla Few-shot Learning (FSL) learns to build a classifier for a new concept from one or very few target examples, with the general assumption that source and target classes are sampled from the same domain. Recently, the task of Cross-Domain Few-Shot Learning (CD-FSL) aims at tackling the FSL where there is a huge domain shift between the source and target datasets. Extensive efforts on CD-FSL have been made via either directly extending the meta-learning paradigm of vanilla FSL methods, or employing massive unlabeled target data to help learn models. In this paper, we notice that in the CD-FSL task, the few labeled target images have never been explicitly leveraged to inform the model in the training stage. However, such a labeled target example set is very important to bridge the huge domain gap. Critically, this paper advocates a more practical training scenario for CD-FSL. And our key insight is to utilize a few labeled target data to guide the learning of the CD-FSL model. Technically, we propose a novel Generalized Meta-learning based Feature-Disentangled Mixup network, namely GMeta-FDMixup. We make three key contributions of utilizing GMeta-FDMixup to address CD-FSL. Firstly, we present two mixup modules - mixup-P and mixup-M that help facilitate utilizing the unbalanced and disjoint source and target datasets. These two novel modules enable diverse image generation for training the model on the source domain. Secondly, to narrow the domain gap explicitly, we contribute a novel feature disentanglement module that learns to decouple the domain-irrelevant and domain-specific features. By stripping the domain-specific features, we alleviate the negative effects caused by the domain inductive bias. Finally, we repurpose a new contrastive learning module, dubbed ConL. ConL prevents the model from only capturing category-related features via introducing contrastive loss. Thus, the generalization ability on novel categories is improved. Extensive experimental results on two benchmarks show the superiority of our setting and the effectiveness of our method. Code and models will be released.
Cross-domain learning for underwater image enhancement

The poor quality of underwater images has become a widely-known cause affecting the performance of the underwater development projects, including mineral exploitation, driving photography, and navigation for autonomous underwater vehicles. In recent years, deep learning-based techniques have achieved remarkable successes in image restoration and enhancement tasks. However, the limited availability of paired training data (underwater images and their corresponding clear images) and the requirement for vivid color correction remain challenging for underwater image enhancement, as almost all learning-based methods require paired data for training. In this study, instead of creating the time-consuming paired data, we explore the unsupervised training strategy. Specifically, we introduce a universal cross-domain GAN-based framework to generate high-quality images to address the dependence on paired training data. To ensure the vivid colorfulness, the color loss is designed to constrain the training process. Also, a feature fusion module (FFM) is proposed to increase the capacity of the whole model as well as the dual discriminator channel adopted in the architecture. Extensive quantitative and perceptual experiments show that our approach overcomes the limitation of paired data and obtains superior performance over the state-of-the-art on several underwater benchmarks in terms of both accuracy and model deployment.
Understanding and Analyzing Factors that Affect Merge Conflicts from the Perspective of Software Developers

Merge conflicts are very common in collaborative software development, which is supported mainly by the use of branches that can be potentially merged. In this context, several studies have proposed mechanisms to avoid conflicts whenever possible and some identified factors that lead to conflicts. In this article, we report on an investigation of factors that can lead to conflicts or that can somehow reduce the chances of conflict from the developers’ perspective. To do so, based on related work, we conducted two empirical studies with Brazilian software developers to both understand and analyze factors that affect merge conflicts. Firstly, we conducted survey research with 109 software developers to understand how they use branches, the occurrence of conflicts and the resolution process, and factors that can lead to or avoid conflicts. Results showed that the use of branches is very common and mostly has the purpose of creating a new feature or fixing a bug. According to the participants, in most projects, developers have the autonomy to create new branches and sometimes conflicts happen. The main factors that can lead to conflicts are “the time a branch is isolated” and “lack of communication”. On the other hand, the factors cited as good practices to avoid conflicts were “improve team communication” and “less branching duration”. Secondly, we conducted a field study based on interviews with 15 software developers to analyze those factors to understand better what leads to or avoids conflicts in a merge. Finally, this work allowed us to conclude that communication with the team, checking code updates, shorter branch duration, and management are important for software developers, especially when they think about what increases and decreases merge conflicts.
Deep learning-based galaxy image deconvolution

With the onset of large-scale astronomical surveys capturing millions of images, there is an increasing need to develop fast and accurate deconvolution algorithms that generalize well to different images. A powerful and accessible deconvolution method would allow for the reconstruction of a cleaner estimation of the sky. The deconvolved images would be helpful to perform photometric measurements to help make progress in the fields of galaxy formation and evolution. We propose a new deconvolution method based on the Learnlet transform. Eventually, we investigate and compare the performance of different Unet architectures and Learnlet for image deconvolution in the astrophysical domain by following a two-step approach: a Tikhonov deconvolution with a closed-form solution, followed by post-processing with a neural network. To generate our training dataset, we extract HST cutouts from the CANDELS survey in the F606W filter (V-band) and corrupt these images to simulate their blurred-noisy versions. Our numerical results based on these simulations show a detailed comparison between the considered methods for different noise levels.
Proactive Moderation of Online Discussions: Existing Practices and the Potential for Algorithmic Support

To address the widespread problem of uncivil behavior, many online discussion platforms employ human moderators to take action against objectionable content, such as removing it or placing sanctions on its authors. Thisreactive paradigm of taking action against already-posted antisocial content is currently the most common form of moderation, and has accordingly underpinned many recent efforts at introducing automation into the moderation process. Comparatively less work has been done to understand other moderation paradigms---such as proactively discouraging the emergence of antisocial behavior rather than reacting to it---and the role algorithmic support can play in these paradigms. In this work, we investigate such a proactive framework for moderation in a case study of a collaborative setting: Wikipedia Talk Pages. We employ a mixed methods approach, combining qualitative and design components for a holistic analysis. Through interviews with moderators, we find that despite a lack of technical and social support, moderators already engage in a number of proactive moderation behaviors, such as preemptively intervening in conversations to keep them on track. Further, we explore how automation could assist with this existing proactive moderation workflow by building a prototype tool, presenting it to moderators, and examining how the assistance it provides might fit into their workflow. The resulting feedback uncovers both strengths and drawbacks of the prototype tool and suggests concrete steps towards further developing such assisting technology so it can most effectively support moderators in their existing proactive moderation workflow.
Multimodal metadata assignment for cultural heritage artifacts

We develop a multimodal classifier for the cultural heritage domain using a late fusion approach and introduce a novel dataset. The three modalities are Image, Text, and Tabular data. We based the image classifier on a ResNet convolutional neural network architecture and the text classifier on a multilingual transformer architecture (XML-Roberta). Both are trained as multitask classifiers. Tabular data and late fusion are handled by Gradient Tree Boosting. We also show how we leveraged a specific data model and taxonomy in a Knowledge Graph to create the dataset and to store classification results.
Machine-Learning Assisted Handwriting Recognition Using Graphene Oxide-Based Hydrogel

Machine-learning assisted handwriting recognition is crucial for development of next-generation biometric technologies. However, most of the currently reported handwriting recognition systems are lacking in flexible sensing and machine learning capabilities, both of which are essential for implementation of intelligent systems. Herein, assisted by machine learning, we develop a new handwriting recognition system, which can be applied as both a recognizer for written texts and an encryptor for confidential information. This flexible and intelligent handwriting recognition system combines a printed circuit board with graphene oxide-based hydrogel sensors. It offers fast response and good sensitivity and allows high-precision recognition of handwritten content from a single letter to words and signatures. By analyzing 690 acquired handwritten signatures obtained from seven participants, we successfully demonstrate a fast recognition time (less than 1 s) and a high recognition rate (∼91.30%). Our developed handwriting recognition system has great potential in advanced human-machine interactions, wearable communication devices, soft robotics manipulators, and augmented virtual reality.
Increasing validation accuracy of a face mask detection by new deep learning model-based classification

During COVID-19, wearing a mask was globally mandated in various workplaces, departments, and offices. New deep learning convolutional neural network (CNN) based classifications were proposed to increase the validation accuracy of face mask detection. This work introduces a face mask model that is able to recognize whether a person is wearing mask or not. The proposed model has two stages to detect and recognize the face mask; at the first stage, the Haar cascade detector is used to detect the face, while at the second stage, the proposed CNN model is used as a classification model that is built from scratch. The experiment was applied on masked faces (MAFA) dataset with images of 160x160 pixels size and RGB color. The model achieved lower computational complexity and number of layers, while being more reliable compared with other algorithms applied to recognize face masks. The findings reveal that the model's validation accuracy reaches 97.55% to 98.43% at different learning rates and different values of features vector in the dense layer, which represents a neural network layer that is connected deeply of the CNN proposed model training. Finally, the suggested model enhances recognition performance parameters such as precision, recall, and area under the curve (AUC).
3D Object Recognition Using Fast Overlapped Block Processing Technique

Three-dimensional (3D) image and medical image processing, which are considered big data analysis, have attracted significant attention during the last few years. To this end, efficient 3D object recognition techniques could be beneficial to such image and medical image processing. However, to date, most of the proposed methods for 3D object recognition experience major challenges in terms of high computational complexity. This is attributed to the fact that the computational complexity and execution time are increased when the dimensions of the object are increased, which is the case in 3D object recognition. Therefore, finding an efficient method for obtaining high recognition accuracy with low computational complexity is essential. To this end, this paper presents an efficient method for 3D object recognition with low computational complexity. Specifically, the proposed method uses a fast overlapped technique, which deals with higher-order polynomials and high-dimensional objects. The fast overlapped block-processing algorithm reduces the computational complexity of feature extraction. This paper also exploits Charlier polynomials and their moments along with support vector machine (SVM). The evaluation of the presented method is carried out using a well-known dataset, the McGill benchmark dataset. Besides, comparisons are performed with existing 3D object recognition methods. The results show that the proposed 3D object recognition approach achieves high recognition rates under different noisy environments. Furthermore, the results show that the presented method has the potential to mitigate noise distortion and outperforms existing methods in terms of computation time under noise-free and different noisy environments.