Reviewerly/exHarmony huggingface dataset:
A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis

The end-to-end aspect-based sentiment analysis (ABSA) task remains to be a long-standing challenge, which aims to extract the aspect term and then identify its sentiment orientation. In previous approaches, the explicit syntactic structure of a sentence, which reflects the syntax properties of natural language and hence is intuitively crucial for aspect term extraction and sentiment recognition, is insufficiently modeled. In this paper, we thus propose a novel dependency syntactic knowledge augmented interactive architecture with multi-task learning for end-to-end ABSA. This model is capable of fully exploiting the syntactic knowledge (dependency relations and types) by leveraging a well-designed Dependency Relation Embedded Graph Convolutional Network. Additionally, we design a simple yet effective message-passing mechanism to ensure that our model learns from multiple related tasks in a multi-task learning framework. Extensive experimental results on three benchmark datasets demonstrate the effectiveness of our approach, which significantly outperforms the existing state-of-the-art method. Besides, we achieve further improvements by using BERT as an additional feature extractor.
A Secure Clinical Diagnosis With Privacy-Preserving Multiclass Support Vector Machine in Clouds

With the rapid growth of machine learning and artificial intelligence in medical cloud systems, cloud-aided medical computing has materialized a concrete platform for rapid realization of the service-oriented computing paradigm. However, using machine learning approaches in medical clouds might cause serious privacy leakage risk, for example, it needs to take the patients&#x2019; sensitive data as the inputs of the machine learning algorithm. In this article, we propose an efficient and privacy-preserving clinical diagnosis scheme that was performed by an (untrusted and malicious) outsourced cloud platform, which can provide the diagnosis assistance for doctors without leaking any sensitive information of patients and service providers. Concretely, we give the security model of privacy-preserving multiclass support vector machine (SVM) in outsourced medical clouds and design a secure clinical diagnosis scheme with privacy-preserving multiclass predecision and diagnosis. We propose a novel encoding approach to implement the encryption of negative number, and design several security building blocks, such as privacy-preserving decision function computing, privacy-preserving classification, and search of max decision function on encrypted fields, to provide the construction of the secure clinical diagnosis with privacy-preserving multiclass SVM. We give the concrete scheme and provide the experiment on Dermatology datasheet. Security analysis and experimental results indicate that the proposed scheme is efficient and practical in privacy-preserving clinical diagnosis systems.
MMBERT: Multimodal BERT Pretraining for Improved Medical VQA

Images in the medical domain are fundamentally different from the general domain images. Consequently, it is infeasible to directly employ general domain Visual Question Answering (VQA) models for the medical domain. Additionally, medical image annotation is a costly and time-consuming process. To overcome these limitations, we propose a solution inspired by self-supervised pretraining of Transformer-style architectures for NLP, Vision, and Language tasks. Our method involves learning richer medical image and text semantic representations using Masked Vision-Language Modeling as the pretext task on a large medical image + caption dataset. The proposed solution achieves new state-of-the-art performance on two VQA datasets for radiology images - VQA-Med 2019 and VQA-RAD, outperforming even the ensemble models of previous best solutions. Moreover, our solution provides attention maps which help in model interpretability.
Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation is receiving great attention due to its low human annotation cost. In this paper, we aim to tackle bounding box supervised semantic segmentation, i.e., training accurate semantic segmentation models using bounding box annotations as supervision. To this end, we propose affinity attention graph neural network ( A2GNN). Following previous practices, we first generate pseudo semantic-aware seeds, which are then formed into semantic graphs based on our newly proposed affinity Convolutional Neural Network (CNN). Then the built graphs are input to our A2GNN, in which an affinity attention layer is designed to acquire the short- and long- distance information from soft graph edges to accurately propagate semantic labels from the confident seeds to the unlabeled pixels. However, to guarantee the precision of the seeds, we only adopt a limited number of confident pixel seed labels for A2GNN, which may lead to insufficient supervision for training. To alleviate this issue, we further introduce a new loss function and a consistency-checking mechanism to leverage the bounding box constraint, so that more reliable guidance can be included for the model optimization. Experiments show that our approach achieves new state-of-the-art performances on Pascal VOC 2012 datasets (val: 76.5 percent, test: 75.2 percent). More importantly, our approach can be readily applied to bounding box supervised instance segmentation task or other weakly supervised semantic segmentation tasks, with state-of-the-art or comparable performance among almot all weakly supervised tasks on PASCAL VOC or COCO dataset. Our source code will be available at https://github.com/zbf1991/A2GNN.
Variational Transformer Networks for Layout Generation

Generative models able to synthesize layouts of different kinds (e.g. documents, user interfaces or furniture arrangements) are a useful tool to aid design processes and as a first step in the generation of synthetic data, among other tasks. We exploit the properties of self-attention layers to capture high level relationships between elements in a layout, and use these as the building blocks of the well-known Variational Autoencoder (VAE) formulation. Our proposed Variational Transformer Network (VTN) is capable of learning margins, alignments and other global design rules without explicit supervision. Layouts sampled from our model have a high degree of resemblance to the training data, while demonstrating appealing diversity. In an extensive evaluation on publicly available benchmarks for different layout types VTNs achieve state-of-the-art diversity and perceptual quality. Additionally, we show the capabilities of this method as part of a document layout detection pipeline.
Multimodal Data Guided Spatial Feature Fusion and Grouping Strategy for E-Commerce Commodity Demand Forecasting

E-commerce offers various merchandise for selling and purchasing with frequent transactions and commodity flows. An accurate prediction of customer needs and optimized allocation of goods is required for cost reduction. The existing solutions have significant errors and are unsuitable for addressing warehouse needs and allocation. That is why businesses cannot respond to customer demands promptly, as they need accurate and reliable demand forecasting. Therefore, this paper proposes spatial feature fusion and grouping strategies based on multimodal data and builds a neural network prediction model for e-commodity demand. The designed model extracts order sequence features, consumer emotional features, and facial value features from multimodal data from e-commerce products. Then, a bidirectional long short-term memory network- (BiLSTM-) based grouping strategy is proposed. The proposed strategy fully learns the contextual semantics of time series data while reducing the influence of other features on the group’s local features. The output features of multimodal data are highly spatially correlated, and this paper employs the spatial dimension fusion strategy for feature fusion. This strategy effectively obtains the deep spatial relations among multimodal data by integrating the features of each column in each group across spatial dimensions. Finally, the proposed model’s prediction effect is tested using e-commerce dataset. The experimental results demonstrate the proposed algorithm’s effectiveness and superiority.
MDC-net: A new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information

Accurate segmentation of nuclei in digital pathology images can assist doctors in diagnosing diseases and evaluating subsequent treatments. Manual segmentation of nuclei from pathology images is time-consuming because of the large number of nuclei and is also error-prone. Therefore, accurate and automatic nucleus segmentation methods are required. Owing to the large variations in the characterization of nuclei, it is difficult to accurately segment nuclei using traditional methods. In this study, we propose a new method for nucleus segmentation. The proposed method uses a deep fully convolutional neural network to perform end-to-end segmentation on pathological tissue slices. Multiple short residual connections were used to fuse feature maps from different scales to better utilize the context information. Dilated convolutions with different dilation ratios were used to increase the receptive fields. In addition, we incorporated the distance map and contour information into the segmentation method to segment touching nuclei, which is difficult via traditional segmentation methods. Finally, post-processing was used to improve the segmentation results. The results demonstrate that our segmentation method can obtain comparable or better performance than other state-of-the-art methods on the public nuclei histopathology datasets.
From Rain Generation to Rain Removal

For the single image rain removal (SIRR) task, the performance of deep learning (DL)-based methods is mainly affected by the designed deraining models and training datasets. Most of current state-of-the-art focus on constructing powerful deep models to obtain better deraining results. In this paper, to further improve the deraining performance, we novelly attempt to handle the SIRR task from the perspective of training datasets by exploring a more efficient way to synthesize rainy images. Specifically, we build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator with the input as some latent variables representing the physical structural rain factors, e.g., direction, scale, and thickness. To solve this model, we employ the variational inference framework to approximate the expected statistical distribution of rainy image in a data-driven manner. With the learned generator, we can automatically and sufficiently generate diverse and non-repetitive training pairs so as to efficiently enrich and augment the existing benchmark datasets. User study qualitatively and quantitatively evaluates the realism of generated rainy images. Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution that not only helps significantly improve the deraining performance of current deep single image derainers, but also largely loosens the requirement of large training sample pre-collection for the SIRR task. Code is available in https://github.com/hongwang01/VRGNet.
The <scp>Flores-101</scp> Evaluation Benchmark for Low-Resource and Multilingual Machine Translation

Abstract One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the Flores-101 evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are fully aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.
DETReg: Unsupervised Pretraining with Region Priors for Object Detection

Recent self-supervised pretraining methods for object detection largely focus on pretraining the backbone of the object detector, neglecting key parts of detection architecture. Instead, we introduce DETReg, a new self-supervised method that pretrains the entire object detection network, including the object localization and embedding components. During pretraining, DETReg predicts object localizations to match the localizations from an unsupervised region proposal generator and simultaneously aligns the corresponding feature embeddings with embeddings from a self-supervised image encoder. We implement DETReg using the DETR family of detectors and show that it improves over competitive baselines when finetuned on COCO, PASCAL VOC, and Airbus Ship benchmarks. In low-data regimes, including semi-supervised and few-shot learning settings, DETReg establishes many state-of-the-art results, e.g., on COCO we see a +6.0 AP improvement for 10-shot detection and over 2 AP improvements when training with only 1 % of the labels. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code: https://www.amirbar.net/detreg/.
Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation

We address the problem of weakly-supervised semantic segmentation (WSSS) using bounding box annotations. Although object bounding boxes are good indicators to segment corresponding objects, they do not specify object boundaries, making it hard to train convolutional neural networks (CNNs) for semantic segmentation. We find that background regions are perceptually consistent in part within an image, and this can be leveraged to discriminate foreground and background regions inside object bounding boxes. To implement this idea, we propose a novel pooling method, dubbed background-aware pooling (BAP), that focuses more on aggregating foreground features inside the bounding boxes using attention maps. This allows to extract high-quality pseudo segmentation labels to train CNNs for semantic segmentation, but the labels still contain noise especially at object boundaries. To address this problem, we also introduce a noise-aware loss (NAL) that makes the networks less susceptible to incorrect labels. Experimental results demonstrate that learning with our pseudo labels already outperforms state-of-the-art weakly- and semi-supervised methods on the PASCAL VOC 2012 dataset, and the NAL further boosts the performance.
IT Governance Framework and Smart Services Integration for Future Development of Dubai Infrastructure Utilizing AI and Big Data, Its Reflection on the Citizens Standard of Living

The development of smart cities worldwide has been necessitated by the unprecedented increase in the global population. The formation of the Dubai Roads and Transport Authority was in response to Dubai's need to develop and upgrade its infrastructure to facilitate ease of commuting through the emirate and recently elevate it to a smart one. One of the challenges of smart cities is the failure to develop proper governance to oversee project implementation This research fills the existing gap in acquiring a holistic view of RTA's IT infrastructure and its impact on service delivery and improvement of people's living standards. The research was conducted by studying RTA and its IT governance framework to analyze Dubai RTA's IT governance framework and how they have integrated smart services for future development of Dubai infrastructure utilizing AI and Big Data to serve the collective community of Dubai better. However, there is a need for other public service providers to be included in the service delivery to increase overall government efficiency.
3D-MAN: 3D Multi-frame Attention Network for Object Detection

3D object detection is an important module in autonomous driving and robotics. However, many existing methods focus on using single frames to perform 3D detection, and do not fully utilize information from multiple frames. In this paper, we present 3D-MAN: a 3D multi-frame attention network that effectively aggregates features from multiple perspectives and achieves state-of-the-art performance on Waymo Open Dataset. 3D-MAN first uses a novel fast single-frame detector to produce box proposals. The box proposals and their corresponding feature maps are then stored in a memory bank. We design a multi-view alignment and aggregation module, using attention networks, to extract and aggregate the temporal features stored in the memory bank. This effectively combines the features coming from different perspectives of the scene. We demonstrate the effectiveness of our approach on the large-scale complex Waymo Open Dataset, achieving state-of-the-art results compared to published single-frame and multi-frame methods.
Perception-Aware Multi-Sensor Fusion for 3D LiDAR Semantic Segmentation

3D LiDAR (light detection and ranging) semantic segmentation is important in scene understanding for many applications, such as auto-driving and robotics. For example, for autonomous cars equipped with RGB cameras and LiDAR, it is crucial to fuse complementary information from different sensors for robust and accurate segmentation. Existing fusion-based methods, however, may not achieve promising performance due to the vast difference between the two modalities. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) to exploit perceptual information from two modalities, namely, appearance information from RGB images and spatio-depth information from point clouds. To this end, we first project point clouds to the camera coordinates to provide spatio-depth information for RGB images. Then, we propose a two-stream network to extract features from the two modalities, separately, and fuse the features by effective residual-based fusion modules. Moreover, we propose additional perception-aware losses to measure the perceptual difference between the two modalities. Extensive experiments on two benchmark data sets show the superiority of our method. For example, on nuScenes, our PMF outperforms the state-of-the-art method by 0.8% in mIoU.
DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network

Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse. To solve this issue, previous works [47], [22] mainly focused on encouraging the correlation between the latent codes and their generated images, while ignoring the relations between images generated from various latent codes. The recent MSGAN [27] tried to encourage the diversity of the generated image but only considers "negative" relations between the image pairs.In this paper, we propose a novel DivCo framework to properly constrain both "positive" and "negative" relations between the generated images specified in the latent space. To the best of our knowledge, this is the first attempt to use contrastive learning for diverse conditional image synthesis. A novel latent-augmented contrastive loss is introduced, which encourages images generated from adjacent latent codes to be similar and those generated from distinct latent codes to be dissimilar. The proposed latent-augmented contrastive loss is well compatible with various cGAN architectures. Extensive experiments demonstrate that the proposed DivCo can produce more diverse images than state-of-the-art methods without sacrificing visual quality in multiple unpaired and paired image generation tasks. Training code and pretrained models are available at https://github.com/ruiliu-ai/DivCo.
Novel application of adaptive swarm intelligence techniques coupled with adaptive network-based fuzzy inference system in predicting photovoltaic power

Accurate photovoltaic (PV) power prediction is necessary for future development of the micro-grids projects and the economic dispatch sector. This study investigates the potential of using a novel hybrid approach of adaptive swarm intelligence techniques and adaptive network-based fuzzy inference system (ANFIS) in estimating the PV power of a solar system at different time horizons, from 0 to 24 h. The developed approach is an integration of ANFIS and particle swarm optimization (PSO) with adaptive and time-varying acceleration coefficients, i.e., ANFIS-APSO (ANFIS-PSO with adaptive acceleration coefficients) and ANFIS-IPSO (ANFIS-PSO with time-varying acceleration coefficients), were developed. The performance of the proposed models was compared with other hybrid ANFIS models, namely ANFIS-PSO (ANFIS coupled with PSO), ANFIS-BBO (ANFIS coupled with biogeography-based optimization), ANFIS-GA (ANFIS coupled with genetic algorithm), and ANFIS-GWO (ANFIS coupled with grey wolf optimization). For this purpose, the climatic variables and historical PV power data of a 960 kWP grid-connected PV system in the south of Italy were used to design and evaluate the models. Several statistical analyses were implemented to evaluate the accuracy of the proposed models and assess the impact of variables that affects the PV power values. The experimental results show that the proposed ANFIS-APSO attained the most accurate prediction of the PV power with R2 = 0.835 and 0.657, RMSE = 0.088 kW and 0.081 kW, and MAE = 0.077 kW and 0.079 kW in the testing phase at time horizons 12 h and 24 h, respectively. Based on the obtained results, the newly constructed ANFIS-APSO outperformed the standard ANFIS-PSO model including other hybrid models, and hence very potential to be a new alternative to assist engineers for predicting the PV power of solar systems at short- and long-time horizons.
SDD-FIQA: Unsupervised Face Image Quality Assessment with Similarity Distribution Distance

In recent years, Face Image Quality Assessment (FIQA) has become an indispensable part of the face recognition system to guarantee the stability and reliability of recognition performance in an unconstrained scenario. For this purpose, the FIQA method should consider both the intrinsic property and the recognizability of the face image. Most previous works aim to estimate the sample-wise embedding uncertainty or pair-wise similarity as the quality score, which only considers the partial information from the intra-class. However, these methods ignore the valuable in-formation from the inter-class, which is for estimating the recognizability of face image. In this work, we argue that a high-quality face image should be similar to its intra-class samples and dissimilar to its inter-class samples. Thus, we propose a novel unsupervised FIQA method that incorporates Similarity Distribution Distance for Face Image Quality Assessment (SDD-FIQA). Our method generates quality pseudo-labels by calculating the Wasserstein Distance (WD) between the intra-class and inter-class similarity distributions. With these quality pseudo-labels, we are capable of training a regression network for quality prediction. Extensive experiments on benchmark datasets demonstrate that the proposed SDD-FIQA surpasses the state-of-the-arts by an impressive margin. Meanwhile, our method shows good generalization across different recognition systems.