Reviewerly/exHarmony huggingface dataset:
Transformer Gate Attention Model: An Improved Attention Model for Visual Question Answering

Given an image and an opened-ended question related to the picture, the Visual Question Answering (VQA) model aims to provide the right answer to the question for the image. This is a challenging task that requires a fine-grained simultaneous realization of the visual content of the image and the textual content of the question. However, most current models ignore the region-to-word interaction and the noise effect of irrelevant words. This paper proposes a novel model called Transformer Gate Attention Model (TGAM) to capture the inter-modal information dependence to solve the above problems. Specifically, TGAM is composed of Adaptive Gate (AG) and Parallel Transformer Module (PTM). AG fuses information between different modalities and reduces noise, and PTM obtains more advanced cross-modal feature representation. Many qualitative and quantitative experiments were conducted on VQA-v2 dataset to verify the validity of TGAM. And extensive ablation studies have been conducted to explore the reasons behind the effectiveness of TGAM. Experimental results show that the performance of TGAM is significantly better than previous state-of-the-art technologies. Our best model achieved 71.28% overall accuracy on the test-dev set and 71.6% on the test-std set.
Rate-1 Incompressible Encryption from Standard Assumptions

Incompressible encryption, recently proposed by Guan, Wichs and Zhandry (EUROCRYPT’22), is a novel encryption paradigm geared towards providing strong long-term security guarantees against adversaries with bounded long-term memory. Given that the adversary forgets just a small fraction of a ciphertext, this notion provides strong security for the message encrypted therein, even if, at some point in the future, the entire secret key is exposed. This comes at the price of having potentially very large ciphertexts. Thus, an important efficiency measure for incompressible encryption is the message-to-ciphertext ratio (also called the rate). Guan et al. provided a low-rate instantiation of this notion from standard assumptions and a rate-1 instantiation from indistinguishability obfuscation (iO). In this work, we propose a simple framework to build rate-1 incompressible encryption from standard assumptions. Our construction can be realized from, e.g. the DDH and additionally the DCR or the LWE assumptions.
Unsupervised MRI Super Resolution Using Deep External Learning and Guided Residual Dense Network With Multimodal Image Priors

Deep learning techniques have led to state-of-the-art image super resolution with natural images. Normally, pairs of high-resolution and low-resolution images are used to train the deep learning models. These techniques have also been applied to medical image super-resolution. The characteristics of medical images differ significantly from natural images in several ways. First, it is difficult to obtain high-resolution images for training in real clinical applications due to the limitations of imaging systems and clinical requirements. Second, other modal high-resolution images are available (e.g., high-resolution T1-weighted images are available for enhancing low-resolution T2-weighted images). In this paper, we propose an unsupervised image super-resolution technique based on simple prior knowledge of the human anatomy. This technique does not require target T2WI high-resolution images for training. Furthermore, we present a guided residual dense network, which incorporates a residual dense network with a guided deep convolutional neural network for enhancing the resolution of low-resolution images by referring to different modal high-resolution images of the same subject. Experiments on a publicly available brain MRI database showed that our proposed method achieves better performance than the state-of-the-art methods.
Reliability of News and Toxicity in Twitter Conversations

Social media platforms like Twitter play a pivotal role in public debates. Recent studies showed that users online tend to join groups of like-minded peers, called echo chambers, in which they frame and reinforce a shared narrative. Such a polarized configuration may trigger heated debates and foment misinformation spreading. In this work, we explore the interplay between the systematic spreading of misinformation and the emergence of toxic conversations. We perform a thorough quantitative analysis on 3.3 M comments by more than 1 M unique users from 25 K conversations involving 60 news outlets active on Twitter from January 2020 to April 2022. By tagging the news triggering the conversation with a specific reliability score provided by an independent fact-checking organization (NewsGuard), we perform a network-based analysis of the structure of toxic conversations on Twitter. We find that users using toxic language are few and evenly distributed over the entire reliability score range, showing no significant evidence for the interplay between toxic speech and misinformation spreading.
Dynamic Expert-Knowledge Ensemble for Generalizable Video Quality Assessment

Despite the impressive progress of supervised methods in quality assessment for in- the-wild videos, models trained on one domain often fail to generalize well to others due to the domain shifts caused by distortion diversity and content variation. Domain generalizable video quality assessment (VQA) methods that can work across domains remain an open research challenge. Although combining more data following the mixed-domain training strategy can improve the generalization performance to a certain extent, the specific knowledge from each source domain, which could potentially be useful for improving unseen domain generalization, is ignored in this principle. Motivated by this, we propose a domain generalizable VQA method named Dynamic Ensemble of Expert-Knowledge (DEEK), a novel framework that dynamically exploits the expert-knowledge from each source domain to achieve a generalizable ensemble prediction. Specifically, based on the multiple experts each trained to specialize in a particular source domain, we aim to exploit complementary information provided by the expert-knowledge. We effectively train an ensemble model by proposing a quality-sensitive InfoNCE loss to regularize the collaborative training of all experts in the contrastive learning formulation, aiming to exploit complementary information provided by the expert-knowledge when forming the ensemble. By dynamically integrating the experts according to their relevances to the target data, these expert-knowledge could be leveraged for better generalization. Experiments on five VQA datasets verify that our approach outperforms the state-of-the-arts by large margins.
Deep Learning Framework for Brain Tumor Classification through Histopathological Image

Further most oncologist varies in their conclusions around the determination utilizing histopathological-pictures, so the manual diagnosis of brain tumour is troublesome, and it too needs elevated involvement. Building a show of brain tumour classification is a fundamental assignment. CAD empowers productive and exact conclusion of this sort of imaging. In addition, it makes a difference early determination of brain tumours. Profound learning has started to advance within the final decades, so convolutional neural systems/networks are utilized to classify brain tumours. We presented the programmed brain tumour classification strategies to assist specialists in examination and treatment, which can naturally classify the histopathological pictures as generous or dangerous. This study’s hypothetical premise is creating a profound learning tumour classification framework. It employs a profound learning approach that is the proposed CNN model comprises of three stages. The primary stage is pre-processing. The moment stage is the highlight extraction organize that takes input as augmented preprocessed pictures. This stage has two divisions. The primary stage comprises of three parallel CNN branches with different filter sizes to extricate distinctive include levels. We blended the three parallel branches within the moment stage, which produced a highlight combination way that diminished the time complexity and upgraded the outlined methods, performance. The third stage classifies the melded highlights as generous or dangerous. The Cancer-Genome-Atlas Glioblastoma Multiform-DB (TCGA-GBM) is utilized &implemented and contains 19,507 files of brain tumour tissue for 237 patients. The data was parted into three sections: 60%, 20%, 20% for training, validation, and testing. A few tests were performed to attain the finest demonstrative precision of 91.37 %. The results are compared to the outcomes about of different methods within the writing. The presented preferences of our method make it offer assistance within the briefest treatment time, particularly within the case of the large-scale histopathological images of the brain tumour dataset where the normal testing time per picture for the proposed strategy was 0.08 seconds.
Rango: An Intuitive Rule Language for Learning Classifier Systems in Cyber-Physical Systems

Self-adaptation is crucial for cyber-physical systems (CPS) to meet their requirements in environments characterized by complexity and uncertainty. As many situations that CPS encounter at runtime are not foreseeable at design time, (online) learning approaches are attractive for such systems. Learning classifier systems (LCS) are a promising learning approach for CPS thanks to their rather low computational complexity. They operate on a set of rules that describe potential adaptation behavior. So far, specifying rules for a learning classifier system is a tedious task that requires expert knowledge. In this paper, we present Rango — an intuitive rule language for learning classifier systems — to overcome this challenge. Compared to existing approaches, Rango has a strong focus on CPS and provides a large variety of corresponding keywords. In addition, Rango rules are automatically transferred into a representation that is usable in a learning classifier system without any modifications. Rango therefore empowers system administrators to formulate rules and, hence, leverage an online learning approach for their use case without having prior experience with learning classifier systems. We evaluate Rango extensively with (i) a complexity analysis of parsing and rule evaluation, (ii) a usefulness study which shows that Rango facilitates both the writing of rules and the understanding of LCS output and (iii) a usability study, which proves that basic programming knowledge is sufficient to understand and formulate Rango rules.
Comparative Analysis of Sensor-Based Human Activity Recognition Using Artificial Intelligence

Human Activity Recognition (HAR) has become one of the most prominent research topics in the field of ubiquitous computing and pattern recognition over the last decade. In this paper, a comparative analysis of 17 different algorithms is done using a 4-core 940mx machine and a 16-core G4dn.4xlarge Elastic Compute (EC2) instance on a public domain HAR dataset using accelerometer and gyroscope data from the inertial sensors in smartphones. The results are evaluated using the metrics accuracy, F1-score, precision, recall, training time, and testing time. The Machine Learning (ML) models implemented include Logistic Regression (LR), Support Vector Classifier (SVC), Random Forest (RF), Decision Trees (DT), Gradient Boosted Decision Trees (GBDT), linear and Radial Basis Function (RBF) kernel Support Vector Machines (SVM), K- Nearest Neighbors (KNN) and Naive Bayes (NB). The Deep Learning (DL) models implemented include Convolutional Neural Networks (CNN), Long Short Term Memory (LSTM), a combination of CNN-LSTM and Bidirectional LSTM. Neural Structure Learning was also implemented over a CNN-LSTM model along with Deep Belief Networks (DBN). It is identified that the Deep Learning models CNN, LSTM, CNN-LSTM & CNN-BLSTM consistently confuse between dynamic activities and that the machine learning models confuse between static activities. A Divide and Conquer approach was implemented on the dataset and CNN achieved an accuracy of 99.92% on the dynamic activities, whereas the CNN-LSTM model achieved an accuracy of 96.73% eliminating confusion between the static and dynamic activities. Maximum classification accuracy of 99.02% was achieved by DBN on the full dataset after Gaussian standardization. The proposed DBN model is much more efficient, lightweight, accurate, and faster in its classification than the existing models.
A study on the distribution of social biases in self-supervised learning visual models

Deep neural networks are efficient at learning the data distribution if it is sufficiently sampled.However, they can be strongly biased by non-relevant factors implicitly incorporated in the training data.These include operational biases, such as ineffective or uneven data sampling, but also ethical concerns, as the social biases are implicitly present-even inadvertently, in the training data or explicitly defined in unfair training schedules.In tasks having impact on human processes, the learning of social biases may produce discriminatory, unethical and untrustworthy consequences.It is often assumed that social biases stem from supervised learning on labelled data, and thus, Self-Supervised Learning (SSL) wrongly appears as an efficient and bias-free solution, as it does not require labelled data.However, it was recently proven that a popular SSL method also incorporates biases.In this paper, we study the biases of a varied set of SSL visual models, trained using ImageNet data, using a method and dataset designed by psychological experts to measure social biases.We show that there is a correlation between the type of the SSL model and the number of biases that it incorporates.Furthermore, the results also suggest that this number does not strictly depend on the model's accuracy and changes throughout the network.Finally, we conclude that a careful SSL model selection process can reduce the number of social biases in the deployed model, whilst keeping high performance.The code is available at https://github.com/vpulab/SB-SSL.
A Deep Learning Ensemble Hate Speech Detection Approach for Sinhala Tweets

We live in an era where social media platforms play a key role in society. These platforms support most of the native languages and this has enabled people to express their opinions conveniently. Also, it is very common to observe that people express very hateful opinions on social media platforms as well. Several studies have been carried out in this area for the Sinhala language with traditional machine learning models and none of them have shown promising results. Further, current approaches are far behind the latest techniques carried out in high-resource languages. Hence this study presents a deep learning-based approach for hate speech detection which has shown outstanding results for other languages. Moreover, a deep learning ensemble was constructed from these models to evaluate performance improvements. These models were trained and tested on a newly created dataset using the Twitter API. Moreover, the model generalizability was further tested by applying it to a completely new dataset. As per the results, it can be observed that the proposing approach has outperformed the traditional machine learning models and is well generalized. Finally, the experimentation with extra features also reveals that there is a positive impact on the performance using extra features.
HubHSP Graph: Effective Data Sampling for Pivot-Based Representation Strategies

Given a finite dataset in a metric space, we investigate the definition of a representative sample. Such a definition is important in data analysis strategies to seed algorithms (such as $$k$$ -means) and for pivot-based data indexing techniques. We discuss the geometrical and statistical facets of such a definition. We propose the Hubness Half Space Partitioning (HubHSP) strategy as an effective sampling heuristic that combines both geometric and statistical constraints. We show that the HubHSP sampling strategy is sound and stable in non-uniform high-dimensional regimes and compares favorably with classical sampling techniques.
Frequency Response Function Identification from Incomplete Data: A Wavelet-based Approach

Frequency Response Function (FRF) identification plays a crucial role in the design, the control, and the analysis of complex dynamical systems, including thermal and motion systems. Especially for applications that require long measurements, missing data samples, e.g., due to interruptions in the data transmission or sensor failure, often occur. The aim of this paper is to accurately identify nonparametric FRF models of periodically excited systems from noisy output measurements with missing samples. The presented method employs a wavelet-based transformation to address the identification problem in the time-frequency plane. A simulation example confirms that the developed techniques produce accurate estimates, even when many samples are missing.
CFL: Cluster Federated Learning in Large-Scale Peer-to-Peer Networks

High bandwidth, data privacy issues, and single point of failure require the development of Federated learning (FL) in large-scale peer-to-peer (P2P) networks. In this paper, we propose the first fine-grained global model training protocol, dubbed CFL, which is efficient and privacy-preserving. Rigorous analyses show that CFL guarantees the privacy and data integrity and authenticity of local model update parameters under two widespread threat models. Ingenious experiments on the Trec06p and Trec07 datasets show that the global model trained by CFL has good classification accuracy, rapid convergence rate, and dropout-robustness. Compared to the first global model training protocol for FL in P2P networks, CFL improves communication efficiency and computational efficiency.
A Zero-Knowledge-Range-Proof-based Privacy-Preserving Blockchain Platform for COVID-19 Contact Tracing

Contact tracing is an effective strategy to slow down the COVID-19 pandemic. However, the use of digital footprints as supportive evidences in the contact tracing process rises the privacy problems since private information must be shared to the contact tracing providers. This paper proposed a novel privacy-preserving contact tracing procedure based on zero-knowledge-range-proof and blockchain platform, which helps users to prove whether they in contact with the confirmed patient without disclosing the exact location they have visited. The blockchain is used to guarantee anonymity through the use of address as an identity, and provide strong non-repudiation from transactions. Finally, we provide a proof-of-concept implementation of our proposal using Hyperledger Fabric and smartphone application. The evaluation showed that the proposed system can work as intended with minimal processing delay.
A Nonparametric Cumulative Sum-Based Fault Detection Method for Rolling Bearings Using High-Level Extended Isolated Forest

As an advanced anomaly detection method, extended isolated forest (EIF) has great potential for use in bearings. A low-dimensional data transformation method based on EIF with high extension level and an early fault detection strategy for rolling bearings based on feature combination are proposed due to the limitation of EIF extension level on low-dimensional data. First, a preliminary sequence base is generated through sliding windows, and the sequences related to the number of transformed dimensions are selected based on the dynamic time warped (DTW) distance of the original sequence to perform weighted DTW barycenter averaging (WDBA), which enables preprocessing of the maximum extension level of EIF. Then, to realize early fault detection of rolling bearings, the nonparametric cumulative sum (NCUSUM) algorithm is used to design a joint threshold discrimination scheme for root mean square (rms) and kurtosis of the new and original sequences. We tested the algorithm on fault simulation using FEMTO-ST and XJTU-SY bearing datasets. The results show that WDBA-EIF algorithm can better remove uncorrelated noise from time series at high extension level. Compared with several anomaly detection methods, under 60% datasets, it has the best detection result of an early weak anomaly in the running process of rolling bearings and has a low false alarm rate.
Performance Objective Extraction of Optimal Controllers: A Hippocampal Learning Approach

Intention inference of autonomous vehicles is crucial to guarantee safety and to mitigate risk.This paper reports a performance objective extraction from expert's data trajectories for experience transference and to uncover the hidden cost associated to the intent.The algorithm is inspired in the hippocampus learning system for experience exploitation that exhibits the human brain.The hippocampus is responsible of memory and to store past experiences to enable transfer learning and fast convergence.The proposed algorithm extracts, from expert's data, the performance matrices associated to a hidden utility function using a complementary approach based on an off-policy policy iteration and a matrix extraction inverse reinforcement learning algorithms.Exact performance extraction is obtained by adding a constraint in terms of the measurements of the utility function in a batch-least squares algorithm.Convergence of the proposed approach is verified using Lyapunov recursions.Simulation studies are carried out to demonstrate the effectiveness of the proposed approach.
Sim VQA: Exploring Simulated Environments for Visual Question Answering

Existing work on VQA explores data augmentation to achieve better generalization by perturbing images in the dataset or modifying existing questions and answers. While these methods exhibit good performance, the diversity of the questions and answers are constrained by the available images. In this work we explore using synthetic computer-generated data to fully control the visual and language space, allowing us to provide more diverse scenarios. We quantify the effectiveness of leveraging synthetic data for real-world VQA. By exploiting 3D and physics simulation platforms, we provide a pipeline to generate synthetic data to expand and replace type-specific questions and answers without risking exposure of sensitive or personal data that might be present in real images. We offer a comprehensive analysis while expanding existing hyper-realistic datasets to be usedfor VQA. We also propose Feature Swapping (F-SWAP) - where we randomly switch object-level features during training to make a VQA model more domain invariant. We show that F-SWAP is effective for improving VQA models on real images without compromising on their accuracy to answer existing questions in the dataset.
Simplex-Based Proximal Multicategory Support Vector Machine

The multicategory support vector machine (MSVM) has been widely used for multicategory classification. Despite its widespread popularity, regular MSVM cannot provide direct probabilistic results and suffers from excessive computational cost, as it is formulated on the hinge loss function and it solves a sum-to-zero constrained quadratic programming problem. In this study, we propose a general refinement of regular MSVM, termed as the simplex-based proximal MSVM (SPMSVM). Our SPMSVM uses a novel family of squared error loss functions in place of the hinge loss and it removes the explicit sum-to-zero constraint by the simplex structure. Consequently, the SPMSVM only requires solving an unconstrained linear system, leading to closed-form solutions. In addition, the SPMSVM can be cast into a weighted regression problem so that it is scalable for large-scale applications. Moreover, the SPMSVM naturally yields an estimate of the conditional category probability, which is more informative than regular MSVM. Theoretically, the SPMSVM is shown to include many existing MSVMs as its special cases, and its asymptotic and finite-sample statistical properties are well established. Simulations and real examples show that the proposed SPMSVM is a stable, scalable and competitive classifier.