Reviewerly/exHarmony huggingface dataset:
Ancient Indian Murals Digital Restoration through Image InPainting

Murals and paintings are an integral part of the history of any civilization. India has deep roots in ancient history and possesses some of its most valuable assets in ancient murals dating back several hundred years. They feature Hindu mythology, traditions, incidents of bravery and cultural events and thus have a substantial social significance in human society. Unfortunately, some of these assets lose value over time due to environmental factors and human negligence. The ground truth regarding their original colour, textures and painting style references is not currently available, due to which their physical restoration is not possible. Therefore, the only way to restore them is through their digital counterparts and image inpainting techniques. In this paper, a complete end-to-end image restoration framework has been presented to virtually repair the damaged regions of the mural paintings using a selective neural style technique for the transfer of realistic-looking damaged textures, a modified U-Net for detecting the damaged regions in an image and a Partial Convolution network for effective image inpainting. The photographs of Rajasthani paintings scrapped from online sources are used as the dataset. The restored results based on the perceptual quality of completion on ancient Indian murals are promising; thus, they may contribute to the digital restoration of murals.
Designing an Efficient Object Detection Model for Autonomous Driving Applications

The detection of road objects plays an essential role in the development of autonomous vehicles in intelligent transportation systems, which has become an emerging field in deep learning. Several objects on the road, like vehicles, pedestrians, etc., are necessary to be accurately identified, which guarantees the safety of other people and vehicles in the surroundings. In this article, we aim to design an efficient object detection model for autonomous driving systems. To achieve this, we investigate the recently developed YOLOv7 and optimize the same for improving the detection performance to satisfy the realtime safety requirements of autonomous vehicles. We perform extensive experimentation and demonstrate the effectiveness of our method on the BDDIOOK dataset. Experimental results show that our proposed method increases the detection accuracy to 82.6% and inference speed to 97.2 FPS compared to the baselines, with no additional increase in model complexity.
Futures of Money – Monies of the Future. Introduction

Abstract This introduction reviews the main themes addressed in the special issue Futures of Money – Monies of the Future. Recalling broader historical transformations of money, this introduction focuses on how the digitalization of money interacts with the central role of states and the financial industry in the issuance and distribution of money. This concerns how money interacts with non-monetary digital information and how digital forms of social interaction transform the rules of exchange where money is used. Exploring these issues, the articles brought together here open fruitful venues to rethink the way we theorize money.
SentiMatch: Sentiment Analysis of Multi-modal Social Media Posts

Social media users are increasingly using images and videos to communicate their feelings. However, researchers usually rely on textual data for sentiment analysis. In this paper, we focus on Twitter posts having both text and image data and analyze the match between their sentiments, when predicted independently. We also propose a method to predict the sentiment of a social media post as a collective entity from the individual sentiments of text and image. Comparing our proposed model to a naive approach where the sentiment of the text is assigned to the image also, we see an improvement of around 20% in prediction of sentiment of Twitter images associated with text.
XRBi-GAC: A hybrid deep learning framework for multilingual toxicity detection

Social media platforms allow people across the globe to share their thoughts and opinions and conveniently communicate with each other. Apart from various advantages of social media, it is also misused by a set of users for hate-mongering with toxic and offensive comments. The majority of the earlier proposed toxicity detection methods are primarily focused on the English language, but there is a lack of research on low-resource languages and multilingual text data. We propose an XRBi-GAC framework comprising XLM-RoBERTa, Bi-GRU with self-attention and capsule networks for multilingual toxic text detection. A loss function is also presented, which fuses the binary cross-entropy loss and focal loss to address the class imbalance problem. We evaluated the proposed framework on two datasets, namely, the Jigsaw Multilingual Toxic Comment dataset and HASOC 2019 dataset and achieved F1-score of 0.865 and 0.829, respectively. The results of the experiments show that the proposed framework has outperformed the state-of-the-art multilingual models XLM-RoBERTa and mBERT on both datasets, which shows the versatility and robustness of the proposed XRBi-GAC framework.
Postprediction Inference for Clinical Characteristics Extracted With Machine Learning on Electronic Health Records

Real-world data (RWD) derived from electronic health records (EHRs) are often used to understand population-level relationships between patient characteristics and cancer outcomes. Machine learning (ML) methods enable researchers to extract characteristics from unstructured clinical notes, and represent a more cost-effective and scalable approach than manual expert abstraction. These extracted data are then used in epidemiologic or statistical models as if they were abstracted observations. Analytical results derived from extracted data in this way may differ from those given by abstracted data, and the magnitude of this difference is not directly informed by standard ML performance metrics.In this paper, we define the task of postprediction inference, which is to recover similar estimation and inference from an ML-extracted variable that would be obtained from abstracting the variable. We consider fitting a Cox proportional hazards model that uses a binary ML-extracted variable as a covariate and evaluate four approaches for postprediction inference in this setting. The first two approaches only require the ML-predicted probability, while the latter two additionally require a labeled (human abstracted) validation data set.Our results for both simulated data and EHR-derived RWD from a national cohort demonstrate that we can improve inference from ML-extracted variables by leveraging a limited amount of labeled data.We describe and evaluate methods for fitting statistical models using ML-extracted variables subject to model error. We show that estimation and inference is generally valid when using extracted data from high-performing ML models. More complex methods that incorporate auxiliary labeled data provide further improvements.
Swarm Intelligence Algorithms for Profit Maximization and Delay Minimization in Internet Data Centers

Internet Data Centers have increasingly gained importance in recent days. organizations require their data centers to run with minimum expenditure to improve profits, and the service requests need to be fulfilled with the minimum service delay possible. Increasing profits and minimizing service delay are challenges faced by data centers on a day-to-day basis. The demand response in a data center is a crucial challenge to be optimized because of its direct correlation with profits and delay in an internet data center. In this paper, meta-heuristic swarm algorithms are analyzed to find the best possible profits and minimum possible delays that can arise in a data center for a particular number of active servers and the amount of workload processed by the data center. The methods are verified with real-world data for pricing schemes of electricity, power consumption, request loads and data center specifications.
Design and Evaluation of a Vehicle Detection System in Low Light Conditions

The advent of autonomous vehicles has motivated researchers to focus their research on developing improved advanced driver assistance systems (ADAS). A key area of study in ADAS is vehicle detection. Many have proposed vehicle detection systems in daylight conditions but very few have proposed vehicle detection systems in night time. We have proposed a vehicle detection system in low light conditions in this paper. Cycle Generative Adversarial Networks (CycleGAN) is used to translate images from night to day. This makes use of unpaired imageto-image translation. Super Resolution Convolutional Neural Networks (SRCNN) is used to improve the resolution of the image. Finally, several object detector models like YOLOv4, Faster-RCNN, SSD, ResNet are used to detect vehicles. The results are compared. We have achieved a maximum mAP of 61.9% using the proposed model.
A Novel Attention Heat Map-Based Video Anomaly Detection Framework

In video surveillance systems for the purpose of public safety, automatic anomaly detection is a vital responsibility. The goal of anomalous activity recognition is to identify the frames with various spatial information. In the proposed methodology we presented a framework where we can classify and localize rare events by using trainable features only through the global average pooling layer rather than flattening the model, using the pre-trained model. This model is trained only on the global average pooling layer while the other layers are non-trainable. Extracted features are further connected to dense layers to make predictions as anomalies or normal images. We first determined all images classified as anomalies by the model to localize anomalies and save those images. Using a heat map with peak value and threshold we locate different anomalies in the image using weights of the last two layers of the trained model. Our method adds the advantage of both frame-level and pixel-level performance where we classify and locate the anomalies. We evaluated our network performance on two standard datasets that outperform the state of art methods by achieving an accuracy of 93.7% for UCSD ped1 and 95.4% for ped2 datasets respectively.
Quality Evaluation of Laparoscopic Videos Using the Interdependency of Luminance and Texture Maps

The perceived quality of the video during minimally invasive surgeries greatly impacts the success of these medical procedures. Consequently, there is high demand for varied methods aimed at quality assessment (QA) of laparoscopic videos. In this work, a video quality prediction model is proposed to explore the statistical dependency between the subband coefficients of luminance and texture components of motion blurred laparoscopic videos. For this, the publicly available motion blurred videos from the Laparoscopic Video Quality (LVQ) database is used. The luminance map and gray-level co-occurrence matrix (GLCM) texture map are computed for each video frame and decomposed using steerable pyramid decomposition method. The decomposed subband coefficients are modelled using Bivariate Generalised Gaussian Distribution (BGGD) and the distribution depicts sharp peaks and long tails. The BGGD $(\alpha,\beta)$ parameters are estimated for each subband level and they are average pooled to estimate the frame-level values. The videos of the LVQ database is randomly divided into 66:34 training-to-testing ratio to perform supervised non-linear regression using Adaptive Neuro Fuzzy Inference System (ANFIS) network. In training phase, frame-level BGGD parameters are used as inputs along with expert and non-expert Mean Opinion Score (MOS) values as training labels for each video. In testing phase, the quality score of each test video is predicted. The performance of the proposed model is compared to other 2D image QA (IQA) and video QA (VQA) algorithms. The Pearson’s Linear Correlation Coefficient (PLCC) and Root Mean Square Error (RMSE) performance metrics are used to evaluate the prediction efficiency of the model. It is observed that the proposed model performs prediction with a PLCC value of 0.9814 for expert training labels. When non-expert training labels are used a PLCC value of 0.9635 is obtained. The model also demonstrates an RMSE value of 0.3939 when expert MOS training labels are used, and an RMSE value of 0.4639 when non-expert MOS training labels are used. The higher correlation scores and lower RMSE values indicate that the model can be used as a VQA technique for analysis of laparoscopic videos affected by blur distortions due to motion.
Domain adaptive learning for document layout analysis and object detection using classifier alignment mechanism

Document structure analysis and object detection is the new research interest, which involves differentiation between distinctive semantic regions of textual and non-textual objects. Document layout analysis remains a major challenge as document elements pose diversified layout structures, shapes, and appearances. Lack of annotated training datasets and domain shifts between datasets further increases the intricacy. The paper proposes an adaptive detection model for cross-domain learning ‘XDOD’ for document structure recognition and object detection. The detection model overcomes the domain shift between documents using, (1) Document Object Attention (DA) module, which learns coarse-to-refined features, and (2) Classifier Alignment (CA) module, which reduces the object misclassification. The XDOD model has been evaluated on various publicly available document datasets belonging to different domains. Extensive experimental results portray that the proposed XDOD model performs significantly better than the existing benchmark model giving more than 97% of detection rate on all datasets.
Track vibration sequence anomaly detection algorithm based on LSTM

Subway structure monitoring obtains structure monitoring data in real time, and the obtained subway track vibration sequence exhibits obvious time series characteristics. Therefore, the difficulties of abnormal detection of subway track vibration sequence include not only the general scarcity and diversity of data, but also the large amount of sample data. In this paper, an anomaly detection method based on the long short-term memory (LSTM) was applied to detect anomalous subway track vibration sequences. Firstly, subway track vibration signals were preprocessed. An approach to extract subway track vibration sequences was proposed. According to this method, whether the collected data constituted running data was determined via the mean square error, and track vibration sequences were then extracted from the original data according to an adaptive threshold value to obtain subway track vibration samples. Afterwards, Savitzky-Golay filtering was performed to smooth the obtained subway track vibration sequences, and then the wavelet transform was applied to denoise the signal. Second, an anomaly detection algorithm based on the LSTM was employed to detect subway track vibration sequences. Finally, compared to other algorithms, the LSTM algorithm performed better in anomaly detection on the subway track vibration dataset with a small anomaly proportion than did the other three methods. However, in the case of a large proportion of anomalies in the signal, the detection effect of the proposed algorithm was close to BPNN and superior to the LOF and OCSVM. The results indicated that the LSTM-based sequence anomaly detection algorithm attained a satisfactory detection effect for subway track vibration sequences. The anomaly detection algorithm can be applied to subway structure monitoring systems, which can monitor subway track vibration signals in real time and determine whether these signals are anomalous to ensure the safe operation of subway structures.
A heterogeneous E-commerce user alignment model based on data enhancement and data representation

Identifying users’ accounts on different platforms is important for the management and marketing of e-commerce platforms. To address the problem of sparse consumer behavior data on e-commerce platforms, this paper designs Double Generative Adversarial Network (Double-GAN) to compensate the data. This method uses not only the data of the one platform , but also of heterogeneous e-commerce platforms for alternate iteration compensation. For the complexity of “user-behavior-commodity” data feature space of e-commerce bookstore platform. Considering Consider the advantages of model JUST in capturing structural and semantic information of multiple types of nodes. This paper proposes a new method UBC2vec to integrate user’s interest in commodities and change the random walk strategy to represent the overall information of data feature space of heterogeneous e-commerce platform. Finally, for the high computational complexity of the user alignment algorithm, this paper constructs a “user-commodity” bipartite graph to delineate the roles of users and reduce the complexity. Experiments show that the model can effectively alleviate the data sparsity problem and reduce the computational complexity of the algorithm. Above the real data set, it can effectively match users’ accounts in different bookstore platforms.
Complex objective optimization in fuzzy environments

Multi-objective optimization can be used to address possible conflicting relationships between multiple objectives. However, some objectives have a fuzzy temporal relationship between them, making it difficult to give a common method to portray the fuzzy temporal relationship. To fill this gap, we propose the concept of complex objectives, which can be described by fuzzy temporal logic that includes both temporal and logical operators. Furthermore, we investigated the optimal control problems of complex objectives and developed a fuzzy system called possibilistic decision systems (PDSs) to establish a framework for optimal control. In PDSs, states of fuzzy systems are determined by a family of variables, and transitions induced by actions between fuzzy states of systems are also fuzzy uncertain and determined by a possibility degree. Importantly, we proved that memoryless strategies are sufficient for optimal control of complex objectives. Finally, the theory presented in this paper is illustrated by a mobile robot simulation.
An Investigation of confusing code patterns in JavaScript

Evolving software is particularly challenging when the code has been poorly written or uses confusing idioms and language constructs, which might increase maintenance efforts and impose a significant cognitive load on developers. Previous research has investigated possible sources of confusion in programs, including the impact of small code patterns (hereafter atoms of confusion) that contribute to misunderstanding the source code. Although researchers have explored atoms of confusion in code written in C, C+ +, and Java, different languages have different features, developer communities, and development cultures. This justifies the exploration of other languages to verify whether they also exhibit confusion-inducing patterns. In this paper we investigate the impact of atoms of confusion on understanding JavaScript code—a dynamically typed language whose popularity is growing in the most diverse application domains. We present the results of a mixed-methods research comprising a mining software repositories (MSR) study, two experiments, and a set of interviews with practitioners. Our MSR effort shows that atom candidates are frequent and used intensively in 72 popular open-source JavaScript projects: four atom candidates appear in 90% of them and two of them occur more than once for every 100 lines of code. This helps motivate the other three studies. The results of both experiments suggest that two code patterns that have been previously observed to confuse C programmers also confuse JavaScript programmers: the comma operator and assignments being used as values. In addition, some code patterns, such as omitted curly braces and change of literal encoding, have caused confusion in participants in one of the experiments. We discover that some JavaScript-specific elements, such as automatic semicolon insertion and object destructuring, also have the potential to cause confusion. For all these cases effect sizes were either medium or high. The interviews we conducted indicate other constructs and idioms that merit investigation in the future.
SemDM: Task-oriented masking strategy for self-supervised visual learning

In this paper, we propose a novel learning scheme for better combining masked image modeling (MIM) and instance discrimination (ID). Motivated by compensating the requirement gap of masking strength between MIM and ID, we propose Semantic Disjoint Masking (SemDM), which decomposes the masking into two manners: preserving the majority of key patterns in images for ID, while dropping out most of them for MIM. Specifically, we utilize attention-guided masking in ID to help keeping the identity of object in image for encoder. While in MIM, we conversely only leave some hints about the object. Then these generated masked views only perform their specified learning task, facilitating more suitable visual priors to be learned in each learning task. Moreover, we introduce product quantization (PQ) to optimize the concept distributions in latent space, which guarantees that a compact set of meaningful visual concepts can be learned. Extensive experiments demonstrate that our method bootstraps meaningful visual concepts to guide visual understanding, and obtains state-of-the-art results on ImageNet-100.
Locally differentially private multi-dimensional data collection via haar transform

Local differential privacy (LDP) has emerged as a privacy standard for collecting distributed data. In multi-dimensional data collection, separately perturbing each dimension is one canonical solution to protect privacy. Yet, it commonly falls short in statistical utility due to excessive noise injection caused by the privacy budget split linearly related to data dimensions. To tackle this problem, we propose a multi-dimensional data collection scheme under LDP, called PPMC, achieving privacy-utility tradeoff through Haar transform-based dimension reduction. Specifically, we apply Haar transform to convert multi-dimensional data into two parts: the average value and eigenvector, so as to lay a foundation for reducing the dimension. We design a probability density-based perturbation mechanism for the average value, which can decrease the noise injection by optimizing the probability distribution. For the eigenvector, a dimension reduction model is presented that promises low utility loss by error-balanced strategy. Further, we develop a global perturbation mechanism for the reduced dimension eigenvector, which can better maintain statistical utility while ensuring privacy via a private sampling strategy. Finally, the noisy multi-dimensional data is generated by utilizing the inverse Haar transform in a locally differential private manner. Theoretical analysis and experiment results confirm the effectiveness of our solution.