Reviewerly/exHarmony huggingface dataset:
Prompt-Based Learning for Unpaired Image Captioning

Unpaired Image Captioning (UIC) has been developed to learn image descriptions from unaligned vision-language sample pairs. Existing works usually tackle this task using adversarial learning and visual concept reward based on reinforcement learning. However, these existing works were only able to learn limited cross-domain information in vision and language domains, which restrains the captioning performance of UIC. Inspired by the success of Vision-Language Pre-Trained Models (VL-PTMs) in this research, we attempt to infer the cross-domain cue information about a given image from the large VL-PTMs for the UIC task. This research is also motivated by recent successes of prompt learning in many downstream multi-modal tasks, including image-text retrieval and vision question answering. In this work, a semantic prompt is introduced and aggregated with visual features for more accurate caption prediction under the adversarial learning framework. In addition, a metric prompt is designed to select high-quality pseudo image-caption samples obtained from the basic captioning model and refine the model in an iterative manner. Extensive experiments on the COCO and Flickr30 K datasets validate the promising captioning ability of the proposed model. We expect that the proposed prompt-based UIC model will stimulate a new line of research for the VL-PTMs based captioning.
Performance evaluation of a novel Conjugate Gradient Method for training feed forward neural network

In this paper, we construct a new conjugate gradient method for solving unconstrained optimization problems. The proposed method satisfies the sufficient decent property irrespective of the line search and the global convergence was established under some suitable. Further, the new method was used to train different sets of data via a feed forward neural network. Results obtained show that the proposed algorithm significantly reduces the computational time by speeding up the directional minimization with a faster convergence rate.
Toward Multi-label Attribute Estimation on Multiple Faces Using CNN

In the past few years, there has been a dramatic increase in the number of intelligent or cognitive applications. These applications can understand, learn, and interact with people in a more natural way than traditional applications. Many of these new applications are based on artificial intelligence (AI) technology, namely the ability of computers to learn and work on their own. One of the earliest examples of an intelligent application is the facial recognition system, based on deep learning models; this system can detect, estimate, and classify human faces based on facial attributes. Our approach is focusing on the estimation of attributes in images or videos. We train the first component, a YOLOv5 network, on Wider Face datasets, to detect faces, crop them, then pass the resulted data to the second component where we train a CNN multi-label network on CelebA datasets. The obtained model is capable of estimating two levels of face images, colored ones, and sketches. In addition, we make a comparison between YOLOv5 and SSD-ResNet (Single-Shot MultiBox Detector) face detectors.
Movie Sentiment Review Analysis

The study aims to investigate the relationship between mental health issues and academic performance among students using data analytics and machine learning. The study was conducted among students aged 18-24 in various courses, with CGPA ranging up to 5. The data was collected through a survey questionnaire that included questions related to mental health issues such as depression, anxiety, and panic attacks, along with demographic and academic performance-related questions. The collected data was cleaned, explored, and modeled using various machine learning algorithms such as logistic regression, decision tree, random forest, and support vector classifier. The random forest model was found to be the most accurate among all the models. The study found that students with mental health issues have lower academic performance than their peers, and female students are more prone to mental health issues than male students. Keywords: Data Analytics, Students, mental health, modelling, classifier, depression, anxiety
Popularity Prediction Model With Context, Time and User Sentiment Information: An Optimization Assisted Deep Learning Technique

In social media, the data-sharing activities have turned out to be more pervasive; individuals and companies have comprehended the significance of promoting info by social media network. However, these individuals and companies face more challenges with the issue of “how to obtain the full benefit that the platforms provide”. Therefore, social media policies to improve the online promotion are turning out to be more significant. The popularization of social media contents are related to public attention and interest of users, thus the popularity fore cast of online contents has considered being the major task in social media analytic and it facilitates several appliances in diverse domain as well. This paper intends to introduce a popularity forecast approach that derives and combines the richest data of “text content encoder, user encoder, time series encoder, and user sentiment analysis”. The extracted features are then predicted via Long Short Term Memory (LSTM). Particularly, to enhance the prediction accuracy of the LSTM, the weights are fine-tuned via Self Adaptive Rain optimization (SA-RO).
Depth Estimation Based on Monocular Camera Sensors in Autonomous Vehicles: A Self-supervised Learning Approach

Abstract Estimating depth from images captured by camera sensors is crucial for the advancement of autonomous driving technologies and has gained significant attention in recent years. However, most previous methods rely on stacked pooling or stride convolution to extract high-level features, which can limit network performance and lead to information redundancy. This paper proposes an improved bidirectional feature pyramid module (BiFPN) and a channel attention module (Seblock: squeeze and excitation) to address these issues in existing methods based on monocular camera sensor. The Seblock redistributes channel feature weights to enhance useful information, while the improved BiFPN facilitates efficient fusion of multi-scale features. The proposed method is in an end-to-end solution without any additional post-processing, resulting in efficient depth estimation. Experiment results show that the proposed method is competitive with state-of-the-art algorithms and preserves fine-grained texture of scene depth.
End-to-End Security for Distributed Event-driven Enclave Applications on Heterogeneous TEEs

This article presents an approach to provide strong assurance of the secure execution of distributed event-driven applications on shared infrastructures, while relying on a small Trusted Computing Base. We build upon and extend security primitives provided by Trusted Execution Environments (TEEs) to guarantee authenticity and integrity properties of applications, and to secure control of input and output devices. More specifically, we guarantee that if an output is produced by the application, it was allowed to be produced by the application’s source code based on an authentic trace of inputs. We present an integrated open-source framework to develop, deploy, and use such applications across heterogeneous TEEs. Beyond authenticity and integrity, our framework optionally provides confidentiality and a notion of availability, and facilitates software development at a high level of abstraction over the platform-specific TEE layer. We support event-driven programming to develop distributed enclave applications in Rust and C for heterogeneous TEE, including Intel SGX, ARM TrustZone, and Sancus. In this article we discuss the workings of our approach, the extensions we made to the Sancus processor, and the integration of our development model with commercial TEEs. Our evaluation of security and performance aspects show that TEEs, together with our programming model, form a basis for powerful security architectures for dependable systems in domains such as Industrial Control Systems and the Internet of Things, illustrating our framework’s unique suitability for a broad range of use cases which combine cloud processing, mobile and edge devices, and lightweight sensing and actuation.
Time Series Symbolization Method for the Data Mining K-Means Algorithm

Time series is a data type frequently encountered in data analysis. With the current depth and breadth of the data and the improvement in computer processing capabilities, the dimensionality and the complexity of time series are getting higher and higher. Time series symbolization is to cluster and assign complex and lengthy time series in the form of symbols to achieve the purpose of reducing the dimensionality of the sequence or making the sequence easier to process. Considering the excellent performance of the K-means algorithm in data mining and processing, as well as in the allocation algorithm for clustering, we plan to develop a simple method for the symbolization of time series for the K-means algorithm and hope that this method can realize the high-dimensional time series dimensionality reduction, processing of the special points in time series, and so on. Based on this, this article proposes an improved sans algorithm based on the K-means algorithm and discusses the representation method and the data processing of time series symbolization. Experimental results show that this method can effectively reduce the dimensionality of high-dimensional time series. After dimensionality reduction, the information retention rate contained in the elevation of the sequence can reach more than 90%, which is very effective for the detection of outliers in low-dimensional sequences.
Alone and Together: Resilience in a Fluid Socio-Technical-Natural System

Disruption to routines is an increasingly common part of everyday life. With the roots of some disruptions in the interconnectedness of the world and environmental and socio-political instability, there is good reason to believe that conditions that cause widespread disruption will persist. Individuals, communities, and systems are thus challenged to engage in resilience practices to deal with both acute and chronic disruption. Our interest is in chronic, everyday resilience, and the role of both technology and non-technical adaptation practices engaged by individuals and communities, with a specific focus on practices centered in nature. Foregrounding nature's role allows close examination of environmental adversity and nature as part of adaptivity. We add to the CSCW and HCI literature on resilience by examining long-distance hikers, for whom both the sources of adversity and the mitigating resilience processes cut across the social, the technical, and the environmental. In interviews with 12 long-distance hikers we find resilience practices that draw upon technology, writ large, and nature in novel assemblages, and leverage fluid configurations of the individual and the community. We place our findings in the context of a definition for resilience that emphasizes a systems view at multiple scales of social organization. We make three primary contributions: (1) we contribute an empirical account of resilience in a contextual setting that complements prior CSCW resilience studies, (2) we add nuance to existing models for resilience to reflect the role of technology as both a resilience tool and a source of adversity, and (3) we identify the need for new designs that integrate nature into systems as a way to foster collaborative resilience. This nuanced understanding of the role of technology in individual and community resilience in and with nature provides direction for technology design that may be useful for everyday disrupted life.
Systemic Gender Inequities in Who Reviews Code

Code review is an essential task for modern software engineers, where the author of a code change assigns other engineers the task of providing feedback on the author's code. In this paper, we investigate the task of code review through the lens of equity, the proposition that engineers should share reviewing responsibilities fairly. Through this lens, we quantitatively examine gender inequities in code review load at Google. We found that, on average, women perform about 25% fewer reviews than men, an inequity with multiple systemic antecedents, including authors' tendency to choose men as reviewers, a recommender system's amplification of human biases, and gender differences in how reviewer credentials are assigned and earned. Although substantial work remains to close the review load gap, we show how one small change has begun to do so.
Optimized numerical gradient and Hessian estimation for variational quantum algorithms

Sampling noisy intermediate-scale quantum devices is a fundamental step that converts coherent quantum-circuit outputs to measurement data for running variational quantum algorithms that utilize gradient and Hessian methods in cost-function optimization tasks. This step, however, introduces estimation errors in the resulting gradient or Hessian computations. To minimize these errors, we discuss tunable numerical estimators, which are the finite difference (including their generalized versions) and scaled parameter-shift estimators [introduced in Phys. Rev. A 103, 012405 (2021)], and propose operational circuit-averaged methods to optimize them. We show that these optimized numerical estimators offer estimation errors that drop exponentially with the number of circuit qubits for a given sampling-copy number, revealing a direct compatibility with the barren-plateau phenomenon. In particular, there exists a critical sampling-copy number below which an optimized difference estimator gives a smaller average estimation error in contrast to the standard (analytical) parameter-shift estimator, which exactly computes gradient and Hessian components. Moreover, this critical number grows exponentially with the circuit-qubit number. Finally, by forsaking analyticity, we demonstrate that the scaled parameter-shift estimators beat the standard unscaled ones in estimation accuracy under any situation, with comparable performances to those of the difference estimators within significant copy-number ranges, and are the best ones if larger copy numbers are affordable.
<i>XploreNAS</i> : Explore Adversarially Robust and Hardware-efficient Neural Architectures for Non-ideal Xbars

Compute In-Memory platforms such as memristive crossbars are gaining focus as they facilitate acceleration of Deep Neural Networks (DNNs) with high area and compute-efficiencies. However, the intrinsic non-idealities associated with the analog nature of computing in crossbars limits the performance of the deployed DNNs. Furthermore, DNNs are shown to be vulnerable to adversarial attacks leading to severe security threats in their large-scale deployment. Thus, finding adversarially robust DNN architectures for non-ideal crossbars is critical to the safe and secure deployment of DNNs on the edge. This work proposes a two-phase algorithm-hardware co-optimization approach called XploreNAS that searches for hardware-efficient & adversarially robust neural architectures for non-ideal crossbar platforms. We use the one-shot Neural Architecture Search (NAS) approach to train a large Supernet with crossbar-awareness and sample adversarially robust Subnets therefrom, maintaining competitive hardware-efficiency. Our experiments on crossbars with benchmark datasets (SVHN, CIFAR10 & CIFAR100) show upto ~8-16% improvement in the adversarial robustness of the searched Subnets against a baseline ResNet-18 model subjected to crossbar-aware adversarial training. We benchmark our robust Subnets for Energy-Delay-Area-Products (EDAPs) using the Neurosim tool and find that with additional hardware-efficiency driven optimizations, the Subnets attain ~1.5-1.6x lower EDAPs than ResNet-18 baseline.
Quantifying <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>T</mml:mi></mml:math> -gate-count improvements for ground-state-energy estimation with near-optimal state preparation

Quantum computers will enable high-accuracy quantum simulations of physical systems, provided that states with reasonable overlap with the ground state can be efficiently prepared. In this Letter, the authors articulate conditions in which quadratic runtime speedups are plausible for ground-state energy estimation by translating a near-optimal ground-state preparation scheme into explicit circuits and estimating the cost in a model relevant to their fault-tolerant implementation.
Moment-based space-variant Shack–Hartmann wavefront reconstruction

This article presents an approach for space-variant Shack–Hartmann wavefront reconstruction based on image moment theory. We derive the relation between the moment of a pair of subimages and the local transformation coefficients. A special solution is obtained from this relation using the square guide ‘star’. Our moment-based wavefront reconstruction has reduced computational complexity compared to the iteration-based algorithm. We execute image restorations using both the tiling strategy with 5 × 5 PSFs and the conventional strategy with a global average PSF. Our approach is supported by visual and quantitative evaluations.
Causal Disentangled Recommendation against User Preference Shifts

Recommender systems easily face the issue of user preference shifts. User representations will become out-of-date and lead to inappropriate recommendations if user preference has shifted over time. To solve the issue, existing work focuses on learning robust representations or predicting the shifting pattern. There lacks a comprehensive view to discover the underlying reasons for user preference shifts. To understand the preference shift, we abstract a causal graph to describe the generation procedure of user interaction sequences. Assuming user preference is stable within a short period, we abstract the interaction sequence as a set of chronological environments. From the causal graph, we find that the changes of some unobserved factors (e.g., becoming pregnant) cause preference shifts between environments. Besides, the fine-grained user preference over item categories sparsely affects the interactions with different items. Inspired by the causal graph, our key considerations to handle preference shifts lie in modeling the interaction generation procedure by: (1) capturing the preference shifts across environments for accurate preference prediction and (2) disentangling the sparse influence from user preference to interactions for accurate effect estimation of preference. To this end, we propose a Causal Disentangled Recommendation (CDR) framework, which captures preference shifts via a temporal variational autoencoder and learns the sparse influence from multiple environments. Specifically, an encoder is adopted to infer the unobserved factors from user interactions while a decoder is to model the interaction generation process. Besides, we introduce two learnable matrices to disentangle the sparse influence from user preference to interactions. Last, we devise a multi-objective loss to optimize CDR. Extensive experiments on three datasets show the superiority of CDR in enhancing the generalization ability under user preference shifts.
Cross-lingual Sentence Embedding for Low-resource Chinese-Vietnamese Based on Contrastive Learning

Cross-lingual sentence embedding’s goal is mapping sentences with similar semantics but in different languages close together and dissimilar sentences farther apart in the representation space. It is the basis of many downstream tasks such as cross-lingual document matching and cross-lingual summary extraction. At present, the works of cross-lingual sentence embedding tasks mainly focus on languages with large-scale corpus. But low-resource languages such as Chinese-Vietnamese are short of sentence-level parallel corpora and clear cross-lingual monitoring signals, and these works on low-resource languages have poor performances. Therefore, we propose a cross-lingual sentence embedding method based on contrastive learning and effectively fine-tune powerful pretraining mode by constructing sentence-level positive and negative samples to avoid the catastrophic forgetting problem of the traditional fine-tuning pre-trained model based only on small-scale aligned positive samples. First, we construct positive and negative examples by taking parallel Chinese Vietnamese sentences as positive examples and non-parallel sentences as negative examples. Second, we construct a siamese network to get contrastive loss by inputting positive and negative samples and fine-tuning our model. The experimental results show that our method can effectively improve the semantic alignment accuracy of cross-lingual sentence embedding in Chinese and Vietnamese contexts.
Wave-Like Class Activation Map With Representation Fusion for Weakly-Supervised Semantic Segmentation

The Class Activation Map (CAM) is widely used to generate pseudo-labels for Weakly Supervised Semantic Segmentation (WSSS), while it does not adequately consider the modeling of foreground-independent information, resulting in prone to false positive pixels. In this paper, we propose a Wave-like Class Activation Map (WaveCAM) from the perspective of representation fusion and dynamic aggregation representation to alleviate the above problem. Specifically, our WaveCAM includes the foreground-aware representation modeling that enhances perception of foreground information, and the foreground-independent representation modeling that enhances perception of foreground-independent information, and a representation-adaptive fusion module that fuses the two representations. Both representations are expressed as wave functions with amplitude and phase to dynamically aggregate representations and extract semantic information after initialization, and they are fused through the adaptive fusion module to obtain an output containing rich semantic information. Extensive experiments on PASCAL VOC 2012 dataset and MS COCO 2014 dataset validate that our WaveCAM can easily embed multi-stage WSSS and end-to-end WSSS, achieving the state-of-the-art performance. The release code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Rongtao-Xu/RepresentationLearning/tree/main/WaveCAM-TMM2023</uri> .
Scheduling the Task of User in Cloud computing using Hybrid Procedure of PSO and Lion Algorithm

Commonly used for tasks that may be performed on a network of remote servers or “processing units,” cloud computing is a popular platform for executing operations on virtual computers. In both the consumer and the provider realms, task scheduling is an important tool. Finding the best resource to complete a given work is a major challenge for any task scheduler. Thus, the researchers saw job scheduling as one of cloud computing's most basic challenges. Several methods have been proposed by academics to handle the optimization issue that is the optimum determination of scheduling criteria in cloud computing. This work presents a novel hybrid approach that combines the concepts of the Lion Algorithm with the Particle Swarm Optimization-(PSO) algorithm to address the aforementioned optimization problem. The research used both traditional parallel workloads and artificial workloads to perform its tests using the CloudSim toolset. The findings are compared to those produced by other similarly-structured, metaheuristic-based methods tested under the identical circumstances. The promising findings of our simulations suggest that our suggested method has real promise in this area. To reduce the time it takes to submit a particular process in a virtual environment, this study proposes improved versions of the PSO and LA algorithm, subject to user-required budgetary limitations. The results also indicate that, when applied to workflow issues operating on different virtual machines, improved versions of the PSO and LA algorithm outperform the original technique in terms of shorter schedule duration.