Reviewerly/exHarmony huggingface dataset:
Contrastive encoder pre-training-based clustered federated learning for heterogeneous data

Federated learning (FL) is a promising approach that enables distributed clients to collaboratively train a global model while preserving their data privacy. However, FL often suffers from data heterogeneity problems, which can significantly affect its performance. To address this, clustered federated learning (CFL) has been proposed to construct personalized models for different client clusters. One effective client clustering strategy is to allow clients to choose their own local models from a model pool based on their performance. However, without pre-trained model parameters, such a strategy is prone to clustering failure, in which all clients choose the same model. Unfortunately, collecting a large amount of labeled data for pre-training can be costly and impractical in distributed environments. To overcome this challenge, we leverage self-supervised contrastive learning to exploit unlabeled data for the pre-training of FL systems. Together, self-supervised pre-training and client clustering can be crucial components for tackling the data heterogeneity issues of FL. Leveraging these two crucial strategies, we propose contrastive pre-training-based clustered federated learning (CP-CFL) to improve the model convergence and overall performance of FL systems. In this work, we demonstrate the effectiveness of CP-CFL through extensive experiments in heterogeneous FL settings, and present various interesting observations.
Methods for identifying emergent concepts in deep neural networks

The present perspective discusses methods to detect concepts in internal representations (hidden layers) of deep neural networks (DNNs), such as network dissection, feature visualization, and testing with concept activation vectors (TCAV). I argue that these methods provide evidence that DNNs are able to learn non-trivial relations between concepts. However, the methods also require users to specify or detect concepts via (sets of) instances. This underdetermines the meaning of concepts, making the methods unreliable. The problem could be overcome, to some extent, by systematically combining the methods and by using synthetic datasets. The perspective also discusses how conceptual spaces—sets of concepts in internal representations—are shaped by a trade-off between predictive accuracy and compression. I argue that conceptual spaces are useful, or even necessary, to understand how concepts are formed in DNNs but that there is a lack of method for studying conceptual spaces.
Multitask Learning Using Feature Extraction Network for Smart Tourism Applications

Recently around half of the world’s current population resides in urban areas and benefit from rich services in the smart city. The majority of smart city services are recommendation-related services, and with the development of Internet, most recommendation services in smart economy are online recommendations. Online travel platforms (OTPs) (like Booking, Airbnb, Ctrip, and Fliggy) provide people sufficient resources and convenient approaches to plan and enjoy their trips in smart city. Hotel recommendation is essential for the success of OTPs. However, it is more challenging compared to item recommendation in typical E-commerce scenarios (e.g., Taobao, Jd, and YouTube). The in-nature characteristics of low-frequency and high unit-price lead to more severe sparse and long-tail data distributions. Moreover, for enhancing user experience and business returns, the recommender system seeks to improve both click-through rate (CTR) and conversion rate (CVR) where the seesaw phenomenon may occur. In order to address the aforementioned shortages in hotel recommendation, a multitask learning (MTL) method with a novel flexible multilevel extraction network [denoted as flexible MTL (FMTL)] is proposed. Particularly, FMTL takes MTL into consideration in a unified representation learning framework and is divided into feature encoding and task prediction. In the feature encoding phase, we introduce a novel multirepresentation extractor with temperature-adjusted gating mechanism (T-MRE) for each task, producing more flexible representations for sparse and long-tail data. Moreover, we fuse different representations for each task with three strategies during the prediction phase and empirically demonstrate that the simple concatenation strategy is superior than other relatively complex gating approaches. Offline and live experiments with regard to both overall metrics and user group analysis based on the scarcity of user behaviors illustrate that without significantly increasing model parameters, our FMTL model outperforms substantially over several state-of-the-art models.
Invariant Node Representation Learning under Distribution Shifts with Multiple Latent Environments

Node representation learning methods, such as graph neural networks, show promising results when testing and training graph data come from the same distribution. However, the existing approaches fail to generalize under distribution shifts when the nodes reside in multiple latent environments. How to learn invariant node representations to handle distribution shifts with multiple latent environments remains unexplored. In this article, we propose a novel I nvariant N ode representation L earning (INL) approach capable of generating invariant node representations based on the invariant patterns under distribution shifts with multiple latent environments by leveraging the invariance principle. Specifically, we define invariant and variant patterns as ego-subgraphs of each node and identify the invariant ego-subgraphs through jointly accounting for node features and graph structures. To infer the latent environments of nodes, we propose a contrastive modularity-based graph clustering method based on the variant patterns. We further propose an invariant learning module to learn node representations that can generalize to distribution shifts. We theoretically show that our proposed method can achieve guaranteed performance under distribution shifts. Extensive experiments on both synthetic and real-world node classification benchmarks demonstrate that our method greatly outperforms state-of-the-art baselines under distribution shifts.
Reinforcement learning-based rate adaptation in dynamic video streaming

Video streaming stands out as the most significant traffic type consumed by mobile devices. This increased demand has been a major driver for research on bitrate adaptation algorithms. Bitrate adaptation ensures high user-perceived quality, which, in turn, correlates with higher profits for content providers and delivery systems. Dynamic adaptive streaming over HTTP (DASH) is a widely adopted video streaming standard utilized by service providers to provide competitive quality of experience (QoE). It is capable of providing seamless streaming via uncertain network conditions by switching across different video qualities and their corresponding video segment bitrates. The complexity of the video streaming environment makes it a good candidate for different learning-based approaches. Accordingly, this paper proposes a reinforcement-learning (RL) deep Q-network called DQNReg, that enhances the classical deep Q-learning method. A segment-wise QoE-based reward function is formulated so that the learning strategy can converge towards maximizing the QoE outcome. The proposed RL-based adaptation approach is evaluated using trace-based simulation for both wireless local area network channels and 5G mobile channels. The performance of this RL-based method is compared to three methods: A heuristic method, a model-based method, and a classical learning-based method. The comparison shows that the RL-based method converges faster while achieving a high QoE score. In addition, it reduces the re-buffering duration while maintaining higher video quality and relatively lower quality variations.
Reducing the number of unnecessary biopsies for mammographic BI-RADS 4 lesions through a deep transfer learning method

Abstract Background In clinical practice, reducing unnecessary biopsies for mammographic BI-RADS 4 lesions is crucial. The objective of this study was to explore the potential value of deep transfer learning (DTL) based on the different fine-tuning strategies for Inception V3 to reduce the number of unnecessary biopsies that residents need to perform for mammographic BI-RADS 4 lesions. Methods A total of 1980 patients with breast lesions were included, including 1473 benign lesions (185 women with bilateral breast lesions), and 692 malignant lesions collected and confirmed by clinical pathology or biopsy. The breast mammography images were randomly divided into three subsets, a training set, testing set, and validation set 1, at a ratio of 8:1:1. We constructed a DTL model for the classification of breast lesions based on Inception V3 and attempted to improve its performance with 11 fine-tuning strategies. The mammography images from 362 patients with pathologically confirmed BI-RADS 4 breast lesions were employed as validation set 2. Two images from each lesion were tested, and trials were categorized as correct if the judgement (≥ 1 image) was correct. We used precision (Pr), recall rate (Rc), F1 score (F1), and the area under the receiver operating characteristic curve (AUROC) as the performance metrics of the DTL model with validation set 2. Results The S5 model achieved the best fit for the data. The Pr, Rc, F1 and AUROC of S5 were 0.90, 0.90, 0.90, and 0.86, respectively, for Category 4. The proportions of lesions downgraded by S5 were 90.73%, 84.76%, and 80.19% for categories 4 A, 4B, and 4 C, respectively. The overall proportion of BI-RADS 4 lesions downgraded by S5 was 85.91%. There was no significant difference between the classification results of the S5 model and pathological diagnosis ( P = 0.110). Conclusion The S5 model we proposed here can be used as an effective approach for reducing the number of unnecessary biopsies that residents need to conduct for mammographic BI-RADS 4 lesions and may have other important clinical uses.
Improving scene text image captioning using transformer-based multilevel attention

Many existing image captioning methods only focus on image objects and their relationships for generating image captions, ignoring the text present in an image. Scene text (ST) contains crucial information to understand an image and facilitating reasoning. The existing methods fail to establish strong correlations between optical character recognition (OCR) tokens, as they have limited OCR representation power. Further, these methods have not efficiently used the positional information of the text. In this work, we have proposed an ST-based image captioning model (Trans-MAtt) based on a multilevel attention mechanism and relation network. We have used relation networks to enhance the connections between ST tokens. We have employed a multi-level attention method, which comprises of spatial, semantic, and appearance attention modules that precisely define the image. To represent context-enriched ST tokens, we use a combination of appearance, location, FastText, and PHOC features. We predict the ST location in the image, which is further integrated with the generated word embeddings for final caption generation. Experiments on the TextCaps dataset demonstrate the effectiveness of the proposed Trans-MAtt model, where it outperforms the current best model by 3.4% on B-4, 2.9% on METEOR, 3.3% on ROUGE-L, 3.1% on CIDEr-D, and 4.1% on SPICE metric scores. Our experiments on the Flickr30k and MSCOCO datasets demonstrated the superiority of our proposed model over existing methods.
Real-time waste segregation at the individual level: the smart waste sorter

Segregating waste accurately at an individual level is paramount for efficient waste management, especially considering the staggering 268 million tons of waste generated each year in the USA, a large portion of which is recyclable. To address this issue, we developed a Smart Waste Sorter (SWS), which is a portable device that can be placed on any waste bin. It uses deep learning models to identify whether a piece of waste is a battery, recyclable, compostable, or recyclable, or trash, and provides a real time alert if the user is about to dispose of the item incorrectly. To develop the SWS image classification model, we utilized a dataset of 4,122 images, obtained from a combination of publicly available and manually collected images from households over several months. We experimented with four models of varying sizes: VGG16, EfficientNetB1, MobileNetV2, and ResNet50, to investigate whether a smaller model could achieve comparable performance, given that our device is portable and requires a compact model that can operate on limited memory without internet connectivity. Our experiments showed that ResNet50 achieved the highest validation and test accuracy of 77.91% and 96.39% respectively over four categories, suggesting that smaller models can be effective. Our results demonstrate the potential of the SWS to improve real-time waste segregation at the individual level, while considering practical constraints for implementation. The proposed solution utilizes a Raspberry Pi to detect motion, capture images and classify them. Our solution provides an effective, practical, and low-cost method for accurately segregating waste and contributing to sustainable waste management.
A Model for the Construction of the Curriculum System of Hua People Students&amp;apos; China Knowledge Education

Hua people students (HPS), Citizens of Chinese descendant of foreign nationality, are unique in their natural affinity for China and have the most value in the development of non-governmental diplomacy.It is an important way to improve China's international communication ability to provide them with good conditions and opportunities to know China personally, experience China on the spot, devote themselves to research, form a true, objective and comprehensive understanding of China, make them grow into international friends who know China.The key to achieving this goal is to build a modern curriculum system that matches the cognitive psychology, acquisition ability and acceptance interest of the HPS.A two-level model is proposed for this curriculum system.The first level model integrates Chinese knowledge education (CKE)'s educational objectives, graduation (complete courses) requirements, curriculum and syllabus, teaching implementation and effect evaluation into one, with forward dependence and reverse reference.The second level model mainly solves the three problems of comprehensive weighing factors, construction principles, curriculum types and specific composition of curriculum system construction.Establish the sequence of degree education and non-degree education, set up education levels according to the two stages of undergraduate and graduate education, and finally grade the curriculum according to the three standards of China, students and content.At the same time, the policy and financial support, social linkage and communication coordination mechanism are included in the construction model as a guarantee system to ensure the effectiveness of teaching activities.
Security Impact of Federated and Transfer Learning on Network Management Systems with Fuzzy DEMATEL Approach

Everyday using of the big data, machine learning algorithms, and related studies, ensuring data privacy and security have become a critical necessity.These features make them more vulnerable to cyber-attacks.The security of the stored data is also critical, and evaluating the processing of information in the autonomous network management of these systems.The criteria considers the account in the processing and security of data entering every field from the widespread industry examined.It is necessary to increase their awareness of negative and attack problems while these systems are working.Applications such as traditional machine learning and the use of cloud computing also involve risks regarding data security and personal data leakage.Cooperative learning pays due attention to the confidentiality of sensitive information by keeping the original training data hidden.By collecting, combining, and integrating heterogeneous data with collaborative learning together with a federated learning structure, data produced and stored.This study discusses the effect of federated and transfer learning on autonomous network management analyzes the security status parameters.The fuzzy DEMATEL method was preferred in exploring the parameters affecting the system state according to the degree of importance.Situational scenarios evaluated by considering the structure in which the features of cyber-physical systems examined together with federated learning.Data security factors discussed with the fuzzy DEMATEL
Privacy Enhancement with Perturbation Method for Multidimensional Grid

With the development of technology, the use of big data is spreading at an increasing rate.The issues of storing, analysing and securing data have brought along the methods that need to be developed.Ensuring data privacy and data security is the case of partial separation and processing of data with the perturbation method of data with the block chain approach.Within the scope of this study, data analysis performed using normalization, geometric rotation, linear regression and scalar data multiplication and comparative classification in precision data mining.
An energy-aware combinatorial auction-based virtual machine scheduling model and heuristics for green cloud computing

Considering the increasing demand for cloud computing, and the financial and environmental impact of the increasing energy consumption trend of data centers, improving energy efficiency is vital for cloud service providers. In this study, an energy-aware virtual machine scheduling model is proposed which is based on the multi-unit nondiscriminatory combinatorial auction. The model includes a powerful bidding language that allows users to declare their complicated virtual machine requests using logical AND and OR relations along with the time constraints. The study also presents the formal definition of the model and the associated optimization problem for determining the optimum schedule and energy-efficient placement of VMs on physical servers. The optimization problem is formulated using integer linear programming and several heuristic solution methods including the Genetic Algorithm are proposed for this problem. The performances of the model and the proposed heuristics are assessed on a comprehensive test suite. The proposed model is estimated to provide approximately a 37% improvement in revenues, and the solution methods are estimated to provide high-quality solutions within only 5% of the optimum which enable the model to be deployed in large-scale clouds.
The Effects of Applied Behavior Analysis-Based Behavioral Contracts on Self-Management Skills of Students with Intellectual Disabilities

In order to examine the effects of applied behavior analysis-based behavioral contracts have on self-management skills of students with intellectual disabilities, in this study, behavioral contracts were implemented for a total of 2 students, one 2nd grade student in an elementary school special class and one 1st grade student attending post-high school specialized vocational program in special school. Based on previous studies, the behavioral contracts were prepared by including the common components required for the behavioral contracts. As for the self-management skills, regular and repetitive daily behaviors with a high probability to perform everyday were selected among the daily life skills required for individual students. In particular, factors related to self-management competency were considered in the whole process starting from selecting target behavior to writing the behavioral contracts, implementing the behavioral contracts, and ending the behavioral contracts. The experiments of student A and student B were conducted at school and at home, respectively, and functional relation was demonstrated by applying a reversal design. As a result of visual analysis, the applied behavior analysis-based behavioral contracts were effective in increasing the self-management skills of students with intellectual disabilities. The effects of intervention observed in school were maintained afterwards, and the effects of intervention at home were also generalized in the school environment. Discussion and suggestions for the overall study have been described.
The State of Art in Machine Learning Applications in Civil Engineering

Machine learning (ML) is one of the methods used by the artificial intelligence approach. Machine learning is used to teach machines how to handle data more efficiently. The purpose of machine learning is to learn from the data. Thanks to machine learning, a certain result can be reached without the need for an expert on the subject. It has very common usage areas from the financial sector to e-mail analysis. Machine learning is also widely used in civil engineering. In this study, machine learning is explained in historical development, and general terms and the studies that have been done are summarized.
THE EFFECT OF TECHNOLOGICAL CHANGES ON EMPLOYEES’ WORK PERFORMANCE AMONG NON-ACADEMIC STAFF

Technological changes are important in improving performance as well as increasing productivity of the employees. The state-of-the-art technologies help placing organization a better position to face competitor and stay relevant through this technological advancement era. In this rapid changing of technologies, employees should fully utilize this opportunity to improve work performance and be able to adapt with these changes to raise their standard work performance. However, not all employees can cope with the changes of new technology in their organization and it is feared that such behaviour can influence job performance, since majority of the employees (non-academic staff) comprises of older generation workers who are against or resistant to this change. The purpose of this study was to determine the relationship between technological changes and employees’ work performance. Quantitative data was collected using a self-administered questionnaire that consisted of items with five-point Likert scale. A total number of 250 non-academic staff was identified from the Administrative Department of Universiti Teknologi MARA (UiTM), Kelantan Branch. A stratified random sampling technique was applied as to ensure that the strata (or layers) in the population are fairly represented in the sample and 100 respondents were later determined. The questionnaires were then distributed randomly to 100 respondents from 10 different non-academic departments in UiTM Kelantan Branch. Result of the findings showed that training of new technology is the most affecting dimension of technological change that affect non-academic staff of UiTM Kelantan Branch. It was also found that New Working Method and Training of New Technology have strong relationship with employees’ work performance. However, Acceptance has no relationship with employees’ work performance. It is therefore recommended that a new model for the independent variable should be developed by future researcher.