Reviewerly/exHarmony huggingface dataset:
Exploring the Impact of Serverless Computing on Peer To Peer Training Machine Learning

The increasing demand for computational power in big data and machine learning has driven the development of distributed training methodologies. Among these, peer-to-peer (P2P) networks provide advantages such as enhanced scalability and fault tolerance. However, they also encounter challenges related to resource consumption, costs, and communication overhead as the number of participating peers grows. In this paper, we introduce a novel architecture that combines serverless computing with P2P networks for distributed training and present a method for efficient parallel gradient computation under resource constraints.Our findings show a significant enhancement in gradient computation time, with up to a 97.34% improvement compared to conventional P2P distributed training methods. As for costs, our examination confirmed that the serverless architecture could incur higher expenses, reaching up to 5.4 times more than instance-based architectures. It is essential to consider that these higher costs are associated with marked improvements in computation time, particularly under resource-constrained scenarios.Despite the cost-time trade-off, the serverless approach still holds promise due to its pay-as-you-go model. Utilizing dynamic resource allocation, it enables faster training times and optimized resource utilization, making it a promising candidate for a wide range of machine learning applications.
Comparative Analysis of BanglaT5 and Pointer Generator Network for Bengali Abstractive Story Summarization

Sequence-to-sequence models, especially transformer-based ones, are the recent baseline for generating a summary of text documents and various natural language processing (NLP) tasks. Transformer-based language models can perform well for text summarization. However, for text summarization, these models often need help to capture the actual context of a summary. This paper compares the BanglaT5 seq-to-seq model with a pointer-generator network for generating novel words with a feature-rich encoder. We aim to build an automatic text summarizer that can summarize literary works. We present a new dataset, namely the Bangla short story collection of Rabindranath Tagore. Technically, this will be a step towards Bangla literature revealing something new. Evaluations from ROUGE-1, ROUGE-2, and ROUGE-L scores show that the pointer-generator network with linguistic features significantly enhances text summarization. We got better ROUGE scores for a pointer-generator network that are explained in the result analysis section.
Prioritization of Identified Data Science Use Cases in Industrial Manufacturing via C-EDIF Scoring

While data science and artificial intelligence (AI) can be highly beneficial for industrial manufacturers, it is not yet readily usable. Therefore, putting it to good use requires to understand the domain challenges and identify opportunities for deploying AI. Our work aims at solving this task by proposing a generalized framework for ❨1❩ exploring companies for use cases and (2) prioritizing them via C-EDIF scoring. This novel approach allows to determine the business importance of any use case by considering the underlying evaluability, data situation, impact and infeasibility. Besides the theoretical framework, our work also provides real-world insights from applying C-EDIF scoring in an extensive use case exploration phase. These results stem from a strategic partnership between data scientists and Wilo SE, a renowned pump manufacturing company, where we successfully identified and rated opportunities for AI.
Delayed Random Partial Gradient Averaging for Federated Learning

Federated learning (FL) is a distributed machine learning paradigm that enables multiple clients to train a shared model collaboratively while preserving privacy. However, the scaling of real-world FL systems is often limited by two communication bottlenecks: (a) while the increasing computing power of edge devices enables the deployment of large-scale Deep Neural Networks (DNNs), the limited bandwidth constraints frequent transmissions over large DNNs; and (b) high latency cost greatly degrades the performance of FL. In light of these bottlenecks, we propose a Delayed Random Partial Gradient Averaging (DPGA) to enhance FL. Under DPGA, clients only share partial local model gradients with the server. The size of the shared part in a local model is determined by the update rate, which is coarsely initialized and subsequently refined over the temporal dimension. Moreover, DPGA largely reduces the system run time by enabling computation in parallel with communication. We conduct experiments on non-IID CIFAR-10/100 to demonstrate the efficacy of our method.
Optimizing Region of Convergence to Preserve Model Privacy from Membership Inference Attack

Recently, membership attacks are a well-known thread to disclose the training data of deep learning models, which can leak sensitive data under several circumstances. To prevent such attacks, improving the model's generalization is one of the key approaches. Such generalization capability of the deep model can be achieved through the careful examination of loss landscape and the usage of large learning rate. Based on the popular stochastic gradient optimizer, our work explores the connection between the training learning rate and the resulting model's loss landscape in defensing against the membership attacks. We found out that flat surfaces in the loss landscape which come from a large learning rate tend to preserve the model's privacy better while maintains a good prediction accuracy. We validate our findings with three model architectures, ResNet-18, VGG-11 and 2-layers MLP on the two popular datasets, FashionMNIST and CIFAR10. The results show that a large learning rate can be used to improve the model privacy from 2–4% with better models' accuracy.
Learning Representations through Contrastive Strategies for a more Robust Stance Detection

Stance Detection refers to the process of determining an author’s position towards a particular issue or target in a text. Previous research suggests that existing systems for Stance Detection are not resilient enough to handle variations and errors in input sentences. In our proposed methodology, we utilize Contrastive Learning to learn sentence representations. We achieve this by bringing semantically similar sentences and those implying the same stance closer to each other in the embedding space. To compare our approach, we use a pretrained transformer model that is directly finetuned with the stance datasets. We evaluate the resilience of the models using char-level and word-level adversarial perturbation attacks and show that our approach performs better and is more robust to the different adversarial perturbations introduced to the test data. Our approach is also shown to perform better on small-sized and class-imbalanced stance datasets. We further experiment with unlabeled stance datasets to make the representation learning independent of domain-specific labels, and the models trained with our approach on unlabeled datasets are still robust and perform comparably to those trained with labeled data.
Direct Sequence Spread Spectrum-Based Radio Steganography

Direct sequence spread spectrum (DSSS) is a spread-spectrum modulation technique primarily used to reduce overall signal interference. DSSS is one of the so-called secondary modulation techniques, which makes the transmitted signal wider in bandwidth than the information bandwidth. On the other hand, this technique allows the transmitted information signal to be hidden in the noise. Thus, DSSS signals are classified as low probability of intercept/low probability of detection (LPI/LPD). DSSS modulation is also used in information hiding for both radio and audio signals. In this case, the DSSS-based steganography algorithms used various information-hiding mechanisms, e.g., modulation parameter modification. In the novel solution proposed in this paper, we use DSSS with variable processing gain for keying additional information. In this case, the hidden message is transmitted in a switching order of spreading sequences taken from a mapping dictionary known only to the notified parties.
TOCOL: Improving Contextual Representation of Pre-trained Language Models via Token-Level Contrastive Learning

Self-attention, which allows transformers to capture deep bidirectional contexts, plays a vital role in BERT-like pre-trained language models. However, the maximum likelihood pre-training objective of BERT may produce an anisotropic word embedding space, which leads to biased attention scores for high-frequency tokens, as they are very close to each other in representation space and thus have higher similarities. This bias may ultimately affect the encoding of global contextual information. To address this issue, we propose TOCOL, a TOken-Level COntrastive Learning framework for improving the contextual representation of pre-trained language models, which integrates a novel self-supervised objective to the attention mechanism to reshape the word representation space and encourages PLM to capture the global semantics of sentences. Results on the GLUE Benchmark show that TOCOL brings considerable improvement over the original BERT. Furthermore, we conduct a detailed analysis and demonstrate the robustness of our approach for low-resource scenarios.
An Anomaly Detection Algorithm of QAR Based on Spatial-Temporal Correlation

QAR (Quick Access Recorder) data contains numerous quality flaws such as anomalies and missing data. It will cause significant problems for subsequent data mining, model training, and analysis if it is not detected. To address these issues, this paper investigates QAR-specific anomaly detection (AD) algorithms before presenting a three-stage QAR data AD algorithm based on QAR spatial-temporal correlation, which includes single parameter AD, parameter correlation analysis, and multi parameter AD. The SST (Singular Spectrum Transformation) is used in this process to analyze the correlation between parameters based on the change point rather than the change trend. Simultaneously, a double K-means clustering algorithm that can automatically select the hyper-parameters K is proposed, followed by a relatively complete empirical experiment. The methods investigated in this paper are implemented in Python code, and their feasibility and effectiveness are demonstrated through simulation analysis. The accuracy rate is increased by 54% when compared to existing literature methods.
Classification and Prioritization of Requirements Smells Using Machine Learning Techniques

Software requirements are a description of what the software is expected to do and behave. Specifying requirements in natural language might face difficulties like clarity, inaccuracy, ambiguity, incompleteness, vagueness etc. An indicator of such problems in requirements are technically termed as requirement smells. They need early detection and rapid response to ensure the quality of requirements. However, previous requirement smells detection approaches have scalability and flexibility problems, and poor performance. In addition, there is a gap on prioritizing detected requirement smells to take an appropriate action based on the order. In this study, we address the gaps by developing a Machine Learning (ML) based approach for the classification and prioritization of requirement smells. We have collected 3100 requirements and labeled them by experts. To prioritize requirements smells, we have used requirement smells severity level collected from experts and requirements importance level from Software Requirement Specification (SRS) document. Then, textual requirements were preprocessed using Natural Language Processing (NLP) techniques, and features were extracted using TF-IDF and BOW. We have used 80/20 train-test split ratio. For both classification and prioritization model building, commonly used classification algorithms LR, NB, SVM, DT, and KNN were applied and their performance is compared. Moreover, we have also used an additional sorting method for prioritizing requirement smells detected from a single project. As a result, LR with TF-IDF achieved highest performance with 94% accuracy for requirement smells classification. For requirement smells prioritization, SVM outperformed other algorithms with 99% accuracy.
DynArt ChatGPT: a platform for the generation of dynamic non-traditional New Year paintings

In a modern, technology-driven society, the preservation and development of China’s Intangible Cultural Heritage (ICH) has encountered three major contradictions: economic development, technological renewal and cultural change, resulting in a lack of people to pass on the heritage, difficulties in innovation and inadequate protection. In order to promote and develop Chinese traditional art, we had a vision: can we combine the conversation function of ChatGPT with the traditional art of ICH? Based on this idea, we have built a system - DynaArt ChatGPT. DynaArt ChatGPT can extract the keywords from the sample given by ChatGPT and then generate a description related to the sample based on the keywords, we input the description into Lumen5 model, Lumen5 will generate the corresponding dynamic video according to the description we input, after the experiment we think the dynamic painting generation After experiments we believe that the system can provide a new interpretation of some familiar Chinese folk tales.
A Summary of the ALQAC 2023 Competition

This paper presents an overview of the third edition of the Automated Legal Question Answering Competition (ALQAC 2023). The primary objective of ALQAC is to address challenges related to processing legal texts in languages with limited linguistic resources. This year's competition focuses on two specific tasks involving Vietnamese statute law documents: Legal Document Retrieval (Task 1) and Legal Question Answering (Task 2). In Task 1, participants are tasked with developing systems capable of retrieving pertinent legal articles in response to a given legal question. In Task 2, the challenge involves creating systems that can provide accurate answers to legal questions. The competition received participation from 5 teams in Task 1, submitting a total of 15 runs, while Task 2 had 4 teams participating with 12 runs. This paper offers a comprehensive description of the designated tasks along with a description of the approaches adopted by each participating team. Furthermore, the competition results are presented, followed by a thorough analysis and discussion.
Enhancing Clinical Note Generation from Doctor-Patient Conversations through Semantic Partition-Oriented Summarization

Clinical note generation from doctor-patient con-versations is an essential task that helps to maintain the medical records of patients. The process of writing clinical notes is a time-consuming task for doctors, and redundant or inaccurate information in clinical notes may have adverse consequences. In this paper, we propose a novel approach to clinical note generation. Our contribution lies in proposing a semantic partitioning and clustering method for the extractive module of this task. We show that our semantic-based partition method provided a way to extract relevant information for specific sections. This enables large language models to produce high-quality clinical notes and give accurate information for each section. Our approach outperformed other systems in ROUGE-1 score on the MEDIQA-Sum 2023 Shared Task dataset.
Supplier qualification document recognition through open-set recognition

Large and medium-sized manufacturing companies are concerned with maintaining their supplier registries with well-reputed suppliers sourced over time. Every supplier periodically undergoes rigorous qualification processes where procurement officers assess, among other factors, the supplier document compliance status. To this end, procurement officers periodically ask suppliers, via digital e-procurement platforms, for qualification documents of different categories. Conversely, suppliers promptly answer by handing out such documents that can, maliciously or inadvertently, be wrong. When wrong qualification documents remain undetected, and the associated suppliers are qualified, a threat to the overall business arises: procurement officers may entrust purchase orders to not compliant suppliers with unpredictable performances. Our claim is that equipping e-procurement platforms with document recognition based on supervised open-set recognition (OSR) could mitigate the problem. In particular, we deem OSR solutions suitable for supplier qualification document recognition due to their simultaneous abilities of i) recognizing documents belonging to relevant categories and ii) rejecting those belonging to unknown categories. Quantitative and qualitative results from a real-world case study in partnership with an Italian manufacturing company show that the proposed solution is viable.
Design Amharic Text Sentiment Analysis Model Using Machine Learning Techniques. In Case of Restaurant Reviews

Sentiment analysis is a type of natural language processing for tracking the attitude of the public about a particular product, service or topic. It is also highly challenging as natural language processing research topic, and covers many novel sub-problems. Now business organizations and academics are putting forward their efforts to find the best system for sentiment analysis. The focus of this study was an Amharic unstructured restaurant review on the web. The objective of the paper was to design Amharic text sentiment analysis model using supervised machine learning techniques and evaluate the performance of classifiers. This paper explored the supervised machine learning classification approaches (naive Bayes, support vector machine and k-nearest neighbor) with different feature selection schemes to obtain a sentiment analysis model for domain specific restaurant review dataset at sentence level. The proposed model has the following components: Data preparation, preprocessing such as tokenization, normalization, filter stop words, feature extraction and selection to prepare feature vector, polarity classification. Performance analysis carried out on classifiers, based on n-grams proposed. From the results of the experimental studies, all algorithms are known to be highly effective classifiers, and are able to achieve good accuracy in this experiment. The experiments show that Term frequency (TF) and the TF -IDF scheme gives maximum accuracy 80.43 % and 79.49 % respectively for SVM in bigram features. Term frequency and term occurrence also give maximum accuracy 78.37 % and 78.00 % respectively for Naive Bayes classifier at bigram features. TF-IDF also give maximum accuracy 78.00% for KNN at 4-gram. The challenge was opinion holders sometimes give objective text to express their opinion, but the classifier did not identify those facts from opinions. These kinds of complexities of natural languages make sentiment mining systems more challenging and to resolving this challenge subjectivity and objectivity classification is needed.
Multi-level Similarity Complementary Fusion for Unsupervised Cross-Modal Hashing

Cross-modal hashing is widely used in multimedia retrieval tasks for its advantages of low storage cost and high retrieval speed. In recent years, many deep unsupervised cross-modal hashing methods were proposed to deal with unlabeled data. These methods usually generate a similarity matrix from raw features as a guidance signal for model training. However, this type of techniques introduce bias in feature measurements as they fail to accurately capture semantic relationships between cross-modal data to eliminate the error caused by the original features. Furthermore, the multiple semantic correlations between instances are underutilized, which makes it difficult to narrow the semantic gap. To address these issues, we propose a novel multi-level similarity complementary fusion (MSCF) method. Specifically, we fuse similarity matrices from different modalities. After feature interaction and reconstruction, we perform multilevel complementary fusion with the similarity in the common Hamming space to form the final instance similarity matrix, which is used to guide the model training. We conduct extensive experiments to verify the improvement of our work over existing works.
Comparing the Effectiveness of PPO and its Variants in Training AI to Play Game

Automated game intelligence is a crucial step in rapid game development. A promising research direction for automated game intelligence is reinforcement learning, and specifically, the proximal policy optimization (PPO) algorithm. Two variants of the PPO, Maskable PPO and Recurrent PPO, further extend the capabilities of the PPO. We compare the performance of these three algorithms in the 2D game Mario and a 3D car racing game environment. We also evaluate their performance and applicability by comparing the experimental results of the original algorithm authors. With our results, we provide recommendations on PPO configuration depending on the target game type, providing future developers with a benchmark to help them decide which algorithm is most applicable for their applications.
On Raising a Virtual Human

I have been fortunate to have experienced and contributed to the software development of virtual humans. Whether they are Avatars, Conversational Agents, game NPCs, or Virtual Influencers, they all have origins in computer graphics (CG). Indeed, the “DNA” of these human models encompasses a large set of CG component technologies including 3-D modeling, animation techniques, and interactive systems. Finding (or purporting to know) “the” path from the origins to the present would require writing a CG and art history book about representing the human form. Here I will try to document portions of an evolutionary path—and my personal journey—to elucidate the context and processes that led up to the explosion of virtual human software “DNA” by the turn of the millennium.