Reviewerly/exHarmony huggingface dataset:
Large Language Models Evaluate Machine Translation via Polishing

Quality of machine translation is usually evaluated by automatic reference-based metrics. However, these conventional metrics have limited correlation with human evaluation and require references for comparison. Human evaluation is considered the most reliable, but it is still plagued by high costs and uncertainties among annotators. With the emergence of large language models (LLMs) and their impressive capabilities, some studies have utilized LLMs for the evaluation of natural language generation. In this paper, we propose EvLP (Evaluation via LLMs Polishing), a reference-free machine translation evaluation method inspired by post-editing that leverages LLMs. We select gpt-3.5-turbo to act as annotators to polish the translated text. Our experiments compare EvLP with other LLM-based evaluation methods and SEG_LM, a representative reference-free metric utilizing cross-lingual pre-trained language models to evaluate machine translation. We select a dataset with two kinds of human evaluation, direct assessment and post-editing. The results of experiments demonstrate that EvLP closely aligns with human evaluation compared to other LLM-based metrics, outperforming SEG_LM on direct assessment and exhibiting a significant advantage in post-editing. We also discuss the bias that LLMs hold towards LLM-generated texts.
Wasserstein Topology Transfer for Joint Distilling Embeddings of Knowledge Graph Entities and Relations

A high-dimensional knowledge graph embedding (KGE) space is usually required for a better reasoning capability in representing entities and relations. However, a high-dimensional KGE can lead to high memory overhead with the increasing number of entities and the diversity of relations, which is limited in mobile or edge devices. Recently, researchers have focused on reducing the memory burden by distilling critical information from a high-dimensional KGE teacher into a low-dimensional KGE student. However, these methods still face two main issues: (1) they neglect the distillation of graph topology from teachers; (2) they distil entities and relations independently instead of in a shared semantic space. To address these issues, we propose Wasserstein Topology Transfer (termed as WTT), which effectively discovers graph structure and transfers graph topology from teachers to students. In particular, we propose two novel optimal transport (OT) regularizers: Reversing Wasserstein Regularizer and Blending Wasserstein Regularizer, to constrain the student’s behaviours and enable an efficient distillation from teachers to students of entity and relation embeddings. The regularizers achieve entities and relations distillation in the same OT procedure and reduce redundant transport procedures by adjacency matrices in KGs. Extensive experimental results have demonstrated the effectiveness of our WTT method.
SARA: Slot Attention for Robust Adversarial Training

In the field of adversarial attacks, deceptive perturbations in attacks are key to influencing model behavior. Exploring how perturbations affect model behavior using interpretability techniques can contribute to the development of robust adversarial training. This is because interpretability methods can provide reasoning attention maps that cover features related to model decisions. In this paper, we propose SARA as a robust adversarial training framework based on slot attention. SARA inputs clean examples and adversarial examples into two separate slot attention models, respectively, where the slot attention trained with adversarial examples is supervised by the attention from the clean example training. This forces the adversarial training model to capture features with a similar shape as the clean model, reducing the impact of deceptive perturbations. In terms of robustness, our training framework significantly outperforms current mainstream adversarial training techniques on ImageNet.
Enhancing Text Comprehension via Fusing Pre-trained Language Model with Knowledge Graph

Pre-trained language models (PLMs) such as BERT and GPTs capture rich linguistic and syntactic knowledge from pre-training over large-scale text corpora, which can be further fine-tuned for specific downstream tasks. However, these models still have limitations as they rely on knowledge gained from plain text and ignore structured knowledge such as knowledge graphs (KGs). Recently, there has been a growing trend of explicitly integrating KGs into PLMs to improve their performance. For instance, K-BERT incorporates KG triples as domain-specific supplements into input sentences. Nevertheless, we have observed that such methods do not consider the semantic relevance between the introduced knowledge and the original input sentence, leading to the issue of knowledge impurities. To address this issue, we propose a semantic matching-based approach that enriches the input text with knowledge extracted from an external KG. The architecture of our model comprises three components: the knowledge retriever (KR), the knowledge injector (KI), and the knowledge aggregator (KA). The KR, built upon the sentence representation learning model (i.e. CoSENT), retrieves triples with high semantic relevance to the input sentence from an external KG to alleviate the issue of knowledge impurities. The KI then integrates the retrieved triples from the KR into the input text by converting the original sentence into a knowledge tree with multiple branches, the knowledge tree is transformed into an accessible sequence of text that can be fed into the KA. Finally, the KA takes the flattened knowledge tree and passes it through an embedding layer and a masked Transformer encoder. We conducted extensive evaluations on eight datasets covering five text comprehension tasks, and the experimental results demonstrate that our approach exhibits competitive advantages over popular knowledge-enhanced PLMs such as K-BERT and ERNIE.
Exploration of Network Optimization Strategies Based On the TSN Model

With the widespread application of deep learning in the field of computer vision, deep learning-based video behavior recognition models have become important tools for video recognition tasks. However, these models currently face two challenges: how to expand and enhance data to improve model accuracy, and how to achieve lightweight networks without increasing computational complexity. The Temporal Segment Network (TSN) model is a deep learning model used for video action recognition. It combines 2D and 3D convolutional neural networks and effectively captures temporal and spatial information in videos while maintaining the requirement for lightweight networks. This paper explores the optimal data augmentation strategy based on the TSN model and proposes a Temporal Feature Fusion Algorithm to optimize the TSN model, providing a solution for high-performance lightweight networks. These findings will contribute to the exploration of data augmentation techniques in future video behavior recognition models and provide insights for lightweight network research.
Drugs for Sale! An Analysis and Estimation of Drug Products on the Cryptomarket Ecosystem

Cryptomarkets are marketplaces on the dark web that facilitate the sale of mostly illicit goods and services between vendors and buyers. There were few to no studies that examine the cryptomarket ecosystem using data from multiple cryptomarkets collected at one point in time. This study strives to fill that gap by collecting all product information from eight large or notable cryptomarkets between June 2021 and January 2022 to understand the cryptomarket drug ecosystem, the products available, and identify factors that encourage or discourage vendors from shipping internationally. The eight cryptomarkets annually trafficked an estimated 13.4 tons of drugs for $239.2 million CAD. The characteristics of drug products will be examined to determine whether it influences the vendor’s willingness to ship internationally. Larger quantities and less expensive products were more likely to be shipped internationally. Specifically, products that cost less than $50 were more likely to be able to ship internationally.
Selective Sharing is Caring: Toward the Design of a Collaborative Tool to Facilitate Team Sharing

Temporary teams are commonly limited by the amount of experience with their new teammates, leading to poor understanding and coordination. Collaborative tools can promote teammate team mental models (e.g., teammate attitudes, tendencies, and preferences) by sharing personal information between teammates during team formation. The current study utilizes 89 participants engaging in real-world temporary teams to better understand user perceptions of sharing personal information. Qualitative and quantitative results revealed unique findings including: 1) Users perceived personality and conflict management style assessments to be accurate and sharing these assessments to be helpful, but had mixed perceptions regarding the appropriateness of sharing; 2) Users of the collaborative tool had higher perceptions of sharing in terms of helpfulness and appropriateness; and 3) User feedback highlighted the need for tools to selectively share less data with more context to improve appropriateness and helpfulness while reducing the amount of time to read.
ICU Outcome Predictions Using Real-Time Signals with Wavelet-Transform-based Convolutional Neural Network

Intensive care units (ICUs) serve patients with life-threatening conditions. The limited ICU resources cause severe economic and healthcare burdens worldwide. It is critical to conduct ICU outcome predictions at an early stage and promote efficient use of ICU resources. However, all the current prediction methods have limitations such as unsatisfactory accuracy and depending on resource-demanding laboratory tests or expert domain knowledge. In this research, we design a wavelet-transformed-based convolutional neural network, WTCNN, which only requires patients’ vital sign series and information at ICU admission for real-time ICU outcome predictions. The model is evaluated using a large real-world ICU database and outperforms state-of-art baselines on both ICU mortality and length-of-stay prediction tasks. We conduct LIME for model interpretation and prescriptive analysis. Our work provides an efficient tool for ICU outcome predictions, allowing healthcare providers to take action promptly on patients at risk and reduce the negative impacts on patient outcomes.
Efektivitas Penggunaan Media Pembelajaran Berbasis Edlink Sebagai Alat Evaluasi Pembelajaran Di SMP Nasional Makassar

Research on the effectiveness of using Ed-link learning media as a learning evaluation tool at Makassar National Middle School aims to determine the effectiveness of using the Ed-link application as a learning evaluation tool. This type of research uses a quantitative method with a pre-experimental research design that uses one sample class in the form of pre-test and post-test. The data collection technique in this study was observation and test data analysis. The technique used was descriptive statistical analysis. The subjects in this study were students of class VII.B Makassar National Middle School in the 2022/2023 academic year, a total of 32 students. Based on data analysis, discussion of research results regarding the effectiveness of using learning media as a learning evaluation tool in Indonesian subjects, literary work entitled that: the ed-Link application is effectively used as an alternative learning evaluation tool at Makassar National Middle School with scores from observations of the use of ed- link in Indonesian subject class VII B is in a very good category which can be seen from the results of student learning through descriptive statistical analysis before using the media application ed-link, the average student score is still below the KKM after using the media, the average student score increases above KKM . It turns out that the value of the post test results, which is 2,422, is greater than the pre-test result value, which is 30.92. From the results of this study it can be interpreted that the use of ed-link as a learning evaluation tool in Indonesian subjects at the Makassar National Middle School is effective.
Pengembangan Media E-Book Berbasis Aplikasi Office Sway pada Bahan Ajar Modul Bahasa Indonesia Kelas X SMA

This study aims to describe the use, design and results of office sway-based media development in class X Indonesian teaching materials that are valid and practical. This type of research is Research and Development. Media research and development procedures use the Borg &amp; Gall development model, namely: surveys and data collection, planning, development of product drafts, expert validation tests, design revisions and product trials. This research was conducted at SMA Muhammadiyah 1 Unismuh Makassar with Class X research objects totaling 24 students and 1 Indonesian teacher. Research data collection techniques were obtained through observation, interviews, product validation to experts, student and teacher response questionnaires, as well as post-test and pre-test scores. The results of the data were analyzed quantitatively and qualitatively. The results of the study show that the developed media is classified as feasible, as can be seen from the validation of material experts with an average percentage of 78.33% and the validation of media experts with an average percentage of 86%. The Indonesian teacher's response stated that the media was good with an average percentage of 82.10% and the student response questionnaire met practical criteria, the completeness score of student learning outcomes was more than 7.5%, which was 8.45%. Based on this, comparison and analysis can be interpreted that the learning product developed in the form of an E-book based on the Office Sway application in the teaching materials of this module can be declared valid and practical to use as an Indonesian language learning medium.
Efektivitas Penggunaan Media Quizizz pada Mata Pelajaran Sejarah Kelas X SMK

Research on the Effectiveness of Using Quizizz Media in History Subjects of diachronic, synchronic, chronological, space and time concepts in history. Class X DKV (Virtual Communication Design) students at SMK Negeri 1 Karossa for the 2022/2023 academic year, with a total of 16 students. Aims to determine the effectiveness of using quizizz media in history subjects. The sample in this study were students of class X DKV (Virtual Communication Design), the method used in this study was an experimental method with a quantitative approach. The instruments in this study used tests, questionnaires, and documentation. Data collection techniques used are tests, questionnaires and documentation. data analysis technique by Descriptive Statistics analysis and Inferential Statistics analysis. The results of the descriptive statistical analysis showed that the average student learning outcomes before using the quizizz media was 47.25, the student's score was still below the KKM score and after using the quizizz media, the average student score was 89.81, an increase above the KKM score. For inferential statistical analysis using the t test formula, it is known that a significant value of 0.000 &lt;0.05 is obtained so that H0 is rejected and H1 is accepted. Based on these results, it shows that the use of quizizz media is effective in history subjects for class X DKV (Virtual Communication Design) SMK Negeri 1 Karossa, Central Mamuju Regency.
Efektivitas Penggunaan Media Interaktif Wizer.Me Terhadap Mata Pelajaran IPA Pada Siswa Kelas XI SMA

Research on the Effectiveness of using wizer.me interactive media on science subjects in class XI IPA II students at SMA Negeri 5 Bantaeng. Aims to determine the effectiveness of the use of wizer.me Interactive media in science subjects. This research is a type of quantitative research using experimental methods and the media used in this research, namely wizer.me. The sample is class XI IPA II students. The instruments in this study provide pre- and post-tests on the sample. Data collection techniques used are observation, tests, and documentation. Data analysis techniques are descriptive statistical analysis and inferential statistical analysis. Comparison of learning outcomes categories, and comparison of completeness levels has proven an increase in student learning outcomes in class XI IPA II at SMA Negeri 5 Bantaeng. It can be seen that student learning outcomes through descriptive statistical analysis before using wizer.me interactive media, the average student score is still below the mark KKM and after using wizer.me interactive media the average student score increased. He obtained that the posttest result value was 80.5, which was greater than the pretest result value, which was 61.75. Furthermore, based on the results of inferential statistical analysis using the t test formula, it is known that the t count obtained is 13.708, at a significant level = 0.05 or 5%, the t table is 1.740. So tCount &gt; tTable or 13.708&gt;1.740 thus H0 is rejected and H1 is accepted. From the results of this study it can be concluded that by using wizer.me interactive media it has been effectively used in learning science for class XI IPA II at SMA Negeri 5 Bantaeng.
Efektifitas Penggunaan Aplikasi Quizlet Terhadap Penilaian Hasil Belajar Siswa Pada Pelajaran IPA Kelas VII Di SMPN 18 Makassar

Research on the Effectiveness of Using the Quizlet Application on Assessment of Student Learning Outcomes in Science Lessons on Substances and Their Changes in Class VII 2 Students at SMP Negeri 18 Makassar in the 2022/2023 Academic Year, a total of 30 students. Aims to determine the effectiveness of student learning outcomes by using the Quizlet application in the Natural Sciences subject matter of substances and their changes. This research is a type of quantitative research using experimental methods and the media used in research, namely the Quizlet application. The sample in the research was class VII 2 students. The instruments in this study provided pre- and post-tests to the sample. Data collection techniques used are observation, tests, and documentation. Data analysis techniques are descriptive statistical analysis and inferential statistical analysis. Comparison of learning outcomes categories, and comparisons of completeness levels has proven an increase in class VII 2 student learning outcomes of SMP Negeri 18 Makassar. It can be seen that student learning outcomes through descriptive statistical analysis before using Quizlet media, the average student score is still below the KKM score and after using media quizlet average student scores increased above the KKM score. The finding that the posttest result value is 84 is greater than the pretest result value which is 62.5. Furthermore, based on the results of inferential statistical analysis using the t test formula, it is known that the tcount obtained is 2.326 with a frequency df = 30-1 = 29, at a significant level = 0.05 or 5%, the ttable is 1.699. So tcount &gt; ttable or H0 is rejected and H1 is accepted. Based on these results it can be concluded that there is a significant increase in student learning outcomes. From the results of this study it can be interpreted that through learning media based on the Quizlet application in learning can have an influence on student learning outcomes in class VII 2 Science class SMP Negeri 18 Makassar and provide a positive response to the learning media.
Producing Diverse Rashomon Sets of Counterfactual Explanations with Niching Particle Swarm Optimization Algorithms

Counterfactual explanation is a popular eXplainable AI technique, that gives contrastive explanations to answer potential "what-if" questions about the workings of machine learning models. However, research into how explanations are understood by human beings has shown that an optimal explanation should be both selected and social, providing multiple varying explanations for the same event that allow a user to select specific explanations based on prior beliefs and cognitive biases. In order to provide such explanations, a Rashomon set of explanations can be created: a set of explanations utilising different features in the data. Current work to generate counterfactual explanations does not take this need into account, only focusing on producing a single optimal counterfactual.
Fast Evolutionary Neural Architecture Search by Contrastive Predictor with Linear Regions

Evolutionary neural architecture search (ENAS) has emerged as a promising approach to finding high-performance neural architectures. However, widespread application has been limited by the expensive computational costs due to the nature of evolutionary algorithms. In this study, we aim to significantly reduce the computational costs of ENAS by involving a training-free performance metric. Specifically, the network performance can be estimated by the training-free metric with only a single forward pass. However, training-free metrics have their own challenges, in particular, an insufficient correlation with ground-truth performance. We adopt a Graph Convolutional Network (GCN) based contrastive predictor which can leverage the low cost of the training-free performance metric yet improve the correlation between the estimated performance and the true performance of the candidate architectures. Combining a training-free metric - the number of linear regions with the GCN-based contrastive predictor and an active learning scheme, we propose Fast-ENAS which can achieve superior search efficiency and performance on the benchmark NAS-Bench-201 and DARTS search spaces. Furthermore, with a single GPU searching on the DARTS space, Fast-ENAS requires only 0.02 (29 minutes) and 0.026 (37 minutes) GPU days to achieve test error rates of 2.50% and 24.30% on CIFAR-10 and ImageNet respectively.
Dynamic Depth for Better Generalization in Continued Fraction Regression

A continued fraction expansion represents a real number as an expression obtained by iteratively extracting the largest whole number from its fractional part and inverting the remainder.
Adaptive Team Cooperative Co-Evolution for a Multi-Rover Distribution Problem

This paper deals with policy learning for a team of heterogeneous robotic agents when the whole team shares a single reward. We address the problem of providing an accurate estimation of the contribution of each agent in tasks where coordination between agents requires joint policy updates of two (or more) agents. This is typically the case when two agents must simultaneously modify their behaviors to perform a joint action that leads to a performance gain for the whole team. We propose a cooperative co-evolutionary algorithm extended with a multi-armed bandit algorithm that dynamically adjusts the number of agents that should update their policies simultaneously, aiming both for performance and learning speed. We use a realistic robotic multi-rover task where agents must physically distribute themselves on points of interest of different natures to complete the task. Results show that the algorithm is able to select the best group size for policy updates that reflects the task's coordination requirements. Surprisingly, we also reveal that coupling between agents' actions in a realistic setup can also emerge from interactions at the phenotypical level, hinting at subtle interactions during learning between the control parameter space and the behavioral space.