Reviewerly/exHarmony huggingface dataset:
Pitfalls in Experiments with DNN4SE: An Analysis of the State of the Practice

Software engineering techniques are increasingly relying on deep learning approaches to support many software engineering tasks, from bug triaging to code generation. To assess the efficacy of such techniques researchers typically perform controlled experiments. Conducting these experiments, however, is particularly challenging given the complexity of the space of variables involved, from specialized and intricate architectures and algorithms to a large number of training hyper-parameters and choices of evolving datasets, all compounded by how rapidly the machine learning technology is advancing, and the inherent sources of randomness in the training process. In this work we conduct a mapping study, examining 194 experiments with techniques that rely on deep neural networks appearing in 55 papers published in premier software engineering venues to provide a characterization of the state-of-the-practice, pinpointing experiments common trends and pitfalls. Our study reveals that most of the experiments, including those that have received ACM artifact badges, have fundamental limitations that raise doubts about the reliability of their findings. More specifically, we find: weak analyses to determine that there is a true relationship between independent and dependent variables (87% of the experiments); limited control over the space of DNN relevant variables, which can render a relationship between dependent variables and treatments that may not be causal but rather correlational (100% of the experiments); and lack of specificity in terms of what are the DNN variables and their values utilized in the experiments (86% of the experiments) to define the treatments being applied, which makes it unclear whether the techniques designed are the ones being assessed, or how the sources of extraneous variation are controlled. We provide some practical recommendations to address these limitations.
Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD Detection, Calibration, and Accuracy

Deep ensembles are capable of achieving state-of-the-art results in classification and out-of-distribution (OOD) detection. However, their effectiveness is limited due to the homogeneity of learned patterns within ensembles. To overcome this issue, our study introduces Saliency Diversified Deep Ensemble (SDDE), a novel approach that promotes diversity among ensemble members by leveraging saliency maps. Through incorporating saliency map diversification, our method outperforms conventional ensemble techniques and improves calibration in multiple classification and OOD detection tasks. In particular, the proposed method achieves state-of-the-art OOD detection quality, calibration, and accuracy on multiple benchmarks, including CIFAR10/100 and large-scale ImageNet datasets.
A Generic Performance Model for Deep Learning in a Distributed Environment

Performance modelling of a deep learning application is essential to improve and quantify the efficiency of the model framework. However, existing performance models are mostly case-specific, with limited capability for the new deep learning frameworks/applications. In this paper, we propose a generic performance model of an application in a distributed environment with a generic expression of the application execution time that considers the influence of both intrinsic factors/operations (e.g. algorithmic parameters/internal operations) and extrinsic scaling factors (e.g. the number of processors, data chunks and batch size). We formulate it as a global optimization problem and solve it using regularization on a cost function and differential evolution algorithm to find the best-fit values of the constants in the generic expression to match the experimentally determined computation time. We have evaluated the proposed model on three deep learning frameworks (i.e., TensorFlow, MXnet, and Pytorch). The experimental results show that the proposed model can provide accurate performance predictions and interpretability. In addition, the proposed work can be applied to any distributed deep neural network without instrumenting the code and provides insight into the factors affecting performance and scalability.
Learning Global-aware Kernel for Image Harmonization

Image harmonization aims to solve the visual inconsistency problem in composited images by adaptively adjusting the foreground pixels with the background as references. Existing methods employ local color transformation or region matching between foreground and background, which neglects powerful proximity prior and independently distinguishes fore-/back-ground as a whole part for harmonization. As a result, they still show a limited performance across varied foreground objects and scenes. To address this issue, we propose a novel Global-aware Kernel Network (GKNet) to harmonize local regions with comprehensive consideration of long-distance background references. Specifically, GKNet includes two parts, \ie, harmony kernel prediction and harmony kernel modulation branches. The former includes a Long-distance Reference Extractor (LRE) to obtain long-distance context and Kernel Prediction Blocks (KPB) to predict multi-level harmony kernels by fusing global information with local features. To achieve this goal, a novel Selective Correlation Fusion (SCF) module is proposed to better select relevant long-distance background references for local harmonization. The latter employs the predicted kernels to harmonize foreground regions with both local and global awareness. Abundant experiments demonstrate the superiority of our method for image harmonization over state-of-the-art methods, \eg, achieving 39.53dB PSNR that surpasses the best counterpart by +0.78dB $\uparrow$; decreasing fMSE/MSE by 11.5\%$\downarrow$/6.7\%$\downarrow$ compared with the SoTA method. Code will be available at \href{https://github.com/XintianShen/GKNet}{here}.
Survival Probability of Unstable States in Coupled-Channels -- nonexponential decay of "threshold-cusp"

We investigate the survival probability of unstable states, the time-dependence of an initial state, in coupled channels. First, we extend the formulation of the survival probability from single channel to coupled channels (two channels). We derive an exact general expression of the two-channel survival probability using uniformization, a method which makes the coupled-channel S matrix single-valued, and the Mittag-Leffler expansion, i.e. a pole expansion. Second, we calculate the time dependence of the two-channel survival probability by employing the derived expression. It is the minimal distance between the pole and the physical region in the complex energy plane, not the imaginary part of the pole energy, which determines not only the energy spectrum of the Green's function but also the survival probability. The survival probability of the "threshold-cusp" caused by a pole on the unusual complex-energy Riemann sheet is shown to decay, not grow in time though the imaginary part of the pole energy is positive. We also show that the decay of the "threshold-cusp" is non-exponential. Thus, the "threshold-cusp" is shown to be a new type of unstable mode, which is found only in coupled channels.
Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantization

Existing vector quantization (VQ) based autoregressive models follow a two-stage generation paradigm that first learns a codebook to encode images as discrete codes, and then completes generation based on the learned codebook. However, they encode fixed-size image regions into fixed-length codes and ignore their naturally different information densities, which results in insufficiency in important regions and redundancy in unimportant ones, and finally degrades the generation quality and speed. Moreover, the fixed-length coding leads to an unnatural raster-scan autoregressive generation. To address the problem, we propose a novel two-stage framework: (1) Dynamic-Quantization VAE (DQ-VAE) which encodes image regions into variable-length codes based on their information densities for an accurate and compact code representation. (2) DQ-Transformer which thereby generates images autoregressively from coarse-grained (smooth regions with fewer codes) to fine-grained (details regions with more codes) by modeling the position and content of codes in each granularity alternately, through a novel stacked-transformer architecture and shared-content, non-shared position input layers designs. Comprehensive experiments on various generation tasks validate our superiorities in both effectiveness and efficiency. Code will be released at https://github.com/CrossmodalGroup/DynamicVectorQuantization.
What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability

In Natural Language Generation (NLG) tasks, for any input, multiple communicative goals are plausible, and any goal can be put into words, or produced, in multiple ways. We characterise the extent to which human production varies lexically, syntactically, and semantically across four NLG tasks, connecting human production variability to aleatoric or data uncertainty. We then inspect the space of output strings shaped by a generation system's predicted probability distribution and decoding algorithm to probe its uncertainty. For each test input, we measure the generator's calibration to human production variability. Following this instance-level approach, we analyse NLG models and decoding strategies, demonstrating that probing a generator with multiple samples and, when possible, multiple references, provides the level of detail necessary to gain understanding of a model's representation of uncertainty. Code available at https://github.com/dmg-illc/nlg-uncertainty-probes.
Inference-time Re-ranker Relevance Feedback for Neural Information Retrieval

Neural information retrieval often adopts a retrieve-and-rerank framework: a bi-encoder network first retrieves K (e.g., 100) candidates that are then re-ranked using a more powerful cross-encoder model to rank the better candidates higher. The re-ranker generally produces better candidate scores than the retriever, but is limited to seeing only the top K retrieved candidates, thus providing no improvements in retrieval performance as measured by Recall@K. In this work, we leverage the re-ranker to also improve retrieval by providing inference-time relevance feedback to the retriever. Concretely, we update the retriever's query representation for a test instance using a lightweight inference-time distillation of the re-ranker's prediction for that instance. The distillation loss is designed to bring the retriever's candidate scores closer to those of the re-ranker. A second retrieval step is then performed with the updated query vector. We empirically show that our approach, which can serve arbitrary retrieve-and-rerank pipelines, significantly improves retrieval recall in multiple domains, languages, and modalities.
Tester-Learners for Halfspaces: Universal Algorithms

We give the first tester-learner for halfspaces that succeeds universally over a wide class of structured distributions. Our universal tester-learner runs in fully polynomial time and has the following guarantee: the learner achieves error $O(\mathrm{opt}) + \epsilon$ on any labeled distribution that the tester accepts, and moreover, the tester accepts whenever the marginal is any distribution that satisfies a Poincar\'e inequality. In contrast to prior work on testable learning, our tester is not tailored to any single target distribution but rather succeeds for an entire target class of distributions. The class of Poincar\'e distributions includes all strongly log-concave distributions, and, assuming the Kannan--L\'{o}vasz--Simonovits (KLS) conjecture, includes all log-concave distributions. In the special case where the label noise is known to be Massart, our tester-learner achieves error $\mathrm{opt} + \epsilon$ while accepting all log-concave distributions unconditionally (without assuming KLS). Our tests rely on checking hypercontractivity of the unknown distribution using a sum-of-squares (SOS) program, and crucially make use of the fact that Poincar\'e distributions are certifiably hypercontractive in the SOS framework.
COMPARATIVE ANALYSIS OF FUTURE FOREIGN LANGUAGE TEACHERS’ INDEPENDENT LEARNING ORGANIZATION IN UKRAINE AND GREAT BRITAIN

The article provides a comparative analysis of the experience of organizing independent extracurricular work of future foreign language teachers in higher education institutions of Ukraine and Great Britain. The analysis of educational and methodological support of the disciplines of the cycle of professional training of foreign language teachers of domestic higher education institutions gives reasons to assert that independent extracurricular work has a significant share in relation to classroom educational activities of students. There is a tendency to increase the hours of independent extracurricular work.Independent extracurricular work of future foreign language teachers in domestic higher education institutions, unlike British ones, has only a practical focus and is aimed at the formation of foreign language competence of future teachers. It was determined that in the organization of independent extracurricular work, Ukrainian scientists and practitioners adhere to didactic (individualization activities, a combination of individual, paired and group forms of the specified work, creativity, visuality) and methodical (autonomy, communicativeness, interconnected learning of speech activities and aspects of language, interrelated learning of language and culture, the dominant role of exercises, the authenticity of educational materials) principles.It was found out that Great Britain has developed quite a powerful experience of organizing independent extracurricular work of future foreign language teachers. Some features in this field are common to both countries, but in higher education institutions of Great Britain there is a much greater amount of work related to the formation of student independence, capable of consciously and responsibly performing various tasks outside the classroom. Changing the authoritarian model of professional education, characteristic of Soviet times, to a person-oriented one involves the maximum individualization of independent extracurricular work of future foreign language teachers, the creation of conditions for self-development and self-improvement, meaningful and independent determination of one's own educational needs, one's capabilities, and individual educational goals. In this aspect, work in higher educational institutions of Ukraine has begun and needs further improvement. In the higher educational institutions of Great Britain, a modern model of higher education has already been created, designed to provide the student with systematized, solid and at the same time operational knowledge, capable of providing conditions that contribute to the realization of a necessary and urgent requirement: to teach the student to learn independently. These processes have a certain tradition and are actively implemented.
PPTTG Pemipil Jagung untuk Petani Jagung Bandar Klippa Kecamatan Percut Sei Tuan

Bandar Klippa Village is one of the villages in Percut Sei Tuan sub-district, Deli Serdang Regency, North Sumatra Province with an area of ​​1,828.4 Ha. In the village of Bandar Klippa, most of the agricultural land out of a total of 575 hectares is planted with corn with an average yield per hectare of 20 tons if there is sufficient rainfall. Generally, corn farmers immediately sell their crops to the factory after they have been harvested. The price of corn if it is sold directly to the factory is around 2800/kg, namely whole corn and the corn cob because the farmers do not yet have a corn sheller. If shelling is done, the price of corn increases to IDR 4,200/kg or almost 2 times that. This service is intended to help corn farmers get added value to their corn yields by providing a corn sheller machine designed by the proposing team and providing training in the use and maintenance of the sheller machine so that it can be used for a long time. After being crushed, the corncobs can also be sold to cattle breeders at a price of around Rp. 1000 to Rp. 2000 for each kilogram. The sheller machine can shell 350 kg of corn every hour
Enhancing Few-shot NER with Prompt Ordering based Data Augmentation

Recently, data augmentation (DA) methods have been proven to be effective for pre-trained language models (PLMs) in low-resource settings, including few-shot named entity recognition (NER). However, conventional NER DA methods are mostly aimed at sequence labeling models, i.e., token-level classification, and few are compatible with unified autoregressive generation frameworks, which can handle a wider range of NER tasks, such as nested NER. Furthermore, these generation frameworks have a strong assumption that the entities will appear in the target sequence with the same left-to-right order as the source sequence. In this paper, we claim that there is no need to keep this strict order, and more diversified but reasonable target entity sequences can be provided during the training stage as a novel DA method. Nevertheless, a naive mixture of augmented data can confuse the model since one source sequence will then be paired with different target sequences. Therefore, we propose a simple but effective Prompt Ordering based Data Augmentation (PODA) method to improve the training of unified autoregressive generation frameworks under few-shot NER scenarios. Experimental results on three public NER datasets and further analyses demonstrate the effectiveness of our approach.
PENERAPAN STRATEGI SELF REGULATED LEARNING PADA PEMBELAJARAN DARING

This study aims to find out how the implementation of the self-regulated learning strategy by class VIII.3 students of SMP Negeri 6 Ambon in PAK learning in online learning. The method used in this study is a qualitative method, the informants used in this study were 19 people. Data collection techniques using interviews. The results showed: self-regulated learning strategies, namely self-evaluation, organizing and changing, setting goals and planning, seeking information, noting important things, managing the environment, self-consequences, repeating and remembering, and seeking social support, have been applied by class students. VIII.3 but not maximal.
Improving Multimodal Joint Variational Autoencoders through Normalizing Flows and Correlation Analysis

We propose a new multimodal variational autoencoder that enables to generate from the joint distribution and conditionally to any number of complex modalities. The unimodal posteriors are conditioned on the Deep Canonical Correlation Analysis embeddings which preserve the shared information across modalities leading to more coherent cross-modal generations. Furthermore, we use Normalizing Flows to enrich the unimodal posteriors and achieve more diverse data generation. Finally, we propose to use a Product of Experts for inferring one modality from several others which makes the model scalable to any number of modalities. We demonstrate that our method improves likelihood estimates, diversity of the generations and in particular coherence metrics in the conditional generations on several datasets.
Complex Claim Verification with Evidence Retrieved in the Wild

Evidence retrieval is a core part of automatic fact-checking. Prior work makes simplifying assumptions in retrieval that depart from real-world use cases: either no access to evidence, access to evidence curated by a human fact-checker, or access to evidence available long after the claim has been made. In this work, we present the first fully automated pipeline to check real-world claims by retrieving raw evidence from the web. We restrict our retriever to only search documents available prior to the claim's making, modeling the realistic scenario where an emerging claim needs to be checked. Our pipeline includes five components: claim decomposition, raw document retrieval, fine-grained evidence retrieval, claim-focused summarization, and veracity judgment. We conduct experiments on complex political claims in the ClaimDecomp dataset and show that the aggregated evidence produced by our pipeline improves veracity judgments. Human evaluation finds the evidence summary produced by our system is reliable (it does not hallucinate information) and relevant to answering key questions about a claim, suggesting that it can assist fact-checkers even when it cannot surface a complete evidence set.
ANALISIS MUATAN HOTS (HIGER ORDER THINKING SKILLS) PADA BUKU TEKS SISWA PENDIDIKAN AGAMA KRISTEN DAN BUDI PEKERTI KELAS VII TERBITAN KEMENDIKBUD EDISI REVISI 2017HOTS (Higher Order Thinking Skill) is a thinking process of students at a higher cognitive le

HOTS (Higher Order Thinking Skill) is a thinking process of students at a higher cognitive level. The main goal of HOTS is to hone the thinking skills of students at a higher level, related to the ability to think critically, creatively, solve problems, and make decisions in complex situations. The student book on Christian Religious Education and Characteristics for SMP class VII published by the Ministry of Education and Culture, the 2017 revised edition is a book intended for students as a guide in learning activities. From the results of data analysis, it shows that HOTS content that appears most in textbooks is to analyze (C4) 46, 07% then create (C6) 33, 34% and the least evaluate (C5) 20, 58%. Therefore, the book is stated to follow the needs of junior high school students because it contains more cognitive abilities to analyze than other cognitive abilities. However, when viewed from the cognitive level, the book is not coherent because it produces (C6) more than evaluates (C5). &#x0D; Keywords: HOTS, Student Textbooks
Analisis Kesalahan Penggunaan Huruf Kapital dan Tanda Baca pada Karangan Deskripsi terhadap Keterampilan Menulis Siswa Kelas 5 SD

The purpose of this study was to find out errors in the use pf capital letters and punctuation in descriptive essays on the writing skills of 5th graders at Amanah Elementary School, Tangerang City. The type of research method used by the researcher is description. The reason the uses a qualitative approach is because the presentation of data and data analysis will be delivered in the form of a sentence description based on the results of tests, interview, and documentation studies. The results of the study found 23 students’ writings that had been analyzed there were 69 errors in the use of capital letters in the first letter of the beginning of the sentence, there are 19 errors in using capital letters in the first letter of people’s names, errors in the use of capital letters in the first letter of the name of the year, month and big day or holiday there are 7 errors, there are 16 errors in the use of capital letters in the first letter of the geographic names. And in the results of the analysis of errors in the use of punctuation, there are 57 errors in the use of punctuation point, 8 errors in the use punctuation marks, 6 errors in the use of hyphens.