Reviewerly/exHarmony huggingface dataset:
Efficient and Accurate Text Detection Combining Differentiable Binarization with Semantic Segmentation

Recently, segmentation-based methods have quickly become the mainstream in scene text detection, owing to their precise description of arbitrary-shape texts. However, the reduced inference speed hinders the practical application of segmentation-based methods. In this paper, we propose an efficient and accurate arbitrary-shaped text detector named ViT-Bilateral DBNet, which improves the efficiency of feature processing approach to achieve a good trade-off between accuracy and real-time performance. Specifically, we first combine Differentiable Binarization (DB) with real-time semantic segmentation BiSeNet V2 which is more suitable to process features for segmentation-based methods. Then three improvements are proposed to optimize the initial integrated network. ViT-Bilateral Network can strengthen the feature extracting capability of neural networks. Attention-driven Aggregation Layer (AAL) can adaptively fuse the details and the semantics achieved by ViT-Bilateral Network. Meanwhile, the auxiliary loss is added to make the training more sufficient. Compared with original DBNet, our method not only gains 1.17% (on IC15) and 1.34% (on CTW 1500) improvements, but also runs 1.38 times and 1.34 times faster. Notably, our detector surpasses the previous best record and maintains a high inference speed.
Linear Self-attention Approximation via Trainable Feedforward Kernel

In pursuit of faster computation, Efficient Transformers demonstrate an impressive variety of approaches -- models attaining sub-quadratic attention complexity can utilize a notion of sparsity or a low-rank approximation of inputs to reduce the number of attended keys; other ways to reduce complexity include locality-sensitive hashing, key pooling, additional memory to store information in compacted or hybridization with other architectures, such as CNN. Often based on a strong mathematical basis, kernelized approaches allow for the approximation of attention with linear complexity while retaining high accuracy. Therefore, in the present paper, we aim to expand the idea of trainable kernel methods to approximate the self-attention mechanism of the Transformer architecture.
Makeup Transfer Based on Generative Adversarial Network for Large Angle Spatial Misalignment

The makeup transfer task aims to transfer makeup styles from a reference makeup image to another non-makeup image. Previous methods achieved great progress with the same face angle, but failed to achieve good results when transferring between images with large spatial misalignment of face angles. In this paper, We propose a method for facial makeup transfer for large-angle spatial misalignment which based on generative adversarial networks. It first utilizes the Neural Head Reconstruction Module to process the reference image to obtain a new reference image. The new reference image maintains the original makeup style but has the same face angle as the source image, and it still has high definition and realism. Thus, the subsequent makeup transfer will be much easier. In addition, in order to ensure the authenticity and clarity of local details as much as possible, we also introduce the concept of local perception to transfer the makeup while keeping the color of the original character’s eyeballs, ears and neck unchanged. Besides, we can also realize controllable and partial makeup transfer. Experimental results show that our method achieves the state-of-the-art compared to existing methods.
Building Excitement for Reading and Building New Friendships

This article describes a collaborative Book Bistro between my pre-service teachers and Kim’s middle school students. First, we will describe how this strategy aligns with scholarship supporting adolescent readers and pre-service teachers. We will then share the logistics and results of our event as well as suggestions for others interested in using this strategy.
Using Choral Reading with Students with Exceptionalities to Improve Reading Fluency

Choral reading, a reading strategy that involves two or more students reading a passage in unison, where striving readers following the reading model provided by more proficient readers in a group, has long been recognized as a vehicle for improving fluency among students with exceptionalities (Kodal, &amp; Akyol, 2018). This instructional strategy, which is the topic of this paper, provides the interpersonal modeling needed by learners with exceptionalities, while providing a safe space for internalization of fluency skills to occur.
The Implication of Educandy Learning Media on Students’ Learning Outcomes in Japanese Learning

This research objective was to determine the effect of using Educandy learning media on Japanese students’ learning outcomes in a state vocational high school in Jakarta for academic year 2020/2021. This study applied experimental quantitative research method, with a purposive sample technique of 16 students. This research used one group Pre Test–Post Test design, and the data collected then analysed using descriptive and inferential statistics. In the preparation of the instrument, several tests consist of validity tests and reliability tests were conducted. The results showed that based on the Shapiro-Wilk Normality Test, the data was normally distributed due to the Sig value, and the data from Pre-test and Post-test showed that the students’ learning outcomes were increased after using Educandy. Moreover, the Levene Homogeneity Test results showed that the data is more than the significance level or the data was homogeneous, while based on the normality test and homogeneity test, the results showed that the value in the Pre-test and Post-test learning outcomes were lower than the level of significance. Based on the analysis, it can be concluded that H0 is rejected and H1 is accepted, which means that there is a significant difference in Pre-test and Post-test student learning outcomes so the Educandy learning media influences student learning outcomes in Japanese subjects.
Object-Centric Image Generation from Layouts

Despite recent impressive results on single-object and single-domain image generation, the generation of complex scenes with multiple objects remains challenging. In this paper, we start with the idea that a model must be able to understand individual objects and relationships between objects in order to generate complex scenes well. Our layout-to-image-generation method, which we call Object-Centric Generative Adversarial Network (or OC-GAN), relies on a novel Scene-Graph Similarity Module (SGSM). The SGSM learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity. We also propose changes to the conditioning mechanism of the generator that enhance its object instance-awareness. Apart from improving image quality, our contributions mitigate two failure modes in previous approaches: (1) spurious objects being generated without corresponding bounding boxes in the layout, and (2) overlapping bounding boxes in the layout leading to merged objects in images. Extensive quantitative evaluation and ablation studies demonstrate the impact of our contributions, with our model outperforming previous state-of-the-art approaches on both the COCO-Stuff and Visual Genome datasets. Finally, we address an important limitation of evaluation metrics used in previous works by introducing SceneFID -- an object-centric adaptation of the popular Fr{\'e}chet Inception Distance metric, that is better suited for multi-object images.
Membership Privacy for Machine Learning Models Through Knowledge Transfer

Large capacity machine learning (ML) models are prone to membership inference attacks (MIAs), which aim to infer whether the target sample is a member of the target model's training dataset. The serious privacy concerns due to the membership inference have motivated multiple defenses against MIAs, e.g., differential privacy and adversarial regularization. Unfortunately, these defenses produce ML models with unacceptably low classification performances. Our work proposes a new defense, called distillation for membership privacy (DMP), against MIAs that preserves the utility of the resulting models significantly better than prior defenses. DMP leverages knowledge distillation to train ML models with membership privacy. We provide a novel criterion to tune the data used for knowledge transfer in order to amplify the membership privacy of DMP. Our extensive evaluation shows that DMP provides significantly better tradeoffs between membership privacy and classification accuracies compared to state-of-the-art MIA defenses. For instance, DMP achieves ~100% accuracy improvement over adversarial regularization for DenseNet trained on CIFAR100, for similar membership privacy (measured using MIA risk): when the MIA risk is 53.7%, adversarially regularized DenseNet is 33.6% accurate, while DMP-trained DenseNet is 65.3% accurate.
Teaching the Old Dog New Tricks: Supervised Learning with Constraints

Adding constraint support in Machine Learning has the potential to address outstanding issues in data-driven AI systems, such as safety and fairness. Existing approaches typically apply constrained optimization techniques to ML training, enforce constraint satisfaction by adjusting the model design, or use constraints to correct the output. Here, we investigate a different, complementary, strategy based on "teaching" constraint satisfaction to a supervised ML method via the direct use of a state-of-the-art constraint solver: this enables taking advantage of decades of research on constrained optimization with limited effort. In practice, we use a decomposition scheme alternating master steps (in charge of enforcing the constraints) and learner steps (where any supervised ML model and training algorithm can be employed). The process leads to approximate constraint satisfaction in general, and convergence properties are difficult to establish; despite this fact, we found empirically that even a na\"ive setup of our approach performs well on ML tasks with fairness constraints, and on classical datasets with synthetic constraints.
Answering Complex Queries in Knowledge Graphs with Bidirectional Sequence Encoders

Representation learning for knowledge graphs (KGs) has focused on the problem of answering simple link prediction queries. In this work we address the more ambitious challenge of predicting the answers of conjunctive queries with multiple missing entities. We propose Bi-Directional Query Embedding (BIQE), a method that embeds conjunctive queries with models based on bi-directional attention mechanisms. Contrary to prior work, bidirectional self-attention can capture interactions among all the elements of a query graph. We introduce a new dataset for predicting the answer of conjunctive query and conduct experiments that show BIQE significantly outperforming state of the art baselines.
Classification by Attention: Scene Graph Classification with Prior Knowledge

A major challenge in scene graph classification is that the appearance of objects and relations can be significantly different from one image to another. Previous works have addressed this by relational reasoning over all objects in an image or incorporating prior knowledge into classification. Unlike previous works, we do not consider separate models for perception and prior knowledge. Instead, we take a multi-task learning approach, where we implement the classification as an attention layer. This allows for the prior knowledge to emerge and propagate within the perception model. By enforcing the model also to represent the prior, we achieve a strong inductive bias. We show that our model can accurately generate commonsense knowledge and that the iterative injection of this knowledge to scene representations leads to significantly higher classification performance. Additionally, our model can be fine-tuned on external knowledge given as triples. When combined with self-supervised learning and with 1% of annotated images only, this gives more than 3% improvement in object classification, 26% in scene graph classification, and 36% in predicate prediction accuracy.
Pemodelan status kelayakan saham di aplikasi ajaib menggunakan metode regresi logit ordinal

The development of the capital market cannot be separated from the contributions of investors and brokers. From 2018 until the end of September 2021, the number of investors has increased every year. PT. Ajaib Sekuritas Asia is one of the brokerage companies listed on the IDX. The purpose is for modelling, analyzing, and interpreting the stock eligibility status in the Ajaib using the ordinal logistic regression with logit function. The data used in this study is secondary data which includes: RoA (Return on Assets), GPM (Gross Profit Margin), OPM (Operating Profit Margin), NPM (Net Profit Margin), and Stock Status. All data is available in the Ajaib investment application, so Ajaib is the main data source in this research. The study result indicate that there are patterns between Return on Assets, Gross Profit Margin, Operating Profit Margin, and Net Profit Margin, to the stock eligibility status. The predictability of the model reached 49.22% and the pseudo R2is 19% with an error rate of 5%. Based on the results of the study, it can be concluded that there is a pattern of relationship between the issuer's financial indicators, which include Return on Assets (RoA), Gross Profit Margin (GPM), Operating Profit Margin (OPM), and Net Profit Margin (NPM), on the feasibility status of existing shares. In the Magic investment app. A positive relationship is shown by RoA, GPM, and NPM to the logit function of stock eligibility status, while a negative relationship is only shown by OPM to stock eligibility status.
Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning

Although recent multi-task learning methods have shown to be effective in improving the generalization of deep neural networks, they should be used with caution for safety-critical applications, such as clinical risk prediction. This is because even if they achieve improved task-average performance, they may still yield degraded performance on individual tasks, which may be critical (e.g., prediction of mortality risk). Existing asymmetric multi-task learning methods tackle this negative transfer problem by performing knowledge transfer from tasks with low loss to tasks with high loss. However, using loss as a measure of reliability is risky since it could be a result of overfitting. In the case of time-series prediction tasks, knowledge learned for one task (e.g., predicting the sepsis onset) at a specific timestep may be useful for learning another task (e.g., prediction of mortality) at a later timestep, but lack of loss at each timestep makes it difficult to measure the reliability at each timestep. To capture such dynamically changing asymmetric relationships between tasks in time-series data, we propose a novel temporal asymmetric multi-task learning model that performs knowledge transfer from certain tasks/timesteps to relevant uncertain tasks, based on feature-level uncertainty. We validate our model on multiple clinical risk prediction tasks against various deep learning models for time-series prediction, which our model significantly outperforms, without any sign of negative transfer. Further qualitative analysis of learned knowledge graphs by clinicians shows that they are helpful in analyzing the predictions of the model. Our final code is available at https://github.com/anhtuan5696/TPAMTL.
Metrics and continuity in reinforcement learning

In most practical applications of reinforcement learning, it is untenable to maintain direct estimates for individual states; in continuous-state systems, it is impossible. Instead, researchers often leverage state similarity (whether explicitly or implicitly) to build models that can generalize well from a limited set of samples. The notion of state similarity used, and the neighbourhoods and topologies they induce, is thus of crucial importance, as it will directly affect the performance of the algorithms. Indeed, a number of recent works introduce algorithms assuming the existence of "well-behaved" neighbourhoods, but leave the full specification of such topologies for future work. In this paper we introduce a unified formalism for defining these topologies through the lens of metrics. We establish a hierarchy amongst these metrics and demonstrate their theoretical implications on the Markov Decision Process specifying the reinforcement learning problem. We complement our theoretical results with empirical evaluations showcasing the differences between the metrics considered.
Joint Semantic Analysis with Document-Level Cross-Task Coherence Rewards

Coreference resolution and semantic role labeling are NLP tasks that capture different aspects of semantics, indicating respectively, which expressions refer to the same entity, and what semantic roles expressions serve in the sentence. However, they are often closely interdependent, and both generally necessitate natural language understanding. Do they form a coherent abstract representation of documents? We present a neural network architecture for joint coreference resolution and semantic role labeling for English, and train graph neural networks to model the 'coherence' of the combined shallow semantic graph. Using the resulting coherence score as a reward for our joint semantic analyzer, we use reinforcement learning to encourage global coherence over the document and between semantic annotations. This leads to improvements on both tasks in multiple datasets from different domains, and across a range of encoders of different expressivity, calling, we believe, for a more holistic approach to semantics in NLP.
Expected Value of Communication for Planning in Ad Hoc Teamwork

A desirable goal for autonomous agents is to be able to coordinate on the fly with previously unknown teammates. Known as "ad hoc teamwork", enabling such a capability has been receiving increasing attention in the research community. One of the central challenges in ad hoc teamwork is quickly recognizing the current plans of other agents and planning accordingly. In this paper, we focus on the scenario in which teammates can communicate with one another, but only at a cost. Thus, they must carefully balance plan recognition based on observations vs. that based on communication. This paper proposes a new metric for evaluating how similar are two policies that a teammate may be following - the Expected Divergence Point (EDP). We then present a novel planning algorithm for ad hoc teamwork, determining which query to ask and planning accordingly. We demonstrate the effectiveness of this algorithm in a range of increasingly general communication in ad hoc teamwork problems.
Weakly-supervised Temporal Action Localization by Uncertainty Modeling

Weakly-supervised temporal action localization aims to learn detecting temporal intervals of action classes with only video-level labels. To this end, it is crucial to separate frames of action classes from the background frames (i.e., frames not belonging to any action classes). In this paper, we present a new perspective on background frames where they are modeled as out-of-distribution samples regarding their inconsistency. Then, background frames can be detected by estimating the probability of each frame being out-of-distribution, known as uncertainty, but it is infeasible to directly learn uncertainty without frame-level labels. To realize the uncertainty learning in the weakly-supervised setting, we leverage the multiple instance learning formulation. Moreover, we further introduce a background entropy loss to better discriminate background frames by encouraging their in-distribution (action) probabilities to be uniformly distributed over all action classes. Experimental results show that our uncertainty modeling is effective at alleviating the interference of background frames and brings a large performance gain without bells and whistles. We demonstrate that our model significantly outperforms state-of-the-art methods on the benchmarks, THUMOS'14 and ActivityNet (1.2 & 1.3). Our code is available at https://github.com/Pilhyeon/WTAL-Uncertainty-Modeling.
Gradient-Based Localization and Spatial Attention for Confidence Measure in Fine-Grained Recognition using Deep Neural Networks

Both theoretical and practical problems in deep learning classification benefit from assessing uncertainty prediction. In addition, current state-of-the-art methods in this area are computationally expensive: for example,~\cite{loquercio2020general} is a general method for uncertainty estimation in deep learning that relies on Monte-Carlo sampling. We propose a new, efficient confidence measure later dubbed Over-MAP that utilizes a measure of overlap between structural attention mechanisms and segmentation methods. It does not rely on sampling or retraining. We show that the classification confidence increases with the degree of overlap. The associated confidence and identification tools are conceptually simple, efficient and of high practical interest as they allow for weeding out misleading examples in training data. Our measure is currently deployed in the real-world on widely used platforms to annotate large-scale data efficiently.
Creative Improvised Interaction with Generative Musical Systems

In this paper we survey the methods for control and cre-ative interaction with pre-trained generative models for au-dio and music. By using reduced (lossy) encoding and sym-bolization steps we are able to examine the level of information that is passing between the environment (the musician) and the agent (machine improvisation). We further use the concept of music information dynamics to find an optimal symbolization in terms of predictive information measure. Methods and strategies for generative models are surveyed in this paper and their implications for creative interaction with the machine are discussed in the musical improvisation framework.