Reviewerly/exHarmony huggingface dataset:
End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking

Machine learning systems perform well on pattern matching tasks, but their ability to perform algorithmic or logical reasoning is not well understood. One important reasoning capability is algorithmic extrapolation, in which models trained only on small/simple reasoning problems can synthesize complex strategies for large/complex problems at test time. Algorithmic extrapolation can be achieved through recurrent systems, which can be iterated many times to solve difficult reasoning problems. We observe that this approach fails to scale to highly complex problems because behavior degenerates when many iterations are applied -- an issue we refer to as "overthinking." We propose a recall architecture that keeps an explicit copy of the problem instance in memory so that it cannot be forgotten. We also employ a progressive training routine that prevents the model from learning behaviors that are specific to iteration number and instead pushes it to learn behaviors that can be repeated indefinitely. These innovations prevent the overthinking problem, and enable recurrent systems to solve extremely hard extrapolation tasks.
Consideration of the output series generated by hysteresis reservoir computing

Reservoir computing is a machine learning model that is widely used for time series tasks due to its advantages of low cost and fast learning. However, conventional reservoir computing often do not achieve the memory capacity and nonlinearity required for the task. To solve this problem, we proposed hysteresis reservoir computing which conventional reservoir neurons are replaced by hysteresis neurons. The model generates various output sequences by changing their parameters. In addition, it has the potential to memorize time series because it presents complex periodic solutions. In this paper, we confirm the dynamics generated by changing the hysteresis reservoir computing parameters. The experimental results show that changing the parameters improves the learning ability and can represent specific series of data. This indicates an important dynamics in terms of memory capacity and the ability to represent.
Addressing Challenging Problems Using Optimized Deep Learning Classification Algorithms on the MNIST Dataset

Using the MNIST dataset, we have addressed three problems that utilized deep learning and special algorithms to solve. The first problem (Problem 0) was to achieve MNIST classification accuracy beyond 99%. The second (Problem 1) addressed maximizing classification accuracy with minimum number of training samples. The third (Problem 2) addressed splitting MNIST digits vertically, shuffle them around, then rematch them with highest accuracy. We solved Problem 0 using an optimized deep learning model, achieving accuracy in the range [99.10, 99.50]. We solved Problem 1 through utilizing random sampling, up-sampling, data augmentation, and accumulative training. We were able to reach accuracy ≥ 90% using only 80–110 images, and ≥ 95% using only 140–210 images. For Problem 2, we trained three classifiers: left-side digit (95% acc., 0–9 classes), right-side digit (94% acc., 0–9 classes), and matching fusion (97% acc., pair match vs. mismatch). Left and right models classify the image part to corresponding digit. Then for each digit, brute-force was executed against identified halves. At each brute-force combination, the input is passed to the matching network to check whether the fused pair match. The combination with the highest match probability was selected. Accordingly, we were able to achieve a maximum matching accuracy of ~ 48% using up-sampled image size and narrowed fusion scope input. The solutions presented in this work can be further improved and extended beyond the MNIST dataset.
DA-FDFtNet: Dual Attention Fake Detection Fine-tuning Network to Detect Various AI-Generated Fake Images

Due to the advancement of Generative Adversarial Networks (GAN), Autoencoders, and other AI technologies, it has been much easier to create fake images such as "Deepfakes". More recent research has introduced few-shot learning, which uses a small amount of training data to produce fake images and videos more effectively. Therefore, the ease of generating manipulated images and the difficulty of distinguishing those images can cause a serious threat to our society, such as propagating fake information. However, detecting realistic fake images generated by the latest AI technology is challenging due to the reasons mentioned above. In this work, we propose Dual Attention Fake Detection Fine-tuning Network (DA-FDFtNet) to detect the manipulated fake face images from the real face data. Our DA-FDFtNet integrates the pre-trained model with Fine-Tune Transformer, MBblockV3, and a channel attention module to improve the performance and robustness across different types of fake images. In particular, Fine-Tune Transformer consists of multiple numbers of an image-based self-attention module and a down-sampling layer. The channel attention module is also connected with the pre-trained model to capture the fake images feature space. We experiment with our DA-FDFtNet with the FaceForensics++ dataset and various GAN-generated datasets, and we show that our approach outperforms the previous baseline models.
Reducing Land Occupancy Using Solar PV Artefact: A Study

In today’s modern times, the world is shifting to renewable energy sources for catering the increasing energy demand. Solar PV plays a very important role in renewable energy due to the rapidly reducing cost of energy generation. Solar PV can be used as an alternative to counter the increasing electricity bills. Solar artefact can be used as a very effective solar PV source because of the numerous advantages it offers, like effective land utilization, reliability, aesthetically pleasing and artistic structure. Also, installation of solar artefacts can be beneficial to raise awareness about solar energy among people. Solar PV power plants are installed at different locations like rooftops, ground, water, buildings. However, in a ground-mounted solar PV power plant constructed in conventional style, huge land area is occupied, which becomes unusable for any other purpose. Conventional-style solar PV installations have certain demerits of more land occupancy and less promotion among public due to less aesthetically pleasing structure. In this paper, the authors have tried to propose a sustainable and smart solution in the form of a research study of solar artefact design compared to a conventional-style ground-mounted solar PV power plant.
Textured Mesh Quality Assessment: Large-Scale Dataset and Deep Learning-based Quality Metric

Over the past decade, 3D graphics have become highly detailed to mimic the real world, exploding their size and complexity. Certain applications and device constraints necessitate their simplification and/or lossy compression, which can degrade their visual quality. Thus, to ensure the best Quality of Experience (QoE), it is important to evaluate the visual quality to accurately drive the compression and find the right compromise between visual quality and data size. In this work, we focus on subjective and objective quality assessment of textured 3D meshes. We first establish a large-scale dataset, which includes 55 source models quantitatively characterized in terms of geometric, color, and semantic complexity, and corrupted by combinations of 5 types of compression-based distortions applied on the geometry, texture mapping and texture image of the meshes. This dataset contains over 343k distorted stimuli. We propose an approach to select a challenging subset of 3000 stimuli for which we collected 148929 quality judgments from over 4500 participants in a large-scale crowdsourced subjective experiment. Leveraging our subject-rated dataset, a learning-based quality metric for 3D graphics was proposed. Our metric demonstrates state-of-the-art results on our dataset of textured meshes and on a dataset of distorted meshes with vertex colors. Finally, we present an application of our metric and dataset to explore the influence of distortion interactions and content characteristics on the perceived quality of compressed textured meshes.
Results of the Intervention Measures to Improve Trustworthiness of Business Students: The Progressing, the Neutralizing and the Declining

Not all-academic interventions yield the same effect to student recipient.Nevertheless, business educators continue to provide intervention program without examining its effect on students.This study tries to measure the effectiveness of an intervention measure among third-year students to improve their level of trustworthiness indicated in the result of the 16 Personality Factor Test (PFT).The intervention measure was embedded in their Human Behavior in Organization course, enriching the subject with cases and activities on trust as an important element in business organization, at the same time, orienting the faculty on the importance of personality on student's future career.The study reveals that there is a significant improvement on the trust among students, those enrolled in the second term, second semester have a higher posttest performance than other periods.Due to student' different learning and faculty teaching style result significantly varies.There are three clusters of students those who respond positively in the intervention measure, those that remained neutral and those that respond negatively.Students in the first cluster, the progressing, grasp the importance of trustworthiness in business.The second cluster, the neutrals, demonstrates trait optimism; these students try to maintain their status quo.The third cluster, the decliners, low self-efficacy attributes to the negative response.
Trustworthy Medical Segmentation with Uncertainty Estimation

Deep Learning (DL) holds great promise in reshaping the healthcare systems given its precision, efficiency, and objectivity. However, the brittleness of DL models to noisy and out-of-distribution inputs is ailing their deployment in the clinic. Most systems produce point estimates without further information about model uncertainty or confidence. This paper introduces a new Bayesian deep learning framework for uncertainty quantification in segmentation neural networks, specifically encoder-decoder architectures. The proposed framework uses the first-order Taylor series approximation to propagate and learn the first two moments (mean and covariance) of the distribution of the model parameters given the training data by maximizing the evidence lower bound. The output consists of two maps: the segmented image and the uncertainty map of the segmentation. The uncertainty in the segmentation decisions is captured by the covariance matrix of the predictive distribution. We evaluate the proposed framework on medical image segmentation data from Magnetic Resonances Imaging and Computed Tomography scans. Our experiments on multiple benchmark datasets demonstrate that the proposed framework is more robust to noise and adversarial attacks as compared to state-of-the-art segmentation models. Moreover, the uncertainty map of the proposed framework associates low confidence (or equivalently high uncertainty) to patches in the test input images that are corrupted with noise, artifacts or adversarial attacks. Thus, the model can self-assess its segmentation decisions when it makes an erroneous prediction or misses part of the segmentation structures, e.g., tumor, by presenting higher values in the uncertainty map.
Re-interpreting Rules Interpretability

Abstract Trustworthy machine learning requires a high level of interpretability of machine learning models , yet many models are inherently black-boxes. Training interpretable models instead — or using them to mimic the black-box model — seems like a viable solution. In practice, however , these interpretable models are still unintelligible due to their size and complexity. In this paper, we present an approach to explain the logic of large interpretable models that can be represented as sets of logical rules by a simple, and thus intelligible, descriptive model. The coarseness of this descriptive model and its fidelity to the original model can be controlled, so that a user can understand the original model in varying levels of depth. We showcase and discuss this approach on three real-world problems from healthcare, material science, and finance.
Exploiting all samples in low-resource sentence classification: early  stopping and initialization parameters

In low resource settings, deep neural models have often shown lower performance due to overfitting. The primary method to solve the overfitting problem is to generalize model parameters. To this end, many researchers have depended on large external resources with various manipulation techniques. In this study, we discuss how to exploit all available samples in low resource settings, without external datasets and model manipulation. This study focuses on natural language processing task. We propose a simple algorithm to find out good initialization parameters that improve robustness to a small sample set. We apply early stopping techniques that enable the use of all samples for training. Finally, the proposed learning strategy is to train all samples with the good initialization parameters and stop the model with the early stopping techniques. Extensive experiments are conducted on seven public sentence classification datasets, and the results demonstrate that the proposed learning strategy achieves better performance than several state-of-the-art works across the seven datasets.
Automated 3D Scanning Device for the Production of Forearm Prostheses and Orthoses

Commercial prostheses and orthoses are expensive and have a slow fabrication process that causes patient discomfort. This paper describes the development of an automated 3D scanning system that can reduce the time and production cost of 3D printed forearm prostheses and orthoses. The system is composed of a rotational axis and 3D scanner support, enabling the precise scanning of body parts with a shorter acquisition time and less image processing compared to existing methods. Measurements were performed to analyze the accuracy of the device. The results showed that the system obtains accurate molds of the forearm, with Technical Error of Measurement between 0.12 and 0.25%, threefold smaller measurement error than the maximum accepted value. The proposed method offers a higher number of personalization possibilities at a lower production cost, due to the use of 3D printing techniques, thereby promoting greater social inclusion.
Bayesian Nonparametric View to Spawning

In tracking multiple objects, it is often assumed that each observation (measurement) is originated from one and only one object. However, we may encounter a situation that each measurement may or may not be associated with multiple objects at each time step --spawning. Therefore, the association of each measurement to multiple objects is a crucial task to perform in order to track multiple objects with birth and death. In this paper, we introduce a novel Bayesian nonparametric approach that models a scenario where each observation may be drawn from an unknown number of objects for which it provides a tractable Markov chain Monte Carlo (MCMC) approach to sample from the posterior distribution. The number of objects at each time step, itself, is also assumed to be unknown. We, then, show through experiments the advantage of nonparametric modeling to scenarios with spawning events. Our experiment results also demonstrate the advantages of our framework over the existing methods.
Towards a Secure and Reliable Federated Learning using Blockchain

Federated learning (FL) is a distributed machine learning (ML) technique that enables collaborative training in which devices perform learning using a local dataset while preserving their privacy. This technique ensures privacy, communication efficiency, and resource conservation. Despite these advantages, FL still suffers from several challenges related to reliability (i.e., unreliable participating devices in training), tractability (i.e., a large number of trained models), and anonymity. To address these issues, we propose a secure and trustworthy blockchain framework (SRB-FL) tailored to FL, which uses blockchain features to enable collaborative model training in a fully distributed and trustworthy manner. In particular, we design a secure FL based on the blockchain sharding that ensures data reliability, scalability, and trustworthiness. In addition, we introduce an incentive mechanism to improve the reliability of FL devices using subjective multi-weight logic. The results show that our proposed SRB-FL framework is efficient and scalable, making it a promising and suitable solution for federated learning.
Multi-Label Classification in Patient-Doctor Dialogues With the RoBERTa-WWM-ext + CNN (Robustly Optimized Bidirectional Encoder Representations From Transformers Pretraining Approach With Whole Word Masking Extended Combining a Convolutional Neural Network) Model: Named Entity Study (Preprint)

<sec> <title>BACKGROUND</title> With the prevalence of online consultation, many patient-doctor dialogues have accumulated, which, in an authentic language environment, are of significant value to the research and development of intelligent question answering and automated triage in recent natural language processing studies. </sec> <sec> <title>OBJECTIVE</title> The purpose of this study was to design a front-end task module for the network inquiry of intelligent medical services. Through the study of automatic labeling of real doctor-patient dialogue text on the internet, a method of identifying the negative and positive entities of dialogues with higher accuracy has been explored. </sec> <sec> <title>METHODS</title> The data set used for this study was from the Spring Rain Doctor internet online consultation, which was downloaded from the official data set of Alibaba Tianchi Lab. We proposed a composite abutting joint model, which was able to automatically classify the types of clinical finding entities into the following 4 attributes: positive, negative, other, and empty. We adapted a downstream architecture in Chinese Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach (RoBERTa) with whole word masking (WWM) extended (RoBERTa-WWM-ext) combining a text convolutional neural network (CNN). We used RoBERTa-WWM-ext to express sentence semantics as a text vector and then extracted the local features of the sentence through the CNN, which was our new fusion model. To verify its knowledge learning ability, we chose Enhanced Representation through Knowledge Integration (ERNIE), original Bidirectional Encoder Representations from Transformers (BERT), and Chinese BERT with WWM to perform the same task, and then compared the results. Precision, recall, and macro-F1 were used to evaluate the performance of the methods. </sec> <sec> <title>RESULTS</title> We found that the ERNIE model, which was trained with a large Chinese corpus, had a total score (macro-F1) of 65.78290014, while BERT and BERT-WWM had scores of 53.18247117 and 69.2795315, respectively. Our composite abutting joint model (RoBERTa-WWM-ext + CNN) had a macro-F1 value of 70.55936311, showing that our model outperformed the other models in the task. </sec> <sec> <title>CONCLUSIONS</title> The accuracy of the original model can be greatly improved by giving priority to WWM and replacing the word-based mask with unit to classify and label medical entities. Better results can be obtained by effectively optimizing the downstream tasks of the model and the integration of multiple models later on. The study findings contribute to the translation of online consultation information into machine-readable information. </sec>
SMGRL: Scalable Multi-resolution Graph Representation Learning

Graph convolutional networks (GCNs) allow us to learn topologically-aware node embeddings, which can be useful for classification or link prediction. However, they are unable to capture long-range dependencies between nodes without adding additional layers -- which in turn leads to over-smoothing and increased time and space complexity. Further, the complex dependencies between nodes make mini-batching challenging, limiting their applicability to large graphs. We propose a Scalable Multi-resolution Graph Representation Learning (SMGRL) framework that enables us to learn multi-resolution node embeddings efficiently. Our framework is model-agnostic and can be applied to any existing GCN model. We dramatically reduce training costs by training only on a reduced-dimension coarsening of the original graph, then exploit self-similarity to apply the resulting algorithm at multiple resolutions. The resulting multi-resolution embeddings can be aggregated to yield high-quality node embeddings that capture both long- and short-range dependencies. Our experiments show that this leads to improved classification accuracy, without incurring high computational costs.
Some bijections for lattice paths

We present three bijections, the first between little Schr\"{o}der paths and a class of growth-constrained integer sequences, the second between lattice paths consisting of steps with nonnegative slope and another class of growth-constrained sequences, the third between a class of lattice paths with steps $(1,1)$ and $(1,-j\,),\ j\ge 1$, and a class with steps $(k,1),\ k\ge 1$, and $(1,-1)$.
Causal Domain Adaptation with Copula Entropy based Conditional Independence Test

Domain Adaptation (DA) is a typical problem in machine learning that aims to transfer the model trained on source domain to target domain with different distribution. Causal DA is a special case of DA that solves the problem from the view of causality. It embeds the probabilistic relationships in multiple domains in a larger causal structure network of a system and tries to find the causal source (or intervention) on the system as the reason of distribution drifts of the system states across domains. In this sense, causal DA is transformed as a causal discovery problem that finds invariant representation across domains through the conditional independence between the state variables and observable state of the system given interventions. Testing conditional independence is the corner stone of causal discovery. Recently, a copula entropy based conditional independence test was proposed with a rigorous theory and a non-parametric estimation method. In this paper, we first present a mathemetical model for causal DA problem and then propose a method for causal DA that finds the invariant representation across domains with the copula entropy based conditional independence test. The effectiveness of the method is verified on two simulated data. The power of the proposed method is then demonstrated on two real-world data: adult census income data and gait characteristics data.
CoursePathVis: Course path visualization using flexible grouping and funnel-augmented Sankey diagram

We present CoursePathVis, a visual analytics tool for exploring and analyzing students' progress through a college curriculum using a Sankey diagram.Focusing on four student cohorts in a department, we group students in multiple ways (by their AP courses, term courses, and a user-specified funnel course) to comprehensively understand the data.CoursePathVis helps us identify patterns or outliers that affect student success with these flexible grouping techniques and the funnel-augmented Sankey diagram.Three stakeholders from the same department formulate design requirements and provide an ad-hoc evaluation.