Miracle-dz/UniFAR-FTUs huggingface dataset:
Relational Learning with Variational Bayes

When comparing our model and training setups between the MNIST and Yale face experiments, their differences can all be attributed to the need for accommodating different data types, e.g., increasing nework size for larger face images (vs. smaller MNIST images), modifying learning objective to adapt real-valued Yale dataset (vs. binay valued MNIST dataset), and not for accomodating different relationships. Taking this into consideration, our results shows that we have solved both class of problems with the exact same principled method despite each class of problems represents a very different kind of relationships (the relationships in MNIST are geometric whereas the relationahips in Yale are high-level perception, e.g., sentiment, external environmental factors). We believe our results further demonstrates that the proposed VRL method is robust, stable, and generalizable to many different relationships.
Rethinking Adversarial Transferability from a Data Distribution Perspective

Adversarial transferability enables attackers to generate adversarial examples from the source model to attack the target model, which has raised security concerns about the deployment of DNNs in practice. In this paper, we rethink adversarial transferability from a data distribution perspective and further enhance transferability by score matching based optimization. We identify that some samples with injecting small Gaussian noise can fool different target models, and their adversarial examples under different source models have much stronger transferability. We hypothesize that these samples are in the low-density region of the ground truth distribution where models are not well trained. To improve the attack success rate of adversarial examples, we match the adversarial attacks with the directions which effectively decrease the ground truth density. We propose Intrinsic Adversarial Attack (IAA), which smooths the activation function and decreases the impact of the later layers of a given normal model, to increase the alignment of adversarial attack and the gradient of joint data distribution. We conduct comprehensive transferable attacks against multiple DNNs and show that our IAA can boost the transferability of the crafted attacks in all cases and go beyond state-of-the-art methods.
Optimizing Neural Networks with Gradient Lexicase Selection

One potential drawback of using aggregated performance measurement in machine learning is that models may learn to accept higher errors on some training cases as compromises for lower errors on others, with the lower errors actually being instances of overfitting. This can lead to both stagnation at local optima and poor generalization. Lexicase selection is an uncompromising method developed in evolutionary computation, which selects models on the basis of sequences of individual training case errors instead of using aggregated metrics such as loss and accuracy. In this paper, we investigate how lexicase selection, in its general form, can be integrated into the context of deep learning to enhance generalization. We propose Gradient Lexicase Selection, an optimization framework that combines gradient descent and lexicase selection in an evolutionary fashion. Our experimental results demonstrate that the proposed method improves the generalization performance of various widely-used deep neural network architectures across three image classification benchmarks. Additionally, qualitative analysis suggests that our method assists networks in learning more diverse representations. Our source code is available on GitHub: https://github.com/ld-ing/gradient-lexicase.
Improving Non-Autoregressive Translation Models Without Distillation

Transformer-based autoregressive (AR) machine translation models have achieved signiﬁcant performance improvements, nearing human-level accuracy on some languages. The AR framework translates one token at a time which can be time consuming, especially for long sequences. To accelerate inference, recent work has been exploring non-autoregressive (NAR) approaches that translate blocks of tokens in parallel. Despite signiﬁcant progress, leading NAR models still lag behind their AR counterparts, and only become competitive when trained with distillation. In this paper we investigate possible reasons behind this performance gap, namely, the indistinguishability of tokens, and mismatch between training and inference. We then propose the Conditional Masked Language Model with Correction (CMLMC) that addresses these problems. Empirically, we show that CMLMC achieves state-of-the-art NAR performance when trained on raw data without distillation, and approaches AR performance on multiple datasets. Code for this work is available here: https://github.com/layer6ai-labs/CMLMC .
Learning Causal Models from Conditional Moment Restrictions by Importance Weighting

We consider learning causal relationships under conditional moment restrictions. Unlike causal inference under unconditional moment restrictions, conditional moment restrictions pose serious challenges for causal inference, especially in high-dimensional settings. To address this issue, we propose a method that transforms conditional moment restrictions to unconditional moment restrictions through importance weighting, using a conditional density ratio estimator. Using this transformation, we successfully estimate nonparametric functions defined under conditional moment restrictions. Our proposed framework is general and can be applied to a wide range of methods, including neural networks. We analyze the estimation error, providing theoretical support for our proposed method. In experiments, we confirm the soundness of our proposed method.
Neural Methods for Logical Reasoning over Knowledge Graphs

Reasoning is a fundamental problem for computers and deeply studied in Artificial Intelligence. In this paper, we specifically focus on answering multi-hop logical queries on Knowledge Graphs (KGs). This is a complicated task because, in real-world scenarios, the graphs tend to be large and incomplete. Most previous works have been unable to create models that accept full First-Order Logical (FOL) queries, which include negative queries, and have only been able to process a limited set of query structures. Additionally, most methods present logic operators that can only perform the logical operation they are made for. We introduce a set of models that use Neural Networks to create one-point vector embeddings to answer the queries. The versatility of neural networks allows the framework to handle FOL queries with Conjunction ($\wedge$), Disjunction ($\vee$) and Negation ($\neg$) operators. We demonstrate experimentally the performance of our model through extensive experimentation on well-known benchmarking datasets. Besides having more versatile operators, the models achieve a 10\% relative increase over the best performing state of the art and more than 30\% over the original method based on single-point vector embeddings.
The determinants of University spinout formation and survival : the UK context of network, investment, and management team effects

UK universities attract increasing attention from policy-makers searching for regional solutions to economic development challenges. Consequently, university spinout companies have featured prominently in UK policy-making, as they embody a transfer of knowledge of the most complex and comprehensive character. However, whilst the positive contribution of spinout companies to regional economies is widely accepted, little is known of how to ensure high quality outcomes from such knowledge commercialisation activities. This thesis aims to improve the understanding of this problem by examining the elements that contribute to the success of academic spinouts in the UK context. It investigates dual meaning of success: spinout formation and survival, conceptualised here as embedded in university networks composed of multiple actors. It is set within a post-positivistic paradigm and employs an explanatory sequential mixed methods research design. The quantitative part identifies the elements contributing to the success of spinout companies using data on 870 spinout companies extracted from university websites and supplemented with financial, economic and educational databases; and leads to the qualitative part, explaining differential performance of spinout companies across UK regions with data collected through semi-structured interviews conducted at four illustrative university networks. It is found that the success of spinout companies depends on networks, investment, and management teams. However, the formation and survival of spinout companies differs across a number of elements: technology transfer offices, business incubators, other actors, and geography; suggesting bi-dimensional complexity across space and success measures. The variable spatial performance of university networks determining spinout company outcomes is explained by connectedness, filtration and time: successful spinout companies originate from university networks that have capability to build and exploit network capital. It is suggested that regional innovation systems require designs oriented towards these diverse spinout success outcomes, formation and survival-based, with strong local adaptations.
Impacts of a Parent-Implemented Language Intervention on Children's Language Development Within Home Visiting

The purpose of this study was to conduct a pilot randomized controlled trial to evaluate the efficacy of an intervention for enhancing the language development of young children enrolled in home visiting programs. The participants were 59 children (18–30 months old) enrolled in one of three types of home visiting programs. Children's language skills were assessed pre- and post-intervention with the Preschool Language Scale. To estimate the impacts of the intervention on children's language scores, we conducted a two-level hierarchical linear model. Results from this model provide promising evidence for the benefits of using systematic parent-implemented language interventions.
This paper is included in the Proceedings of the 2022 USENIX Annual Technical Conference.

Identifying a binary’s compiler configuration enables developers and analysts to locate potential security issues caused by optimization side-effects, identify binary clones, and build compatible binary patches. Existing work focuses on identifying compiler family, version and optimization level of a binary using semantic features and deep learning techniques. Unfortunately, in practice, bi-naries are an amalgamation of objects and functions that can be compiled at different optimization levels with a variety of individual, fine-grained, optimizations that may be applied depending on the structure of the code. Hence, rather than recovering high-level artifacts, i.e., compiler family, version, and optimization level, we explore the recovery of individual, fine-grained, optimization passes for each function in a binary. To do so, we develop an approach using specially crafted features alongside intuitive and understandable machine learning models. Our evaluation on 15 popular open-source repositories shows that our approach compares favorable with the state-of-the-art deep learning approach in compiler family, compiler version and optimization level identification. For fine-grained optimization passes, our evaluation on 149,814 functions from 552 binaries in four popular open-source repositories shows that our approach achieves an average F-1 score of 92.1% for all optimization passes and an average F-1 score of 89.8% for optimization passes that could have negative impacts on security. Moreover, our approach includes experimental support for dynamic feature extraction via binary emulation, and our results shows that such features offer promising potential in improving the accuracy of optimization pass identification.
Factors Affecting the Performance of Indonesian Government´s Internal Supervisory

: e role of the Indonesian government's internal oversight apparatus (APIP) in guiding and supervising government institutions obliges them to maintain and improve its performance. is paper investigates the eﬀect of management support, competence, objectivity, and integrity on the performance of the government's internal oversight apparatus. e tool used in this data processing is SPSS. e results show that support for management, objectivity and integrity have a signiﬁcant eﬀect on APIP performance, while competition has a non-signiﬁcant eﬀect on APIP performance. Governments should be involved in supporting APIP performance by creating policies that can improve the management of internal oversight oﬃcers.
Treatment of inguinal hernias in girls with laparoscopic Burnia technique versus open surgery

Introduction: At present, the debate is expanding on which technique would be better, comparing open surgery with the laparoscopic technique and the future cosmetic interest in female patients who require surgery, so the objective of the present study was to describe girls with inguinal hernia undergoing laparoscopic correction techniques (Burnia technique) versus conventional surgery for inguinal hernias. Methods: The present observational study was carried out at the Dr. Roberto Gilbert Elizalde Children's Hospital, in Guayaquil, Ecuador, from January 2021 to February 2022. With a nonprobabilistic sample, girls undergoing surgical treatment entered the study for inguinal hernia. Age, surgical technique, hospital stay, surgical time, and operative complications were recorded. Bivariate analysis is performed comparing the surgical techniques: open versus laparoscopic. Proportions are compared with Chi-square and means with Mann-Whitney U. Results: 89 patients were included in the study, 76 cases by open technique (AT) and 13 cases by Burnia technique (TB). The average age was 3.8 years, 5.05 days of hospitalization in unilateral TA and 2.3 days in unilateral TB (P=0.03), 7.2 days of hospitalization in bilateral TA, and 2.25 days in bilateral TB (P=0.026). Surgical time was 46.9 minutes in unilateral TA and 40.38 minutes in unilateral BT (P=0.232). Operative time of 64.7 min in bilateral TA and 42.5 min in bilateral TB (P=0.038). Complications 2 cases (2.6%) in TA and 1 case (7.7%) in TB (P=0.35). Conclusions: Using the laparoscopic technique decreased hospital stay times and surgical time when it is bilateral. From the aesthetic point of view, the scars were less visible in the minimally invasive surgical treatment.
Challenges when university researchers work collaboratively with industry in aviation safety: Why can’t we all just get along?

Increasing interest exists for collaboration between university researchers and industry partners, however there have been few attempts to analyse the nature of these collaborations (Calvert & Patel, 2003). This presents a void in terms of the challenges present, as well as the mutual benefits. In order to identify the most salient concerns, six researchers from Australian universities, who have worked with corporations on aviation research projects, and eight individuals from industry, who have worked with academic collaborators on aviation research projects, were sampled. Survey respondents were also asked what benefits were realized from the cross academic/industry partnerships. Findings revealed the most frequent challenges that each group face, as well as the primary benefits. Qualitative comments are provided to illustrate the three most common challenges and benefits. Results provide both valuable insights into the common predicaments when conducting applied aviation research and an indication as to which areas to focus on, in order to improve future collaborations. Further analyses are being conducted to identify challenges within other safety domains and concerns that may exist within international partnerships.
Contrôle stochastique appliqué à la ﬁnance

This PhD thesis considers the optimal trading problem from the stochastic control approach and consists of four parts. In the first part, we begin with the study of the impacts generated by volumes on the price process. We introduce a structural model in which price movements are due to not only the last trade’s volume but also to those of earlier trades, weakened by a decay factor. Considering a similar continuous version, we provide a condition ensuring the optimality of a strategy for the minimization of the execution cost in a mean-variance framework, and solve it numerically. In the second part, we propose a general model to optimize the way trading algorithms are used. Using an impulse control approach, we model the execution of a large order by a sequence (τi,δi,Ei)i, which is defined so that the i-th slice is executed in [τi,τi+δi] with parameter Ei. We characterize the value function as a viscosity solution of a system of PDE. We provide a numerical scheme and prove its convergence. Numerical illustrations are given for a real case. We deal with the problem of pricing an option on the book liquidation in presence of impact where the classical pricing by neutral risk measure fails. We begin with an abstract model generalized from the work of Bouchard- Eile-Touzi (2008), and then apply to compute the price of a VWAP guaranteed contract. We establish in the last part an equivalence result between stochastic target problems and standard optimal control. We derive the classical HJB equation from the PDE obtained in the stochastic target framework.
Acquisition of Voicing Alternations

"Morpho-phonological alternations are central to phonological theory, but little is known about how they are acquired. Acquiring alternations amounts to dealing with variation in a morpheme’s shape depending on its morphological context. It is generally assumed that children start with an initial stage of phonotactic learning, after which morphology is acquired, i.e. the analysis of words into meaningful units or morphemes. In Optimality Theory (Prince and Smolensky 1993), alternations occur when two or more output forms share a single input form, the lexical or underlying representation. Hence, alternating forms are characterised by a lack of faithfulness to the underlying form of a morpheme. It is not fully understood how children store lexical items and represent the relationship between their contextual variants. Moreover, it is unclear whether early lexical representations differ from those of adults. Here, the various models of morphological processing, such as ‘dual-route’ models (e.g. Baayen et al. 1997) or ‘whole word’ models (e.g. Bybee 1995) will not be addressed"
Acoustic Bloch Wave Propagation in a Periodic Waveguide

Abstract : The propagation of linear, dissipative acoustic Bloch waves in a periodic waveguide is investigated theoretically and experimentally. The waveguide under study is a rigid, fluid-filled, rectangular waveguide that is loaded with a periodic array of rigidly terminated rectangular side branches. Conditions for the existence of Bloch waves in the periodic waveguide are presented. Analytic expressions for the dispersion relation, the impedance function, and the Bloch wave function are derived. Measurements verify the predicted passband/stopband structure of the dispersion relation and show good agreement with the theory.
What caused the wage changes in tourism-related industries? A demand-side analysis based on an extended input-output model

ABSTRACT Previous studies that have explored the factors influencing wage changes in tourism have generally been confined to supply-side analysis, ignoring the impact of final demand on wage changes through the mechanism of inter-industry linkages. To this end, this study proposes a novel framework that combines input-output analysis with the logarithmic mean Divisia index method to examine wage levels and their changes in tourism-related industries and to clarify how final demand affects wage changes. The empirical results for China reveal significant wage differences among tourism-related industries, with accommodation and catering sectors consistently exhibiting a low-wage profile. In addition, the decomposition results show that the wage changes from 2002 to 2017 were mainly caused by the positive role of the total final demand effect and the negative role of the labour compensation coefficient effect. The final demand structure effect and Leontief structural effect are potential drivers of wage growth.
BRFP: An Efficient and Universal Sentence Embedding Learning Model Method Supporting Fused Syntax Combined with Graph Embedding Representation Algorithm

Due to the rapidly growing volume of data on the Internet, the methods of efficiently and accurately processing massive text information have been the focus of research. In natural language processing theory, sentence embedding representation is an important method. This paper proposes a new sentence embedding learning model called BRFP (Factorization Process with Bidirectional Restraints) that fuses syntactic information, uses matrix decomposition to learn syntactic information, and fuses and calculates with word vectors to obtain the embedded representation of sentences. In the experimental chapter, text similarity experiments are conducted to verify the rationality and effectiveness of the model and analyzed experimental results on Chinese and English texts with the current mainstream learning methods, and potential improvement directions are summarized. The experimental results on Chinese and English datasets, including STS, AFQMC, and LCQMC, show that the model proposed in this paper outperforms the CNN method in terms of accuracy and F1 value by 7.6% and 4.8. The comparison experiment with the word vector weighted model shows that when the sentence length is longer, or the corresponding syntactic structure is complex, the model’s advantages in this paper are more prominent than TF-IDF and SIF methods. Compared with the TF-IDF method, the effect improved by 14.4%. Compared with the SIF method, it has a maximum advantage of 7.9%, and the overall improvement in each comparative experimental task is between 4 and 6 percentage points. In the neural network model comparison experiment, the model in this paper compared the CNN, RNN, LSTM, ST, QT, and InferSent models, and the effect significantly improved on the 14’OnWN, 14’Tweet-news, and 15’Ans. -forum datasets. For example, in the 14’OnWN dataset, the BRFP method has a 10.9% improvement over the ST method. The 14’Tweet-news dataset has a 22.9% advantage over the LSTM method, and the 15’Ans. -forum dataset has a 24.07% improvement over the RNN method. The article also demonstrates the generality of the model, proving that the model proposed in this paper is also a universal learning framework.