Miracle-dz/UniFAR-FTUs huggingface dataset:
Node Dependent Local Smoothing for Scalable Graph Learning

Recent works reveal that feature or label smoothing lies at the core of Graph Neural Networks (GNNs). Concretely, they show feature smoothing combined with simple linear regression achieves comparable performance with the carefully designed GNNs, and a simple MLP model with label smoothing of its prediction can outperform the vanilla GCN. Though an interesting finding, smoothing has not been well understood, especially regarding how to control the extent of smoothness. Intuitively, too small or too large smoothing iterations may cause under-smoothing or over-smoothing and can lead to sub-optimal performance. Moreover, the extent of smoothness is node-specific, depending on its degree and local structure. To this end, we propose a novel algorithm called node-dependent local smoothing (NDLS), which aims to control the smoothness of every node by setting a node-specific smoothing iteration. Specifically, NDLS computes influence scores based on the adjacency matrix and selects the iteration number by setting a threshold on the scores. Once selected, the iteration number can be applied to both feature smoothing and label smoothing. Experimental results demonstrate that NDLS enjoys high accuracy -- state-of-the-art performance on node classifications tasks, flexibility -- can be incorporated with any models, scalability and efficiency -- can support large scale graphs with fast training.
Toward a Theory of Justice for Artificial Intelligence

Abstract This essay explores the relationship between artificial intelligence and principles of distributive justice. Drawing upon the political philosophy of John Rawls, it holds that the basic structure of society should be understood as a composite of sociotechnical systems, and that the operation of these systems is increasingly shaped and influenced by AI. Consequently, egalitarian norms of justice apply to the technology when it is deployed in these contexts. These norms entail that the relevant AI systems must meet a certain standard of public justification, support citizens' rights, and promote substantively fair outcomes, something that requires particular attention to the impact they have on the worst-off members of society.
Time‐Resolved Evolution of the 3D Nanoporous Structure of Sintered Ag by X‐Ray Nanotomography: Role of the Interface with a Copper Substrate

The evolution of the nanoporous structure of cylindrical sintered silver samples during high temperature aging, ranging from 200 to 350 °C for 350 min, is studied through in situ computed X‐ray tomography. Investigations are carried out for two types of specimens: pure sintered silver and specimens containing a silver‐copper interface. It is shown that the overall pore evolution is driven by the evolution of very few large ones. The smaller pores, although being more numerous, do not really evolve before being absorbed by the few bigger ones. In pure silver, pore evolution is driven by diffusion (Ostwald ripening) but the presence of an interface promotes faster growth kinetics until the aging time reaches a threshold value, after which a deviation from Ostwald ripening occurs. The transition is a function of the aging temperature. This behavior is associated with the competition between elastic relaxation and surface energy minimization.
Evidential Softmax for Sparse Multimodal Distributions in Deep Generative Models

Many applications of generative models rely on the marginalization of their highdimensional output probability distributions. Normalization functions that yield sparse probability distributions can make exact marginalization more computationally tractable. However, sparse normalization functions usually require alternative loss functions for training since the log-likelihood is undefined for sparse probability distributions. Furthermore, many sparse normalization functions often collapse the multimodality of distributions. In this work, we present ev-softmax, a sparse normalization function that preserves the multimodality of probability distributions. We derive its properties, including its gradient in closed-form, and introduce a continuous family of approximations to ev-softmax that have full support and can be trained with probabilistic loss functions such as negative log-likelihood and Kullback-Leibler divergence. We evaluate our method on a variety of generative models, including variational autoencoders and auto-regressive architectures. Our method outperforms existing dense and sparse normalization techniques in distributional accuracy. We demonstrate that ev-softmax successfully reduces the dimensionality of probability distributions while maintaining multimodality.
JACC: An OpenACC Runtime Framework with Kernel-Level and Multi-GPU Parallelization

The rapid development in computing technology has paved the way for directive-based programming models towards a principal role in maintaining software portability of performance-critical applications. Efforts on such models involve a least engineering cost for enabling computational acceleration on multiple architectures while programmers are only required to add meta information upon sequential code. Optimizations for obtaining the best possible efficiency, however, are often challenging. The insertions of directives by the programmer can lead to side-effects that limit the available compiler optimization possible, which could result in performance degradation. This is exacerbated when targeting multi-GPU systems, as pragmas do not automatically adapt to such systems, and require expensive and time consuming code adjustment by programmers. This paper introduces JACC, an OpenACC runtime framework which enables the dynamic extension of OpenACC programs by serving as a transparent layer between the program and the compiler. We add a versatile code-translation method for multi-device utilization by which manually-optimized applications can be distributed automatically while keeping original code structure and parallelism. We show in some cases nearly linear scaling on the part of kernel execution with the NVIDIA V100 GPUs. While adaptively using multi-GPUs, the resulting performance improvements amortize the latency of GPU-to-GPU communications.
Overleaf Example

Many real-world problems require the combined application of multiple reasoning abilities—employing suitable abstractions, commonsense knowledge, and creative synthesis of problem-solving strategies. To help advance AI systems towards such capabilities, we propose a new reasoning challenge, namely Fermi Problems (FPs), which are questions whose answers can only be approximately estimated because their precise computation is either impractical or impossible. For example, “How much would the sea level rise if all ice in the world melted? ” FPs are commonly used in quizzes and interviews to bring out and evaluate the creative reasoning abilities of humans. To do the same for AI systems, we present two datasets: 1) A collection of 1k real-world FPs sourced from quizzes and olympiads; and 2) a bank of 10k synthetic FPs of intermediate complexity to serve as a sandbox for the harder real-world challenge. In addition to question-answer pairs, the datasets contain detailed solutions in the form of an executable program and supporting facts, helping in supervision and evaluation of intermediate steps. We demonstrate that even extensively fine-tuned large-scale language models perform poorly on these datasets, on average making estimates that are off by two orders of magnitude. Our contribution is thus the crystallization of several unsolved AI problems into a single, new challenge that we hope will spur further advances in building systems that can reason.
An improved multiobjective evolutionary algorithm based on decomposition and adaptive multi-reference points

Many real-world optimization problems such as engineering design are ﬁnally modeled as a multiobjective optimization problem (MOP) which must be solved to get a set of trade-oﬀs. Multiobjective evolutionary algorithm based on decomposition (MOEA/D) has been regarded as a very promising approach for solving MOPs, which oﬀers a general algorithmic framework of evolutionary multiobjective optimization. Recent studies have shown that MOEA/D with uniformly distributed weight vectors is well-suited to MOPs with regular Pareto optimal front, but its performance in terms of diversity deteriorates on MOPs with irregular Pareto optimal front such as highly nonlinear and convex. In this way, the solution set obtained by the algorithm can not provide more reasonable choices for decision makers. In order to eﬃciently overcome this shortcoming, in this paper, we propose an improved MOEA/D algorithm by virtue of the well-known Pascoletti-Seraﬁni scalarization method and a new strategy of multi-reference points. Speciﬁcally, this strategy consists of the setting and adaptation of reference points generated by the techniques of equidistant partition and projection. For performance assessment, the proposed algorithm is compared with existing four state-of-the-art multiobjective evolutionary algorithms on both benchmark test problems with various types of Pareto optimal fronts and two real-world MOPs including the hatch cover design and the rocket injector design in engineering optimization. Experimental results reveal that the proposed algorithm is better than that of the other compared algorithms in diversity.
Deep learning via message passing algorithms based on belief propagation

Message-passing algorithms based on the belief propagation (BP) equations constitute a well-known distributed computational scheme. They yield exact marginals on tree-like graphical models and have also proven to be effective in many problems defined on loopy graphs, from inference to optimization, from signal processing to clustering. The BP-based schemes are fundamentally different from stochastic gradient descent (SGD), on which the current success of deep networks is based. In this paper, we present and adapt to mini-batch training on GPUs a family of BP-based message-passing algorithms with a reinforcement term that biases distributions towards locally entropic solutions. These algorithms are capable of training multi-layer neural networks with performance comparable to SGD heuristics in a diverse set of experiments on natural datasets including multi-class image classification and continual learning, while being capable of yielding improved performances on sparse networks. Furthermore, they allow to make approximate Bayesian predictions that have higher accuracy than point-wise ones.
Approximation Methods for Mixed Models with Probit Link Functions

We study approximation methods for a large class of mixed models with a probit link function that includes mixed versions of the binomial model, the multinomial model, and generalized survival models. The class of models is special because the marginal likelihood can be expressed as Gaussian weighted integrals or as multivariate Gaussian cumulative density functions. The latter approach is unique to the probit link function models and has been proposed for parameter estimation in complex, mixed effects models. However, it has not been investigated in which scenarios either form is preferable. Our simulations and data example show that neither form is preferable in general and give guidance on when to approximate the cumulative density functions and when to approximate the Gaussian weighted integrals and, in the case of the latter, which general purpose method to use among a large list of methods.
Dream to Explore: Adaptive Simulations for Autonomous Systems

One's ability to learn a generative model of the world without supervision depends on the extent to which one can construct abstract knowledge representations that generalize across experiences. To this end, capturing an accurate statistical structure from observational data provides useful inductive biases that can be transferred to novel environments. Here, we tackle the problem of learning to control dynamical systems by applying Bayesian nonparametric methods, which is applied to solve visual servoing tasks. This is accomplished by first learning a state space representation, then inferring environmental dynamics and improving the policies through imagined future trajectories. Bayesian nonparametric models provide automatic model adaptation, which not only combats underfitting and overfitting, but also allows the model's unbounded dimension to be both flexible and computationally tractable. By employing Gaussian processes to discover latent world dynamics, we mitigate common data efficiency issues observed in reinforcement learning and avoid introducing explicit model bias by describing the system's dynamics. Our algorithm jointly learns a world model and policy by optimizing a variational lower bound of a log-likelihood with respect to the expected free energy minimization objective function. Finally, we compare the performance of our model with the state-of-the-art alternatives for continuous control tasks in simulated environments.
MIRA: Multihop Relation Prediction in Temporal Knowledge Graphs

In knowledge graph reasoning, we observe a trend to analyze temporal data evolving over time. The additional temporal dimension is attached to facts in a knowledge base resulting in quadruples between entities such as (Nintendo, released, Super Mario, Sep-13-1985) , where the relation between two entities is associated to a specific time interval or point in time. Multi-hop reasoning on inferred subgraphs connecting entities within a knowledge graph can be formulated as a reinforcement learning task where the agent sequentially performs inference upon the explored subgraph. The task in this work is to infer the predicate between a subject and an object entity, i.e., (subject, ? , object, time) , being valid at a certain timestamp or time interval. Given query entities, our agent starts to gather temporal relevant information about the neighborhood of the subject and object. The encoding of information about the explored graph structures is referred to as fingerprints . Subsequently, we use the two fingerprints as input to a Q-Network. Our agent decides sequentially which relational type needs to be explored next expanding the local subgraphs of the query entities in order to find promising paths between them. The evaluation shows that the proposed method not only yields results being in line with state-of-the-art embedding algorithms for temporal Knowledge Graphs (tKG), but we also gain information about the relevant structures between subjects and objects.
Does the Data Induce Capacity Control in Deep Learning?

We show that the input correlation matrix of typical classification datasets has an eigenspectrum where, after a sharp initial drop, a large number of small eigenvalues are distributed uniformly over an exponentially large range. This structure is mirrored in a network trained on this data: we show that the Hessian and the Fisher Information Matrix (FIM) have eigenvalues that are spread uniformly over exponentially large ranges. We call such eigenspectra"sloppy"because sets of weights corresponding to small eigenvalues can be changed by large magnitudes without affecting the loss. Networks trained on atypical datasets with non-sloppy inputs do not share these traits and deep networks trained on such datasets generalize poorly. Inspired by this, we study the hypothesis that sloppiness of inputs aids generalization in deep networks. We show that if the Hessian is sloppy, we can compute non-vacuous PAC-Bayes generalization bounds analytically. By exploiting our empirical observation that training predominantly takes place in the non-sloppy subspace of the FIM, we develop data-distribution dependent PAC-Bayes priors that lead to accurate generalization bounds using numerical optimization.
CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense

Alternatively inferring on the visual facts and commonsense is fundamental for an advanced visual question answering (VQA) system. This ability requires models to go beyond the literal understanding of commonsense. The system should not just treat objects as the entrance to query background knowledge, but fully ground commonsense to the visual world and imagine the possible relationships between objects, e.g., “fork, can lift, food”. To comprehensively evaluate such abilities, we propose a VQA benchmark, Compositional Reasoning on vIsion and Commonsense(CRIC), which introduces new types of questions about CRIC, and an evaluation metric integrating the correctness of answering and commonsense grounding. To collect such questions and rich additional annotations to support the metric, we also propose an automatic algorithm to generate question samples from the scene graph associated with the images and the relevant knowledge graph. We further analyze several representative types of VQA models on the CRIC dataset. Experimental results show that grounding the commonsense to the image region and joint reasoning on vision and commonsense are still challenging for current approaches. The dataset is available at https://cricvqa.github. io.
Insights into Oxidative Stress in Bone Tissue and Novel Challenges for Biomaterials

The presence of Reactive Oxygen Species (ROS) in bone can influence resident cells behaviour as well as the extra-cellular matrix composition and the tissue architecture. Aging, in addition to excessive overloads, unbalanced diet, smoking, predisposing genetic factors, lead to an increase of ROS and, if it is accompanied with an inappropriate production of scavengers, promotes the generation of oxidative stress that encourages bone catabolism. Furthermore, bone injuries can be triggered by numerous events such as road and sports accidents or tumour resection. Although bone tissue possesses a well-known repair and regeneration capacity, these mechanisms are inefficient in repairing large size defects and bone grafts are often necessary. ROS play a fundamental role in response after the implant introduction and can influence its success. This review provides insights on the mechanisms of oxidative stress generated by an implant in vivo and suitable ways for its modulation. The local delivery of active molecules, such as polyphenols, enhanced bone biomaterial integration evidencing that the management of the oxidative stress is a target for the effectiveness of an implant. Polyphenols have been widely used in medicine for cardiovascular, neurodegenerative, bone disorders and cancer, thanks to their antioxidant and anti-inflammatory properties. In addition, the perspective of new smart biomaterials and molecular medicine for the oxidative stress modulation in a programmable way, by the use of ROS responsive materials or by the targeting of selective molecular pathways involved in ROS generation, will be analysed and discussed critically.
Investigation of the interaction between emotion and working memory load using spatiotemporal pattern similarity analysis

Objective. Accumulating evidence has revealed that emotions can be provided with the modulatory effect on working memory (WM) and WM load is an important factor for the interaction between emotion and WM. However, it remains controversial whether emotions inhibit or facilitate WM and the interaction between cognitive task, processing load and emotional processing remains unclear. Approach. In this study, we used a change detection paradigm wherein memory items have four different load sizes and emotion videos to induce three emotions (negative, neutral, and positive). We performed an event-related spectral perturbation (ERSP) analysis and a spatiotemporal pattern similarity (STPS) analysis on the electroencephalography data. Main results. The ERSP results indicated that alpha and beta oscillations can reflect the difference among WM load sizes and also can reflect the difference among emotions under middle high WM load over posterior brain region in the maintenance stage. Moreover, the STPS results demonstrated a significant interaction between emotion and WM load size in the posterior region and found significantly higher similarity indexes for the negative emotion to the neutral emotion under the middle high WM load during WM maintenance. In addition, The STPS results also revealed that both positive emotion and negative emotion could interfere with the distinction of load sizes. Significance. The consistence of the behavioral, ERSP and STPS results suggested that when the memory load approaches the limit of WM capacity, negative emotion could facilitate WM through the top–down attention modulation promoting the most relevant information storage during WM maintenance.
Medication related osteonecrosis of the jaws (MRONJ): Factors related to recurrence after treatment with surgery and platelet rich plasma (PRP) placement

Background Medication-related osteonecrosis of the jaws (MRONJ) is a well-known complication associated with antiresorptive and antiangiogenic therapies. The purpose of this study was to analyse if there is any predictive factor of recurrence after local debridement plus platelet rich plasma (PRP) placement in MRONJ patients. Material and Methods Seventy MRONJ patients treated at the department of Oral and Maxillofacial Surgery in La Paz Hospital (Madrid, Spain) were included in this retrospective study. All of them were treated surgically by local debridement and PRP placement. The observation period was between January 2012 and January 2019. Information regarding use, type, administration, and duration of therapy with BP/denosumab was recorded. The follow-up period ranged from 2-52 months. A descriptive analysis, a bivariate and a multivariate study were performed. Results Most of the patients were women (82.9%) between 50-70 years old (64.3%), with a stage II disease (74.3%). The therapy lasted more than 12 months in 54.8% of them. Zoledronic acid was the main antiresorptive used (44.3%), followed by oral administered BPs (29 patients, 41.4%) and denosumab (10 patients, 14.3%). Osteoporosis (48.6%), breast cancer (30%) and multiple myeloma (11.4%) were the main diseases because the patients were taking antirresorptives. 13 patients (18.6%) experienced recurrence. We found that breast cancer patients (p>0.0001), smokers (p>0.016), and administration of zoledronic acid (p>0.0001) were related to recurrence. After performing the multivariate model, we found that the only factor related to recurrence was smoking habit (Wald 3.837, p=0.05, OR 6.12). Conclusions recurrence after local debridement plus PRP placement in our MRONJ series affected to 18.6% of patients. It seems to be more frequent in breast cancer patients, smokers, and after zoledronic acid administration. Smoking habit was the only independent factor related to recurrence in our series. Key words:Osteonecrosis of the jaw, recurrence, risk factor, bisphosphonates, zoledronic acid, denosumab, platelet rich plasma.