Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning to Improve Representations by Communicating About Perspectives

Effective latent representations need to capture abstract features of the external world. We hypothesise that the necessity for a group of agents to reconcile their subjective interpretations of a shared environment state is an essential factor in-ﬂuencing this property. To test this hypothesis, we propose an architecture where individual agents in a population receive different observations of the same underlying state and learn latent representations that they communicate to each other. We highlight a fundamental link between emergent communication and representation learning: the role of language as a cognitive tool and the opportunities conferred by subjectivity, an inherent property of most multi-agent systems. We present a minimal architecture comprised of a population of autoencoders, where we deﬁne loss functions, capturing different aspects of effective communication, and examine their effect on the learned representations. We show that our proposed architecture allows the emergence of aligned representations. The subjectivity introduced by presenting agents with distinct perspectives of the environment state contributes to learning abstract representations that outperform those learned by both a single au-toencoder and a population of autoencoders, presented with identical perspectives. Altogether, our results demonstrate how communication from subjective perspectives can lead to the acquisition of more abstract representations in multi-agent systems, opening promising perspectives for future research at the intersection of representation learning and emergent communication.
Solar cell patent classification method based on keyword extraction and deep neural network

With the growing impact of ESG on businesses, research related to renewable energy is receiving great attention. Solar cells are one of them, and accordingly, it can be said that the research value of solar cell patent analysis is very high. Patent documents have high research value. Being able to accurately analyze and classify patent documents can reveal several important technical relationships. It can also describe the business trends in that technology. And when it comes to investment, new industrial solutions will also be inspired and proposed to make important decisions. Therefore, we must carefully analyze patent documents and utilize the value of patents. To solve the solar cell patent classification problem, we propose a keyword extraction method and a deep neural network-based solar cell patent classification method. First, solar cell patents are analyzed for pretreatment. It then uses the KeyBERT algorithm to extract keywords and key phrases from the patent abstract to construct a lexical dictionary. We then build a solar cell patent classification model according to the deep neural network. Finally, we use a deep neural network-based solar cell patent classification model to classify power patents, and the training accuracy is greater than 95%. Also, the validation accuracy is about 87.5%. It can be seen that the deep neural network method can not only realize the classification of complex and difficult solar cell patents, but also have a good classification effect.
BERT-Beta: A Proactive Probabilistic Approach to Text Moderation

Text moderation for user generated content, which helps to promote healthy interaction among users, has been widely studied and many machine learning models have been proposed. In this work, we explore an alternative perspective by augmenting reactive reviews with proactive forecasting. Specifically, we propose a new concept text toxicity propensity to characterize the extent to which a text tends to attract toxic comments. Beta regression is then introduced to do the probabilistic modeling, which is demonstrated to function well in comprehensive experiments. We also propose an explanation method to communicate the model decision clearly. Both propensity scoring and interpretation benefit text moderation in a novel manner. Finally, the proposed scaling mechanism for the linear model offers useful insights beyond this work.
TVRecap: A Dataset for Generating Stories with Character Descriptions

We introduce TVR ECAP , a story generation dataset that requires generating detailed TV show episode recaps from a brief summary and a set of documents describing the characters involved. Unlike other story generation datasets, TVR ECAP contains stories that are authored by professional screenwriters and that feature complex interactions among multiple characters. Generating stories in TVR ECAP requires drawing relevant information from the lengthy provided documents about characters based on the brief summary. In addition, by swapping the input and output, TVR ECAP can serve as a challenging testbed for abstractive summarization. We create TVR ECAP from fan-contributed websites, which allows us to collect 26k episode recaps with 1868.7 tokens on average. Empirically, we take a hierarchical story generation approach and ﬁnd that the neural model that uses oracle content se-lectors for character descriptions demonstrates the best performance on automatic metrics, showing the potential of our dataset to inspire future research on story generation with constraints. Qualitative analysis shows that the best-performing model sometimes generates content that is unfaithful to the short summaries, suggesting promising directions for future work. 1
Physical Layer Anonymous Precoding: The Path to Privacy-Preserving Communications

Next-generation systems aim to increase both the speed and responsiveness of wireless communications while supporting compelling applications such as edge-cloud computing, remote health, vehicle-to-infrastructure communications, etc. As these applications are expected to carry confidential personal data, ensuring user privacy becomes a critical issue. In contrast to traditional security and privacy designs that aim to prevent confidential information from being eavesdropped upon by adversaries, or learned by unauthorized parties, in this article we consider designs that mask the user's identity during communication, resulting in anonymous communications. In particular, we examine the recent interest in physical layer (PHY) anonymous solutions. This line of research departs from conventional higher layer anonymous authentication/encryption and routing protocols, and judiciously manipulates the signaling pattern of transmitted signals in order to mask the sender's PHY characteristics. We first discuss the concept of anonymity at the PHY, then illustrate a strategy that is able to unmask the sender's identity by analyzing his/her PHY information only – that is, signalling patterns and the inherent fading characteristics. Subsequently, we review the emerging area of anonymous precoding to preserve the sender's anonymity while ensuring high receiver-side signal-to-interference-plus-noise ratio for communication. This family of anonymous precoding designs represents a new approach to providing anonymity at the PHY, introducing a new dimension for privacy-preserving techniques.
Design of Conformal Array of Rectangular Waveguide-fed Metasurfaces

We present a systematic design method for a cylindrical conformal array of rectangular waveguide-fed metasurfaces. The conformal metasurface consists of multiple curved rectangular waveguides loaded with metamaterial elements—electrically small apertures—inserted into the upper conducting walls of the waveguides. Each element radiates energy into free space to contribute to an overall radiation pattern. Thus, the geometry or electrical configuration of each of the elements needs to be tailored to generate a desired pattern. Generally, due to difficulties in modeling the effect of curvature, the design of conformal metasurface arrays has relied on full-wave simulations or experiments. In this study, we propose a design method utilizing the semianalytical model of a planar metasurface accounting for metamaterial elements' locations and orientations over a curved surface. Although approximate, we demonstrate that the alteration along with the framework of dipolar modeling of planar elements can be used for the analysis of conformal arrays with small curvature. We then design a conformal array metasurface using the method combined with the covariance matrix adaptation evolution strategy (CMA-ES) optimizer. Through numerical simulations, we confirm the validity of the proposed design method. Applications include the design of metasurfaces for radar, communications, and imaging systems for mobile platforms.
Proteome-informed machine learning studies of cocaine addiction

No anti-cocaine addiction drugs have been approved by the Food and Drug Administration despite decades of effort. The main challenge is the intricate molecular mechanisms of cocaine addiction, involving synergistic interactions among proteins upstream and downstream of the dopamine transporter. However, it is difficult to study so many proteins with traditional experiments, highlighting the need for innovative strategies in the field. We propose a proteome-informed machine learning (ML) platform for discovering nearly optimal anti-cocaine addiction lead compounds. We analyze proteomic protein-protein interaction networks for cocaine dependence to identify 141 involved drug targets and build 32 ML models for cross-target analysis of more than 60,000 drug candidates or experimental drugs for side effects and repurposing potentials. We further predict their ADMET (absorption, distribution, metabolism, excretion, and toxicity) properties. Our platform reveals that essentially all of the existing drug candidates fail in our cross-target and ADMET screenings but identifies several nearly optimal leads for further optimization.
Coloring mixed QCD/QED evolution

Parton showers are crucial components of high-energy physics calculations. Improving their modelling of QCD is an active research area since shower approximations are stumbling blocks for precision event generators. Naively, the interference between sub-dominant Standard-Model interactions and QCD can be of similar size to subleading QCD corrections. This article assesses the impact of QCD/QED interference effects in parton showers, by developing a sophisticated shower including QED, QCD at fixed color, and employing complete tree-level matrix element corrections for individual N_C=3NC=3 color configurations to embed interference. The resulting simulation indicates that QCD/QED interference effects are small for a simple test case and dwarfed by electro-weak resonance effects.
Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection

Bayesian Optimization (BO) is a method for globally optimizing black-box functions. While BO has been successfully applied to many scenarios, developing effective BO algorithms that scale to functions with high-dimensional domains is still a challenge. Optimizing such functions by vanilla BO is extremely time-consuming. Alternative strategies for high-dimensional BO that are based on the idea of embedding the high-dimensional space to the one with low dimension are sensitive to the choice of the embedding dimension, which needs to be pre-specified. We develop a new computationally efficient high-dimensional BO method that exploits variable selection. Our method is able to automatically learn axis-aligned sub-spaces, i.e. spaces containing selected variables, without the demand of any pre-specified hyperparameters. We theoretically analyze the computational complexity of our algorithm and derive the regret bound. We empirically show the efficacy of our method on several synthetic and real problems.
Information Dynamics and The Arrow of Time

Time appears to pass irreversibly. In light of CPT symmetry, the Universe’s initial condition is thought to be somehow responsible. We propose a model, the stochastic partitioned cellular automaton (SPCA), in which to study the mechanisms and consequences of emergent irreversibility. While their most natural definition is probabilistic, we show that SPCA dynamics can be made deterministic and reversible, by attaching randomly initialized degrees of freedom. This property motivates analogies to classical field theories. We develop the foundations of non-equilibrium statistical mechanics on SPCAs. Of particular interest are the second law of thermodynamics, and a mutual information law which proves fundamental in non-equilibrium settings. We believe that studying the dynamics of information on SPCAs will yield insights on foundational topics in computer engineering, the sciences, and the philosophy of mind. As evidence of this, we discuss several such applications, including an extension of Landauer’s principle, and sketch a physical justification of the causal decision theory that underlies the so-called psychological arrow of time.
Extragalactic Magnetism with SOFIA (Legacy Program) - II: A Magnetically Driven Flow in the Starburst Ring of NGC 1097

Galactic bars are frequent in disk galaxies and they may support the transfer of matter toward the central engine of active nuclei. The barred galaxy NGC 1097 has magnetic forces controlling the gas flow at several kpc scales, which suggest that magnetic fields (B-fields) are dynamically important along the bar and nuclear ring. However, the effect of the B-field on the gas flows in the central kpc scale has not been characterized. Using thermal polarized emission at 89 μm with HAWC+/SOFIA, here, we measure that the polarized flux is spatially located at the contact regions of the outer bar with the starburst ring. The linear polarization decomposition analysis shows that the 89 μm and radio (3.5 and 6.2 cm) polarization traces two different modes, m, of the B-field: a constant B-field orientation and dominated by m = 0 at 89 μm, and a spiral B-field dominated by m = 2 at radio. We show that the B-field at 89 μm is concentrated in the warmest region of a shock driven by the galactic-bar dynamics in the contact regions between the outer bar with the starburst ring. Radio polarization traces a superposition of the spiral B-field outside and within the starburst ring. According to Faraday rotation measures between 3.5 and 6.2 cm, the radial component of the B-field along the contact regions points toward the galaxy's center on both sides. We conclude that gas streams outside and within the starburst ring follow the B-field, which feeds the black hole with matter from the host galaxy.
Geometric Interpretation of Running Nystr\"{o}m-Based Kernel Machines and Error Analysis

Recently, Nystr\"{o}m method has proved its prominence empirically and theoretically in speeding up the training of kernel machines while retaining satisfactory performances and accuracy. So far, there are several different approaches proposed to exploit Nystr\"{o}m method in scaling up kernel machines. However, there is no comparative study over these approaches, and they were individually analyzed for specific types of kernel machines. Therefore, it remains a question that the philosophy of which approach is more promising when it extends to other kernel machines. In this work, motivated by the column inclusion property of Gram matrices, we develop a new approach with a clear geometric interpretation for running Nystr\"{o}m-based kernel machines. We show that the other two well-studied approaches can be equivalently transformed to be our proposed one. Consequently, analysis established for the proposed approach also works for these two. Particularly, our proposed approach makes it possible to develop approximation errors in a general setting. Besides, our analysis also manifests the relations among the aforementioned two approaches and another naive one. First, the analytical forms of the corresponding approximate solutions are only at odds with one term. Second, the naive approach can be implemented efficiently by sharing the same training procedure with others. These analytical results lead to the conjecture that the naive approach can provide more accurate approximate solutions than the other two sophisticated approaches. Since our analysis also offers ways for computing the accuracy of these approximate solutions, we run experiments with classification tasks to confirm our conjecture.
One-loop constituent quark contributions to the vector and axial-vector meson curvature mass

The renormalized contribution of fermions to the curvature masses of vector and axial-vector mesons is derived with two different methods at leading order in the loop expansion applied to the (2 + 1)-flavor constituent quark-meson model. The corresponding contribution to the curvature masses of the scalar and pseudoscalar mesons, already known in the literature, is rederived in a transparent way. The temperature dependence of the curvature mass of various (axial-)vector modes obtained by decomposing the curvature mass tensor is investigated along with the (axial-)vector– (pseudo)scalar mixing. All fermionic corrections are expressed as simple integrals that involve at finite temperature only the Fermi-Dirac distribution function modified by the Polyakov-loop degrees of freedom. The renormalization of the (axial-)vector curvature mass allows us to lift a redundancy in the original Lagrangian of the globally symmetric extended linear sigma model, in which terms already generated by the covariant derivative were reincluded with different coupling constants.
de novo gene birth as an inevitable consequence of adaptive evolution

Phylostratigraphy suggests that new genes are continually born de novo from non-genic sequences. The genes that persist found new lineages, contributing to the adaptive evolution of organisms. While recent evidence supports the view that de novo gene birth is frequent and widespread, the mechanisms underlying this process are yet to be discovered. Here we hypothesize and examine a potential general mechanism of gene birth driven by the accumulation of beneficial mutations at non-genic loci. To demonstrate this possibility, we model this mechanism, within the boundaries set by current knowledge on mutation effects. Estimates from this simple analysis are in line with observations of recurrent and extensive gene birth in genomics studies. Thus, we propose that rather than being inactive and silent, non-genic regions are likely to be dynamic storehouses of potential genes.
On the nonorientable four-ball genus of torus knots

The nonorientable four-ball genus of a knot $K$ in $S^3$ is the minimal first Betti number of nonorientable surfaces in $B^4$ bounded by $K$. By amalgamating ideas from involutive knot Floer homology and unoriented knot Floer homology, we give a new lower bound on the smooth nonorientable four-ball genus $\gamma_4$ of any knot. This bound is sharp for several families of torus knots, including $T_{4n,(2n\pm 1)^2}$ for even $n\ge 2$, a family Longo showed were counterexamples to Batson's conjecture. We also prove that, whenever $p$ is an even positive integer and $\frac{p}{2}$ is not a perfect square, the torus knot $T_{p,q}$ does not bound a locally flat M\"obius band for almost all integers $q$ relatively prime to $p$.
Sharp Concentration Inequalities for the Centered Relative Entropy

We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha>0$ (i.e. when the true underlying distribution is bounded sufficiently away from the boundary of the simplex), we prove an upper bound on the moment generating function of the centered relative entropy that matches (up to logarithmic factors in the alphabet size and $\alpha$) the optimal asymptotic rates, subsequently leading to a sharp concentration inequality for the centered relative entropy. As a corollary of this result we also obtain confidence intervals and moment bounds for the centered relative entropy that are sharp up to logarithmic factors in the alphabet size and $\alpha$.
A Framework for Institutional Risk Identification using Knowledge Graphs and Automated News Profiling

Organizations around the world face an array of risks impacting their operations globally. It is imperative to have a robust risk identification process to detect and evaluate the impact of potential risks before they materialize. Given the nature of the task and the current requirements of deep subject matter expertise, most organizations utilize a heavily manual process. In our work, we develop an automated system that (a) continuously monitors global news, (b) is able to autonomously identify and characterize risks, (c) is able to determine the proximity of reaching triggers to determine the distance from the manifestation of the risk impact and (d) identifies organization's operational areas that may be most impacted by the risk. Other contributions also include: (a) a knowledge graph representation of risks and (b) relevant news matching to risks identified by the organization utilizing a neural embedding model to match the textual description of a given risk with multi-lingual news.
Temporal Knowledge Graph Completion using Box Embeddings

Knowledge graph completion is the task of inferring missing facts based on existing data in a knowledge graph. Temporal knowledge graph completion (TKGC) is an extension of this task to temporal knowledge graphs, where each fact is additionally associated with a time stamp. Current approaches for TKGC primarily build on existing embedding models which are developed for static knowledge graph completion, and extend these models to incorporate time, where the idea is to learn latent representations for entities, relations, and timestamps and then use the learned representations to predict missing facts at various time steps. In this paper, we propose BoxTE, a box embedding model for TKGC, building on the static knowledge graph embedding model BoxE. We show that BoxTE is fully expressive, and possesses strong inductive capacity in the temporal setting. We then empirically evaluate our model and show that it achieves state-of-the-art results on several TKGC benchmarks
Back-translation for Large-Scale Multilingual Machine Translation

This paper illustrates our approach to the shared task on large-scale multilingual machine translation in the sixth conference on machine translation (WMT-21). In this work, we aim to build a single multilingual translation system with a hypothesis that a universal cross-language representation leads to better multilingual translation performance. We extend the exploration of different back-translation methods from bilingual translation to multilingual translation. Better performance is obtained by the constrained sampling method, which is different from the finding of the bilingual translation. Besides, we also explore the effect of vocabularies and the amount of synthetic data. Surprisingly, the smaller size of vocabularies perform better, and the extensive monolingual English data offers a modest improvement. We submitted to both the small tasks and achieve the second place.