Miracle-dz/UniFAR-FTUs huggingface dataset:
Fine-Tuning of Transformer Models for High Quality Screenplay Generation

Screenplays contain semantically and structurally rich text as the average movie screenplay is thousands of words (tokens) long and contains long range dependencies of entity relations and contextual plot elements throughout. Large-scale pre-trained language models (like GPT-2) perform very well in open-domain text generation when the generated outputs are only ten to low hundreds of tokens long. This project aims to test how well current large transformer models perform at producing long, coherent texts for the task of movie screenplay generation. We compared the outputs of several different models such as GPT-2, GPT-2 finetuned for 1 epoch, GPT-2 finteuned for 3-epochs, and a recently published non-monotonic, progressive generation approach [ProGeT] to see which model and architecture could best support high quality screenplay generation. We found that non-monotonic generation approach performed best on a set of automated evaluation metrics, including the contextual embedding similarity approach known as BERTScore. Looking at the example model outputs [appendix], we can see that the non-montonic approach, ProGeT, has outputs that read most similarly to human written movie screenplays.
Robust QA System with Task-Adaptive Pretraining, Data Augmentation, and Hyperparameter Tuning

In this project, we aim to build a robust question answering (QA) system by improving the DistiIBERT model[10]. A robust system eliminates the need for prohibitively large datasets in training models for use in different domains. To accomplish this goal, we implement task-adaptive pretraining (TAPT), model tuning such as transformer block re-initialization and increasing the number of training epochs, and ensemble methods. Since there is a limited amount of out-of-domain data, we also use data augmentation techniques for both pretraining and finetuning steps. Our final ensemble model achieves dev scores of Fl = 52.57, EM = 37.96 and test scores of Fl = 60.42, EM = 43.42.
Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization

This paper investigates correct variable selection in finite samples via $\ell_1$ and $\ell_1+\ell_2$ type penalization schemes. The asymptotic consistency of variable selection immediately follows from this analysis. We focus on logistic and linear regression models. The following questions are central to our paper: given a level of confidence $1-\delta$, under which assumptions on the design matrix, for which strength of the signal and for what values of the tuning parameters can we identify the true model at the given level of confidence? Formally, if $\widehat{I}$ is an estimate of the true variable set $I^*$, we study conditions under which $\mathbb{P}(\widehat{I}=I^*)\geq 1-\delta$, for a given sample size $n$, number of parameters $M$ and confidence $1-\delta$. We show that in identifiable models, both methods can recover coefficients of size $\frac{1}{\sqrt{n}}$, up to small multiplicative constants and logarithmic factors in $M$ and $\frac{1}{\delta}$. The advantage of the $\ell_1+\ell_2$ penalization over the $\ell_1$ is minor for the variable selection problem, for the models we consider here. Whereas the former estimates are unique, and become more stable for highly correlated data matrices as one increases the tuning parameter of the $\ell_2$ part, too large an increase in this parameter value may preclude variable selection.
Composite Surface Roughness and Color Change Following Air � ow Usage

Azam Valian Shahid Beheshti University of Medical Sciences School of Dentistry Zahra Jaber Ansari Shahid Beheshti University of Medical Sciences School of Dentistry Mohammad Moien Rezaie Shahid Beheshti University of Medical Sciences School of Dentistry Roja Askian (  askianroja841@gmail.com ) Shahid Beheshti University of Medical Sciences School of Dentistry https://orcid.org/0000-0001-67315785
Theoretical Calculation for Disproportionation of Methyltrichlorosilane and Trimethylchlorosilane Catalyzed by NaAlCl4/ZSM-5

Organic silicon material is widely used in many aspects [1]. But the commonly adopted direct synthesis approach [2] for preparing dimethyldichlorosilance (M2) would produce a number of chemical byproducts of which methyltrichlorosilane (M1) is in the majority. The disposal of the chemical byproducts refers to the future and destiny of the whole organic silicon industry. Now mostly researching AlCl3 is used as catalyst for the disposal of M1. But AlCl3 dissolves in reaction system easily, which leads to difficulties for subsequent separations and greatly influences the purity of the products. Previous experiments [3] put forward that catalyst of ZSM-5 zeolite supported by AlCl3 catalyzed the disproportionation under atmospheric and low temperature (70 °C, 100 kPa), the yield of M2 was more than 88 %. But AlCl3 sublimated and dissolved in samples. If AlCl3 can be prepared to be double salt NaAlCl4 of the supported catalyst, we would solve the above problem. Experiment [4] indicated that the catalytic effect of NaAlCl4-ZSM-5 and AlCl3/ZSM-5 catalyst was approximate, but the stability of complex catalyst was better than AlCl3/ZSM-5. In order to explain the result of the experiment, this paper discussed that the reaction mechanism of the disproportionation with complex catalyst. Yu et al. [5] studied Theoretical Calculation for Disproportionation of Methyltrichlorosilane and Trimethylchlorosilane Catalyzed by NaAlCl4/ZSM-5
Beyond Private Benefits of Control - the Case of Italian Pyramidal Groups

This study aims at exploring the ownership dynamics and corporate governance practices of Italian financial pyramidal groups, by providing a comprehensive analysis of the private benefits of control they capture. The unit of analysis is a set of family-owned conglomerates layered in various levels of public and unlisted firms, conceived to leverage the amount of capital controlled per unit of capital paid. On top of presenting a novel measure to quantify private benefits of control, arguably contributing new insights to the existing literature, this research suggests an implicit relationship between firms’ ownership structure and hosting country’s long-term economic growth.
Does Firm Life Stage Affect Investor Perceptions? Evidence from Earnings Announcement Reactions

This paper argues that firms in certain life stages may be more subjectively valued by individual investors, leading to an overoptimistic bias in stock prices that is subsequently corrected upon the release of earnings news. Using a cash flow-based life stage classification, introduction and decline stage companies exhibit three-day cumulative abnormal returns (CARs) around earnings announcements that are at least 112 bps lower than firms in growth, maturity, and shake-out stages. Specifically, introduction and decline stage stocks exhibit less positive reactions to positive earnings surprises and more negative reactions to negative earnings surprises relative to companies in other life stages. Lottery stocks’ excess returns around earnings announcements (Liu, Wang, Yu, and Zhao 2020) also vary based on firm life stage. Our findings suggest that individual investors’ overoptimistic expectations for introduction and decline stage stocks are met with disappointment when value-relevant earnings news is released. This study demonstrates that firm life stage has real implications for stock price reactions to earnings announcements in financial markets.
Maximum Likelihood Training of Score-Based Diffusion Models

Score-based diffusion models synthesize samples by reversing a stochastic process that diffuses data to noise, and are trained by minimizing a weighted combination of score matching losses. The log-likelihood of score-based diffusion models can be tractably computed through a connection to continuous normalizing flows, but log-likelihood is not directly optimized by the weighted combination of score matching losses. We show that for a specific weighting scheme, the objective upper bounds the negative log-likelihood, thus enabling approximate maximum likelihood training of score-based diffusion models. We empirically observe that maximum likelihood training consistently improves the likelihood of score-based diffusion models across multiple datasets, stochastic processes, and model architectures. Our best models achieve negative log-likelihoods of 2.83 and 3.76 bits/dim on CIFAR-10 and ImageNet 32x32 without any data augmentation, on a par with state-of-the-art autoregressive models on these tasks.
DOCTOR: A Simple Method for Detecting Misclassification Errors

Deep neural networks (DNNs) have shown to perform very well on large scale object recognition problems and lead to widespread use for real-world applications, including situations where DNN are implemented as"black boxes". A promising approach to secure their use is to accept decisions that are likely to be correct while discarding the others. In this work, we propose DOCTOR, a simple method that aims to identify whether the prediction of a DNN classifier should (or should not) be trusted so that, consequently, it would be possible to accept it or to reject it. Two scenarios are investigated: Totally Black Box (TBB) where only the soft-predictions are available and Partially Black Box (PBB) where gradient-propagation to perform input pre-processing is allowed. Empirically, we show that DOCTOR outperforms all state-of-the-art methods on various well-known images and sentiment analysis datasets. In particular, we observe a reduction of up to $4\%$ of the false rejection rate (FRR) in the PBB scenario. DOCTOR can be applied to any pre-trained model, it does not require prior information about the underlying dataset and is as simple as the simplest available methods in the literature.
Glance-and-Gaze Vision Transformer

Recently, there emerges a series of vision Transformers, which show superior performance with a more compact model size than conventional convolutional neural networks, thanks to the strong ability of Transformers to model long-range dependencies. However, the advantages of vision Transformers also come with a price: Self-attention, the core part of Transformer, has a quadratic complexity to the input sequence length. This leads to a dramatic increase of computation and memory cost with the increase of sequence length, thus introducing difficulties when applying Transformers to the vision tasks that require dense predictions based on high-resolution feature maps. In this paper, we propose a new vision Transformer, named Glance-and-Gaze Transformer (GG-Transformer), to address the aforementioned issues. It is motivated by the Glance and Gaze behavior of human beings when recognizing objects in natural scenes, with the ability to efficiently model both long-range dependencies and local context. In GG-Transformer, the Glance and Gaze behavior is realized by two parallel branches: The Glance branch is achieved by performing self-attention on the adaptively-dilated partitions of the input, which leads to a linear complexity while still enjoying a global receptive field; The Gaze branch is implemented by a simple depth-wise convolutional layer, which compensates local image context to the features obtained by the Glance mechanism. We empirically demonstrate our method achieves consistently superior performance over previous state-of-the-art Transformers on various vision tasks and benchmarks. The codes and models will be made available at https://github.com/yucornetto/GG-Transformer.
Global existence, uniform boundedness, and stabilization in a chemotaxis system with density-suppressed motility and nutrient consumption

Abstract Well-posedness and uniform-in-time boundedness of classical solutions are investigated for a three-component parabolic system which describes the dynamics of a population of cells interacting with a chemoattractant and a nutrient. The former induces a chemotactic bias in the diffusive motion of the cells and is accounted for by a density-suppressed motility. Well-posedness is first established for generic positive and non-increasing motility functions vanishing at infinity. Growth conditions on the motility function guaranteeing the uniform-in-time boundedness of solutions are next identified. Finally, for sublinearly decaying motility functions, convergence to a spatially homogeneous steady state is shown, with an exponential rate for consumption rates behaving linearly near zero.
A very high energy hadron collider on the Moon

The long-term prospect of building a hadron collider around the circumference of a great circle of the Moon is sketched. A circular collider on the Moon (CCM) of ∼11 000 km in circumference could reach a proton–proton center-of-mass collision energy of 14 PeV—a thousand times higher than the Large Hadron Collider at CERN—optimistically assuming a dipole magnetic field of 20 T. Several aspects of such a project are presented, including siting, construction, availability of necessary materials on the Moon, and powering, as well as a discussion of future studies and further information needed to determine the more concrete feasibility of each. Machine parameters and vacuum requirements are explored, and an injection scheme is delineated. Other unknowns are set down. Due to the strong interest from multiple organizations in establishing a permanent Moon presence, a CCM could be the (next-to-) next-to-next-generation discovery machine for high-energy particle physics and a natural successor to next-generation machines, such as the proposed future circular collider at CERN or a super proton–proton collider in China, and other future machines, such as a collider in the Sea, in the Gulf of Mexico. A CCM would serve as an important stepping stone toward a Planck-scale collider sited in our Solar System.
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering

Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neural scene representations are a promising approach to 3D scene understanding. In this work, we propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field parameterized via a neural implicit representation. Rendering a ray from an LFN requires only a single network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or volumetric based renderers in 3D-structured neural scene representations. In the setting of simple scenes, we leverage meta-learning to learn a prior over LFNs that enables multi-view consistent light field reconstruction from as little as a single image observation. This results in dramatic reductions in time and memory complexity, and enables real-time rendering. The cost of storing a 360-degree light field via an LFN is two orders of magnitude lower than conventional methods such as the Lumigraph. Utilizing the analytical differentiability of neural implicit representations and a novel parameterization of light space, we further demonstrate the extraction of sparse depth maps from LFNs.
A Nearly Optimal All-Pairs Min-Cuts Algorithm in Simple Graphs

We give an <tex>$n^{2+o(1)}$</tex>-time algorithm for finding <tex>$s-t$</tex> min-cuts for all pairs of vertices <tex>$s$</tex> and <tex>$t$</tex> in a simple, undirected graph on <tex>$n$</tex> vertices. We do so by constructing a Gomory-Hu tree (or cut equivalent tree) in the same running time, thereby improving on the recent bound of <tex>$\tilde{O}(n^{2.5})$</tex> by Abboud et al. (STOC 2021). Our running time is nearly optimal as a function of <tex>$n$</tex>.
Training Energy-Efficient Deep Spiking Neural Networks with Time-to-First-Spike Coding

The tremendous energy consumption of deep neural networks (DNNs) has become a serious problem in deep learning. Spiking neural networks (SNNs), which mimic the operations in the human brain, have been studied as prominent energy-efficient neural networks. Due to their event-driven and spatiotemporally sparse operations, SNNs show possibilities for energy-efficient processing. To unlock their potential, deep SNNs have adopted temporal coding such as time-to-first-spike (TTFS)coding, which represents the information between neurons by the first spike time. With TTFS coding, each neuron generates one spike at most, which leads to a significant improvement in energy efficiency. Several studies have successfully introduced TTFS coding in deep SNNs, but they showed restricted efficiency improvement owing to the lack of consideration for efficiency during training. To address the aforementioned issue, this paper presents training methods for energy-efficient deep SNNs with TTFS coding. We introduce a surrogate DNN model to train the deep SNN in a feasible time and analyze the effect of the temporal kernel on training performance and efficiency. Based on the investigation, we propose stochastically relaxed activation and initial value-based regularization for the temporal kernel parameters. In addition, to reduce the number of spikes even further, we present temporal kernel-aware batch normalization. With the proposed methods, we could achieve comparable training results with significantly reduced spikes, which could lead to energy-efficient deep SNNs.
Fre-GAN: Adversarial Frequency-consistent Audio Synthesis

Although recent works on neural vocoder have improved the quality of synthesized audio, there still exists a gap between generated and ground-truth audio in frequency space. This difference leads to spectral artifacts such as hissing noise or reverberation, and thus degrades the sample quality. In this paper, we propose Fre-GAN which achieves frequency-consistent audio synthesis with highly improved generation quality. Specifically, we first present resolution-connected generator and resolution-wise discriminators, which help learn various scales of spectral distributions over multiple frequency bands. Additionally, to reproduce high-frequency components accurately, we leverage discrete wavelet transform in the discriminators. From our experiments, Fre-GAN achieves high-fidelity waveform generation with a gap of only 0.03 MOS compared to ground-truth audio while outperforming standard models in quality.
Finding and Fixing Spurious Patterns with Explanations

Image classifiers often use spurious patterns, such as"relying on the presence of a person to detect a tennis racket, which do not generalize. In this work, we present an end-to-end pipeline for identifying and mitigating spurious patterns for such models, under the assumption that we have access to pixel-wise object-annotations. We start by identifying patterns such as"the model's prediction for tennis racket changes 63% of the time if we hide the people. "Then, if a pattern is spurious, we mitigate it via a novel form of data augmentation. We demonstrate that our method identifies a diverse set of spurious patterns and that it mitigates them by producing a model that is both more accurate on a distribution where the spurious pattern is not helpful and more robust to distribution shift.
Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path

The recently discovered Neural Collapse (NC) phenomenon occurs pervasively in today's deep net training paradigm of driving cross-entropy (CE) loss towards zero. During NC, last-layer features collapse to their class-means, both classifiers and class-means collapse to the same Simplex Equiangular Tight Frame, and classifier behavior collapses to the nearest-class-mean decision rule. Recent works demonstrated that deep nets trained with mean squared error (MSE) loss perform comparably to those trained with CE. As a preliminary, we empirically establish that NC emerges in such MSE-trained deep nets as well through experiments on three canonical networks and five benchmark datasets. We provide, in a Google Colab notebook, PyTorch code for reproducing MSE-NC and CE-NC: at https://colab.research. google.com/github/neuralcollapse/neuralcollapse/blob/main/neuralcollapse.ipynb. The analytically-tractable MSE loss offers more mathematical opportunities than the hard-to-analyze CE loss, inspiring us to leverage MSE loss towards the theoretical investigation of NC. We develop three main contributions: (I) We show a new decomposition of the MSE loss into (A) terms directly interpretable through the lens of NC and which assume the last-layer classifier is exactly the least-squares classifier; and (B) a term capturing the deviation from this least-squares classifier. (II) We exhibit experiments on canonical datasets and networks demonstrating that term-(B) is negligible during training. This motivates us to introduce a new theoretical construct: the central path, where the linear classifier stays MSE-optimal for feature activations throughout the dynamics. (III) By studying renormalized gradient flow along the central path, we derive exact dynamics that predict NC.
A differentiable N-body code for transit timing and dynamical modeling – I. Algorithm and derivatives

When fitting N-body models to astronomical data – such as transit times, radial velocity, and astrometric positions at observed times – the derivatives of the model outputs with respect to the initial conditions can help with model optimization and posterior sampling. Here we describe a general-purpose symplectic integrator for arbitrary orbital architectures, including those with close encounters, which we have recast to maintain numerical stability and precision for small step sizes. We compute the derivatives of the N-body coordinates and velocities as a function of time with respect to the initial conditions and masses by propagating the Jacobian along with the N-body integration. For the first time we obtain the derivatives of the transit times with respect to the initial conditions and masses using the chain rule, which is quicker and more accurate than using finite differences or automatic differentiation. We implement this algorithm in an open source package, NbodyGradient.jl, written in the Julia language, which has been used in the optimization and error analysis of transit-timing variations in the TRAPPIST-1 system. We present tests of the accuracy and precision of the code, and show that it compares favorably in speed to other integrators which are written in C.
On the Strategyproofness of the Geometric Median

The geometric median, an instrumental component of the secure machine learning toolbox, is known to be effective when robustly aggregating models (or gradients), gathered from potentially malicious (or strategic) users. What is less known is the extent to which the geometric median incentivizes dishonest behaviors. This paper addresses this fundamental question by quantifying its strategyproofness. While we observe that the geometric median is not even approximately strategyproof, we prove that it is asymptotically $\alpha$-strategyproof: when the number of users is large enough, a user that misbehaves can gain at most a multiplicative factor $\alpha$, which we compute as a function of the distribution followed by the users. We then generalize our results to the case where users actually care more about specific dimensions, determining how this impacts $\alpha$. We also show how the skewed geometric medians can be used to improve strategyproofness.