Miracle-dz/UniFAR-FTUs huggingface dataset:
Cochlear implantation impairs intracochlear microcirculation and counteracts iNOS induction in guinea pigs

Introduction Preservation of residual hearing remains a great challenge during cochlear implantation. Cochlear implant (CI) electrode array insertion induces changes in the microvasculature as well as nitric oxide (NO)-dependent vessel dysfunction which have been identified as possible mediators of residual hearing loss after cochlear implantation. Methods A total of 24 guinea pigs were randomized to receive either a CI (n = 12) or a sham procedure (sham) by performing a cochleostomy without electrode array insertion (n = 12). The hearing threshold was determined using frequency-specific compound action potentials. To gain visual access to the stria vascularis, a microscopic window was created in the osseous cochlear lateral wall. Cochlear blood flow (CBF) and cochlear microvascular permeability (CMP) were evaluated immediately after treatment, as well as after 1 and 2 h, respectively. Finally, cochleae were resected for subsequent immunohistochemical analysis of the iNOS expression. Results The sham control group showed no change in mean CBF after 1 h (104.2 ± 0.7%) and 2 h (100.8 ± 3.6%) compared to baseline. In contrast, cochlear implantation resulted in a significant continuous decrease in CBF after 1 h (78.8 ± 8.1%, p < 0.001) and 2 h (60.6 ± 11.3%, p < 0.001). Additionally, the CI group exhibited a significantly increased CMP (+44.9% compared to baseline, p < 0.0001) and a significant increase in median hearing threshold (20.4 vs. 2.5 dB SPL, p = 0.0009) compared to sham after 2 h. Intriguingly, the CI group showed significantly lower iNOS-expression levels in the organ of Corti (329.5 vs. 54.33 AU, p = 0.0003), stria vascularis (596.7 vs. 48.51 AU, p < 0.0001), interdental cells (564.0 vs. 109.1 AU, p = 0.0003) and limbus fibrocytes (119.4 vs. 18.69 AU, p = 0.0286). Conclusion Mechanical and NO-dependent microvascular dysfunction seem to play a pivotal role in residual hearing loss after CI electrode array insertion. This may be facilitated by the implantation associated decrease in iNOS expression. Therefore, stabilization of cochlear microcirculation could be a therapeutic strategy to preserve residual hearing.
Cost-effectiveness and budget impact of pembrolizumab+axitinib versus sunitinib in patients with advanced clear-cell renal cell carcinoma in the Netherlands

Background The phase 3 clinical trial KEYNOTE-426 suggested a higher efficacy regarding overall survival (OS) and progression-free survival (PFS) of pembrolizumab+axitinib compared to sunitinib as a first-line treatment for patients with advanced renal cell carcinoma. In this analysis, the potential cost-effectiveness of this combination treatment versus sunitinib for patients with advanced clear-cell renal cell carcinoma (accRCC) was examined from the societal perspective in the Netherlands. Methods For this analysis, a partitioned survival model was constructed. Clinical data were obtained from the published KEYNOTE-426 trial reports; data on costs and (dis-)utilities were derived from published literature. Costs outside of the healthcare sector included treatment-related travel, informal care and productivity loss. Next to a probabilistic scenario analysis, various scenario analyses were performed that aimed at survival extrapolation, different utility values, treatment duration and drug pricing, as well as restricting the cohort to patients with an intermediate or poor prognosis. Further, a budget impact analysis over three years was conducted, in which a sensitivity analysis concerning ranges in costs and the number of patients was applied. Moreover, a scenario concerning increasing market penetration of pembrolizumab+axitinib up to a market share of 80% in the third year was analyzed. Results The incremental cost-effectiveness ratio (ICER) of pembrolizumab+axitinib was estimated at €368,396/quality-adjusted life year (QALY) gained, with an incremental QALY gain of 0.55 over sunitinib. The probability of cost-effectiveness at a willingness-to-pay threshold of €80,000/QALY was estimated at 0%, a 50% probability was estimated at €340,000/QALY. Cost-effectiveness was not achieved in any of the applied scenarios. The budget impact over three years amounted to €417.3 million upon instantaneous and full replacement of sunitinib, and to €214.9 million with increasing market penetration. Conclusion Pembrolizumab+axitinib was not estimated to be cost-effective compared to sunitinib as a first-line treatment for patients with accRCC in the Netherlands from a societal perspective. In none of the analyzed scenarios, cost-effectiveness was achieved. However, price reductions and shorter treatment durations might lead to a more favorable ICER.
Evolutionary transition between bee pollination and hummingbird pollination in Salvia: Comparing means, variances and covariances of corolla traits

Covariation among traits can modify the evolutionary trajectory of complex structures. This process is thought to operate at a microevolutionary scale, but its long‐term effects remain controversial because trait covariation can itself evolve. Flower morphology, and particularly floral trait (co)variation, has been envisioned as the product of pollinator‐mediated selection. Available evidence suggests that major changes in pollinator assemblages may affect the joint expression of floral traits and their phenotypic integration. We expect species within a monophyletic lineage sharing the same pollinator type will show not only similarity in trait means but also similar phenotypic variance‐covariance structures. Here, we tested this expectation using eighteen Salvia species pollinated either by bees or by hummingbirds. Our findings indicated a nonsignificant multivariate phylogenetic signal and a decoupling between means and variance‐covariance phenotypic matrices of floral traits during the evolution to hummingbird pollination. Mean trait value analyses revealed significant differences between bee‐ and hummingbird‐pollinated Salvia species although fewer differences were detected in the covariance structure between groups. Variance‐covariance matrices were much more similar among bee‐ than hummingbird‐pollinated species. This pattern is consistent with the expectation that, unlike hummingbirds, bees physically manipulate the flower, presumably exerting stronger selection pressures favouring morphological convergence among species. Overall, we conclude that the evolution of hummingbird pollination proceeded through different independent transitions. Thus, although the evolution of hummingbird pollination led to a new phenotypic optimum, the process involved the diversification of the covariance structure.
Development and Prospect of Image Multiscale Geometric Analysis

The aim of Multiscale Geometric Analysis is to find a kind of optimal representation of high dimension function in the sense of nonlinear approximation.Based on the nonlinear approximation of 2-D function,the mathematical and neurophysiological backgrounds of Image Multiscale Geometric Analysis are studied on this paper,and its development history,current and future challenges are reviewed in details.
Methodological ignorance: A comment on field experiments and methodological intolerance

Glenn Harrison [Journal of Economic Methodology, 2013, 20, 103–117] discusses four related forms of methodological intolerance with respect to field experiments: field experiments should rely on some form of randomization, should be disconnected from theory, the concept of causality should only be defined in terms of observables, and the role of laboratory experiments is dismissed. As is often the case, the cause of intolerance is ignorance, as it is here. To acquire knowledge about potential influences, which we need for both the evaluation of internal and external validity of experimental results, we cannot do without theory. A purely empiricist (inductive) methodology will be unable to give us sufficient understanding of the validity of these results. An account of causality only based on directly observed things, is an account based on factual influences only. This account will be too restricted, because it will not deal with the unobserved potential influences, which we need – again – for the evaluation of the internal and external validity of the experimental results. Every investigation in a laboratory that is feasible regarding a specific potential influence may lead to deep knowledge. It is simply scientifically irrational to dismiss this kind of knowledge.
Modulation classification of M-QAM signals

This paper proposed a modulation recognition algorithm for M-QAM signals. This scheme does not require the prior information about baud rate and carry frequency of the received signals. These parameters are estimated first, then interpolation, deconvolution, clustering and recognition are performed. Computer simulations show that the correct recognition ratio is larger than 95% under the condition that the SNR is higher than 8dB and more than 400 symbols are used. So it is suitable for the practical application of signal detection and fast recognition in non-cooperation communication systems.
Contrastive Pretraining of minBERT to Improve Performance in Downstream Tasks

Bidirectional Encoder Representations from Transformers (BERT) is a transformer-based model that generates contextual word representations. By extending BERT, these learned representations of language can be applied across a range of tasks. In this project, we evaluate the performance of a BERT-based multitask model in sentiment analysis, paraphrase detection, and semantic textual similarity, using both pretrained and fine-tuned weights. We then evaluate the effects of contrastive pre-training on model performance using the SimCSE framework. We demonstrate that pretraining using contrastive learning objectives yields sentence embeddings that outperform baseline methods after task-specific fine-tuning. Finally, we perform an error analysis to qualitatively characterize the effects of contrastive pretraining and the limitations of our model.
Efficient Semiring-Weighted Earley Parsing

We present Earley’s (1970) context-free parsing algorithm as a deduction system, incorporating various known and new speed-ups. In particular, our presentation supports a known worst-case runtime improvement from Earley’s (1970) O(N3|G||R|), which is unworkable for the large grammars that arise in natural language processing, to O(N3|G|), which matches the complexity of CKY on a binarized version of the grammar G. Here N is the length of the sentence, |R| is the number of productions in G, and |G| is the total length of those productions. We also provide a version that achieves runtime of O(N3|M|) with |M| leq |G| when the grammar is represented compactly as a single finite-state automaton M (this is partly novel). We carefully treat the generalization to semiring-weighted deduction, preprocessing the grammar like Stolcke (1995) to eliminate the possibility of deduction cycles, and further generalize Stolcke’s method to compute the weights of sentence prefixes. We also provide implementation details for efficient execution, ensuring that on a preprocessed grammar, the semiring-weighted versions of our methods have the same asymptotic runtime and space requirements as the unweighted methods, including sub-cubic runtime on some grammars.
Solving the Ill-Conditioning in Neural Network

In this paper we investigate the feed-forward learning prob- lem. The well-known ill-conditioning which is present in most feed-for- ward learning problems is shown to be the result of the structure of the network. Also, the well-known problem that weights between 'higher' layers in the network have to settle before 'lower' weights can converge is addressed. We present a solution to these problems by modifying the structure of the network through the addition of linear connections which carry shared weights. We call the new network structure the linearly augmented feed-forward network, and it is shown that the universal ap- proximation theorems are still valid. Simulation experiments show the validity of the new method, and demonstrate that the new network is less sensitive to local minima and learns faster than the original network.
Multi-task NLP with BERT

We extend BERT to preform multiple Natural Language Processing (NLP) tasks. The tasks are (1) multi-class sentence classification task in the form of sentiment analysis of the Stanford Sentiment Treebank dataset of movie reviews, (2) multi-class sentence pair classification task in the form of predicting the similarity between sentences of the SemEval STS Benchmark dataset, and (3) a binary-class sentence pair classification task in the form of de-duplicating Quora questions. The original model is provided by Stanford CS224N teaching staff. Our best preforming model changed that base model to instead operate as suggested in the BERT paper whereby a linear layer (plus a task discriminating feature) is placed a top the BERT model which classifies the [CLS] token generated by feeding in a single or multiple sentences. We attempted gradient surgery (4), task embedding (our own novelty), and task attention embedding (3) but in the end all models preformed more or less the same although their time to best result differed.
Strategies in TOEFL Reading Comprehension for Chinese Students

With China’s development, an increasing number of students choose to further their studies abroad. However, many Chinese students do not know how to prepare for the TOEFL test by themselves.Thus, there are three sections in my literature review, which are the introduction of TOEFL, TOEFL Reading Comprehension Strategies for Native Chinese Speakers, and TOEFL test preparation for Chinese students.After that, this project includes original materials so that beginning level students can get a preliminary understanding of the TOEFL reading comprehension tests. The material is deliberately based on Chinese culture, which makes it relevant and easier for beginners to understand. This project can improve students’ motivation to study for the TOEFL. There are four units in this project, and three of them introduce types of TOEFL reading questions. Each of the units contains one reading passage and several questions. The last unit is grammar-based and it presents the adjective clause system. The materials can be used by students who want to improve their English proficiency abilities while increasing their motivation for learning English. Furthermore, students can increase their confidence by being better prepared for the TOEFL.
BERT and Learnie: Multi-task Classification Across Semantics Street

This paper explores various approaches to improving the performance of the BERT model across the three tasks of sentiment analysis, paraphrase detection, and semantic textual similarity evaluation. We focused particularly on exploring methods to train all three tasks simultaneously on a singular BERT backbone that can perform well across all three downstream tasks. We did this by exploring multiple model architectures, supplemented by our implementation of a wide range of hyperparameter search algorithms, scheduling techniques, regularization methods, and optimization techniques such as gradient surgery. Our results showed that using a singular BERT backbone but separate linear layers for all three tasks led to significantly better performance than the baseline model. Furthermore, we also found that implementing gradient surgery, data augmentation, variable learning rate with linear warmup and linear decay, early stopping, and performing aggressive hyperparameter search using a population-based tuning algorithm yielded our best-performing model, which was 102% better than our baseline implementation. We hope that our study will provide a well-performing and easily extensible baseline on which others can quickly train a small BERT model that is efficient across multiple downstream tasks.
BERTer Multi-task Fine-tuning for Sentence-Level Tasks

In our project, we implement a BERT model with additional pre-training using masked language modeling (MLM), cosine similarity fine-tuning, and multitask fine-tuning with gradient surgery. We attempt to display improved performance in three downstream tasks (sentiment analysis, paraphrase detection, and semantic textual similarity) over a baseline BERT model with linear projection heads. The most notable result was the significant positive impact of cosine-similarity fine-tuning on STS (semantic textual similarity). On the other hand, both MLM and gradient surgery had negligible impact in increasing the ceiling of our model for the downstream tasks. However, when cross-comparing the accuracies in each epoch with our baseline, the gradient surgery model appears to converge much faster overall. With our model, we achieved a top 33% ranking in the test set leaderboard with an overall dev score of 0.707 (individual scores: 0.529 SST test accuracy, 0.747 paraphrase test accuracy, and 0.845 STS test correlation).
Abstractive Summarization of Legal Text Corpuses Using Transfer Learning

Abstractive summarization using transformers is difficult to perform on long passages of text due to the limited token length that current models like BERT and T5 can process (usually 1024 tokens). This challenge is particularly significant when processing legal documents, which frequently span dozens of pages. In this paper, we utilize the Big Bird architecture proposed by Zaheer et al. (2020) which has an expanded maximum input length of 4096 tokens. Using this extended context window, we explore the effects of various fine-tuning methods on generating ab-stractive summaries of legal documents that are 2 . 5 times larger than our model’s maximum context window - on average. We find that partitioning a document into chunks of 4096 tokens, where the last 500 tokens from the previous chunk are present in the next chunk, produces summaries with a significantly higher level of fluency and accuracy when compared to vanilla fine-tuning methods.
Development of Wind Turbine Fault Analysis Setup based on DFIG Hardware in the Loop Simulator

Article History: Received 16 July 2022 Reviewed 15 October 2022 Revised 29 October 2022 Accepted 07 November 2022 Background and Objectives: Renewable energy, like wind turbines, is growing rapidly in the world today due to environmental pollution, so their maintenance plans are very important. Fault diagnosis and fault-tolerant approaches are typical methods to reduce the cost of energy production and downtime of Wind Turbines (WTs). Methods: In this paper, a new Hardware In the Loop (HIL) simulator based on Double Feed Induction Generator (DFIG) for fault diagnosis and fault-tolerant control is proposed. The system developed as a laboratory bed uses a generator with a power of about 90 kW, which is connected from two sides to a back-to-back power converter with a power of one-third of the generator power. The generator is connected to a motor as a propulsion and wind energy replacement with a power of about 110 kW, and this connection is established through a gearbox with a gear ratio of more than three. Results: The effectiveness of the proposed simulator is evaluated based on different fault representations back-to-back converter and generator. Conclusion: The experiment shows that the Condition Based Maintenance (CBM) is improved by the proposed simulator and the fault is modeled before serious damage occurs. This setup is effective for the development of wind turbine fault analysis software. As the testing on real WTs is very expensive, to improve and develop the research fields of condition monitoring and WT control, this low-cost setup is effective.
Generalizing BERT through Multi-Task Learning

Bidirectional Encoder Representations from Transformers or BERT is a model that uses word representations to perform tasks such as sentiment analysis, paraphrase detection, and semantic textual similarity. In this paper, a basic BERT model is implemented as a baseline model. Then, in an attempt to improve BERT’s ability to generalize well across the three downstream tasks listed above, this paper implements a multi-task method of training BERT
Fine-Tuning BERT for Sentiment Analysis, Paraphrase Detection and Semantic Text Similarity NLP Tasks

In this default final project, we train BERT model to perform well on Sentiment Analysis, Paraphrase Detection, and Semantic Text Similarity tasks. Furthermore, we implement several core components of BERT and fine-tune a pre-trained BERT to perform well on the three aforementioned NLP tasks. In our experiments, we demonstrate advantages of first further pre-training on target data, followed by targeted fine-tuning on individual tasks while keeping the BERT layers frozen, and lastly end to end multi-task fine-tuning of the whole network. We find that this strategy, coupled with a scheduled freezing of BERT layers, gradient surgery
minBERT and Multiple Downstream Tasks

BERT (Bidirectional Encoder Representations from Transformers) [1] has significantly advanced natural language processing (NLP) by leveraging transformer architecture to learn contextualized word representations from large amounts of text data. In this paper, we present the minBERT implementation that simultaneously performs three tasks: sentiment analysis [2], paraphrase detection [3], and semantic textual similarity (STS) [4]. Rather than fine-tuning BERT on individual tasks, we leverage multi-task learning to update BERT model, which has been shown to improve performance (Bi et al. [5]). We also apply hyperparameter optimization to further improve the model's performance. Our results demonstrate the effectiveness of minBERT in achieving compelling performance on all three tasks, while also providing insights into the optimal hyperparameters for each task.
Measuring the Long-Term Impact of Environmental Externalities

We exploit a unique dataset of the Danish population observed over more than four decades to measure the long-term health effects of toxic pollution. Making use of incinerators’ openings and closings, we explore the interactions among cumulative exposure, intensity, and latency as determinants of lung cancer incidence. On average, the impact of an incinerator becomes detectable 25-30 years after the opening and persists for 10-15 years after the closing. An average amount of cumulative exposure increases the risk of lung cancer by 8%. At low levels of intensity, cumulative exposure impacts lung cancer with a decade-long latency (up to 15 years). At high levels, latency is much shorter (less than 5 years). Not accounting for latency leads to underestimate the health effects of cumulative exposure to incinerators by 90 to 100%, depending on intensity. Preliminary and Incomplete Current Draft: March 2023 JEL Classification: JEL I1, K32, Q5, Q53