Reviewerly/exHarmony huggingface dataset:
FTX Collapse: A Ponzi Story

FTX used to be the third-largest centralized exchange (CEX) in crypto space, managing over $10B in daily trading volume before its downfall. Such a giant, however, failed to avoid the fate of mania, panic, and crash. In this work, we revisit the FTX’s crash by telling it as a Ponzi story. In regards to why FTX could not sustain this Ponzi game, we extract and demonstrate the three drivers of the FTX collapse, namely FTT, leverage, and diversion. The unfunctionality in each factor can iteratively magnify the impact of damages when the panic is triggered. Rooted in the unstable ground, FTX eventually suffered insolvency and rapidly crashed in Nov. 2022. The crisis of FTX is not an isolated event; it consequently results in the collapse of a chain of associated companies in the entire market. Recall this painful experience, we discuss possible paths for a way forward for both CeFi and DeFi implementations.
Artificial Intelligence for the Management of Breast Cancer: An Overview

Abstract: Breast cancer is a severe global health problem, and early detection, accurate diagnosis, and personalized treatment is the key to improving patient outcomes. Artificial intelligence (AI) and machine learning (ML) have emerged as promising breast cancer research and clinical practice tools in recent years. Various projects are underway in early detection, diagnosis, prognosis, drug discovery, advanced image analysis, precision medicine, predictive modeling, and personalized treatment planning using artificial intelligence and machine learning. These projects use different algorithms, including convolutional neural networks (CNNs), support vector machines (SVMs), decision trees, and deep learning methods, to analyze and improve different types of data, such as clinical, genomic, and imaging data for breast cancer management. The success of these projects has the potential to transform breast cancer care, and continued research and development in this area is likely to lead to more accurate and personalized breast cancer diagnosis, treatment, and outcomes.
Short Squeeze in DeFi Lending Market: Decentralization in Jeopardy?

Anxiety levels in the Aave community spiked in November 2022 as Avi Eisenberg performed an attack on Aave. Eisenberg attempted to short the CRV token by using funds borrowed on the protocol to artificially deflate the value of CRV. While the attack was ultimately unsuccessful, it left the Aave community scared and even raised question marks regarding the feasibility of large lending platforms under decentralized governance. In this work, we analyze Avi Eisenberg’s actions and show how he was able to artificially lower the price of CRV by selling large quantities of borrowed CRV for stablecoins on both decentralized and centralized exchanges. Despite the failure of his attack, it still led to irretrievable debt worth more than 1.5 Mio USD at the time and, thereby, quadrupled the protocol’s irretrievable debt. Furthermore, we highlight that his attack was enabled by the vast proportion of CRV available to borrow as well as Aave’s lending protocol design hindering rapid intervention. We stress Eisenberg’s attack exposes a predicament of large DeFi lending protocols: limit the scope or compromise on ‘decentralization’.
Modulation recognition method of mixed signals based on cyclic spectrum projection

Abstract The signal in the receiver is mainly a combination of different modulation types due to the complex electromagnetic environment, which makes the modulation recognition of the mixed signal a hot topic in recent years. In response to the poor adaptability of existing mixed signals recognition methods, this paper proposes a new recognition method for mixed signals based on cyclic spectrum projection and deep neural network. Firstly, through theoretical derivation, we prove the feasibility of using cyclic spectrum for mixed communication signal identification. Then, we adopt grayscale projections on the two-dimensional cyclic spectrum as identifying representation. And a new nonlinear piecewise mapping and directed pseudo-clustering method are used to enhance the above-mentioned grayscale images, which reduces the impact of energy ratios and symbol rates on signal identification. Finally, we use deep neural networks to extract deep abstract modulation information to achieve effective recognition of mixed signals. Simulation results show that the proposed method is robust against noise. When signal-to-noise ratio is not less than 0 dB, the average recognition rate is greater than 95%. Furthermore, this method exhibits good robustness towards the changes in signal symbol rates and energy ratios between mixed signals.
A Natural Language Generation Algorithm for Greek by Using Hole Semantics and a Systemic Grammatical Formalism

This work is about the progress of previous related work based on an experiment to improve the intelligence of robotic systems, with the aim of achieving more linguistic communication capabilities between humans and robots. In this paper, the authors attempt an algorithmic approach to natural language generation through hole semantics and by applying the OMAS-III computational model as a grammatical formalism. In the original work, a technical language is used, while in the later works, this has been replaced by a limited Greek natural language dictionary. This particular effort was made to give the evolving system the ability to ask questions as well as the authors developed an initial dialogue system using these techniques. The results show that the use of these techniques the authors apply can give us a more sophisticated dialogue system in the future.
Fast and Accurate: Efficient Full-Domain Functional Bootstrap and Digit Decomposition for Homomorphic Computation

The functional bootstrap in FHEW/TFHE allows for fast table lookups on ciphertexts and is a powerful tool for privacy-preserving computations. However, the functional bootstrap suffers from two limitations: the negacyclic constraint of the lookup table (LUT) and the limited ability to evaluate large-precision LUTs. To overcome the first limitation, several full-domain functional bootstraps (FDFB) have been developed, enabling the evaluation of arbitrary LUTs. Meanwhile, algorithms based on homomorphic digit decomposition have been proposed to address the second limitation. Although these algorithms provide effective solutions, they are yet to be optimized. This paper presents four new FDFB algorithms and two new homomorphic decomposition algorithms that improve the state-of-the-art. Our FDFB algorithms reduce the output noise, thus allowing for more efficient and compact parameter selection. Across all parameter settings, our algorithms reduce the runtime by up to 39.2%. Our homomorphic decomposition algorithms also run at 2.0x and 1.5x the speed of prior algorithms. We have implemented and benchmarked all previous FDFB and homomorphic decomposition algorithms and our methods in OpenFHE.
A Methodological Path for the Creation Of a Support Device to the Development and Evaluation of Digital Competences in the 1st Cicle of Basic Education

This article presents the methodological path followed for the construction of a device to support the development and assessment of digital competences of students in the 1st Cycle of Basic Education. In addition to the epistemological foundations that support an interventionist approach, based on the pragmatic tradition and aligned with the principles of Design-Based Research (DBR), the purposes that justify the decisions taken with regard to the instrumentation and procedures for collecting and analysis of data that characterize the originality of the methodological path developed within the scope of a master's research, in the area of ​​Innovation in Education at the University of Lisbon. Despite the limitations experienced, it is concluded that the methodological path adopted, in addition to constituting a promising approach for the realization of more accessible products for their end users, may also inspire other researchers who are interested in advancing scientific knowledge in education through the use of appreciation of professional wisdom.
Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

Abstract The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (a) a form of Nonlinear Vector Autoregression (NVAR), and (b) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi‐Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures.
Evolutionary Experience-Driven Particle Swarm Optimization with Dynamic Searching

Particle swarm optimization (PSO) algorithms have been successfully used for various complex optimization problems. However, balancing the diversity and convergence is still a problem that requires continuous research. Therefore, an evolutionary experience-driven particle swarm optimization with dynamic searching (EEDSPSO) is proposed in this paper. For purpose of extracting the effective information during population evolution, an adaptive framework of evolutionary experience is presented. And based on this framework, an experience-based neighborhood topology adjustment (ENT) is used to control the size of the neighborhood range, thereby effectively keeping the diversity of population. Meanwhile, experience-based elite archive mechanism (EEA) adjusts the weights of elite particles in the late evolutionary stage, thus enhancing the convergence of the algorithm. In addition, a Gaussian crisscross learning strategy (GCL) adopts crosslearning method to further balance the diversity and convergence. Finally, extensive experiments use the CEC2013 and CEC2017. The experiment results show that EEDSPSO outperforms current excellent PSO variants.
Self-Prompting Perceptual Edge Learning for Dense Prediction

Numerous studies have employed prompt learning structures to enhance dense prediction tasks by integrating additional semantic or geometric information. While the inclusion of extra information has shown improvements in performance, it also poses challenges for applications that cannot provide extra input. To address this issue, this study evaluates the performance of different prompts and introduces an additional-input-free method, called self-prompting perceptual edge learning (SPPEL), which extracts edge-embedded semantic prompts directly from the image feature itself using trainable handcrafted edge operators within a plug-and-play module. To obtain the edge features, our approach incorporates an adversarial structure that compares the similarity between two edge features generated by the Hog and Kirsch operators, where the edge features are measured using multiplication, finetuned through a trainable all-one embedding, and enhanced with channel-to-channel attention. We conduct extensive evaluations of SPPEL on 7 tasks, utilizing 7 different backbones and applying 5 distinct methods. Our experimental results demonstrate that SPPEL achieves strong competitiveness in various settings with an average improvement of 1.7% across all 7 tasks, including ADE20K, COCO (Instance Segmentation), COCO (Object Detection), Pascal VOC2012, STARE, CHASE DB1, and HRF, while incurring a parameter increase of less than 3% (the detailed computation analysis of parameters and Gflops are shown in different experimental tables). Code will be released at: https://github.com/chenhao-zju/sppel.
Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generation

Understanding Arabic text and generating human-like responses is a challenging task. While many researchers have proposed models and solutions for individual problems, there is an acute shortage of a comprehensive Arabic natural language generation toolkit that is capable of handling a wide range of tasks. In this work, we present a robust Arabic text-to-text Transformer model, namely AraT5v2, methodically trained on extensive and diverse data, utilizing an extended sequence length of 2,048 tokens. We explore various pretraining strategies including unsupervised, supervised, and joint pertaining, under both single and multitask settings. Our models outperform competitive baselines with large margins. We take our work one step further by developing and publicly releasing OCTOPUS, a Python-based package and command-line toolkit tailored for eight Arabic generation tasks all exploiting a single model. We provide a link to the models and the toolkit through our public repository.
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages

Zheng Xin Yong, Ruochen Zhang, Jessica Forde, Skyler Wang, Arjun Subramonian, Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Lintang Sutawika, Jan Christian Blaise Cruz, Yin Lin Tan, Long Phan, Long Phan, Rowena Garcia, Thamar Solorio, Alham Aji. Proceedings of the 6th Workshop on Computational Approaches to Linguistic Code-Switching. 2023.
Emergent Linear Representations in World Models of Self-Supervised Sequence Models

How do sequence models represent their decision-making process? Prior work suggests that Othello-playing neural network learned nonlinear models of the board state (Li et al., 2023a). In this work, we provide evidence of a closely related linear representation of the board. In particular, we show that probing for “my colour” vs. “opponent’s colour” may be a simple yet powerful way to interpret the model’s internal state. This precise understanding of the internal representations allows us to control the model’s behaviour with simple vector arithmetic. Linear representations enable significant interpretability progress, which we demonstrate with further exploration of how the world model is computed.
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data

Chain-of-thought (CoT) advances the reasoning abilities of large language models (LLMs) and achieves superior performance in complex reasoning tasks. However, most CoT studies rely on carefully designed human-annotated rational chains to prompt LLMs, posing challenges for real-world applications where labeled data is available without rational chains. This paper proposes a new strategy, AutomateCoT (Automatic Prompt Augmentation and Selection with Chain-of-Thought), that can bypass human engineering of CoT by automatically augmenting rational chains from a small labeled dataset, and then pruning low-quality chains to construct a candidate pool of machinegenerated rationale chains based on the labels. Finally, it selects the optimal combination of several rationale chains from the pool for CoT prompting by employing a variance-reduced policy gradient strategy to estimate the significance of each example. Automate-CoT enables a quick adaptation of the CoT technique to different tasks. Experimental results demonstrate the effectiveness of our method, where competitive results are achieved on arithmetic reasoning (+2.7%), commonsense reasoning (+3.4%), symbolic reasoning (+3.2%), and non-reasoning tasks (+2.5%).
Query Rewriting in Retrieval-Augmented Large Language Models

Large Language Models (LLMs) play powerful, black-box readers in the retrieve-then-read pipeline, making remarkable progress in knowledge-intensive tasks. This work introduces a new framework, Rewrite-Retrieve-Read instead of the previous retrieve-then-read for the retrieval-augmented LLMs from the perspective of the query rewriting. Unlike prior studies focusing on adapting either the retriever or the reader, our approach pays attention to the adaptation of the search query itself, for there is inevitably a gap between the input text and the needed knowledge in retrieval. We first prompt an LLM to generate the query, then use a web search engine to retrieve contexts. Furthermore, to better align the query to the frozen modules, we propose a trainable scheme for our pipeline. A small language model is adopted as a trainable rewriter to cater to the black-box LLM reader. The rewriter is trained using the feedback of the LLM reader by reinforcement learning. Evaluation is conducted on downstream tasks, open-domain QA and multiple-choice QA. Experiments results show consistent performance improvement, indicating that our framework is proven effective and scalable, and brings a new framework for retrieval-augmented LLM.
Error Detection for Text-to-SQL Semantic Parsing

Despite remarkable progress in text-to-SQL semantic parsing in recent years, the performance of existing parsers is still far from perfect. Specifically, modern text-to-SQL parsers based on deep learning are often over-confident, thus casting doubt on their trustworthiness when deployed for real use. In this paper, we propose a parser-independent error detection model for text-to-SQL semantic parsing. Using a language model of code as its bedrock, we enhance our error detection model with graph neural networks that learn structural features of both natural language questions and SQL queries. We train our model on realistic parsing errors collected from a cross-domain setting, which leads to stronger generalization ability. Experiments with three strong text-to-SQL parsers featuring different decoding mechanisms show that our approach outperforms parser-dependent uncertainty metrics. Our model could also effectively improve the performance and usability of text-to-SQL semantic parsers regardless of their architectures.
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models

Large language models (LLMs) have been shown to possess impressive capabilities, while also raising crucial concerns about the faithfulness of their responses. A primary issue arising in this context is the management of (un)answerable queries by LLMs, which often results in hallucinatory behavior due to overconfidence. In this paper, we explore the behavior of LLMs when presented with (un)answerable queries. We ask: do models represent the fact that the question is (un)answerable when generating a hallucinatory answer? Our results show strong indications that such models encode the answerability of an input query, with the representation of the first decoded token often being a strong indicator. These findings shed new light on the spatial organization within the latent representations of LLMs, unveiling previously unexplored facets of these models. Moreover, they pave the way for the development of improved decoding techniques with better adherence to factual generation, particularly in scenarios where query (un)answerability is a concern.
Prompting is not a substitute for probability measurements in large language models

Prompting is now a dominant method for evaluating the linguistic knowledge of large language models (LLMs). While other methods directly read out models’ probability distributions over strings, prompting requires models to access this internal information by processing linguistic input, thereby implicitly testing a new type of emergent ability: metalinguistic judgment. In this study, we compare metalinguistic prompting and direct probability measurements as ways of measuring models’ linguistic knowledge. Broadly, we find that LLMs’ metalinguistic judgments are inferior to quantities directly derived from representations. Furthermore, consistency gets worse as the prompt query diverges from direct measurements of next-word probabilities. Our findings suggest that negative results relying on metalinguistic prompts cannot be taken as conclusive evidence that an LLM lacks a particular linguistic generalization. Our results also highlight the value that is lost with the move to closed APIs where access to probability distributions is limited.
Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation

Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses. Existing methods either manually annotate or employ LLM (e.g., GPT-series) to generate data for instruction tuning. However, they often overlook associating instructions with existing annotated datasets. In this paper, we propose Dynosaur, a dynamic growth paradigm for the automatic curation of instruction-tuning data. Based on the metadata of existing datasets, we use LLMs to automatically construct instruction-tuning data by identifying relevant data fields and generating appropriate instructions. By leveraging the existing annotated datasets, Dynosaur offers several advantages: 1) it reduces the API cost for generating instructions (e.g., it costs less than $12 USD by calling GPT-3.5-turbo for generating 800K instruction tuning samples; 2) it provides high-quality data for instruction tuning (e.g., it performs better than Alpaca and Flan on Super-NI and Longform with comparable data sizes); and 3) it supports the continuous improvement of models by generating instruction-tuning data when a new annotated dataset becomes available. We further investigate a continual learning scheme for learning with the ever-growing instruction-tuning dataset, and demonstrate that replaying tasks with diverse instruction embeddings not only helps mitigate forgetting issues but generalizes to unseen tasks better. Code and data are available at https://github.com/WadeYin9712/Dynosaur.
Investigating Efficiently Extending Transformers for Long Input Summarization

While large pretrained Transformer models have proven highly capable at tackling natural language tasks, handling long sequence inputs still poses a significant challenge. One such task is long input summarization, where inputs are longer than the maximum input context of most models. Through an extensive set of experiments, we investigate what model architectural changes and pretraining paradigms most efficiently adapt a pretrained Transformer for long input summarization. We find that a staggered, block-local Transformer with global encoder tokens strikes a good balance of performance and efficiency, and that an additional pretraining phase on long sequences meaningfully improves downstream summarization performance. Based on our findings, we introduce PEGASUS-X, an extension of the PEGASUS model with additional long input pretraining to handle inputs of up to 16K tokens, which achieves strong performance on long input summarization tasks comparable with much larger models.