Venue: NAACL 2024

# Automating the Generation of a Functional Semantic Types Ontology with

Foundational Models

 Sachin Konan

Two Sigma Investments, LP

sachin@twosigma.com &Scott Affens

Two Sigma Investments, LP

scott.affens@twosigma.com &Larry Rudolph

Two Sigma Investments, LP

rudolph@csail.mit.edu

###### Abstract

The rise of data science, the inherent diritiness of data, and the proliferation of vast data providers have increased the value proposition of Semantic Types. Semantic Types encode contextual information onto a data schema, informing the user about the definitional meaning of data, its broader context, and relationships to other types. Previous work focuses on the recognition of statically-defined types, but what happens when a type-set isn't known apriori and how do we connect Semantic Types to downstream use-cases? FSTO_Gen_ addresses these questions by leveraging LLMs to generate Functional Semantic Types that include normalization, validation, and casting code to efficiently automate human-intensive data tasks.

## 1 Introduction

Onboarding datasets at scale is human-intensive because recognizing semantics about how data was generated, its bounds or other idiosyncrasies is critical to how it is processed and eventually used. Normalization requires understanding the form/representation as it is ingested, as well as the target form used by other datasets or people in an organization. Much of the effort involved in semantic understanding can be automated when values are typed with something richer than the basic primitives (e.g. integer, float, or strings types). In fields like finance, medicine, or broadly-spanning AI systems, new data is constantly being added, and automation can defray some of the ingestion cost.

The underlying pain-point in data onboarding stems from humans inconsistently naming tables and columns without knowing who may use them. Even when using data from a high-quality source [25], the consumer often invests significant human capital to develop to normalize and map data to a canonical representation as there are many challenges associated with normalizing columnar tables at scale. In addition to handling null/not-a-number values, inconsistent data formats, or out-of-distribution values, it is critical to understand the units of the values, which maybe implicit (e.g. dollars when giving the price of housing in the US), written in prose (e.g. revenue may be in millions of dollars as noted in the text of a 10k filing), or easily inferred (e.g. a summer temperature of 100 degrees is Fahrenheit and not Kelvin or Celsius). Additionally, to compare, join, or group values from columns in different tables, it may be necessary to recast values (e.g. convert a 12-hour am/pm time value to a 24-hour clock value or a country code to a country name). Identifying these semantics is behind the reliance on humans during onboarding.

We introduce _Functional Semantic Types_ (FST) to automatically annotate columnar data with Semantic Types and provide a library of functional attributes to normalize, validate, and cast real data values. FSTs are placed in a synthetically generated ontology, which directly maps columnar datasets to their hierarchically organized FSTS (this process is FSTO_-_Gen_). The generation of FSTs relies upon Large Language Models (LLMs)[5] to assign and generate a Python class definition that contains semantic details and functional characteristics about the data. By their training breadth, LLMs offer a general solution for entity recognition [8], because they can leverage the distributional properties/real values of data, tabular metadata, and data dictionaries to construct more accurate type annotations. Furthermore, LLMs have shown the ability to generate code, and therefore are capable of transforming semantic context into functional attributes.

Our work takes advantage of the natural hierarchical organization of tabular data. We refer to a collection of tables as a product. The information within a product is assumed to be initially constructed, maintained, and labeled by the same community of interest. As such, columns and tables, as well as context, are correlated. Our system is likely to generate the same FST for multiple columns be

longing to the tables within a product. The automatic generation of semantic types saves human labor, and even more so when multiple columns are assigned the same type. The system verifies this by merging a subset of values from these columns and checking that they all pass the same validation test.

At the final stage, we identify common FSTs across different products. First semantic types with the same name are agglomerated. Then all the uniquely named FSTs are turned into a graph by finding semantically similar FSTs in representation space and generating _cross-type-cast_ functions to transfer data values between FSTs. Human verification shows that this first attempt identifies communities of semantically-_identical_ entities.

There have been many other efforts to automatically assign semantic types to columnar data, but with some major differences (see Section 2). They assume the existence of an ontology or knowledge graph, do not build a bespoke ontology that is used for cross-type casting, and most of all they do not generate the functions that define the Semantic Type. Hence, the contributions of this work are:

* Automatically generating _Functional Semantic Type_ Python class definitions with fields and functional methods that characterize, transform, and validate columnar data values.
* Aggregating commonly named FSTs across products, and generating conversion code.
* Demonstrating success in real-world collections of data and evaluating the functionality and correctness of each of these ontologies.
* Showing that generated ontologies have utility in downstream data discovery, joining, validation, and normalization applications.

## 2 Related Works

The association of columnar tables with entities has been previously treated as a multi-class prediction problem over some user-defined distribution of entity types/properties. Methods such as Sherlock[12], SATO[29], DoDuo[26], and TableGPT[8] use 78 semantic types described by the T2Dv2 Gold Standard[4] which matches properties from the DBpedia ontology with column headers from the WebTables corpus. AutoType[28] made predictions over 112 manually procured types that spanned different industries. However, the types used in these works are often too broad for industry-specific datasets (e.g. in finance, EBITA is a commonplace term, but missing from DBPedia[1] and WordNet[6] knowledge graphs).

We summarize the related works along several dimensions (although TableGPT[8] and Foundational Models[21] cover nearly all of these). _Table Question and Answer:_ Table Cell Identification[27], Semantic Parsing[22], TabFact[3]. _Row-to-Row Transformation:_ TDE[9]. _Entity Matching Between Rows:_ Ditto[17], Deep Entity Matching[20], Auto-EM[31]. _Schema Matching Between Tables:_ Valentine[14], SMAT[30]. _Data Imputation:_ DataWig[2], Eracer[18], IMP[19], HoloClean[24].

A trend that spans most of these works is the success of LLMs as natural language processing engines for directly operating on real data values. Archetype[7] showed that LLMs are performant zero-shot annotators of semantic types across various domains. Additionally, LLMs have shown success in code generation tasks [16], specifically, for data processing and ingestion coding [11; 15]. Our work builds on these efforts but is unique in that it directly attaches any functional normalization during the entity recognition process (encapsulated in a FST), and hierarchically groups FSTs to build an ontology that identifies semantically identical entities across products. Additionally, unlike previous work in semantic type annotation, we do not specify the set of types beforehand; the aggregation of generated types across a universe of data becomes the type set (App. A.7).

## 3 Problem Formulation

Our FST system is applied to a _universe_ of data, hierarchically composed of _tables_ and _products_. A product consists of at least one columnar table. Columns, tables, and products all have labels. The tables within a product are assumed to have some informational similarity. In addition, our set of FSTs can either be a subclass of DatasetSemanticType (Fig. 1), or of GenericSemanticType (Fig. 2). DatasetSemanticType FSTs correspond to the standard types of data: numeric (with/without units), boolean, strings, and categorical. Examining the values in a column, as well as its relation to other columns, is all that is needed to make this type of assignment. In addition to a human-readable name, its definition includes descriptive characteristics about the type's semantics, domain, and example values, as well as a func 

tional transformation from raw data to normalized, types. GenericSemanticTypes collate identically-named ColumnSemanticTypes across products by applying a standardized normalization procedure that undergoes self-validation (see Fig. 2). The FSTs generated at the table and product levels are called T-FSTs and P-FSTs (each a subclass of DatasetSemanticType ), while those generated at the universe level are called G-FSTs (subclass of GenericSemanticType).

**Goal**: Given a table, extract the underlying semantic entities (if any) per column and generate a DatasetSemanticType with a descriptive class name, instance fields, and a cast() that transforms a single columnar value to the format dictated by the class. Commonly named FSTs across products are used to generate a subclass of a GenericSemanticType that merges the semantics of all the input DatasetSemanticType definitions, consolidates their transform logic into a single cast(), and evaluates the correctness with validate(). Some G-FSTs may be convertible, so cross_type_cast()'s transform their values. Unlike traditional ontologies, the result is a synthetic ontology where _the hierarchy between_T-FST, P-FST, _and_G-FST_ represent increasing layers of transformational generality_. Edges between G-FSTs represent a knowledge graph, but whose edges are not just semantic relationships, but functional transformations.

FST Ontology Generation (Fsto-Gen) Step 1: table\(\rightarrow\) T-FST - For each table in our universe, an LLM is provided with a serialized format of the table (App. A.1.1) and identifies the _subset_ of columns that correspond to semantic entities that might be more broadly applicable to other datasets. For each identified column, the LLM generates the corresponding T-FST subclasses and a mapping from column name to subclass. Abstract syntax trees are used to parse the output string and store any class definition and this mapping dictionary. In our experiments, the LLM tends to create identical subclass names (but not always identical fields) for columns in the same product (App. A.1.2).

* There exists many identical T-FSTs within a product, so we agglomerate identically-named ones into a single P-FST (App. A.2). For a given T-FST group, the unique columnar values spanned by the T-FSTs are aggregated and tested via the cast() to assess the number of values that pass and the number that changed. The T-FST that achieves the max criteria is selected as the P-FST for the group (App. A.2).****Step 3: P-FST \(\rightarrow\) G-FST (general)
- Across products there may exist identically-named, but functionally/seemantically different P-FSTs. The LLM understand the differences in each P-FST based in all available information to generate a G-FST whose super_cast() handles the output of each P-FST'scast(). Validity or sanity checks of the values is achieved by performing two consecutive transformations at the product and general levels. The validate() is responsible for performing type, bound, or value-based checks on the output, and is where the LLM use external lookups to establish a ground truth (App. A.3).

* There exist many G-FSTs that may represent identical (differently named) or distinct entities, that may be castable. For a given source G-FST, the \(k\)-nearest G-FSTs neighbors is identified by vectorizing each G-FST using an embedding model (App.A.4.1). An LLM determines the subset of the \(k\) neighbors that are convertible and for each, it generates a cross_type_cast() that transforms any output of the source G-FST to a value that would be accepted by the neighbors validate() (App. A.4.2).**

## 5 Experimental Evaluation

We evaluated the P-FSTs, G-FSTs, and cross_type_cast()'s on three data universes, two of which are freely available to the public (as well as all our code and prompts) with results shown here and code definitions in the Appendix.

Figure 1: The subclasses of DatasetSemanticType Each class has specific and inherited instance fields, as well as an implementation of the cast().

 

- Our universes source from two open-source data providers, Kaggle and Harvard Dataverse, as well as a commercial financial dataset, we refer to by a fictional name FData. For Kaggle, we selected the 707 most commonly used Kaggle-_datasets_ and extracted their associated tabular data files (files and datasets are represented as tables and products in our nomenclature). For Harvard, we selected the top 484 most downloaded datasets and organized them similarly to Kaggle, but Harvard required additional preparation due to the large number of fully null columns (App. A.6). FDA consists of 428 tables containing a wide breadth of financial terminology. Table 1 shows the aggregated details of each universe. Notice that Kaggle contains the largest number of tables, products, and columns while Harvard and FData contain differing levels of product granularity and table widths, which are both factors that affected the performance of the product and general stages.

* We used OpenAI's gpt-4-0613 LLM with 8k context for the table, general, and cross stages of FSTO-_Gen_.

* To evaluate the functionality of FSTO-_Gen_, we considered the throughput performance of each pipeline stage. Our analysis of the functional characteristics of each stage provides intuition about the behavior of each transformation; however, to demonstrate correctness, a human evaluation of the P-FSTs and cross_type_cast()'s was needed to assess the LLM's ability to select subsets of information (relevant columns that refer to semantic entities in table or relevant semantically similar general FSTs that are castable), as well generate accurate code.

For each P-FST, we iterated over each child FST, sampled 1000 values from the columns covered by the FST, and aggregated a unique set. For each \(x\) in this set, P-FST's cast(\(x\)) gives a value \(w\). For the G-FST parent of P-FST, G-FST's super_cast(\(w\)) gives a value \(y\), which is validated using G-FST's validate(). Then for a neighboring G-FST, cross_type_cast(\(y\)) yields a value \(z\), which we validated in the neighbor's validate(). For each cast function (cast(), super_cast(), cross_type_cast()), we record if it passes (indicated by ) or errors out (\(\chi\)). The result of a cast can either be Complex (differs from the original), Identity (identical to the original), or an Exception (erroneous input or cast logic).

### Functional Throughput Results

**Column \(\rightarrow\)P-Fst:** The results are summarized in Table 2. Non-null values mostly pass through unchanged, i.e. the data is already well-formatted and fits the canonical form of the P-FST. Some values change after casting, indicating normalization was necessary. Complex transformations range from rounding floats (App. A.8.1, A.8.2) to mapping

Figure 3: Unshaded rectangles represent columns, shaded rectangles represent T-FSTs, triangles represent P-FSTs, and shaded circles represent G-FSTs. Each color represents a specific semantic entity (i.e. ZipCode, City, State, etc). Circles at the general level are the generalization of the P-FSTs. At the cross level is a graph representing the relationship between G-FSTs.

 

country abbreviations to full names with lookup tables (App. A.8.3). A few times, a RunTimeError in the cast() is thrown.

P-Fst\({}^{*}\)@g-FST:The results are summarized in Table 3. In general, values are left unchanged, indicating that data already achieved normalization in the P-FST, which is expected considering that there are many 1-1 correspondences between P-FFTs and G-FSTs (per Table 1). When the super_cast() is a Complex transformation, this indicates that different communities of interest (in this case products) have differing, yet locally standardized ways of representing the same data (App. A.8.4). In some of these cases, the validate() fails, indicating insufficient normalization in the P-FST or G-FST, incorrect validate(), or deeper insights are needed into the domain restriction of the G-FST (App. A.8.5). However, more frequently, Complex transforms pass validate() (App. A.8.6 A.7), indicating that LLMs can derive a common standard for a large variety of entity types at scale.

G-Fst\({}^{*}\)@g-Fst:We found (Table 4) many semantically _identical_ entities that differed only by class name (App: A.8.8). The generation of T-FSTs is a generative process with no source of truth, and since LLMs are stochastic, it is likely to name identical semantic entities with slightly differing naming conventions. An artifact of generating ontologies from the bottom-up is that entities at the most general level may be too specific or too broad; however, this is the fundamental purpose of the cross_type_cast(). While these FSTs differ by name, they shouldn't differ in their semantics (and therefore are close in the vectorized representation space of their corresponding classes). The next most common outcomes were Complex transforms, where we witnessed nontrivial behavior involving lookups, mappings, etc.(App. A.8.9, A.8.10), as well as Identity mappings that failed the neighbor's validate() (App. A.8.11). We attribute these cases to hallucinations, whereby the LLM will justify a cross_type_cast() with faulty logic.

### Human Evaluation Results

As there is no ground truth, we performed a human evaluation to assess _quality_ and _correctness_. We evaluated P-FSTs using 50 tables from Kaggle, and 20 tables each fromHarvard and FData for a total of \(\bar{1}000\) columns. Humans were tasked with labeling whether a column should have an FST associated with it. Our results show (Figure 4) that LLMs have high precision and recall in _recognizing semantic entities_ and were able to _generate correctly-scoped_, and functionally correct FSTs.

We counted when P-FSTs were either too broad or too specific (Table 5). For those with correct scope, we labeled cases of when the generated class completely differe

\begin{table}
\begin{tabular}{|c|c|l|c|c|c|c|} \hline \multirow{2}{*}{Col Entry} & Cast & Return & Validate & \multicolumn{3}{c|}{Universe} \\  & \multirow{2}{*}{Func} & \multirow{2}{*}{Val} & \multicolumn{1}{c|}{Func} & **Kaggle** & **Harvard** & **FData** \\ \cline{5-8}  & & & & Fast & 14.26\% & 18.34\% & 11.01\% \\ \multirow{3}{*}{Non-Null} & \multirow{3}{*}{✓} & \multirow{3}{*}{Complex} & \multicolumn{1}{c|}{Fail} & 0.95\% & 2.30\% & 0.97\% \\ \cline{3-8}  & & & Pass & 77.30\% & 63.55\% & 78.75\% \\ \cline{3-8}  & & & & Fall & 7.16\% & 14.69\% & 7.58\% \\ \cline{3-8}  & \(\chi\) & Exception & & Fall & 0.29\% & 0.45\% & 0.20\% \\ \hline \multirow{3}{*}{Null} & \multirow{3}{*}{✓} & \multirow{3}{*}{Complex} & \multicolumn{1}{c|}{Pass} & 0.00\% & 0.22\% & 1.52\% \\ \cline{3-8}  & & & & Fall & 0.00\% & 0.40\% & 1.67\% \\ \cline{3-8}  & & & & Pass & 0.00\% & 0.00\% & 0.00\% \\ \cline{3-8}  & & & & Fall & 0.00\% & 0.00\% & 0.00\% \\ \cline{3-8}  & \(\chi\) & Exception & & Fall & 0.00\% & 0.00\% & 0.00\% \\ \hline \end{tabular}
\end{table}
Table 4: The distribution of outcomes after the application of cross_type_cast() between general FSTs and the target’s validate(). Values tend to undergo Identity transformation, indicating the existence of semantically-duplicative FSTs.

Figure 4: Confusion Matrices of the LLM’s ability to recognize and generate P-FSTs. We witness high True Positive, low False Positive, and low False Negative rates, indicating both high precision and recall.

 

the entity (Totally Incorrect), contained slightly erroneous fields or functional attributes (Slightly Wrong), or was (Just Right). In Kaggle and Harvard, these types were generally right, while in FData they were either too broad or perfect. We hypothesize LLMs tend to generalize unfamiliar concepts (e.g. it labeled a finance-specific growth rate with "Growth Rate"), making any cast or validation ineffectual. Finally, even when P-FSTs were scoped properly, some contained errors such as mismatches between its name or description or made a false assertion (App. A.8.11).

To assess the quality of the cross_type_cast(), we sampled 20 G-FSTs from each universe, as well as its \(k\)=20 neighbors in representation space. For each neighbor, we recorded whether there should exist a cross_type_cast() between the source and destination. This served as a ground-truth comparison to the edges present in the generated ontology, with which we classified if the LLM was able to recognize when two G-FSTs were truly cross-type-castle. As seen in Fig 5, we witnessed acceptable recall on Kaggle and Harvard, low recall on FData, and low precision across all universes. The reasons for low recall on FData are related to the analysis in Table 5: when an LLM can't recognize semantic entities, it's difficult to assess whether a cross-type-cast is allowed. Additionally, because LLMs tend to hallucinate, the false positive rate was large across all universes, especially for FData, which is an artifact of the generality of the FSTs. LLMs found cross-type-casts between types like "Percent" and "Ratio", which might refer to entirely different entities, but when generalized, hallucinations become more likely.

## 6 Cost Analysis

FTO-Gen utilizes OpenAI's GPT-4 LLM to perform Steps 1, 3, and 4, which attaches external and human-in-the-loop costs. During Step 1, there is one LLM call per table in the Universe, and during Steps 3 and 4, another single LLM call per G-FST. The execution time for each call is a summation of the time to send the request + the inference time of LLM + the time to receive the response payload. However, since inferences operate on independent tables or G-FSTs, we batched our calls into groups of \(24\) (number of cores on our machine). This makes FSTO-Gen extensible to larger universes, pursuant to increased parallelism. Finally, since the response may sometimes contain errors in its construction or run-time dependencies that weren't explicitly detailed in the prompt, human correction was necessary. A human adjusted these outputs to ensure compilability. These were minimal relative to the number of LLM calls (Table 6).

## 7 Applications

* Data Normalization is baked into the construction of our ontology. The cast() in T-FSTs, super_cast() in G-FSTs, and cross_type_cast()'s represent normalization layers for typing a columnar value. Listing 1 shows the super_cast() generated for a G-FST for "Income-Level". Across tables, Income-Level can be represented as a range or a specific amount,

Figure 5: Confusion Matrices of LLM’s ability to recognize true, castable relationships between G–FSTs. We witness lower levels of recall and precision, due to the LLMs hallucination.

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|} \hline  & \multicolumn{2}{c|}{Incorrect Scope} & \multicolumn{2}{c|}{Correct Scope} \\ \cline{2-6} Universes & Too Broad & Too Specific & Toatlored & Signbly & Just \\ \hline
**Kaggle** & 15.44/ & 1.52x & 2.78x & 5.06x & 75.19x \\ \hline
**Harvard** & 17.54/ & 0.88x & 1.32x & 0.44x & 79.82 \\ \hline
**FDAa** & 49.07x & 0x & 3.27x & 1.87x & 45.79x \\ \hline \end{tabular}
\end{table}
Table 5: Quality Distribution of True Positive, product-level FSTs. The LLM generates bytes that are too broad, or perfect.

 

and the aggregation process results in a single all-inclusive transformation covering both cases.

```
defsuper_cast(self,val): ifisinstance(val,str): ifval=='lessThen5000';return0.0 elifval='5001800*1000*1retran750a0.0 elifval=='10002-2000*1retnum1500.0 elifval=='MoreThan20000':return2000.6 else:raiseException('Invalidincome level') elifisinstance(val,{int,float}): ifval>n:returnfloat(val) else:raiseException('Invalidincomelevel') else:raiseException('Invalidincomelevel')
```

Listing 1: The super_cast() of G-FST, "IncomeLevel," which normalizes range or number inputs to a number.

* Validation of a semantic entity generally requires knowledge about the distributional properties of a column or worldly knowledge about the domain set of the entity. For accelerometer data, the LLM asserted bounds between \([-1,1]\) (App. A.8.1). Additionally, there is robustness in generating fault-tolerant T-FSTS, such as the "Precipitation" T-FST (App. A.8.2. Precipitation data is usually numeric, but there were several occurrences of the letter 'T'. Leveraging worldly knowledge resulted in assigning 'T' to a value of 0 as it generally refers to trace amounts of rainfall. Finally, when dealing with entities with large domain sets, LLMs use external lookup tables, (e.g. the set of US States configured in pycountry) to assert the correctness of state names to their two-letter abbreviation (Listing 2). An interesting avenue for future work is utilizing validation to perform better normalization. While we provide a summary report and a subset of values, these may not be enough to construct robust validation over an entire column, so an iterative approach that couples feedback from normalization may be used to improve validation.

```
defvalidate(self,val): casted_val=self.super_cast(val) us_states=pycountry.subdivisions.get('US') forstateinlist(us_states): ifstate.name.title()==casted_val: returnTrue
```

Listing 2: The validate() of G-FST, "USState," which uses pycountry to checks the validity of super_cast().

* Data Fusion is automatically unlocked with the graph construction of the product and general stages of FSTO-_Gen_ to join tables on the common columnar entity. In general, any two tables can be merged using a common P-FST or G-FST ancestor, and the explicitness of the ancestor-child relationships in FSTO-_Gen_ reduces the complexity of finding a semantic match. Additionally, the construction of cross_type_cast()'s allow for non-trivial joins between tables. For example, the LLM generated a cross_type_cast() to convert education level from string to integer enums by mapping their domains (App. A.8.9).

* Discovery is achieved by allowing practitioners to semantically search over the informational/functional properties of FSTs or traverse the relationships in the synthetic ontology. For example, FSTO-_Gen_ generated a FST for lifeexpectancy that spanned four products, two of which came from the "WHO-world-health-statistics-2020" product where a column was named "First ToolTip", and one from "Barcelona-data-sets" product where columns associated with life-expectancy were named by time-ranges. It is doubtful that someone searching for data related to life-expectancy would have known that these products would be related, but automated contextualization with LLMs combined with the hierarchical composition of FSTO-_Gen_ found these relations.

## 8 Conclusion and Limitations

FSTO-_Gen_ is an LLM-powered framework for automating the generation of G-FSTs and their relations from columnar data. Our synthetic ontology is hierarchical and functional, such that successive layers represent the transformation of semantically identical columns into a singular representation. These ontologies are useful in automating various data onboarding tasks: normalization, validation, fusion, and discovery. FSTO-_Gen_ is less successful with domain-specific datasets, because LLMs are less familiar with terminology, leading to incorrectly scoped FSTs. We are exploring fine-tuning [10] and retrieval-augmented-generation [13] to overcome this gap. However, even in its current state, FSTO-_Gen_ shows promise in reducing the tedious job of data onboarding.

Figure 6: Two Data Tables, belonging to different products, contain columns associated with the semantic entity “LifeExpectancy”, but neither are explicitly named. The LLM uses table context to perform this association and FSTO-_Gen_ identifies the relation as a graph.

 



## References

* Auer et al. [2007] Soren Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyaniak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In _international semantic web conference_, pages 722-735. Springer.
* Biessmann et al. [2019] Felix Biessmann, Tammo Rukat, Phillipp Schmidt, Prathik Naidu, Sebastian Schelter, Andrey Taptunov, Dustin Lange, and David Salinas. 2019. Datawig: Missing value imputation for tables. _Journal of Machine Learning Research_, 20(175):1-6.
* Chen et al. [2019] Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2019. Tabfact: A large-scale dataset for table-based fact verification. _arXiv preprint arXiv:1909.02164_.
* Cremaschi et al. [2020] Marco Cremaschi, Flavio De Paoli, Anisa Rula, and Blerina Spahiu. 2020. A fully automated approach to a complete semantic table interpretation. _Future Generation Computer Systems_, 112:478-500.
* Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. _arXiv preprint arXiv:1810.04805_.
* Fellbaum [2010] Christiane Fellbaum. 2010. Wordnet. In _Theory and applications of ontology: computer applications_, pages 231-243. Springer.
* Feuer et al. [2023] Benjamin Feuer, Yurong Liu, Chinmay Hegde, and Juliana Freire. 2023. Archetype: A novel framework for open-source column type annotation using large language models.
* Gong et al. [2020] Heng Gong, Yawei Sun, Xiaocheng Feng, Bing Qin, Wei Bi, Xiaojiang Liu, and Ting Liu. 2020. Tablegpt: Few-shot table-to-text generation with table structure reconstruction and content matching. In _Proceedings of the 28th International Conference on Computational Linguistics_, pages 1978-1988.
* He et al. [2018] Yeye He, Xu Chu, Kris Ganjam, Yudian Zheng, Vivek Narasiya, and Surajit Chaudhuri. 2018. Transform-data-by-example (tde) an extensible search engine for data transformations. _Proceedings of the VLDB Endowment_, 11(10):1165-1177.
* Hu et al. [2021] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.
* Huang et al. [2022] Junjie Huang, Chenglong Wang, Jipeng Zhang, Cong Yan, Haotian Cui, Jeevana Priya Inala, Colin Clement, Nan Duan, and Jianfeng Gao. 2022. Execution-based evaluation for data science code generation models. _arXiv preprint arXiv:2211.09374_.
* Hulsebos et al. [2019] Madelon Hulsebos, Kevin Hu, Michiel Bakker, Emanuel Zgragen, Arvind Satyanarayan, Tim Kraska, Cagatay Demiralp, and Cesar Haldago. 2019. Sherlock: A deep learning approach to semantic data type detection. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 1500-1508.
* Khattab et al. [2023] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Varadhaman, Saiful Hajq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazman, Heather Miller, Matei Zaharia, and Christopher Potts. 2023. DSPy: Compiling declarative language model calls into self-improving pipelines. _arXiv preprint arXiv:2310.03714_.
* Koutras et al. [2021] Christos Koutras, George Siachamis, Andrea Ionescu, Kyriakos Psarakis, Jerry Brons, Marios Fragkoulis, Christoph Lofi, Angela Bonifati, and Asterios Katsifodimos. 2021. Valentine: Evaluating matching techniques for dataset discovery. In _2021 IEEE 37th International Conference on Data Engineering (ICDE)_, pages 468-479. IEEE.
* Lai et al. [2023] Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. Ds-1000: A natural and reliable benchmark for data science code generation. In _International Conference on Machine Learning_, pages 18319-18345. PMLR.
* Li et al. [2022] Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Remi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode. _Science_, 378(6624):1092-1097.
* Li et al. [2020] Yuliang Li, Jingfeng Li, Yoshihiko Suhara, AnHai Doan, and Wang-Chiew Tan. 2020. Deep entity matching with pre-trained language models. _arXiv preprint arXiv:2004.00584_.
* Mayfield et al. [2010] Chris Mayfield, Jennifer Neville, and Sunil Prabhakar. 2010. Fracer: a database approach for statistical inference and data cleaning. In _Proceedings of the 2010 ACM SIGMOD International Conference on Management of data_, pages 75-86.
* Mei et al. [2021] Yinnan Mei, Shaoxu Song, Chenguang Fang, Haifeng Yang, Jingyun Fang, and Jiang Long. 2021. Capturing semantics for imputation with pre-trained language models. In _2021 IEEE 37th International Conference on Data Engineering (ICDE)_, pages 61-72. IEEE.
* Mudgal et al. [2018] Sidharth Mudgal, Han Li, Theodoros Rekatsinas, AnHai Doan, Youngchoon Park, Ganesh Krishnan, Rohit Deep, Esteban Arcaute, and Vijay Raghavendra. 2018. Deep learning for entity matching: A design space exploration. In _Proceedings of the 2018 International Conference on Management of Data_, pages 19-34.
* Narayan et al. [2022] Avanka Narayan, Ines Chami, Laurel Orr, and Christopher Re. 2022. Can foundation models wrangle your data? _arXiv preprint arXiv:2205.09911_.
* Pasupat and Liang [2015] Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. _arXiv preprint arXiv:1508.00305_

* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. _arXiv preprint arXiv:1908.10084_.
* Rekatsinas et al. (2017) Theodoros Rekatsinas, Xu Chu, Ihab F Ilyas, and Christopher Re. 2017. Holocenean: Holistic data repairs with probabilistic inference. _arXiv preprint arXiv:1702.00820_.
* aws data exchange. Https://aws.amazon.com/data-exchange.
* Suhara et al. (2022) Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang, Cagatay Demiralp, Chen Chen, and Wang-Chiew Tan. 2022. Annotating columns with pre-trained language models. In _Proceedings of the 2022 International Conference on Management of Data_, SIGMOD '22, page 1493-1503, New York, NY, USA. Association for Computing Machinery.
* Sun et al. (2016) Huan Sun, Hao Ma, Xiaodong He, Wen-tau Yih, Yu Su, and Xifeng Yan. 2016. Table cell search for question answering. In _Proceedings of the 25th International Conference on World Wide Web_, pages 771-782.
* Yan and He (2018) Cong Yan and Yeye He. 2018. Synthesizing type-detection logic for rich semantic data types using open-source code. In _Proceedings of the 2018 International Conference on Management of Data_, SIGMOD '18, page 35-50, New York, NY, USA. Association for Computing Machinery.
* Zhang et al. (2020) Dan Zhang, Madelon Hulsebos, Yoshihiko Suhara, Cagatay Demiralp, Jingfeng Li, and Wang-Chiew Tan. 2020. Sato: contextual semantic type detection in tables. _Proc. VLDB Endow._, 13(12):1835-1848.
* Zhang et al. (2021) Jing Zhang, Bongsun Shin, Jinho D Choi, and Joyce C Ho. 2021. Smat: An attention-based deep learning solution to the automation of schema matching. In _Advances in Databases and Information Systems: 25th European Conference, ADBIS 2021, Tartu, Estonia, August 24-26, 2021, Proceedings 25_, pages 260-274. Springer.
* Zhao and He (2019) Chen Zhao and Yeye He. 2019. Auto-em: End-to-end fuzzy entity-matching using pre-trained deep models and transfer learning. In _The World Wide Web Conference_, pages 2413-2424.

 

Appendix

### Step 1: (table\(\rightarrow\)T-Fst)

#### a.1.1 Table Serialization

In step 1 of FSTO_Gen_, an LLM is tasked with converting a table into a mapping dictionary which maps some subset of the tables' columns to FSTs names and the definition of FSTs themselves. The motivation for this scheme sources from the serialization process in Sherlock[12] and DoDuo[26], which showed that semantic type annotation is enhanced when the annotation engine can understand table-level and column-level semantics. In fact, in our experiments, there were many cases where columns were badly named, but table/product names helped the LLM perform inference. The problem with table serialization is fitting it into the context window of the LLM, so given a table as a pandas dataframe, we convert it to a string using serialize() as defined in Listing 3.

```
importpandassaspd defserialize(df:pd.DataFrame,data_dict: dict[str,str])->str; string="forcolindfocolsums:is_numeric=...#checkifcolis numeric/categorical quartile_1,median,quartile_3= df['col'].quantitile([C.25,0.5,0.75]) ifis_numerical:string=""-col:(col) *description:(data_dict[col]) *mean:(df[col].mean()) *std:(df[col].std()) *min:(df[col].ind()) *0.25:(quartile_1) *0.5:(median) *0.75:(quartile_3) *max:(df[col].max()) *first_five:{df[col].iloe[C:S].values} *num_na:(df[col].isna().sum()) ==!else: string+=""-col:(col) *description:(data_dict[col]) *num_na:(len[df[col].unique())) *top_5.most.freement: (df[col].value_counts().nlargest(5)) *num_na:(df[col].isna().sum()) returnString
```

Listing 3: serialize() function which converts a pandas dataframe table to a string.

#### a.1.2 LLM Prompt

Below we denote the prompt (denoted in the typewriter font) used to generate a set of T-FSTs and their mappings to columnar names for a particular table. The prompt is formatted using Jinja templating ([https://jinja.palletsprojects.com/en/3.1.x/](https://jinja.palletsprojects.com/en/3.1.x/)) which is materialized using the following inputs:

1. base_class_definitions - T-fST Base Classes as seen in Fig. 1
2. numeric_col_summary - App. A.1.1
3. categorical_col_summary - App. A.1.1
4. end_to_end_example - End to End Example with column summary and resultant T-FSTs and mapping.
5. dataset_name - Name of the table
6. dataset_description - Description of the table or empty string if it doesn't exist
7. column_dict - App. A.1.1
8. necessary_imports - App. A.5

``` <START_PROMPT> YouareSemanticGPTan assistantthatidentifiesSemanticTypesforttabularata.SemanticTypesaredatatypesintangrainsemanticcontextintoanetity.SemanticTypesarevaluablebecausetheyrestrictthedomainwithwhichaccolumncanspan,meaningthatSemanticTypeshaveafixedomainofvaluesandformat.HerearePythonbaseclassSemanticTypedefinitions:{(base_class_definitions)} Iamgoingtoprovideyouwiththemameofthedataset,alongwithaburbhenatedlistofthecolumnsenclosedin'',aswellascertainpropertiespereachcolumnthatrepresentasummaryofallcolumns.Ifthecolumnisinterredtobenumeric,thenupillobeintheformof:{(categorical_col_summary)}.Thecommoniseinferredtocbnon-numeric,theinputwillbecinfocomsoft:{(categorical_col_summary)}.Yourgoalistoreadthroughthecolumnsmumaryandtrytofigureout1)ifthereexistsapSemanticTypeforagivencolumn2)Ifyohaven'talreadyconstructedapSemanticTypesdefinitionforthecolumn,constructonethatinheritstheMOSTSPECIFICclassdefinitionfromtheprovidedbaseclass 

definitions. 3) Assign each column to the constructed Semantic Type, using a dictionary. Note, columns may be mapped to the same Semantic Type or not at all (I expect there to be a small set of constructed Semantic Types). To aid in this process, I created a decision-tree i want you to follow:

 - Does the column relate to a semantic type?  - YES: should the column only take in two values?  - YES: BooleanSemanticType  - NO: does the column represent  a numerical semantic type?  - YES:  NumericSemanticTypeWithUnits  || NumericSemanticType  - CategoricalEnumSemanticType  || CategoricalSemanticType  - NO: do nothing

{{end_to_end_example}}. Now I want you to generate the corresponding Python SemanticType definitions for the given table. It is of UPMOST importance that your code compiles and runs. Do not add any extra text, "", or "python" prefixes. I just want the class definitions and the Mapping dictionary. Also the class names should be real-world entities and spelled correctly. For any string column, think about how to extract a uniform representation in the cast() function. - SUPER IMPORTANT: I will provide you with the list of libraries to start with, don't import anything else. Just start writing the classes definitions and Mapping dictionary. Make sure your class definition names don't conflict with the imports. COLUMNS="" dataset_name:{{dataset_name}} {{dataset_description}} {{column_dict}}  #! {(necessary_imports}) <END_PROMPT>

### Step 2: (T-Fst \(\rightarrow\) P-Fst) Details

```
importnumpyasnp defagglenerate(tfsts:list[DataSetSemanticType],  unique_set:list[Any]):  mat = np.zeros((len(tfsts), 2))  for ix in range[len(trts)]:  num_passes = 0  run_changes = 0  fork x unique_set:  try:  y = tfsts[ix].cast(x)  num_passes += 1  num_changes = ; ly = x except Exception as e:  pass  nat[ix] = [num_passes, num_changes] max_ix = mat[int:, 0] == mat[:, 0].max()].argmax() returnfts[max_ix]
```

Listing 4: The agglomerate function is used to combine identically-named T-FFSTs (belonging to the same data-product) into a single P-FST. Since our ontology increases in functional generalization, we pick the T-FST with the most general cast() as the P-FST.

After Step 1 of FSTO-_Gen_, for each product, there exist many T-FFSTs with the same name, which tend to also have the same semantics, so we perform an agglomeration of them into a single P-FST to reduce the number of LLM calls at the general stage. Given a grouping G of T-FFSTs, we ideally would perform an agglomeration similar to the general stage, because while the T-FFSTs have the same name, they may still have different fields or cross_type_cast()'s. However, this is computationally expensive across the many groupings in the product stage, so instead we pick the most functionally general T-FST that performs meaningful normalization on the columnar input. We seek to maximize throughput through the cast(), so we pick the T-FST that throws the least errors and performs the most Complex transformation on a set of columnar values. To assemble this set, we sample 1000 values from each column associated with a T-FST and create a unique set. We show this in the agglomerate() (Listing 4) which takes in a grouping of T-FFSTs and a unique set of values, and generates a P-FST.

### Step 3: (P-Fst \(\rightarrow\) G-Fst) Prompt

Below we denote the prompt used to generate a single G-FST from a list of P-FFSTs with a common name. We use the following variables:

1. general_sem_type_class_def - Fig. 2
2. end_to_end_example - Example where we provide a list of P-FFSTs and an ideal G-FST 

3. class_defs - P-FSTs to aggregate
4. necessary_imports - App. A.5

<START_PROMPT> You are GroupGPT, an assistant that will receive a list of Python class definitions and return a single class that spans them all. For context, each class represents a Semantic Type, which is a real-world entity that corresponds to some piece of columnar data I have collected. Each type has its own attributes, specified formatting, and acast() function that takes as input a single value from the column and returns a formatted, casted value. You will populate the following class using the instructions from the comments: {(general_sem_type_class_def}). - For super_cast(), I want your code to be ROBUST, and handle ALL of the outputs generated by the cast() of the provided classes. - For validate(), I want your code to sanity-check that the value is correct. - MOST IMPORTANT: Make sure that your solution compiles and will execute when I instantiate the class. - A useful strategy for "picking" a canonical format, is to pick the format of ONE class, and convert any output from the cast() of the provided classes to that format.

{{end_to_end_example}}.

I want you to output a single class that INHERITS GenericSemanticType (but change the class name). Do not add any extra text, "" or "python" prefixes. The class names should be real-world entities, spelled correctly, and SHOULD BE LOWERcase. - SUPER IMPORTANT: I will provide you with the list of libraries to start with, don't import anything else. Just start writing the classes definitions and Mapping dictionary. Make sure your class definition names don't conflict with the imports. CLASSES = ""

{{class_defs}} RETURN= {{necessary_imports}} <END_PROMPT>

### Step 4: **(6-fst\(\rightarrow\)6-fest)**

#### a.4.1 Algorithm Details

There exist many G-FSTs that are semantically similar, or even identical, so to identify joins across products, we generate cross_type_cast()'s between similar G-FSTs. First, we serialize each G-FST into a string by concatenating the class name with its description instance field. Then for each string we vectorize the class using an embeddings model (we use the all-MiniLM-L6-v2 model[23]) to convert the model into a 384-length vector and find the nearest \(k\) neighbors using kNN (we use \(k=20\) to reduce the number of tokens in the cross LLM prompt). For each G-FST, we concatenate the cross prompt with the G-FST and each of its 20 neighbors to receive a maximum of 20 output cross_type_cast()'s.

#### a.4.2 LLM Prompt

Below we denote the prompt used to generate up to \(k\)cross_type_cast()'s. We use the following variables in the prompt:

1. len_targets - \(k\) G-FSTs to compare to
2. simple_example - Example where we provide a list of P-FSTs and an ideal G-FST.
3. class_defs - P-FSTs to aggregate
4. necessary_imports - App. A.5
5. simple_partial_example - Partial Example of two castable G-FSTs.
6. full_example - Full Example with input class definitions and generated cross_type_cast()'s.
7. partial_example_1 - Partial Example that shows a true positive
8. partial_example_2 - Partial Example that shows a false positive
9. partial_example_3 - Partial Example that shows a false positive
10. src_class_def - Source G-FST
11. target_class_defs - Target G-FST 

* <START_PROMPT> You are CastGPT, an agent that will help me convert between two Semantic Type Class Definitions. These class definitions have a super_cast() method, which converts a value to the class's canonical format, and a validate() method which sanity-checks the result of the validate() method. Given a root class definition and {{len_targets}} target class definitions, I want you to generate at MOST {{len_targets}} functions. For example, given root class a and target class b you will generate a method called cross_type_cast_between_a_and_b(val). This is how it will be used:

"

casted_a_val = a().super_cast(val) casted_b_val = cross_type_cast_between_a_and_b( casted_val ) b().validate(casted_b_val) "" There are two main challenges here. The first is that you need to figure out if class a and class b represent the same type of information, and whether the result of a().super_cast(val) can be casted to the form/function described by b().super_cast(val). If that is possible, then you need to generate the right python mapping code to perform the conversion of a().super_cast(val) to the format of b().

{(simple_partial_example)} The full form of the function is defined as follows. For each (a,b) pairing that is valid (maximum {{len_targets}}), I want you to generate: {{cross_type_cast_def}}.

{{full_example}} {{partial_example_1}} {{partial_example_2}} {{partial_example_3}}

Now I want you to try on the following examples. Like the example, generate the cross_type_cast() functions according to the template I gave you, don't give me anything else but code! - DO NOT generate an empty cross_type_cast() function, just skip it. - Also, if you need bizzare mapping code, DO NOT generate a cross_type_cast() function. - I want you to be EXTREMELY conservative with your conversions. There shouldn't be a lot of conversions that work, because only small numbers of entities actually represent the same type of information. - SUPER IMPORTANT: I will provide you with the list of libraries to start with, don't import anything else. SOURCE="" {src_class_def} "

TARGETS="' {target_class_defs}} " FUNCTIONS = "" {necessary_imports}} <END_PROMPT>

### FST allowed Imports

To perform data manipulation tasks and lookups, in the declaration of each FST, we allowed the following set of imports:

1. numpy - to perform array manipulation.
2. pandas - to handle na/null values.
3.datetime - to perform date string operations.
4. math - to perform rounding.
5. pycountry/countryinfo - to perform geographic lookups.

### Data Curation Details

The Kaggle dataverse was sourced from the top 1000 most downloaded Kaggle Datasets, and we extracted the ".csv" files present into each dataset. Each Kaggle dataset was termed as a product, while the ".csv" files as a table. Across all universes, we selected tables that had at least 80% of columns with at least more than one-null value. From the set of 1000, we selected the top 707. In the end, 

there were 237 products with an average of 3 tables per product. We performed a similar process for the Harvard dataverse, except we selected the top 500 by filtering on whether the datasets contained "tab" files (tabular data files) and if they were released publicly. Additionally, Harvard datasets are generally mapped directly to a single tabular file, so to enhance the product stage we categorized datasets by their subject tag, which could be any one of: "agriculturalsciences, business-sandmanagement, earthandenvironmentalsciences, law, socialsciences, artsandhumanities, chemistry, engineering, mathematicalsciences, astronomy/adaprophysics, computeranformationscience, medicinehealthndilliesciences". With a small number of products, there is less opportunity for aggregation to occur at the product stage as columns are less likely to have semantically similar information in wider groupings. This explains the greater decrease in the number of types from FSTs to P-FSTs in Kaggle versus Harvard (Table 1). The FData universe is a commercial, proprietary universe and its details can't be revealed under confidentiality agreements.

### Type Distribution

In Fig. 7 we show the distribution of G-FSTs, sorted by usage across data products, in Kaggle and Harvard. Notice how the most frequently occurring G-FSTs are common across both Kaggle and Harvard, for example, "age", "date", "year", "gender", "latitude", "longitude", "country", "education", "currency", etc. are repeated. The aforementioned G-FSTs are canonical examples of Semantic Types, however as you move closer to the tails of the histograms, there are more dataset-specific types like "confirmedcases", "salary", "treatmenttype", or "speciesname". The middle and right-tail of this distribution uncovers domain-specific datatypes, and the benefit of FSTO-_Gen_ is that these types can be discovered through automation rather than a human exhaustively iterating through a universe.

### Examples

#### BodyAcceleration

```
class bodyacceleration(NumericSennanticTypeWithUnits): def__init__(self,args,..kwargs): self.description='The mean body accelerationincertaindirection' self.valid_range=[-1.@,1.@] self.dtype=float self.format='Bodyaccelerationshouldbe afloatingpointnumberbetween-1and1 self.units*:'Theunitofbodyaccelerationislg,where$istheaccelerationduetogravity' self.examples=[-1.@,-0.5,0.@,0.5,1.@] defcast(self,val): num=float(val) ifnum<-1.@ornum>1.@: raiseException(~Invalidbodyacceleration') returnround(num,6)
```

Listing 5: Body Acceleration P-FST

This P-FST (Listing 5) was generated from the "human-activity-recognition-with-smartphones" product from Kaggle. This FST represents accelerometer values, and the generated cast() function will float-cast and round the number. However, these values are stored as strings and contain various rounding conventions. The cast() standardizes the number of decimal points to 6, and converts all strings to floats. Additionally, using its understanding of accelerometer data, the LLM it assigned a unit of "IgG" for acceleration. It also used the min/max values from App. A.1.1 to create bounds. These may not be right, but these are rules enforced by the data and the LLM's

Figure 7: Distribution of types and their usage across data products (num_across_dps) and data tables (num_across_table) in Kaggle and Harvard universes. We constructed a histogram of type usage and sorted the G-FSTs from high to low usage across data products. The top 50 are shown in the above histograms.

 

knowledge about accelerometer values.

#### a.8.2 Precipitation

```
classprecipitation(NumericScenanticTypeWithunits): def__init_(self,args,**kwargs): self.description='Precipitationlevelsin inches'self.valid_range=[0,float('inf')] self.dtype=float self.format='Precipitationshouldbeatfloitingpointnumberindicinesofprecipitation.'self.units='Inches'self.examples=[0,0.254,0.508,0.762,1.016] defcast(self,val): ifval=='T': return0.0returnround(float(val),3)
```

Listing 6: Precipitation P-FST

In the construction of a _precipitation_ FST (Listing 6) in the "weatherww2" product from Kaggle, the product contains weather information during World War 2, and each table contains data relative to specific geographic locations. A column, named "precip.", consists of a mixture of floating-point values and a single letter 'T'. Without context, it could be confusing to any user of the data, or even cause failures in any downstream pipelines that rely upon the column being floating-point. The table and product stages of FSTO-_Gen_ identified a few important features: 1) the table corresponded to U.S. weather conditions 2) "precip." refers to "precipitation" 3) precipitation in the U.S. is measured in inches, so it sub-classed a NumericeSemanticTypeWithUnitsandadded "inches" as a unit 4) identified the existence of a value 'T', which canonically refers to when _trace_ amounts of rain occur and replaced 'T' with 0. Encapsulated in an FST, this type allows any user to understand how it was processed.

#### a.8.3 NationalityName

Each table in the "fifa-22-complete-player-dataset" product contains information about individual players in the soccer videogame, FIFA 2022. One type that arose in this product was a P-FST (Listing 7) that represents the _nationality_ of a soccer player, and the generated cast() function checks that the country is valid by using a lookup from the pycountry library. It produced an Exception for the value "China PR", which is the name of the soccer team, not a nationality name.

#### a.8.4 Latitude

These two T-FSTS (Listing 8) were generated in Harvard's "earthandenvironmentalsciences" product, and merged during the product stage of FSTO-_Gen_. This example shows how table-level generation can produce a class with the same name, but slightly different semantics. The first class signifies that the _latitude_ corresponds to a loss event, while the second class refers to the most general notion of _latitude_ and contains a bound check within its cast().

#### a.8.5 Duration

The duration G-FST in Listing 9 represents the temporal _duration_ between events and correctly throws an error when the value is less than 0, indicating that an event was in the past. Generally, the duration between events is seen as a scalar, regardless of whether the event was passed in or not, 

so this function failed validate() on the column value of "-27.83". Whether or not this is the correct behavior, alerts any user of this G-FST about how negative numbers are being handled and allows them to alter the behavior.

#### a.8.6 Gender

``` classgender(GeneralSemanticType): def__init_(self,args,**kwargs): self.description='Agender' self.format.='Inlower^caseandasasing string' self.examples=['male',female',male',female',male'] defsupercast(self,val): str_val=str(val).lower() ifstr_valin<'male','female','m','f','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|','|',''|','|','|',''|','|',''|','|',''|','|',''|',''|','|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|',''|','''|',''|',''|','''|',''|','''|',''|','''|',''|','''|',''|','''|',''|','''|',''|','''|','''|',''|','''|' 

classconfirmedcases(GeneralSemanticType):  def__init_(self,args,**kwargss): self.description="Numberofconfirmed COVID-19cases'  self.format="Thenumbershowdbbeaconnnegativeinteger.

classcovidacases(GeneralSemanticType):  def__init_(self,args,**kwargss): self.description="COVID-19Cases'  positiveinteger,representingthenumberofcases' classcasescount(GeneralSemanticType):  def__init_(self,args,**kwargss): self.description="CountofcovID-19cases'  self.format="Countofexcessshouldbecantimegertwithhocdecimalplaces' classuncases(GeneralSemanticType):  def__init_(self,args,**kwargss): self.description="NumberofcovID-19 cases'  self.format="Numberofcasesshouldbdeaconpositiveinteger' classeducation(GeneralSemanticType):  def__init_(self,args,**kwargss): self.description="Levelofeducation'  self.formats="Incapitalizedstringfor'  self.examples=['secondary/SecondarySpecial','HigherEducatient','Incompletehigher','LowerSecondary','AcademicDegree'] classnothereduction(GeneralSemanticType):  def__init_(self,args,**kwargss): self.description="Educationlevelofthestudet'smother'  self.format="Reductionlevelshouldbecantimeger'  self.examples=[@,1,2,3,4] defcross_type_cast_between_education_} and_nothereduction(val):  reason="BoththeEducationaland  motherreductionclassesrepresentthemsame  realworldentity,whichishedebateculequivalent.However,theyrepresentthis  informationindifferentformats. The educationclassrepresentseducatio  levelsasstrings,whilenothereduction  representingthemasintegers.Wecanmap  thestringrepresentationtotheinteger  representationbyidentifyingkeywords in  the stringthatcorrespondtdifferent integervalues.' mapping<"{"noeducation';@,'educationlevel1';1,'educationlevel2';2,'educationlevel3';3,'educationlevel4';4'educationlevel5';5,'educationlevel6';6,'highereducation';7} returnmapping.get(val.lower(),@) ```
Listing 13: Conversion Between Two Categorical Enum G-FSTs representing education (only the name and its description and format fields are shown for brevity).

#### a.8.9 Nontrivial Education super_cast()

In Listing 13, we show a Complex transformation between two G-FSTs, where the source represents education as a set of enum strings, while the latter represent it as numbers. Using the set of unique values in the former and the range bounds in the latter, the LLM generates a cross_type_cast() that works on real-data values. The reasoning string was generated by the LLM to justify its behavior.

#### a.8.10 Nontrivial Currency super_cast()
``` classcurrencyvalue(GeneralSemanticType): def__init_(self,args,**kwargs): self.description="AUSD currencyvalue'  self.format="CurrencyValueshouldbeaconfloatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $4397164978.0,49124417794.0,  $55224891.0,1918358283.0] classescurrencyint(GeneralSemanticType):  def__init_(self,args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $4397164978.0,49124417794.0,  $55224891.0,1918358283.0]  classcurrencyint(GeneralSemanticType):  def__init_(self,args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyshouldbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self,**args,**kwargs): self.description="CurrencyvalueinIN'  self.format="Currencyallowbecarrepresentedasaf floatingpointnumber'  self.examples=[\(\tt 113524789243.0,  $5497164978.0,49124417794.0,  $55224891.0,1918358283.0] classcurrencyint(GeneralSemanticType):  def__init_(self, 

the LLM and performing a consensus, or using more examples in the prompt, could help alleviate these issues.