Package: categorical-gpt
CategoricalGPTWhen analyzing heterogeneous data comprising numerical and categorical attributes, it is common to treat the different data types separately or transform the categorical attributes to numerical ones. The transformation has the advantage of facilitating an integrated multi-variate analysis of all attributes. We propose a novel technique for transforming categorical data into interpretable numerical feature vectors using Large Language Models (LLMs). The LLMs are used to identify the categorical attributes' main characteristics and assign numerical values to these characteristics, thus generating a multi-dimensional feature vector. The transformation can be computed fully automatically, but due to the interpretability of the characteristics, it can also be adjusted intuitively by an end user. We provide a respective interactive tool that aims to validate and possibly improve the AI-generated outputs.
Having transformed a categorical attribute, we propose novel methods for ordering and color-coding the categories based on the similarities of the feature vectors.InstallationInstall via PIPpipinstallcategorical-gptor install from sourcepipinstall-e.Getting StartedThis guide gives you an introduction on how to use CategoricalGPT to perform an automatic (unsupervised) data transformation.Define a LLM DriverThis package utilizes thellmmodule to interact with different LLMs.
You can define the model parameters for each step of the pipeline. Look up thellmdocumentation for more information on how to install new models and which parameters each model can use.Additionally, you can define a cache path. Prompts to the given LLM can be cached to avoid unnecessary API calls.fromcategorical_gptimportLLMDriverllm_driver=LLMDriver('gpt-3.5-turbo',api_key='sk-123456789abc',cache_path='path/to/optional/cache',model_params={"characteristics":{"n":1,"temperature":0.9,"top_p":1},"heuristic":{"n":1,"temperature":0.7,"top_p":1},"apply_heuristic":{"n":1,"temperature":0.01,"top_p":0.1}})Run the transformationfromcategorical_gptimportCategoricalGPT# First define a LLM driver.llm_driver=...# Initialize the CategoricalGPT object with the driver, category name and category optionscategory_name='Car Brand'category_options=['Mercedes','Ferrari','Nissan','Kia','Ford','BMW','Bugatti']cgpt=CategoricalGPT(llm_driver=llm_driver,category_name=category_name,options=category_options)# Perform the full transfromation in one go. You can assign the model parameters for each step of the pipeline.feature_vectors=cgpt.transform(characteristic_model_params={"n":1,"temperature":0.7,"top_p":1},heuristic_model_params={"n":1,"temperature":0.7,"top_p":1},apply_heuristic_model_params={"n":1,"temperature":0.01,"top_p":0.1})More granular transformationThe above approach shows a single function-call to perform the transformation. You can also use it more granular as follows:fromcategorical_gptimportCategoricalGPTllm_driver=...# Initialize the CategoricalGPT object with the driver, category name and category optionscgpt=CategoricalGPT(llm_driver=llm_driver,category_name='Car Brand',options=['Mercedes','Ferrari','Nissan','Kia','Ford','BMW','Bugatti'])# Load the characteristics and the frequencies (relative), given the number n of calls to the APIcharacteristics,characteristic_frequencies=cgpt.get_characteristics(n=1,temperature=0.7,top_p=1,model="gpt-3.5-turbo",prevent_cache=False)# For each characteristic, we are going to load the heuristic and the value assignments.forcharacteristicincharacteristics:# Load the heuristic for the given characteristic and also get information about the (relative) frequencies depending on the number n of calls to the APIheuristic,heuristic_frequencies=cgpt.get_heuristic(characteristic=characteristic,n=1,temperature=0.7,top_p=1,model="gpt-3.5-turbo",prevent_cache=False)# Given the heuristic and characteristic, load the value assignments for all category options.# If n > 1, we return the mean value of all calls to the API. The statistics consists of (value_stds, value_max_diffs).value_assignments,value_assignment_statistics=cgpt.apply_heuristic(characteristic=characteristic,heuristic=heuristic)# After loading all required data, we can create the feature vectorfeature_vectors=cgpt.make_feature_vector()Visualization applications for the transformed dataGiven a set of transformedfeature_vectors, we can apply some approaches that are useful for visualization.EmbeddingWe can perform a Multidimensional Scaling (MDS) embedding on the given feature vectors. Possible distance metrics are['euclidean', 'cosine', 'pearson']. Possible normalization strategies are['minmax', 'meanstd', 'custom']. For the optioncustom, you have define it like this:custom:min_val,max_val, wheremin_valandmax_valare the numerical values for the normalization.fromcategorical_gptimportCategoricalGPT,make_mds_embeddingcgpt=CategoricalGPT(...)# Make sure that you have a fully loaded cgpt instance with the feature vectors built.mds,mds_eigen,distance_matrix=make_mds_embedding(cgpt,distance_metric='euclidean',n_dim=3,tolist=False,normalization='minmax')Color-mapThe color-map functionality assigns colors to each category option. The colors are assinged such that similar options share a similar color.fromcategorical_gptimportCategoricalGPT,color_codingcgpt=CategoricalGPT(...)# Make sure that you have a fully loaded cgpt instance with the feature vectors built.color_mapping=color_coding(cgpt)OrderingWe can order the category options, so that category options that are similar are ordered close to each other. We provide multiple methods on how to create ordering:['mds', 'tsp', 'tsp+mds', 'umap', 'lexicographic']fromcategorical_gptimportCategoricalGPT,ordered_optionscgpt=CategoricalGPT(...)# Make sure that you have a fully loaded cgpt instance with the feature vectors built.ordered_options=ordered_options(cgpt,ordering_method='tsp+mds',embedding_distance_metric='euclidean')Fine-tuning GUIIn order to supervise the transformatin process, we offer a graphical user interface that can be run via a webserver (Flask).
See theFlaskdocumentation for more information.fromcategorical_gptimportstart_guillm_driver=...# See Flask documentation for all available parametersstart_gui(llm_driver,debug=True,threaded=True,host="0.0.0.0",port=5001)This will start up the webserver, that can be visited viahttp://localhost:5001/. Full guide for the GUI can be found here.