App now retains original index following cleaning to allow for referring back to original data

Allowed for app running on AWS to use smaller embedding model and not to load representation LLM (due to size restrictions).

Only aggregate topics not 'other', allowed for minimum sentence length, default max_topics now will auto aggregate topics. Added Cognito Auth functionality (boto3 with AWS).

Can split passages into sentences. Improved embedding, LLM representation models, improved zero shot capabilities

Updated packages. Improve hierarchy vis. Better models - mixedbread and phi3. Now option to split texts into sentences before modelling.

Reduce outliers now more efficient and relabels with correct vectoriser. Default topic labels now tidier. Hiearchical topics outputs more useful for joining to df afterwards. Switched low resource reduction algorithm to UMAP as default is not good.

Should now parse custom regex correctly. Will now wipe previously created embeddings if 'low resource mode' option switched.

Allowed for uploading custom regex for cleaning. Fixed calculate all probabilities, reduce outliers. Added text tree for hierarchical modelling.

Upgraded to Gradio 4.16.0. Guide for converting to exe added.

Note about LLM not working now successfully added!

Some text changes. Fixed a couple of TF-IDF embeddings issues

Switched embeddings to low resource TF-IDF by default. Some text changes.

Added clean data options, improved re-representation options and visualisation. General format changes

Allowed for loading in external topic labels. A few visualisation modifications.

Lots of general fixes. New visualisations, fixed hierarchical vis for zero shot. Added calc all probabilities.

Changed Phi model to smaller StableLM 2 1.6. Fixed a None type detection error.

Disabled console logging as it was getting in the way of file load into the app

Switched embeddings model to BGE Small 1.5 as Jina seemed unable to do zero shot topic modelling properly

Added minimum similarity slider for zero shot topic modelling

model and hierarchy details should now save properly

Split off LLM representation, visualisation, and reduce outliers from main function. Added hierarchical visualisation and logs

More efficient embeddings save and representations load/process. Custom visualisation hover option added, formatting improvements. Version 0.1?

App should now check if embeddings are loaded before topic modelling. And will save only once.

Added option to reduce outliers based on closest topic

Returned TruncatedSVD components to 100 - higher values don't seem to help

Greatly increased low resource process dimensions for higher quality. Visualisations disabled by default to increase speed.

Greatly improved low resource mode speed (at cost of potential quality)

Changed zero shot min similarity to 0.5

Added controls for saving topic models and visualisation. Removed custom UMAP layer

Now should save embeddings by default. Added random seed to representation

Fixed llm_config, environmental variable, zero shot topic model errors with quick embeddings

Model export changed to safetensors. Improved representational model function. Got zero shot topic modelling working

Removed reference to environmental variable

first commit