DOI: 10.5281/zenodo.17666438
title: How Type Hints are Used in Python Libraries and Frameworks: Early Insights

description: Dataset Overview

projects_discovery.zipIncludes all auxiliary files used during dataset construction and during the generation of the CSV files contained in repositories_dataset.zip.





clone_repos.py – Clones all repositories listed in top_repos.txt into the source directory. The official repository list used in the study is provided in top_repos_official.txt.




create_repo_links.py / extract_repo_links.py – Utility scripts responsible for normalizing, cleaning, and generating correct GitHub repository paths for Python projects.




filter_repos_annotations.py – Filters repositories based on a minimum threshold of type-annotation coverage. Only projects meeting or exceeding the specified annotation percentage are retained.




get_about.py – Extracts the “About” and descriptive metadata from GitHub repositories to support repository categorization.




merge_topics.py – Merges topic metadata from multiple CSV sources into a unified structure.



type_hints.zipContains all scripts responsible for extracting, aggregating, and processing type-annotation information.





type_extractor.py – Recursively scans all Python files inside each repository under repos/. For every file, it records all detected type annotations and outputs a CSV located at output/{repository}.csv. Each row contains:["file", "member_name", "type", "member_type", "context_code"].




type_extractor_metrics.py – Generates a per-repository summary from the extractor output. Produces a summary.csv including:["repo", "total", "typed_total", "total_typed_%", "variables_total", "variables_typed", "variables_typed_%", "parameters_total", "parameters_typed", "parameters_typed_%", "returns_total", "returns_typed", "returns_typed_%"].A final GLOBAL row aggregates all repositories to provide overall statistics.




strip_annotations.py – Produces multi-level code snippets (local, medium, full scope) for all annotated regions. Each repository receives a CSV containing the extracted snippets. These files are used for evaluating small LLMs on automatic type-inference tasks.




delimiter.py / drop_duplicates.py – Utility scripts for data cleaning, duplicate removal, and consistent CSV formatting.


pipeline.py - Contains a small pipeline where each snippet is inputted into a small size LLM model, deepseek-r1:7b in this case,  for automatic type inference performance.

type_summary.py - Type to extract and create a category for each type seem at each repository unique csv file.

add_category_column.py - Helper file to append the category column at the summary dataset.


resulting_dataset_sheets/Contains the primary consolidated CSV files used for analysis in the paper.





repos_merged.csv – Integrates GitHub repository URLs, classification (library vs. non-library), and descriptive metadata such as repository topics and “About” fields.




summary.csv – Merges all computed metrics required to answer each research question.



repositories_dataset/A reduced sample of the full 136-GB dataset. Each file corresponds to the output of type_extractor.py for one repository.Columns are:





file – File path of the Python source where the annotation was found.




member_name – Identifier of the function, variable, or member containing the annotation.




type – The explicitly declared type annotation.




context_code – The code snippet surrounding the annotation.



The complete dataset (not fully uploaded due to size constraints) contains the full set of extracted annotations across all analyzed repositories.