Package: dlt
data load tool (dlt) — the open-source Python library for data loadingBe it a Google Colab notebook, AWS Lambda function, an Airflow DAG, your local laptop,or a GPT-4 assisted development playground—dltcan be dropped in anywhere.🚀 Join our thriving community of likeminded developers and build the future together!Installationdlt supports Python 3.8+.pipinstalldltQuick StartLoad chess game data from chess.com API and save it in DuckDB:importdltfromdlt.sources.helpersimportrequests# Create a dlt pipeline that will load# chess player data to the DuckDB destinationpipeline=dlt.pipeline(pipeline_name='chess_pipeline',destination='duckdb',dataset_name='player_data')# Grab some player data from Chess.com APIdata=[]forplayerin['magnuscarlsen','rpragchess']:response=requests.get(f'https://api.chess.com/pub/player/{player}')response.raise_for_status()data.append(response.json())# Extract, normalize, and load the datapipeline.run(data,table_name='player')Try it out in ourColab DemoFeaturesAutomatic Schema:Data structure inspection and schema creation for the destination.Data Normalization:Consistent and verified data before loading.Seamless Integration:Colab, AWS Lambda, Airflow, and local environments.Scalable:Adapts to growing data needs in production.Easy Maintenance:Clear data pipeline structure for updates.Rapid Exploration:Quickly explore and gain insights from new data sources.Versatile Usage:Suitable for ad-hoc exploration to advanced loading infrastructures.Start in Seconds with CLI:Powerful CLI for managing, deploying and inspecting local pipelines.Incremental Loading:Load only new or changed data and avoid loading old records again.Open Source:Free and Apache 2.0 Licensed.Ready to use Sources and DestinationsExplore ready to use sources (e.g. Google Sheets) in theVerified Sources docsand supported destinations (e.g. DuckDB) in theDestinations docs.DocumentationFor detailed usage and configuration, please refer to theofficial documentation.ExamplesYou can find examples for various use cases in theexamplesfolder.Adding as dependencydltfollows the semantic versioning with theMAJOR.MINOR.PATCHpattern. Currently, we are usingpre-release versioningwith the major version being 0.minorversion change means breaking changespatchversion change means new features that should be backward compatibleany suffix change, e.g.,post10->post11, is considered a patchWe suggest that you allow onlypatchlevel updates automatically:Using theCompatible Release Specifier. For exampledlt~=0.3.10allows only versions>=0.3.10and less than<0.4Poetrycaret requirements. For example^0.3.10allows only versions>=0.3.10to<0.4Get InvolvedThe dlt project is quickly growing, and we're excited to have you join our community! Here's how you can get involved:Connect with the Community: Join other dlt users and contributors on ourSlackReport issues and suggest features: Please use theGitHub Issuesto report bugs or suggest new features. Before creating a new issue, make sure to search the tracker for possible duplicates and add a comment if you find one.Track progress of our work and our plans: Please check out ourpublic Github projectContribute Verified Sources: Contribute your custom sources to thedlt-hub/verified-sourcesto help other folks in handling their data tasks.Contribute code: Check out ourcontributing guidelinesfor information on how to make a pull request.Improve documentation: Help us enhance the dlt documentation.LicenseDLT is released under theApache 2.0 License.