Project: Prefect (Orchestration & Management / Scheduling & Orchestration)
File: tasks.md

description: Learn the basics of writing tasks. tags: tutorial getting started basics tasks caching concurrency subflows A is any Python function decorated with a `@task` decorator. A flow is like a recipe for connecting a known sequence of tasks together. Tasks, and the dependencies between them, are displayed in the flow run graph, enabling you to break down a complex flow into something you can observe, understand, and control at a more granular level. When a function becomes a task, it can execute concurrently and you can cache its return value. Flows and tasks share some common features: Both are defined using their respective decorator, which accepts settings for that flow / task (see all / ). You can provide them with a `name`, `description` and `tags` for organization and bookkeeping. They provide functionality for retries, timeouts, and other hooks to handle failure and completion events. Network calls (such as our `GET` requests to the GitHub API) are particularly useful as tasks because they take advantage of task features such as , , and . !!! tip "You can call tasks from other tasks" As of `prefect 2.18.x`, you can call tasks from within other tasks. This removes the need to use subflows for simple task composition. !!! note "When to use tasks" Not all functions in a flow need be tasks. Use them only when their features are useful. Let's use the previous flow and move the request into a task: ```python hllines="2 5-9 15" title="repoinfo.py" import httpx from prefect import flow, task from typing import Optional @task def get_url(url: str, params: Optional[dict[str, any]] = None): response = httpx.get(url, params=params) response.raiseforstatus() return response.json() @flow(retries=3, retrydelayseconds=5, log_prints=True) def getrepoinfo(repo_name: str = "PrefectHQ/prefect"): url = f"https://api.github.com/repos/{repo_name}" repostats = geturl(url) print(f"{repo_name} repository statistics :") print(f"Stars : {repostats['stargazerscount']}") print(f"Forks : {repostats['forkscount']}") if name == "main": getrepoinfo() ``` Running the flow in your terminal will result in something like this: <div class="terminal"> ```bash 09:55:55.412 | INFO | prefect.engine - Created flow run 'great-ammonite' for flow 'get-repo-info' 09:55:55.499 | INFO | Flow run 'great-ammonite' - Created task run 'geturl-0' for task 'geturl' 09:55:55.500 | INFO | Flow run 'great-ammonite' - Executing 'get_url-0' immediately... 09:55:55.825 | INFO | Task run 'get_url-0' - Finished in state Completed() 09:55:55.827 | INFO | Flow run 'great-ammonite' - PrefectHQ/prefect repository statistics : 09:55:55.827 | INFO | Flow run 'great-ammonite' - Stars : 12157 09:55:55.827 | INFO | Flow run 'great-ammonite' - Forks : 1251 09:55:55.849 | INFO | Flow run 'great-ammonite' - Finished in state Completed('All states completed.') ``` </div> You can now see this task run tracked in the UI as well. Tasks support the ability to cache their return value. Caching allows you to efficiently reuse of tasks that may be expensive to reproduce with every flow run. You can also reuse cached results if the inputs to a task have not

To enable caching, specify a `cachekeyfn` a function that returns a cache key on your task. You may optionally provide a `cache_expiration` timedelta indicating when the cache expires. Define a task that is cached based on its inputs by using the Prefect . Let's add caching to the `get_url` task: ```python hl_lines="2 4 7" import httpx from datetime import timedelta from prefect import flow, task, getrunlogger from prefect.tasks import taskinputhash from typing import Optional @task(cachekeyfn=taskinputhash, cache_expiration=timedelta(hours=1), ) def get_url(url: str, params: Optional[dict[str, any]] = None): response = httpx.get(url, params=params) response.raiseforstatus() return response.json() ``` You can test this caching behavior by using a personal repository as your workflow parameter - give it a star, or remove a star and see how the output of this task changes (or doesn't) by running your flow multiple times. !!! warning "Task results and caching" Task results are cached in memory during a flow run and persisted to your home directory by default. Prefect Cloud only stores the cache key, not the data itself. Tasks enable concurrency, allowing you to execute multiple tasks asynchronously. This concurrency can greatly enhance the efficiency and performance of your workflows. Let's expand the script to calculate the average open issues per user. This requires making more requests: ```python hllines="14-24 30-31 35" title="repoinfo.py" import httpx from datetime import timedelta from prefect import flow, task from prefect.tasks import taskinputhash from typing import Optional @task(cachekeyfn=taskinputhash, cache_expiration=timedelta(hours=1)) def get_url(url: str, params: Optional[dict[str, any]] = None): response = httpx.get(url, params=params) response.raiseforstatus() return response.json() def getopenissues(reponame: str, openissuescount: int, perpage: int = 100): issues = [] pages = range(1, -(openissuescount // -per_page) + 1) for page in pages: issues.append( get_url( f"https://api.github.com/repos/{repo_name}/issues", params={"page": page, "perpage": perpage, "state": "open"}, ) ) return [i for p in issues for i in p] @flow(retries=3, retrydelayseconds=5, log_prints=True) def getrepoinfo(repo_name: str = "PrefectHQ/prefect"): repostats = geturl(f"https://api.github.com/repos/{repo_name}") issues = getopenissues(reponame, repostats["openissuescount"]) issuesperuser = len(issues) / len(set( for i in issues])) print(f"{repo_name} repository statistics :") print(f"Stars : {repostats['stargazerscount']}") print(f"Forks : {repostats['forkscount']}") print(f"Average open issues per user : {issuesperuser:.2f}") if name == "main": getrepoinfo() ``` Now you're fetching the data you need, but the requests are happening sequentially. Tasks expose a method that changes the execution from sequential to concurrent. In this example, you also need to use the method because you are unpacking a list of return values: ```python hl_lines="6 11" def getopenissues(reponame: str, openissuescount: int, perpage: int = 100): issues = [] pages = range(1, -(openissuescount // -per_page) + 1) for page in pages: issues.append( get_url.submit( f"https://api.github.com/repos/{repo_name}/issues", params={"page": page, "perpage": perpage, "state": "open"}, ) ) return [i for p in issues for i in p.result()] ``` The logs show that each task is running concurrently: <div class="terminal"> ```bash 12:45:28.241 | INFO | prefect.engine - Created flow run 'intrepid-coua' for flow 'get-repo-info' 12:45:28.311 | INFO | Flow run 'intrepid-coua' - Created task run 'geturl-0' for task 'geturl' 12:45:28.312 | INFO | Flow run 'intrepid-coua' - Executing 'get_url-0' immediately...

| INFO | Task run 'get_url-0' - Finished in state Completed() 12:45:28.583 | INFO | Flow run 'intrepid-coua' - Created task run 'geturl-1' for task 'geturl' 12:45:28.584 | INFO | Flow run 'intrepid-coua' - Submitted task run 'get_url-1' for execution. 12:45:28.594 | INFO | Flow run 'intrepid-coua' - Created task run 'geturl-2' for task 'geturl' 12:45:28.594 | INFO | Flow run 'intrepid-coua' - Submitted task run 'get_url-2' for execution. 12:45:28.609 | INFO | Flow run 'intrepid-coua' - Created task run 'geturl-4' for task 'geturl' 12:45:28.610 | INFO | Flow run 'intrepid-coua' - Submitted task run 'get_url-4' for execution. 12:45:28.624 | INFO | Flow run 'intrepid-coua' - Created task run 'geturl-5' for task 'geturl' 12:45:28.625 | INFO | Flow run 'intrepid-coua' - Submitted task run 'get_url-5' for execution. 12:45:28.640 | INFO | Flow run 'intrepid-coua' - Created task run 'geturl-6' for task 'geturl' 12:45:28.641 | INFO | Flow run 'intrepid-coua' - Submitted task run 'get_url-6' for execution. 12:45:28.708 | INFO | Flow run 'intrepid-coua' - Created task run 'geturl-3' for task 'geturl' 12:45:28.708 | INFO | Flow run 'intrepid-coua' - Submitted task run 'get_url-3' for execution. 12:45:29.096 | INFO | Task run 'get_url-6' - Finished in state Completed() 12:45:29.565 | INFO | Task run 'get_url-2' - Finished in state Completed() 12:45:29.721 | INFO | Task run 'get_url-5' - Finished in state Completed() 12:45:29.749 | INFO | Task run 'get_url-4' - Finished in state Completed() 12:45:29.801 | INFO | Task run 'get_url-3' - Finished in state Completed() 12:45:29.817 | INFO | Task run 'get_url-1' - Finished in state Completed() 12:45:29.820 | INFO | Flow run 'intrepid-coua' - PrefectHQ/prefect repository statistics : 12:45:29.820 | INFO | Flow run 'intrepid-coua' - Stars : 12159 12:45:29.821 | INFO | Flow run 'intrepid-coua' - Forks : 1251 Average open issues per user : 2.27 12:45:29.838 | INFO | Flow run 'intrepid-coua' - Finished in state Completed('All states completed.') ``` </div> In addition to calling tasks within a flow, you can also call other flows. Child flows are calledand allow you to efficiently manage, track, and version common multi-task logic. Subflows are a great way to organize your workflows and offer more visibility within the UI. Let's add a `flow` decorator to the `getopenissues` function: ```python hl_lines="1" @flow def getopenissues(reponame: str, openissuescount: int, perpage: int = 100): issues = [] pages = range(1, -(openissuescount // -per_page) + 1) for page in pages: issues.append( get_url.submit( f"https://api.github.com/repos/{repo_name}/issues", params={"page": page, "perpage": perpage, "state": "open"}, ) ) return [i for p in issues for i in p.result()] ``` Whenever you run the parent flow, the subflow is called and runs. In the UI each subflow run is linked to its parent and can be individually inspected. You now have a flow with tasks, subflows, retries, logging, caching, and concurrent execution. In the next section, you will deploy this flow to run it on a schedule and/or external infrastructure. Learn how to .