Project: Prefect (Orchestration & Management / Scheduling & Orchestration)
File: host.md

description: Hosting Prefect Sever tags: UI dashboard Prefect Server Observability Events Serve Database SQLite search: boost: 2 Learn how to host your own Prefect server instance. !!! note If you would like to host a Prefect server instance on Kubernetes, check out the prefect-server . After installing Prefect, you have a Python SDK client that can communicate with , and an instance backed by a database and a UI. Spin up a local Prefect server UI with the `prefect server start` CLI command in the terminal: <div class="terminal"> ```bash prefect server start ``` </div> Open the URL for the Prefect server UI ( by default) in a browser. Shut down the Prefect server with <kdb> ctrl </kbd> + <kdb> c </kbd> in the terminal. A self-hosted Prefect server instance and Prefect Cloud share a common set of features. Prefect Cloud includes the following additional features: &mdash; isolated environments to organize your flows, deployments, and flow runs. &mdash; configure triggers, actions, and notifications in response to real-time monitoring events. &mdash; send email alerts from Prefect's servers based on automation triggers. &mdash; configure API access for running workers or executing flow runs on remote infrastructure. &mdash; assign users granular permissions to perform activities within an account or workspace. &mdash; authentication using your identity provider. &mdash; a record of user activities to monitor security and compliance. Read more about Prefect Cloud in the section. Go to your terminal session and run this command to set the API URL to point to a Prefect server instance: ```bash prefect config set PREFECTAPIURL="http://127.0.0.1:4200/api" ``` !!! tip "`PREFECTAPIURL` required when running Prefect inside a container" You must set the API server address to use Prefect within a container, such as a Docker container. You can save the API server address in a . Whenever that profile is active, the API endpoint is at that address. See for more information on profiles and configurable Prefect settings. The Prefect database persists data to track the state of your flow runs and related Prefect concepts, including: Flow run and task run state Run history Logs Deployments Flow and task run concurrency limits Storage blocks for flow and task results Variables Artifacts Work pool status Currently Prefect supports the following databases: SQLite (default in Prefect): Recommended for lightweight, single-server deployments. SQLite requires essentially no setup. PostgreSQL: Best for connecting to external databases, but requires additional setup (such as Docker). Prefect uses the extension, so it must be installed and

A local SQLite database is the default database and is configured upon Prefect installation. The database is located at `~/.prefect/prefect.db` by default. To reset your database, run the CLI command: <div class="terminal"> ```bash prefect server database reset -y ``` </div> This command clears all data and reapplies the schema. Prefect provides several settings for configuring the database. The default settings are: <div class="terminal"> ```bash PREFECTAPIDATABASECONNECTIONURL='sqlite+aiosqlite:///${PREFECT_HOME}/prefect.db' PREFECTAPIDATABASE_ECHO='False' PREFECTAPIDATABASEMIGRATEON_START='True' PREFECTAPIDATABASE_PASSWORD='None' ``` </div> Save a setting to your active Prefect profile with `prefect config set`. Connect Prefect to a PostgreSQL database by setting the following environment variable: <div class="terminal"> ```bash prefect config set PREFECTAPIDATABASECONNECTIONURL="postgresql+asyncpg://postgres:yourTopSecretPassword@localhost:5432/prefect" ``` </div> The above environment variable assumes: You have a username called `postgres` Your password is set to `yourTopSecretPassword` Your database runs on the same host as the Prefect server instance, `localhost` You use the default PostgreSQL port `5432` Your PostgreSQL instance has a database called `prefect` Quickly start a PostgreSQL instance to use as your Prefect database with the following command (which will start a Docker container running PostgreSQL): <div class="terminal"> ```bash docker run -d --name prefect-postgres -v prefectdb:/var/lib/postgresql/data -p 5432:5432 -e POSTGRESUSER=postgres -e POSTGRESPASSWORD=yourTopSecretPassword -e POSTGRES_DB=prefect postgres:latest ``` </div> The above command: Pulls the version of the official `postgres` Docker image, which is compatible with Prefect. Starts a container with the name `prefect-postgres`. Creates a database `prefect` with a user `postgres` and `yourTopSecretPassword` password. Mounts the PostgreSQL data to a Docker volume called `prefectdb` to provide persistence if you ever have to restart or rebuild that container. Run the command below to set your current Prefect Profile to the PostgreSQL database instance running in your Docker container. <div class="terminal"> ```bash prefect config set PREFECTAPIDATABASECONNECTIONURL="postgresql+asyncpg://postgres:yourTopSecretPassword@localhost:5432/prefect" ``` </div> Inspect your Prefect profile to confirm that the environment variable has been properly set: <div class="terminal"> ```bash prefect config view --show-sources ``` </div> <div class="terminal"> ```bash You should see output similar to the following: PREFECTPROFILE='myprofile' PREFECTAPIDATABASECONNECTIONURL='' (from profile) PREFECTAPIURL='http://127.0.0.1:4200/api' (from profile) ``` </div> Start the Prefect server to use your PostgreSQL database instance: <div class="terminal"> ```bash prefect server start ``` </div> To use an in-memory SQLite database, set the following environment variable: <div class="terminal"> ```bash prefect config set PREFECTAPIDATABASECONNECTIONURL="sqlite+aiosqlite:///file::memory:?cache=shared&uri=true&checksamethread=false" ``` </div> !!! warning "Use SQLite database for testing only" SQLite is only supported by Prefect for testing purposes and is not compatible with multiprocessing. Prefect uses to manage database migrations. Alembic is a database migration tool to use with the SQLAlchemy Database Toolkit for

Alembic provides a framework for generating and applying schema changes to a database. Apply migrations to your database with the following commands: To upgrade: <div class="terminal"> ```bash prefect server database upgrade -y ``` </div> To downgrade: <div class="terminal"> ```bash prefect server database downgrade -y ``` </div> Use the `-r` flag to specify a specific migration version to upgrade or downgrade to. For example, to downgrade to the previous migration version, run: <div class="terminal"> ```bash prefect server database downgrade -y -r -1 ``` </div> or to downgrade to a specific revision: <div class="terminal"> ```bash prefect server database downgrade -y -r d20618ce678e ``` </div> To downgrade all migrations, use the `base` revision. See the to create new database migrations. gives you access to a hosted platform with Workspace & User controls, Events, and Automations. Prefect Cloud has an option for automation notifications. The more limited Notifications option is provided for the self-hosted Prefect server. Notifications enable you to set up alerts that are sent when a flow enters any state you specify. When your flow and task runs changes , Prefect notes the state change and checks whether the new state matches any notification policies. If it does, a new notification is queued. Prefect supports sending notifications through: Custom webhook Discord webhook Mattermost webhook Microsoft Teams webhook Opsgenie webhook PagerDuty webhook Sendgrid email Slack webhook Twilio SMS !!! cloud-ad "Notifications in Prefect Cloud" Prefect Cloud uses the robust interface to enable notifications related to flow run state changes and work pool status. To configure a notification in a Prefect server, go to the Notifications page and select Create Notification or the + button. You can choose: Which run states should trigger a notification Tags to filter which flow runs are covered by the notification Whether to send an email, a Slack message, Microsoft Teams message, or use another services For email notifications (supported on Prefect Cloud only), the configuration requires email addresses to which the message is sent. For Slack notifications, the configuration requires webhook credentials for your Slack and the channel to which the message is sent. For example, to get a Slack message if a flow with a `daily-etl` tag fails, the notification will read: If a run of any flow with daily-etl tag enters a failed state, send a notification to my-slack-webhook When the conditions of the notification are triggered, youll receive a message: The fuzzy-leopard run of the daily-etl flow entered a failed state at 22-06-27 16:21:37 EST. On the Notifications page you can pause, edit, or delete any configured notification.