> ## Documentation Index > Fetch the complete documentation index at: https://prefect-bd373955-pytest-markdown.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Run Flows with Docker > Learn how to store your flow code in a Docker image and serve your flow on any docker-compatible infrastructure. The [Deployments](/3.0rc/deploy/serve-flows/) guide showed how to serve a flow on a schedule. With a Python script, you can build a Docker image for that script, allowing you to serve your flow This guide demonstrates running a flow in Kubernetes, but you can use any Docker-compatible infrastructure. This guide covers how to: * Write a Dockerfile to build an image that stores your Prefect flow code. * Build a Docker image for your flow. * Deploy and run your Docker image on a Kubernetes cluster. * Look at the Prefect-maintained Docker images and discuss options for use. In this guide you will create a Dockerfile from scratch. Alternatively, Prefect makes it convenient to build a Docker image as part of deployment creation. You can even include environment variables and specify additional Python packages to install at runtime. If creating a deployment with a `prefect.yaml` file, the build step makes it easy to customize your Docker image and push it to the registry of your choice. See an [example](/3.0rc/deploy/dynamic-infra/deploy-kubernetes/#define-a-deployment). Deployment creation with a Python script that includes `flow.deploy` similarly allows you to customize your Docker image with keyword arguments as shown below. ```python ... if __name__ == "__main__": hello_world.deploy( name="my-first-deployment", work_pool_name="above-ground", image='my_registry/hello_world:demo', job_variables={"env": { "EXTRA_PIP_PACKAGES": "boto3" } } ) ``` ## Prerequisites * A Python script that defines and serves a flow. This guide uses the flow script and deployment from the [Deployments](/3.0rc/deploy/serve-flows/) guide. * Access to a running Prefect API server. Sign up for a forever free [Prefect Cloud account](https://docs.prefect.io/cloud/), or run a Prefect API server locally with `prefect server start`. * [Docker Desktop](https://docs.docker.com/desktop/) installed on your machine. ## Write a Dockerfile Make a clean directory to work from, called `prefect-docker-guide`. ```bash mkdir prefect-docker-guide cd prefect-docker-guide ``` In this directory, create a sub-directory named `flows` and put your flow script from the [Deployments](/3.0rc/deploy/serve-flows/) guide in it. ```bash mkdir flows cd flows touch prefect-docker-guide-flow.py ``` Here's the flow code for reference: ```python prefect-docker-guide-flow.py import httpx from prefect import flow @flow(log_prints=True) def get_repo_info(repo_name: str = "PrefectHQ/prefect"): url = f"https://api.github.com/repos/{repo_name}" response = httpx.get(url) response.raise_for_status() repo = response.json() print(f"{repo_name} repository statistics 🤓:") print(f"Stars 🌠 : {repo['stargazers_count']}") print(f"Forks 🍴 : {repo['forks_count']}") if __name__ == "__main__": get_repo_info.serve(name="prefect-docker-guide") ``` Next, add a `requirements.txt` to the `prefect-docker-guide` directory. Include all dependencies required for your `prefect-docker-guide-flow.py` script in the Docker image to build. ```bash # ensure you run this line from the top level of the `prefect-docker-guide` directory touch requirements.txt ``` Put the following in your `requirements.txt` file: ```txt requirements.txt prefect>=3.0.0 httpx ``` Create a `Dockerfile` to create a Docker image that also stores the flow code: ```bash touch Dockerfile ``` Add the following content to your `Dockerfile`: ```dockerfile Dockerfile # Using the latest version of Prefect with Python 3.10 FROM prefecthq/prefect:2-python3.10 # Add your requirements.txt file to the image and install dependencies COPY requirements.txt . RUN pip install -r requirements.txt --trusted-host pypi.python.org --no-cache-dir # Add your flow code to the image COPY flows /opt/prefect/flows # Run your flow script when the container starts CMD ["python", "flows/prefect-docker-guide-flow.py"] ``` ## Build a Docker image Now that you have a Dockerfile, build your image by running: ```bash docker build -t prefect-docker-guide-image . ``` Check that your build worked by running a container from your new image. Your container needs an API URL and an API key to communicate with Prefect Cloud. * You can get an API key from the [API Keys](https://docs.prefect.io/2.12.0/cloud/manage-users/api-keys/) section of the user settings in the Prefect UI. * You can get your API URL by running `prefect config view` and copying the `PREFECT_API_URL` value. Provide both these values to your container by passing them as environment variables with the `-e` flag. ```bash docker run -e PREFECT_API_URL=YOUR_PREFECT_API_URL -e PREFECT_API_KEY=YOUR_API_KEY prefect-docker-guide-image ``` After running the above command, the container should start up and serve the flow within the container. Your container needs an API URL and network access to communicate with the Prefect API. For this guide, assume the Prefect API is running on the same machine that you will run your container on and the Prefect API was started with `prefect server start`. If you're running a different setup, check out the [Hosting a Prefect server guide](/3.0rc/manage/self-host/) for information on how to connect to your Prefect API instance. To ensure that your flow container can communicate with the Prefect API, set your `PREFECT_API_URL` to `http://host.docker.internal:4200/api`. If you're running Linux, set your `PREFECT_API_URL` to `http://localhost:4200/api` and use the `--network="host"` option instead. ```bash docker run --network="host" -e PREFECT_API_URL=http://host.docker.internal:4200/api prefect-docker-guide-image ``` After running the above command, the container should start and the flow should run within the container. ## Deploy to a remote environment Now that you have a Docker image with your flow code embedded, you can deploy it to a remote environment. This guide simulates a remote environment by using Kubernetes locally with Docker Desktop. Use the instructions provided by Docker to [set up Kubernetes locally.](https://docs.docker.com/desktop/kubernetes/) ### Create a Kubernetes deployment manifest To ensure the process serving your flow is always running, create a [Kubernetes deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/). If your flow's container crashes, Kubernetes automatically restarts it, ensuring you don't miss any scheduled runs. First, create a `deployment-manifest.yaml` file in your `prefect-docker-guide` directory: ```bash touch deployment-manifest.yaml ``` Add the following content to your `deployment-manifest.yaml` file: ```yaml deployment-manifest.yaml apiVersion: apps/v1 kind: Deployment metadata: name: prefect-docker-guide spec: replicas: 1 selector: matchLabels: flow: get-repo-info template: metadata: labels: flow: get-repo-info spec: containers: - name: flow-container image: prefect-docker-guide-image:latest env: - name: PREFECT_API_URL value: YOUR_PREFECT_API_URL - name: PREFECT_API_KEY value: YOUR_API_KEY # Never pull the image because we're using a local image imagePullPolicy: Never ``` **Keep your API key secret** In the above manifest, you are passing the Prefect API URL and API key as environment variables. This approach is simple, but not secure. If you are deploying your flow to a remote cluster, use a [Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/) to store your API key. ```yaml deployment-manifest.yaml apiVersion: apps/v1 kind: Deployment metadata: name: prefect-docker-guide spec: replicas: 1 selector: matchLabels: flow: get-repo-info template: metadata: labels: flow: get-repo-info spec: containers: - name: flow-container image: prefect-docker-guide-image:latest env: - name: PREFECT_API_URL value: # Never pull the image because we're using a local image imagePullPolicy: Never ``` **Linux users** If you're running Linux, set your `PREFECT_API_URL` to use the IP address of your machine instead of `host.docker.internal`. This manifest defines how your image runs when deployed in your Kubernetes cluster. Note that you will run a single replica of your flow container. If you want to run multiple replicas of your flow container to keep up with an active schedule, or because your flow is resource-intensive, you can increase the `replicas` value. ### Deploy your flow to the cluster You now have a deployment manifest. To deploy your flow to the cluster, use: ```bash kubectl apply -f deployment-manifest.yaml ``` To monitor the status of your Kubernetes deployment, use: ```bash kubectl get deployments ``` Once the deployment successfully starts, check the logs of your flow container by running the following: ```bash kubectl logs -l flow=get-repo-info ``` You're now serving your flow in your cluster. To trigger a flow run, use: ```bash prefect deployment run get-repo-info/prefect-docker-guide ``` Navigate to the URL provided by the `prefect deployment run` command and follow the flow run through the logs in the Prefect UI. ## Prefect-maintained Docker images Every release of Prefect results in several new Docker images. These images are all named [prefecthq/prefect](https://hub.docker.com/r/prefecthq/prefect) and their **tags** identify their differences. ### Image tags When a release is published, images are built for all of Prefect's supported Python versions. These images are tagged to identify the combination of Prefect and Python versions contained. Additionally, we have "convenience" tags which are updated with each release to facilitate automatic updates. For example, when release `3.1.1` is published: 1. Images with the release packaged are built for each supported Python version (3.9, 3.10, 3.11, 3.12) with both standard Python and Conda. 2. These images are tagged with the full description. For example, `prefect:3.1.1-python3.10` and `prefect:3.1.1-python3.10-conda`. 3. For users that want more specific pins, these images are also tagged with the SHA of the git commit of the release. For example, `sha-88a7ff17a3435ec33c95c0323b8f05d7b9f3f6d2-python3.10`. 4. To be on the latest `3.1.x` release and receiving patch updates, we update a tag without the patch version to this release. For example, `prefect.3.1-python3.10`. 5. To be on the latest `3.x.y` release and receiving minor version updates, we update a tag without the minor or patch version to this release. For example, `prefect.3-python3.10`. 6. For users who want the latest `3.x.y` release without specifying a Python version, we update `3-latest` to the image for our highest supported Python version, which in this case is equivalent to `prefect:3.1.1-python3.10`. **Choose image versions carefully** It's a good practice to use Docker images with specific Prefect versions in production. Be careful when employing images that automatically update to new versions (such as `prefecthq/prefect:2-python3.11` or `prefecthq/prefect:3-latest`). ### Standard Python Standard Python images are based on the official Python `slim` images, such as `python:3.10-slim`. | Tag | Prefect Version | Python Version | | ---------------------- | :-------------------------: | -------------: | | 3-latest | most recent v2 PyPi version | 3.10 | | 3-python3.12 | most recent v2 PyPi version | 3.12 | | 3-python3.11 | most recent v2 PyPi version | 3.11 | | 3-python3.10 | most recent v2 PyPi version | 3.10 | | 3-python3.9 | most recent v2 PyPi version | 3.9 | | 3.X-python3.12 | 2.X | 3.12 | | 3.X-python3.11 | 2.X | 3.11 | | 3.X-python3.10 | 2.X | 3.10 | | 3.X-python3.9 | 2.X | 3.9 | | sha-\-python3.12 | \ | 3.12 | | sha-\-python3.11 | \ | 3.11 | | sha-\-python3.10 | \ | 3.10 | | sha-\-python3.9 | \ | 3.9 | ### Conda-flavored Python Conda-flavored images are based on `continuumio/miniconda3`. Prefect is installed into a conda environment named `prefect`. | Tag | Prefect Version | Python Version | | ---------------------------- | :-------------------------: | -------------: | | 3-latest-conda | most recent v3 PyPi version | 3.10 | | 3-python3.12-conda | most recent v3 PyPi version | 3.12 | | 3-python3.11-conda | most recent v3 PyPi version | 3.11 | | 3-python3.10-conda | most recent v3 PyPi version | 3.10 | | 3-python3.9-conda | most recent v3 PyPi version | 3.9 | | 3.X-python3.12-conda | 3.X | 3.12 | | 3.X-python3.11-conda | 3.X | 3.11 | | 3.X-python3.10-conda | 3.X | 3.10 | | 3.X-python3.9-conda | 3.X | 3.9 | | sha-\-python3.12-conda | \ | 3.12 | | sha-\-python3.11-conda | \ | 3.11 | | sha-\-python3.10-conda | \ | 3.10 | | sha-\-python3.9-conda | \ | 3.9 | ## Build your own image If your flow relies on dependencies not found in the default `prefecthq/prefect` images, you may want to build your own image. You can either base it off of one of the provided `prefecthq/prefect` images, or build your own image. See the [Work pool deployment guide](/3.0rc/deploy/work-pools/prefect-deploy) to build custom images with dependencies specified in a `requirements.txt` file. By default, Prefect [work pools](/3.0rc/deploy/work-pools/control-runs) that use containers refer to the `3-latest` image. You can specify another image at work pool creation. You can override the work pool image choice in individual deployments. ### Extend the `prefecthq/prefect` image manually Here is an example `Dockerfile` for building an image based on `prefecthq/prefect:2-latest`, but with `scikit-learn` installed: ```dockerfile FROM prefecthq/prefect:3-latest RUN pip install scikit-learn ``` ### Choose an image strategy The options described above have different complexity and performance characteristics. We recommend the following when choosing a strategy: * If your flow only makes use of tasks defined in the same file as the flow, or tasks that are part of `prefect` itself, you can rely on the default provided `prefecthq/prefect` image. * If your flow requires a few extra dependencies found on PyPI, you can use the default `prefecthq/prefect` image and specify additional packages to install at runtime when you create a deployment or a work pool. * If the installation process requires compiling code or other expensive operations, consider building a custom image instead. * If your flow requires extra dependencies or shared libraries, we recommend building a shared custom image with all the extra dependencies and shared task definitions you need. Your flows can all rely on the same image, but have their source stored externally. This option eases development, since the shared image only needs to be rebuilt when dependencies change, not when a flow's code changes. ## Next steps * You can extend this setup to serve multiple flows in a single Docker image by updating your Python script to use `flow.to_deployment` and `serve` to [serve multiple flows or the same flow with different configuration](/3.0rc/develop/write-flows#serving-multiple-flows-at-once). * To learn more about deploying flows, check out the [Deployments](/3.0rc/deploy/serve-flows/) concept doc. * For advanced infrastructure requirements, such as executing each flow run within its own dedicated Docker container, learn more in the [Work pool deployment guide](/3.0rc/deploy/work-pools/prefect-deploy).