Enhance Docker Compose Using Provider Services

Exciting Developments in Docker Compose: Introducing Provider Services

In a significant update, Docker Compose v2.36.0 has introduced an innovative feature known as provider services. This development is set to enhance Docker Compose’s capabilities by enabling it to interact not only with containers but also with various external systems. At the center of this transformation remains the familiar Compose file, ensuring continuity in workflow while expanding functionality.

Understanding Provider Services

Provider services represent an important milestone for Docker Compose, traditionally used by developers to orchestrate applications that rely on multiple containers. As development environments have grown more complex, integrating non-container dependencies has become a pressing need. Applications today often depend on external resources such as managed databases, SaaS APIs, cloud-hosted message queues, or even VPN tunnels. These elements typically exist outside the realm of Docker Compose, requiring developers to resort to additional scripts or tools, which can complicate workflows and hinder team collaboration.

By introducing provider services, Docker Compose offers a solution to this challenge. Developers can now define and manage external resources directly within their compose.yaml files. This is achieved by delegating the lifecycle management of these resources to the provider binary, which coordinates with Docker Compose as part of its service lifecycle. This allows Docker Compose to become a more comprehensive tool for full-stack development, adaptable to both local and remote environments.

Implementing Provider Services in Your Compose File

To utilize provider services, developers declare them in the Compose file much like any other service. However, instead of specifying a container image, a provider is defined with a particular type and optional parameters. This type should correspond to a binary in the user’s system path that adheres to the Docker Compose provider specification.

For instance, the Telepresence provider plugin is a practical example. This plugin reroutes Kubernetes traffic to a local service, facilitating live cloud debugging. Such functionality is invaluable for testing how local services integrate with real Kubernetes clusters.

Upon executing docker compose up, Docker Compose interacts with the compose-telepresence plugin. Here’s how it operates:

Up Action:

• Verifies whether the Telepresence traffic manager is present in the Kubernetes cluster and installs it if absent.
• Establishes an intercept to reroute traffic from the Kubernetes service to the local service.
  

  Down Action:
  

• Removes the intercept previously set up.
• Uninstalls the Telepresence traffic manager from the cluster.
• Ends the active Telepresence session.

  It’s important to note that the options field’s structure and content are specific to each provider. Plugin authors are responsible for defining and documenting the expected parameters.

  For those unfamiliar with configuring provider services, the Compose Language Server (LSP) offers step-by-step guidance with inline suggestions and validation.

  For more examples and workflows, refer to the official documentation.

  The Mechanics of Provider Services
  

  Under the hood, when Docker Compose encounters a service using the provider key, it searches for an executable in the user’s system path that matches the provider type name. For example, it may look for docker-model cli plugin or compose-telepresence. Docker Compose then executes the binary, passing service options as command-line flags. The provider receives all necessary configurations through these arguments.

  The provider binary must handle JSON-formatted requests via standard input and output. This interaction facilitates seamless communication between Docker Compose and the provider.

  Communication Protocol
  

  Docker Compose communicates essential information to the provider binary by transforming options attributes into flags. It also provides project and service names. For instance, in the compose-telepresence provider example, the following command is executed during the up process:

  <br /> $ compose-telepresence compose --project-name my-project up --name api --port 5732:api-80 --namespace avatars --service api dev-api<br />

  Providers can also send runtime messages to Docker Compose, including:

• info: Status updates displayed in Docker Compose’s logs.
• error: Error reports displayed as failure reasons.
• setenv: Exposes environment variables to dependent services.
• debug: Debug messages visible only when running Docker Compose with the -verbose option.

  This versatile protocol simplifies the integration of new types and supports the development of rich provider integrations. For a detailed structure and examples, refer to the official protocol specification.

  Creating Your Own Provider Plugin
  

  The true potential of provider services lies in their extensibility. Developers can create plugins in any programming language that adhere to the protocol. A typical provider binary implements logic to handle Docker Compose commands with up and down subcommands.

  The source code of the compose-telepresence-plugin serves as an excellent starting point. Written in Go, this plugin wraps the Telepresence CLI to connect a local development container with a remote Kubernetes service. Here’s a snippet from its up implementation:

  

  To create your own provider plugin, follow these steps:

  1. Study the full extension protocol specification.
  2. Parse options as flags to gather the complete configuration required by the provider.
  3. Implement JSON response handling over standard output.
  4. Include debug messages for detailed insights during implementation.
  5. Compile the binary and place it in your system path.
  6. Reference it in your Compose file using provider.type.

     You can develop a range of solutions, from service emulators to remote cloud service initiators, and Docker Compose will automatically invoke your binary when needed.

     Looking Ahead
     

     The introduction of provider services marks a new chapter in Docker Compose’s evolution. Future enhancements will be guided by user feedback to ensure provider services continue to meet real-world needs effectively.

     Envisioning the future, Docker Compose aims to become a comprehensive hub for full-stack development environments. This includes containers, local tools, remote services, and even AI runtimes. Whether you’re connecting to a cloud-hosted database, launching a tunnel, or orchestrating machine learning inference, Docker Compose’s provider services offer a native way to extend your development environment without the need for additional tools or hacks.

     We welcome your ideas on new providers you’d like to build or see added. The community’s creativity will undoubtedly take this feature to exciting new heights.

     Stay tuned for more updates and happy coding!

For more Information, Refer to this article.