In the ever-evolving landscape of infrastructure as code (IaC), managing sensitive data securely is a critical concern. Before the release of Terraform 1.11, handling sensitive values such as passwords and tokens presented significant challenges because these values were inevitably stored in the Terraform plan artifact and state files. This situation necessitated treating the state as sensitive data, a practice strongly recommended by both Terraform developers and users. To address this, Terraform introduced the concept of "ephemerality," providing a more secure method of handling sensitive information without leaving a trace.
Understanding Ephemerality in Terraform
Ephemerality, a term borrowed from the computing world, refers to the creation of temporary or short-lived data that is discarded once it fulfills its purpose. In the context of Terraform, this means managing sensitive data in a way that ensures it doesn’t persist beyond its immediate use. With the introduction of new language constructs, Terraform can now track values only during runtime, making them transient and inherently ephemeral. These constructs encompass ephemeral values, including ephemeral resources, input variables, and write-only arguments.
Exploring Ephemeral Resources
Ephemeral resources in Terraform are designed to be temporary. They serve the purpose of reading data from sources like a secrets manager or establishing a connection, while their attributes can be referenced elsewhere without being stored in the Terraform plan artifact or state file. It’s crucial to understand that ephemeral resources rely on the existence of all their dependencies, as they are executed during both the plan and apply stages. If an ephemeral resource attempts to access a non-existent secret from a secrets manager, it results in an error. However, Terraform can delay the execution of an ephemeral resource to the apply stage if an input argument references a value that is unknown during the plan stage but will be resolved during apply.
For instance, consider an ephemeral password resource that generates a random password using a cryptographic random number generator. This password can be utilized as input for a write-only attribute on a managed or ephemeral resource, ensuring sensitive data remains secure and transient.
Write-Only Arguments in Terraform
Write-only arguments are another crucial aspect of managing sensitive data securely in Terraform. These are resource attributes configured by users but are not stored in the Terraform plan artifact or state file. Terraform providers implement write-only arguments on managed resources to handle sensitive values such as passwords and tokens securely. A typical example is the secret_string_wo argument on the aws_secretsmanager_secret_version resource. Assigning an ephemeral resource attribute to a write-only argument allows for end-to-end ephemerality in Terraform, ensuring sensitive values are not persisted.
It’s essential to note that write-only arguments are sent to the provider during every operation. To track changes, providers typically incorporate write-only version arguments, which are stored in the state file. Users can update a write-only value by incrementing the version number, prompting Terraform to detect and propagate this change to the provider.
Deferring Ephemeral Resources
Terraform’s ability to defer the execution of ephemeral resources is a powerful feature. If an input argument of an ephemeral resource references a value that is unknown during the plan stage but will be resolved during or after the plan, Terraform defers the resource’s execution to the apply stage. This ensures that ephemeral resources are evaluated at the correct time, preventing premature execution.
Ephemeral resources form nodes in Terraform’s dependency graph, and when a managed resource or data source relies on an attribute of an ephemeral resource, Terraform ensures the ephemeral resource is provisioned first. By modeling the Terraform configuration to align with the dependency graph, users can ensure the correct execution order and prevent unintended behavior.
Using the example of an ephemeral random password and AWS Secrets Manager, Terraform securely provisions an AWS RDS instance by leveraging ephemerality and its deferring logic. During the apply stage, Terraform generates a secret, fetches it using another ephemeral resource, and writes it to the appropriate write-only argument.
Lifecycle of an Ephemeral Resource
The lifecycle of ephemeral resources differs from other resources and data sources. It comprises three steps:
- Opening: When Terraform requires the result of an ephemeral resource, it opens it, similar to reading a secret from a secrets manager.
- Renewing: If Terraform needs access to the ephemeral resource for a duration longer than the remote system’s expiration time, it requests the provider to periodically renew it.
- Closing: Once Terraform no longer requires an ephemeral resource, it closes it after the dependent providers complete their tasks for the current Terraform run phase.
While all ephemeral resources implement opening logic, not all include closing or renewing logic. This depends on the ephemeral nature of the resource. For instance, managing tunnel connections may necessitate periodic renewal and proper closure, while leasing temporary credentials from Vault implies renewal and revocation when they are no longer needed.
Persisting Ephemeral Secrets
While ephemerality prevents introspection of the value itself, which is intentional, it can pose challenges. When using an ephemeral random password, for instance, the generated secret must be stored in a secrets manager to ensure accessibility. In scenarios where existing secrets are read from a secrets manager managed outside the current Terraform module, there is no need to persist them again.
Conclusion
The introduction of ephemerality in Terraform represents a significant advancement in securely managing sensitive data. By leveraging ephemeral values, resources, and write-only arguments, users can ensure that sensitive information is handled securely without persisting in the Terraform plan artifact or state file. This approach not only enhances security but also simplifies the management of sensitive data in infrastructure as code.
For further reading and a deeper understanding of ephemeral values and their implementation in Terraform, refer to the original documentation on HashiCorp’s website.
For more Information, Refer to this article.
