Infrastructure as Workflow: Beyond Static Terraform
SchemaBridge Team · 2026-01-18 · IaC, DevOps, Cloud Automation
Managing Day 2 cloud operations. Treating infrastructure as a stateful, long-running business process.
The Limit of "Infrastructure as Code": Day 1 vs. Day 2
We love Terraform (IaC). It solved the problem of "Day 1" provisioning. You describe your desired state (10 EC2 instances, 1 RDS, 1 VPC), you run terraform apply, and the cloud provider makes it so. This is perfect for static resources. It is declarative, idempotent, and version-controlled. It is the bedrock of modern DevOps.
But the reality of cloud operations is that Day 2 is where the complexity lives. Day 1 is the wedding; Day 2 is the marriage. Day 2 is about Processes, not just Resources. It is about the living, breathing lifecycle of the system as it changes over time.
Consider the lifecycle of a critical database upgrade. This is not a static event; it is a Workflow:
1. Preparation: Snapshot the primary DB to S3 for safety.
2. Wait: You must wait for the snapshot to complete. For a large multi-TB database, this could take 30 to 45 minutes.
3. Provisioning: Spin up a new DB instance from the snapshot using the new engine version.
4. Wait: Wait for the new instance to become "Available" and "Healthy."
5. Migration: Connect to the new instance and run a schema migration script (Flyway or Liquibase). This could take 1 hour.
6. Verification: Run a set of smoke tests against the new DB to ensure data integrity.
7. Switchover: If successful, update the DNS CNAME to point to the new DB.
8. Rollback: If any step fails, you must revert the DNS and destroy the broken instance.
Terraform is a poor fit for this. It is declarative, not imperative. time_sleep and local-exec provisioners exist as escape hatches for "Wait for 30 minutes" or "Run a SQL script and check the exit code," but Terraform has no durable, resumable, approval-gated process model. To handle this, teams usually resort to wrapping Terraform in Jenkins pipelines, CircleCI jobs, or Python scripts. We are back to the "Glue Code Crisis" discussed earlier in this series, but now for infrastructure. These scripts are brittle, stateless, and impossible to debug when they fail halfway through a 4-hour operation.
Treating Operations as Durable Workflows
At SchemaBridge, we believe that Infrastructure Operations are Business Workflows. They have the same requirements as a payment processing flow: reliability, auditability, state management, and error recovery.
By moving your Day 2 operations into SchemaBridge, you gain the power of a Durable Orchestrator for your cloud:
- Durable Waiting: Need to wait 4 hours for a data warehouse export? The workflow sleeps durably without consuming a Jenkins executor or a container.
- Approval Gates: Need a senior SRE or a Compliance Officer to approve the DNS switch? The workflow pauses at an Approval step until someone with approval rights approves it, in the Approvals page or through the API. The state is preserved whether that takes 5 minutes or 5 days (the default approval timeout is 7 days), and a Slack step just before it can post a message linking to the pending approval.
- Explicit Rollback Paths: SchemaBridge does not run hidden compensation for you, and that is a feature: rollback is a path you draw. Model the outcomes you expect as data — a status check that returns
failed, or a gateway configured to turn a rejected (4xx) call into aninvalid_inputoutcome — and route those to a cleanup branch that restores the old configuration. An unexpected step failure stops the run where it is, so an operator can inspect it and retry from the failed step rather than having a half-understood rollback fire automatically.
The "Control Plane" Pattern: Integrating with AWS, K8s, and Terraform
SchemaBridge doesn't replace Terraform; it orchestrates it. We use the Control Plane Pattern to unify the static and dynamic worlds.
1. The Trigger: A developer commits code or triggers a "Provision Dev Env" workflow from your developer portal by calling the workflow's webhook URL.
2. The Provisioning: SchemaBridge calls the Terraform Cloud API (or your own runner's HTTP endpoint) to start the run that creates the physical resources. SchemaBridge does not shell out to terraform itself: workflow scripts run in a GraalVM sandbox with no process or file-system access, by design.
3. The Wait: A durable Delay step waits, then a Gateway step checks the Terraform run's status. Workflows are acyclic graphs, so instead of an infinite polling loop you chain a bounded number of wait-and-check steps, or have the workflow re-spawn itself after a Delay — the same recursion our own gradual-rollout system workflow uses.
4. The Post-Provisioning (The "Logic Layer"): Once the infrastructure exists, SchemaBridge calls the endpoints that run the DB migrations, seed the test data, and kick off the acceptance tests — your CI system's API, or a small ops service you control inside the new cluster.
This pattern allows you to keep your resources defined in HCL (HashiCorp Configuration Language) while keeping your operational logic defined in a Visual Graph.
Ephemeral Environments: The Holy Grail of Developer Velocity
Every modern engineering team wants Ephemeral Environments—a full replica of production for every Pull Request (PR). This allows for true isolation and preventing bugs before they merge to main.
Traditional approaches fail because they are hard to clean up. You spin up resources for PR-123, but the developer forgets to close the PR, or the cleanup script fails. Your cloud bill explodes with "Zombie RDS Instances" and "Orphaned Load Balancers."
With SchemaBridge, an Environment is a workflow with a defined lifecycle.
1. Start: Spin up resources (RDS, a cache, ECS Services) through your provisioning API.
2. Wait: A durable Delay step arms the teardown at creation time — for example a 24-hour TTL. The wait holds no worker or container; the wake-up time is persisted.
3. Cleanup: When the timer expires, the workflow wakes up and calls your provisioning API to destroy the environment. For "tear down as soon as the PR merges", point your Git provider's merge webhook at a separate teardown workflow, and make the teardown idempotent so it does no harm if both fire.
Because the teardown timer is armed in the same run that created the environment, it is hard to forget. Even if the entire SchemaBridge cluster restarts, the persisted timer still fires and PR-123's environment is still destroyed on schedule. This is "Garbage Collection for the Cloud."
Illustrative Scenario: Zero-Touch Blue/Green Deployment for a Latency-Sensitive Service
Consider a team running a latency-sensitive matching engine that needs to update their core service without a microsecond of downtime.
The Challenge
A standard Kubernetes rolling update wouldn't be safe enough if they needed to verify the accuracy of the new version on live data for 10 minutes before switching traffic. They would need a complex "Shadow Mode" deployment.
The SchemaBridge Approach
They could build a "Blue/Green Deployment Workflow" in SchemaBridge:
1. Deploy Green: Spin up the new version of the engine alongside the old one (Green).
2. Shadow Traffic: Configure the API Gateway to send a generic copy of live traffic to Green (fire-and-forget). The responses from Green are not sent to users but are captured.
3. Verify: After a 10-minute Delay, the workflow fetches a sample of Green's captured responses and the matching Blue (live) responses over HTTP, and uses JSONata to compare them.
4. Decision Point: Conditional edges route on the comparison result:
- If accuracy < 100%, the workflow posts an alert and calls the deploy API to destroy
Green. - If accuracy == 100%, the workflow proceeds.
5. Switch: The workflow updates the load balancer to switch real user traffic to Green.
6. Clean up: It waits another hour (for easy rollback) and then destroys Blue.
The Result
This kind of workflow could substantially reduce deployment risk. An SRE team could trigger a deployment and step away, knowing the workflow would run the verification and the explicit cleanup path they designed — turning risky, infrequent releases into safer, routine ones.
Cost Management: The "Workflow" of FinOps
Cloud cost optimization (FinOps) is often a manual process of nagging developers to turn things off. SchemaBridge allows you to automate this governance.
The Night Watchman Pattern
You can deploy a "Night Watchman" workflow that runs every evening at 8:00 PM.
1. Scan: Ask your ops API (for example a small Lambda or internal service that wraps your cloud SDK) for all non-production tagged resources. SchemaBridge talks to it over HTTP; it has no native AWS connector.
2. Check Activity: Have the same endpoint report CPU usage for the last hour, and keep only the resources under 5%.
3. Shutdown: If idle, call the endpoint to stop the instance (don't terminate).
4. Notify: Send a Slack message to the owner: "We paused your dev box to save money."
5. Resume: A separate "start my box" workflow, triggered from your ops tooling via its webhook URL, starts the instance again in the morning.
This kind of simple workflow can meaningfully reduce idle dev-environment spend, cutting the ongoing cost of forgotten, always-on non-production resources.
Comparison: Jenkins/GitLab CI vs. SchemaBridge Operations
| Feature | CI/CD Pipelines (Jenkins) | SchemaBridge Operations |
| :--- | :--- | :--- |
| State | Tied to an executor; long waits hold resources | Durable (persisted in PostgreSQL; survives restarts) |
| Duration | Minutes/Hours | Hours/Days (durable timers) |
| Logic | Scripted (Bash/Groovy) | Visual (Graph) |
| Approval | Input step in the pipeline | Approval step (UI or API), with a timeout |
| Recovery | Restart from a stage | Retry from the failed step |
| Parallelism | Limited by Executors | Fan-out (Spawner) with autoscaled workers |
Expert Checklist for Ops Automation
To modernize your operational stack, follow these heuristics:
1. Don't script waits: If you are writing sleep 60 in bash to wait for an ALB, you are doing it wrong. Use a durable Delay step followed by a status check.
2. Automate Deletion first: Write the cleanup logic before the creation logic. Ensure every creation event has a matching destruction path.
3. Use Approval Gates: Don't be afraid to put a human in the loop for high-risk actions. A "Pause for Approval" is a feature, not a bug.
4. Audit the Operator: Log who triggered the environment and why. Have the trigger pass the requester's ID in the payload and stamp it on every resource you create.
5. Treat Ops as Code: Version control your operational workflows just like your application code. Export them as bundles, keep them in your repository for review, and deploy them through the Deployments page (plan, review, apply) or a configuration import.
Conclusion: The Cloud is a State Machine
Your infrastructure is not a static pile of servers; it is a living, breathing component of your business. By treating it as a state machine, you can automate the complex, multi-step operational dances that currently consume your SRE team's life. You can move from "Ticket-based Ops" to "Self-Service Ops" with safety and durability built in.
See also: The Cold Start Myth: Optimizing the JVM for Events — why long-running workers beat per-event cold starts.