On the Limits of Drupal Orchestration and Shared Infrastructure

Developer and ECA maintainer Jürgen Haas wants Drupal to offer a common modelling surface for processes carried out by different workflow engines, without requiring those engines to work alike. His experience across ECA, the Modeler API, Tool API and the Admin theme shapes a distinction between what Drupal should coordinate and what it should execute.

Speaking with The DropTimes, Haas discusses the evidence behind that direction, the gaps in tracking failed multi-engine workflows, and the safeguards he wants when AI can propose changes to automation. He also explains where shared abstractions may fail, and why maintaining them cannot depend on one developer.

TDT [1]: Across ECA, the Modeler API, Tool API, the Workflow Modeler, AI integrations and your work around Drupal’s Admin interface, you repeatedly end up building layers that let other systems do more without each having to solve the same problem again. How do you recognise when something should become shared Drupal infrastructure rather than remain a feature of one project?

Jürgen Haas: I rarely have to recognise it, because generic is where I start. If code solves exactly one use case, it's probably not worth writing. Almost any problem can be abstracted so that others can use it, configure it and extend it. That costs more time at the drawing board. The solution gets better for it anyway, even if nobody else ever shows up.

Usually somebody does. ECA is the famous example. We developed it in the open from the start. A few weeks in, another maintainer joined full-time and helped us get 1.0 done in a way we couldn't have managed alone. Code written for one client never gets that. Generic code brings shared maintenance cost, visibility, and people who discover it and join.

Generic by default doesn't mean right the first time. ECA had two modelers from day one, but model owners and modelers still knew too much about each other. For years, the answer to "can we use the modeler for migrations?" was no. The Modeler API fixed that. Writing it was the quick part. Seeing it took years.

What moves something out of one project and into a shared layer is vocabulary. When Dries, Randy, Shibin and I lined up ECA, Maestro, FlowDrop and core, we found the same concepts under different names: Trigger, Step, Condition, Workflow, Run. Three projects inventing the same thing means a floor is missing. That floor belongs to everybody, ideally in core.

TDT [2]: With orchestration, that instinct now reaches beyond Drupal itself. What kinds of work do you think Drupal should coordinate but not execute, and where would you draw the boundary beyond which Drupal should simply hand responsibility to another system?

Jürgen Haas: Drupal should execute what touches its own data and rules: entities, forms, access, rendering. That's ECA's home turf, and nobody can do it from the outside.

Beyond that, the boundary is maintenance, not capability. Drupal could talk to my accountant's portal just as well as Activepieces does. That's not the point. Activepieces and n8n maintain hundreds of connectors, and every one needs attention whenever a vendor changes its API. Nobody maintains that long tail well on the side. For them it's the core business. For us it would be a hobby with a growing issue queue. I maintain 87 projects. I know what a long tail does to a maintainer.

So my billing run splits cleanly. Drupal knows that the invoice went out, to whom, and who approved it. How to log in to the accountant's portal is Activepieces' job. Drupal records the outcome. It doesn't copy the data to pretend it's in charge. Copying instead of connecting is exactly why the Excel sheet fails.

Dries wrote that the CMS "is not necessarily the orchestrator of that ecosystem." I agree, and I still want the modeling surface in Drupal. Those are two different claims. The engines can run anywhere. The place where a human sees the whole job and stays in control belongs in Drupal, because that's where the Modeler API already is.

TDT [3]: The vision increasingly assumes that users should be able to describe a process without deciding whether ECA, Maestro, FlowDrop or an external engine runs every step. Before building further towards that, what evidence from real organisations or workflows tells you that choosing the engine is actually a problem users want the platform to solve for them?

Jürgen Haas: Two findings, both anecdotes.

The first is a silence. Nobody asks which engine should run their step. People who need orchestration never get far enough to ask. The complexity keeps them away long before that. In Post 8 I invited readers to bring me their spreadsheets. Nobody did. I don't read that as missing demand. Everybody orchestrates, most of them by hand, and they don't hang out in our Slack channel.

The second came from our own expert round. Randy built a prototype in which Activepieces, FlowDrop, ECA and Maestro work together on a properly complex task, and recorded a video. I was impressed like rarely before. Then I asked myself who else could have built it. Randy knows all four components inside out. Anyone else would have shouted that this is rubbish. Context switch after context switch. Four UIs, four sets of keyboard shortcuts. Everything looks similar and still behaves differently. No regular user exposes themselves to that. Nobody wants to choose an engine. They want the invoice sent.

Behind both sits first-semester UX. "Consistency and standards" is the fourth of Jakob Nielsen's ten usability heuristics, published in 1994: users should not have to wonder whether different words, situations or actions mean the same thing. That's older than Drupal. We just hadn't applied it across project borders.

The evidence is anecdotal. The principle isn't.

TDT [4]: If Drupal does start making more of those decisions for the user, what should it never hide? Which choices about execution, data movement, permissions, cost or responsibility must remain visible to the person operating the workflow even if the interface could technically abstract them away?

Jürgen Haas: Fair challenge. In Post 8 I wrote that the user never learns which system ran which step. I stand by that for the engine. Which engine executes a step is plumbing, like the power plant behind the light switch.

Four things aren't plumbing, and they stay visible.

1. Data leaving Drupal. If a step sends personal data to an external service, the step says so, by name, before anything runs. I'm German. Nobody here gets to treat GDPR as optional reading.
2. Whose permissions a step uses. "Runs as the editor" and "runs as an admin account" look identical on a diagram and very different in an audit.
3. Cost. An AI call or an Activepieces task costs money per run. A workflow that fires on every node save should say so before the invoice does.
4. Irreversibility. An email can't be unsent, and a card charge can't be retried away. Those steps need to look different from "update a field".

The rule behind it: hide what the user can't influence, show what they're accountable for. Whatever stays hidden by default must be one click away. Test and replay from Post 5 is the model. After a run, every step and every token value is there to inspect.

TDT [5]: ECA, Maestro and FlowDrop were built around genuinely different assumptions about state, duration and execution. Which of those differences should remain differences? Where would trying to create one common abstraction actually make the individual systems worse?

Jürgen Haas: The engines should stay different. That's where the value is.

ECA has no running instance, on purpose. It reacts inside the request, with the live entity in hand, and can throw a validation error in a pre-save hook. Give it a persisted run, and tens of thousands of sites pay for state they never use. Maestro is the opposite. Its process tracks loop-backs, reassignments and ancestor history over days. Make that stateless and you've deleted Maestro. FlowDrop insists that data on the wires is typed and serializable, so an agent can reason about it. Make it pass live entities between steps the way ECA does, and you lose the property that makes it safe for AI.

An abstraction general enough to cover all three would have to be reactive and durable, hook-deep and AI-clean, all at once. It would be mediocre at everything, and each maintainer would spend their time working around it.

What should be shared is the edge, not the core. A common vocabulary: Trigger, Step, Condition, Workflow, Run. A contract for handing data from one step to the next. A way for one engine to start another and get the result back. What happens inside an engine stays its maintainer's business. Composition beats merger.

That doesn't contradict generic by default. Generic belongs where the concepts really are the same. At the edges they are. Inside the engines they aren't.

TDT [6]: Imagine a process starts in Drupal, passes through two different workflow engines and an external service, and then fails halfway through. Where should the authoritative state of that run live? Who decides what can be retried, what has already produced an irreversible side effect, and where a human needs to intervene?

Jürgen Haas: Honest answer first: today, nowhere. Each engine knows its own part, nobody knows the whole run. In Post 6 we listed the Run as a shared primitive with no standard in core. In Post 8 I didn't even put it on the to-do list. It belongs there, near the top.

Where it should live: in Drupal. One run record, owned by the coordinating layer. Each engine stays authoritative for what happens inside its own step. Maestro knows its process, Activepieces knows its flow. The run record knows the sequence, what each step reported back, and where it stopped. Two sources of truth for the same fact is how you end up back in Excel.

Who decides about retries: the step definition. Not the engine, and not whoever is on call at 2 a.m. The dispatcher I sketched in Post 8 reads properties from each step. Does it wait for a human? Does it leave Drupal? Add two more. Is it idempotent? Is it reversible? Updating a field: retry freely. Sending the invoice email: check before retrying. Charging a card: stop.

When a step that can't be retried fails, a human takes over. We already have an engine that's very good at waiting days for a human: Maestro. That's not a separate workflow. It's a step within the same one, with a named person on it.

TDT [7]: A common data contract can tell one system what another system produced, but reliable orchestration may also need to carry identity, permissions, provenance and context across that boundary. How much of that belongs in a shared Drupal contract, and how much should remain the responsibility of the individual execution engine?

Jürgen Haas: Identity and provenance belong in the contract. Permissions don't.

Every payload that crosses an engine boundary should say who started the run, on whose behalf this step acts, which step produced the data, and which run it belongs to. Without that, no engine can make a safe decision, and nobody can reconstruct what happened.

Permissions are different. If the contract carries "this user may do X", every engine has to trust every other engine's idea of X. That's a skeleton key passed from hand to hand. Each engine checks access in its own system, against the identity it received. Drupal checks Drupal permissions. Activepieces uses the connection somebody configured on purpose.

Context comes in two kinds. One is a reference. A step receives an ID and loads the entity itself, under its own access rules. That's FlowDrop's approach, and it's the safe default: nothing travels that the next step didn't ask for.

The other is work in progress. One step loads an entity and changes it. The next step changes it further and maybe hands it back for more. Nothing is saved yet. Load it fresh from the ID and you get the old version, without any of those changes. ECA does this all the time within a single request, through tokens. Across an engine boundary, the contract has to carry the modified entity itself, serialized, not just its ID.

Permissions still don't need to travel for that. Whichever step finally saves the entity checks whether the identity it received may do so.

TDT [8]: AI now adds a more difficult authority problem. An agent can use Drupal tools, participate inside a workflow and increasingly help create the workflow itself. How much authority should an agent ever have to create or modify the automation that governs what that same agent is allowed to do? What decisions should always require independent human approval?

Jürgen Haas: To propose it: plenty. To approve it: none. An agent never approves its own changes, least of all to the automation that constrains it. That would be a rubber stamp with extra steps. Separation of duties is much older than AI. Accountants don't sign off their own expenses either.

Both directions exist today. AI agents in Drupal can use ECA, through tools or through events. And the Workflow Modeler has an embedded AI chatbot that calls an LLM to contribute to the model. Put both into one agent without a gate, and it writes itself a new tool.

So the rule I'd hard-wire: agents draft, humans deploy. A model an agent creates should land disabled, as a proposal, with a readable diff. ECA models are config, so the diff already exists.

A human approves, always, when a change touches one of five things: roles, permissions, the tools an agent may call, data leaving Drupal, or anything irreversible.

Which human approves has nothing to do with AI. Whoever may make a change without an agent may approve it with one. Often that's the person who gave the agent the task. Not everyone may touch a production system, agent or not.

TDT [9]: Making powerful systems approachable is also a thread through your work on ECA’s in-context customisation and Drupal’s Admin theme. What have you learned from working on both about the difference between removing unnecessary complexity and merely hiding complexity that users eventually need to understand?

Jürgen Haas: Removing complexity means the user never needs it. Hiding it means they will, and won't find it. The difference shows up months later, usually as a support ticket.

In-context customization taught me the test. Click the lightning bolt on a form, apply a template, and you get a real ECA model, the same kind a developer would build. Later it opens in the full modeler like any other model. Nothing is hidden. The complexity just isn't mandatory on day one. Had in-context customization produced some lighter object instead, it would have been a dead end: easy to start, impossible to grow out of.

The Workflow Modeler shows what real removal looks like. We started with four panels and are down to two. The toolbars shrank dramatically, and not a single function got lost. We just arranged things better.

The admin theme taught me what hiding looks like. Gin had sticky action buttons, first configurable, later mandatory. When Gin merged into core, the default admin theme kept the code but switched it off. Only a setting in settings.php turns the buttons back on. Hidden so well that I, the subsystem maintainer, first claimed in the issue that they had never made it into core. Now we're discussing whether core needs both, the regular buttons and the sticky ones.

We hid them on purpose. Nobody had done the UX research to decide, and removing them without it would have meant guessing. Removing well takes research, not opinions. Mine included.

Emma and Mark's testing gave us one bar: first-time users succeed without documentation. I hold a second one just as high. Experienced users never feel locked out.

TDT [10]: Shared APIs and workflow infrastructure become difficult to replace once other projects depend on them, but maintaining that infrastructure is less visible than launching new features. Who should ultimately be responsible for the long-term governance and maintenance of this orchestration layer, and what would make you conclude that the current architectural direction itself needs to change?

Jürgen Haas: Not one person, and definitely not me alone. I already maintain 87 projects. Something every engine depends on can't rest on one maintainer.

The step and the data-handoff contract should end up in Drupal core, probably the trigger and the condition too. The run stays with the engines and with whatever coordinates them. Core has what contrib lacks: subsystem maintainers, a release cycle everybody plans around, and a process that outlives any individual. I see that every week as subsystem maintainer for the Admin theme. Until then, the people with a component carry it together: ECA, Maestro, FlowDrop, Tool API, the AI module. The orchestration design spec, where decisions are append-only and rejected options stay on record, is the start of that governance.

Maintenance also needs money, and invisible work is the hardest to fund. That was the whole point of Post 9.

Three things would change my direction. If Maestro or FlowDrop as a second model owner on one canvas needs special cases everywhere, the Modeler API is the wrong abstraction. If the engines can't agree on a data-handoff contract, the shared floor is a fantasy, and bridges are the honest answer. And if people who use the single UI still ask which engine runs their step, the premise of Post 8 was wrong.