Inside the Running System

The claim under examination is this:

The availability of Tidewave MCP on my app is extremely helpful, and few if any other environments provide such effective tool use for software development.

The first half is straightforwardly true and worth explaining precisely, because the reason it is true is more specific than “the agent can run code.” The second half is directionally right and overstated as written. This piece separates them.

What Tidewave actually does

Tidewave, from José Valim and Dashbit, embeds an MCP server inside a running web application. The framing they use is runtime intelligence: the idea that static code analysis and live application behaviour are different sources of truth, and that agents have historically had access only to the first.

The tools it exposes in an Elixir application are:

• project_eval – evaluate arbitrary code inside the running application process
• get_docs – documentation for any module or function, including dependencies
• get_source_location – where a module or function is defined, including inside deps
• get_logs – log output, filterable by level and pattern
• execute_sql_query and get_ecto_schemas – database access, where Ecto is present
• get_ash_resources – resource definitions, where Ash is present
• browser_eval – evaluate JavaScript in a connected browser

Which of those appear depends on what the application uses. In an application built on a custom SQLite layer rather than Ecto, the Ecto and Ash tools simply do not register, and everything routes through project_eval instead. That detail matters later.

Support currently covers Phoenix and Rails, with Django thinner, and it works with the usual agent clients.

The distinction that does the work

The phrase “the agent can execute code” flattens two very different things, and the whole value sits in the gap between them.

Shell access starts a new process. An agent running mix run -e "..." or rails runner "..." boots a fresh instance: cold caches, no accumulated state, no connections, no supervision tree in the shape production has. It answers questions about code. It cannot answer questions about this system, right now.

Runtime evaluation happens inside the process that is already serving. The registries are populated. The GenServers hold whatever state they have accumulated. The database connections are open with their real contents. The permission map for a specific user is loaded. Nothing needs to be reconstructed, because it is all already there.

That difference sounds academic until a bug is only reachable through accumulated state. Then it is the whole game.

The clearest illustration I have is a workflow that had been failing repeatedly. The step results recorded an error of :not_found. That was ambiguous: it could have meant the target did not exist, the identifier was wrong, or the caller lacked access. Static reading of the code narrowed it but did not settle it.

What settled it was running the same function twice inside the live system, once as each of two users:

Permissions.FunctionNodes.call(perms_a, "user_a", "instance", "Mathy", "add", [100, 200])
#=> {:ok, 300}

Permissions.FunctionNodes.call(perms_b, "user_b", "instance", "Mathy", "add", [100, 200])
#=> {:error, :not_found}

That is a controlled experiment, run against production, with real permission maps loaded from the real database, taking about a minute. The answer was that instances resolve by owner and the failing runs were being created under a different account. No amount of log reading would have produced that, because the logs recorded the symptom and not the discriminator. Reproducing it locally would have required rebuilding two users, their permissions and their resources first, which is more work than the bug was worth.

The general shape: runtime evaluation turns debugging from inference into measurement.

What it made possible in practice

A partial list from recent work on one application, all of which would otherwise have been slower or guesswork.

Confirming environment rather than assuming it. Whether an API key was actually present in the running process, and what path a native extension resolved to. Config files say what should be true. The runtime says what is true, and the two diverge more often than anyone likes.

Reading real data structures. Query the live tables through the application’s own data layer, seeing exactly what the running code sees, including the double-encoded JSON envelopes that a tool response actually stores. Reading the schema would have suggested what those columns contained. Reading the rows showed it.

Testing a code path without building a client. An MCP tool handler takes a request frame that a real client normally constructs. Building one by hand in the live node and calling the handler directly exercised the real dispatch, the real permission checks and the real business logic, with no client, no transport and no test harness. Every layer that mattered was covered; only the wire was not.

Discovering a migration gap. A newly added column existed in the schema definition but not in a database whose process had started before the change. The failure surfaced as a plain error, and the fix was one call to the setup function in the live node. Both the diagnosis and the repair took under a minute, and the underlying design gap, that migrations run only when a database process starts, became visible in a way that reading the code had not made obvious.

Verifying a security property empirically. Whether an unauthenticated route was reachable, and whether the exemption that made it reachable had accidentally widened to cover anything else. Two calls through the real router: one path answering 404, the other still answering 401. That is the kind of check that is easy to convince yourself of by reading and easy to get wrong.

The common thread is that each of these is a question about a specific running system rather than about code in general, and each was answered by asking the system instead of reasoning about it.

Why the BEAM amplifies this

The capability is not equally good everywhere it exists, and the runtime matters.

Hot code loading. Editing a module and having the running node pick it up means the agent’s changes and the agent’s observations happen in the same process. There is no rebuild-and-restart cycle between writing and seeing.

Process isolation. An evaluation that raises kills its own process, not the node. The blast radius of an agent’s mistake is smaller than in a runtime where an unhandled error takes down the server.

Addressable state. Registered names, registries and supervision trees mean live state has stable identifiers. Asking what a particular process holds is a normal operation rather than an act of archaeology.

Distribution. A node can be reached from elsewhere as a first-class operation, so the same technique extends to remote systems without inventing a transport.

Put together, the BEAM was designed for systems that are inspected and modified while running. Attaching an agent to that is less a new capability than a new consumer of an old one.

Now the second half of the claim

“Few if any other environments provide such effective tool use for software development” is where the claim needs qualifying, and it is worth doing carefully, because the qualified version is more persuasive.

The underlying capability is decades old. Live-image development is one of the oldest ideas in the field. Smalltalk made the image the environment. Common Lisp with SLIME or SLY has offered evaluation, inspection and redefinition inside a running image since long before any of this. Clojure’s nREPL is the canonical modern implementation. Erlang and Elixir have had remote shells, observer and recon for years. Rails has had rails console, Django has had its shell, and both attach to real data.

Anyone claiming this is new is describing their own recent discovery rather than the state of the art.

Even the specific product is not exclusive. Tidewave supports Rails as well as Phoenix, from the same vendor. A Rails developer gets a comparable capability today.

And there are adjacent agent-accessible runtimes. Chrome DevTools Protocol servers give agents live access to a running browser. Database MCP servers give live data. Jupyter kernels give a persistent evaluation context. None of these is nothing.

So what is actually rare?

The combination is rare, not the ingredient. Live-image introspection existed but was aimed at a human at a keyboard. Agent tooling proliferated but mostly stopped at the filesystem and the shell. Wiring the first to the second is recent, and the number of runtimes where it works well is small, because it requires the runtime to tolerate being poked while serving, to keep state addressable, and to survive mistakes.

The BEAM satisfies all three unusually well. Most runtimes satisfy at most two.

So the defensible version of the claim is not that few environments have this. It is:

Evaluation inside the live application process is categorically more useful for debugging than shell access, most agent tooling does not offer it, and the runtimes where it works well are a short list on which the BEAM sits near the top.

That is narrower than the original and considerably harder to argue with.

The costs, stated honestly

An evaluation that only lists benefits is advocacy. Four real costs, each observed rather than hypothesised.

project_eval is arbitrary code execution in production. This is not a caveat attached to the feature. It is the feature. The same call that reads a permission map can delete a table, mutate user data or grant privileges. In recent work I created and deleted records in a real user’s database, granted a permission to a real account, and altered a live schema. Nothing in the tool objected, because nothing in the tool can distinguish a diagnostic from a mutation. Anyone running this against production is accepting that an agent, a compromised token or a mistyped expression has the same reach as a developer with a root console, which is precisely what it is.

There is no rollback. Every action lands immediately against real state. There is no transaction wrapping the session and no undo. The discipline has to come from the operator, because it does not come from the tool.

Remote connections are not always reliable. In recent use the remote connection timed out on several occasions, mid-investigation, with no partial result. When the technique works it is fast enough to feel free, which makes the failures more disruptive than their frequency suggests, because the workflow has already come to depend on them.

It does not replace tests, and can quietly discourage them. Runtime evaluation answers “is it working right now,” which is seductive precisely because it is so much faster than writing a test that answers “will it keep working.” In the same period, a test suite failed to compile for reasons unrelated to any of this, and no amount of runtime introspection surfaced that, because nothing was asking. mix test found it in seconds. The two tools answer different questions, and the fast one is not a substitute for the slow one.

A fifth, milder cost: the tool surface is shaped by the framework. An application that does not use the expected data layer loses the structured database tools and falls back to project_eval for everything. That is more powerful and less safe, since a structured query tool can constrain what it will do and an arbitrary evaluator cannot.

The verdict

On the first half: correct, and for a more specific reason than usually stated. The value is not that an agent can run code. It is that an agent can run code inside the process that already holds the state, which converts a class of debugging from inference into measurement. On a runtime built for live inspection, that is a large practical difference, and the examples above are ordinary rather than exceptional.

On the second half: directionally right, factually overstated. The capability is old, the specific product covers more than one ecosystem, and adjacent agent-accessible runtimes exist. What is genuinely uncommon is the combination of live-image introspection with agent access on a runtime that tolerates it well. That is a real advantage and does not need inflating to be worth having.

And a caveat that grows with the benefit. The more effective this becomes, the more it is worth governing. A tool that gives an agent unmediated evaluation inside a production system is the strongest possible argument for the surrounding controls: authenticate the caller, scope what a delegated credential may reach, audit every call, and keep the ability to revoke. An agent with a live console is enormously useful and is also the most capable insider you will ever grant access to.

The right conclusion is not to use it less. It is to notice that the reason it is so useful is the same reason it deserves a permission boundary around it.