What an Agent Doesn’t Know It Doesn’t Know

3–5 minutos

·

·

Agent context boundary exposing an implicit decision outside its visible context

This one is part of the Graph Saga, a series about what I am learning and needing around agentic AI applied to code, and to projects like ALDC. The aim is always to bring it closer to Business Central. Not every post will do that on the surface, but underneath and between the lines it is there. The pieces are not sequential. They are, if you allow me the licence, a graph: interconnected, readable in any order.

And in that same spirit, a line of my own: what a person does not know is what the agent does not know that it does not know.

That is the collision point. The point of no return where you go into a spin of technical debt in agentic programming, and in AL it is probably sharper than elsewhere. In the previous post I left you some questions and laid out the two camps. Here I give you my own read.

Agent context boundary exposing an implicit decision outside its visible context

Ng accidentally put a name on it

In his latest piece, Andrew Ng lays out his map of AI engineering skills. When he reaches software fundamentals, I think he has unintentionally named what the more sceptical camp from the previous post was calling strategic programming.

A warning before I go on: Ng writes for people building AI applications, not for someone who has spent fifteen years putting code into an ERP. The translation into Business Central is mine, and if I get it wrong that is on me, not on him.

His idea is simpler than it looks. Anyone who programs by firing off instructions, without understanding how the thing they are building actually works, can get a simple extension out of the door. Along the way they will make maintenance cost decisions on their own, and the problem, according to Ng, is not that they decide badly. It is that they never knew there was something to decide. That nuance is practically the whole article.

The gap is not in writing code

The gap is not in writing code. It is in not knowing what to ask. In not knowing which questions to put to people before a single line gets written.

That brought back something I read a few weeks ago about false edges in a flow: at every step, the next one needs the result of the previous one. What Ng describes works as a similar test, but applied to judgement rather than to the flow.

For each thing you hand to the agent: do you know what is being decided at that moment? Do you know what gets lost by choosing one option and not the other? If the honest answer is no, congratulations. You have just found your frontier.

Cherny’s uncomfortable question

And this connects directly with Cherny’s uncomfortable question. Do you remember it?

When somebody tells me they feel outpaced by Claude Code or by Copilot, they almost always talk about speed, and about the agent writing better code than they do. Possibly true.

Fine. But scratch a little and the person who feels outpaced is usually the one who never had to decide whether that went in a table extension or in a new table, whether that calculation was a flow field or should be persisted some other way. What happened to that integration when the customer had ten times, a hundred times more records in the feature or in the tenant? Were all those decisions taken by someone else, or by nobody at all? We have all suffered the result in one development or another.

So the question changes shape

The question from the previous post changes shape. It is no longer tactical or strategic. It is how much of what has to be decided in an extension you are delegating without saying so, and whether you even know what is being decided there.

In Business Central that list tends to be very long. You will almost never find it directly in the repository, although part of it is there.

And I will say it up front: you are not going to find the answer in any frontier model either. Not in the most powerful one available.

So, a reminder. Are you really the one taking the decisions? Who is the actual copilot here?

A final note

You can carry on with any of the Graph Saga posts in any order, as I mentioned. I hope you enjoy them, I am looking forward to your comments, and let us see whether we manage not to get lost along the frontier.

Strategic programming in Business Central, and how to find the line where your own judgement runs out.

Deja un comentario

Feature is an online magazine made by culture lovers. We offer weekly reflections, reviews, and news on art, literature, and music.

Please subscribe to our newsletter to let us know whenever we publish new content. We send no spam, and you can unsubscribe at any time.

← Volver

Gracias por tu respuesta. ✨

Designed with WordPress.