The validator that would have passed while checking nothing

What I was doing

I pulled my agent teams into one central hold, a single place where the definitions live, and then surfaced them back into each project through a symlink so every project still sees its own team where it expects to. Sensible cleanup. One source of truth, many views onto it.

There is a contract validator that runs over those agent and skill definitions to make sure they are internally consistent. As long as it stays green, I trust the definitions are sound. So this consolidation was not supposed to touch correctness at all. It was supposed to be plumbing.

Where it went sideways

The symlink raised a question I almost waved off as cosmetic: do I link the whole directory, or do I link each individual file inside it. Both produce a working layout. The project sees the files either way.

Here is the part that turned a layout choice into a correctness choice. My validator deliberately skips symlinked files, in several separate layers of its checks. That is a reasonable thing for it to do in isolation. But it means the granularity of the link decides what the validator can see. If I had linked file by file, every single file would have been a symlink, the validator would have skipped every one of them, and the whole suite would have passed by examining none of them.

That is the trap. Not a red that I would have chased down, but a green. A clean, confident all-green that actually checked zero files and would have happily hidden every contract violation sitting underneath it. I would have read that green as "the definitions are sound" when it really meant "I looked at nothing."

Why it is worth logging

The validator's green was contingent on a design choice it never announced. Nothing in the output would have said "by the way, I skipped everything." It would have looked exactly like a real pass. Pick the wrong link shape and the same green flips from meaningful to meaningless, silently.

Pick the wrong link shape and the same green means the validator looked at nothing. A pass is only worth the files it actually inspected.

So I linked at the directory level. That keeps the underlying files real from the validator's point of view, real enough that it opens them and checks actual content instead of skipping past a symlink. A file-level link would have made each one invisible to it. The difference between a check that means something and a check that means nothing came down to the granularity of a single link, and I would never have guessed that from the outside. It is the same shape as the other verification failures I keep running into. The check can be present, named, and green, and still be inspecting nothing. The only way to know is to understand what it actually touched.