Your AI Saying “It Works” Does Not Mean It Works

AI finishes implementing your feature and gives you a detailed summary.

Everything (and beyond) is complete. Errors are handled. The design is responsive. Architecture is enterprise grade and security is quantum safe (yea that’s an actual term).

You open the application. The page crashes.

So what exactly did it verify?

Sometimes, way less than the summary suggested. The AI changed the code, looked at what it wrote and decided that it should work. Which is a reasonable starting point, but quite an optimistic place to stop.

Writing the code is only part of the task

Let’s say you ask AI to add a contact form.

It creates the fields, adds validation (hopefully), connects an email service and displays a success message after submission. Everything you requested appears to be present in the code.

But has it actually submitted the form?

Does the email arrive? What happens if the email service is unavailable? Is there any retry mechanism, and and does it eventually stop instead of trying forever? Can someone click the submit button five times while the first request is still processing?

These are different questions from “did you add a contact form?”

If you are building without a software background, you should not need to anticipate every possible implementation mistake. You can, however, require the AI to check its work and explain what those checks covered.

Start with the checks the project already has

Most established projects have commands that can catch certain problems automatically.

Depending on the setup, these include:

• Build: checks whether the application can be prepared for deployment.
• Type checking: catches incompatible values and incorrect use of typed code.
• Linting: checks code against configured rules that can flag mistakes and inconsistent patterns.
• Tests: check specific behaviors someone has written assertions for.

You do not need to memorize the commands. Ask AI to inspect the project and run the relevant ones.

God to keep in mind though: passing these checks does not prove the whole application works.

A build can pass while your contact form sends nothing. Tests can pass because none of them cover the feature that just changed.

A green terminal is useful, but does not confirm that an email arrived.

Ask it to actually use the feature

If your AI tool has browser access, ask it to open the application and try the flow it just implemented. How it does it exactly may vary - it may actually open a tab in your browser - if you have the plugin enabled - or run a headless (no window / tab actually opens) instance and even use something like playwright (integration / e2e testing framework) under the hood.

For the contact form, that might mean:

• submitting valid information;
• submitting an empty form or an invalid email address;
• checking what appears while the request is processing;
• checking what happens when submission fails.

It should also verify the intended result where it has access. Seeing “Message sent” on the page is not the same as confirming delivery.

The checks should fit the change. Updating a button label does not require an afternoon of automated testing. Adding a checkout flow deserves considerably more attention.

Also ask it to check existing behavior that the change could reasonably affect. If it modified a shared form component, the contact page may not be the only place that needs a look.

Sometimes it cannot check everything

The AI may not have browser access, test credentials or access to the service needed to verify the result.

That is fine, provided it tells you.

“Implemented, build passed, email delivery not verified because I cannot access the email service” gives you useful information.

“Everything works perfectly” does not, if delivery was never checked.

You can then perform the missing check yourself or provide an appropriate test environment. The important part is knowing what remains uncertain.

A prompt you can reuse

Add this to your implementation request, or keep it in your project instructions:

Treat verification as part of the task. Inspect the project instructions and available scripts. After making changes, run the relevant build, type checks, linting and existing tests. Report failures, including whether they appear related to your changes. Verify the changed behavior directly where your tools allow it. For user-facing changes, use the browser if available. Check the normal flow, relevant error cases and existing behavior likely to be affected. Do not treat a passing build as proof that the feature works. Verify the intended result, not just the appearance of a success message. Do not remove, disable or weaken checks just to make them pass. In your final response, briefly state: - What changed. - What you actually ran or tried, and the results. - What you could not verify and why. - Any specific checks I still need to perform. Clearly distinguish verified behavior from behavior you only expect to work.

This will not catch every bug. Even properly tested software has bugs, as anyone who has installed a software update can probably confirm.

But it gives you a much better basis for deciding whether the work is finished.

The next time AI says “it works,” you should be able to see what that statement is based on.