That last sentence is the whole article. The cost of writing a functional test has collapsed. The cost of knowing what to test, and of checking that what was written is right, has not moved. If anything, it now matters more, because it is the only part left that a person has to do.
What we showed
The setup was deliberately ordinary. A clickable prototype of a home banking application, the kind we build early in an engagement so a client can see screens before anyone writes production code. Playwright, the open source browser automation framework, installed in the repository. Claude Code, the coding agent from Anthropic, connected to Playwright through the Model Context Protocol, which is the standard that lets an agent call an external tool as if it were one of its own functions. Nothing here runs in a special cloud. It runs on the engineer's laptop, against a code base the engineer already has open.
The first thing I did was ask the agent to run an existing test. It found the command, ran it, and a browser tab flashed open and closed. Playwright produced its usual report: which test ran, which steps executed, how long each took, pass or fail. So far this is just a test runner with a chat window in front of it.
The second thing is where it gets interesting. I gave the agent a short paragraph in plain language. Open this page. There is an access gate, here is the key. Sign in as a private customer with these credentials. Go to transfers. Make a SEPA transfer of 250 euros to this IBAN. Confirm with the one time password. Tell me what you click at every step.
The agent opened the browser and did exactly that, narrating as it went. Dashboard shows this customer, this balance, these accounts. Transfers page, origin account selected, destination filled, amount entered. Confirmation screen, password accepted, success message displayed. It was reading the screen as it went, so when I asked it to confirm the elements were where it expected them, it could.
Then the third step, and this is the one that changes the economics. I asked it to take the flow it had just walked through and write it down as a Playwright test file, reusing the existing login helper. It wrote a script that any engineer can read: the steps, the selectors it clicks, the waits, the inputs, the assertions at the end. From that moment the test runs without the agent. It runs from the terminal, in the pipeline, on every deploy from development to quality to production, and it consumes no tokens to do so.
The negative case, and the thirty after it
A test suite that only checks the happy path is a demo, not a suite. So I asked for a second case in the same file: the same transfer, but with an IBAN that is too short. PT50 followed by four digits, where a real one has twenty one. Expected result: the page rejects it with an invalid IBAN message. The agent added the case, ran both, and the report now showed the successful transfer and the rejected one side by side.
Then I asked a lazier question, on purpose. Can you identify positive, negative and edge use cases for testing the transfers journey? It came back with a list. Transfer between own accounts. International transfer with the currency parameter changed. Scheduled transfer. Standing order. Save as template. Missing origin account. Insufficient balance. Cancel mid flow. Continue button stays disabled until the form is valid. Around thirty of them.
I asked it to script them all. It split the work across several sub agents. One wrote the negative validation cases. One read the transfers code and the existing test files to match conventions. One checked how the wizard steps and labels were built so the selectors would be stable. Within a few minutes there were new test files in the repository with names like transfers validation negative path coverage, and the agent was still adding to them when we closed the call.
My estimate, said out loud on the call, was this. If I had written those thirty tests by hand, it would have been about a week of work. Reviewing what the agent wrote, one by one, to confirm each test checks what it claims to check, is perhaps four hours. That ratio is the reason to care.
Where the bottleneck moved
Here is what I told the team, and what I would tell any client who asks whether this is real.
The pain of scripting tests is gone. The pain of thinking about tests is exactly where it was. When I typed a short, lazy prompt, I got a reasonable list. That list came from what the agent could infer from the code and from thin documentation in the repository. It was not wrong, but it was generic. If I had handed it the specification of what was actually built, the personas who can perform each action, the business rules for what is valid and what is not, and the flows that matter to the client, the list would have been sharper and the assertions stronger. The quality of the tests is bounded by the quality of the context, and the context is a human deliverable.
This is not a small point for how we run engagements. It is why we are moving our discovery and shaping work away from user stories and towards detailed specifications. A user story was designed to prompt a conversation between a product owner and a developer. When the developer is an agent, the conversation does not happen. What the agent needs is the specification itself: the rules, the states, the exceptions, written down. The same document that lets an agent build a feature correctly is the document that lets it test the feature completely.
Speed without structure is risk. We say that about generated code, and it applies with equal force to generated tests. Thirty scripts that assert the wrong thing give a team false confidence faster than no scripts at all.
What does not change
Three things stay with the people.
The first is validation. The agent drafts. A tester decides whether the draft is a test worth keeping. The tester is not removed from the process. The tester is removed from the typing. On the call I was careful to say that I had not opened any of the generated files, and that I would not trust them until I had. That is the correct posture, and it is the posture we hold in client work.
The second is the feedback loop from real users. During acceptance testing, someone will always report that they pressed a button and the wrong thing happened. That report used to become a ticket. Now it becomes a ticket and a test. The scenario the user described is fed back to the agent, scripted, and added to the suite, so the next deploy checks for it automatically. Every defect a client finds makes the regression suite permanently better. That is the compounding effect we are after.
The third is deciding where the tests live and what they are allowed to touch. On the call, one of our engineers raised a client whose data sensitivity is high and non negotiable. The answer is that the AI is only involved when the tests are being authored, and it runs on the engineer's machine against a development environment. The scripts it produces are plain code, reviewed and committed, and they run deterministically afterwards with no model in the loop. Production data never enters a prompt. That was already our rule, and this way of working does not require relaxing it.
The enterprise platform case
Much of our work runs on enterprise platforms such as OutSystems, where the application code base does not live in a repository you can drop test files into. The pattern is the same, with one adjustment: the tests live in their own repository and run against the deployed application over the browser, exactly as Playwright is designed to do. The agent helps author them, the pipeline runs them.
Nor does it need one specific agent. In organisations that have standardised on a different assistant, the same recipe works. The assistant may not orchestrate five sub agents and produce thirty scripts unattended, but it will plan the cases with you, help you implement the first, and then the second, and the working pattern is identical. The recipe matters more than the tool.
What this means for a client
If you are a technical lead, it means regression testing stops being the task that gets cut when the sprint runs long. Every feature ships with a suite that runs on every deploy, and extending that suite is measured in minutes.
If you own the business outcome, it means acceptance is evidence based. You get a report that shows which journeys were exercised, with which inputs, and what happened. When your users report a problem, you can ask to see the test that now covers it.
For us at Bool, it is one more place where the same principle holds. AI handles the repetitive. Our team focuses on the context, the specification, and the judgment about what good looks like. Human teams, AI amplified. Anyone can generate test scripts with an agent. Our job is to make sure they test the right things, run in the right place, and touch only the data they are allowed to touch.
Try it before you believe it
The homework I gave the team applies to anyone reading this. Install Playwright. Point your coding agent at a public website, Wikipedia is a good candidate because it has real buttons and real links and no login. Ask the agent to drive a journey, then to script it, then to find the negative cases. Read what it wrote. Decide what you would have done differently.
You will learn two things in an afternoon. The first is that the mechanics are simpler than you expect. The second is that the hard part was never the mechanics.
If you want to see how this fits into a real delivery, with the specification work in front of it and the governance around it, we are happy to show you. That is what we do.
