Automated Control Testing: What an AI Agent May Do

Automated control testing in a bank means using software to run, repeat and document the checks that show whether a control works. An AI agent can take over a defined part of that work: it drafts test steps, pulls evidence from source systems and assembles the file. It does not decide whether a control is effective. The control owner makes and signs that decision, and every step the agent takes is logged.

This article sets out where that line sits, why it matters under Swiss and EU supervision, and how to design an agent so that a second line, internal audit or a supervisor can follow what it did.

S-PRO

Make Control Testing Easier to Review

S-PRO builds AI agents for financial services, including solutions designed around regulatory requirements and human review.

Explore S-PRO’s AI solutions

TL;DR

An AI agent can draft test steps, collect evidence and prepare the working file. The control owner reviews the result and signs off.
Every action, input, output and approval is logged, so a reviewer can reconstruct the test later.
FINMA’s AI survey of April 2025 shows AI is already in day-to-day use at Swiss institutions, with data quality, data protection and explainability among the main concerns.
DORA requires regular ICT testing for in-scope entities. It does not say the testing must be done by an AI agent, and neither does FINMA.
Start with one control family, a narrow evidence scope and a human approval step, then widen.

What automated control testing covers
What an AI agent may do in control testing
What an AI agent may not do
The accountability model: who signs, who reviews
What the logs must contain
What Swiss supervision says
What DORA adds for EU-facing institutions
Where to start: a first use case in six steps
Common failure modes
Our perspective
Building the agent with a partner
How S-PRO helps
FAQ
Sources

What automated control testing covers

Control testing answers one question: did this control operate as designed, over a defined period? The answer rests on evidence. For a manual control, such as a four-eyes approval on a client limit change, the tester reviews documents or observes the process. For an automated control, such as a system-enforced access restriction, the tester looks at configuration, system logs or exception reports.

In practice, “automated control testing” covers three layers that are often mixed up:

Rule-based test scripts. Deterministic checks that run on a schedule: does every leaver have disabled access within the policy window, does every payment above a threshold carry two approvals. These need no AI and should not use it.
Evidence collection. Pulling logs, tickets, screenshots, extracts and approvals from several systems into one file, with timestamps and source references.
Judgment-heavy preparation. Reading a policy, a procedure document or a ticket history and drafting the test steps or a first assessment of whether the evidence fits the control description.

AI agents add most in layers two and three. Layer one is better served by plain code, which is cheaper to validate and behaves identically every time. A good design uses each tool where it fits.

What an AI agent may do in control testing

The workable rule is simple: the agent prepares, the human decides. In concrete terms, an agent can:

Draft test steps from the control description, the risk it mitigates and the relevant policy text, for a tester to edit and approve.
Select samples according to a sampling rule that the control owner or audit function has defined, and record how the sample was drawn.
Collect evidence from read-only connections to ticketing, identity, core banking, document management and logging systems, and attach each item to the right test step.
Compare evidence with the control description and flag gaps: a missing approval, a date outside the period, an exception without a ticket.
Draft the working paper, including a summary of what was found and which items need human attention.
Keep the file current by re-running the same steps in the next cycle and highlighting what changed.

The common thread is that each output is a proposal inside a defined workflow, with a visible trail back to the source data.

What an AI agent may not do

The boundaries matter as much as the capabilities. As a design rule for regulated institutions, an agent used in control testing does not:

Make the control decision. Whether a control is effective, deficient or needs remediation is a judgment owned by a named person. The agent may suggest a rating draft; it cannot record one as final.
Sign off its own work. Approval needs a human identity, a timestamp and a recorded outcome.
Change production systems. Evidence access is read-only. An agent that can alter the thing it tests no longer provides independent evidence.
Widen its own scope. The systems, controls and periods it can touch are set by configuration under change management, not by the agent.
Work without a log. If a step cannot be logged, the agent does not take it.

These limits are design choices, not legal quotations. No Swiss or EU text we cite below says “an agent must not sign off”. They follow from how supervisors read accountability: a person owns the control, and a person owns the conclusion about it.

The accountability model: who signs, who reviews

A clean split keeps the roles readable to an auditor.

Step	Agent	Human
Draft test steps	Proposes from control description and policy	Control owner or tester edits and approves
Define sample	Applies the approved rule and records the draw	Audit or second line owns the rule
Collect evidence	Retrieves with read-only access, tags source and time	Reviews completeness on exceptions
Assess evidence	Flags matches and gaps against the control text	Decides effectiveness
Record conclusion	Prepares the draft file	Control owner signs off
Remediation	Suggests a ticket draft	Owner assigns and tracks

The human column is where responsibility stays. If you cannot name the person in each row, the design is not ready.

What the logs must contain

A log is only useful if it lets someone else reproduce the reasoning. For each agent action, we recommend capturing:

the control, test step and period it belongs to;
the input: which documents, queries or system extracts were read, with versions or hashes;
the instruction and model configuration in force at the time;
the output, including what the agent flagged and what it left out;
the human action that followed: edited, accepted, rejected, with user and time;
the final sign-off and the identity behind it.

Store these logs where the agent cannot edit them. Treat them like any other audit record, with retention rules set by your compliance function.

What Swiss supervision says

Switzerland has no AI-specific statute for financial institutions. FINMA’s approach is technology-neutral and principle-based, and existing requirements on governance, risk management and internal controls apply to AI too. That is the thrust of FINMA Guidance 08/2024 on governance and risk management when using AI, which asks supervised institutions to identify, assess and manage the specific risks of the AI they deploy. For a control-testing agent, that means the agent itself becomes something your control framework covers: an inventory entry, a risk assessment, an owner, validation and monitoring.

FINMA Circular 2023/1 on operational risks and resilience, in force since 1 January 2024, sets the frame for how banks manage operational risk, including technology and critical functions. Control testing is one of the ways an institution shows that this frame operates in practice. Nothing in the circular requires that testing be automated. What it asks for is that you can demonstrate your controls work, and a well-logged agent can make that evidence easier to assemble.

FINMA also looked at how widespread AI has become. In its survey of around 400 institutions, published on 24 April 2025, about half reported using AI or having first applications in place. The survey names data quality, data protection and explainability among the top risks. Each of the three lands directly on a control-testing agent:

Data quality: an agent that reads incomplete or stale evidence produces a confident but wrong draft. Completeness checks belong in the workflow.
Data protection: control evidence often contains client data. Hosting, access and retention need the same care as in production.
Explainability: the reviewer must see why the agent flagged a gap. A summary with links to the source items is better than a score.

What DORA adds for EU-facing institutions

If you fall under DORA, Regulation (EU) 2022/2554, for example through an EU subsidiary or EU clients served from a branch, the regulation requires financial entities to test their ICT systems regularly. The tests range from basic vulnerability assessments to threat-led penetration testing, and they are meant to show whether preventive, detection, response and recovery capabilities hold up.

Testing under DORA is proportionate to risk. Larger, systemic and more ICT-mature institutions face more demanding requirements than smaller entities. Check which tier applies to you before you scope an agent around it.

Two practical points follow. First, DORA testing is a defined programme with its own governance; an agent can help prepare and document the work, but the programme owner remains accountable. Second, if the agent supports ICT testing, it becomes part of your ICT estate and the third-party and risk rules that apply to other ICT tooling are relevant to it.

If the AI system also falls under the EU AI Act, classify it and document it accordingly. A control-testing assistant working on internal evidence is typically a different case from a system making decisions about clients, but the classification is your institution’s to make, with legal review.

Where to start: a first use case in six steps

Pick something narrow enough to validate in a quarter and meaningful enough that the control owner wants it.

Choose one control family. User access reviews, change management approvals or payment authorization limits work well because evidence sits in a few systems.
Write the control and test description in plain terms. If humans disagree about what the control requires, an agent will not settle it.
Connect read-only evidence sources. Start with two or three systems and confirm the data is complete and timestamped.
Define the human gates. Who approves test steps, who reviews exceptions, who signs off. Name roles, then names.
Run in parallel with the manual test for one cycle. Compare the agent’s file with the tester’s file and document the differences, both ways.
Decide on scope with evidence. Extend to the next control family only when the logs and the parallel run support it.

We do not publish time-saving figures for this kind of work, because they depend on your control library, the quality of your evidence and your sampling rules. Measure your own baseline in step five and decide from that.

Common failure modes

Treating the agent’s draft as the conclusion. Reviewers skim, accept and move on. Counter it by requiring a recorded action on every flagged item and by sampling accepted items for second review.
Evidence without provenance. A screenshot with no source or time is weak. Capture where each item came from.
Agent access that is too broad. Give it only what the test needs, and review that access like any service account.
No change control on prompts and models. A changed instruction or model version can change outputs. Version them and log which version produced which file.
Ignoring the control library’s quality. Vague control descriptions produce vague tests, whether run by a person or an agent.

Our perspective

Control testing is a good example of where restraint pays. The value is not in an agent that appears to do everything. It is in an agent that does the repetitive, evidence-heavy preparation reliably and leaves the judgment where the regulator expects it: with a named person.

A pilot built to reach production has a few traits. The scope is narrow. The human approval step is designed first, not added after a review. Logging is treated as a product feature, since it is what lets a second line, internal audit or FINMA follow the work. And documentation for the EU AI Act, FINMA and DORA is prepared alongside the build, not after it.

We would also caution against reading regulation as a push toward automation for its own sake. The sources are clear about governance, risk management and regular testing. They do not say that testing must be automated, and they do not shift accountability from people to systems. An agent earns its place when it makes your existing accountability easier to evidence.

— Igor Izraylevych

Building the agent with a partner

If you are weighing a control-testing agent, the build decisions that matter are access model, logging design, human gates and hosting. Our team designs and delivers AI agents for banks and financial services with audit trails and human-in-the-loop controls, from pilot through to production. S-PRO is a Certified partner in the Claude Partner Network, and our team includes Claude-certified engineers and architects, which is relevant if your institution is evaluating Claude-based agents.

How S-PRO helps

S-PRO is a Zurich-based software development company for banks, private banks, wealth managers, asset managers and fintechs, founded in 2014, with 250+ engineers and ISO 27001 and ISO 27701 certification. For control testing and wider compliance workflows, we:

scope a first use case with your control owners and second line, including the human sign-off design;
build agents that collect evidence from your existing systems with read-only access and full action logs;
prepare the documentation your compliance and risk teams need for the EU AI Act, FINMA and DORA;
take the agent from pilot to production and integrate it with the systems you already run, including fintech software development work where your platforms need to change.

To discuss a first control family, start from our AI agent development page.

FAQ

Does FINMA require automated control testing?

No. FINMA’s guidance and circulars set expectations for governance, risk management and operational resilience. They do not mandate that controls be tested by automated means. Automation is one way to produce consistent, well-documented evidence, and the choice is yours.

Can an AI agent decide whether a control is effective?

In the design we recommend, no. The agent drafts steps, collects evidence and flags gaps. The control owner makes the effectiveness judgment and signs off, and the sign-off is logged with the person’s identity and time.

What is the difference between automated control testing and automated compliance testing?

The terms overlap. Control testing checks whether a specific control operated as designed. Compliance testing is broader and checks conformity with a regulation or policy, often by testing many controls. The same agent pattern applies to both: prepare, collect, flag, then human sign-off.

Does DORA apply to a Swiss bank?

DORA is an EU regulation, so it applies to EU financial entities. A Swiss bank may be affected through EU subsidiaries, branches or contractual requirements from EU clients. Check your exposure with legal counsel, and attribute requirements to the right regime in your documentation.

Which controls are good candidates for a first agent?

Controls where evidence sits in a small number of systems and the test is repeated every cycle, such as access reviews or change approvals. Avoid starting with controls whose descriptions are ambiguous or whose evidence is scattered across paper files.