Building Audit-Ready AI: FloQast Transform's Approach to Auditability

"We're screwed if it starts hallucinating and doing random things."

Billy Klein doesn't mince words when the topic turns to AI in accounting in the latest episode of Blood, Sweat, and Balance Sheets. As a Solutions Consultant at FloQast working on Transform, our AI automation platform, he has held hundreds of conversations with controllers and CFOs who are currently evaluating AI tools.

While headlines often focus on existential concerns about job displacement, there is a much more practical, immediate question keeping finance leaders up at night:

How do you make AI auditable?

When the auditors arrive, "the robot did it" is not a valid explanation. Here is why the distinction between "free-thinking" AI and deterministic scripts matters for your month-end close.

The Problem With "Free-Thinking" AI in Accounting

The challenge is consistency. Some AI tools can produce different outputs when processing the same data, making it difficult to establish a reliable audit trail.

"If I'm getting a different output every time, how do I review that?" Billy explains. "I'm learning a new workbook every time. That's craziness. There's no way that's efficient for anybody."

For accounting teams working under tight close deadlines, this is not just an efficiency issue; it is an audit nightmare. Your external auditors need to understand and verify your processes. If your AI tool produces different results each time it runs, explaining that variance to firms like EY, Deloitte, or KPMG becomes nearly impossible.

You simply cannot explain it away. Consequently, adoption stalls, and the team goes back to manual spreadsheets.

Letting Accountants Own the Edits 

The stakes become even clearer when you consider the severe time constraints of the month-end close. We all know the drill: the books need to be closed, and the clock is ticking.

Billy offers a concrete scenario that every accountant will recognize:

"If I'm helping you automate close, that's kind of a crazy time. You're going to have three days to do everything… If something were to break and I have to put a ticket into IT, if I have to wait three days for IT to get back to me and fix the problem, that's a huge issue for us. I'm going to get yelled at because we're not closing the books on time."

Revenue recognition cannot wait for IT ticket response times. Reconciliations cannot pause while you troubleshoot why the AI gave you a different result than it did yesterday. In the world of accounting, repeatability is not a "nice-to-have" feature — it is table stakes.

Two Types of "Auditability" (And Why You Need Both)

Here is where the conversation often gets murky. Billy has found that "auditability" is a broad term that companies often misunderstand.

"There are two components that we talk about," he explains. "We have our SOC report, ISO 42001 — those are different than auditability. Those are a component of auditability. But auditability is how we build our product with the audit in mind."

Let's break that down into two distinct buckets.

Component 1: Compliance Certifications

This category includes the standard badges of honor you see in website footers:

• SOC 2 Type II reports
• ISO certifications
• Security and data handling protocols

These are vital. They prove your vendor takes security seriously and has established controls. Almost every enterprise software vendor will put their SOC reports on their website. However, these certifications alone do not solve the auditability challenge regarding the work the AI is doing.

Component 2: Building With Audit in Mind

This is the deeper work. It involves designing your AI system so that the outputs themselves are auditable, reviewable, and ready to pass scrutiny from external auditors.

"Who do you want to partner with?" Billy asks. "Who are the nerds about audit and are going to make sure that the technology you're using is going to get the stamp of approval from EY, Deloitte, KPMG?"

One Approach to Auditability: Using AI to Generate Scripts

So, how does FloQast make AI auditable? The answer lies in shifting what the AI is actually doing.

"AI is really, really good at coding," Billy emphasizes. "Like really. Like the best thing. It is insanely good at it. And it writes very, very complex code too, especially for accounting processes where things change, the data is a little different every time, a lot of complex rules."

Instead of having AI generate free-form outputs that might vary based on how it "feels" that day, FloQast Transform uses AI to write repeatable scripts. Here is how the workflow operates:

1. Analysis: The AI analyzes your specific accounting process.
2. Creation: The AI writes code (a script) that executes that process.
3. Execution: The script runs the same way every single time.
4. Verification: You can review, edit, and audit the script itself.

"We are actually using it to write that code so that it is repeatable, it is auditable, it is going to return the exact same result every time," Billy explains.

With scripting, you are not reviewing a mysterious black box. You are reviewing a defined process—one that happens to be written by AI but executes deterministically.

The Internal Checkpoint: Compliance’s Seal of Approval

FloQast takes this audit-first approach seriously enough to have built internal checkpoints before anything reaches customers. Billy references Vicky LeVay, FloQast's Vice President, Information Security & Compliance, who serves as a crucial testing ground.

"We make sure that all of that passes. We've done our own independent audits, work with our internal auditors and just like, okay, break this. What is this not doing that you would just not be okay with? What do I need to show you for you to be like, yep, good. That's doing exactly what I want. You've checked all the boxes. We're good."

It does not stop there. FloQast works with implementation partners who also perform audits, getting their stamp of approval before going to market.

"They don't want to be deploying technology that's not going to be auditable either," Billy notes.

The result is a comprehensive three-layer verification process:

1. Internal compliance review 
2. External audit partner review
3. Customer's own audit requirements

What This Means for AI Evaluation

If you are evaluating AI tools for your accounting team, you need to look beyond the marketing hype. Here are the five questions you should be asking vendors:

1. "How does your AI generate outputs?"

Look for answers about scripting, deterministic processes, or code generation. Be wary if the answer is simply "our AI is really smart."

2. "Will I get the same result if I run this twice?"

If the answer is anything other than "yes, every time," you need to dig deeper. Variability is the enemy of the audit.

3. "How do you handle auditability beyond SOC 2 compliance?"

Security certifications are important, but they do not solve the audit trail problem for the actual accounting work.

4. "Can I review and edit the automation if something breaks?"

During a three- to five-day close window, you cannot afford to wait for vendor support tickets. You need control over your own tools.

5. "Have external audit firms reviewed your approach?"

If they have worked with Big Four or regional firms to validate the audit trail, that is a strong signal of reliability.

The Bottom Line

"How we go about that auditability via the script is crucial," Billy says. "And whether they ask it or not, I make sure that they hear."

The future of AI in accounting is not about replacing accountants. It is about giving them tools they can actually use and trust. We need tools that auditors will sign off on, tools that produce the same result every time, and tools that free up accountants to do the strategic work that actually adds value—like explaining the "why" behind variance analysis instead of just documenting the "what."

However, none of that happens without thoughtful approaches to auditability from the ground up.

‍

Want to go deeper on AI automation in accounting?

Billy Klein shares more insights on FloQast Transform's implementation approaches, what customers actually want from AI tools, and his journey from Big Four auditor to Solutions Consultant in the full Blood, Sweat & Balance Sheets episode.

‍