AI Governance in Companies: How to Comply with the EU AI Act Without “Black Boxes”

(Updated October 2026) - Many medium-sized companies already use AI in some process: a support chatbot , an email sorter, an assistant that summarizes contracts. The challenge now is to bring it under control, because the EU AI Act already has obligations in place and clients are starting to demand guarantees. Much of that control depends on how the system is designed, so it's as much a technical issue as a legal one.

We call this AI governance in companies : knowing what each system does, what data it works with, and who is responsible for what it produces.

AI is often described as a black box. Data goes in, a response comes out, and nobody can explain what happened in between. In a company, that's equivalent to having an undocumented risk.

AI governance is the set of processes, architectural decisions, and controls that allow us to understand and document how automated systems behave. In practice, for each system, we must be able to answer these questions:

• What data does the model use and what services are sent?
• Why did he give this answer or make this decision?
• Who validates the result?

If an agent serves customers or processes internal documentation, the company needs to be able to review what they have done and justify it when someone asks.

The European regulation on artificial intelligence is being implemented in phases starting in 2024. In Spain, the reference body is the Spanish Agency for the Supervision of Artificial Intelligence (AESIA). The EU AI Act in Spain classifies systems according to their level of risk:

• Unacceptable risk. Practices banned from February 2025, such as social scoring or subliminal manipulation.
• High risk. Systems that affect fundamental rights or critical infrastructure, such as AI that filters resumes or assesses a client's creditworthiness. These require technical documentation, risk management, record-keeping, and human oversight.
• Limited risk. Chatbots and synthetic content generation. Here, the main obligation is transparency; that is, the user must know they are talking to a machine and the generated content must be clearly marked.
• Minimal risk. Spam or AI filters in video games, with no additional obligations.

Transparency obligations apply from August 2026. For high-risk systems the planned date was also August 2026, but the Digital Omnibus package, in force since July 2026, delayed it to 2 December 2027 for Annex III systems (employment, credit, education or access to essential services) and to 2 August 2028 for AI integrated into regulated products.

This window of opportunity should be seized, because B2B customers are likely to demand guarantees about AI use before signing with a vendor, without waiting until 2027. For CTOs and product managers, AI compliance begins with taking stock of the AI systems in the company, what each one is used for, and what level of risk it poses.

To comply, more than just a compliance document is needed. The architecture must be documented, and integrations with external providers like OpenAI, Gemini, or Claude must be monitored. In a technically focused AI audit, we review three areas.

1. Inventory and mapping of dependencies

The code and integrations are reviewed to identify all calls to external language models and what company information is transmitted within them. If these calls pass through a single point, it's easier to control what is sent, log usage, and switch providers when necessary.

2. Data control and hallucinations

Generative models generate answers when they lack information or when their scope is not properly defined. The standard way to limit this is through a Retrieval-Augmented Generation (RAG) architecture, in which the model responds based on validated company documentation. During the audit, we verify whether the system adheres to these sources or has room to improvise, and what it does when it cannot find the answer.

3. Traceability and records

Every decision that AI makes or suggests must be recorded. If a customer complains about something that an automated system initiated, the technical team must be able to see which prompt , context, and model version produced that result.

At Softspring, we work with what we call responsible AI engineering . AI proposes, and a person decides, especially when the decision affects customers, money, or someone's rights.

This approach, known as human-in-the-loop , improves safety and also the quality of the outcome. There are well-known cases of what happens without it. In 2024, a Canadian court ordered Air Canada to compensate a passenger because the airline's chatbot had explained a refund policy that didn't exist. The company argued that the chatbot was responsible for its own responses, but the court rejected this argument.

In our projects, AI prepares drafts, extracts data, and summarizes, while approval is handled by a team member. For example, in "The Psychologist at Home," the recommender suggests psychologists to the coordinator based on their specialization and availability, and the coordinator makes the final selection. We also ensure that the review process is quick, because if it takes too long, people end up approving without looking.

With our technology consulting service, we help technology and product teams review their existing systems, take an AI inventory, and streamline their architecture.

If something new needs to be built or something that cannot be audited needs to be redone, we do it within our custom AI development service, using Symfony, Python, and deployment on Google Cloud, with the logs and human oversight planned from the design stage.

We've been building software for businesses for over 15 years. If you'd like to learn more about how we work, you can meet our team .

If you want to know the risk level of your systems or need to develop a new one that can be audited, tell us what you have and we'll look at it together.

Contact our engineering team