— 11 reading minutes
When a company starts consulting its documents with an AI assistant, someone inevitably asks where those documents go. It's usually the legal department or a client filling out a vendor questionnaire, and the question almost always arises after the prototype is already functional. We're often asked this in discussions about custom AI development , and answering it requires reviewing several vendors.
If the requirement is that no data leaves Europe, this can be met today with open models served in Europe and the same RAG architecture we would use anywhere else. This comes at a cost in terms of convenience and operational work, and there are projects where it's not worthwhile to pay for it.
What is RAG?
Retrieval Augmented Generation (RAG) means that, before responding, the system searches your documents for relevant snippets and passes them to the template along with the question, so the answer comes from your sources and can cite them. To find snippets by their meaning, even if they don't share words, each piece of text is converted into an embedding —a list of numbers that represents what the topic is about. We explain the complete process and its limitations in the article on corporate RAG with your company's documentation .
Which providers see your data in a RAG assistant?
The server where the application runs and the database where the documents, their fragments and embeddings are stored .
The embedding model . Every document you index and every user query passes through it. It's configured at the beginning of the project, making it easy to avoid having to check where it's running again.
The model that generates the answers. In each query, the question and the retrieved fragments travel to the site where the inference is made.
Backups. In some services, where they are stored is a different setting from the database region.
Logs and monitoring. Application logs typically include the user's question verbatim, and the monitoring tool may be from a third-party company.
Transactional mail and payments, which handle names, email addresses, and invoices.
When we migrated Softime, our time tracking tool, to a European provider, we discovered that their payment gateway had its European headquarters in Dublin but was part of a US-based group, so we switched to a Dutch one. [CONFIRM: status of Softime migration and whether it can be mentioned before launch]
Under the GDPR, each of these providers is a data processor or sub-processor, with its own contract and location. If even one is located outside of Europe, it can no longer be argued that the data does not leave Europe.
The law to which the provider is subject also matters. A US company with a data center in Frankfurt stores your data in Frankfurt, but the US Cloud Act allows its authorities to request data from companies under its jurisdiction regardless of where it is stored (US Department of Justice ). How this affects you depends on the type of data you handle, and it's a question for your legal team and each provider: whether they can receive requests from authorities outside the EU and what they would do with them.
How we built it on a European provider
In September, we rebuilt our existing infrastructure on Scaleway, a French cloud provider, and wrote it entirely in Terraform. Terraform describes infrastructure as code: the same recipe produces the same environment, so it can be destroyed and recreated without constantly clicking buttons in a console, and the next project builds upon something already tested. We divided it into separate recipes for the core infrastructure, application deployment, and monitoring, so each part can be changed without affecting the others.
All the pieces are in the Paris region.
A private network with a managed and encrypted PostgreSQL database accessible only from within. Scaleway's managed PostgreSQL supports pgvector , the extension we use to store and search embeddings in our RAG projects, so documents, snippets, and vectors all reside in the same database.
The application uses serverless containers , which scale with traffic and do not require server management.
Monitoring is done with the provider's own metrics and logs service and Grafana dashboards, so the monitoring data also stays with the same provider.
Credentials are handled in such a way that they are never written to Terraform's state files.
For models, Scaleway's Generative APIs serve open models (Mistral, Llama, Qwen, Gemma, and gpt-oss, among others) and two embedding models . The provider states that it hosts them all in a data center in Paris and that it does not collect, read, reuse, or analyze your prompts or generated responses ( Scaleway Generative APIs , accessed October 2026).
The database using pgvector, the method for splitting documents, the semantic search, and the instructions that force the model to adhere to the retrieved fragments are the same as those we would use in Google Cloud. Only the service from which the embeds and responses are requested changes. When we set up our internal chatbot that queries our documents in Google Cloud with Gemini, we made the model and storage interchangeable, so a change like this doesn't require rewriting the application.
The September trial covered the infrastructure. The containers we deployed were sample applications, still without a RAG assistant on top of that. Shortly after, we moved Softime to the same vendor, and now we have a real application running there.
What did not work first time, and what you miss from Google Cloud
Initially, the Grafana dashboards were empty. Metrics and logs were being collected, but the data sources hadn't synced with Grafana, and the container dashboard didn't know where to read from. We forced the synchronization and added a step to the monitoring recipe that repeats it every time the sources change, so this wouldn't happen again in the next environment.
There were no pre-configured alert rules for the database or containers, so the alert system was activated without any rules. Here, you have to write them yourself and decide who to notify, something that large providers usually offer with reasonable default values.
In general, there are fewer managed services, so some of what a large provider delivers ready-made has to be set up and maintained by your team while the system is running. The range of models is also smaller: in a purely European infrastructure, you won't find the large, proprietary models from US labs. To answer using your own documents, a medium-sized open model is usually sufficient, but I would test it with your actual documents and questions before deciding, because in some cases the difference compared to a large model is significant.
When does a purely European infrastructure make sense?
A European-only infrastructure makes sense when a contract or a client requires European suppliers, when documents contain sensitive personal data (health, human resources, legal records), when you work in a regulated sector, or when "where is our data?" is a recurring question in your sales and you want an answer that fits in one sentence.
It's more difficult to justify if the documents don't contain personal data, if a large supplier with a European presence and a good contract covers everything your legal team requires, if you need the best available model for a demanding task, or if there's no one to operate the extra parts. In those cases, we would go to the large supplier's European region and focus our efforts on obtaining permits, conducting the evaluation, and documenting which supplier handles what.
There's also a middle ground: the application, database, and embeds are all located in Europe, while the generation model uses a provider that processes data within a European region, with this decision documented and explained to anyone who asks. In many projects, this is sufficient.
What GDPR asks, and what the AI Act asks
The GDPR does not require personal data to remain in Europe. It requires a legal basis for processing, a contract with each processor specifying what they can do with the data and when they can use sub-processors ( Article 28 of the GDPR ), and, if the data leaves the European Economic Area, a valid transfer mechanism as per Chapter V. Keeping the entire chain within Europe saves you from analyzing international transfers; however, contracts, the legal basis, and access permissions still need to be reviewed.
The AI Act regulates what an AI system does and how it is used, leaving the question of data storage to the GDPR. The transparency obligations of Article 50, such as notifying users that they are interacting with an AI, have been in effect since August 2, 2026. The obligations for high-risk systems in Annex III were postponed to December 2, 2027, by Regulation (EU) 2026/1744 (see FPF timeline ). An assistant that responds with internal documentation is not typically considered high-risk. We review the dates on which Article 50 of the AI Act already applies, what comes into effect in December , and the traceability aspect of AI governance without black boxes .
In our custom AI projects, we review the supply chain with you, supplier by supplier, and tell you if a European-only infrastructure is worthwhile. Tell us about your situation .