“Private AI” can mean an enterprise account with an external provider, an API under particular terms, a model in a controlled cloud environment or servers owned by the organisation. The label alone tells you neither where requests are processed nor who can inspect logs.
Start with the task. An assistant for public manuals may need a different design from one searching case files containing personal data. Map each data path and its owner before comparing deployment options.
This is a technical selection method. Legal basis, contracts and international transfers for a specific processing activity belong with data-protection and legal specialists.
Map the full data path first
List user input, retrieved documents, the prompt received by the model, its response, application logs and backups. Include external tools. A local model offers limited protection if another service receives the documents for indexing.
A checklist for a document assistant. Each transfer needs a purpose, permission and known destination.
- Person
Asks under an identity and access rules.
- Application
Checks access and prepares the query.
- Search
Retrieves only authorised passages.
- Model
Processes context and produces an answer.
- Logs
Retain only what operations and incident review need.
Portocelo’s conceptual diagram. Documents, indexes and logs may reside in different locations.
Before using real data, ask where inputs are processed, what is stored and for how long, who can access it, how a query is deleted and what changes when optional history or web search is enabled. The Spanish Data Protection Agency’s 2026 recommendations for its own generative-AI systems are a useful method reference, not a universal prescription for one architecture. Source checked on 29 September 2026.
Five ways to deploy the same task
| Option | Control and operating burden | Costs to check | When to consider it |
|---|---|---|---|
| Enterprise SaaS | Provider runs the service; your team configures users and permitted use. | Licences, limits, integration and data exit. | A standard task with sufficient features and acceptable terms. |
| Managed API | Your team controls the application; provider runs the model. | Usage, optional storage, development and monitoring. | You need a custom process without operating GPUs. |
| Private or dedicated cloud | More control of network and resources; maintenance varies by contract. | Reserved capacity, operations, security and demand peaks. | Isolation or specific deployment conditions matter. |
| Own infrastructure | Your team operates model, servers, access, updates and continuity. | Equipment, energy, people, replacement and idle capacity. | There is a verifiable local-control requirement and a team to maintain it. |
| Hybrid | Explicit rules route each task to a defined environment. | Two environments, routing, tests and observability. | Tasks have different requirements and routing can be audited. |
“Private cloud” does not automatically guarantee residency. Microsoft’s service documentation distinguishes storage and processing locations by deployment type and explains optional history features. Verify the exact product and terms before buying.
Amazon Bedrock documents invocation logging that can retain inputs and outputs in CloudWatch or S3 when configured. Your own application may also log prompts. Examine all layers, not just model terms. Sources checked on 29 September 2026.
Example of application: searching internal manuals
A company wants staff to search technical procedures and receive cited answers. Some manuals are public; others describe internal incidents. A pilot separates collections, applies the user’s permissions before retrieval and checks that answers cite authorised passages. The RAG assistant guide covers the functional design.
A managed API may suffice for public manuals. For internal incidents, compare an enterprise API under suitable terms with a model in a controlled environment. Test both on the same questions, measure latency and cost of updates. Local hosting cannot rescue poor answers or an unmaintained system.
This is a hypothetical example of application, not a customer deployment or result.
Total cost includes operation
Comparing an API price per query with the cost of one GPU server is incomplete. Local deployment needs peak capacity, model and dependency updates, access controls, backups, incident response and people to handle failures. An API still needs integration and supervision.
- Estimate daily and peak-hour queries, input size and acceptable latency.
- Test the whole task, including retrieval and human review.
- Separate setup from licences, consumption, infrastructure, staff and hardware replacement.
- Price an outage and define a manual or alternative route.
- Repeat the comparison when volume, models or terms change.
No universal savings figure applies. A small pilot with measured workload gives a better estimate than a tariff alone.
Privacy and security responsibilities before contracting
Where personal data is involved, the GDPR requires an assessment of purpose, minimisation, security and processor arrangements; international transfers need their own review. On-premises hosting does not remove these duties, and a European region does not establish a legal basis by itself. Source checked on 29 September 2026.
Document who controls the processing, which data enters, who can inspect it, where copies and logs remain, how rights requests are handled and what happens at contract end. Data-protection and legal teams must assess the specific case, including whether an impact assessment is needed.
A concise decision record can capture task, data, destinations, rejected options, quality test, operating cost, owners and stop conditions. Our private AI service can assess the architecture; AI governance helps assign ongoing reviews.
From knowledge to a working process
Where should your data and model run?
We can map the real data flow, compare options on the same tasks and estimate who will operate each part.
Assess the architecture ↗
