Company knowledge needs more than a vector database
Private RAG that respects documents, permissions and uncertainty.
Retrieval-augmented generation can ground an AI response in approved company sources. The hard parts are document quality, permissions, retrieval evaluation and human review - not simply installing a chat interface.
Use RAG when answers should be grounded in a controlled source set. Do not use it as a substitute for source governance or professional judgement.
Buyer field note
Let the work and the room reject the wrong machine.
- 01 / Fit
- Start with the accepted task, not a chassis or GPU headline.
- 02 / Site
- Check power, cooling, noise, network and maintenance access.
- 03 / Alternative
- Keep cloud, colocation or a smaller system in the decision.
A transparent data path
Documents are parsed, split, embedded and stored for retrieval. At question time, relevant passages are passed to the model and the answer should cite them.
Each stage needs a named owner, retention rule and permission model. A local deployment can keep that path close, but it does not correct poor source data.
Capacity budget
Memory demand has more than one layer
- Model
- Weights and quantisation.
- Context
- Working input and cache.
- Concurrency
- Simultaneous active work.
- Headroom
- Operations and change allowance.
Connect the category to the physical and operating system.
Platform views and technical plates show what must be tested beyond a product label or aggregate specification.
Permissions must survive ingestion
A single company-wide index can leak information between teams. The design must preserve access groups or use separate collections where the source systems require it.
The user interface should show sources and make it obvious when retrieval confidence is weak.
Site boundary
The room is part of the specification
- Electrical Circuit, protection and UPS policy.
- Thermal Airflow, room heat and cooling route.
- Network Access, bandwidth and isolation.
- Service Rack space, noise and maintenance access.
Evaluate retrieval and answer quality separately
The system can retrieve the wrong passage even when the language model writes a fluent answer. Conversely, the right passage can be misinterpreted.
A representative question set should score retrieval, citation, answer support and refusal behaviour before wider access.
Procurement record
Make the route to acceptance inspectable
- Fit brief Workload and alternatives recorded.
- Reference test Conditions and results retained.
- Exact build Bill of materials and substitutions visible.
- Handover Owners, versions and exclusions signed off.
Hardware depends on usage, not document count alone
Ingestion can be a batch task. Day-to-day GPU demand is driven more by model, context, concurrent questions and response target.
Storage size, backup and document update frequency still matter, but millions of file names are not a useful server-sizing headline by themselves.
Questions answered
Straight answers to common questions
What is a private RAG server?
A server that runs retrieval, vector storage and a language model inside a customer-controlled environment so staff can ask questions over approved documents.
Will RAG stop hallucinations?
No. It can ground answers and expose sources, but retrieval and generation can still fail. Evaluation, citations, refusal behaviour and human review remain necessary.
Can it connect to SharePoint or Google Drive?
Connectors can be scoped separately. The initial appliance baseline does not include every integration or permission mapping by default.
Continue the decision
Useful next steps
Put the claim to work
Turn this guidance into a testable requirement.
The brief asks about workload and operating conditions - not just budget.