Secure Enterprise AI Assistant
- Sector
- U.S. government transit authority
- My role
- Lead developer
- Key decisions
- 6 covered below
In plain terms
A private, ChatGPT-style assistant that lets thousands of employees ask questions about internal documents — with every conversation logged and nothing leaving the organisation.
The problem
Thousands of employees needed a conversational way into internal documentation. As a government body, the organisation couldn't accept any data exposure, and every interaction had to be attributable and auditable.
Architecture
Scroll sideways to see the whole diagram.
Key decisions
- 01
Answers from the documents, not from memory
A full retrieval pipeline — parse, chunk, embed, retrieve, generate — so responses come from the organisation's own corpus rather than whatever the model learned in training.
- 02
Feels like the tools people already use
Persistent session memory, context-aware follow-ups and streamed responses, so it behaves like the consumer AI tools employees are used to.
- 03
Checks on the way out
Structured validation on every response makes sure nothing sensitive is surfaced at any point in the exchange.
- 04
Only employees get in
Enterprise SSO with role-based access control, scoped strictly to staff.
- 05
Visible to the security team
Admin analytics, audit logs and SIEM monitoring, so security sees this system the same way they see everything else on the network.
- 06
Prompt-level caching
Repeated context — system prompts and retrieved passages — is cached, cutting both latency and token cost on busy sessions.
Built with
- FastAPI
- React
- PostgreSQL
- AWS (S3, Cognito)
- OpenAI API
- SQLAlchemy
- Enterprise SSO