AI / RAG / Full Stack
BackAI Meeting Intelligence
Turns recorded meetings into a searchable, multilingual knowledge base you can ask questions against.
Overview
Cross-language meetings have two separate problems: participants who don't share a language can't follow along live, and once the meeting ends, everything discussed is effectively unsearchable. This platform addresses both — real-time multilingual transcription and voice-cloned dubbing during the meeting, and a RAG-powered knowledge base over past meetings afterward.
Problem
Teams with international, multilingual participants lose people mid-meeting when the conversation shifts languages, and there's no good live solution short of a human interpreter. Separately, meeting recordings pile up in storage and are effectively write-only — searching them means scrubbing through video by hand, and knowledge shared verbally in one meeting is invisible to everyone who wasn't in the room.
Approach
For live meetings, audio is transcribed in real time with Faster-Whisper and translated on the fly, with XTTS v2 voice cloning used to dub the translated speech back in something close to the original speaker's voice — across 10+ languages for concurrent participants. Separately, uploaded recordings go through a batch pipeline: transcription, speaker diarization, translation, and auto-generated meeting minutes. Transcripts are chunked and embedded into a vector store, so instead of browsing a list of recordings, a user can ask a question — "what did we decide about the pricing model in last month's calls?" — and get an answer synthesized from the relevant meetings, with retrieval-augmented generation grounding the response in the actual transcript content rather than a model's guess.
- Live multilingual transcription (10+ languages)
- Real-time voice-cloned translation/dubbing
- Audio/video file processing
- Speaker diarization
- Auto-generated meeting minutes
- Semantic retrieval over past meetings
- Conversational Q&A (RAG)
- Authentication
- Containerized microservices
Architecture
Challenges
Real-time latency budget for live translation
Live dubbing only feels usable if the delay between someone speaking and hearing the translated, voice-cloned audio stays small. That put a hard latency budget on transcription, translation, and voice synthesis for concurrent participants — a very different constraint from the offline batch pipeline, which could trade latency for accuracy.
Chunking transcripts without losing context
Naive fixed-length chunking split conversations mid-thought and hurt retrieval quality. Chunk boundaries needed to respect speaker turns (from diarization) and topic shifts so retrieved passages stayed coherent when handed to the LLM.
Multilingual transcription and translation accuracy
Meeting audio quality varies widely — cross-talk, accents, and background noise all degrade transcription. The pipeline needed a translation step that ran reliably on Faster-Whisper's output without compounding transcription errors.
Grounding answers in retrieved content
A general-purpose LLM will happily answer from prior knowledge instead of the retrieved transcript chunks. Prompting and retrieval needed to be tight enough that answers stayed traceable back to specific meetings rather than sounding plausible but being unsupported.
Isolating storage and identity per service
Recordings, transcripts, and embeddings all needed durable, access-controlled storage independent of the application layer, and authentication needed to be handled centrally rather than re-implemented per service.
Decisions
Faster-Whisper for transcription, XTTS v2 for voice cloning
Both run self-hosted with strong multilingual accuracy, which mattered for keeping meeting audio under the platform's own control rather than sending it to a third-party API — and for keeping inference latency low enough for live dubbing.
Go + Python microservices split
Go for the latency-sensitive real-time path (streaming audio, live translation orchestration), Python for the ML-heavy work (transcription, diarization, RAG) — each language doing what it's actually good at instead of forcing one runtime to do both well.
A dedicated vector database for retrieval
Semantic search over meeting content needed to scale independently of the primary data store and support similarity search patterns that a relational database isn't built for.
Keycloak for authentication
Centralized identity and access control across services rather than building and maintaining custom auth, with room to add SSO later.
Docker for every service
Transcription, translation, retrieval, and the API each have different runtime dependencies. Containerizing each one kept the pipeline reproducible and easy to deploy as independent, scalable services.
Outcome
The result is a platform that handles both sides of the multilingual meeting problem: live transcription and voice-cloned dubbing for participants in the room, and a queryable, cited knowledge base afterward — covering transcription, diarization, translation, retrieval, and conversational Q&A end to end.