Ensuring high-performance audio transcription and diarization entirely on consumer hardware
Optimized Whisper and PyAnnote models for efficient execution using quantization techniques
A local-first transcription product for diarized speech-to-text and notes without sending private audio to a cloud service.
Context
The product was motivated by a simple constraint: transcription is useful, but many meetings and recordings should not leave the machine. That shifts the problem from API integration to local model orchestration, latency management, and interface clarity.
Build
I combined Whisper for transcription, PyAnnote for diarization, llama.cpp for local summarization, and a responsive web UI for managing transcripts and settings. The architecture balances model quality with consumer-hardware limits, keeping the user experience fast enough to feel practical.
Engineering
Optimized Whisper and PyAnnote models for efficient execution using quantization techniques
Implemented asynchronous workflows with FastAPI for scalable and responsive audio processing
Leveraged llama.cpp to efficiently run lightweight summarization models directly on-device
Outcome
Stack