Real-Time Transcript Capture Microservice
Heavy contributor to a production Ruby microservice: live Zoom transcript capture via dual-WebSocket RTMS, thread-safe IVar signaling, async S3 persistence.
Overview
This is a de-identified production microservice. Joined an existing architecture and contributed heavily across the implementation: WebSocket connection management, keep-alive monitoring, periodic S3 uploads, and the RSpec test suite covering all connection states and file adapter paths. The service connects to Zoom's Real-Time Media Streaming (RTMS) API to capture live transcripts from provider-patient video calls. Zoom's RTMS protocol requires two concurrent WebSocket connections: a signaling channel (authentication, keep-alive heartbeats, stream control messages) and a separate transcript data channel whose URL is only known after the signaling handshake completes. The runner coordinates these using `Concurrent::IVar` — WebSocket event callbacks fire on background threads, and the main Sidekiq job thread blocks on the IVar value with a configurable timeout. A `Concurrent::TimerTask` handles periodic uploads of the in-progress transcript to S3 so no data is lost if a job is interrupted. On retry, the runner downloads any existing S3 content before reconnecting, resuming from wherever the previous attempt left off. Job state events (processing, completed, failed) are reported back to the main web application via HTTP after every terminal outcome. The engineering challenge is robustness across a stateful, time-bound process: a three-hour medical appointment that cannot be restarted, where transcript data loss has clinical consequences. The architecture separates failure modes cleanly — retryable exceptions are raised so Sidekiq retries them; non-retryable terminal states (e.g., invalid server URL) signal failure without raising, ending the job gracefully. Keep-alive monitoring detects silent disconnections that the WebSocket library would otherwise miss. *Code examples on this page are representative illustrations of the architectural patterns used — they are not actual proprietary source code from the production system.*
Technical Stack
Runtime & Jobs
- ▸Ruby
- ▸Rack
- ▸Sidekiq
- ▸Einhorn
- ▸concurrent-ruby
Networking
- ▸Zoom RTMS API
- ▸websocket-client-simple
- ▸Signaling WebSocket
- ▸Transcript WebSocket
Storage & Ops
- ▸AWS S3
- ▸RSpec
- ▸CircleCI
- ▸Docker
Key Features
Dual WebSocket connection management: signaling channel (auth + keep-alive) and transcript data channel opened in sequence
Cross-thread IVar signaling — WebSocket callbacks on background threads communicate the transcript URL and terminal outcomes to the main job thread
Concurrent::TimerTask for periodic in-progress transcript uploads to S3 during live sessions
Resume-on-retry: downloads any existing S3 transcript before reconnecting so interrupted jobs continue from where they left off
Keep-alive monitoring with configurable timeout — detects silent Zoom disconnections that the WebSocket library does not surface
Deterministic S3 key derivation from job ID + meeting UUID, enabling good bucket partitioning and collision-free retries
Clean failure taxonomy: raises for retryable errors (Sidekiq retries), returns Failure objects for terminal states (no raise, no retry)
Job event reporting to the main web application after every state transition (processing, completed, failed)
Code Examples
Technical Challenges
Coordinating two sequential WebSocket connections where the second URL is unknown until the first handshake completes
Managing cross-thread state safely between the WebSocket event thread and the Sidekiq job thread using IVar
Detecting silent WebSocket disconnections mid-session without relying on the underlying library's close event
Designing a retry strategy that preserves transcript data across job restarts without duplicating content
Distinguishing retryable from terminal failure modes in a stateful, long-running process where raising incorrectly causes data loss