Address 5 critical issues found in comprehensive code review:
1. Opus Encoder Configuration Failures (CRITICAL)
- Split encoder settings into critical vs non-critical
- Critical settings (bitrate, VBR, FEC) now fail initialization on error
- Non-critical settings (complexity, DTX) log warnings but continue
- Prevents silent audio quality degradation from misconfigured encoder
2. V4L2 Sample Rate Detection Error Reporting (CRITICAL)
- Add specific error messages for different failure modes
- Distinguish permission errors, device not found, and no signal
- Validate detected sample rates are in reasonable range (8-192kHz)
- Improves debuggability when HDMI audio detection fails
3. Mutex Handling in ALSA Error Recovery (CRITICAL)
- Refactor handle_alsa_error() to NEVER unlock mutex internally
- Caller now always responsible for unlocking after checking return
- Eliminates complex mutex ownership semantics that caused deadlocks
- Consistent lock/unlock patterns prevent double-unlock bugs
4. Async Audio Start Error Propagation (CRITICAL)
- Make SetAudioOutputEnabled/SetAudioInputEnabled synchronous
- Add 5-second timeout for audio initialization
- Return errors to caller instead of only logging
- Revert state on failure to maintain consistency
- Users now get immediate feedback if audio fails to start
5. CgoSource Race Condition (CRITICAL)
- Hold c.mu mutex during C function calls in ReadMessage/WriteMessage
- Prevents use-after-free when Disconnect() called concurrently
- Lock order (c.mu -> capture_mutex) is consistent, no deadlock risk
- Fixes potential crash from accessing freed ALSA/codec resources
These changes eliminate silent failures, improve error visibility, and
prevent race conditions that could cause crashes or audio degradation.
Refactor audio processing to enhance stability and code clarity:
- Remove soft-clipping from audio capture pipeline
- Fix hardware frame size calculation for variable sample rates
- Add comprehensive error codes for audio initialization failures
- Clear stop flags after cleanup to prevent initialization deadlocks
- Improve mutex handling during device initialization
- Simplify constant validation and remove redundant comments
- Add DevPod setup instructions for Apple Silicon users
- Enforce Go cache clearing in dev_deploy.sh for CGO reliability
These changes improve audio capture stability when switching between
HDMI and USB audio sources, and fix race conditions during device
initialization and teardown.
Replace ALSA plugin layer resampling with libspeexdsp for improved audio
quality and reliability. This implementation uses direct hardware access
(hw:) instead of ALSA plugins (plughw:) and handles sample rate conversion
with SpeexDSP's high-quality sinc-based resampler.
Key changes:
- Add libspeexdsp 1.2.1 with ARM NEON optimizations to build dependencies
- Switch from plughw: to hw: device access for lower latency
- Implement conditional resampling (only when hardware rate ≠ 48kHz)
- Use SPEEX_RESAMPLER_QUALITY_DESKTOP for high-quality interpolation
- Add automatic audio dependency building in dev_deploy.sh
Quality improvements:
- Fix race condition in resampler cleanup with mutex protection
- Fix memory leak on resampler re-initialization
- Add buffer overflow validation (3840 frame limit for 192kHz)
- Improve error logging for resampling, encoding, and ALSA configuration
- Simplify code structure while maintaining all functionality
Technical details:
- Hardware negotiates actual sample rate (e.g., HDMI may vary)
- SpeexDSP converts hardware rate → 48kHz for Opus encoding
- USB Audio Gadget hardcoded to 48kHz (no resampling overhead)
- Static buffer allocation for zero allocation in hot path
- WebRTC requires 48kHz RTP clock rate per RFC 7587
Changes the audio subsystem from hw: (direct hardware access) to plughw:
(plugin layer with rate conversion) to enable configurable sample rates.
Changes:
- Update ALSA build to include plug,rate,linear,copy plugins
- Change device names from hw: to plughw: in C and Go code
- Remove 48kHz hardcoding for HDMI audio output
- Keep USB at 48kHz since hardware is fixed at that rate
- Update all comments to reflect plughw usage
Technical details:
- hw: devices bypass all ALSA plugins and require exact hardware rate match
- plughw: devices enable the ALSA plugin layer for automatic rate conversion
- Hardware still receives at native rate (48kHz), resampling happens in userspace
- HDMI can now use 8k/12k/16k/24k/48kHz, USB remains at 48kHz
- NEON-optimized resampling provides good performance on Cortex-A7
Requires rebuilding ALSA library with updated plugin configuration.
USB Audio Gadget (hw:1,0) hardware only supports 48kHz for both capture
and playback due to configfs p_srate/c_srate being hardcoded. This commit
ensures both audio paths respect this hardware limitation:
- Output path: Force 48kHz when using hw:1,0, allow configurable rates for HDMI
- Input path: Always use 48kHz regardless of UI configuration
- Calculate frame size dynamically based on actual sample rate used
Also removes redundant comments that don't add debugging or maintainability value.
Changes:
- Consolidate duplicate stop logic into helper functions
- Fix RPC getAudioConfig to return actual runtime values instead of
inconsistent defaults (bitrate was returning 128 vs actual 192)
- Improve setAudioTrack mutex handling to eliminate nested locking
- Simplify ALSA error retry logic by reorganizing conditional branches
- Split CGO Connect() into separate input/output methods for clarity
- Use map lookup for sample rate validation instead of long if-chain
- Add inline comments documenting validation steps
All changes preserve existing functionality while reducing code
duplication and improving readability. Tested with both HDMI and
USB audio sources.
- Separate capture_channels (stereo HDMI) from playback_channels (mono mic)
to prevent initialization conflicts that were breaking stereo output
- Optimize defaults for LAN use: 192kbps bitrate, complexity 8, 0% packet
loss compensation, DTX disabled (eliminates static and improves clarity)
- Add comprehensive race condition protection in C audio layer with handle
validity checks and mutex-protected cleanup operations
- Enable USB audio volume control and configure microphone as mono
- Add centralized AUDIO_DEFAULTS constant in UI with localized labels
- Add missing time import to fix compilation
This resolves audio quality issues and crash scenarios when switching
between HDMI and USB audio sources.
Moved all start/stop of sources into audio (out of jsonrpc)
Clean up duplicated code, made direction a bool, more logging, made all source/relay atomics.
Eliminate SetConfig since we always set it during start.
Eliminate the extra initialized flag.
Properly detect when USB audio was previously active.
Relay has the pointer to the source, not a copy.
CgoSource (and stub) expose the AudioSource interface.
Removed obvious comments that don't add value:
- cgo_source.go: Removed redundant status check comments
- audio.go: Consolidated mutex pattern comments
Kept important comments that explain non-obvious patterns:
- Why mutex is released before C calls (deadlock prevention)
- Why operations happen outside mutex (avoid blocking on CGO)
- Why single critical section is used (race condition prevention)
Problem:
When switching audio sources (USB to HDMI or vice versa), the application
would hang indefinitely. This was caused by a deadlock between Go and C
layers:
1. Main thread calls SetAudioOutputSource() → stopOutputAudio()
2. stopOutputAudio() calls outputRelay.Stop() which waits for goroutine
3. Goroutine is blocked in ReadMessage() holding Go mutex
4. ReadMessage() calls blocking C function jetkvm_audio_read_encode()
5. C function is blocked reading from ALSA device
6. Disconnect() can't acquire Go mutex to clean up
7. Deadlock: Main thread waiting for goroutine, goroutine waiting for ALSA
Solution:
Release the Go mutex BEFORE calling blocking C functions in ReadMessage()
and WriteMessage(). The C layer has its own pthread mutex protection and
handles stop requests via atomic flags. This allows:
- Disconnect() to acquire the mutex immediately
- C layer to detect stop request and return quickly
- Goroutines to exit cleanly
- Audio source switching to work flawlessly
Fixes:
- internal/audio/cgo_source.go:ReadMessage() - Release mutex before C call
- internal/audio/cgo_source.go:WriteMessage() - Release mutex before C call
This fix eliminates the hang when switching between USB and HDMI audio
sources.
- Update default EDID with registered manufacturer ID (Dell) and proper 24-inch display dimensions (52x32cm) for better macOS/OS compatibility
- Add configurable sample rate (32/44.1/48/96 kHz) to support different HDMI audio sources
- Add packet loss compensation percentage control for FEC overhead tuning
- Fix config migration to ensure new audio parameters get defaults for existing configs
- Update all language translations for new audio settings
- Add config fields for bitrate, complexity, DTX, FEC, buffer periods
- Add RPC methods for get/set audio config and restart
- Add UI settings page with controls for all audio parameters
- Add Apply Settings button to restart audio with new config
- Add config migration for backwards compatibility
- Add translations for all 9 languages
- Clean up redundant comments and optimize log levels
Remove all subprocess-based audio code to simplify the audio system and
reduce complexity. Audio now uses CGO in-process mode exclusively.
Changes:
- Remove subprocess mode: Deleted Supervisor, IPCSource, embed.go
- Remove audio mode selection from UI (Settings → Audio)
- Remove audio mode from backend config (AudioMode field)
- Remove JSON-RPC handlers: getAudioMode/setAudioMode
- Remove Makefile targets: build_audio_output/input/binaries
- Remove standalone C binaries: jetkvm_audio_{input,output}.c
- Remove IPC protocol implementation: ipc_protocol.{c,h}
- Remove unused IPC functions from audio_common.{c,h}
- Simplify audio.go: startAudio() instead of startAudioSubprocesses()
- Update all function calls and comments to remove subprocess references
- Add constants to cgo_source.go (ipcMaxFrameSize, ipcMsgTypeOpus)
- Keep update_opus_encoder_params() for potential future runtime config
Benefits:
- Simpler codebase: -1,734 lines of code
- Better performance: No IPC overhead on embedded hardware
- Easier maintenance: Single audio implementation
- Smaller binary: No embedded audio subprocess binaries
The audio system now works exclusively via CGO direct C function calls,
with ALSA device selection (HDMI vs USB) still configurable via settings.
Alex P
Marc Brooks