Address 5 critical issues found in comprehensive code review:
1. Opus Encoder Configuration Failures (CRITICAL)
- Split encoder settings into critical vs non-critical
- Critical settings (bitrate, VBR, FEC) now fail initialization on error
- Non-critical settings (complexity, DTX) log warnings but continue
- Prevents silent audio quality degradation from misconfigured encoder
2. V4L2 Sample Rate Detection Error Reporting (CRITICAL)
- Add specific error messages for different failure modes
- Distinguish permission errors, device not found, and no signal
- Validate detected sample rates are in reasonable range (8-192kHz)
- Improves debuggability when HDMI audio detection fails
3. Mutex Handling in ALSA Error Recovery (CRITICAL)
- Refactor handle_alsa_error() to NEVER unlock mutex internally
- Caller now always responsible for unlocking after checking return
- Eliminates complex mutex ownership semantics that caused deadlocks
- Consistent lock/unlock patterns prevent double-unlock bugs
4. Async Audio Start Error Propagation (CRITICAL)
- Make SetAudioOutputEnabled/SetAudioInputEnabled synchronous
- Add 5-second timeout for audio initialization
- Return errors to caller instead of only logging
- Revert state on failure to maintain consistency
- Users now get immediate feedback if audio fails to start
5. CgoSource Race Condition (CRITICAL)
- Hold c.mu mutex during C function calls in ReadMessage/WriteMessage
- Prevents use-after-free when Disconnect() called concurrently
- Lock order (c.mu -> capture_mutex) is consistent, no deadlock risk
- Fixes potential crash from accessing freed ALSA/codec resources
These changes eliminate silent failures, improve error visibility, and
prevent race conditions that could cause crashes or audio degradation.
When HDMI is unplugged during active audio capture, the blocking
snd_pcm_readi() call was holding the mutex, preventing clean shutdown.
This caused snd_pcm_drop() to race with the blocking read, leading to
undefined behavior and crashes.
Solution mirrors PiKVM's approach:
- Release mutex before snd_pcm_readi()/snd_pcm_writei()
- Reacquire mutex after I/O completes
- Verify handle and stop flag before proceeding
This allows snd_pcm_drop() to immediately abort pending I/O when the
device is closed, ensuring clean shutdown during HDMI hotplug events.
Replace ALSA plugin layer resampling with libspeexdsp for improved audio
quality and reliability. This implementation uses direct hardware access
(hw:) instead of ALSA plugins (plughw:) and handles sample rate conversion
with SpeexDSP's high-quality sinc-based resampler.
Key changes:
- Add libspeexdsp 1.2.1 with ARM NEON optimizations to build dependencies
- Switch from plughw: to hw: device access for lower latency
- Implement conditional resampling (only when hardware rate ≠ 48kHz)
- Use SPEEX_RESAMPLER_QUALITY_DESKTOP for high-quality interpolation
- Add automatic audio dependency building in dev_deploy.sh
Quality improvements:
- Fix race condition in resampler cleanup with mutex protection
- Fix memory leak on resampler re-initialization
- Add buffer overflow validation (3840 frame limit for 192kHz)
- Improve error logging for resampling, encoding, and ALSA configuration
- Simplify code structure while maintaining all functionality
Technical details:
- Hardware negotiates actual sample rate (e.g., HDMI may vary)
- SpeexDSP converts hardware rate → 48kHz for Opus encoding
- USB Audio Gadget hardcoded to 48kHz (no resampling overhead)
- Static buffer allocation for zero allocation in hot path
- WebRTC requires 48kHz RTP clock rate per RFC 7587
Changes the audio subsystem from hw: (direct hardware access) to plughw:
(plugin layer with rate conversion) to enable configurable sample rates.
Changes:
- Update ALSA build to include plug,rate,linear,copy plugins
- Change device names from hw: to plughw: in C and Go code
- Remove 48kHz hardcoding for HDMI audio output
- Keep USB at 48kHz since hardware is fixed at that rate
- Update all comments to reflect plughw usage
Technical details:
- hw: devices bypass all ALSA plugins and require exact hardware rate match
- plughw: devices enable the ALSA plugin layer for automatic rate conversion
- Hardware still receives at native rate (48kHz), resampling happens in userspace
- HDMI can now use 8k/12k/16k/24k/48kHz, USB remains at 48kHz
- NEON-optimized resampling provides good performance on Cortex-A7
Requires rebuilding ALSA library with updated plugin configuration.
USB Audio Gadget (hw:1,0) hardware only supports 48kHz for both capture
and playback due to configfs p_srate/c_srate being hardcoded. This commit
ensures both audio paths respect this hardware limitation:
- Output path: Force 48kHz when using hw:1,0, allow configurable rates for HDMI
- Input path: Always use 48kHz regardless of UI configuration
- Calculate frame size dynamically based on actual sample rate used
Also removes redundant comments that don't add debugging or maintainability value.
- Clarify sample rate is configurable (8k/12k/16k/24k/48k), not fixed at 48kHz
- Expand mutex comment to include full lifecycle protection scope
- Document that ALSA playback init fails immediately with no fallback
- Add async behavior documentation to audio enable/restart functions
- Restore build_audio_deps target lost during merge
- Restore lint-fix, lint-go, lint-ui Makefile targets
- Fix variable alignment per linter
- Remove silent fallback to ALSA 'default' device on playback init failure
- Return error from SetAudioOutputSource for invalid source values
- Fix misleading comment about mutex scope in C audio code
- Clarify inputSourceMutex purpose for WebRTC packet serialization
- Replace helper function in getAudioConfig with explicit validation
- Consolidate audio default application in LoadConfig
- Streamline relay retry logic with inline conditions
- Extract closeFile and openHidFile helpers in USB gadget
- Simplify setPendingInputTrack pointer handling
- Improve error handling clarity in startAudio and updateUsbRelatedConfig
- Clean up processInputPacket mutex usage
Make audio start asynchronous to prevent blocking the RPC response.
Previously, enabling audio would block until ALSA initialization completed,
which can take 30-60 seconds for HDMI audio due to TC358743 hardware.
This also fixes the -1 decode errors that occurred when packets arrived
during the synchronous restart window.
Matches the existing async pattern used in SetAudioOutputSource().
ALSA now forces the configured sample rate (default 48kHz) instead of
auto-detecting the source rate. This prevents Opus encoder initialization
failures when HDMI sources output 44.1kHz audio, which Opus doesn't support.
Changes:
- Use snd_pcm_hw_params_set_rate() to force exact rate (48kHz by default)
- ALSA performs software resampling if hardware rate differs
- Update valid rates to Opus-compatible only (8k, 12k, 16k, 24k, 48k)
- Remove auto-adaptation logic that caused Opus failures with 44.1kHz
This ensures audio capture works reliably with any HDMI source rate.
- Fix validateAndApply comment to clarify it returns values, doesn't apply them
- Correct capture_channels comment about hardware capabilities
- Fix opus_packet_loss_perc default value from 0 to 20 (matches backend default)
- Fix handle_alsa_error return value documentation (return 0 also unlocks mutex)
When switching audio output source between HDMI and USB, the HDMI
audio device (hw:0,0) can take 18-31 seconds to initialize due to
hardware characteristics of the TC358743 chip. This caused the UI
to freeze during source changes.
Changes:
- Move startAudio() to background goroutine in SetAudioOutputSource
- Move SaveConfig() to background goroutine to avoid blocking on disk I/O
- Return immediately after updating in-memory config
- Audio will initialize in background while UI remains responsive
The in-memory config is updated synchronously so subsequent calls
see the new source immediately. Both async operations are protected
by their respective mutexes (audioMutex, configLock).
Changes:
- Consolidate duplicate stop logic into helper functions
- Fix RPC getAudioConfig to return actual runtime values instead of
inconsistent defaults (bitrate was returning 128 vs actual 192)
- Improve setAudioTrack mutex handling to eliminate nested locking
- Simplify ALSA error retry logic by reorganizing conditional branches
- Split CGO Connect() into separate input/output methods for clarity
- Use map lookup for sample rate validation instead of long if-chain
- Add inline comments documenting validation steps
All changes preserve existing functionality while reducing code
duplication and improving readability. Tested with both HDMI and
USB audio sources.
Critical Fixes:
- Fix race condition in handleInputTrackForSession by reloading source inside mutex
- Fix ALSA handle cleanup atomicity (nullify before close to prevent use-after-free)
- Bounds check for opus buffer already present (verified)
Configuration Alignment:
- Align audio bitrate default to 192 kbps across all layers (C, Go defaults, config)
- Align audio complexity default to 8 across all layers
- Align DTX default to enabled (true/1) across all layers for bandwidth efficiency
Documentation Improvements:
- Update C header comment to reflect accurate 192 kbps default
- Clarify NEON requirement (not just "always available")
- Fix ALSA device mapping comments to reflect environment variable usage
- Document fallback behavior in playback init
Code Quality:
- Add validation logging for out-of-range audio configuration values
- Improve error visibility for configuration issues
All changes thoroughly analyzed before implementation.
- Separate capture_channels (stereo HDMI) from playback_channels (mono mic)
to prevent initialization conflicts that were breaking stereo output
- Optimize defaults for LAN use: 192kbps bitrate, complexity 8, 0% packet
loss compensation, DTX disabled (eliminates static and improves clarity)
- Add comprehensive race condition protection in C audio layer with handle
validity checks and mutex-protected cleanup operations
- Enable USB audio volume control and configure microphone as mono
- Add centralized AUDIO_DEFAULTS constant in UI with localized labels
- Add missing time import to fix compilation
This resolves audio quality issues and crash scenarios when switching
between HDMI and USB audio sources.
Moved all start/stop of sources into audio (out of jsonrpc)
Clean up duplicated code, made direction a bool, more logging, made all source/relay atomics.
Eliminate SetConfig since we always set it during start.
Eliminate the extra initialized flag.
Properly detect when USB audio was previously active.
Relay has the pointer to the source, not a copy.
CgoSource (and stub) expose the AudioSource interface.
Removed obvious comments that don't add value:
- cgo_source.go: Removed redundant status check comments
- audio.go: Consolidated mutex pattern comments
Kept important comments that explain non-obvious patterns:
- Why mutex is released before C calls (deadlock prevention)
- Why operations happen outside mutex (avoid blocking on CGO)
- Why single critical section is used (race condition prevention)
Reduced mutex locking from 3 separate lock/unlock cycles to 1, eliminating
race window and improving performance. New relay is prepared within mutex,
then old resources are stopped and new relay started outside mutex to avoid
blocking during CGO calls.
- Update default EDID with registered manufacturer ID (Dell) and proper 24-inch display dimensions (52x32cm) for better macOS/OS compatibility
- Add configurable sample rate (32/44.1/48/96 kHz) to support different HDMI audio sources
- Add packet loss compensation percentage control for FEC overhead tuning
- Fix config migration to ensure new audio parameters get defaults for existing configs
- Update all language translations for new audio settings
- Add config fields for bitrate, complexity, DTX, FEC, buffer periods
- Add RPC methods for get/set audio config and restart
- Add UI settings page with controls for all audio parameters
- Add Apply Settings button to restart audio with new config
- Add config migration for backwards compatibility
- Add translations for all 9 languages
- Clean up redundant comments and optimize log levels
Previous fix protected WriteMessage() but missed Connect() calls. During
session transfers, multiple goroutines could simultaneously initialize the
audio playback system, causing SIGABRT crashes in jetkvm_audio_playback_init.
Extended inputSourceMutex to serialize both Connect() and WriteMessage()
operations, ensuring atomic state transitions and preventing concurrent
C library initialization.
During session transfers, multiple handleInputTrackForSession goroutines
could call WriteMessage() concurrently on the shared inputSource, causing
Opus SILK decoder state corruption and assertion failures.
Added inputSourceMutex to serialize WriteMessage() calls and prevent
race conditions in both input and output audio paths.
When an old connection closed while a new one started, the audio cleanup
would hold audioMutex for up to 37 seconds during CGO disconnect calls,
blocking the new session from initializing.
Use capture-clear-release pattern to minimize mutex hold time:
- Capture references to sources/relays while holding mutex
- Clear globals immediately so new sessions can proceed
- Release mutex before calling blocking Stop/Disconnect operations
This eliminates the 37-second hang during connection transitions.
- Replace mutex-protected inputSource with atomic.Pointer for lock-free reads
- Eliminate ~100 mutex operations per second in audio packet hot path
- Add defer pattern for safer mutex unlock in enable/disable functions
- Add error logging for audio write failures
- Consolidate Makefile lint commands for brevity
When browsers recreate audio tracks during long sessions, the backend
now properly handles the new track instead of ignoring it. Old track
handlers detect when they've been superseded and exit cleanly to
prevent goroutine leaks.
- Add AudioInputAutoEnable and AudioOutputEnabled fields to backend config
- Implement RPC methods for get/set audio input auto-enable preference
- Load audio output enabled state from config on startup
- Make manual microphone toggle call backend to enable audio pipeline
- Auto-enable preference now persists across incognito sessions
- Reset microphone state to off when reaching login pages
- Fix issue where manual mic toggle required auto-enable to be on
Temporarily remove the ability to switch between HDMI and USB audio
output sources. The application now uses USB audio (hw:1,0) exclusively
until HDMI audio capture issues are resolved.
Changes:
- Remove AudioOutputSource config field
- Remove audio source switching logic and UI
- Hardcode USB audio output device
- Remove related RPC methods
Remove all subprocess-based audio code to simplify the audio system and
reduce complexity. Audio now uses CGO in-process mode exclusively.
Changes:
- Remove subprocess mode: Deleted Supervisor, IPCSource, embed.go
- Remove audio mode selection from UI (Settings → Audio)
- Remove audio mode from backend config (AudioMode field)
- Remove JSON-RPC handlers: getAudioMode/setAudioMode
- Remove Makefile targets: build_audio_output/input/binaries
- Remove standalone C binaries: jetkvm_audio_{input,output}.c
- Remove IPC protocol implementation: ipc_protocol.{c,h}
- Remove unused IPC functions from audio_common.{c,h}
- Simplify audio.go: startAudio() instead of startAudioSubprocesses()
- Update all function calls and comments to remove subprocess references
- Add constants to cgo_source.go (ipcMaxFrameSize, ipcMsgTypeOpus)
- Keep update_opus_encoder_params() for potential future runtime config
Benefits:
- Simpler codebase: -1,734 lines of code
- Better performance: No IPC overhead on embedded hardware
- Easier maintenance: Single audio implementation
- Smaller binary: No embedded audio subprocess binaries
The audio system now works exclusively via CGO direct C function calls,
with ALSA device selection (HDMI vs USB) still configurable via settings.
Remove dynamic gain code and rely on Opus encoder quality improvements:
- Increase Opus complexity from 2 to 5 for better quality
- Change bandwidth from FULLBAND (20kHz) to SUPERWIDEBAND (16kHz) for better quality at 128kbps
- Disable FEC to allocate all bits to audio quality
- Increase ALSA buffer from 40ms to 80ms for stability
The dynamic gain code was adding complexity without solving the underlying
issue: TC358743 HDMI chip captures digital audio at whatever volume the
source outputs. Users should adjust volume at the source or in their browser.
Alex P
Marc Brooks