Hardware sample rate is auto-negotiated by ALSA, and frame size is derived
from it. These parameters were being passed but ignored - remove them from
update_audio_constants() and update_audio_decoder_constants() signatures.
- Copy opus data in ReadMessage to prevent aliasing with internal buffer
- Remove unused ctx field from OutputRelay struct
- Fix OPUS_BANDWIDTH comment accuracy (20kHz passband)
- Add missing __sync_synchronize() in capture init to match playback init path
- Add error handling for os.Setenv calls with warning logs on failure
- Add logging when ALSA channel map is unavailable (assumes standard L/R order)
- Fix OPUS_BANDWIDTH constant from 1104 to 1105 (fullband 20kHz vs superwideband 12kHz)
- Fix PacketLossPerc default from 0 to 20 to match C layer default
- Add early return in FEC recovery when pcm_frames <= 0 to avoid zero-frame ALSA writes
- Remove unused InputRelay fields (source, ctx, cancel) and unused NewInputRelay parameter
- Add async cleanup on timeout in SetAudioOutputEnabled/SetAudioInputEnabled
Resolved conflicts by:
- Preserving HDMI hotplug fix in video_restart_streaming() that checks detected_signal
- Keeping both isSecureContext and doRpcHidHandshake imports in devices.$id.tsx
No functionality was changed or broken in this merge.
The sleep mode feature from dev (PR #999) prevented video from auto-starting
when streaming_status == 0, which broke HDMI hotplug functionality.
Modified video_restart_streaming() to check detected_signal:
- No signal + not streaming -> don't start (preserve sleep mode)
- Has signal -> always stop cleanly then restart (fix hotplug)
This ensures proper encoder cleanup and prevents corrupted video output
after HDMI disconnect/reconnect cycles.
Implement periodic polling (every ~1 second) to detect HDMI audio sample
rate changes and trigger automatic reconfiguration. This prevents audio
distortion when switching between 44.1kHz and 48kHz sources.
Key changes:
- Poll TC358743 V4L2 control every 50 frames in capture hot path
- Trigger reconnection when sample rate changes
- Optimize logging to only output on rate changes (reduces log spam)
- Add proper state tracking to prevent duplicate logging
- Fix comment accuracy and ensure all state updates are consistent
Performance impact: ~100-500μs overhead every second (~0.01-0.05% CPU)
Address 5 critical issues found in comprehensive code review:
1. Opus Encoder Configuration Failures (CRITICAL)
- Split encoder settings into critical vs non-critical
- Critical settings (bitrate, VBR, FEC) now fail initialization on error
- Non-critical settings (complexity, DTX) log warnings but continue
- Prevents silent audio quality degradation from misconfigured encoder
2. V4L2 Sample Rate Detection Error Reporting (CRITICAL)
- Add specific error messages for different failure modes
- Distinguish permission errors, device not found, and no signal
- Validate detected sample rates are in reasonable range (8-192kHz)
- Improves debuggability when HDMI audio detection fails
3. Mutex Handling in ALSA Error Recovery (CRITICAL)
- Refactor handle_alsa_error() to NEVER unlock mutex internally
- Caller now always responsible for unlocking after checking return
- Eliminates complex mutex ownership semantics that caused deadlocks
- Consistent lock/unlock patterns prevent double-unlock bugs
4. Async Audio Start Error Propagation (CRITICAL)
- Make SetAudioOutputEnabled/SetAudioInputEnabled synchronous
- Add 5-second timeout for audio initialization
- Return errors to caller instead of only logging
- Revert state on failure to maintain consistency
- Users now get immediate feedback if audio fails to start
5. CgoSource Race Condition (CRITICAL)
- Hold c.mu mutex during C function calls in ReadMessage/WriteMessage
- Prevents use-after-free when Disconnect() called concurrently
- Lock order (c.mu -> capture_mutex) is consistent, no deadlock risk
- Fixes potential crash from accessing freed ALSA/codec resources
These changes eliminate silent failures, improve error visibility, and
prevent race conditions that could cause crashes or audio degradation.
Simplify channel swap detection and improve performance based on
IDisposable's review comments:
- Pass bool pointer directly instead of encoding in bit flag
- Remove redundant channel count check (already verified earlier)
- Use ARM NEON SIMD for channel swapping (4x faster)
- Process 4 frames (8 samples) per iteration with vrev32q_s16
These changes improve code clarity and boost channel swap performance
by ~4x using vectorized operations.
Query TC358743 HDMI receiver for detected audio sample rate before
initializing ALSA capture device. This fixes distortion issues when
HDMI sources send 44.1kHz audio (e.g., Armbian SBC) instead of 48kHz.
Previously, the code always requested 48kHz from ALSA, but in I2S slave
mode, the RV1106 I2S controller receives whatever clock rate the TC358743
master provides. This caused a sample rate mismatch where ALSA thought
it was 48kHz but hardware was actually running at 44.1kHz, resulting in
incorrect SpeexDSP resampling and audio distortion.
Changes:
- Add V4L2 ioctl to query TC358743's audio_sampling_rate control
- Use detected rate when configuring ALSA (falls back to 48kHz if unavailable)
- SpeexDSP resampler now gets correct input rate (44.1k, 48k, etc.)
- Supports all HDMI audio sample rates: 32k, 44.1k, 48k, 88.2k, 96k, etc.
Replace EDID with version that only advertises 60Hz timing modes
(1920x1080@60Hz and 1280x720@60Hz), removing the 1920x1080@50Hz mode
that was causing HDMI sources to prefer 50fps over 60fps output.
Refactor audio processing to enhance stability and code clarity:
- Remove soft-clipping from audio capture pipeline
- Fix hardware frame size calculation for variable sample rates
- Add comprehensive error codes for audio initialization failures
- Clear stop flags after cleanup to prevent initialization deadlocks
- Improve mutex handling during device initialization
- Simplify constant validation and remove redundant comments
- Add DevPod setup instructions for Apple Silicon users
- Enforce Go cache clearing in dev_deploy.sh for CGO reliability
These changes improve audio capture stability when switching between
HDMI and USB audio sources, and fix race conditions during device
initialization and teardown.
Defensive programming to prevent undefined behavior when closing ALSA
PCM handles. While the previous commit disabled assertions with -DNDEBUG,
adding explicit NULL checks ensures graceful handling even if handles are
unexpectedly NULL.
All error paths that call snd_pcm_close() now verify the handle is non-NULL
before closing, preventing potential crashes in edge cases.
Production builds should not include debug assertions. ALSA's assert()
calls cause aborts when internal invariants are violated, even for
recoverable error conditions.
The crash occurred when snd_pcm_close() was called with a NULL pointer,
triggering assertion failure at pcm.c:779 instead of graceful error
handling.
Stack trace:
pcm.c:779: snd_pcm_close: Assertion `pcm' failed
SIGABRT in jetkvm_audio_capture_init()
Adding -DNDEBUG disables all assert() calls in ALSA, Opus, and SpeexDSP
libraries for production robustness.
Previous EDID was configured for 29.95Hz (30 FPS). Updated to standard
1920x1080@60Hz using CEA timing (148.5 MHz pixel clock, 2200x1125 total)
for smoother video capture.
- Add input validation: NULL checks, bounds checking (max 7680 samples)
- Change return type to int for error propagation
- Use saturating NEON arithmetic (vqaddq_s16, vqsubq_s16) to prevent overflow
- Fix type consistency: use int16_t instead of short throughout
- Update documentation: precise threshold (0.9375 or 15/16), describe 4:1 compression
- Remove redundant clamping operations (mathematically proven unnecessary)
- Add stdbool.h include for bool type support
- Handle soft-clip errors at call site to prevent encoding corrupted audio
Implements SIMD-optimized soft-clipping before Opus encoding to prevent
digital clipping distortion on sharp transient attacks (e.g., plastic cup
impacts, percussive sounds). Uses smooth saturation curve starting at
±30720 (~94% of max amplitude) to preserve audio quality while eliminating
crackles and pops.
Processes 8 samples per iteration using ARM NEON intrinsics for optimal
performance on the ARM Cortex-A7 platform.
We use direct hw: device access with SpeexDSP for resampling, so the
ALSA plugin layer (plug) and rate conversion plugin are not needed.
This reduces library size while maintaining all required functionality.
Query the ALSA channel map (snd_pcm_get_chmap) to detect hardware that
reports non-standard channel ordering (R,L instead of L,R). When detected,
swap channels after capture to ensure correct left/right positioning.
This properly handles hardware quirks (like TC358743 HDMI audio) without
hardcoding device names, making the solution portable and correct.
Changes default EDID to JetKVM branded display configuration:
- Display name: JetKVM
- Full HD resolution (1920x1080@60Hz)
- Digital RGB 8-bit color support
- CEA-861 extension with PCM audio capability
- Broader compatibility with source devices
This EDID declares audio support which may improve HDMI audio detection
on certain source hardware.
When HDMI is unplugged during active audio capture, the blocking
snd_pcm_readi() call was holding the mutex, preventing clean shutdown.
This caused snd_pcm_drop() to race with the blocking read, leading to
undefined behavior and crashes.
Solution mirrors PiKVM's approach:
- Release mutex before snd_pcm_readi()/snd_pcm_writei()
- Reacquire mutex after I/O completes
- Verify handle and stop flag before proceeding
This allows snd_pcm_drop() to immediately abort pending I/O when the
device is closed, ensuring clean shutdown during HDMI hotplug events.
Replace ALSA plugin layer resampling with libspeexdsp for improved audio
quality and reliability. This implementation uses direct hardware access
(hw:) instead of ALSA plugins (plughw:) and handles sample rate conversion
with SpeexDSP's high-quality sinc-based resampler.
Key changes:
- Add libspeexdsp 1.2.1 with ARM NEON optimizations to build dependencies
- Switch from plughw: to hw: device access for lower latency
- Implement conditional resampling (only when hardware rate ≠ 48kHz)
- Use SPEEX_RESAMPLER_QUALITY_DESKTOP for high-quality interpolation
- Add automatic audio dependency building in dev_deploy.sh
Quality improvements:
- Fix race condition in resampler cleanup with mutex protection
- Fix memory leak on resampler re-initialization
- Add buffer overflow validation (3840 frame limit for 192kHz)
- Improve error logging for resampling, encoding, and ALSA configuration
- Simplify code structure while maintaining all functionality
Technical details:
- Hardware negotiates actual sample rate (e.g., HDMI may vary)
- SpeexDSP converts hardware rate → 48kHz for Opus encoding
- USB Audio Gadget hardcoded to 48kHz (no resampling overhead)
- Static buffer allocation for zero allocation in hot path
- WebRTC requires 48kHz RTP clock rate per RFC 7587
Changes the audio subsystem from hw: (direct hardware access) to plughw:
(plugin layer with rate conversion) to enable configurable sample rates.
Changes:
- Update ALSA build to include plug,rate,linear,copy plugins
- Change device names from hw: to plughw: in C and Go code
- Remove 48kHz hardcoding for HDMI audio output
- Keep USB at 48kHz since hardware is fixed at that rate
- Update all comments to reflect plughw usage
Technical details:
- hw: devices bypass all ALSA plugins and require exact hardware rate match
- plughw: devices enable the ALSA plugin layer for automatic rate conversion
- Hardware still receives at native rate (48kHz), resampling happens in userspace
- HDMI can now use 8k/12k/16k/24k/48kHz, USB remains at 48kHz
- NEON-optimized resampling provides good performance on Cortex-A7
Requires rebuilding ALSA library with updated plugin configuration.
USB Audio Gadget (hw:1,0) hardware only supports 48kHz for both capture
and playback due to configfs p_srate/c_srate being hardcoded. This commit
ensures both audio paths respect this hardware limitation:
- Output path: Force 48kHz when using hw:1,0, allow configurable rates for HDMI
- Input path: Always use 48kHz regardless of UI configuration
- Calculate frame size dynamically based on actual sample rate used
Also removes redundant comments that don't add debugging or maintainability value.
- Add sample rate dropdown in UI with Opus-supported rates (8k/12k/16k/24k/48kHz)
- Add sampleRate parameter to setAudioConfig RPC handler
- Validate sample rate is one of the 5 Opus-compatible values
- Configuration takes effect on next audio restart (Apply button)
- Clarify sample rate is configurable (8k/12k/16k/24k/48k), not fixed at 48kHz
- Expand mutex comment to include full lifecycle protection scope
- Document that ALSA playback init fails immediately with no fallback
- Add async behavior documentation to audio enable/restart functions
- Restore build_audio_deps target lost during merge
- Restore lint-fix, lint-go, lint-ui Makefile targets
- Fix variable alignment per linter
- Remove silent fallback to ALSA 'default' device on playback init failure
- Return error from SetAudioOutputSource for invalid source values
- Fix misleading comment about mutex scope in C audio code
- Clarify inputSourceMutex purpose for WebRTC packet serialization
- Replace helper function in getAudioConfig with explicit validation
- Consolidate audio default application in LoadConfig
- Streamline relay retry logic with inline conditions
- Extract closeFile and openHidFile helpers in USB gadget
- Simplify setPendingInputTrack pointer handling
- Improve error handling clarity in startAudio and updateUsbRelatedConfig
- Clean up processInputPacket mutex usage
Use snd_pcm_hw_params_set_rate_resample(1) to enable ALSA's rate plugin,
which provides software resampling even with hw: device interface.
This fixes audio distortion when HDMI sources output non-48kHz rates
(e.g., 44.1kHz from SBCs). ALSA now automatically resamples any input
rate to the configured 48kHz that Opus expects.
The rate plugin is available because ALSA is compiled with
--with-pcm-plugins=rate in install_audio_deps.sh
Make audio start asynchronous to prevent blocking the RPC response.
Previously, enabling audio would block until ALSA initialization completed,
which can take 30-60 seconds for HDMI audio due to TC358743 hardware.
This also fixes the -1 decode errors that occurred when packets arrived
during the synchronous restart window.
Matches the existing async pattern used in SetAudioOutputSource().
ALSA now forces the configured sample rate (default 48kHz) instead of
auto-detecting the source rate. This prevents Opus encoder initialization
failures when HDMI sources output 44.1kHz audio, which Opus doesn't support.
Changes:
- Use snd_pcm_hw_params_set_rate() to force exact rate (48kHz by default)
- ALSA performs software resampling if hardware rate differs
- Update valid rates to Opus-compatible only (8k, 12k, 16k, 24k, 48k)
- Remove auto-adaptation logic that caused Opus failures with 44.1kHz
This ensures audio capture works reliably with any HDMI source rate.
Integrated latest dev branch changes including:
- Native process refactoring with gRPC architecture
- OTA update system refactor with new component-based updates
- Updated build system and dependencies
- UI improvements and bug fixes
Post-merge fixes applied:
- Remove duplicate OTA RPC function declarations (now in ota.go)
- Fix GetDefaultEDID reference to use native.DefaultEDID constant
- Fix IsUpdatePending to use otaState.IsUpdatePending() method
- Add missing OTA RPC handler registrations for new update system
All audio functionality from feat/audio-support preserved.
All dev branch functionality preserved.
Alex P
Aveline
Siyuan
Adam Shiervani