Compare commits

...

312 Commits

Author SHA1 Message Date
Alex 91d25f2c7f
Merge 14b741c3dd into 1d1e58f036 2025-11-03 09:49:46 +01:00
Alex P 14b741c3dd temp: disable audio source selection while HDMI audio issues are diagnosed
Temporarily remove the ability to switch between HDMI and USB audio
output sources. The application now uses USB audio (hw:1,0) exclusively
until HDMI audio capture issues are resolved.

Changes:
- Remove AudioOutputSource config field
- Remove audio source switching logic and UI
- Hardcode USB audio output device
- Remove related RPC methods
2025-11-03 10:49:35 +02:00
Alex P de74ae1a12 fix: update default EDID to enable audio support
The previous default EDID did not advertise audio capabilities,
preventing HDMI audio capture from working. This update ensures
the JetKVM properly identifies itself to source devices and
enables audio capture out of the box.

Changes:
- Display identifies as "JetKVM HDMI" (manufacturer ID: JTK)
- Includes full audio support (2/6/8-channel LPCM, up to 192 kHz)
- Advertises all TC358743XBG capabilities (1080p60, YCbCr, Deep Color)

This allows HDMI audio to work by default without manual EDID
configuration.
2025-10-30 22:33:04 +02:00
Alex 2ea65c7a96
Merge branch 'dev' into feat/audio-support 2025-10-30 22:27:30 +02:00
Alex P 516a953f41 Merge branch 'dev' into feat/audio-support 2025-10-30 02:14:30 +02:00
Alex P eeacceb667 fix: wait for audio relay goroutine to exit before disconnecting source 2025-10-29 00:44:02 +02:00
Alex P 802166ba23 fix: use atomic.Bool for audio source to prevent mutex contention during switching 2025-10-29 00:26:48 +02:00
Alex P 4bc60c3f1b Updates: revert VERSION to 0.4.8 2025-10-28 22:36:52 +02:00
Alex P e7e6d7cb9d Updates: bump Makefile VERSION to 0.4.9 2025-10-28 22:32:18 +02:00
Alex P 65bbcf85ad fix: prevent race condition in audio output source switching by reading from in-memory state with proper synchronization 2025-10-28 22:11:11 +02:00
Alex P cd7a098f76 fix: track audio autoplay status separately to handle Safari blocking audio while allowing video 2025-10-28 15:47:36 +02:00
Alex P ef86af8afc fix: enable audio playback via manual start stream button
Browser autoplay policy blocks audio without user interaction.
Store audio elements in ref and trigger playback when user clicks
the existing 'Manually start stream' button.
2025-10-28 14:28:30 +02:00
Alex P b2a57a64e9 Merge branch 'dev' into feat/audio-support 2025-10-27 19:22:45 +02:00
Alex P be2bb518ed Merge branch 'dev' into feat/audio-support 2025-10-27 19:19:41 +02:00
Alex P 0f43a84551 fix: remove continuous sleep mode check to prevent audio interference
The continuous ensure_sleep_mode_disabled() call in the format detection
loop caused repeated I2C transactions to the TC358743, disrupting HDMI
audio capture. Sleep mode is already disabled once during video_init(),
which is sufficient.
2025-10-27 17:28:14 +02:00
Alex P 557b3bf3e6 Fix: app not loading via HTTP 2025-10-27 11:21:49 +02:00
Alex P 450836daba chore: revert golangci-lint config and remove audio stub
- Revert .golangci.yml to dev branch state (removed custom build-tags)
- Remove internal/audio/cgo_source_stub.go (not needed with proper cross-compilation)
- Fix import ordering in ui/src/utils.ts

Use 'make lint-go' for proper ARM cross-compilation environment.
2025-10-24 01:31:06 +03:00
Alex P 68b1bc54ce fix: add stub implementation for CGO audio on non-ARM platforms
Provides no-op AudioSource implementations for platforms that don't
support ARM CGO audio (x86_64, darwin, etc.). This allows golangci-lint
to run successfully on any platform without requiring ARM cross-compilation
toolchain.

The stub implementations return errors when called, ensuring that if
they're accidentally used at runtime on non-ARM platforms, it will fail
gracefully with a clear error message rather than undefined symbols.

Build constraints ensure the real CGO implementation is used on linux/arm
and linux/arm64, while stubs are used everywhere else.
2025-10-24 00:04:05 +03:00
Alex P 54cbd98781 fix: verify audio source change and show accurate feedback
After setting audio output source, fetch the actual value from backend
to verify the change was applied successfully. This prevents the UI
from showing incorrect state when the backend fails to apply the change.

The optimistic update provides immediate feedback, but we now verify
the actual result and show error if backend returned a different value
than expected.
2025-10-23 23:59:02 +03:00
Alex P 47782856f3 feat: add translations and fix audio settings page
Added comprehensive translations for audio settings page:
- Page title and description
- Audio output/input settings
- Audio source selector (HDMI/USB labels)
- Success/error messages for all operations
- Translations added for all 9 languages

Updated devices.$id.settings.audio.tsx:
- Import and use translation keys (m.audio_settings_*)
- Use existing audio_output_* and audio_input_* keys for notifications
- Fix dropdown not updating: update UI optimistically, revert on error
- This prevents the dropdown from appearing stuck after source change

The optimistic update pattern improves UX by immediately reflecting
the user's choice while the backend processes the change.
2025-10-23 23:56:15 +03:00
Alex P f75f5eb58d docs: document JetKVM ARMv7 architecture and lint requirements
Added documentation to .golangci.yml explaining:
- JetKVM uses ARMv7 (Rockchip RK3308, Cortex-A7) not ARM64
- golangci-lint should be run via 'make lint-go' not directly
- Audio CGO code requires cross-compilation environment from Makefile

The build-tags are explicitly listed to match the actual build configuration
used in the Makefile (netgo, timetzdata, nomsgpack).

Running 'make lint' or 'make lint-go' works correctly as it sets up
the full ARM cross-compilation environment with audio library dependencies.
2025-10-23 23:36:23 +03:00
Alex P c96c3e215a fix: use translation key for audio button in ActionBar
Changed hardcoded 'Audio' string to use m.action_bar_audio()
translation key for proper internationalization.
2025-10-23 23:26:23 +03:00
Alex P a20c13d7f3 fix: add AudioOutputSource field to Config struct
The audio.go file references config.AudioOutputSource but the field
was missing from the Config struct definition, causing type errors.

Added AudioOutputSource string field with json tag 'audio_output_source'
to support configuration of audio output source ('hdmi' or 'usb').
2025-10-23 23:24:44 +03:00
Alex P 9a82df662f feat: add proper translations for audio features across all languages
Added comprehensive translations for audio-related UI strings:
- Action bar audio button
- Audio popover (title, output/input settings)
- Enable/Disable button labels
- Success/error notification messages
- USB device audio preset labels

Translations added for all 9 supported languages:
- English (en)
- Danish (da)
- German (de)
- Spanish (es)
- French (fr)
- Italian (it)
- Norwegian Bokmål (nb)
- Swedish (sv)
- Chinese Simplified (zh)

Updated AudioPopover.tsx to use translation keys instead of
hardcoded English strings for all UI elements.

Total of 19 new translation keys per language:
- action_bar_audio
- audio_disable/enable
- audio_input_* (title, description, enabled/disabled, failed)
- audio_output_* (title, description, enabled/disabled, failed)
- audio_popover_title
- usb_device_enable_audio_* (title, description)
- usb_device_keyboard_mouse_mass_storage_and_audio
2025-10-23 23:22:38 +03:00
Alex P 55f40cb729 fix: move ipcMsgTypeOpus constant to source.go for cross-platform builds 2025-10-23 23:13:23 +03:00
Alex P 50b938d13e Merge dev into feat/audio-support
- Resolved conflicts in .gitignore, ActionBar, UsbDeviceSetting, and devices.$id.tsx
- Added audio-related translations to all language files:
  - action_bar_audio
  - usb_device_enable_audio_title
  - usb_device_enable_audio_description
  - usb_device_keyboard_mouse_mass_storage_and_audio
- Updated UI components to use localization messages
- Added separate USB preset for keyboard/mouse/storage without audio
- Kept OPUS_STEREO_PARAMS import from audio branch
- Preserved AudioPopover component from audio branch
2025-10-23 23:11:17 +03:00
Marc Brooks 374fc310d3
Merge branch 'dev' into feat/audio-support 2025-10-22 16:35:00 -05:00
Alex P a81544070b Merge branch 'dev' into feat/audio-support
This merge integrates the latest dev branch changes while preserving all
audio support functionality. Key changes include:

- Network stack refactoring: migrated from internal/network to pkg/nmlite
- New NetworkManager architecture with jetdhcpc client
- Function-based config pattern to avoid shared pointer bugs
- Updated UI components for network settings
- GitHub workflow and PR templates

- config.go: Retained AudioOutputSource field in Config struct
- config.go: Kept Audio: true in defaultUsbDevices
- config.go: Set AudioOutputSource: "usb" as default
- config.go: Adopted dev's types.NetworkConfig import path
- config.go: Adopted dev's function-based getDefaultConfig() pattern

1. config.go Config struct: Combined audio fields with dev's network refactoring
2. config.go default config: Adopted dev's function-based pattern while preserving audio defaults
3. ui/src/utils/jsonrpc.ts: Fixed unused error variable in catch block

All linters pass with 0 errors and 0 warnings.
2025-10-17 00:33:32 +03:00
Alex 0951f150bf
Merge branch 'dev' into feat/audio-support 2025-10-09 22:42:16 +03:00
Alex 71553bcef7
Merge branch 'dev' into feat/audio-support 2025-10-07 14:30:07 +03:00
Alex P 532e83e514 Cleanup: reduce drift from dev 2025-10-07 14:00:49 +03:00
Alex P bb5634be58 refactor: Remove subprocess audio infrastructure, use CGO-only
Remove all subprocess-based audio code to simplify the audio system and
reduce complexity. Audio now uses CGO in-process mode exclusively.

Changes:
- Remove subprocess mode: Deleted Supervisor, IPCSource, embed.go
- Remove audio mode selection from UI (Settings → Audio)
- Remove audio mode from backend config (AudioMode field)
- Remove JSON-RPC handlers: getAudioMode/setAudioMode
- Remove Makefile targets: build_audio_output/input/binaries
- Remove standalone C binaries: jetkvm_audio_{input,output}.c
- Remove IPC protocol implementation: ipc_protocol.{c,h}
- Remove unused IPC functions from audio_common.{c,h}
- Simplify audio.go: startAudio() instead of startAudioSubprocesses()
- Update all function calls and comments to remove subprocess references
- Add constants to cgo_source.go (ipcMaxFrameSize, ipcMsgTypeOpus)
- Keep update_opus_encoder_params() for potential future runtime config

Benefits:
- Simpler codebase: -1,734 lines of code
- Better performance: No IPC overhead on embedded hardware
- Easier maintenance: Single audio implementation
- Smaller binary: No embedded audio subprocess binaries

The audio system now works exclusively via CGO direct C function calls,
with ALSA device selection (HDMI vs USB) still configurable via settings.
2025-10-07 13:34:03 +03:00
Alex P 1bca60ae6b Merge branch 'feat/dual-audio-mode' into feat/audio-support 2025-10-07 11:26:09 +03:00
Alex P dd09cbcdc3 Cleanup: PR Comments 2025-10-07 11:14:16 +03:00
Alex P 035ba4c35f [WIP] Updates: support in-process mode 2025-10-07 11:00:59 +03:00
Alex P 00b8da45d9 [WIP] Updates: support in-process mode 2025-10-07 10:17:06 +03:00
Alex P 9abb2aa026 [WIP] Updates: support in-process mode 2025-10-07 10:13:11 +03:00
Alex P 0f16e0b11a [WIP] Updates: support in-process mode 2025-10-07 09:54:48 +03:00
Alex P 9d7fd878a1 [WIP] Updates: support in-process mode 2025-10-07 09:51:08 +03:00
Alex P 58fad71112 [WIP] Updates: support in-process mode 2025-10-07 08:49:49 +03:00
Alex P 9c72db913b feat: Optimize audio quality and default to USB audio
Audio quality improvements:
- Enable constrained VBR to prevent bitrate starvation at low volumes
- Increase Opus complexity from 2 to 5 for better quality
- Enable DTX for bandwidth optimization
- Enable FEC (Forward Error Correction)
- Add DTX and FEC signaling in SDP (usedtx=1;useinbandfec=1)

Default configuration changes:
- Change default audio output source from HDMI to USB
- Enable USB Audio device by default
- USB audio works on current stable image (HDMI requires newer device tree)

These changes fix crackling issues at low volumes and provide better
overall audio quality for both USB and HDMI audio paths.
2025-10-07 01:38:42 +03:00
Alex P 19fe908426 refactor: Simplify audio implementation
Remove dynamic gain code and rely on Opus encoder quality improvements:
- Increase Opus complexity from 2 to 5 for better quality
- Change bandwidth from FULLBAND (20kHz) to SUPERWIDEBAND (16kHz) for better quality at 128kbps
- Disable FEC to allocate all bits to audio quality
- Increase ALSA buffer from 40ms to 80ms for stability

The dynamic gain code was adding complexity without solving the underlying
issue: TC358743 HDMI chip captures digital audio at whatever volume the
source outputs. Users should adjust volume at the source or in their browser.
2025-10-07 00:25:45 +03:00
Alex P 04dd37f58f fix: Add noise gate to prevent amplifying silence artifacts
Add noise gate threshold at peak > 256 (-42dB) to prevent dynamic gain
from amplifying quantization noise and hardware noise floor. This fixes
crackling, buzzing, and static-like noise when HDMI audio is at very
low volume or during silence.

Without the gate, signals below -42dB (peak < 256) would get 8x gain
applied, amplifying noise floor to audible levels. Now these signals
pass through unmodified, eliminating the artifacts.
2025-10-06 22:23:23 +03:00
Alex P 5158c89103 fix: Replace 'any' with proper React.ChangeEvent type in AudioPopover
Replace 'as any' type assertions with proper React.ChangeEvent<HTMLInputElement>
type for synthetic events passed to audio toggle handlers.
2025-10-06 22:13:21 +03:00
Alex P 141e2f9099 fix: Address linting errors in audio code
- Check SetReadDeadline error in IPC client
- Explicitly ignore Kill() error (process may be dead)
- Remove init() function and rely on explicit ExtractEmbeddedBinaries() call
2025-10-06 22:12:20 +03:00
Alex P 7872ddc8fc Refactor: Simplify / rewrite Audio 2025-10-06 21:59:44 +03:00
Alex P 67447e4e5e [WIP] Updates: reduce PR complexity 2025-10-02 00:08:42 +03:00
Alex P c8e220334d [WIP] Updates: reduce PR complexity 2025-10-01 22:58:37 +03:00
Alex P 178c7486cc [WIP] Updates: reduce PR complexity 2025-10-01 22:54:37 +03:00
Alex P 257993ec20 [WIP] Updates: reduce PR complexity 2025-10-01 22:21:04 +03:00
Alex P 56c02f1067 [WIP] Updates: reduce PR complexity 2025-10-01 22:07:45 +03:00
Alex P 4c12783107 [WIP] Updates: reduce PR complexity 2025-10-01 21:20:30 +03:00
Alex P 6ccd9fdf19 [WIP] Updates: use native C binaries for audio 2025-10-01 20:13:13 +03:00
Alex P 74f73d9496 Merge branch 'dev' into feat/audio-support 2025-10-01 13:09:03 +00:00
Alex P ef5c25efcf Updates: integrate all dev branch changes 2025-10-01 15:50:39 +03:00
Alex P bdcac6a468 [WIP] Updates: update build flows to work with the CGO jetkvm_native 2025-10-01 10:04:42 +03:00
Alex P 160a925f40 Merge branch 'dev' into feat/audio-support 2025-09-30 20:08:33 +00:00
Alex P 70ef7193fd Cleanup: remove silence detection 2025-09-30 14:32:36 +00:00
Alex P 35b5dbd034 [WIP] Cleanup: cleanup audio code after HDMI switch 2025-09-30 13:32:56 +00:00
Alex P 7dc57bcdf3 [WIP] Fix: crackling sound when seeking forward after migrating to HDMI Audio 2025-09-30 13:14:15 +00:00
Alex P 05b347fe74 [WIP] Fix: crackling sound when seeking forwars 2025-09-30 13:11:16 +00:00
Alex P e989cad633 [WIP] Fix: crackling sound when seeking forwars 2025-09-30 12:58:49 +00:00
Alex P fc38830af1 [WIP] Updates: simplify audio system 2025-09-30 12:36:41 +00:00
Alex P 76b80da157 Updates: adjust gain to avoid audio artifacts 2025-09-30 12:05:03 +00:00
Alex P 01719e01dd [WIP] Updates: simplify audio system 2025-09-30 11:41:17 +00:00
Alex P 753c613708 [WIP] Updates: simplify audio system 2025-09-30 11:03:34 +00:00
Alex P f6dd605ea6 [WIP] Updates: simplify audio system 2025-09-30 09:36:19 +00:00
Alex P 680607e82e [WIP] Updates: simplify audio system 2025-09-30 09:08:55 +00:00
Alex P 6c6a1def28 [WIP] Updates: simplify audio system 2025-09-30 09:04:07 +00:00
Alex P dcce0fefb7 Merge branch 'dev' into feat/audio-support 2025-09-26 12:53:23 +00:00
Alex P 630571da25 [WIP] Improvements, Bugfixes: Improve audio experience when running in HTTP mode 2025-09-21 22:05:19 +00:00
Alex P d311dee4c6 [WIP] Improvements, Bugfixes: Improve audio experience when running in HTTP mode 2025-09-21 21:20:05 +00:00
Alex P 7060f9e8d6 [WIP] Improvements, Bugfixes: Improve audio experience when running in HTTP mode 2025-09-21 21:11:30 +00:00
Alex P b63af01d73 [WIP] Improvements, Bugfixes: Improve audio experience when running in HTTP mode 2025-09-21 21:07:41 +00:00
Alex P 4dec696c4a [WIP] Improvements, Bugfixes: Improve audio experience when running in HTTP mode 2025-09-21 21:04:08 +00:00
Alex P 093f2bbe22 Improvement: improve audio input restauration mechanism 2025-09-21 17:41:12 +00:00
Alex P dec0b9d3db [WIP] Improvements: Improve audio resume mechanism after Hardware Settings deactivation & reactivation 2025-09-21 18:17:43 +03:00
Alex P f2ad918dfd [WIP] Improvements: Improve audio resume mechanism after Hardware Settings deactivation & reactivation 2025-09-21 17:52:55 +03:00
Alex P a84f63c0c4 Fix: make sure audio output enable / disable doesn't need a refresh in order for audio to become audible again 2025-09-20 21:07:41 +00:00
Alex P 439f57c3c8 [WIP] CLeanup: Remove unused or redundant code or comments 2025-09-20 23:38:03 +03:00
Alex P b6d093f399 [WIP] Cleanup: PR cleanup - restore commented logs 2025-09-20 20:12:01 +00:00
Alex P cd87aa499c [WIP] Cleanup: PR Cleanup 2025-09-20 22:57:19 +03:00
Alex P 274854b198 [WIP] Cleanup: PR Cleanup 2025-09-20 01:38:16 +03:00
Alex P 6ee79b79c3 [WIP] Cleanup: PR Cleanup 2025-09-20 01:14:41 +03:00
Alex P 8b86124be1 [WIP] Cleanup: PR Cleanup 2025-09-20 00:57:57 +03:00
Alex P f2edfa66f0 [WIP] Cleanup: PR Cleanup 2025-09-20 00:54:14 +03:00
Alex P 432303e228 [WIP] Cleanup: PR Cleanup 2025-09-20 00:21:09 +03:00
Alex P 1dbc6c9d06 [WIP] Cleanup: PR cleanup 2025-09-19 23:53:40 +03:00
Alex P 3e24a3c186 Merge branch 'dev' into feat/audio-support 2025-09-19 12:41:46 +00:00
Alex P 17c3c4be9a PR Cleanup: reset .golangci.yml to its original state 2025-09-19 11:38:35 +00:00
Alex P 140a803ccf perf(audio): add ARM NEON SIMD optimizations for audio processing
Implement SIMD-optimized audio operations using ARM NEON for Cortex-A7 targets
Update Makefile and CI configuration to support NEON compilation flags
Add SIMD implementations for common audio operations including:
- Sample clearing and interleaving
- Volume scaling and format conversion
- Channel manipulation and balance adjustment
- Endianness swapping and prefetching
2025-09-16 18:18:19 +00:00
Alex P eca3c52513 PR Review Optimization: As recommended, use ternary operators instead of if/else for better readability 2025-09-16 16:17:56 +03:00
Alex P 55bcfb5a22 Consistency: keep if block multi-line 2025-09-16 16:08:16 +03:00
Alex P 0027001390 Cleanup: removed redundant code 2025-09-16 16:03:20 +03:00
Alex P caa0a60ebb Cleanup: removed redundant code 2025-09-16 16:00:55 +03:00
Alex P a5fb3bf30c Fix: remove misplaced const 2025-09-16 15:52:53 +03:00
Alex P 26e71806cb Cleanup, Optimizations: Small aaudio optimizations 2025-09-16 15:46:55 +03:00
Alex P 2f7bf55f22 Cleanup, Optimizations: Small aaudio optimizations 2025-09-16 15:45:03 +03:00
Alex P 8a3f1b6c32 Cleanup, Optimizations: Small aaudio optimizations 2025-09-16 15:37:23 +03:00
Alex P 7ffb9e1d59 Cleanup: removed redundant code, comments, etc. 2025-09-16 15:31:10 +03:00
Alex P 647eca4292 Cleanup: removed redundant code, comments, etc. 2025-09-16 15:23:16 +03:00
Alex P a8b58b5d34 [WIP] Cleanup: removed redundant code 2025-09-16 15:17:49 +03:00
Alex P b23cc50d6c [WIP] Cleanup: removed redundant code 2025-09-16 15:14:00 +03:00
Alex P 1f88dab95f [WIP] Maintainability Improvement: Add debug logging throughout the audio system for easy debugging and troubleshooting 2025-09-16 15:05:08 +03:00
Alex P 0944c886e5 [WIP] Maintainability Improvement: Add debug logging throughout the audio system for easy debugging and troubleshooting 2025-09-16 11:27:18 +03:00
Alex P 5e257b3144 [WIP] Add debug logging throughout the audio system 2025-09-16 11:26:48 +03:00
Alex P fb98c4edcb [WIP] Maintainability: Add debug / trace logs to make it easy to debug audio input issues 2025-09-16 11:11:18 +03:00
Alex P e894470ca8 [WIP] Cleanup: function naming 2025-09-16 07:33:34 +00:00
Alex P 996016b0da [WIP] Cleanup: remove unnecessary complexity 2025-09-15 23:00:03 +00:00
Alex P 7ab4a0e41d [WIP] Simplification: PR Simplification 2025-09-16 00:44:26 +03:00
Alex P ebb79600b0 Fix: pcm_snd_wait won't work when device is busy 2025-09-16 00:32:19 +03:00
Alex P b040b8feaf [WIP] Optimizations: Optimize audio system 2025-09-15 21:22:02 +00:00
Alex P ca38ebee0c [WIP] Fix: add commented code back 2025-09-15 12:04:24 +00:00
Alex P cca1fe720d [WIP] Cleanpup: Remove audio input quality settings to reduce complexity 2025-09-15 12:02:47 +00:00
Alex P 9d6bd997d9 [WIP] Cleanpup: Remove audio input quality settings to reduce complexity 2025-09-15 11:59:21 +00:00
Alex P e29694921b Fix: indentation 2025-09-15 14:29:02 +03:00
Alex P c8630e7c7f [WIP] Cleanpup: Remove audio input quality settings to reduce complexity 2025-09-15 08:40:28 +00:00
Alex P b6858ab155 [WIP] Cleanup / Refinement: PR Review 2025-09-14 21:55:31 +00:00
Alex P 0eaad6ba16 Merge branch 'dev' into feat/audio-support 2025-09-13 06:35:25 +00:00
Alex P 557aa5891a Cleanup: Simplify Makefile by exporting all needed env vars at the top 2025-09-12 09:47:11 +00:00
Alex P 49d62f8eb0 Cleanup: only set PKG_CONFIG_PATH once 2025-09-12 09:22:14 +00:00
Alex P 9e4392127e Fix: add accidentally removed test file back, devcontainer build process on clean devcontainer 2025-09-12 09:10:31 +00:00
Alex P 15baf9323b Merge branch 'dev' into feat/audio-support 2025-09-12 01:18:46 +03:00
Alex P 0e76023c39 Improvement (Maintainability): Make all C code more manageable by moving it to its own dedicated file 2025-09-11 13:27:10 +03:00
Alex P 5da357ba01 [WIP] Cleanup: Remove hid optimization code, as it is out of scope 2025-09-09 23:31:58 +00:00
Alex P eab0261344 Cleanup: remove devLog with calculated param 2025-09-09 22:44:33 +00:00
Alex P e0b6e612c0 Updates: defer the mutex unlock 2025-09-09 22:03:35 +00:00
Alex P f48c3fe25a [WIP] Updates, Cleanup: use uint64 for non-negative values 2025-09-09 21:54:36 +00:00
Alex P d4c10aef87 Updates: use uint64 since we won't have negative numbers here 2025-09-09 21:37:08 +00:00
Alex P 2a81497d34 Improvements: input performance 2025-09-09 20:58:34 +00:00
Alex P 8cff7d600b pr-optimizations,perf(input): optimize JSON-RPC input handling with ultra-fast path
Add ultra-fast path for input methods that completely bypasses float64 conversions and reflection
Use direct JSON unmarshaling to target types for maximum
2025-09-09 18:56:54 +00:00
Alex P eca1e6a80d Cleanup: implement PR Review suggestions 2025-09-09 18:09:19 +00:00
Alex P 02acee0c75 Fix: Audio Output Enable / Disable 2025-09-09 10:39:47 +00:00
Alex P 0f2aa9abe4 feat(audio): improve socket handling and validation performance
- Add retry logic for socket file removal and listener creation
- Optimize message writing by combining header and data writes
- Move socket paths from temp dir to /var/run
- Refactor OPUS parameter lookup to use map for better readability
- Simplify validation functions for better performance in hotpaths
2025-09-09 10:16:53 +00:00
Alex P 5d4f4d8e10 UX Improvement: keep Mic state accross page refreshes 2025-09-09 09:43:39 +00:00
Alex P a5d1ef1225 refactor(audio): optimize performance and simplify code
- Replace mutex locks with atomic operations for counters
- Remove redundant logging calls to reduce overhead
- Simplify error handling and buffer validation
- Add exponential backoff for audio relay stability
- Streamline CGO audio operations for hotpath optimization
2025-09-09 09:12:05 +00:00
Alex P bda92b4a62 [Milestone] Fix: in-flight audio input quality updates 2025-09-09 08:56:24 +00:00
Alex P 3c6184d0e8 [Milestone] Improvement: In-flight audio output quality update 2025-09-09 07:44:37 +00:00
Alex P 2bc7e50391 [WIP] Cleanup, Refactor: Reduce PR complexity, common IPC layer 2025-09-09 07:08:32 +00:00
Alex P f71d18039b [WIP] Cleanup: reduce PR complexity 2025-09-09 06:59:55 +00:00
Alex P 00e5148eef [WIP] Cleanup: reduce PR complexity 2025-09-09 06:52:40 +00:00
Alex P 0ebfc762f7 [WIP] Cleanup: PR SImplification 2025-09-09 05:41:20 +00:00
Alex P 845eadec18 [WIP] Fix: Audio Latency issues: move audio to a dedicated media stream
For more details please see: https://groups.google.com/g/discuss-webrtc/c/ZvAHvkHsb0E
2025-09-09 00:23:15 +00:00
Alex P aa21b4b459 Updates: increase congestion treshold multiplier 2025-09-08 22:58:49 +00:00
Alex P 89e68f5cdb [WIP] Change playback latency spikes on Audio Output Quality changes 2025-09-08 22:55:19 +00:00
Alex P f873b50469 fix(audio): adjust congestion and CPU thresholds for single-core system
Update congestion threshold multiplier and CPU thresholds to better suit single-core ARM RV1106G3 processor characteristics. Adjust memory thresholds for systems with 200MB total memory.
2025-09-08 22:03:11 +00:00
Alex P 0893eb88ac feat(audio): improve reliability with graceful degradation and async updates
- Implement graceful degradation for congestion handling with configurable thresholds
- Refactor audio relay track updates to be async to prevent deadlocks
- Add timeout-based supervisor stop during quality changes
- Optimize buffer pool configuration and cleanup strategies
2025-09-08 21:47:39 +00:00
Alex P 8cf0b639af perf(audio): increase buffer sizes and timeouts for quality change bursts
Significantly increase message pool, channel buffer, and adaptive buffer sizes to better handle quality change bursts. Adjust timeouts and intervals for improved responsiveness.
2025-09-08 21:17:06 +00:00
Alex P 6f10010d71 refactor(audio): remove redundant config variable assignments
Replace repeated local config variable assignments with direct Config access
to reduce memory allocations and improve code maintainability
2025-09-08 21:04:07 +00:00
Alex P 1d1658db15 refactor(audio): replace GetConfig() calls with direct Config access
This change replaces all instances of GetConfig() function calls with direct access to the Config variable throughout the audio package. The modification improves performance by eliminating function call overhead and simplifies the codebase by removing unnecessary indirection.

The commit also includes minor optimizations in validation logic and connection handling, while maintaining all existing functionality. Error handling remains robust with appropriate fallbacks when config values are not available.

Additional improvements include:
- Enhanced connection health monitoring in UnifiedAudioClient
- Optimized validation functions using cached config values
- Reduced memory allocations in hot paths
- Improved error recovery during quality changes
2025-09-08 17:30:49 +00:00
Alex P 91f9dba4c6 feat(audio): improve audio quality handling and recovery mechanisms
- Add server stats reset and frame drop recovery functions
- Implement global audio server instance management
- Add WebRTC audio track replacement capability
- Improve audio relay initialization with retry logic
- Enhance quality change handling with adaptive buffer management
- Add global helper functions for audio quality control
2025-09-08 12:48:22 +00:00
Alex P 219c972e33 Merge branch 'dev' into feat/audio-support 2025-09-08 11:17:08 +00:00
Alex P df58e04846 feat(audio): implement zero-copy batch processing with reference counting
Add batch reference counting and zero-copy frame management for optimized audio processing. Includes:
- BatchReferenceManager for efficient reference counting
- ZeroCopyFrameSlice utilities for frame management
- BatchZeroCopyProcessor for high-performance batch operations
- Adaptive optimization interval based on stability metrics
- Improved memory management with zero-copy frames
2025-09-08 09:08:07 +00:00
Alex P 323d2587b7 refactor(audio): improve memory management with atomic operations and chunk allocation
- Replace mutex-protected refCount with atomic operations in ZeroCopyFramePool
- Implement chunk-based allocation in AudioBufferPool to reduce allocations
- Add proper reference counting with atomic operations in ZeroCopyAudioFrame
- Optimize buffer pool sizing based on buffer size
2025-09-08 08:25:42 +00:00
Alex P a6913bf33b perf(audio): make refCount operations atomic and optimize frame pooling
Replace mutex-protected refCount operations with atomic operations to improve performance in concurrent scenarios.
Simplify frame release logic and add hitCount metric for pool usage tracking.
2025-09-08 08:20:43 +00:00
Alex P 6890f17a54 refactor(audio): consolidate supervision logic into base implementation
Move common supervision loop logic to BaseSupervisor with configurable parameters
Simplify input/output supervisor implementations by using base template
Update function comments to be more concise
2025-09-08 05:53:06 +00:00
Alex P a2a87b46b8 refactor(audio): move channel and process management to base supervisor
Consolidate duplicate channel and process management code from input/output supervisors into BaseSupervisor
Add new methods for channel initialization and cleanup
Standardize process termination and monitoring behavior
2025-09-07 20:14:33 +00:00
Alex P 96a6a0f8f9 refactor(audio): improve process management and error handling
- Remove unused setRunning method from BaseSupervisor
- Refactor IPC input reader to use running flag and mutex
- Add atomic state management to InputSupervisor
- Implement proper channel cleanup and process termination
- Improve error handling and logging throughout
2025-09-07 19:35:14 +00:00
Alex P bfdbbdc557 [WIP] Simplification 2025-09-07 19:13:35 +00:00
Alex P e3b4bb2002 refactor(audio): replace mute functionality with start/stop for microphone
- Replace MuteMicrophone calls with StartMicrophone/StopMicrophone for clearer behavior
- Update microphone state broadcasting to reflect actual subprocess status
- Modify UI to use enable/disable terminology instead of mute/unmute
- Ensure microphone device changes properly restart the active microphone
2025-09-07 18:32:42 +00:00
Alex P 7d39a2741e [WIP] Improvements: improve Audio Input Activation / Deactivation process so it is faster 2025-09-07 16:17:06 +00:00
Alex P e27f1cfa59 fix(AudioControlPopover): swap mute button theme and labels for clarity & consistency 2025-09-06 20:37:18 +00:00
Alex P b267348084 feat(audio): add microphone stop endpoint and improve mute handling
- Implement new POST /microphone/stop endpoint
- Refactor mute handling to properly start/stop audio processes
- Add callback mechanism for audio relay to reconnect to current session
- Simplify UI microphone controls by combining mute/start-stop functionality
2025-09-06 20:17:29 +00:00
Alex P 5a0dce9984 Fix: Microphone & audio on page load, PID field parsing in audio input supervisor 2025-09-06 06:11:29 +00:00
Alex P d3e2b2dff2 fix(audio): correct mute state broadcast in audio control service
Use BroadcastAudioMuteChanged instead of BroadcastAudioDeviceChanged to properly reflect mute state changes
2025-09-05 22:10:14 +00:00
Alex P 947b4f9528 [WIP] Updates: Reduce PR complexity 2025-09-05 21:47:21 +00:00
Alex P 8a189ba1b9 [WIP] Updates / FIles reorg: reduce PR complexity 2025-09-05 21:34:23 +00:00
Alex P d3bbe1bf0a refactor(audio): standardize component logging names and cleanup
- Add component name constants for consistent logging across audio modules
- Remove redundant debug logs and outdated comments
- Delete obsolete naming standards documentation file
2025-09-05 21:12:17 +00:00
Alex P 158437352c Fix: quality not updating in the Audio Control Popover when changed 2025-09-05 17:36:43 +00:00
Alex P e45bec4a9c Fix: currently selected preset not coming from the API 2025-09-05 17:33:13 +00:00
Alex P 2c2f2d416b Refactoring: Move most audio business logic into the audio package 2025-09-05 17:22:14 +00:00
Alex P 0a38451c95 feat(audio): add audio handlers and refactor session management
- Extract audio-related handlers into separate file for better organization
- Simplify session creation logic by removing duplicate code paths
- Add new Prometheus metrics for connection monitoring
- Reduce websocket ping interval from 30s to 15s for better responsiveness
2025-09-05 16:30:09 +00:00
Alex P d9072673c0 Updates: merge 'dev' intu 'feat/audio-support' 2025-09-05 15:07:54 +00:00
Alex P 9cb976ab8d Merge branch 'dev' into feat/audio-support 2025-09-05 14:40:23 +00:00
Alex P fcd07b2b59 perf(audio): enhance buffer pool performance with adaptive caching
- Increase goroutine cache size from 4 to 8 buffers for better hit rates
- Add adaptive resize and cache warmup based on usage patterns
- Implement enhanced cleanup with size limits and better TTL management
- Optimize buffer clearing and preallocation strategies
2025-09-05 13:53:00 +00:00
Alex P 1a0c7a84bc refactor(audio): rename latency config fields for clarity
Update config field names to better reflect their specific usage contexts in adaptive buffer and optimizer components. This improves code maintainability by making the purpose of each latency target more explicit.
2025-09-05 12:27:35 +00:00
Alex P 463f34e40b Fix: audio output not working due to too agressive timeout 2025-09-05 12:04:38 +00:00
Alex P 4075057c2b [WIP] Cleanup: Reduce PR Complexity 2025-09-04 22:06:48 +00:00
Alex P c1cc8dd832 refactor(audio): remove audio input subprocess pre-warming feature
The pre-warming feature was removed to simplify the audio input supervisor implementation. This feature added complexity and was not providing significant latency improvements in practice.
2025-09-04 19:48:38 +00:00
Alex P c40459664f Audio Input resiliency. Make sure the IPC client always recovers 2025-09-04 10:51:07 +00:00
Alex P cdf6731639 [WIP] Performance Updates:
Add LSB depth parameter for improved bit allocation and disable MMAP access for compatibility.
Adjust buffer sizing logic to better handle constrained environments while maintaining stability.
2025-09-04 08:47:40 +00:00
Alex P 6f15fdf965 perf(build): update ARM optimization flags for better compatibility
Replace -mcpu with -mtune in compiler flags to improve binary compatibility
while maintaining performance for Cortex-A7 targets
2025-09-04 08:25:16 +00:00
Alex P b63404c26b [WIP] Cleanup: reduce PR complexity 2025-09-03 23:50:05 +00:00
Alex P b497444d6d [WIP] Cleanup: reduce PR complexity 2025-09-03 23:13:36 +00:00
Alex P 476a245598 [WIP] Cleanup: Reduce PR complexity 2025-09-03 23:01:08 +00:00
Alex P 5dc04321a1 [WIP] Cleanup: decrease PR complexity 2025-09-03 22:48:25 +00:00
Alex P a3702dadd9 Cleanup: reduce PR complexity 2025-09-03 22:35:52 +00:00
Alex P 2568660149 feat(metrics): add configurable metrics collection and performance optimizations
- Add config flags to enable/disable metrics collection, goroutine monitoring, and latency profiling
- Optimize batch processor with configurable queue sizes and thread pinning thresholds
- Skip metrics operations when disabled to reduce overhead
- Update default config with performance-related settings
2025-09-03 20:18:07 +00:00
Alex P ca365f1acd feat(audio): add subprocess pre-warming to reduce activation latency
implement batched metrics updates to reduce atomic operations
optimize thread locking for high-throughput scenarios only
2025-09-03 19:51:18 +00:00
Alex P 5c55da0787 perf(audio): optimize metrics collection and logging overhead
- Replace direct atomic updates with sampling to reduce contention
- Simplify metrics tracking by removing buffering and using direct updates
- Optimize logging by adding level checks and sampling
- Improve validation performance using cached config values
2025-09-03 19:41:20 +00:00
Alex P 260f62efc3 perf(audio): optimize audio processing paths and reduce overhead
- Replace CGO function variable aliases with direct function calls to eliminate indirection
- Simplify audio frame validation by using cached max size and removing error formatting
- Optimize buffer pool operations by removing metrics collection and streamlining cache access
- Improve batch audio processor by pre-calculating values and reducing config lookups
- Streamline IPC message processing with inline validation and reduced error logging
2025-09-03 17:51:08 +00:00
Alex P a741f05829 fix(audio): add proper locking for config cache updates
Add mutex locking around config cache expiration checks to prevent race conditions. The cache now properly checks initialization status before attempting updates.
2025-09-03 17:08:19 +00:00
Alex P a557987629 perf(audio): optimize hotpath by removing redundant checks and logging
- Skip logging in frame validation to reduce overhead
- Only update cache when expired to avoid unnecessary operations
- Remove duplicate config caching system and simplify buffer handling
- Optimize batch processing with pre-allocated buffers and conditional time tracking
2025-09-03 16:55:39 +00:00
Alex P 5353c1cab2 perf(audio): optimize opus decode-write with separate buffers
- Add PCM buffer pool and config for optimized decode-write operations
- Implement separate buffer handling in CGO audio processing
- Update batch processor to support both legacy and optimized paths
2025-09-03 16:28:25 +00:00
Alex P 370178e43b fix(audio): remove arm architecture restriction from cgo build tags 2025-09-03 15:49:07 +00:00
Alex P 9f1dd28ad6 feat(audio): add batch write processing and improve thread management
- Add batch write processing functionality to match existing read processing
- Improve thread pinning logic with separate controls for read/write
- Add new batch processing configuration parameters
- Update build tags to exclude arm architecture
2025-09-03 15:21:35 +00:00
Alex P 2ab90e76e0 feat(audio): add validation cache fields to AudioConfigCache
Add atomic fields to AudioConfigCache for validation parameters to enable lock-free access
Optimize validation functions to use cached values for common cases
Move AudioFrameBatch to separate file and update validation logic
2025-09-03 14:48:41 +00:00
Alex P 1b7198aec2 feat(audio): implement sized buffer pool and config caching
Add SizedBufferPool for efficient memory management with size tracking and statistics
Introduce AudioConfigCache to minimize GetConfig() calls in hot paths
Add batch processing support for audio frames to reduce CGO overhead
Extend AudioBufferPoolDetailedStats with total bytes and average size metrics
2025-09-03 14:00:12 +00:00
Alex P f9781f170c perf(audio): increase worker pool sizes and optimize worker management
Double worker counts and queue sizes to handle higher load scenarios
Modify worker management to maintain minimum 2 persistent workers with longer idle timeout
2025-09-03 12:54:07 +00:00
Alex P d7b67e5012 feat(audio): implement goroutine pool for task processing
Add goroutine pool implementation to manage reusable workers for audio processing tasks
Add configuration constants for pool sizing and behavior
Modify audio server components to use pool for goroutine management
Add fallback to direct goroutine creation when pools are full
2025-09-03 12:41:27 +00:00
Alex P 8110be6cc6 feat(audio): optimize audio processing with batch processing and goroutine monitoring
- Add batch audio processing to reduce CGO call overhead
- Implement goroutine monitoring and cleanup for leak prevention
- Optimize buffer pool with TTL-based cache and latency-aware cleanup
- Add configurable parameters for batch processing and monitoring
- Improve CGO audio read performance with config caching
2025-09-03 12:10:54 +00:00
Alex P 950ca2bd99 fix(audio): improve process termination handling in input supervisor
Add more robust process state checking and error handling during audio input server shutdown. Use signal 0 to verify process existence before killing and handle various edge cases. Also improve logging to better track shutdown outcomes.
2025-09-01 11:08:13 +00:00
Alex P dfbf9249b9 fix(audio): improve audio initialization and process termination handling
- Add retry logic for CGO audio initialization
- Enhance process termination checks in input supervisor
- Skip initialization check in cgo_audio to avoid compilation issues
- Add proper error handling for audio system initialization failures
2025-09-01 10:24:26 +00:00
Alex P f51f6da2de fix(audio): improve logging for Opus config and subprocess status
Add detailed logging when sending Opus configuration to audio input subprocess
Include supervisor running status in log when subprocess is not connected
2025-09-01 08:07:53 +00:00
Alex P fd7608384a feat(audio): implement dynamic Opus config updates and optimize audio params
Add support for dynamic Opus encoder configuration updates without requiring subprocess restart. This allows quality changes to be applied immediately while maintaining audio stream continuity.

Optimize audio quality parameters to reduce CPU load and prevent mouse lag on RV1106 devices. Lower bitrates and complexity while adjusting signal types and bandwidths for better performance.

Add build tags for CGO requirements and improve audio input supervisor behavior to check for existing processes before starting new ones.
2025-09-01 08:02:43 +00:00
Alex P 6adcc26ff2 feat(audio): add goroutine cache cleanup and process reuse
Implement periodic cleanup of stale goroutine buffer caches to prevent memory leaks
Add ability to detect and reuse existing audio input server processes
2025-08-29 17:05:37 +00:00
Alex P 858859e317 perf(audio): optimize audio config for RV1106 SoC compatibility
Adjust bitrates, frame sizes, and OPUS parameters to balance quality and performance on RV1106. Reduce channel count for low quality to minimize CPU load. Update CGO constants for better memory efficiency.
2025-08-28 22:25:11 +00:00
Alex P 9c0aff4489 feat(audio): implement audio input supervisor and opus config management
add audio input supervisor with opus configuration support
update audio quality presets and configuration handling
restructure audio subprocess management with environment variables
2025-08-28 22:02:22 +00:00
Alex P 0d4176cf98 refactor(audio): centralize metrics collection with new registry
Introduce MetricsRegistry to serve as single source of truth for audio metrics
Remove duplicate metrics collection logic from web endpoints
Add callback mechanism for metrics updates
2025-08-28 10:01:35 +00:00
Alex P fe4571956d refactor(audio): remove granular latency metrics and histogram functionality
This commit removes the granular latency metrics collection and histogram visualization functionality across the codebase. The changes include:
- Removing latency histogram tracking from audio input/output processing
- Removing latency histogram UI components and related types
- Removing granular metrics collector's latency tracking capabilities
- Updating Prometheus metrics to use milliseconds instead of seconds
- Removing related tests and benchmarks
2025-08-28 08:44:09 +00:00
Alex P f9adb4382d feat(audio): add latency metrics collection for input and output
Add granular metrics collection for audio processing latency in both input and output paths. This enables better performance monitoring through histograms.

Also update build tags to include ARM platform and rename UI label for clarity.
2025-08-28 00:35:51 +00:00
Alex P 758bbbfff6 feat(audio): add latency histogram metrics collection and visualization
- Add LatencyHistogramData interface and implement histogram collection in granular metrics
- Create LatencyHistogram component for visualizing latency distribution
- Update audio metrics events to include histogram data
- Add comprehensive tests for histogram functionality
- Improve error handling for OPUS encoder parameter updates
- Optimize validation cache initialization
2025-08-28 00:24:30 +00:00
Alex P 3efe2f2a1d refactor(audio/validation): improve frame data validation with specific error messages
Split generic frame data validation error into specific cases for empty and oversized frames to provide better error context. The optimized validation maintains performance while improving debuggability.
2025-08-27 23:54:40 +00:00
Alex P ece36ce5fd feat(audio): optimize validation and add dynamic opus encoder configuration
Consolidate audio frame validation functions into a single optimized implementation and add dynamic OPUS encoder parameter updates based on quality settings. Initialize validation cache at startup for consistent performance.

Add latency profiler for end-to-end audio pipeline monitoring. Update test cases to use unified validation function and initialize cache.

The changes improve performance by reducing function call overhead and enabling runtime optimization of audio encoding parameters based on quality presets.
2025-08-27 23:44:16 +00:00
Alex P cdf0b20bc7 perf(audio): optimize validation and buffer pool with caching
- Cache max frame size in validation for hot path performance
- Add lock-free per-goroutine buffer cache to reduce contention
2025-08-27 22:57:07 +00:00
Alex P 25363cef90 perf(audio): replace frame validation with ultra-fast version
Use ValidateAudioFrameUltraFast in critical audio paths to reduce processing overhead
Reduce minimum frame size to 1 byte to allow smaller frames
2025-08-27 22:44:34 +00:00
Alex P e3e7b898b5 style(audio): fix formatting and add missing newlines
- Fix indentation and alignment in performance tests
- Add missing newlines at end of files
- Clean up error message formatting for consistency
2025-08-27 20:54:50 +00:00
Alex P 9dda569523 feat(audio): enhance validation and add logging standards
refactor validation functions to be more comprehensive and add detailed error messages
add new logging standards implementation for audio components
remove deprecated validation_enhanced.go file
add performance-critical tests for validation functions
2025-08-27 20:46:47 +00:00
Alex P 6355dd87be feat(audio): add pool hit tracking and optimize buffer management
refactor(ui): extract audio config display and status indicator components
refactor(audio): simplify validation and error handling
refactor(config): remove duplicate audio quality constants
perf(buffer): optimize buffer pool allocation and tracking
docs(audio): streamline package documentation
2025-08-27 19:37:34 +00:00
Alex P cb20956445 refactor(AudioControlPopover): simplify audio metrics display and remove unused code
Remove advanced metrics toggle and related complex display logic, replacing it with a simplified quick status summary. Also clean up unused imports, state variables, and helper functions.
2025-08-27 18:33:52 +00:00
Alex P 50e04192bf refactor(audio): standardize log levels and messages across components
- Change Info logs to Debug for routine operations
- Standardize log message formatting to lowercase
- Improve error message clarity and consistency
- Add new metrics for device health and memory monitoring
- Simplify config constants documentation
2025-08-27 18:11:06 +00:00
Alex P dc2db8ed2d feat(audio): add comprehensive input validation and base components
refactor(audio): restructure metrics and supervisor components into base implementations
feat(audio): add validation for all audio configurations and frames
fix(audio): fix atomic alignment in metrics structures
refactor(audio): consolidate common functionality into base manager and supervisor
feat(audio): add output IPC manager and config validation
2025-08-27 17:47:39 +00:00
Alex P 8fb0b9f9c6 feat(audio): centralize audio configuration and improve debugging
- Add debug utilities with development-only logging
- Create centralized audio configuration constants
- Implement audio quality service for managing presets
- Replace console logging with debug utilities
- Update audio metrics with unified structure
- Improve microphone error handling and state management
2025-08-27 13:01:56 +00:00
Alex P e8d12bae4b style(audio): fix formatting and add missing newlines
- Fix indentation in test files and supervisor code
- Add missing newlines at end of files
- Clean up documentation formatting
- Fix buffer pool pointer return type
2025-08-26 16:49:41 +00:00
Alex P 6a68e23d12 refactor(audio): improve error handling and memory management
- remove redundant error logging in audio supervisor stop calls
- add buffer pool for memory optimization in audio relay and ipc
- return default metrics when process is not running
- add channel closed flags to prevent double closing
- standardize component naming and logging
- add comprehensive documentation for audio components
- improve test coverage with new unit tests
2025-08-26 14:36:07 +00:00
Alex P b1f85db7de feat(audio): enhance error handling and add device health monitoring
- Implement robust error recovery with progressive backoff in audio streaming
- Add comprehensive device health monitoring system
- Improve ALSA device handling with enhanced retry logic
- Refactor IPC message handling to use shared pools
- Add validation utilities for audio frames and configuration
- Introduce atomic utilities for thread-safe metrics tracking
- Update latency histogram to use configurable buckets
- Add documentation for new metrics and configuration options
2025-08-26 12:51:11 +00:00
Alex P e4ed2b8fad refactor(audio): rename audio components for clarity and add validation
Rename audio server/client components to be more specific (AudioOutputServer/Client). Add new validation.go and ipc_common.go files for shared IPC functionality. Improve error handling and cleanup in input/output IPC components.

Disable granular metrics logging to reduce log pollution. Reset metrics on failed start and ensure proper cleanup. Add common IPC message interface and optimized message pool for reuse.
2025-08-26 10:42:25 +00:00
Alex P fff2d2b791 feat(usbgadget): add nil checks for gadget operations and cleanup tests
refactor(usbgadget): reorganize test files into logical categories

test(usbgadget): add integration tests for audio and usb gadget interactions

fix(dev_deploy): clean up /tmp directory before copying test files
2025-08-25 22:24:41 +00:00
Alex P 6898a6ef1b feat(audio): add granular metrics collection and comprehensive error handling
- Implement granular metrics collection for latency and buffer pool operations
- Add detailed error messages with context for all IPC operations
- Enhance logging with additional operational details
- Introduce comprehensive test suite including unit and integration tests
- Add package documentation explaining architecture and components
2025-08-25 21:00:54 +00:00
Alex P 34f8829e8a refactor(audio): improve configuration handling and validation
- Replace hardcoded values with configurable parameters in output streamer
- Add detailed validation rules for socket buffer configuration
- Enhance CPU percent calculation with bounds checking and validation
- Document message format and validation for IPC communication
- Update latency monitor to use configurable thresholds
- Improve adaptive buffer calculations with validation and documentation
2025-08-25 20:36:26 +00:00
Alex P 60a6e6c5c5 refactor(audio): centralize config values and improve documentation
Move hardcoded values to config and add detailed validation rules documentation for audio components. Update function comments to clearly describe validation logic and thresholds.

The changes ensure consistent configuration management and better maintainability while providing comprehensive documentation of validation rules and system behavior.
2025-08-25 20:35:40 +00:00
Alex P c5216920b3 refactor(audio): centralize config constants and update usage
Move hardcoded values to config constants and update all references to use centralized configuration. Includes:
- Audio processing timeouts and intervals
- CGO sleep durations
- Batch processing parameters
- Event formatting and timeouts
- Process monitor calculations
2025-08-25 19:30:57 +00:00
Alex P 9e343b3cc7 refactor(audio): move hardcoded values to config for better flexibility
- Replace hardcoded values with configurable parameters in audio components
- Add new config fields for adaptive buffer sizes and frame pool settings
- Implement memory guard in ZeroCopyFramePool to prevent excessive allocations
2025-08-25 19:02:29 +00:00
Alex P 35a666ed31 refactor(audio): centralize configuration constants in audio module
Replace hardcoded values with centralized config constants for better maintainability and flexibility. This includes sleep durations, buffer sizes, thresholds, and various audio processing parameters.

The changes affect multiple components including buffer pools, latency monitoring, IPC, and audio processing. This refactoring makes it easier to adjust parameters without modifying individual files.

Key changes:
- Replace hardcoded sleep durations with config values
- Centralize buffer sizes and pool configurations
- Move thresholds and limits to config
- Update audio quality presets to use config values
2025-08-25 18:08:12 +00:00
Alex P 7ec583ed6a refactor(audio): centralize config and remove debug logs
- Move hardcoded constants to centralized config system
- Remove verbose debug logging statements
- Clean up unused code and improve error handling
2025-08-25 16:49:48 +00:00
Alex P d1c192bf8b feat(audio): add real-time USB audio config updates and validation
- Add audio device change listener in UI to update USB config
- Implement audio configuration validation before enabling USB audio
- Broadcast audio device change events for all state changes
2025-08-25 14:21:49 +00:00
Alex P c89d678963 refactor(audio): remove unused context from audio input manager
Simplify AudioInputIPCManager by removing unused context and cancellation logic. The context was not providing any meaningful functionality.

fix(ui): handle audio device changes with proper sync

Add delayed microphone state synchronization when audio devices change to prevent race conditions during USB audio reconfiguration.
2025-08-25 13:53:29 +00:00
Alex P 6f02870c90 fix(audio): improve audio device state management
Add proper cleanup for both audio input manager and output supervisor when disabling audio
Ensure complete shutdown of audio processes before USB reconfiguration
2025-08-25 13:19:29 +00:00
Alex P 1a0377bbdf Improvement: automatically resume audio when the audio usb gadget is re-enabled from settings 2025-08-25 12:36:04 +00:00
Alex P f24443e072 Improvement: automatically resume audio when the audio usb gadget is re-enabled from settings 2025-08-25 11:05:42 +00:00
Alex P 2afe2ca539 Fix: USB Gadgets updates 2025-08-25 10:41:53 +00:00
Alex P bc53523fbb Fix: USB Gadgets update 2025-08-25 09:19:03 +00:00
Alex P 44a35aa5c2 feat(audio): add socket buffer configuration and monitoring
Add socket buffer configuration support with metrics collection for audio IPC connections. This improves performance monitoring and allows tuning socket buffers for optimal audio streaming performance.

- Introduce SocketBufferConfig struct with default and high-load presets
- Add socket buffer configuration to AudioServer and AudioInputServer
- Implement socket buffer metrics collection (size, utilization, overflow)
- Add new Prometheus metrics for socket buffer monitoring
2025-08-24 23:56:58 +00:00
Alex P 3a28105f56 Fix: linting errors 2025-08-24 23:36:29 +00:00
Alex P a9a1082bcc Docs: Updated documentation 2025-08-24 23:15:46 +00:00
Alex e0f7b1d930
Merge branch 'dev' into feat/audio-support 2025-08-25 02:08:12 +03:00
Marc Brooks 5188717bb9 fix: useJsonRpc "any" issue
PR #743 didn't have all the files included in the commit.
Mea culpa, many apologies.
2025-08-24 22:58:35 +00:00
Alex Ballas 70e49a1cac chore: ensure that rpc messages get processed sequentially and avoid phantom and repeated key presses (#744) 2025-08-24 22:57:37 +00:00
Marc Brooks 9d40263eed fix: compiler error (#743)
Using { send } gives the resp a type instead of any
2025-08-24 22:54:26 +00:00
Adam Shiervani 0651faeceb feat: improve custom jiggler settings and add timezone support (#742)
* feat: add timezone support to jiggler and fix custom settings persistence

- Add timezone field to JigglerConfig with comprehensive IANA timezone list
- Fix custom settings not loading current values
- Remove business hours preset, add as examples in custom settings
- Improve error handling for invalid cron expressions

* fix: format jiggler.go with gofmt

* fix: add embedded timezone data and validation

- Import time/tzdata to embed timezone database in binary
- Add timezone validation in runJigglerCronTab() to gracefully fallback to UTC
- Add timezone to debug logging in rpcSetJigglerConfig
- Fixes 'unknown time zone' errors when system lacks timezone data

* refactor: add timezone field comments from jiggler options

* chore: move tzdata to backend

* refactor: fix JigglerSetting linting

- Adjusted useEffect dependency to include send function for better data fetching
- Modified layout classes for improved responsiveness and consistency
- Cleaned up code formatting for better readability

---------

Co-authored-by: Siyuan Miao <i@xswan.net>
2025-08-24 22:54:26 +00:00
Serhii 199cca83ed Update mount list for new Debian 13 release (#739)
* Update mount list for new Debian 13 release

* Keep Debian 12 Bookworm as old-stable release
2025-08-24 22:54:26 +00:00
Marc Brooks a3b2b46f49 Lint fix from last merge. (#733) 2025-08-24 22:54:26 +00:00
jackislanding f729675a3f Added crontab scheduler for jiggler (#316) 2025-08-24 22:54:26 +00:00
Marc Brooks 785a68d923 chore(ui)/package upgrades (#724)
| Package                          | From     | To           |
| -------------------------------- | ----------- | ------------ |
| @headlessui/react        | 2.2.4     | 2.2.7        |
| framer-motion              | 12.23.3 | 12.23.12 |
| react                                | 19.1.0   | 19.1.1     |
| react-dom                       | 19.1.0   | 19.1.1    |
| react-simple-keyboard | 3.8.93   | 3.8.106  |
|@eslint/js                         | 9.30.1   | 9.32.0    |
| @types/react                  | 19.1.8   | 19.1.9    |
| @types/react-dom         | 19.1.8   | 19.1.9   |
|eslint                                 | 9.30.1   | 9.32.0    |
|eslint-config-prettier       | 10.1.5   | 10.1.8   |
| typescript                         |  5.8.3    | 5.9.2     |
2025-08-24 22:54:26 +00:00
Alex P 57b7bafcc1 feat(audio): implement comprehensive audio optimization system
- Add AdaptiveOptimizer for real-time parameter adjustment based on latency metrics
- Add AdaptiveBufferConfig for dynamic buffer sizing based on system load
- Implement BatchAudioProcessor for reduced CGO call overhead
- Add AudioBufferPool with sync.Pool for optimized memory allocation
- Implement LatencyMonitor with exponential moving averages
- Add MemoryMetrics for comprehensive memory usage tracking
- Implement PriorityScheduler with SCHED_FIFO for real-time audio processing
- Add zero-copy operations to minimize memory copying in audio pipeline
- Enhance IPC architecture with intelligent frame dropping
- Add comprehensive Prometheus metrics for performance monitoring
- Implement triple-goroutine architecture for audio input processing
- Add adaptive buffering and performance feedback loops
2025-08-24 22:33:49 +00:00
Alex P 88679cda2f refactor(audio): improve process monitoring with dynamic clock ticks
- Extract monitoring constants and configuration into centralized locations
- Implement dynamic clock ticks detection for more accurate CPU metrics
- Add warmup samples and bounds checking for CPU percentage calculation
- Replace hardcoded values with constants for better maintainability
2025-08-23 23:35:38 +00:00
Alex P 76174f4486 refactor(audio): improve performance and simplify code structure
- Move audio server logic to dedicated package and simplify main.go
- Optimize buffer pool implementation and remove redundant logging
- Improve process monitoring with synchronized metrics updates
- Enhance microphone contention manager with simplified logic
- Replace mutex with atomic operations for metrics tracking
2025-08-23 22:54:01 +00:00
Alex P 27a999c58a [WIP] Updates: audio output & input subprocesses memory & cpu usage 2025-08-23 21:51:24 +00:00
Alex P ddc2f90016 [WIP] Updates: audio output & input subprocesses memory & cpu usage 2025-08-23 21:36:57 +00:00
Alex P 692f7ddb2d [WIP] Updates: audio output & input subprocesses memory & cpu usage 2025-08-23 21:19:28 +00:00
Alex P 38ad145863 [WIP] Updates: audio output & input subprocesses memory & cpu usage 2025-08-23 21:06:02 +00:00
Alex P 879ea5e472 Fix: fix audio input by reverting change 2025-08-23 16:41:45 +00:00
Alex P 2082b1a671 refactor(audio): rename audio-server flag to audio-output-server for clarity
docs: update development documentation with new make targets
refactor: simplify audio quality presets implementation
style: remove redundant comments and align error handling
chore: add lint-ui-fix target to Makefile
2025-08-23 12:18:33 +00:00
Alex P 5e28a6c429 feat(audio): add system memory endpoint and process metrics monitoring
- Add new /system/memory endpoint to expose total system memory
- Implement process metrics collection for audio and microphone processes
- Update UI to display real-time process metrics with charts
- Replace environment variable check with CLI flag for audio input server
- Improve audio metrics broadcasting with 1-second intervals
- Add memory usage capping for CPU percentage metrics
2025-08-23 11:41:03 +00:00
Alex P 0e1c896aa2 Fix: go lint errors 2025-08-22 23:23:07 +00:00
Alex P 0ed84257f6 Improvements, Fixes: enhanced audio metrics (including prometheus format), fixed lint errors 2025-08-22 23:20:22 +00:00
Alex P 32055f5762 Cleanup: remove polling fallback for /audio/mute status 2025-08-22 22:54:05 +00:00
Alex P 97bcb3c1ea Fix: linter errors 2025-08-22 22:29:48 +00:00
Alex P 6ecb829334 Fix: linter errors 2025-08-22 22:28:15 +00:00
Alex P e360348829 Fix: linter errors 2025-08-22 22:26:15 +00:00
Alex P 1e1677b35a Fix: linter errors 2025-08-22 22:23:50 +00:00
Alex P 3c1e9b8dc2 Fix: audio subprocess handling, avg atency audio metric 2025-08-22 22:21:41 +00:00
Alex P 62d4ec2f89 Fix: audio subprocess handling 2025-08-22 22:17:27 +00:00
Alex P aeb7a12c72 Fix: linting errors 2025-08-22 22:07:35 +00:00
Alex P 671d875890 Fix: literal /home/vscode in cache paths 2025-08-22 21:49:15 +00:00
Alex P 7129bd5521 Fix: workflow indentation 2025-08-22 21:43:51 +00:00
Alex P bd4fbef6dc Tweak: steps order 2025-08-22 21:43:17 +00:00
Alex P b3373e56de Improvement: use cache save/restore actions 2025-08-22 21:41:44 +00:00
Alex P 73e8897fc3 Improvement: Automatically invalidate cache 2025-08-22 21:37:53 +00:00
Alex P de0077a351 Fix: always save cache 2025-08-22 21:34:27 +00:00
Alex P 4875c243d3 Fix: Lint env vars 2025-08-22 21:30:24 +00:00
Alex P 071129a9ec Fix: use absolute path for caching 2025-08-22 21:26:37 +00:00
Alex P dee8a0b5a1 Fix: golangci-lint 2025-08-22 21:21:09 +00:00
Alex P a976ce1da9 Updates: set LDFLAGS and CFLAGS for the lint steps 2025-08-22 21:15:25 +00:00
Alex P d5295d0e4b Updates: golangci-lint workflow 2025-08-22 21:06:40 +00:00
Alex P 423d5775e3 [WIP] Performance Enhancements: move audion processing into a separate process 2025-08-22 12:04:30 +00:00
Adam Shiervani 7e83015932 refactor(vite.config): fix local ui development proxy 2025-08-13 18:53:30 +02:00
Alex P 629cdf59a7 perf(audio): optimize audio processing with batching and atomic operations
- Implement batch audio processing to reduce CGO overhead
- Replace mutexes with atomic operations for contention management
- Add buffer pooling to reduce allocations
- Optimize microphone operation cooldown with lock-free approach
- Improve error handling with pre-allocated error objects
2025-08-13 14:49:08 +00:00
Alex P 767311ec04 [WIP] Fix: performance issues 2025-08-13 11:35:11 +00:00
Alex P c51bdc50b5 Fix: linter errors 2025-08-12 13:59:21 +00:00
Alex P 1f2c46230c build(audio): update cgo LDFLAGS to use env vars for library versions 2025-08-12 13:35:39 +00:00
Alex P 4688f9e6ca perf(build): add ARM Cortex-A7 optimization flags for audio builds
Add compiler optimization flags targeting ARM Cortex-A7 with NEON support
to improve performance of audio library builds and Go binaries. The flags
enable vectorization, fast math, and loop unrolling for better execution
speed on the target hardware.
2025-08-12 11:20:19 +00:00
Alex P a9a92c52ab feat(rpc): optimize input handling with direct path for performance
perf(audio): make audio library versions configurable in build

test(input): add comprehensive tests for input RPC validation
2025-08-12 10:56:09 +00:00
Alex P 4b693b4279 perf(usbgadget): reduce input latency by pre-opening HID files and removing throttling
Pre-open HID files during initialization to minimize I/O overhead during operation. Remove mouse event throttling mechanism to improve input responsiveness. Keep HID files open on write errors to avoid repeated file operations.
2025-08-12 10:07:58 +00:00
Alex P 5f905e7eee Fix: session duplication detection, dev_deploy.sh script 2025-08-07 10:12:50 +00:00
Alex P 94ca3fa3f4 Stability: prevent race condition when clicking on Mic Start, Stop buttons in quick succession 2025-08-05 09:02:21 +03:00
Alex P 3c1f96d49c Separation of Concerns: Move the audio-related code into the audio internal package 2025-08-05 02:04:37 +03:00
Alex P a208715cc6 Fix: goimports 2025-08-05 01:49:09 +03:00
Alex P 638d08cdc5 Fix: goimports 2025-08-05 01:47:50 +03:00
Alex P 520c218598 Efficiency Improvements: Switch to websocket-based communication for audio metrics & status 2025-08-05 01:43:58 +03:00
Alex P 3158ca59f7 Finetuning: further perf. tunning 2025-08-05 01:05:06 +03:00
Alex P 612dca3fca Fix: eslint errors 2025-08-05 00:17:08 +03:00
Alex P 3444607021 Improvements, Fixes: reduce mouse lag when audio is on 2025-08-04 23:29:47 +03:00
Alex P 3dc196bab5 Fix: lint errors 2025-08-04 20:30:39 +03:00
Alex P 575abb75f0 [WIP] Updates: audio input support 2025-08-04 20:08:54 +03:00
Alex P 09ac8c5e37 Cleanup / Fix: linting errors, code formatting, etc 2025-08-02 17:45:24 +00:00
Alex P 4f47d62079 [#315] Updates: add advanced audio support 2025-08-02 02:23:37 +00:00
Qishuai Liu 28a8fa05cc
feat: use native jetkvm-audio 2025-06-26 00:30:00 +09:00
Qishuai Liu c529c903d0
Merge remote-tracking branch 'upstream/dev' into feat/usb-audio 2025-06-22 00:36:01 +09:00
Qishuai Liu 9d12dd1e54
fix: audio rtp timestamp 2025-05-16 23:11:22 +09:00
Qishuai Liu cc83e4193f
feat: add audio encoder 2025-05-14 23:41:48 +09:00
Qishuai Liu 466271d935
feat: add usb gadget audio config 2025-05-14 23:15:45 +09:00
42 changed files with 2773 additions and 49 deletions

View File

@ -5,7 +5,7 @@ function sudo() {
if [ "$UID" -eq 0 ]; then
"$@"
else
${SUDO_PATH} "$@"
${SUDO_PATH} -E "$@"
fi
}
@ -17,7 +17,7 @@ sudo apt-get install -y --no-install-recommends \
iputils-ping \
build-essential \
device-tree-compiler \
gperf g++-multilib gcc-multilib \
gperf \
libnl-3-dev libdbus-1-dev libelf-dev libmpc-dev dwarves \
bc openssl flex bison libssl-dev python3 python-is-python3 texinfo kmod cmake \
wget zstd \
@ -31,7 +31,35 @@ pushd "${BUILDKIT_TMPDIR}" > /dev/null
wget https://github.com/jetkvm/rv1106-system/releases/download/${BUILDKIT_VERSION}/buildkit.tar.zst && \
sudo mkdir -p /opt/jetkvm-native-buildkit && \
sudo tar --use-compress-program="unzstd --long=31" -xvf buildkit.tar.zst -C /opt/jetkvm-native-buildkit && \
sudo tar --use-compress-program="zstd -d --long=31" -xvf buildkit.tar.zst -C /opt/jetkvm-native-buildkit && \
rm buildkit.tar.zst
popd
# Install audio dependencies (ALSA and Opus) for JetKVM
echo "Installing JetKVM audio dependencies..."
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
PROJECT_ROOT="$(dirname "${SCRIPT_DIR}")"
AUDIO_DEPS_SCRIPT="${PROJECT_ROOT}/install_audio_deps.sh"
if [ -f "${AUDIO_DEPS_SCRIPT}" ]; then
echo "Running audio dependencies installation..."
# Pre-create audio libs directory with proper permissions
sudo mkdir -p /opt/jetkvm-audio-libs
sudo chmod 777 /opt/jetkvm-audio-libs
# Run installation script (now it can write without sudo)
bash "${AUDIO_DEPS_SCRIPT}"
echo "Audio dependencies installation completed."
if [ -d "/opt/jetkvm-audio-libs" ]; then
echo "Audio libraries installed in /opt/jetkvm-audio-libs"
# Set recursive permissions for all subdirectories and files
sudo chmod -R 777 /opt/jetkvm-audio-libs
echo "Permissions set to allow all users access to audio libraries"
else
echo "Error: /opt/jetkvm-audio-libs directory not found after installation."
exit 1
fi
else
echo "Warning: Audio dependencies script not found at ${AUDIO_DEPS_SCRIPT}"
echo "Skipping audio dependencies installation."
fi
rm -rf "${BUILDKIT_TMPDIR}"

View File

@ -0,0 +1,74 @@
#!/bin/bash
# .devcontainer/install_audio_deps.sh
# Build ALSA and Opus static libs for ARM in /opt/jetkvm-audio-libs
set -e
# Sudo wrapper function
SUDO_PATH=$(which sudo 2>/dev/null || echo "")
function use_sudo() {
if [ "$UID" -eq 0 ]; then
"$@"
elif [ -n "$SUDO_PATH" ]; then
${SUDO_PATH} -E "$@"
else
"$@"
fi
}
# Accept version parameters or use defaults
ALSA_VERSION="${1:-1.2.14}"
OPUS_VERSION="${2:-1.5.2}"
AUDIO_LIBS_DIR="/opt/jetkvm-audio-libs"
BUILDKIT_PATH="/opt/jetkvm-native-buildkit"
BUILDKIT_FLAVOR="arm-rockchip830-linux-uclibcgnueabihf"
CROSS_PREFIX="$BUILDKIT_PATH/bin/$BUILDKIT_FLAVOR"
# Create directory with proper permissions
use_sudo mkdir -p "$AUDIO_LIBS_DIR"
use_sudo chmod 777 "$AUDIO_LIBS_DIR"
cd "$AUDIO_LIBS_DIR"
# Download sources
[ -f alsa-lib-${ALSA_VERSION}.tar.bz2 ] || wget -N https://www.alsa-project.org/files/pub/lib/alsa-lib-${ALSA_VERSION}.tar.bz2
[ -f opus-${OPUS_VERSION}.tar.gz ] || wget -N https://downloads.xiph.org/releases/opus/opus-${OPUS_VERSION}.tar.gz
# Extract
[ -d alsa-lib-${ALSA_VERSION} ] || tar xf alsa-lib-${ALSA_VERSION}.tar.bz2
[ -d opus-${OPUS_VERSION} ] || tar xf opus-${OPUS_VERSION}.tar.gz
# Optimization flags for ARM Cortex-A7 with NEON (simplified to avoid FD_SETSIZE issues)
OPTIM_CFLAGS="-O2 -mfpu=neon -mtune=cortex-a7 -mfloat-abi=hard"
export CC="${CROSS_PREFIX}-gcc"
export CFLAGS="$OPTIM_CFLAGS"
export CXXFLAGS="$OPTIM_CFLAGS"
# Build ALSA
cd alsa-lib-${ALSA_VERSION}
if [ ! -f .built ]; then
chown -R $(whoami):$(whoami) .
# Use minimal ALSA configuration to avoid FD_SETSIZE issues in devcontainer
CFLAGS="$OPTIM_CFLAGS" ./configure --host $BUILDKIT_FLAVOR \
--enable-static=yes --enable-shared=no \
--with-pcm-plugins=rate,linear \
--disable-seq --disable-rawmidi --disable-ucm \
--disable-python --disable-old-symbols \
--disable-topology --disable-hwdep --disable-mixer \
--disable-alisp --disable-aload --disable-resmgr
make -j$(nproc)
touch .built
fi
cd ..
# Build Opus
cd opus-${OPUS_VERSION}
if [ ! -f .built ]; then
chown -R $(whoami):$(whoami) .
CFLAGS="$OPTIM_CFLAGS" ./configure --host $BUILDKIT_FLAVOR --enable-static=yes --enable-shared=no --enable-fixed-point
make -j$(nproc)
touch .built
fi
cd ..
echo "ALSA and Opus built in $AUDIO_LIBS_DIR"

2
.gitignore vendored
View File

@ -14,4 +14,4 @@ node_modules
#internal/native/include
#internal/native/lib
ui/reports
ui/reports

View File

@ -6,6 +6,8 @@ ENV GOPATH=/go
ENV PATH=$GOPATH/bin:/usr/local/go/bin:$PATH
COPY install-deps.sh /install-deps.sh
COPY install_audio_deps.sh /install_audio_deps.sh
RUN /install-deps.sh
# Create build directory
@ -21,4 +23,4 @@ RUN go mod download && go mod verify
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT [ "/entrypoint.sh" ]
ENTRYPOINT [ "/entrypoint.sh" ]

104
Makefile
View File

@ -1,10 +1,52 @@
BRANCH := $(shell git rev-parse --abbrev-ref HEAD)
BUILDDATE := $(shell date -u +%FT%T%z)
BUILDTS := $(shell date -u +%s)
REVISION := $(shell git rev-parse HEAD)
# Build ALSA and Opus static libs for ARM in /opt/jetkvm-audio-libs
build_audio_deps:
bash .devcontainer/install_audio_deps.sh $(ALSA_VERSION) $(OPUS_VERSION)
# Prepare everything needed for local development (toolchain + audio deps + Go tools)
dev_env: build_audio_deps
$(CLEAN_GO_CACHE)
@echo "Installing Go development tools..."
go install golang.org/x/tools/cmd/goimports@latest
@echo "Development environment ready."
JETKVM_HOME ?= $(HOME)/.jetkvm
BUILDKIT_PATH ?= /opt/jetkvm-native-buildkit
BUILDKIT_FLAVOR ?= arm-rockchip830-linux-uclibcgnueabihf
AUDIO_LIBS_DIR ?= /opt/jetkvm-audio-libs
BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD)
BUILDDATE ?= $(shell date -u +%FT%T%z)
BUILDTS ?= $(shell date -u +%s)
REVISION ?= $(shell git rev-parse HEAD)
VERSION_DEV := 0.4.9-dev$(shell date +%Y%m%d%H%M)
VERSION := 0.4.8
# Audio library versions
ALSA_VERSION ?= 1.2.14
OPUS_VERSION ?= 1.5.2
# Set PKG_CONFIG_PATH globally for all targets that use CGO with audio libraries
export PKG_CONFIG_PATH := $(AUDIO_LIBS_DIR)/alsa-lib-$(ALSA_VERSION)/utils:$(AUDIO_LIBS_DIR)/opus-$(OPUS_VERSION)
# Common command to clean Go cache with verbose output for all Go builds
CLEAN_GO_CACHE := @echo "Cleaning Go cache..."; go clean -cache -v
# Optimization flags for ARM Cortex-A7 with NEON SIMD
OPTIM_CFLAGS := -O3 -mfpu=neon -mtune=cortex-a7 -mfloat-abi=hard -ftree-vectorize -ffast-math -funroll-loops -mvectorize-with-neon-quad -marm -D__ARM_NEON
# Cross-compilation environment for ARM - exported globally
export GOOS := linux
export GOARCH := arm
export GOARM := 7
export CC := $(BUILDKIT_PATH)/bin/$(BUILDKIT_FLAVOR)-gcc
export CGO_ENABLED := 1
export CGO_CFLAGS := $(OPTIM_CFLAGS) -I$(BUILDKIT_PATH)/$(BUILDKIT_FLAVOR)/include -I$(BUILDKIT_PATH)/$(BUILDKIT_FLAVOR)/sysroot/usr/include
export CGO_LDFLAGS := -L$(BUILDKIT_PATH)/$(BUILDKIT_FLAVOR)/lib -L$(BUILDKIT_PATH)/$(BUILDKIT_FLAVOR)/sysroot/usr/lib -lrockit -lrockchip_mpp -lrga -lpthread -lm -ldl
# Audio-specific flags (only used for audio C binaries, NOT for main Go app)
AUDIO_CFLAGS := $(CGO_CFLAGS) -I$(AUDIO_LIBS_DIR)/alsa-lib-$(ALSA_VERSION)/include -I$(AUDIO_LIBS_DIR)/opus-$(OPUS_VERSION)/include -I$(AUDIO_LIBS_DIR)/opus-$(OPUS_VERSION)/celt
AUDIO_LDFLAGS := $(AUDIO_LIBS_DIR)/alsa-lib-$(ALSA_VERSION)/src/.libs/libasound.a $(AUDIO_LIBS_DIR)/opus-$(OPUS_VERSION)/.libs/libopus.a -lm -ldl -lpthread
PROMETHEUS_TAG := github.com/prometheus/common/version
KVM_PKG_NAME := github.com/jetkvm/kvm
@ -55,22 +97,26 @@ build_native:
./scripts/build_cgo.sh; \
fi
build_dev: build_native
build_dev: build_native build_audio_deps
$(CLEAN_GO_CACHE)
@echo "Building..."
$(GO_CMD) build \
go build \
-ldflags="$(GO_LDFLAGS) -X $(KVM_PKG_NAME).builtAppVersion=$(VERSION_DEV)" \
$(GO_RELEASE_BUILD_ARGS) \
-o $(BIN_DIR)/jetkvm_app -v cmd/main.go
build_test2json:
$(CLEAN_GO_CACHE)
$(GO_CMD) build -o $(BIN_DIR)/test2json cmd/test2json
build_gotestsum:
$(CLEAN_GO_CACHE)
@echo "Building gotestsum..."
$(GO_CMD) install gotest.tools/gotestsum@latest
cp $(shell $(GO_CMD) env GOPATH)/bin/linux_arm/gotestsum $(BIN_DIR)/gotestsum
build_dev_test: build_test2json build_gotestsum
build_dev_test: build_audio_deps build_test2json build_gotestsum
$(CLEAN_GO_CACHE)
# collect all directories that contain tests
@echo "Building tests for devices ..."
@rm -rf $(BIN_DIR)/tests && mkdir -p $(BIN_DIR)/tests
@ -80,7 +126,7 @@ build_dev_test: build_test2json build_gotestsum
test_pkg_name=$$(echo $$test | sed 's/^.\///g'); \
test_pkg_full_name=$(KVM_PKG_NAME)/$$(echo $$test | sed 's/^.\///g'); \
test_filename=$$(echo $$test_pkg_name | sed 's/\//__/g')_test; \
$(GO_CMD) test -v \
go test -v \
-ldflags="$(GO_LDFLAGS) -X $(KVM_PKG_NAME).builtAppVersion=$(VERSION_DEV)" \
$(GO_BUILD_ARGS) \
-c -o $(BIN_DIR)/tests/$$test_filename $$test; \
@ -117,9 +163,10 @@ dev_release: frontend build_dev
rclone copyto bin/jetkvm_app r2://jetkvm-update/app/$(VERSION_DEV)/jetkvm_app
rclone copyto bin/jetkvm_app.sha256 r2://jetkvm-update/app/$(VERSION_DEV)/jetkvm_app.sha256
build_release: frontend build_native
build_release: frontend build_native build_audio_deps
$(CLEAN_GO_CACHE)
@echo "Building release..."
$(GO_CMD) build \
go build \
-ldflags="$(GO_LDFLAGS) -X $(KVM_PKG_NAME).builtAppVersion=$(VERSION)" \
$(GO_RELEASE_BUILD_ARGS) \
-o bin/jetkvm_app cmd/main.go
@ -133,4 +180,39 @@ release:
@echo "Uploading release..."
@shasum -a 256 bin/jetkvm_app | cut -d ' ' -f 1 > bin/jetkvm_app.sha256
rclone copyto bin/jetkvm_app r2://jetkvm-update/app/$(VERSION)/jetkvm_app
rclone copyto bin/jetkvm_app.sha256 r2://jetkvm-update/app/$(VERSION)/jetkvm_app.sha256
rclone copyto bin/jetkvm_app.sha256 r2://jetkvm-update/app/$(VERSION)/jetkvm_app.sha256
# Run both Go and UI linting
lint: lint-go lint-ui
@echo "All linting completed successfully!"
# Run golangci-lint locally with the same configuration as CI
lint-go: build_audio_deps
@echo "Running golangci-lint..."
@mkdir -p static && touch static/.gitkeep
golangci-lint run --verbose
# Run both Go and UI linting with auto-fix
lint-fix: lint-go-fix lint-ui-fix
@echo "All linting with auto-fix completed successfully!"
# Run golangci-lint with auto-fix
lint-go-fix: build_audio_deps
@echo "Running golangci-lint with auto-fix..."
@mkdir -p static && touch static/.gitkeep
golangci-lint run --fix --verbose
# Run UI linting locally (mirrors GitHub workflow ui-lint.yml)
lint-ui:
@echo "Running UI lint..."
@cd ui && npm ci
@cd ui && npm run lint
# Run UI linting with auto-fix
lint-ui-fix:
@echo "Running UI lint with auto-fix..."
@cd ui && npm ci
@cd ui && npm run lint:fix
# Legacy alias for UI linting (for backward compatibility)
ui-lint: lint-ui

253
audio.go Normal file
View File

@ -0,0 +1,253 @@
package kvm
import (
"io"
"sync"
"sync/atomic"
"github.com/jetkvm/kvm/internal/audio"
"github.com/jetkvm/kvm/internal/logging"
"github.com/pion/webrtc/v4"
"github.com/rs/zerolog"
)
var (
audioMutex sync.Mutex
outputSource audio.AudioSource
inputSource audio.AudioSource
outputRelay *audio.OutputRelay
inputRelay *audio.InputRelay
audioInitialized bool
activeConnections atomic.Int32
audioLogger zerolog.Logger
currentAudioTrack *webrtc.TrackLocalStaticSample
inputTrackHandling atomic.Bool
audioOutputEnabled atomic.Bool
audioInputEnabled atomic.Bool
)
func initAudio() {
audioLogger = logging.GetDefaultLogger().With().Str("component", "audio-manager").Logger()
audioOutputEnabled.Store(true)
audioInputEnabled.Store(true)
audioLogger.Debug().Msg("Audio subsystem initialized")
audioInitialized = true
}
// startAudio starts audio sources and relays (skips already running ones)
func startAudio() error {
audioMutex.Lock()
defer audioMutex.Unlock()
if !audioInitialized {
audioLogger.Warn().Msg("Audio not initialized, skipping start")
return nil
}
// Start output audio if not running and enabled
if outputSource == nil && audioOutputEnabled.Load() {
alsaDevice := "hw:1,0" // USB audio
outputSource = audio.NewCgoOutputSource(alsaDevice)
if currentAudioTrack != nil {
outputRelay = audio.NewOutputRelay(outputSource, currentAudioTrack)
if err := outputRelay.Start(); err != nil {
audioLogger.Error().Err(err).Msg("Failed to start audio output relay")
}
}
}
// Start input audio if not running, USB audio enabled, and input enabled
ensureConfigLoaded()
if inputSource == nil && audioInputEnabled.Load() && config.UsbDevices != nil && config.UsbDevices.Audio {
alsaPlaybackDevice := "hw:1,0" // USB speakers
// Create CGO audio source
inputSource = audio.NewCgoInputSource(alsaPlaybackDevice)
inputRelay = audio.NewInputRelay(inputSource)
if err := inputRelay.Start(); err != nil {
audioLogger.Error().Err(err).Msg("Failed to start input relay")
}
}
return nil
}
// stopOutputLocked stops output audio (assumes mutex is held)
func stopOutputLocked() {
if outputRelay != nil {
outputRelay.Stop()
outputRelay = nil
}
if outputSource != nil {
outputSource.Disconnect()
outputSource = nil
}
}
// stopInputLocked stops input audio (assumes mutex is held)
func stopInputLocked() {
if inputRelay != nil {
inputRelay.Stop()
inputRelay = nil
}
if inputSource != nil {
inputSource.Disconnect()
inputSource = nil
}
}
// stopAudioLocked stops all audio (assumes mutex is held)
func stopAudioLocked() {
stopOutputLocked()
stopInputLocked()
}
// stopAudio stops all audio
func stopAudio() {
audioMutex.Lock()
defer audioMutex.Unlock()
stopAudioLocked()
}
func onWebRTCConnect() {
count := activeConnections.Add(1)
if count == 1 {
if err := startAudio(); err != nil {
audioLogger.Error().Err(err).Msg("Failed to start audio")
}
}
}
func onWebRTCDisconnect() {
count := activeConnections.Add(-1)
if count == 0 {
// Stop audio immediately to release HDMI audio device which shares hardware with video device
stopAudio()
}
}
func setAudioTrack(audioTrack *webrtc.TrackLocalStaticSample) {
audioMutex.Lock()
defer audioMutex.Unlock()
currentAudioTrack = audioTrack
if outputRelay != nil {
outputRelay.Stop()
outputRelay = nil
}
if outputSource != nil {
outputRelay = audio.NewOutputRelay(outputSource, audioTrack)
if err := outputRelay.Start(); err != nil {
audioLogger.Error().Err(err).Msg("Failed to start output relay")
}
}
}
func setPendingInputTrack(track *webrtc.TrackRemote) {
audioMutex.Lock()
defer audioMutex.Unlock()
// Start input track handler only once per WebRTC session
if inputTrackHandling.CompareAndSwap(false, true) {
go handleInputTrackForSession(track)
}
}
// SetAudioOutputEnabled enables or disables audio output
func SetAudioOutputEnabled(enabled bool) error {
if audioOutputEnabled.Swap(enabled) == enabled {
return nil // Already in desired state
}
if enabled {
if activeConnections.Load() > 0 {
return startAudio()
}
} else {
audioMutex.Lock()
stopOutputLocked()
audioMutex.Unlock()
}
return nil
}
// SetAudioInputEnabled enables or disables audio input
func SetAudioInputEnabled(enabled bool) error {
if audioInputEnabled.Swap(enabled) == enabled {
return nil // Already in desired state
}
if enabled {
if activeConnections.Load() > 0 {
return startAudio()
}
} else {
audioMutex.Lock()
stopInputLocked()
audioMutex.Unlock()
}
return nil
}
// handleInputTrackForSession runs for the entire WebRTC session lifetime
// It continuously reads from the track and sends to whatever relay is currently active
func handleInputTrackForSession(track *webrtc.TrackRemote) {
defer inputTrackHandling.Store(false)
audioLogger.Debug().
Str("codec", track.Codec().MimeType).
Str("track_id", track.ID()).
Msg("starting session-lifetime track handler")
for {
// Read RTP packet (must always read to keep track alive)
rtpPacket, _, err := track.ReadRTP()
if err != nil {
if err == io.EOF {
audioLogger.Debug().Msg("audio track ended")
return
}
audioLogger.Warn().Err(err).Msg("failed to read RTP packet")
continue
}
// Extract Opus payload
opusData := rtpPacket.Payload
if len(opusData) == 0 {
continue
}
// Only send if input is enabled
if !audioInputEnabled.Load() {
continue // Drop frame but keep reading
}
// Get source in single mutex operation (hot path optimization)
audioMutex.Lock()
source := inputSource
audioMutex.Unlock()
if source == nil {
continue // No relay, drop frame but keep reading
}
if !source.IsConnected() {
if err := source.Connect(); err != nil {
continue
}
}
if err := source.WriteMessage(0, opusData); err != nil {
source.Disconnect()
}
}
}

View File

@ -151,6 +151,7 @@ var (
RelativeMouse: true,
Keyboard: true,
MassStorage: true,
Audio: true,
}
)

728
internal/audio/c/audio.c Normal file
View File

@ -0,0 +1,728 @@
/*
* JetKVM Audio Processing Module
*
* Bidirectional audio processing optimized for ARM NEON SIMD:
* TODO: Remove USB Gadget audio once new system image release is made available
* - OUTPUT PATH: TC358743 HDMI or USB Gadget audio Client speakers
* Pipeline: ALSA hw:0,0 or hw:1,0 capture Opus encode (128kbps, FEC enabled)
*
* - INPUT PATH: Client microphone Device speakers
* Pipeline: Opus decode (with FEC) ALSA hw:1,0 playback
*
* Key features:
* - ARM NEON SIMD optimization for all audio operations
* - Opus in-band FEC for packet loss resilience
* - S16_LE @ 48kHz stereo, 20ms frames (960 samples)
*/
#include <alsa/asoundlib.h>
#include <opus.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <sched.h>
#include <time.h>
#include <signal.h>
// ARM NEON SIMD support (always available on JetKVM's ARM Cortex-A7)
#include <arm_neon.h>
// RV1106 (Cortex-A7) has 64-byte cache lines
#define CACHE_LINE_SIZE 64
#define SIMD_ALIGN __attribute__((aligned(16)))
#define CACHE_ALIGN __attribute__((aligned(CACHE_LINE_SIZE)))
#define SIMD_PREFETCH(addr, rw, locality) __builtin_prefetch(addr, rw, locality)
// Compile-time trace logging - disabled for production (zero overhead)
#define TRACE_LOG(...) ((void)0)
// ALSA device handles
static snd_pcm_t *pcm_capture_handle = NULL; // OUTPUT: TC358743 HDMI audio → client
static snd_pcm_t *pcm_playback_handle = NULL; // INPUT: Client microphone → device speakers
// ALSA device names
static const char *alsa_capture_device = NULL;
static const char *alsa_playback_device = NULL;
// Opus codec instances
static OpusEncoder *encoder = NULL;
static OpusDecoder *decoder = NULL;
// Audio format (S16_LE @ 48kHz stereo)
static uint32_t sample_rate = 48000;
static uint8_t channels = 2;
static uint16_t frame_size = 960; // 20ms frames at 48kHz
static uint32_t opus_bitrate = 128000;
static uint8_t opus_complexity = 5; // Higher complexity for better quality
static uint16_t max_packet_size = 1500;
// Opus encoder constants (hardcoded for production)
#define OPUS_VBR 1 // VBR enabled
#define OPUS_VBR_CONSTRAINT 1 // Constrained VBR (prevents bitrate starvation at low volumes)
#define OPUS_SIGNAL_TYPE 3002 // OPUS_SIGNAL_MUSIC (better transient handling)
#define OPUS_BANDWIDTH 1104 // OPUS_BANDWIDTH_SUPERWIDEBAND (16kHz)
#define OPUS_DTX 1 // DTX enabled (bandwidth optimization)
#define OPUS_LSB_DEPTH 16 // 16-bit depth
// ALSA retry configuration
static uint32_t sleep_microseconds = 1000;
static uint32_t sleep_milliseconds = 1;
static uint8_t max_attempts_global = 5;
static uint32_t max_backoff_us_global = 500000;
int jetkvm_audio_capture_init();
void jetkvm_audio_capture_close();
int jetkvm_audio_read_encode(void *opus_buf);
int jetkvm_audio_playback_init();
void jetkvm_audio_playback_close();
int jetkvm_audio_decode_write(void *opus_buf, int opus_size);
void update_audio_constants(uint32_t bitrate, uint8_t complexity,
uint32_t sr, uint8_t ch, uint16_t fs, uint16_t max_pkt,
uint32_t sleep_us, uint8_t max_attempts, uint32_t max_backoff);
void update_audio_decoder_constants(uint32_t sr, uint8_t ch, uint16_t fs, uint16_t max_pkt,
uint32_t sleep_us, uint8_t max_attempts, uint32_t max_backoff);
int update_opus_encoder_params(uint32_t bitrate, uint8_t complexity);
/**
* Sync encoder configuration from Go to C
*/
void update_audio_constants(uint32_t bitrate, uint8_t complexity,
uint32_t sr, uint8_t ch, uint16_t fs, uint16_t max_pkt,
uint32_t sleep_us, uint8_t max_attempts, uint32_t max_backoff) {
opus_bitrate = bitrate;
opus_complexity = complexity;
sample_rate = sr;
channels = ch;
frame_size = fs;
max_packet_size = max_pkt;
sleep_microseconds = sleep_us;
sleep_milliseconds = sleep_us / 1000; // Precompute for snd_pcm_wait
max_attempts_global = max_attempts;
max_backoff_us_global = max_backoff;
}
/**
* Sync decoder configuration from Go to C (no encoder-only params)
*/
void update_audio_decoder_constants(uint32_t sr, uint8_t ch, uint16_t fs, uint16_t max_pkt,
uint32_t sleep_us, uint8_t max_attempts, uint32_t max_backoff) {
sample_rate = sr;
channels = ch;
frame_size = fs;
max_packet_size = max_pkt;
sleep_microseconds = sleep_us;
sleep_milliseconds = sleep_us / 1000; // Precompute for snd_pcm_wait
max_attempts_global = max_attempts;
max_backoff_us_global = max_backoff;
}
/**
* Initialize ALSA device names from environment variables
* Must be called before jetkvm_audio_capture_init or jetkvm_audio_playback_init
*/
static void init_alsa_devices_from_env(void) {
// Always read from environment to support device switching
alsa_capture_device = getenv("ALSA_CAPTURE_DEVICE");
if (alsa_capture_device == NULL || alsa_capture_device[0] == '\0') {
alsa_capture_device = "hw:1,0"; // Default to USB gadget
}
alsa_playback_device = getenv("ALSA_PLAYBACK_DEVICE");
if (alsa_playback_device == NULL || alsa_playback_device[0] == '\0') {
alsa_playback_device = "hw:1,0"; // Default to USB gadget
}
}
// SIMD-OPTIMIZED BUFFER OPERATIONS (ARM NEON)
/**
* Clear audio buffer using NEON (16 samples/iteration with 2x unrolling)
*/
static inline void simd_clear_samples_s16(short * __restrict__ buffer, uint32_t samples) {
const int16x8_t zero = vdupq_n_s16(0);
uint32_t i = 0;
// Process 16 samples at a time (2x unrolled for better pipeline utilization)
uint32_t simd_samples = samples & ~15U;
for (; i < simd_samples; i += 16) {
vst1q_s16(&buffer[i], zero);
vst1q_s16(&buffer[i + 8], zero);
}
// Handle remaining 8 samples
if (i + 8 <= samples) {
vst1q_s16(&buffer[i], zero);
i += 8;
}
// Scalar: remaining samples
for (; i < samples; i++) {
buffer[i] = 0;
}
}
// INITIALIZATION STATE TRACKING
static volatile sig_atomic_t capture_initializing = 0;
static volatile sig_atomic_t capture_initialized = 0;
static volatile sig_atomic_t playback_initializing = 0;
static volatile sig_atomic_t playback_initialized = 0;
/**
* Update Opus encoder settings at runtime (does NOT modify FEC or hardcoded settings)
*
* NOTE: Currently unused but kept for potential future runtime configuration updates.
* In the current CGO implementation, encoder params are set once via update_audio_constants()
* before initialization. This function would be useful if we add runtime bitrate/complexity
* adjustment without restarting the encoder.
*
* @return 0 on success, -1 if not initialized, >0 if some settings failed
*/
int update_opus_encoder_params(uint32_t bitrate, uint8_t complexity) {
if (!encoder || !capture_initialized) {
return -1;
}
// Update runtime-configurable parameters
opus_bitrate = bitrate;
opus_complexity = complexity;
// Apply settings to encoder
int result = 0;
result |= opus_encoder_ctl(encoder, OPUS_SET_BITRATE(opus_bitrate));
result |= opus_encoder_ctl(encoder, OPUS_SET_COMPLEXITY(opus_complexity));
return result;
}
// ALSA UTILITY FUNCTIONS
/**
* Open ALSA device with exponential backoff retry
* @return 0 on success, negative error code on failure
*/
// Helper: High-precision sleep using nanosleep (better than usleep)
static inline void precise_sleep_us(uint32_t microseconds) {
struct timespec ts = {
.tv_sec = microseconds / 1000000,
.tv_nsec = (microseconds % 1000000) * 1000
};
nanosleep(&ts, NULL);
}
static int safe_alsa_open(snd_pcm_t **handle, const char *device, snd_pcm_stream_t stream) {
uint8_t attempt = 0;
int err;
uint32_t backoff_us = sleep_microseconds;
while (attempt < max_attempts_global) {
err = snd_pcm_open(handle, device, stream, SND_PCM_NONBLOCK);
if (err >= 0) {
snd_pcm_nonblock(*handle, 0);
return 0;
}
attempt++;
// Exponential backoff with bit shift (faster than multiplication)
if (err == -EBUSY || err == -EAGAIN) {
precise_sleep_us(backoff_us);
backoff_us = (backoff_us << 1 < max_backoff_us_global) ? (backoff_us << 1) : max_backoff_us_global;
} else if (err == -ENODEV || err == -ENOENT) {
precise_sleep_us(backoff_us << 1);
backoff_us = (backoff_us << 1 < max_backoff_us_global) ? (backoff_us << 1) : max_backoff_us_global;
} else if (err == -EPERM || err == -EACCES) {
precise_sleep_us(backoff_us >> 1);
} else {
precise_sleep_us(backoff_us);
backoff_us = (backoff_us << 1 < max_backoff_us_global) ? (backoff_us << 1) : max_backoff_us_global;
}
}
return err;
}
/**
* Configure ALSA device (S16_LE @ 48kHz stereo with optimized buffering)
* @param handle ALSA PCM handle
* @param device_name Unused (for debugging only)
* @return 0 on success, negative error code on failure
*/
static int configure_alsa_device(snd_pcm_t *handle, const char *device_name) {
snd_pcm_hw_params_t *params;
snd_pcm_sw_params_t *sw_params;
int err;
if (!handle) return -1;
snd_pcm_hw_params_alloca(&params);
snd_pcm_sw_params_alloca(&sw_params);
err = snd_pcm_hw_params_any(handle, params);
if (err < 0) return err;
err = snd_pcm_hw_params_set_access(handle, params, SND_PCM_ACCESS_RW_INTERLEAVED);
if (err < 0) return err;
err = snd_pcm_hw_params_set_format(handle, params, SND_PCM_FORMAT_S16_LE);
if (err < 0) return err;
err = snd_pcm_hw_params_set_channels(handle, params, channels);
if (err < 0) return err;
err = snd_pcm_hw_params_set_rate(handle, params, sample_rate, 0);
if (err < 0) {
unsigned int rate = sample_rate;
err = snd_pcm_hw_params_set_rate_near(handle, params, &rate, 0);
if (err < 0) return err;
}
snd_pcm_uframes_t period_size = frame_size; // Optimized: use full frame as period
if (period_size < 64) period_size = 64;
err = snd_pcm_hw_params_set_period_size_near(handle, params, &period_size, 0);
if (err < 0) return err;
snd_pcm_uframes_t buffer_size = period_size * 4; // 4 periods = 80ms buffer for stability
err = snd_pcm_hw_params_set_buffer_size_near(handle, params, &buffer_size);
if (err < 0) return err;
err = snd_pcm_hw_params(handle, params);
if (err < 0) return err;
err = snd_pcm_sw_params_current(handle, sw_params);
if (err < 0) return err;
err = snd_pcm_sw_params_set_start_threshold(handle, sw_params, period_size);
if (err < 0) return err;
err = snd_pcm_sw_params_set_avail_min(handle, sw_params, period_size);
if (err < 0) return err;
err = snd_pcm_sw_params(handle, sw_params);
if (err < 0) return err;
return snd_pcm_prepare(handle);
}
// AUDIO OUTPUT PATH FUNCTIONS (TC358743 HDMI Audio → Client Speakers)
/**
* Initialize OUTPUT path (TC358743 HDMI capture Opus encoder)
* Opens hw:0,0 (TC358743) and creates Opus encoder with optimized settings
* @return 0 on success, -EBUSY if initializing, -1/-2/-3 on errors
*/
int jetkvm_audio_capture_init() {
int err;
init_alsa_devices_from_env();
if (__sync_bool_compare_and_swap(&capture_initializing, 0, 1) == 0) {
return -EBUSY;
}
if (capture_initialized) {
capture_initializing = 0;
return 0;
}
if (encoder) {
opus_encoder_destroy(encoder);
encoder = NULL;
}
if (pcm_capture_handle) {
snd_pcm_close(pcm_capture_handle);
pcm_capture_handle = NULL;
}
err = safe_alsa_open(&pcm_capture_handle, alsa_capture_device, SND_PCM_STREAM_CAPTURE);
if (err < 0) {
fprintf(stderr, "Failed to open ALSA capture device %s: %s\n",
alsa_capture_device, snd_strerror(err));
fflush(stderr);
capture_initializing = 0;
return -1;
}
err = configure_alsa_device(pcm_capture_handle, "capture");
if (err < 0) {
snd_pcm_close(pcm_capture_handle);
pcm_capture_handle = NULL;
capture_initializing = 0;
return -2;
}
int opus_err = 0;
encoder = opus_encoder_create(sample_rate, channels, OPUS_APPLICATION_AUDIO, &opus_err);
if (!encoder || opus_err != OPUS_OK) {
if (pcm_capture_handle) {
snd_pcm_close(pcm_capture_handle);
pcm_capture_handle = NULL;
}
capture_initializing = 0;
return -3;
}
// Configure encoder with optimized settings
opus_encoder_ctl(encoder, OPUS_SET_BITRATE(opus_bitrate));
opus_encoder_ctl(encoder, OPUS_SET_COMPLEXITY(opus_complexity));
opus_encoder_ctl(encoder, OPUS_SET_VBR(OPUS_VBR));
opus_encoder_ctl(encoder, OPUS_SET_VBR_CONSTRAINT(OPUS_VBR_CONSTRAINT));
opus_encoder_ctl(encoder, OPUS_SET_SIGNAL(OPUS_SIGNAL_TYPE));
opus_encoder_ctl(encoder, OPUS_SET_BANDWIDTH(OPUS_BANDWIDTH));
opus_encoder_ctl(encoder, OPUS_SET_DTX(OPUS_DTX));
opus_encoder_ctl(encoder, OPUS_SET_LSB_DEPTH(OPUS_LSB_DEPTH));
opus_encoder_ctl(encoder, OPUS_SET_INBAND_FEC(1));
opus_encoder_ctl(encoder, OPUS_SET_PACKET_LOSS_PERC(20));
capture_initialized = 1;
capture_initializing = 0;
return 0;
}
/**
* Read HDMI audio, encode to Opus (OUTPUT path hot function)
* @param opus_buf Output buffer for encoded Opus packet
* @return >0 = Opus packet size in bytes, -1 = error
*/
__attribute__((hot)) int jetkvm_audio_read_encode(void * __restrict__ opus_buf) {
static short CACHE_ALIGN pcm_buffer[960 * 2]; // Cache-aligned
unsigned char * __restrict__ out = (unsigned char*)opus_buf;
int32_t pcm_rc, nb_bytes;
int32_t err = 0;
uint8_t recovery_attempts = 0;
const uint8_t max_recovery_attempts = 3;
// Prefetch for write (out) and read (pcm_buffer) - RV1106 has small L1 cache
SIMD_PREFETCH(out, 1, 0); // Write, immediate use
SIMD_PREFETCH(pcm_buffer, 0, 0); // Read, immediate use
SIMD_PREFETCH(pcm_buffer + 64, 0, 1); // Prefetch next cache line
if (__builtin_expect(!capture_initialized || !pcm_capture_handle || !encoder || !opus_buf, 0)) {
TRACE_LOG("[AUDIO_OUTPUT] jetkvm_audio_read_encode: Failed safety checks - capture_initialized=%d, pcm_capture_handle=%p, encoder=%p, opus_buf=%p\n",
capture_initialized, pcm_capture_handle, encoder, opus_buf);
return -1;
}
retry_read:
// Read 960 frames (20ms) from ALSA capture device
pcm_rc = snd_pcm_readi(pcm_capture_handle, pcm_buffer, frame_size);
if (__builtin_expect(pcm_rc < 0, 0)) {
if (pcm_rc == -EPIPE) {
recovery_attempts++;
if (recovery_attempts > max_recovery_attempts) {
return -1;
}
err = snd_pcm_prepare(pcm_capture_handle);
if (err < 0) {
snd_pcm_drop(pcm_capture_handle);
err = snd_pcm_prepare(pcm_capture_handle);
if (err < 0) return -1;
}
goto retry_read;
} else if (pcm_rc == -EAGAIN) {
// Wait for data to be available
snd_pcm_wait(pcm_capture_handle, sleep_milliseconds);
goto retry_read;
} else if (pcm_rc == -ESTRPIPE) {
recovery_attempts++;
if (recovery_attempts > max_recovery_attempts) {
return -1;
}
uint8_t resume_attempts = 0;
while ((err = snd_pcm_resume(pcm_capture_handle)) == -EAGAIN && resume_attempts < 10) {
snd_pcm_wait(pcm_capture_handle, sleep_milliseconds);
resume_attempts++;
}
if (err < 0) {
err = snd_pcm_prepare(pcm_capture_handle);
if (err < 0) return -1;
}
return 0;
} else if (pcm_rc == -ENODEV) {
return -1;
} else if (pcm_rc == -EIO) {
recovery_attempts++;
if (recovery_attempts <= max_recovery_attempts) {
snd_pcm_drop(pcm_capture_handle);
err = snd_pcm_prepare(pcm_capture_handle);
if (err >= 0) {
goto retry_read;
}
}
return -1;
} else {
recovery_attempts++;
if (recovery_attempts <= 1 && pcm_rc == -EINTR) {
goto retry_read;
} else if (recovery_attempts <= 1 && pcm_rc == -EBUSY) {
snd_pcm_wait(pcm_capture_handle, 1); // Wait 1ms for device
goto retry_read;
}
return -1;
}
}
// Zero-pad if we got a short read
if (__builtin_expect(pcm_rc < frame_size, 0)) {
uint32_t remaining_samples = (frame_size - pcm_rc) * channels;
simd_clear_samples_s16(&pcm_buffer[pcm_rc * channels], remaining_samples);
}
nb_bytes = opus_encode(encoder, pcm_buffer, frame_size, out, max_packet_size);
return nb_bytes;
}
// AUDIO INPUT PATH FUNCTIONS (Client Microphone → Device Speakers)
/**
* Initialize INPUT path (Opus decoder device speakers)
* Opens hw:1,0 (USB gadget) or "default" and creates Opus decoder
* @return 0 on success, -EBUSY if initializing, -1/-2 on errors
*/
int jetkvm_audio_playback_init() {
int err;
init_alsa_devices_from_env();
if (__sync_bool_compare_and_swap(&playback_initializing, 0, 1) == 0) {
return -EBUSY;
}
if (playback_initialized) {
playback_initializing = 0;
return 0;
}
if (decoder) {
opus_decoder_destroy(decoder);
decoder = NULL;
}
if (pcm_playback_handle) {
snd_pcm_close(pcm_playback_handle);
pcm_playback_handle = NULL;
}
err = safe_alsa_open(&pcm_playback_handle, alsa_playback_device, SND_PCM_STREAM_PLAYBACK);
if (err < 0) {
fprintf(stderr, "Failed to open ALSA playback device %s: %s\n",
alsa_playback_device, snd_strerror(err));
fflush(stderr);
err = safe_alsa_open(&pcm_playback_handle, "default", SND_PCM_STREAM_PLAYBACK);
if (err < 0) {
playback_initializing = 0;
return -1;
}
}
err = configure_alsa_device(pcm_playback_handle, "playback");
if (err < 0) {
snd_pcm_close(pcm_playback_handle);
pcm_playback_handle = NULL;
playback_initializing = 0;
return -1;
}
int opus_err = 0;
decoder = opus_decoder_create(sample_rate, channels, &opus_err);
if (!decoder || opus_err != OPUS_OK) {
snd_pcm_close(pcm_playback_handle);
pcm_playback_handle = NULL;
playback_initializing = 0;
return -2;
}
playback_initialized = 1;
playback_initializing = 0;
return 0;
}
/**
* Decode Opus, write to device speakers (INPUT path hot function)
* Processing pipeline: Opus decode (with FEC) ALSA playback with error recovery
* @param opus_buf Encoded Opus packet from client
* @param opus_size Size of Opus packet in bytes
* @return >0 = PCM frames written, 0 = frame skipped, -1/-2 = error
*/
__attribute__((hot)) int jetkvm_audio_decode_write(void * __restrict__ opus_buf, int32_t opus_size) {
static short CACHE_ALIGN pcm_buffer[960 * 2]; // Cache-aligned
unsigned char * __restrict__ in = (unsigned char*)opus_buf;
int32_t pcm_frames, pcm_rc, err = 0;
uint8_t recovery_attempts = 0;
const uint8_t max_recovery_attempts = 3;
// Prefetch input buffer - locality 0 for immediate use
SIMD_PREFETCH(in, 0, 0);
if (__builtin_expect(!playback_initialized || !pcm_playback_handle || !decoder || !opus_buf || opus_size <= 0, 0)) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Failed safety checks - playback_initialized=%d, pcm_playback_handle=%p, decoder=%p, opus_buf=%p, opus_size=%d\n",
playback_initialized, pcm_playback_handle, decoder, opus_buf, opus_size);
return -1;
}
if (opus_size > max_packet_size) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Opus packet too large - size=%d, max=%d\n", opus_size, max_packet_size);
return -1;
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Processing Opus packet - size=%d bytes\n", opus_size);
// Decode Opus packet to PCM (FEC automatically applied if embedded in packet)
// decode_fec=0 means normal decode (FEC data is used automatically when present)
pcm_frames = opus_decode(decoder, in, opus_size, pcm_buffer, frame_size, 0);
if (__builtin_expect(pcm_frames < 0, 0)) {
// Decode failed - attempt packet loss concealment using FEC from previous packet
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Opus decode failed with error %d, attempting packet loss concealment\n", pcm_frames);
// decode_fec=1 means use FEC data from the NEXT packet to reconstruct THIS lost packet
pcm_frames = opus_decode(decoder, NULL, 0, pcm_buffer, frame_size, 1);
if (pcm_frames < 0) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Packet loss concealment also failed with error %d\n", pcm_frames);
return -1;
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Packet loss concealment succeeded, recovered %d frames\n", pcm_frames);
} else
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Opus decode successful - decoded %d PCM frames\n", pcm_frames);
retry_write:
// Write decoded PCM to ALSA playback device
pcm_rc = snd_pcm_writei(pcm_playback_handle, pcm_buffer, pcm_frames);
if (__builtin_expect(pcm_rc < 0, 0)) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: ALSA write failed with error %d (%s), attempt %d/%d\n",
pcm_rc, snd_strerror(pcm_rc), recovery_attempts + 1, max_recovery_attempts);
if (pcm_rc == -EPIPE) {
recovery_attempts++;
if (recovery_attempts > max_recovery_attempts) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Buffer underrun recovery failed after %d attempts\n", max_recovery_attempts);
return -2;
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Buffer underrun detected, attempting recovery (attempt %d)\n", recovery_attempts);
err = snd_pcm_prepare(pcm_playback_handle);
if (err < 0) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: snd_pcm_prepare failed (%s), trying drop+prepare\n", snd_strerror(err));
snd_pcm_drop(pcm_playback_handle);
err = snd_pcm_prepare(pcm_playback_handle);
if (err < 0) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: drop+prepare recovery failed (%s)\n", snd_strerror(err));
return -2;
}
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Buffer underrun recovery successful, retrying write\n");
goto retry_write;
} else if (pcm_rc == -ESTRPIPE) {
recovery_attempts++;
if (recovery_attempts > max_recovery_attempts) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Device suspend recovery failed after %d attempts\n", max_recovery_attempts);
return -2;
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Device suspended, attempting resume (attempt %d)\n", recovery_attempts);
uint8_t resume_attempts = 0;
while ((err = snd_pcm_resume(pcm_playback_handle)) == -EAGAIN && resume_attempts < 10) {
snd_pcm_wait(pcm_playback_handle, sleep_milliseconds);
resume_attempts++;
}
if (err < 0) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Device resume failed (%s), trying prepare fallback\n", snd_strerror(err));
err = snd_pcm_prepare(pcm_playback_handle);
if (err < 0) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Prepare fallback failed (%s)\n", snd_strerror(err));
return -2;
}
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Device suspend recovery successful, skipping frame\n");
return 0;
} else if (pcm_rc == -ENODEV) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Device disconnected (ENODEV) - critical error\n");
return -2;
} else if (pcm_rc == -EIO) {
recovery_attempts++;
if (recovery_attempts <= max_recovery_attempts) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: I/O error detected, attempting recovery\n");
snd_pcm_drop(pcm_playback_handle);
err = snd_pcm_prepare(pcm_playback_handle);
if (err >= 0) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: I/O error recovery successful, retrying write\n");
goto retry_write;
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: I/O error recovery failed (%s)\n", snd_strerror(err));
}
return -2;
} else if (pcm_rc == -EAGAIN) {
recovery_attempts++;
if (recovery_attempts <= max_recovery_attempts) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Device not ready (EAGAIN), waiting and retrying\n");
snd_pcm_wait(pcm_playback_handle, 1); // Wait 1ms
goto retry_write;
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Device not ready recovery failed after %d attempts\n", max_recovery_attempts);
return -2;
} else {
recovery_attempts++;
if (recovery_attempts <= 1 && (pcm_rc == -EINTR || pcm_rc == -EBUSY)) {
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Transient error %d (%s), retrying once\n", pcm_rc, snd_strerror(pcm_rc));
snd_pcm_wait(pcm_playback_handle, 1); // Wait 1ms
goto retry_write;
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Unrecoverable error %d (%s)\n", pcm_rc, snd_strerror(pcm_rc));
return -2;
}
}
TRACE_LOG("[AUDIO_INPUT] jetkvm_audio_decode_write: Successfully wrote %d PCM frames to device\n", pcm_frames);
return pcm_frames;
}
// CLEANUP FUNCTIONS
/**
* Close INPUT path (thread-safe with drain)
*/
void jetkvm_audio_playback_close() {
while (playback_initializing) {
sched_yield();
}
if (__sync_bool_compare_and_swap(&playback_initialized, 1, 0) == 0) {
return;
}
if (decoder) {
opus_decoder_destroy(decoder);
decoder = NULL;
}
if (pcm_playback_handle) {
snd_pcm_drain(pcm_playback_handle);
snd_pcm_close(pcm_playback_handle);
pcm_playback_handle = NULL;
}
}
/**
* Close OUTPUT path (thread-safe with drain)
*/
void jetkvm_audio_capture_close() {
while (capture_initializing) {
sched_yield();
}
if (__sync_bool_compare_and_swap(&capture_initialized, 1, 0) == 0) {
return;
}
if (encoder) {
opus_encoder_destroy(encoder);
encoder = NULL;
}
if (pcm_capture_handle) {
snd_pcm_drain(pcm_capture_handle);
snd_pcm_close(pcm_capture_handle);
pcm_capture_handle = NULL;
}
}

View File

@ -0,0 +1,101 @@
/*
* JetKVM Audio Common Utilities
*
* Shared functions for audio processing
*/
#include "audio_common.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <time.h>
// GLOBAL STATE FOR SIGNAL HANDLER
// Pointer to the running flag that will be set to 0 on shutdown
static volatile sig_atomic_t *g_running_ptr = NULL;
// SIGNAL HANDLERS
static void signal_handler(int signo) {
if (signo == SIGTERM || signo == SIGINT) {
printf("Audio server: Received signal %d, shutting down...\n", signo);
if (g_running_ptr != NULL) {
*g_running_ptr = 0;
}
}
}
void audio_common_setup_signal_handlers(volatile sig_atomic_t *running) {
g_running_ptr = running;
struct sigaction sa;
memset(&sa, 0, sizeof(sa));
sa.sa_handler = signal_handler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
sigaction(SIGTERM, &sa, NULL);
sigaction(SIGINT, &sa, NULL);
// Ignore SIGPIPE (write to closed socket should return error, not crash)
signal(SIGPIPE, SIG_IGN);
}
int32_t audio_common_parse_env_int(const char *name, int32_t default_value) {
const char *str = getenv(name);
if (str == NULL || str[0] == '\0') {
return default_value;
}
return (int32_t)atoi(str);
}
const char* audio_common_parse_env_string(const char *name, const char *default_value) {
const char *str = getenv(name);
if (str == NULL || str[0] == '\0') {
return default_value;
}
return str;
}
// COMMON CONFIGURATION
void audio_common_load_config(audio_config_t *config, int is_output) {
// ALSA device configuration
if (is_output) {
config->alsa_device = audio_common_parse_env_string("ALSA_CAPTURE_DEVICE", "hw:1,0");
} else {
config->alsa_device = audio_common_parse_env_string("ALSA_PLAYBACK_DEVICE", "hw:1,0");
}
// Common Opus configuration
config->opus_bitrate = audio_common_parse_env_int("OPUS_BITRATE", 128000);
config->opus_complexity = audio_common_parse_env_int("OPUS_COMPLEXITY", 2);
// Audio format
config->sample_rate = audio_common_parse_env_int("AUDIO_SAMPLE_RATE", 48000);
config->channels = audio_common_parse_env_int("AUDIO_CHANNELS", 2);
config->frame_size = audio_common_parse_env_int("AUDIO_FRAME_SIZE", 960);
// Log configuration
printf("Audio %s Server Configuration:\n", is_output ? "Output" : "Input");
printf(" ALSA Device: %s\n", config->alsa_device);
printf(" Sample Rate: %d Hz\n", config->sample_rate);
printf(" Channels: %d\n", config->channels);
printf(" Frame Size: %d samples\n", config->frame_size);
if (is_output) {
printf(" Opus Bitrate: %d bps\n", config->opus_bitrate);
printf(" Opus Complexity: %d\n", config->opus_complexity);
}
}
void audio_common_print_startup(const char *server_name) {
printf("JetKVM %s Starting...\n", server_name);
}
void audio_common_print_shutdown(const char *server_name) {
printf("Shutting down %s...\n", server_name);
}

View File

@ -0,0 +1,135 @@
/*
* JetKVM Audio Common Utilities
*
* Shared functions used by both audio input and output servers
*/
#ifndef JETKVM_AUDIO_COMMON_H
#define JETKVM_AUDIO_COMMON_H
#include <signal.h>
#include <stdint.h>
// SHARED CONSTANTS
// Audio processing parameters
#define AUDIO_MAX_PACKET_SIZE 1500 // Maximum Opus packet size
#define AUDIO_SLEEP_MICROSECONDS 1000 // Default sleep time in microseconds
#define AUDIO_MAX_ATTEMPTS 5 // Maximum retry attempts
#define AUDIO_MAX_BACKOFF_US 500000 // Maximum backoff in microseconds
// Error handling
#define AUDIO_MAX_CONSECUTIVE_ERRORS 10 // Maximum consecutive errors before giving up
// Performance monitoring
#define AUDIO_TRACE_MASK 0x3FF // Log every 1024th frame (bit mask for efficiency)
// SIGNAL HANDLERS
/**
* Setup signal handlers for graceful shutdown.
* Handles SIGTERM and SIGINT by setting the running flag to 0.
* Ignores SIGPIPE to prevent crashes on broken pipe writes.
*
* @param running Pointer to the volatile running flag to set on shutdown
*/
void audio_common_setup_signal_handlers(volatile sig_atomic_t *running);
/**
* Parse integer from environment variable.
* Returns default_value if variable is not set or empty.
*
* @param name Environment variable name
* @param default_value Default value if not set
* @return Parsed integer value or default
*/
int32_t audio_common_parse_env_int(const char *name, int32_t default_value);
/**
* Parse string from environment variable.
* Returns default_value if variable is not set or empty.
*
* @param name Environment variable name
* @param default_value Default value if not set
* @return Environment variable value or default (not duplicated)
*/
const char* audio_common_parse_env_string(const char *name, const char *default_value);
// COMMON CONFIGURATION
/**
* Common audio configuration structure
*/
typedef struct {
const char *alsa_device; // ALSA device path
int opus_bitrate; // Opus bitrate
int opus_complexity; // Opus complexity
int sample_rate; // Sample rate
int channels; // Number of channels
int frame_size; // Frame size in samples
} audio_config_t;
/**
* Load common audio configuration from environment
* @param config Output configuration
* @param is_output true for output server, false for input
*/
void audio_common_load_config(audio_config_t *config, int is_output);
/**
* Print server startup message
* @param server_name Name of the server (e.g., "Audio Output Server")
*/
void audio_common_print_startup(const char *server_name);
/**
* Print server shutdown message
* @param server_name Name of the server
*/
void audio_common_print_shutdown(const char *server_name);
// ERROR TRACKING
/**
* Error tracking state for audio processing loops
*/
typedef struct {
uint8_t consecutive_errors; // Current consecutive error count
uint32_t frame_count; // Total frames processed
} audio_error_tracker_t;
/**
* Initialize error tracker
*/
static inline void audio_error_tracker_init(audio_error_tracker_t *tracker) {
tracker->consecutive_errors = 0;
tracker->frame_count = 0;
}
/**
* Record an error and check if we should give up
* Returns 1 if too many errors, 0 to continue
*/
static inline uint8_t audio_error_tracker_record_error(audio_error_tracker_t *tracker) {
tracker->consecutive_errors++;
return (tracker->consecutive_errors >= AUDIO_MAX_CONSECUTIVE_ERRORS) ? 1 : 0;
}
/**
* Record success and increment frame count
*/
static inline void audio_error_tracker_record_success(audio_error_tracker_t *tracker) {
tracker->consecutive_errors = 0;
tracker->frame_count++;
}
/**
* Check if we should log trace info for this frame
*/
static inline uint8_t audio_error_tracker_should_trace(audio_error_tracker_t *tracker) {
return ((tracker->frame_count & AUDIO_TRACE_MASK) == 1) ? 1 : 0;
}
#endif // JETKVM_AUDIO_COMMON_H

View File

@ -0,0 +1,212 @@
//go:build linux && (arm || arm64)
package audio
/*
#cgo CFLAGS: -O3 -ffast-math -I/opt/jetkvm-audio-libs/alsa-lib-1.2.14/include -I/opt/jetkvm-audio-libs/opus-1.5.2/include
#cgo LDFLAGS: /opt/jetkvm-audio-libs/alsa-lib-1.2.14/src/.libs/libasound.a /opt/jetkvm-audio-libs/opus-1.5.2/.libs/libopus.a -lm -ldl -lpthread
#include <stdlib.h>
#include "c/audio.c"
*/
import "C"
import (
"fmt"
"os"
"sync"
"unsafe"
"github.com/jetkvm/kvm/internal/logging"
"github.com/rs/zerolog"
)
const (
ipcMaxFrameSize = 1024 // Max Opus frame size: 128kbps @ 20ms = ~600 bytes
)
// CgoSource implements AudioSource via direct CGO calls to C audio functions (in-process)
type CgoSource struct {
direction string // "output" or "input"
alsaDevice string
initialized bool
connected bool
mu sync.Mutex
logger zerolog.Logger
opusBuf []byte // Reusable buffer for Opus packets
}
// NewCgoOutputSource creates a new CGO audio source for output (HDMI/USB → browser)
func NewCgoOutputSource(alsaDevice string) *CgoSource {
logger := logging.GetDefaultLogger().With().Str("component", "audio-output-cgo").Logger()
return &CgoSource{
direction: "output",
alsaDevice: alsaDevice,
logger: logger,
opusBuf: make([]byte, ipcMaxFrameSize),
}
}
// NewCgoInputSource creates a new CGO audio source for input (browser → USB speakers)
func NewCgoInputSource(alsaDevice string) *CgoSource {
logger := logging.GetDefaultLogger().With().Str("component", "audio-input-cgo").Logger()
return &CgoSource{
direction: "input",
alsaDevice: alsaDevice,
logger: logger,
opusBuf: make([]byte, ipcMaxFrameSize),
}
}
// Connect initializes the C audio subsystem
func (c *CgoSource) Connect() error {
c.mu.Lock()
defer c.mu.Unlock()
if c.connected {
return nil
}
// Set ALSA device via environment for C code to read via init_alsa_devices_from_env()
if c.direction == "output" {
// Set capture device for output path via environment variable
os.Setenv("ALSA_CAPTURE_DEVICE", c.alsaDevice)
// Initialize constants
C.update_audio_constants(
C.uint(128000), // bitrate
C.uchar(5), // complexity
C.uint(48000), // sample_rate
C.uchar(2), // channels
C.ushort(960), // frame_size
C.ushort(1500), // max_packet_size
C.uint(1000), // sleep_us
C.uchar(5), // max_attempts
C.uint(500000), // max_backoff_us
)
// Initialize capture (HDMI/USB → browser)
rc := C.jetkvm_audio_capture_init()
if rc != 0 {
c.logger.Error().Int("rc", int(rc)).Msg("Failed to initialize audio capture")
return fmt.Errorf("jetkvm_audio_capture_init failed: %d", rc)
}
} else {
// Set playback device for input path via environment variable
os.Setenv("ALSA_PLAYBACK_DEVICE", c.alsaDevice)
// Initialize decoder constants
C.update_audio_decoder_constants(
C.uint(48000), // sample_rate
C.uchar(2), // channels
C.ushort(960), // frame_size
C.ushort(1500), // max_packet_size
C.uint(1000), // sleep_us
C.uchar(5), // max_attempts
C.uint(500000), // max_backoff_us
)
// Initialize playback (browser → USB speakers)
rc := C.jetkvm_audio_playback_init()
if rc != 0 {
c.logger.Error().Int("rc", int(rc)).Msg("Failed to initialize audio playback")
return fmt.Errorf("jetkvm_audio_playback_init failed: %d", rc)
}
}
c.connected = true
c.initialized = true
return nil
}
// Disconnect closes the C audio subsystem
func (c *CgoSource) Disconnect() {
c.mu.Lock()
defer c.mu.Unlock()
if !c.connected {
return
}
if c.direction == "output" {
C.jetkvm_audio_capture_close()
} else {
C.jetkvm_audio_playback_close()
}
c.connected = false
}
// IsConnected returns true if currently connected
func (c *CgoSource) IsConnected() bool {
c.mu.Lock()
defer c.mu.Unlock()
return c.connected
}
// ReadMessage reads the next audio frame from C audio subsystem
// For output path: reads HDMI/USB audio and encodes to Opus
// For input path: not used (input uses WriteMessage instead)
// Returns message type (0 = Opus), payload data, and error
func (c *CgoSource) ReadMessage() (uint8, []byte, error) {
c.mu.Lock()
defer c.mu.Unlock()
if !c.connected {
return 0, nil, fmt.Errorf("not connected")
}
if c.direction != "output" {
return 0, nil, fmt.Errorf("ReadMessage only supported for output direction")
}
// Call C function to read HDMI/USB audio and encode to Opus
// Returns Opus packet size (>0) or error (<0)
opusSize := C.jetkvm_audio_read_encode(unsafe.Pointer(&c.opusBuf[0]))
if opusSize < 0 {
return 0, nil, fmt.Errorf("jetkvm_audio_read_encode failed: %d", opusSize)
}
if opusSize == 0 {
// No data available (silence/DTX)
return 0, nil, nil
}
// Return slice of opusBuf - caller must use immediately
return ipcMsgTypeOpus, c.opusBuf[:opusSize], nil
}
// WriteMessage writes an Opus packet to the C audio subsystem for playback
// Only used for input path (browser → USB speakers)
func (c *CgoSource) WriteMessage(msgType uint8, payload []byte) error {
c.mu.Lock()
defer c.mu.Unlock()
if !c.connected {
return fmt.Errorf("not connected")
}
if c.direction != "input" {
return fmt.Errorf("WriteMessage only supported for input direction")
}
if msgType != ipcMsgTypeOpus {
// Ignore non-Opus messages
return nil
}
if len(payload) == 0 {
return nil
}
// Call C function to decode Opus and write to USB speakers
rc := C.jetkvm_audio_decode_write(unsafe.Pointer(&payload[0]), C.int(len(payload)))
if rc < 0 {
return fmt.Errorf("jetkvm_audio_decode_write failed: %d", rc)
}
return nil
}

158
internal/audio/relay.go Normal file
View File

@ -0,0 +1,158 @@
package audio
import (
"context"
"fmt"
"sync/atomic"
"time"
"github.com/jetkvm/kvm/internal/logging"
"github.com/pion/webrtc/v4"
"github.com/pion/webrtc/v4/pkg/media"
"github.com/rs/zerolog"
)
// OutputRelay forwards audio from AudioSource (CGO) to WebRTC (browser)
type OutputRelay struct {
source AudioSource
audioTrack *webrtc.TrackLocalStaticSample
ctx context.Context
cancel context.CancelFunc
logger zerolog.Logger
running atomic.Bool
sample media.Sample
stopped chan struct{}
// Stats (Uint32: overflows after 2.7 years @ 50fps, faster atomics on 32-bit ARM)
framesRelayed atomic.Uint32
framesDropped atomic.Uint32
}
// NewOutputRelay creates a relay for output audio (device → browser)
func NewOutputRelay(source AudioSource, audioTrack *webrtc.TrackLocalStaticSample) *OutputRelay {
ctx, cancel := context.WithCancel(context.Background())
logger := logging.GetDefaultLogger().With().Str("component", "audio-output-relay").Logger()
return &OutputRelay{
source: source,
audioTrack: audioTrack,
ctx: ctx,
cancel: cancel,
logger: logger,
stopped: make(chan struct{}),
sample: media.Sample{
Duration: 20 * time.Millisecond,
},
}
}
// Start begins relaying audio frames
func (r *OutputRelay) Start() error {
if r.running.Swap(true) {
return fmt.Errorf("output relay already running")
}
go r.relayLoop()
r.logger.Debug().Msg("output relay started")
return nil
}
// Stop stops the relay and waits for goroutine to exit
func (r *OutputRelay) Stop() {
if !r.running.Swap(false) {
return
}
r.cancel()
<-r.stopped
r.logger.Debug().
Uint32("frames_relayed", r.framesRelayed.Load()).
Uint32("frames_dropped", r.framesDropped.Load()).
Msg("output relay stopped")
}
// relayLoop continuously reads from audio source and writes to WebRTC
func (r *OutputRelay) relayLoop() {
defer close(r.stopped)
const reconnectDelay = 1 * time.Second
for r.running.Load() {
// Ensure connected
if !r.source.IsConnected() {
if err := r.source.Connect(); err != nil {
r.logger.Debug().Err(err).Msg("failed to connect, will retry")
time.Sleep(reconnectDelay)
continue
}
}
// Read message from audio source
msgType, payload, err := r.source.ReadMessage()
if err != nil {
// Connection error - reconnect
if r.running.Load() {
r.logger.Warn().Err(err).Msg("read error, reconnecting")
r.source.Disconnect()
time.Sleep(reconnectDelay)
}
continue
}
// Handle message
if msgType == ipcMsgTypeOpus && len(payload) > 0 {
// Reuse sample struct (zero-allocation hot path)
r.sample.Data = payload
if err := r.audioTrack.WriteSample(r.sample); err != nil {
r.framesDropped.Add(1)
r.logger.Warn().Err(err).Msg("failed to write sample to WebRTC")
} else {
r.framesRelayed.Add(1)
}
}
}
}
// InputRelay forwards audio from WebRTC (browser microphone) to AudioSource (USB audio)
type InputRelay struct {
source AudioSource
ctx context.Context
cancel context.CancelFunc
logger zerolog.Logger
running atomic.Bool
}
// NewInputRelay creates a relay for input audio (browser → device)
func NewInputRelay(source AudioSource) *InputRelay {
ctx, cancel := context.WithCancel(context.Background())
logger := logging.GetDefaultLogger().With().Str("component", "audio-input-relay").Logger()
return &InputRelay{
source: source,
ctx: ctx,
cancel: cancel,
logger: logger,
}
}
// Start begins relaying audio frames
func (r *InputRelay) Start() error {
if r.running.Swap(true) {
return fmt.Errorf("input relay already running")
}
r.logger.Debug().Msg("input relay started")
return nil
}
// Stop stops the relay
func (r *InputRelay) Stop() {
if !r.running.Swap(false) {
return
}
r.cancel()
r.logger.Debug().Msg("input relay stopped")
}

28
internal/audio/source.go Normal file
View File

@ -0,0 +1,28 @@
package audio
// IPC message types
const (
ipcMsgTypeOpus = 0 // Message type for Opus audio data
)
// AudioSource provides audio frames via CGO (in-process) C audio functions
type AudioSource interface {
// ReadMessage reads the next audio message
// Returns message type, payload data, and error
// Blocks until data is available or error occurs
// Used for output path (device → browser)
ReadMessage() (msgType uint8, payload []byte, err error)
// WriteMessage writes an audio message
// Used for input path (browser → device)
WriteMessage(msgType uint8, payload []byte) error
// IsConnected returns true if the source is connected and ready
IsConnected() bool
// Connect initializes the C audio subsystem
Connect() error
// Disconnect closes the connection and releases resources
Disconnect()
}

View File

@ -752,7 +752,6 @@ void *run_detect_format(void *arg)
while (!should_exit)
{
ensure_sleep_mode_disabled();
memset(&dv_timings, 0, sizeof(dv_timings));
if (ioctl(sub_dev_fd, VIDIOC_QUERY_DV_TIMINGS, &dv_timings) != 0)

View File

@ -8,8 +8,8 @@ import (
const sleepModeFile = "/sys/devices/platform/ff470000.i2c/i2c-4/4-000f/sleep_mode"
// DefaultEDID is the default EDID for the video stream.
const DefaultEDID = "00ffffffffffff0052620188008888881c150103800000780a0dc9a05747982712484c00000001010101010101010101010101010101023a801871382d40582c4500c48e2100001e011d007251d01e206e285500c48e2100001e000000fc00543734392d6648443732300a20000000fd00147801ff1d000a202020202020017b"
// DefaultEDID is the default EDID (identifies as "JetKVM HDMI" with full TC358743 audio/video capabilities).
const DefaultEDID = "00ffffffffffff002a8b01000100000001230104800000782ec9a05747982712484c00000000d1c081c0a9c0b3000101010101010101083a801871382d40582c450000000000001e011d007251d01e206e28550000000000001e000000fc004a65746b564d2048444d490a20000000fd00187801ff1d000a20202020202001e102032e7229097f070d07070f0707509005040302011f132220111214061507831f000068030c0010003021e2050700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000047"
var extraLockTimeout = 5 * time.Second

View File

@ -1,7 +1,9 @@
package usbgadget
import (
"context"
"fmt"
"time"
"github.com/rs/zerolog"
"github.com/sourcegraph/tf-dag/dag"
@ -114,7 +116,20 @@ func (c *ChangeSetResolver) resolveChanges(initial bool) error {
}
func (c *ChangeSetResolver) applyChanges() error {
return c.applyChangesWithTimeout(45 * time.Second)
}
func (c *ChangeSetResolver) applyChangesWithTimeout(timeout time.Duration) error {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
for _, change := range c.resolvedChanges {
select {
case <-ctx.Done():
return fmt.Errorf("USB gadget reconfiguration timed out after %v: %w", timeout, ctx.Err())
default:
}
change.ResetActionResolution()
action := change.Action()
actionStr := FileChangeResolvedActionString[action]
@ -126,7 +141,7 @@ func (c *ChangeSetResolver) applyChanges() error {
l.Str("action", actionStr).Str("change", change.String()).Msg("applying change")
err := c.changeset.applyChange(change)
err := c.applyChangeWithTimeout(ctx, change)
if err != nil {
if change.IgnoreErrors {
c.l.Warn().Str("change", change.String()).Err(err).Msg("ignoring error")
@ -139,6 +154,20 @@ func (c *ChangeSetResolver) applyChanges() error {
return nil
}
func (c *ChangeSetResolver) applyChangeWithTimeout(ctx context.Context, change *FileChange) error {
done := make(chan error, 1)
go func() {
done <- c.changeset.applyChange(change)
}()
select {
case err := <-done:
return err
case <-ctx.Done():
return fmt.Errorf("change application timed out for %s: %w", change.String(), ctx.Err())
}
}
func (c *ChangeSetResolver) GetChanges() ([]*FileChange, error) {
localChanges := c.changeset.Changes
changesMap := make(map[string]*FileChange)

View File

@ -59,6 +59,23 @@ var defaultGadgetConfig = map[string]gadgetConfigItem{
// mass storage
"mass_storage_base": massStorageBaseConfig,
"mass_storage_lun0": massStorageLun0Config,
// audio (UAC1 - USB Audio Class 1)
"audio": {
order: 4000,
device: "uac1.usb0",
path: []string{"functions", "uac1.usb0"},
configPath: []string{"uac1.usb0"},
attrs: gadgetAttributes{
"p_chmask": "3", // Playback: stereo (2 channels)
"p_srate": "48000", // Playback: 48kHz sample rate
"p_ssize": "2", // Playback: 16-bit (2 bytes)
"p_volume_present": "0", // Playback: no volume control
"c_chmask": "3", // Capture: stereo (2 channels)
"c_srate": "48000", // Capture: 48kHz sample rate
"c_ssize": "2", // Capture: 16-bit (2 bytes)
"c_volume_present": "0", // Capture: no volume control
},
},
}
func (u *UsbGadget) isGadgetConfigItemEnabled(itemKey string) bool {
@ -73,6 +90,8 @@ func (u *UsbGadget) isGadgetConfigItemEnabled(itemKey string) bool {
return u.enabledDevices.MassStorage
case "mass_storage_lun0":
return u.enabledDevices.MassStorage
case "audio":
return u.enabledDevices.Audio
default:
return true
}
@ -182,6 +201,9 @@ func (u *UsbGadget) Init() error {
return u.logError("unable to initialize USB stack", err)
}
// Pre-open HID files to reduce input latency
u.PreOpenHidFiles()
return nil
}
@ -191,11 +213,17 @@ func (u *UsbGadget) UpdateGadgetConfig() error {
u.loadGadgetConfig()
// Close HID files before reconfiguration to prevent "file already closed" errors
u.CloseHidFiles()
err := u.configureUsbGadget(true)
if err != nil {
return u.logError("unable to update gadget config", err)
}
// Reopen HID files after reconfiguration
u.PreOpenHidFiles()
return nil
}

View File

@ -1,10 +1,12 @@
package usbgadget
import (
"context"
"fmt"
"path"
"path/filepath"
"sort"
"time"
"github.com/rs/zerolog"
)
@ -52,22 +54,49 @@ func (u *UsbGadget) newUsbGadgetTransaction(lock bool) error {
}
func (u *UsbGadget) WithTransaction(fn func() error) error {
u.txLock.Lock()
defer u.txLock.Unlock()
return u.WithTransactionTimeout(fn, 60*time.Second)
}
err := u.newUsbGadgetTransaction(false)
if err != nil {
u.log.Error().Err(err).Msg("failed to create transaction")
return err
}
if err := fn(); err != nil {
u.log.Error().Err(err).Msg("transaction failed")
return err
}
result := u.tx.Commit()
u.tx = nil
// WithTransactionTimeout executes a USB gadget transaction with a specified timeout
// to prevent indefinite blocking during USB reconfiguration operations
func (u *UsbGadget) WithTransactionTimeout(fn func() error, timeout time.Duration) error {
// Create a context with timeout for the entire transaction
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
return result
// Channel to signal when the transaction is complete
done := make(chan error, 1)
// Execute the transaction in a goroutine
go func() {
u.txLock.Lock()
defer u.txLock.Unlock()
err := u.newUsbGadgetTransaction(false)
if err != nil {
u.log.Error().Err(err).Msg("failed to create transaction")
done <- err
return
}
if err := fn(); err != nil {
u.log.Error().Err(err).Msg("transaction failed")
done <- err
return
}
result := u.tx.Commit()
u.tx = nil
done <- result
}()
// Wait for either completion or timeout
select {
case err := <-done:
return err
case <-ctx.Done():
return fmt.Errorf("USB gadget transaction timed out after %v: %w", timeout, ctx.Err())
}
}
func (tx *UsbGadgetTransaction) addFileChange(component string, change RequestedFileChange) string {

View File

@ -19,6 +19,7 @@ type Devices struct {
RelativeMouse bool `json:"relative_mouse"`
Keyboard bool `json:"keyboard"`
MassStorage bool `json:"mass_storage"`
Audio bool `json:"audio"`
}
// Config is a struct that represents the customizations for a USB gadget.
@ -39,6 +40,7 @@ var defaultUsbGadgetDevices = Devices{
RelativeMouse: true,
Keyboard: true,
MassStorage: true,
Audio: true,
}
type KeysDownState struct {
@ -188,3 +190,63 @@ func (u *UsbGadget) Close() error {
return nil
}
// CloseHidFiles closes all open HID files
func (u *UsbGadget) CloseHidFiles() {
u.log.Debug().Msg("closing HID files")
// Close keyboard HID file
if u.keyboardHidFile != nil {
if err := u.keyboardHidFile.Close(); err != nil {
u.log.Debug().Err(err).Msg("failed to close keyboard HID file")
}
u.keyboardHidFile = nil
}
// Close absolute mouse HID file
if u.absMouseHidFile != nil {
if err := u.absMouseHidFile.Close(); err != nil {
u.log.Debug().Err(err).Msg("failed to close absolute mouse HID file")
}
u.absMouseHidFile = nil
}
// Close relative mouse HID file
if u.relMouseHidFile != nil {
if err := u.relMouseHidFile.Close(); err != nil {
u.log.Debug().Err(err).Msg("failed to close relative mouse HID file")
}
u.relMouseHidFile = nil
}
}
// PreOpenHidFiles opens all HID files to reduce input latency
func (u *UsbGadget) PreOpenHidFiles() {
// Add a small delay to allow USB gadget reconfiguration to complete
// This prevents "no such device or address" errors when trying to open HID files
time.Sleep(100 * time.Millisecond)
if u.enabledDevices.Keyboard {
if err := u.openKeyboardHidFile(); err != nil {
u.log.Debug().Err(err).Msg("failed to pre-open keyboard HID file")
}
}
if u.enabledDevices.AbsoluteMouse {
if u.absMouseHidFile == nil {
var err error
u.absMouseHidFile, err = os.OpenFile("/dev/hidg1", os.O_RDWR, 0666)
if err != nil {
u.log.Debug().Err(err).Msg("failed to pre-open absolute mouse HID file")
}
}
}
if u.enabledDevices.RelativeMouse {
if u.relMouseHidFile == nil {
var err error
u.relMouseHidFile, err = os.OpenFile("/dev/hidg2", os.O_RDWR, 0666)
if err != nil {
u.log.Debug().Err(err).Msg("failed to pre-open relative mouse HID file")
}
}
}
}

View File

@ -678,7 +678,8 @@ func rpcSetUsbConfig(usbConfig usbgadget.Config) error {
LoadConfig()
config.UsbConfig = &usbConfig
gadget.SetGadgetConfig(config.UsbConfig)
return updateUsbRelatedConfig()
wasAudioEnabled := config.UsbDevices != nil && config.UsbDevices.Audio
return updateUsbRelatedConfig(wasAudioEnabled)
}
func rpcGetWakeOnLanDevices() ([]WakeOnLanDevice, error) {
@ -890,23 +891,42 @@ func rpcGetUsbDevices() (usbgadget.Devices, error) {
return *config.UsbDevices, nil
}
func updateUsbRelatedConfig() error {
func updateUsbRelatedConfig(wasAudioEnabled bool) error {
ensureConfigLoaded()
// Stop input audio before USB reconfiguration (input uses USB)
audioMutex.Lock()
stopInputLocked()
audioMutex.Unlock()
if err := gadget.UpdateGadgetConfig(); err != nil {
return fmt.Errorf("failed to write gadget config: %w", err)
}
if err := SaveConfig(); err != nil {
return fmt.Errorf("failed to save config: %w", err)
}
// Restart audio if USB audio is enabled with active connections
if activeConnections.Load() > 0 && config.UsbDevices != nil && config.UsbDevices.Audio {
if err := startAudio(); err != nil {
logger.Warn().Err(err).Msg("Failed to restart audio after USB reconfiguration")
}
}
return nil
}
func rpcSetUsbDevices(usbDevices usbgadget.Devices) error {
wasAudioEnabled := config.UsbDevices != nil && config.UsbDevices.Audio
config.UsbDevices = &usbDevices
gadget.SetGadgetDevices(config.UsbDevices)
return updateUsbRelatedConfig()
return updateUsbRelatedConfig(wasAudioEnabled)
}
func rpcSetUsbDeviceState(device string, enabled bool) error {
wasAudioEnabled := config.UsbDevices != nil && config.UsbDevices.Audio
switch device {
case "absoluteMouse":
config.UsbDevices.AbsoluteMouse = enabled
@ -916,11 +936,29 @@ func rpcSetUsbDeviceState(device string, enabled bool) error {
config.UsbDevices.Keyboard = enabled
case "massStorage":
config.UsbDevices.MassStorage = enabled
case "audio":
config.UsbDevices.Audio = enabled
default:
return fmt.Errorf("invalid device: %s", device)
}
gadget.SetGadgetDevices(config.UsbDevices)
return updateUsbRelatedConfig()
return updateUsbRelatedConfig(wasAudioEnabled)
}
func rpcGetAudioOutputEnabled() (bool, error) {
return audioOutputEnabled.Load(), nil
}
func rpcSetAudioOutputEnabled(enabled bool) error {
return SetAudioOutputEnabled(enabled)
}
func rpcGetAudioInputEnabled() (bool, error) {
return audioInputEnabled.Load(), nil
}
func rpcSetAudioInputEnabled(enabled bool) error {
return SetAudioInputEnabled(enabled)
}
func rpcSetCloudUrl(apiUrl string, appUrl string) error {
@ -1241,6 +1279,10 @@ var rpcHandlers = map[string]RPCHandler{
"getUsbDevices": {Func: rpcGetUsbDevices},
"setUsbDevices": {Func: rpcSetUsbDevices, Params: []string{"devices"}},
"setUsbDeviceState": {Func: rpcSetUsbDeviceState, Params: []string{"device", "enabled"}},
"getAudioOutputEnabled": {Func: rpcGetAudioOutputEnabled},
"setAudioOutputEnabled": {Func: rpcSetAudioOutputEnabled, Params: []string{"enabled"}},
"getAudioInputEnabled": {Func: rpcGetAudioInputEnabled},
"setAudioInputEnabled": {Func: rpcSetAudioInputEnabled, Params: []string{"enabled"}},
"setCloudUrl": {Func: rpcSetCloudUrl, Params: []string{"apiUrl", "appUrl"}},
"getKeyboardLayout": {Func: rpcGetKeyboardLayout},
"setKeyboardLayout": {Func: rpcSetKeyboardLayout, Params: []string{"layout"}},

View File

@ -36,6 +36,7 @@ func Main() {
initDisplay()
initNative(systemVersionLocal, appVersionLocal)
initAudio()
http.DefaultClient.Timeout = 1 * time.Minute
@ -132,7 +133,10 @@ func Main() {
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
<-sigs
logger.Log().Msg("JetKVM Shutting Down")
logger.Info().Msg("JetKVM Shutting Down")
stopAudio()
//if fuseServer != nil {
// err := setMassStorageImage(" ")
// if err != nil {

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Selvsigneret",
"access_tls_updated": "TLS-indstillingerne er blevet opdateret",
"access_update_tls_settings": "Opdater TLS-indstillinger",
"action_bar_audio": "Lyd",
"action_bar_connection_stats": "Forbindelsesstatistik",
"audio_disable": "Deaktiver",
"audio_enable": "Aktiver",
"audio_input_description": "Aktiver mikrofonindgang til målet",
"audio_input_disabled": "Lydindgang deaktiveret",
"audio_input_enabled": "Lydindgang aktiveret",
"audio_input_failed_disable": "Kunne ikke deaktivere lydindgang: {error}",
"audio_input_failed_enable": "Kunne ikke aktivere lydindgang: {error}",
"audio_input_title": "Lydindgang (Mikrofon)",
"audio_output_description": "Aktiver lyd fra mål til højttalere",
"audio_output_disabled": "Lydudgang deaktiveret",
"audio_output_enabled": "Lydudgang aktiveret",
"audio_output_failed_disable": "Kunne ikke deaktivere lydudgang: {error}",
"audio_output_failed_enable": "Kunne ikke aktivere lydudgang: {error}",
"audio_output_title": "Lydudgang",
"audio_popover_title": "Lyd",
"audio_settings_description": "Konfigurer lydindgangs- og lydudgangsindstillinger for din JetKVM-enhed",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Aktiver eller deaktiver mikrofon lyd til fjerncomputeren",
"audio_settings_input_title": "Lydindgang",
"audio_settings_output_description": "Aktiver eller deaktiver lyd fra fjerncomputeren",
"audio_settings_output_source_description": "Vælg lydoptagelsesenheden (HDMI eller USB)",
"audio_settings_output_source_failed": "Kunne ikke indstille lydudgangskilde: {error}",
"audio_settings_output_source_success": "Lydudgangskilde opdateret med succes",
"audio_settings_output_source_title": "Lydudgangskilde",
"audio_settings_output_title": "Lydudgang",
"audio_settings_title": "Lyd",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Udvidelse",
"action_bar_fullscreen": "Fuldskærm",
"action_bar_settings": "Indstillinger",
@ -790,6 +818,8 @@
"usb_device_description": "USB-enheder, der skal emuleres på målcomputeren",
"usb_device_enable_absolute_mouse_description": "Aktivér absolut mus (markør)",
"usb_device_enable_absolute_mouse_title": "Aktivér absolut mus (markør)",
"usb_device_enable_audio_description": "Aktiver tovejs lyd",
"usb_device_enable_audio_title": "Aktiver USB-lyd",
"usb_device_enable_keyboard_description": "Aktivér tastatur",
"usb_device_enable_keyboard_title": "Aktivér tastatur",
"usb_device_enable_mass_storage_description": "Nogle gange skal det muligvis deaktiveres for at forhindre problemer med bestemte enheder.",
@ -799,6 +829,7 @@
"usb_device_failed_load": "Kunne ikke indlæse USB-enheder: {error}",
"usb_device_failed_set": "Kunne ikke indstille USB-enheder: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Tastatur, mus og masselagring",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Tastatur, mus, masselager og lyd",
"usb_device_keyboard_only": "Kun tastatur",
"usb_device_restore_default": "Gendan til standard",
"usb_device_title": "USB-enhed",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Selbstsigniert",
"access_tls_updated": "TLS-Einstellungen erfolgreich aktualisiert",
"access_update_tls_settings": "TLS-Einstellungen aktualisieren",
"action_bar_audio": "Audio",
"action_bar_connection_stats": "Verbindungsstatistiken",
"audio_disable": "Deaktivieren",
"audio_enable": "Aktivieren",
"audio_input_description": "Mikrofoneingang zum Ziel aktivieren",
"audio_input_disabled": "Audioeingang deaktiviert",
"audio_input_enabled": "Audioeingang aktiviert",
"audio_input_failed_disable": "Fehler beim Deaktivieren des Audioeingangs: {error}",
"audio_input_failed_enable": "Fehler beim Aktivieren des Audioeingangs: {error}",
"audio_input_title": "Audioeingang (Mikrofon)",
"audio_output_description": "Audio vom Ziel zu Lautsprechern aktivieren",
"audio_output_disabled": "Audioausgang deaktiviert",
"audio_output_enabled": "Audioausgang aktiviert",
"audio_output_failed_disable": "Fehler beim Deaktivieren des Audioausgangs: {error}",
"audio_output_failed_enable": "Fehler beim Aktivieren des Audioausgangs: {error}",
"audio_output_title": "Audioausgang",
"audio_popover_title": "Audio",
"audio_settings_description": "Konfigurieren Sie Audio-Eingangs- und Ausgangseinstellungen für Ihr JetKVM-Gerät",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Mikrofonaudio zum entfernten Computer aktivieren oder deaktivieren",
"audio_settings_input_title": "Audioeingang",
"audio_settings_output_description": "Audio vom entfernten Computer aktivieren oder deaktivieren",
"audio_settings_output_source_description": "Wählen Sie das Audioaufnahmegerät (HDMI oder USB)",
"audio_settings_output_source_failed": "Fehler beim Festlegen der Audioausgabequelle: {error}",
"audio_settings_output_source_success": "Audioausgabequelle erfolgreich aktualisiert",
"audio_settings_output_source_title": "Audioausgabequelle",
"audio_settings_output_title": "Audioausgang",
"audio_settings_title": "Audio",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Erweiterung",
"action_bar_fullscreen": "Vollbild",
"action_bar_settings": "Einstellungen",
@ -790,6 +818,8 @@
"usb_device_description": "USB-Geräte zum Emulieren auf dem Zielcomputer",
"usb_device_enable_absolute_mouse_description": "Absolute Maus (Zeiger) aktivieren",
"usb_device_enable_absolute_mouse_title": "Absolute Maus (Zeiger) aktivieren",
"usb_device_enable_audio_description": "Bidirektionales Audio aktivieren",
"usb_device_enable_audio_title": "USB-Audio aktivieren",
"usb_device_enable_keyboard_description": "Tastatur aktivieren",
"usb_device_enable_keyboard_title": "Tastatur aktivieren",
"usb_device_enable_mass_storage_description": "Manchmal muss es möglicherweise deaktiviert werden, um Probleme mit bestimmten Geräten zu vermeiden",
@ -799,6 +829,7 @@
"usb_device_failed_load": "USB-Geräte konnten nicht geladen werden: {error}",
"usb_device_failed_set": "Fehler beim Festlegen der USB-Geräte: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Tastatur, Maus und Massenspeicher",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Tastatur, Maus, Massenspeicher und Audio",
"usb_device_keyboard_only": "Nur Tastatur",
"usb_device_restore_default": "Auf Standard zurücksetzen",
"usb_device_title": "USB-Gerät",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Self-signed",
"access_tls_updated": "TLS settings updated successfully",
"access_update_tls_settings": "Update TLS Settings",
"action_bar_audio": "Audio",
"action_bar_connection_stats": "Connection Stats",
"audio_disable": "Disable",
"audio_enable": "Enable",
"audio_input_description": "Enable microphone input to target",
"audio_input_disabled": "Audio input disabled",
"audio_input_enabled": "Audio input enabled",
"audio_input_failed_disable": "Failed to disable audio input: {error}",
"audio_input_failed_enable": "Failed to enable audio input: {error}",
"audio_input_title": "Audio Input (Microphone)",
"audio_output_description": "Enable audio from target to speakers",
"audio_output_disabled": "Audio output disabled",
"audio_output_enabled": "Audio output enabled",
"audio_output_failed_disable": "Failed to disable audio output: {error}",
"audio_output_failed_enable": "Failed to enable audio output: {error}",
"audio_output_title": "Audio Output",
"audio_popover_title": "Audio",
"audio_settings_description": "Configure audio input and output settings for your JetKVM device",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Enable or disable microphone audio to the remote computer",
"audio_settings_input_title": "Audio Input",
"audio_settings_output_description": "Enable or disable audio from the remote computer",
"audio_settings_output_source_description": "Select the audio capture device (HDMI or USB)",
"audio_settings_output_source_failed": "Failed to set audio output source: {error}",
"audio_settings_output_source_success": "Audio output source updated successfully",
"audio_settings_output_source_title": "Audio Output Source",
"audio_settings_output_title": "Audio Output",
"audio_settings_title": "Audio",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Extension",
"action_bar_fullscreen": "Fullscreen",
"action_bar_settings": "Settings",
@ -790,6 +818,8 @@
"usb_device_description": "USB devices to emulate on the target computer",
"usb_device_enable_absolute_mouse_description": "Enable Absolute Mouse (Pointer)",
"usb_device_enable_absolute_mouse_title": "Enable Absolute Mouse (Pointer)",
"usb_device_enable_audio_description": "Enable bidirectional audio",
"usb_device_enable_audio_title": "Enable USB Audio",
"usb_device_enable_keyboard_description": "Enable Keyboard",
"usb_device_enable_keyboard_title": "Enable Keyboard",
"usb_device_enable_mass_storage_description": "Sometimes it might need to be disabled to prevent issues with certain devices",
@ -799,6 +829,7 @@
"usb_device_failed_load": "Failed to load USB devices: {error}",
"usb_device_failed_set": "Failed to set USB devices: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Keyboard, Mouse and Mass Storage",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Keyboard, Mouse, Mass Storage and Audio",
"usb_device_keyboard_only": "Keyboard Only",
"usb_device_restore_default": "Restore to Default",
"usb_device_title": "USB Device",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Autofirmado",
"access_tls_updated": "La configuración de TLS se actualizó correctamente",
"access_update_tls_settings": "Actualizar la configuración de TLS",
"action_bar_audio": "Audio",
"action_bar_connection_stats": "Estadísticas de conexión",
"audio_disable": "Desactivar",
"audio_enable": "Activar",
"audio_input_description": "Habilitar entrada de micrófono al objetivo",
"audio_input_disabled": "Entrada de audio desactivada",
"audio_input_enabled": "Entrada de audio activada",
"audio_input_failed_disable": "Error al desactivar la entrada de audio: {error}",
"audio_input_failed_enable": "Error al activar la entrada de audio: {error}",
"audio_input_title": "Entrada de audio (Micrófono)",
"audio_output_description": "Habilitar audio del objetivo a los altavoces",
"audio_output_disabled": "Salida de audio desactivada",
"audio_output_enabled": "Salida de audio activada",
"audio_output_failed_disable": "Error al desactivar la salida de audio: {error}",
"audio_output_failed_enable": "Error al activar la salida de audio: {error}",
"audio_output_title": "Salida de audio",
"audio_popover_title": "Audio",
"audio_settings_description": "Configure los ajustes de entrada y salida de audio para su dispositivo JetKVM",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Habilitar o deshabilitar el audio del micrófono a la computadora remota",
"audio_settings_input_title": "Entrada de audio",
"audio_settings_output_description": "Habilitar o deshabilitar el audio de la computadora remota",
"audio_settings_output_source_description": "Seleccione el dispositivo de captura de audio (HDMI o USB)",
"audio_settings_output_source_failed": "Error al configurar la fuente de salida de audio: {error}",
"audio_settings_output_source_success": "Fuente de salida de audio actualizada correctamente",
"audio_settings_output_source_title": "Fuente de salida de audio",
"audio_settings_output_title": "Salida de audio",
"audio_settings_title": "Audio",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Extensión",
"action_bar_fullscreen": "Pantalla completa",
"action_bar_settings": "Ajustes",
@ -790,6 +818,8 @@
"usb_device_description": "Dispositivos USB para emular en la computadora de destino",
"usb_device_enable_absolute_mouse_description": "Habilitar el puntero absoluto del ratón",
"usb_device_enable_absolute_mouse_title": "Habilitar el puntero absoluto del ratón",
"usb_device_enable_audio_description": "Habilitar audio bidireccional",
"usb_device_enable_audio_title": "Habilitar audio USB",
"usb_device_enable_keyboard_description": "Habilitar el teclado",
"usb_device_enable_keyboard_title": "Habilitar el teclado",
"usb_device_enable_mass_storage_description": "A veces puede ser necesario desactivarlo para evitar problemas con ciertos dispositivos.",
@ -799,6 +829,7 @@
"usb_device_failed_load": "No se pudieron cargar los dispositivos USB: {error}",
"usb_device_failed_set": "No se pudieron configurar los dispositivos USB: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Teclado, ratón y almacenamiento masivo",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Teclado, ratón, almacenamiento masivo y audio",
"usb_device_keyboard_only": "Sólo teclado",
"usb_device_restore_default": "Restaurar a valores predeterminados",
"usb_device_title": "Dispositivo USB",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Auto-signé",
"access_tls_updated": "Les paramètres TLS ont été mis à jour avec succès",
"access_update_tls_settings": "Mettre à jour les paramètres TLS",
"action_bar_audio": "Audio",
"action_bar_connection_stats": "Statistiques de connexion",
"audio_disable": "Désactiver",
"audio_enable": "Activer",
"audio_input_description": "Activer l'entrée microphone vers la cible",
"audio_input_disabled": "Entrée audio désactivée",
"audio_input_enabled": "Entrée audio activée",
"audio_input_failed_disable": "Échec de la désactivation de l'entrée audio : {error}",
"audio_input_failed_enable": "Échec de l'activation de l'entrée audio : {error}",
"audio_input_title": "Entrée audio (Microphone)",
"audio_output_description": "Activer l'audio de la cible vers les haut-parleurs",
"audio_output_disabled": "Sortie audio désactivée",
"audio_output_enabled": "Sortie audio activée",
"audio_output_failed_disable": "Échec de la désactivation de la sortie audio : {error}",
"audio_output_failed_enable": "Échec de l'activation de la sortie audio : {error}",
"audio_output_title": "Sortie audio",
"audio_popover_title": "Audio",
"audio_settings_description": "Configurez les paramètres d'entrée et de sortie audio pour votre appareil JetKVM",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Activer ou désactiver l'audio du microphone vers l'ordinateur distant",
"audio_settings_input_title": "Entrée audio",
"audio_settings_output_description": "Activer ou désactiver l'audio de l'ordinateur distant",
"audio_settings_output_source_description": "Sélectionnez le périphérique de capture audio (HDMI ou USB)",
"audio_settings_output_source_failed": "Échec de la configuration de la source de sortie audio : {error}",
"audio_settings_output_source_success": "Source de sortie audio mise à jour avec succès",
"audio_settings_output_source_title": "Source de sortie audio",
"audio_settings_output_title": "Sortie audio",
"audio_settings_title": "Audio",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Extension",
"action_bar_fullscreen": "Plein écran",
"action_bar_settings": "Paramètres",
@ -790,6 +818,8 @@
"usb_device_description": "Périphériques USB à émuler sur l'ordinateur cible",
"usb_device_enable_absolute_mouse_description": "Activer la souris absolue (pointeur)",
"usb_device_enable_absolute_mouse_title": "Activer la souris absolue (pointeur)",
"usb_device_enable_audio_description": "Activer l'audio bidirectionnel",
"usb_device_enable_audio_title": "Activer l'audio USB",
"usb_device_enable_keyboard_description": "Activer le clavier",
"usb_device_enable_keyboard_title": "Activer le clavier",
"usb_device_enable_mass_storage_description": "Parfois, il peut être nécessaire de le désactiver pour éviter des problèmes avec certains appareils",
@ -799,6 +829,7 @@
"usb_device_failed_load": "Échec du chargement des périphériques USB : {error}",
"usb_device_failed_set": "Échec de la configuration des périphériques USB : {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Clavier, souris et stockage de masse",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Clavier, souris, stockage de masse et audio",
"usb_device_keyboard_only": "Clavier uniquement",
"usb_device_restore_default": "Restaurer les paramètres par défaut",
"usb_device_title": "périphérique USB",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Autofirmato",
"access_tls_updated": "Impostazioni TLS aggiornate correttamente",
"access_update_tls_settings": "Aggiorna le impostazioni TLS",
"action_bar_audio": "Audio",
"action_bar_connection_stats": "Statistiche di connessione",
"audio_disable": "Disabilita",
"audio_enable": "Abilita",
"audio_input_description": "Abilita l'ingresso del microfono al target",
"audio_input_disabled": "Ingresso audio disabilitato",
"audio_input_enabled": "Ingresso audio abilitato",
"audio_input_failed_disable": "Impossibile disabilitare l'ingresso audio: {error}",
"audio_input_failed_enable": "Impossibile abilitare l'ingresso audio: {error}",
"audio_input_title": "Ingresso audio (Microfono)",
"audio_output_description": "Abilita l'audio dal target agli altoparlanti",
"audio_output_disabled": "Uscita audio disabilitata",
"audio_output_enabled": "Uscita audio abilitata",
"audio_output_failed_disable": "Impossibile disabilitare l'uscita audio: {error}",
"audio_output_failed_enable": "Impossibile abilitare l'uscita audio: {error}",
"audio_output_title": "Uscita audio",
"audio_popover_title": "Audio",
"audio_settings_description": "Configura le impostazioni di ingresso e uscita audio per il tuo dispositivo JetKVM",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Abilita o disabilita l'audio del microfono al computer remoto",
"audio_settings_input_title": "Ingresso audio",
"audio_settings_output_description": "Abilita o disabilita l'audio dal computer remoto",
"audio_settings_output_source_description": "Seleziona il dispositivo di acquisizione audio (HDMI o USB)",
"audio_settings_output_source_failed": "Impossibile impostare la sorgente di uscita audio: {error}",
"audio_settings_output_source_success": "Sorgente di uscita audio aggiornata con successo",
"audio_settings_output_source_title": "Sorgente di uscita audio",
"audio_settings_output_title": "Uscita audio",
"audio_settings_title": "Audio",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Estensione",
"action_bar_fullscreen": "A schermo intero",
"action_bar_settings": "Impostazioni",
@ -790,6 +818,8 @@
"usb_device_description": "Dispositivi USB da emulare sul computer di destinazione",
"usb_device_enable_absolute_mouse_description": "Abilita mouse assoluto (puntatore)",
"usb_device_enable_absolute_mouse_title": "Abilita mouse assoluto (puntatore)",
"usb_device_enable_audio_description": "Abilita audio bidirezionale",
"usb_device_enable_audio_title": "Abilita audio USB",
"usb_device_enable_keyboard_description": "Abilita tastiera",
"usb_device_enable_keyboard_title": "Abilita tastiera",
"usb_device_enable_mass_storage_description": "A volte potrebbe essere necessario disattivarlo per evitare problemi con determinati dispositivi",
@ -799,6 +829,7 @@
"usb_device_failed_load": "Impossibile caricare i dispositivi USB: {error}",
"usb_device_failed_set": "Impossibile impostare i dispositivi USB: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Tastiera, mouse e memoria di massa",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Tastiera, mouse, archiviazione di massa e audio",
"usb_device_keyboard_only": "Solo tastiera",
"usb_device_restore_default": "Ripristina impostazioni predefinite",
"usb_device_title": "Dispositivo USB",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Selvsignert",
"access_tls_updated": "TLS-innstillingene er oppdatert",
"access_update_tls_settings": "Oppdater TLS-innstillinger",
"action_bar_audio": "Lyd",
"action_bar_connection_stats": "Tilkoblingsstatistikk",
"audio_disable": "Deaktiver",
"audio_enable": "Aktiver",
"audio_input_description": "Aktiver mikrofoninngang til målet",
"audio_input_disabled": "Lydinngang deaktivert",
"audio_input_enabled": "Lydinngang aktivert",
"audio_input_failed_disable": "Kunne ikke deaktivere lydinngang: {error}",
"audio_input_failed_enable": "Kunne ikke aktivere lydinngang: {error}",
"audio_input_title": "Lydinngang (Mikrofon)",
"audio_output_description": "Aktiver lyd fra mål til høyttalere",
"audio_output_disabled": "Lydutgang deaktivert",
"audio_output_enabled": "Lydutgang aktivert",
"audio_output_failed_disable": "Kunne ikke deaktivere lydutgang: {error}",
"audio_output_failed_enable": "Kunne ikke aktivere lydutgang: {error}",
"audio_output_title": "Lydutgang",
"audio_popover_title": "Lyd",
"audio_settings_description": "Konfigurer lydinngangs- og lydutgangsinnstillinger for JetKVM-enheten din",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Aktiver eller deaktiver mikrofonlyd til den eksterne datamaskinen",
"audio_settings_input_title": "Lydinngang",
"audio_settings_output_description": "Aktiver eller deaktiver lyd fra den eksterne datamaskinen",
"audio_settings_output_source_description": "Velg lydopptaksenhet (HDMI eller USB)",
"audio_settings_output_source_failed": "Kunne ikke angi lydutgangskilde: {error}",
"audio_settings_output_source_success": "Lydutgangskilde oppdatert vellykket",
"audio_settings_output_source_title": "Lydutgangskilde",
"audio_settings_output_title": "Lydutgang",
"audio_settings_title": "Lyd",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Forlengelse",
"action_bar_fullscreen": "Fullskjerm",
"action_bar_settings": "Innstillinger",
@ -790,6 +818,8 @@
"usb_device_description": "USB-enheter som skal emuleres på måldatamaskinen",
"usb_device_enable_absolute_mouse_description": "Aktiver absolutt mus (peker)",
"usb_device_enable_absolute_mouse_title": "Aktiver absolutt mus (peker)",
"usb_device_enable_audio_description": "Aktiver toveis lyd",
"usb_device_enable_audio_title": "Aktiver USB-lyd",
"usb_device_enable_keyboard_description": "Aktiver tastatur",
"usb_device_enable_keyboard_title": "Aktiver tastatur",
"usb_device_enable_mass_storage_description": "Noen ganger må det kanskje deaktiveres for å forhindre problemer med visse enheter.",
@ -799,6 +829,7 @@
"usb_device_failed_load": "Klarte ikke å laste inn USB-enheter: {error}",
"usb_device_failed_set": "Kunne ikke angi USB-enheter: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Tastatur, mus og masselagring",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Tastatur, mus, masselagring og lyd",
"usb_device_keyboard_only": "Kun tastatur",
"usb_device_restore_default": "Gjenopprett til standard",
"usb_device_title": "USB-enhet",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "Självsignerad",
"access_tls_updated": "TLS-inställningarna har uppdaterats",
"access_update_tls_settings": "Uppdatera TLS-inställningar",
"action_bar_audio": "Ljud",
"action_bar_connection_stats": "Anslutningsstatistik",
"audio_disable": "Inaktivera",
"audio_enable": "Aktivera",
"audio_input_description": "Aktivera mikrofoningång till målet",
"audio_input_disabled": "Ljudingång inaktiverad",
"audio_input_enabled": "Ljudingång aktiverad",
"audio_input_failed_disable": "Det gick inte att inaktivera ljudingången: {error}",
"audio_input_failed_enable": "Det gick inte att aktivera ljudingången: {error}",
"audio_input_title": "Ljudingång (Mikrofon)",
"audio_output_description": "Aktivera ljud från mål till högtalare",
"audio_output_disabled": "Ljudutgång inaktiverad",
"audio_output_enabled": "Ljudutgång aktiverad",
"audio_output_failed_disable": "Det gick inte att inaktivera ljudutgången: {error}",
"audio_output_failed_enable": "Det gick inte att aktivera ljudutgången: {error}",
"audio_output_title": "Ljudutgång",
"audio_popover_title": "Ljud",
"audio_settings_description": "Konfigurera ljudinmatnings- och ljudutgångsinställningar för din JetKVM-enhet",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "Aktivera eller inaktivera mikrofonljud till fjärrdatorn",
"audio_settings_input_title": "Ljudingång",
"audio_settings_output_description": "Aktivera eller inaktivera ljud från fjärrdatorn",
"audio_settings_output_source_description": "Välj ljudinspelningsenhet (HDMI eller USB)",
"audio_settings_output_source_failed": "Det gick inte att ställa in ljudutgångskälla: {error}",
"audio_settings_output_source_success": "Ljudutgångskälla uppdaterades framgångsrikt",
"audio_settings_output_source_title": "Ljudutgångskälla",
"audio_settings_output_title": "Ljudutgång",
"audio_settings_title": "Ljud",
"audio_settings_usb_label": "USB",
"action_bar_extension": "Förlängning",
"action_bar_fullscreen": "Helskärm",
"action_bar_settings": "Inställningar",
@ -790,6 +818,8 @@
"usb_device_description": "USB-enheter att emulera på måldatorn",
"usb_device_enable_absolute_mouse_description": "Aktivera absolut mus (pekare)",
"usb_device_enable_absolute_mouse_title": "Aktivera absolut mus (pekare)",
"usb_device_enable_audio_description": "Aktivera dubbelriktad ljud",
"usb_device_enable_audio_title": "Aktivera USB-ljud",
"usb_device_enable_keyboard_description": "Aktivera tangentbord",
"usb_device_enable_keyboard_title": "Aktivera tangentbord",
"usb_device_enable_mass_storage_description": "Ibland kan det behöva inaktiveras för att förhindra problem med vissa enheter.",
@ -799,6 +829,7 @@
"usb_device_failed_load": "Misslyckades med att ladda USB-enheter: {error}",
"usb_device_failed_set": "Misslyckades med att ställa in USB-enheter: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "Tangentbord, mus och masslagring",
"usb_device_keyboard_mouse_mass_storage_and_audio": "Tangentbord, mus, masslagring och ljud",
"usb_device_keyboard_only": "Endast tangentbord",
"usb_device_restore_default": "Återställ till standard",
"usb_device_title": "USB-enhet",

View File

@ -47,7 +47,35 @@
"access_tls_self_signed": "自签名",
"access_tls_updated": "TLS 设置更新成功",
"access_update_tls_settings": "更新 TLS 设置",
"action_bar_audio": "音频",
"action_bar_connection_stats": "连接统计",
"audio_disable": "禁用",
"audio_enable": "启用",
"audio_input_description": "启用麦克风输入到目标设备",
"audio_input_disabled": "音频输入已禁用",
"audio_input_enabled": "音频输入已启用",
"audio_input_failed_disable": "禁用音频输入失败:{error}",
"audio_input_failed_enable": "启用音频输入失败:{error}",
"audio_input_title": "音频输入(麦克风)",
"audio_output_description": "启用从目标设备到扬声器的音频",
"audio_output_disabled": "音频输出已禁用",
"audio_output_enabled": "音频输出已启用",
"audio_output_failed_disable": "禁用音频输出失败:{error}",
"audio_output_failed_enable": "启用音频输出失败:{error}",
"audio_output_title": "音频输出",
"audio_popover_title": "音频",
"audio_settings_description": "配置 JetKVM 设备的音频输入和输出设置",
"audio_settings_hdmi_label": "HDMI",
"audio_settings_input_description": "启用或禁用到远程计算机的麦克风音频",
"audio_settings_input_title": "音频输入",
"audio_settings_output_description": "启用或禁用来自远程计算机的音频",
"audio_settings_output_source_description": "选择音频捕获设备HDMI 或 USB",
"audio_settings_output_source_failed": "设置音频输出源失败:{error}",
"audio_settings_output_source_success": "音频输出源更新成功",
"audio_settings_output_source_title": "音频输出源",
"audio_settings_output_title": "音频输出",
"audio_settings_title": "音频",
"audio_settings_usb_label": "USB",
"action_bar_extension": "扩展",
"action_bar_fullscreen": "全屏",
"action_bar_settings": "设置",
@ -790,6 +818,8 @@
"usb_device_description": "在目标计算机上仿真的 USB 设备",
"usb_device_enable_absolute_mouse_description": "启用绝对鼠标(指针)",
"usb_device_enable_absolute_mouse_title": "启用绝对鼠标(指针)",
"usb_device_enable_audio_description": "启用双向音频",
"usb_device_enable_audio_title": "启用 USB 音频",
"usb_device_enable_keyboard_description": "启用键盘",
"usb_device_enable_keyboard_title": "启用键盘",
"usb_device_enable_mass_storage_description": "有时可能需要禁用它以防止某些设备出现问题",
@ -799,6 +829,7 @@
"usb_device_failed_load": "无法加载 USB 设备: {error}",
"usb_device_failed_set": "无法设置 USB 设备: {error}",
"usb_device_keyboard_mouse_and_mass_storage": "键盘、鼠标和大容量存储器",
"usb_device_keyboard_mouse_mass_storage_and_audio": "键盘、鼠标、大容量存储和音频",
"usb_device_keyboard_only": "仅限键盘",
"usb_device_restore_default": "恢复默认设置",
"usb_device_title": "USB 设备",

View File

@ -1,6 +1,6 @@
import { Fragment, useCallback, useRef } from "react";
import { MdOutlineContentPasteGo } from "react-icons/md";
import { LuCable, LuHardDrive, LuMaximize, LuSettings, LuSignal } from "react-icons/lu";
import { LuCable, LuHardDrive, LuMaximize, LuSettings, LuSignal, LuVolume2 } from "react-icons/lu";
import { FaKeyboard } from "react-icons/fa6";
import { Popover, PopoverButton, PopoverPanel } from "@headlessui/react";
import { CommandLineIcon } from "@heroicons/react/20/solid";
@ -19,6 +19,7 @@ import PasteModal from "@components/popovers/PasteModal";
import WakeOnLanModal from "@components/popovers/WakeOnLan/Index";
import MountPopopover from "@components/popovers/MountPopover";
import ExtensionPopover from "@components/popovers/ExtensionPopover";
import AudioPopover from "@components/popovers/AudioPopover";
import { m } from "@localizations/messages.js";
export default function Actionbar({
@ -201,6 +202,36 @@ export default function Actionbar({
onClick={() => setVirtualKeyboardEnabled(!isVirtualKeyboardEnabled)}
/>
</div>
<Popover>
<PopoverButton as={Fragment}>
<Button
size="XS"
theme="light"
text={m.action_bar_audio()}
LeadingIcon={LuVolume2}
onClick={() => {
setDisableVideoFocusTrap(true);
}}
/>
</PopoverButton>
<PopoverPanel
anchor="bottom start"
transition
className={cx(
"z-10 flex w-[420px] flex-col overflow-visible!",
"flex origin-top flex-col transition duration-300 ease-out data-closed:translate-y-8 data-closed:opacity-0",
)}
>
{({ open }) => {
checkIfStateChanged(open);
return (
<div className="mx-auto w-full max-w-xl">
<AudioPopover />
</div>
);
}}
</PopoverPanel>
</Popover>
</div>
<div className="flex flex-wrap items-center gap-x-2 gap-y-2">

View File

@ -24,6 +24,7 @@ export interface UsbDeviceConfig {
absolute_mouse: boolean;
relative_mouse: boolean;
mass_storage: boolean;
audio: boolean;
}
const defaultUsbDeviceConfig: UsbDeviceConfig = {
@ -31,17 +32,30 @@ const defaultUsbDeviceConfig: UsbDeviceConfig = {
absolute_mouse: true,
relative_mouse: true,
mass_storage: true,
audio: true,
};
const usbPresets = [
{
label: m.usb_device_keyboard_mouse_and_mass_storage(),
label: m.usb_device_keyboard_mouse_mass_storage_and_audio(),
value: "default",
config: {
keyboard: true,
absolute_mouse: true,
relative_mouse: true,
mass_storage: true,
audio: true,
},
},
{
label: m.usb_device_keyboard_mouse_and_mass_storage(),
value: "keyboard_mouse_and_mass_storage",
config: {
keyboard: true,
absolute_mouse: true,
relative_mouse: true,
mass_storage: true,
audio: false,
},
},
{
@ -52,6 +66,7 @@ const usbPresets = [
absolute_mouse: false,
relative_mouse: false,
mass_storage: false,
audio: false,
},
},
{
@ -219,6 +234,17 @@ export function UsbDeviceSetting() {
/>
</SettingsItem>
</div>
<div className="space-y-4">
<SettingsItem
title={m.usb_device_enable_audio_title()}
description={m.usb_device_enable_audio_description()}
>
<Checkbox
checked={usbDeviceConfig.audio}
onChange={onUsbConfigItemChange("audio")}
/>
</SettingsItem>
</div>
</div>
<div className="mt-6 flex gap-x-2">
<Button

View File

@ -26,8 +26,10 @@ import { m } from "@localizations/messages.js";
export default function WebRTCVideo({ hasConnectionIssues }: { hasConnectionIssues: boolean }) {
// Video and stream related refs and states
const videoElm = useRef<HTMLVideoElement>(null);
const audioElementsRef = useRef<HTMLAudioElement[]>([]);
const { mediaStream, peerConnectionState } = useRTCStore();
const [isPlaying, setIsPlaying] = useState(false);
const [audioAutoplayBlocked, setAudioAutoplayBlocked] = useState(false);
const [isPointerLockActive, setIsPointerLockActive] = useState(false);
const [isKeyboardLockActive, setIsKeyboardLockActive] = useState(false);
@ -334,13 +336,34 @@ export default function WebRTCVideo({ hasConnectionIssues }: { hasConnectionIssu
peerConnection.addEventListener(
"track",
(e: RTCTrackEvent) => {
addStreamToVideoElm(e.streams[0]);
if (e.track.kind === "video") {
addStreamToVideoElm(e.streams[0]);
} else if (e.track.kind === "audio") {
const audioElm = document.createElement("audio");
audioElm.srcObject = e.streams[0];
audioElm.style.display = "none";
document.body.appendChild(audioElm);
audioElementsRef.current.push(audioElm);
audioElm.play().then(() => {
setAudioAutoplayBlocked(false);
}).catch(() => {
console.debug("[Audio] Autoplay blocked, will be started by user interaction");
setAudioAutoplayBlocked(true);
});
}
},
{ signal },
);
return () => {
abortController.abort();
audioElementsRef.current.forEach((audioElm) => {
audioElm.srcObject = null;
audioElm.remove();
});
audioElementsRef.current = [];
setAudioAutoplayBlocked(false);
};
},
[addStreamToVideoElm, peerConnection],
@ -454,11 +477,12 @@ export default function WebRTCVideo({ hasConnectionIssues }: { hasConnectionIssu
const hasNoAutoPlayPermissions = useMemo(() => {
if (peerConnection?.connectionState !== "connected") return false;
if (isPlaying) return false;
if (hdmiError) return false;
if (videoHeight === 0 || videoWidth === 0) return false;
return true;
}, [hdmiError, isPlaying, peerConnection?.connectionState, videoHeight, videoWidth]);
if (!isPlaying) return true;
if (audioAutoplayBlocked) return true;
return false;
}, [audioAutoplayBlocked, hdmiError, isPlaying, peerConnection?.connectionState, videoHeight, videoWidth]);
const showPointerLockBar = useMemo(() => {
if (settings.mouseMode !== "relative") return false;
@ -519,7 +543,6 @@ export default function WebRTCVideo({ hasConnectionIssues }: { hasConnectionIssu
controls={false}
onPlaying={onVideoPlaying}
onPlay={onVideoPlaying}
muted
playsInline
disablePictureInPicture
controlsList="nofullscreen"
@ -551,6 +574,11 @@ export default function WebRTCVideo({ hasConnectionIssues }: { hasConnectionIssu
show={hasNoAutoPlayPermissions}
onPlayClick={() => {
videoElm.current?.play();
audioElementsRef.current.forEach(audioElm => {
audioElm.play().then(() => {
setAudioAutoplayBlocked(false);
}).catch(() => undefined);
});
}}
/>
</div>

View File

@ -0,0 +1,127 @@
import { useCallback, useEffect, useState } from "react";
import { LuVolume2 } from "react-icons/lu";
import { JsonRpcResponse, useJsonRpc } from "@/hooks/useJsonRpc";
import { GridCard } from "@components/Card";
import { SettingsItem } from "@components/SettingsItem";
import { Button } from "@components/Button";
import notifications from "@/notifications";
import { m } from "@localizations/messages.js";
export default function AudioPopover() {
const { send } = useJsonRpc();
const [audioOutputEnabled, setAudioOutputEnabled] = useState<boolean>(true);
const [audioInputEnabled, setAudioInputEnabled] = useState<boolean>(true);
const [usbAudioEnabled, setUsbAudioEnabled] = useState<boolean>(false);
const [loading, setLoading] = useState(false);
useEffect(() => {
send("getAudioOutputEnabled", {}, (resp: JsonRpcResponse) => {
if ("error" in resp) {
console.error("Failed to load audio output enabled:", resp.error);
} else {
setAudioOutputEnabled(resp.result as boolean);
}
});
send("getAudioInputEnabled", {}, (resp: JsonRpcResponse) => {
if ("error" in resp) {
console.error("Failed to load audio input enabled:", resp.error);
} else {
setAudioInputEnabled(resp.result as boolean);
}
});
send("getUsbDevices", {}, (resp: JsonRpcResponse) => {
if ("error" in resp) {
console.error("Failed to load USB devices:", resp.error);
} else {
const usbDevices = resp.result as { audio: boolean };
setUsbAudioEnabled(usbDevices.audio || false);
}
});
}, [send]);
const handleAudioOutputEnabledToggle = useCallback(() => {
const enabled = !audioOutputEnabled;
setLoading(true);
send("setAudioOutputEnabled", { enabled }, (resp: JsonRpcResponse) => {
setLoading(false);
if ("error" in resp) {
const errorMsg = enabled
? m.audio_output_failed_enable({ error: String(resp.error.data || m.unknown_error()) })
: m.audio_output_failed_disable({ error: String(resp.error.data || m.unknown_error()) });
notifications.error(errorMsg);
} else {
setAudioOutputEnabled(enabled);
const successMsg = enabled ? m.audio_output_enabled() : m.audio_output_disabled();
notifications.success(successMsg);
}
});
}, [send, audioOutputEnabled]);
const handleAudioInputEnabledToggle = useCallback(() => {
const enabled = !audioInputEnabled;
setLoading(true);
send("setAudioInputEnabled", { enabled }, (resp: JsonRpcResponse) => {
setLoading(false);
if ("error" in resp) {
const errorMsg = enabled
? m.audio_input_failed_enable({ error: String(resp.error.data || m.unknown_error()) })
: m.audio_input_failed_disable({ error: String(resp.error.data || m.unknown_error()) });
notifications.error(errorMsg);
} else {
setAudioInputEnabled(enabled);
const successMsg = enabled ? m.audio_input_enabled() : m.audio_input_disabled();
notifications.success(successMsg);
}
});
}, [send, audioInputEnabled]);
return (
<GridCard>
<div className="space-y-4 p-4 py-3">
<div className="space-y-4">
<div className="flex items-center gap-2 text-slate-900 dark:text-slate-100">
<LuVolume2 className="h-5 w-5" />
<h3 className="font-semibold">{m.audio_popover_title()}</h3>
</div>
<div className="space-y-3">
<SettingsItem
loading={loading}
title={m.audio_output_title()}
description={m.audio_output_description()}
>
<Button
size="SM"
theme={audioOutputEnabled ? "light" : "primary"}
text={audioOutputEnabled ? m.audio_disable() : m.audio_enable()}
onClick={handleAudioOutputEnabledToggle}
/>
</SettingsItem>
{usbAudioEnabled && (
<>
<div className="h-px w-full bg-slate-800/10 dark:bg-slate-300/20" />
<SettingsItem
loading={loading}
title={m.audio_input_title()}
description={m.audio_input_description()}
>
<Button
size="SM"
theme={audioInputEnabled ? "light" : "primary"}
text={audioInputEnabled ? m.audio_disable() : m.audio_enable()}
onClick={handleAudioInputEnabledToggle}
/>
</SettingsItem>
</>
)}
</div>
</div>
</div>
</GridCard>
);
}

View File

@ -368,6 +368,12 @@ export interface SettingsState {
videoContrast: number;
setVideoContrast: (value: number) => void;
// Audio settings
audioOutputEnabled: boolean;
setAudioOutputEnabled: (enabled: boolean) => void;
audioInputEnabled: boolean;
setAudioInputEnabled: (enabled: boolean) => void;
}
export const useSettingsStore = create(
@ -415,6 +421,12 @@ export const useSettingsStore = create(
videoContrast: 1.0,
setVideoContrast: (value: number) => set({ videoContrast: value }),
// Audio settings with defaults
audioOutputEnabled: true,
setAudioOutputEnabled: (enabled: boolean) => set({ audioOutputEnabled: enabled }),
audioInputEnabled: true,
setAudioInputEnabled: (enabled: boolean) => set({ audioInputEnabled: enabled }),
}),
{
name: "settings",

View File

@ -41,6 +41,7 @@ const SettingsKeyboardRoute = lazy(() => import("@routes/devices.$id.settings.ke
const SettingsAdvancedRoute = lazy(() => import("@routes/devices.$id.settings.advanced"));
const SettingsHardwareRoute = lazy(() => import("@routes/devices.$id.settings.hardware"));
const SettingsVideoRoute = lazy(() => import("@routes/devices.$id.settings.video"));
const SettingsAudioRoute = lazy(() => import("@routes/devices.$id.settings.audio"));
const SettingsAppearanceRoute = lazy(() => import("@routes/devices.$id.settings.appearance"));
const SettingsGeneralIndexRoute = lazy(() => import("@routes/devices.$id.settings.general._index"));
const SettingsGeneralRebootRoute = lazy(() => import("@routes/devices.$id.settings.general.reboot"));
@ -191,6 +192,10 @@ if (isOnDevice) {
path: "video",
element: <SettingsVideoRoute />,
},
{
path: "audio",
element: <SettingsAudioRoute />,
},
{
path: "appearance",
element: <SettingsAppearanceRoute />,
@ -324,6 +329,10 @@ if (isOnDevice) {
path: "video",
element: <SettingsVideoRoute />,
},
{
path: "audio",
element: <SettingsAudioRoute />,
},
{
path: "appearance",
element: <SettingsAppearanceRoute />,

View File

@ -0,0 +1,93 @@
import { useEffect } from "react";
import { SettingsItem } from "@components/SettingsItem";
import { SettingsPageHeader } from "@components/SettingsPageheader";
import { useSettingsStore } from "@/hooks/stores";
import { JsonRpcResponse, useJsonRpc } from "@/hooks/useJsonRpc";
// import { SelectMenuBasic } from "@components/SelectMenuBasic";
import Checkbox from "@components/Checkbox";
import { m } from "@localizations/messages.js";
import notifications from "../notifications";
export default function SettingsAudioRoute() {
const { send } = useJsonRpc();
const settings = useSettingsStore();
useEffect(() => {
send("getAudioOutputEnabled", {}, (resp: JsonRpcResponse) => {
if ("error" in resp) {
return;
}
settings.setAudioOutputEnabled(resp.result as boolean);
});
send("getAudioInputEnabled", {}, (resp: JsonRpcResponse) => {
if ("error" in resp) {
return;
}
settings.setAudioInputEnabled(resp.result as boolean);
});
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [send]);
const handleAudioOutputEnabledChange = (enabled: boolean) => {
send("setAudioOutputEnabled", { enabled }, (resp: JsonRpcResponse) => {
if ("error" in resp) {
const errorMsg = enabled
? m.audio_output_failed_enable({ error: String(resp.error.data || m.unknown_error()) })
: m.audio_output_failed_disable({ error: String(resp.error.data || m.unknown_error()) });
notifications.error(errorMsg);
return;
}
settings.setAudioOutputEnabled(enabled);
const successMsg = enabled ? m.audio_output_enabled() : m.audio_output_disabled();
notifications.success(successMsg);
});
};
const handleAudioInputEnabledChange = (enabled: boolean) => {
send("setAudioInputEnabled", { enabled }, (resp: JsonRpcResponse) => {
if ("error" in resp) {
const errorMsg = enabled
? m.audio_input_failed_enable({ error: String(resp.error.data || m.unknown_error()) })
: m.audio_input_failed_disable({ error: String(resp.error.data || m.unknown_error()) });
notifications.error(errorMsg);
return;
}
settings.setAudioInputEnabled(enabled);
const successMsg = enabled ? m.audio_input_enabled() : m.audio_input_disabled();
notifications.success(successMsg);
});
};
return (
<div className="space-y-4">
<SettingsPageHeader
title={m.audio_settings_title()}
description={m.audio_settings_description()}
/>
<div className="space-y-4">
<SettingsItem
title={m.audio_settings_output_title()}
description={m.audio_settings_output_description()}
>
<Checkbox
checked={settings.audioOutputEnabled || false}
onChange={(e) => handleAudioOutputEnabledChange(e.target.checked)}
/>
</SettingsItem>
<SettingsItem
title={m.audio_settings_input_title()}
description={m.audio_settings_input_description()}
>
<Checkbox
checked={settings.audioInputEnabled || false}
onChange={(e) => handleAudioInputEnabledChange(e.target.checked)}
/>
</SettingsItem>
</div>
</div>
);
}

View File

@ -6,6 +6,7 @@ import {
LuMouse,
LuKeyboard,
LuVideo,
LuVolume2,
LuCpu,
LuShieldCheck,
LuWrench,
@ -169,6 +170,17 @@ export default function SettingsRoute() {
</div>
</NavLink>
</div>
<div className="shrink-0">
<NavLink
to="audio"
className={({ isActive }) => (isActive ? "active" : "")}
>
<div className="flex items-center gap-x-2 rounded-md px-2.5 py-2.5 text-sm transition-colors hover:bg-slate-100 dark:hover:bg-slate-700 in-[.active]:bg-blue-50 in-[.active]:text-blue-700! md:in-[.active]:bg-transparent dark:in-[.active]:bg-blue-900 dark:in-[.active]:text-blue-200! dark:md:in-[.active]:bg-transparent">
<LuVolume2 className="h-4 w-4 shrink-0" />
<h1>Audio</h1>
</div>
</NavLink>
</div>
<div className="shrink-0">
<NavLink
to="hardware"

View File

@ -12,7 +12,7 @@ import notifications from "@/notifications";
import { m } from "@localizations/messages.js";
const defaultEdid =
"00ffffffffffff0052620188008888881c150103800000780a0dc9a05747982712484c00000001010101010101010101010101010101023a801871382d40582c4500c48e2100001e011d007251d01e206e285500c48e2100001e000000fc00543734392d6648443732300a20000000fd00147801ff1d000a202020202020017b";
"00ffffffffffff002a8b01000100000001230104800000782ec9a05747982712484c00000000d1c081c0a9c0b3000101010101010101083a801871382d40582c450000000000001e011d007251d01e206e28550000000000001e000000fc004a65744b564d2048444d490a20000000fd00187801ff1d000a20202020202001e102032e7229097f070d07070f0707509005040302011f132220111214061507831f000068030c0010003021e2050700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000047";
const edids = [
{
value: defaultEdid,

View File

@ -15,7 +15,7 @@ import { motion, AnimatePresence } from "framer-motion";
import useWebSocket from "react-use-websocket";
import { cx } from "@/cva.config";
import { CLOUD_API } from "@/ui.config";
import { CLOUD_API, OPUS_STEREO_PARAMS } from "@/ui.config";
import api from "@/api";
import { checkAuth, isInCloud, isOnDevice } from "@/main";
import {
@ -172,6 +172,30 @@ export default function KvmIdRoute() {
) {
setLoadingMessage(m.setting_remote_description());
// Enable stereo in remote answer SDP
if (remoteDescription.sdp) {
const opusMatch = remoteDescription.sdp.match(/a=rtpmap:(\d+)\s+opus\/48000\/2/i);
if (!opusMatch) {
console.warn("[SDP] Opus 48kHz stereo not found in answer - stereo may not work");
} else {
const pt = opusMatch[1];
const fmtpRegex = new RegExp(`a=fmtp:${pt}\\s+(.+)`, 'i');
const fmtpMatch = remoteDescription.sdp.match(fmtpRegex);
if (fmtpMatch && !fmtpMatch[1].includes('stereo=')) {
remoteDescription.sdp = remoteDescription.sdp.replace(
fmtpRegex,
`a=fmtp:${pt} ${fmtpMatch[1]};${OPUS_STEREO_PARAMS}`
);
} else if (!fmtpMatch) {
remoteDescription.sdp = remoteDescription.sdp.replace(
opusMatch[0],
`${opusMatch[0]}\r\na=fmtp:${pt} ${OPUS_STEREO_PARAMS}`
);
}
}
}
try {
await pc.setRemoteDescription(new RTCSessionDescription(remoteDescription));
console.log("[setRemoteSessionDescription] Remote description set successfully to: " + remoteDescription.sdp);
@ -434,6 +458,29 @@ export default function KvmIdRoute() {
makingOffer.current = true;
const offer = await pc.createOffer();
// Enable stereo for Opus audio codec
if (offer.sdp) {
const opusMatch = offer.sdp.match(/a=rtpmap:(\d+)\s+opus\/48000\/2/i);
if (!opusMatch) {
console.warn("[SDP] Opus 48kHz stereo not found in offer - stereo may not work");
} else {
const pt = opusMatch[1];
const fmtpRegex = new RegExp(`a=fmtp:${pt}\\s+(.+)`, 'i');
const fmtpMatch = offer.sdp.match(fmtpRegex);
if (fmtpMatch) {
// Modify existing fmtp line
if (!fmtpMatch[1].includes('stereo=')) {
offer.sdp = offer.sdp.replace(fmtpRegex, `a=fmtp:${pt} ${fmtpMatch[1]};${OPUS_STEREO_PARAMS}`);
}
} else {
// Add new fmtp line after rtpmap
offer.sdp = offer.sdp.replace(opusMatch[0], `${opusMatch[0]}\r\na=fmtp:${pt} ${OPUS_STEREO_PARAMS}`);
}
}
}
await pc.setLocalDescription(offer);
const sd = btoa(JSON.stringify(pc.localDescription));
const isNewSignalingEnabled = isLegacySignalingEnabled.current === false;
@ -476,11 +523,35 @@ export default function KvmIdRoute() {
};
pc.ontrack = function (event) {
setMediaStream(event.streams[0]);
if (event.track.kind === "video") {
setMediaStream(event.streams[0]);
}
};
setTransceiver(pc.addTransceiver("video", { direction: "recvonly" }));
const audioTransceiver = pc.addTransceiver("audio", { direction: "sendrecv" });
if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
navigator.mediaDevices.getUserMedia({
audio: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
channelCount: 2, // Request stereo input if available
}
}).then((stream) => {
const audioTrack = stream.getAudioTracks()[0];
if (audioTrack && audioTransceiver.sender) {
audioTransceiver.sender.replaceTrack(audioTrack);
}
}).catch((err) => {
console.warn("Microphone access denied or unavailable:", err.message);
});
} else {
console.warn("navigator.mediaDevices.getUserMedia is not available in this browser/context");
}
const rpcDataChannel = pc.createDataChannel("rpc");
rpcDataChannel.onclose = () => console.log("rpcDataChannel has closed");
rpcDataChannel.onerror = (ev: Event) => console.error(`Error on DataChannel '${rpcDataChannel.label}': ${ev}`);

View File

@ -2,3 +2,6 @@ export const CLOUD_API = import.meta.env.VITE_CLOUD_API;
// In device mode, an empty string uses the current hostname (the JetKVM device's IP) as the API endpoint
export const DEVICE_API = "";
// Opus codec parameters for stereo audio with error correction
export const OPUS_STEREO_PARAMS = 'stereo=1;sprop-stereo=1;maxaveragebitrate=128000;usedtx=1;useinbandfec=1';

View File

@ -22,6 +22,7 @@ import (
type Session struct {
peerConnection *webrtc.PeerConnection
VideoTrack *webrtc.TrackLocalStaticSample
AudioTrack *webrtc.TrackLocalStaticSample
ControlChannel *webrtc.DataChannel
RPCChannel *webrtc.DataChannel
HidChannel *webrtc.DataChannel
@ -323,6 +324,39 @@ func newSession(config SessionConfig) (*Session, error) {
}
}
}()
session.AudioTrack, err = webrtc.NewTrackLocalStaticSample(
webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeOpus},
"audio",
"kvm-audio",
)
if err != nil {
scopedLogger.Warn().Err(err).Msg("Failed to create AudioTrack (non-fatal)")
} else {
_, err = peerConnection.AddTransceiverFromTrack(session.AudioTrack, webrtc.RTPTransceiverInit{
Direction: webrtc.RTPTransceiverDirectionSendrecv,
})
if err != nil {
scopedLogger.Warn().Err(err).Msg("Failed to add AudioTrack transceiver (non-fatal)")
session.AudioTrack = nil
} else {
setAudioTrack(session.AudioTrack)
peerConnection.OnTrack(func(track *webrtc.TrackRemote, receiver *webrtc.RTPReceiver) {
scopedLogger.Info().
Str("codec", track.Codec().MimeType).
Str("track_id", track.ID()).
Msg("Received incoming audio track from browser")
// Store track for connection when audio starts
// OnTrack fires during SDP exchange, before ICE connection completes
setPendingInputTrack(track)
})
scopedLogger.Info().Msg("Audio tracks configured successfully")
}
}
var isConnected bool
peerConnection.OnICECandidate(func(candidate *webrtc.ICECandidate) {
@ -353,6 +387,8 @@ func newSession(config SessionConfig) (*Session, error) {
}
if connectionState == webrtc.ICEConnectionStateClosed {
scopedLogger.Debug().Msg("ICE Connection State is closed, unmounting virtual media")
// Only clear currentSession if this is actually the current session
// This prevents race condition where old session closes after new one connects
if session == currentSession {
// Cancel any ongoing keyboard report multi when session closes
cancelKeyboardMacro()
@ -396,10 +432,12 @@ func onActiveSessionsChanged() {
func onFirstSessionConnected() {
_ = nativeInstance.VideoStart()
onWebRTCConnect()
stopVideoSleepModeTicker()
}
func onLastSessionDisconnected() {
_ = nativeInstance.VideoStop()
onWebRTCDisconnect()
startVideoSleepModeTicker()
}