Compare commits

..

27 Commits

Author SHA1 Message Date
Alex P 02acee0c75 Fix: Audio Output Enable / Disable 2025-09-09 10:39:47 +00:00
Alex P 0f2aa9abe4 feat(audio): improve socket handling and validation performance
- Add retry logic for socket file removal and listener creation
- Optimize message writing by combining header and data writes
- Move socket paths from temp dir to /var/run
- Refactor OPUS parameter lookup to use map for better readability
- Simplify validation functions for better performance in hotpaths
2025-09-09 10:16:53 +00:00
Alex P 5d4f4d8e10 UX Improvement: keep Mic state accross page refreshes 2025-09-09 09:43:39 +00:00
Alex P a5d1ef1225 refactor(audio): optimize performance and simplify code
- Replace mutex locks with atomic operations for counters
- Remove redundant logging calls to reduce overhead
- Simplify error handling and buffer validation
- Add exponential backoff for audio relay stability
- Streamline CGO audio operations for hotpath optimization
2025-09-09 09:12:05 +00:00
Alex P bda92b4a62 [Milestone] Fix: in-flight audio input quality updates 2025-09-09 08:56:24 +00:00
Alex P 3c6184d0e8 [Milestone] Improvement: In-flight audio output quality update 2025-09-09 07:44:37 +00:00
Alex P 2bc7e50391 [WIP] Cleanup, Refactor: Reduce PR complexity, common IPC layer 2025-09-09 07:08:32 +00:00
Alex P f71d18039b [WIP] Cleanup: reduce PR complexity 2025-09-09 06:59:55 +00:00
Alex P 00e5148eef [WIP] Cleanup: reduce PR complexity 2025-09-09 06:52:40 +00:00
Alex P 0ebfc762f7 [WIP] Cleanup: PR SImplification 2025-09-09 05:41:20 +00:00
Alex P 845eadec18 [WIP] Fix: Audio Latency issues: move audio to a dedicated media stream
For more details please see: https://groups.google.com/g/discuss-webrtc/c/ZvAHvkHsb0E
2025-09-09 00:23:15 +00:00
Alex P aa21b4b459 Updates: increase congestion treshold multiplier 2025-09-08 22:58:49 +00:00
Alex P 89e68f5cdb [WIP] Change playback latency spikes on Audio Output Quality changes 2025-09-08 22:55:19 +00:00
Alex P f873b50469 fix(audio): adjust congestion and CPU thresholds for single-core system
Update congestion threshold multiplier and CPU thresholds to better suit single-core ARM RV1106G3 processor characteristics. Adjust memory thresholds for systems with 200MB total memory.
2025-09-08 22:03:11 +00:00
Alex P 0893eb88ac feat(audio): improve reliability with graceful degradation and async updates
- Implement graceful degradation for congestion handling with configurable thresholds
- Refactor audio relay track updates to be async to prevent deadlocks
- Add timeout-based supervisor stop during quality changes
- Optimize buffer pool configuration and cleanup strategies
2025-09-08 21:47:39 +00:00
Alex P 8cf0b639af perf(audio): increase buffer sizes and timeouts for quality change bursts
Significantly increase message pool, channel buffer, and adaptive buffer sizes to better handle quality change bursts. Adjust timeouts and intervals for improved responsiveness.
2025-09-08 21:17:06 +00:00
Alex P 6f10010d71 refactor(audio): remove redundant config variable assignments
Replace repeated local config variable assignments with direct Config access
to reduce memory allocations and improve code maintainability
2025-09-08 21:04:07 +00:00
Alex P 1d1658db15 refactor(audio): replace GetConfig() calls with direct Config access
This change replaces all instances of GetConfig() function calls with direct access to the Config variable throughout the audio package. The modification improves performance by eliminating function call overhead and simplifies the codebase by removing unnecessary indirection.

The commit also includes minor optimizations in validation logic and connection handling, while maintaining all existing functionality. Error handling remains robust with appropriate fallbacks when config values are not available.

Additional improvements include:
- Enhanced connection health monitoring in UnifiedAudioClient
- Optimized validation functions using cached config values
- Reduced memory allocations in hot paths
- Improved error recovery during quality changes
2025-09-08 17:30:49 +00:00
Alex P 91f9dba4c6 feat(audio): improve audio quality handling and recovery mechanisms
- Add server stats reset and frame drop recovery functions
- Implement global audio server instance management
- Add WebRTC audio track replacement capability
- Improve audio relay initialization with retry logic
- Enhance quality change handling with adaptive buffer management
- Add global helper functions for audio quality control
2025-09-08 12:48:22 +00:00
Alex P 219c972e33 Merge branch 'dev' into feat/audio-support 2025-09-08 11:17:08 +00:00
Adam Shiervani c98592a412
feat(ui): Enhance EDID settings with loading state (#691)
* feat(ui): Enhance EDID settings with loading state and Fieldset component

* fix(ui): Improve notifications and adjust styling in custom EDID component

* fix(ui): specify JsonRpcResponse type
2025-09-08 11:38:49 +02:00
Alex P df58e04846 feat(audio): implement zero-copy batch processing with reference counting
Add batch reference counting and zero-copy frame management for optimized audio processing. Includes:
- BatchReferenceManager for efficient reference counting
- ZeroCopyFrameSlice utilities for frame management
- BatchZeroCopyProcessor for high-performance batch operations
- Adaptive optimization interval based on stability metrics
- Improved memory management with zero-copy frames
2025-09-08 09:08:07 +00:00
Marc Brooks 8fbad0112e
fix(ui): Don't render a button in a button (#782)
Gets rid of warning at initial page load.
2025-09-08 11:06:08 +02:00
Claus Holst 8a90555fad
Update URL Mount entries for Ubuntu, Fedora and Arch Linux (#783) 2025-09-08 11:02:46 +02:00
Adam Shiervani a7db0e8408
Enhance connection stats sidebar (#748)
* feat: add Metric component for data visualization

* refactor: update ConnectionStatsSidebar to use Metric component for improved data visualization

* feat: add someIterable utility function and update Metric components for consistent metric handling

- Introduced `someIterable` function to check for the presence of a metric in an iterable.
- Updated `CustomTooltip` and `Metric` components to use `metric` instead of `stat` for improved clarity.
- Refactored `StatChart` to align with the new metric naming convention.

* refactor: rename variable for clarity in Metric component

* docs: add JSDoc comments to createChartArray function in Metric component for better documentation

* feat: do an actual avg reference calc

* feat: Dont collect stats without a video track

* refactor: rename variables for clarity
2025-09-08 10:59:36 +02:00
Alex P 323d2587b7 refactor(audio): improve memory management with atomic operations and chunk allocation
- Replace mutex-protected refCount with atomic operations in ZeroCopyFramePool
- Implement chunk-based allocation in AudioBufferPool to reduce allocations
- Add proper reference counting with atomic operations in ZeroCopyAudioFrame
- Optimize buffer pool sizing based on buffer size
2025-09-08 08:25:42 +00:00
Alex P a6913bf33b perf(audio): make refCount operations atomic and optimize frame pooling
Replace mutex-protected refCount operations with atomic operations to improve performance in concurrent scenarios.
Simplify frame release logic and add hitCount metric for pool usage tracking.
2025-09-08 08:20:43 +00:00
55 changed files with 3229 additions and 3294 deletions

View File

@ -22,6 +22,14 @@ func initAudioControlService() {
audio.SetCurrentSessionCallback(func() audio.AudioTrackWriter { audio.SetCurrentSessionCallback(func() audio.AudioTrackWriter {
return GetCurrentSessionAudioTrack() return GetCurrentSessionAudioTrack()
}) })
// Set up callback for audio relay to replace WebRTC audio track
audio.SetTrackReplacementCallback(func(newTrack audio.AudioTrackWriter) error {
if track, ok := newTrack.(*webrtc.TrackLocalStaticSample); ok {
return ReplaceCurrentSessionAudioTrack(track)
}
return nil
})
} }
} }
@ -92,6 +100,60 @@ func ConnectRelayToCurrentSession() error {
return nil return nil
} }
// ReplaceCurrentSessionAudioTrack replaces the audio track in the current WebRTC session
func ReplaceCurrentSessionAudioTrack(newTrack *webrtc.TrackLocalStaticSample) error {
if currentSession == nil {
return nil // No session to update
}
err := currentSession.ReplaceAudioTrack(newTrack)
if err != nil {
logger.Error().Err(err).Msg("failed to replace audio track in current session")
return err
}
logger.Info().Msg("successfully replaced audio track in current session")
return nil
}
// SetAudioQuality is a global helper to set audio output quality
func SetAudioQuality(quality audio.AudioQuality) error {
initAudioControlService()
audioControlService.SetAudioQuality(quality)
return nil
}
// SetMicrophoneQuality is a global helper to set microphone quality
func SetMicrophoneQuality(quality audio.AudioQuality) error {
initAudioControlService()
audioControlService.SetMicrophoneQuality(quality)
return nil
}
// GetAudioQualityPresets is a global helper to get available audio quality presets
func GetAudioQualityPresets() map[audio.AudioQuality]audio.AudioConfig {
initAudioControlService()
return audioControlService.GetAudioQualityPresets()
}
// GetMicrophoneQualityPresets is a global helper to get available microphone quality presets
func GetMicrophoneQualityPresets() map[audio.AudioQuality]audio.AudioConfig {
initAudioControlService()
return audioControlService.GetMicrophoneQualityPresets()
}
// GetCurrentAudioQuality is a global helper to get current audio quality configuration
func GetCurrentAudioQuality() audio.AudioConfig {
initAudioControlService()
return audioControlService.GetCurrentAudioQuality()
}
// GetCurrentMicrophoneQuality is a global helper to get current microphone quality configuration
func GetCurrentMicrophoneQuality() audio.AudioConfig {
initAudioControlService()
return audioControlService.GetCurrentMicrophoneQuality()
}
// handleAudioMute handles POST /audio/mute requests // handleAudioMute handles POST /audio/mute requests
func handleAudioMute(c *gin.Context) { func handleAudioMute(c *gin.Context) {
type muteReq struct { type muteReq struct {
@ -202,10 +264,8 @@ func handleAudioStatus(c *gin.Context) {
// handleAudioQuality handles GET requests for audio quality presets // handleAudioQuality handles GET requests for audio quality presets
func handleAudioQuality(c *gin.Context) { func handleAudioQuality(c *gin.Context) {
initAudioControlService() presets := GetAudioQualityPresets()
current := GetCurrentAudioQuality()
presets := audioControlService.GetAudioQualityPresets()
current := audioControlService.GetCurrentAudioQuality()
c.JSON(200, gin.H{ c.JSON(200, gin.H{
"presets": presets, "presets": presets,
@ -224,16 +284,24 @@ func handleSetAudioQuality(c *gin.Context) {
return return
} }
initAudioControlService() // Check if audio output is active before attempting quality change
// This prevents race conditions where quality changes are attempted before initialization
if !IsAudioOutputActive() {
c.JSON(503, gin.H{"error": "audio output not active - please wait for initialization to complete"})
return
}
// Convert int to AudioQuality type // Convert int to AudioQuality type
quality := audio.AudioQuality(req.Quality) quality := audio.AudioQuality(req.Quality)
// Set the audio quality // Set the audio quality using global convenience function
audioControlService.SetAudioQuality(quality) if err := SetAudioQuality(quality); err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
// Return the updated configuration // Return the updated configuration
current := audioControlService.GetCurrentAudioQuality() current := GetCurrentAudioQuality()
c.JSON(200, gin.H{ c.JSON(200, gin.H{
"success": true, "success": true,
"config": current, "config": current,
@ -242,9 +310,9 @@ func handleSetAudioQuality(c *gin.Context) {
// handleMicrophoneQuality handles GET requests for microphone quality presets // handleMicrophoneQuality handles GET requests for microphone quality presets
func handleMicrophoneQuality(c *gin.Context) { func handleMicrophoneQuality(c *gin.Context) {
initAudioControlService() presets := GetMicrophoneQualityPresets()
presets := audioControlService.GetMicrophoneQualityPresets() current := GetCurrentMicrophoneQuality()
current := audioControlService.GetCurrentMicrophoneQuality()
c.JSON(200, gin.H{ c.JSON(200, gin.H{
"presets": presets, "presets": presets,
"current": current, "current": current,
@ -258,21 +326,22 @@ func handleSetMicrophoneQuality(c *gin.Context) {
} }
if err := c.ShouldBindJSON(&req); err != nil { if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) c.JSON(400, gin.H{"error": err.Error()})
return return
} }
initAudioControlService()
// Convert int to AudioQuality type // Convert int to AudioQuality type
quality := audio.AudioQuality(req.Quality) quality := audio.AudioQuality(req.Quality)
// Set the microphone quality // Set the microphone quality using global convenience function
audioControlService.SetMicrophoneQuality(quality) if err := SetMicrophoneQuality(quality); err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
// Return the updated configuration // Return the updated configuration
current := audioControlService.GetCurrentMicrophoneQuality() current := GetCurrentMicrophoneQuality()
c.JSON(http.StatusOK, gin.H{ c.JSON(200, gin.H{
"success": true, "success": true,
"config": current, "config": current,
}) })

View File

@ -57,25 +57,25 @@ type AdaptiveBufferConfig struct {
func DefaultAdaptiveBufferConfig() AdaptiveBufferConfig { func DefaultAdaptiveBufferConfig() AdaptiveBufferConfig {
return AdaptiveBufferConfig{ return AdaptiveBufferConfig{
// Conservative buffer sizes for 256MB RAM constraint // Conservative buffer sizes for 256MB RAM constraint
MinBufferSize: GetConfig().AdaptiveMinBufferSize, MinBufferSize: Config.AdaptiveMinBufferSize,
MaxBufferSize: GetConfig().AdaptiveMaxBufferSize, MaxBufferSize: Config.AdaptiveMaxBufferSize,
DefaultBufferSize: GetConfig().AdaptiveDefaultBufferSize, DefaultBufferSize: Config.AdaptiveDefaultBufferSize,
// CPU thresholds optimized for single-core ARM Cortex A7 under load // CPU thresholds optimized for single-core ARM Cortex A7 under load
LowCPUThreshold: GetConfig().LowCPUThreshold * 100, // Below 20% CPU LowCPUThreshold: Config.LowCPUThreshold * 100, // Below 20% CPU
HighCPUThreshold: GetConfig().HighCPUThreshold * 100, // Above 60% CPU (lowered to be more responsive) HighCPUThreshold: Config.HighCPUThreshold * 100, // Above 60% CPU (lowered to be more responsive)
// Memory thresholds for 256MB total RAM // Memory thresholds for 256MB total RAM
LowMemoryThreshold: GetConfig().LowMemoryThreshold * 100, // Below 35% memory usage LowMemoryThreshold: Config.LowMemoryThreshold * 100, // Below 35% memory usage
HighMemoryThreshold: GetConfig().HighMemoryThreshold * 100, // Above 75% memory usage (lowered for earlier response) HighMemoryThreshold: Config.HighMemoryThreshold * 100, // Above 75% memory usage (lowered for earlier response)
// Latency targets // Latency targets
TargetLatency: GetConfig().AdaptiveBufferTargetLatency, // Target 20ms latency TargetLatency: Config.AdaptiveBufferTargetLatency, // Target 20ms latency
MaxLatency: GetConfig().LatencyMonitorTarget, // Max acceptable latency MaxLatency: Config.MaxLatencyThreshold, // Max acceptable latency
// Adaptation settings // Adaptation settings
AdaptationInterval: GetConfig().BufferUpdateInterval, // Check every 500ms AdaptationInterval: Config.BufferUpdateInterval, // Check every 500ms
SmoothingFactor: GetConfig().SmoothingFactor, // Moderate responsiveness SmoothingFactor: Config.SmoothingFactor, // Moderate responsiveness
} }
} }
@ -91,7 +91,6 @@ type AdaptiveBufferManager struct {
config AdaptiveBufferConfig config AdaptiveBufferConfig
logger zerolog.Logger logger zerolog.Logger
processMonitor *ProcessMonitor
// Control channels // Control channels
ctx context.Context ctx context.Context
@ -119,7 +118,7 @@ func NewAdaptiveBufferManager(config AdaptiveBufferConfig) *AdaptiveBufferManage
currentOutputBufferSize: int64(config.DefaultBufferSize), currentOutputBufferSize: int64(config.DefaultBufferSize),
config: config, config: config,
logger: logger, logger: logger,
processMonitor: GetProcessMonitor(),
ctx: ctx, ctx: ctx,
cancel: cancel, cancel: cancel,
lastAdaptation: time.Now(), lastAdaptation: time.Now(),
@ -152,6 +151,42 @@ func (abm *AdaptiveBufferManager) GetOutputBufferSize() int {
// UpdateLatency updates the current latency measurement // UpdateLatency updates the current latency measurement
func (abm *AdaptiveBufferManager) UpdateLatency(latency time.Duration) { func (abm *AdaptiveBufferManager) UpdateLatency(latency time.Duration) {
// Use exponential moving average for latency tracking
// Weight: 90% historical, 10% current (for smoother averaging)
currentAvg := atomic.LoadInt64(&abm.averageLatency)
newLatencyNs := latency.Nanoseconds()
if currentAvg == 0 {
// First measurement
atomic.StoreInt64(&abm.averageLatency, newLatencyNs)
} else {
// Exponential moving average
newAvg := (currentAvg*9 + newLatencyNs) / 10
atomic.StoreInt64(&abm.averageLatency, newAvg)
}
// Log high latency warnings only for truly problematic latencies
// Use a more reasonable threshold: 10ms for audio processing is concerning
highLatencyThreshold := 10 * time.Millisecond
if latency > highLatencyThreshold {
abm.logger.Debug().
Dur("latency_ms", latency/time.Millisecond).
Dur("threshold_ms", highLatencyThreshold/time.Millisecond).
Msg("High audio processing latency detected")
}
}
// BoostBuffersForQualityChange immediately increases buffer sizes to handle quality change bursts
// This bypasses the normal adaptive algorithm for emergency situations
func (abm *AdaptiveBufferManager) BoostBuffersForQualityChange() {
// Immediately set buffers to maximum size to handle quality change frame bursts
maxSize := int64(abm.config.MaxBufferSize)
atomic.StoreInt64(&abm.currentInputBufferSize, maxSize)
atomic.StoreInt64(&abm.currentOutputBufferSize, maxSize)
abm.logger.Info().
Int("buffer_size", int(maxSize)).
Msg("Boosted buffers to maximum size for quality change")
} }
// adaptationLoop is the main loop that adjusts buffer sizes // adaptationLoop is the main loop that adjusts buffer sizes
@ -199,30 +234,9 @@ func (abm *AdaptiveBufferManager) adaptationLoop() {
// The algorithm runs periodically and only applies changes when the adaptation interval // The algorithm runs periodically and only applies changes when the adaptation interval
// has elapsed, preventing excessive adjustments that could destabilize the audio pipeline. // has elapsed, preventing excessive adjustments that could destabilize the audio pipeline.
func (abm *AdaptiveBufferManager) adaptBufferSizes() { func (abm *AdaptiveBufferManager) adaptBufferSizes() {
// Collect current system metrics // Use fixed system metrics for stability
metrics := abm.processMonitor.GetCurrentMetrics() systemCPU := 50.0 // Assume moderate CPU usage
if len(metrics) == 0 { systemMemory := 60.0 // Assume moderate memory usage
return // No metrics available
}
// Calculate system-wide CPU and memory usage
totalCPU := 0.0
totalMemory := 0.0
processCount := 0
for _, metric := range metrics {
totalCPU += metric.CPUPercent
totalMemory += metric.MemoryPercent
processCount++
}
if processCount == 0 {
return
}
// Store system metrics atomically
systemCPU := totalCPU // Total CPU across all monitored processes
systemMemory := totalMemory / float64(processCount) // Average memory usage
atomic.StoreInt64(&abm.systemCPUPercent, int64(systemCPU*100)) atomic.StoreInt64(&abm.systemCPUPercent, int64(systemCPU*100))
atomic.StoreInt64(&abm.systemMemoryPercent, int64(systemMemory*100)) atomic.StoreInt64(&abm.systemMemoryPercent, int64(systemMemory*100))
@ -237,7 +251,7 @@ func (abm *AdaptiveBufferManager) adaptBufferSizes() {
latencyFactor := abm.calculateLatencyFactor(currentLatency) latencyFactor := abm.calculateLatencyFactor(currentLatency)
// Combine factors with weights (CPU has highest priority for KVM coexistence) // Combine factors with weights (CPU has highest priority for KVM coexistence)
combinedFactor := GetConfig().CPUMemoryWeight*cpuFactor + GetConfig().MemoryWeight*memoryFactor + GetConfig().LatencyWeight*latencyFactor combinedFactor := Config.CPUMemoryWeight*cpuFactor + Config.MemoryWeight*memoryFactor + Config.LatencyWeight*latencyFactor
// Apply adaptation with smoothing // Apply adaptation with smoothing
currentInput := float64(atomic.LoadInt64(&abm.currentInputBufferSize)) currentInput := float64(atomic.LoadInt64(&abm.currentInputBufferSize))
@ -401,8 +415,8 @@ func (abm *AdaptiveBufferManager) GetStats() map[string]interface{} {
"input_buffer_size": abm.GetInputBufferSize(), "input_buffer_size": abm.GetInputBufferSize(),
"output_buffer_size": abm.GetOutputBufferSize(), "output_buffer_size": abm.GetOutputBufferSize(),
"average_latency_ms": float64(atomic.LoadInt64(&abm.averageLatency)) / 1e6, "average_latency_ms": float64(atomic.LoadInt64(&abm.averageLatency)) / 1e6,
"system_cpu_percent": float64(atomic.LoadInt64(&abm.systemCPUPercent)) / GetConfig().PercentageMultiplier, "system_cpu_percent": float64(atomic.LoadInt64(&abm.systemCPUPercent)) / Config.PercentageMultiplier,
"system_memory_percent": float64(atomic.LoadInt64(&abm.systemMemoryPercent)) / GetConfig().PercentageMultiplier, "system_memory_percent": float64(atomic.LoadInt64(&abm.systemMemoryPercent)) / Config.PercentageMultiplier,
"adaptation_count": atomic.LoadInt64(&abm.adaptationCount), "adaptation_count": atomic.LoadInt64(&abm.adaptationCount),
"last_adaptation": lastAdaptation, "last_adaptation": lastAdaptation,
} }

View File

@ -82,20 +82,16 @@ type batchWriteResult struct {
// NewBatchAudioProcessor creates a new batch audio processor // NewBatchAudioProcessor creates a new batch audio processor
func NewBatchAudioProcessor(batchSize int, batchDuration time.Duration) *BatchAudioProcessor { func NewBatchAudioProcessor(batchSize int, batchDuration time.Duration) *BatchAudioProcessor {
// Get cached config to avoid GetConfig() calls
cache := GetCachedConfig()
cache.Update()
// Validate input parameters with minimal overhead // Validate input parameters with minimal overhead
if batchSize <= 0 || batchSize > 1000 { if batchSize <= 0 || batchSize > 1000 {
batchSize = cache.BatchProcessorFramesPerBatch batchSize = Config.BatchProcessorFramesPerBatch
} }
if batchDuration <= 0 { if batchDuration <= 0 {
batchDuration = cache.BatchProcessingDelay batchDuration = Config.BatchProcessingDelay
} }
// Use optimized queue sizes from configuration // Use optimized queue sizes from configuration
queueSize := cache.BatchProcessorMaxQueueSize queueSize := Config.BatchProcessorMaxQueueSize
if queueSize <= 0 { if queueSize <= 0 {
queueSize = batchSize * 2 // Fallback to double batch size queueSize = batchSize * 2 // Fallback to double batch size
} }
@ -104,8 +100,7 @@ func NewBatchAudioProcessor(batchSize int, batchDuration time.Duration) *BatchAu
// Pre-allocate logger to avoid repeated allocations // Pre-allocate logger to avoid repeated allocations
logger := logging.GetDefaultLogger().With().Str("component", "batch-audio").Logger() logger := logging.GetDefaultLogger().With().Str("component", "batch-audio").Logger()
// Pre-calculate frame size to avoid repeated GetConfig() calls frameSize := Config.MinReadEncodeBuffer
frameSize := cache.GetMinReadEncodeBuffer()
if frameSize == 0 { if frameSize == 0 {
frameSize = 1500 // Safe fallback frameSize = 1500 // Safe fallback
} }
@ -120,13 +115,11 @@ func NewBatchAudioProcessor(batchSize int, batchDuration time.Duration) *BatchAu
writeQueue: make(chan batchWriteRequest, queueSize), writeQueue: make(chan batchWriteRequest, queueSize),
readBufPool: &sync.Pool{ readBufPool: &sync.Pool{
New: func() interface{} { New: func() interface{} {
// Use pre-calculated frame size to avoid GetConfig() calls
return make([]byte, 0, frameSize) return make([]byte, 0, frameSize)
}, },
}, },
writeBufPool: &sync.Pool{ writeBufPool: &sync.Pool{
New: func() interface{} { New: func() interface{} {
// Use pre-calculated frame size to avoid GetConfig() calls
return make([]byte, 0, frameSize) return make([]byte, 0, frameSize)
}, },
}, },
@ -166,17 +159,13 @@ func (bap *BatchAudioProcessor) Stop() {
bap.cancel() bap.cancel()
// Wait for processing to complete // Wait for processing to complete
time.Sleep(bap.batchDuration + GetConfig().BatchProcessingDelay) time.Sleep(bap.batchDuration + Config.BatchProcessingDelay)
bap.logger.Info().Msg("batch audio processor stopped") bap.logger.Info().Msg("batch audio processor stopped")
} }
// BatchReadEncode performs batched audio read and encode operations // BatchReadEncode performs batched audio read and encode operations
func (bap *BatchAudioProcessor) BatchReadEncode(buffer []byte) (int, error) { func (bap *BatchAudioProcessor) BatchReadEncode(buffer []byte) (int, error) {
// Get cached config to avoid GetConfig() calls in hot path
cache := GetCachedConfig()
cache.Update()
// Validate buffer before processing // Validate buffer before processing
if err := ValidateBufferSize(len(buffer)); err != nil { if err := ValidateBufferSize(len(buffer)); err != nil {
// Only log validation errors in debug mode to reduce overhead // Only log validation errors in debug mode to reduce overhead
@ -221,7 +210,7 @@ func (bap *BatchAudioProcessor) BatchReadEncode(buffer []byte) (int, error) {
select { select {
case result := <-resultChan: case result := <-resultChan:
return result.length, result.err return result.length, result.err
case <-time.After(cache.BatchProcessingTimeout): case <-time.After(Config.BatchProcessorTimeout):
// Timeout, fallback to single operation // Timeout, fallback to single operation
// Use sampling to reduce atomic operations overhead // Use sampling to reduce atomic operations overhead
if atomic.LoadInt64(&bap.stats.SingleReads)%10 == 0 { if atomic.LoadInt64(&bap.stats.SingleReads)%10 == 0 {
@ -235,10 +224,6 @@ func (bap *BatchAudioProcessor) BatchReadEncode(buffer []byte) (int, error) {
// BatchDecodeWrite performs batched audio decode and write operations // BatchDecodeWrite performs batched audio decode and write operations
// This is the legacy version that uses a single buffer // This is the legacy version that uses a single buffer
func (bap *BatchAudioProcessor) BatchDecodeWrite(buffer []byte) (int, error) { func (bap *BatchAudioProcessor) BatchDecodeWrite(buffer []byte) (int, error) {
// Get cached config to avoid GetConfig() calls in hot path
cache := GetCachedConfig()
cache.Update()
// Validate buffer before processing // Validate buffer before processing
if err := ValidateBufferSize(len(buffer)); err != nil { if err := ValidateBufferSize(len(buffer)); err != nil {
// Only log validation errors in debug mode to reduce overhead // Only log validation errors in debug mode to reduce overhead
@ -283,7 +268,7 @@ func (bap *BatchAudioProcessor) BatchDecodeWrite(buffer []byte) (int, error) {
select { select {
case result := <-resultChan: case result := <-resultChan:
return result.length, result.err return result.length, result.err
case <-time.After(cache.BatchProcessingTimeout): case <-time.After(Config.BatchProcessorTimeout):
// Use sampling to reduce atomic operations overhead // Use sampling to reduce atomic operations overhead
if atomic.LoadInt64(&bap.stats.SingleWrites)%10 == 0 { if atomic.LoadInt64(&bap.stats.SingleWrites)%10 == 0 {
atomic.AddInt64(&bap.stats.SingleWrites, 10) atomic.AddInt64(&bap.stats.SingleWrites, 10)
@ -295,10 +280,6 @@ func (bap *BatchAudioProcessor) BatchDecodeWrite(buffer []byte) (int, error) {
// BatchDecodeWriteWithBuffers performs batched audio decode and write operations with separate opus and PCM buffers // BatchDecodeWriteWithBuffers performs batched audio decode and write operations with separate opus and PCM buffers
func (bap *BatchAudioProcessor) BatchDecodeWriteWithBuffers(opusData []byte, pcmBuffer []byte) (int, error) { func (bap *BatchAudioProcessor) BatchDecodeWriteWithBuffers(opusData []byte, pcmBuffer []byte) (int, error) {
// Get cached config to avoid GetConfig() calls in hot path
cache := GetCachedConfig()
cache.Update()
// Validate buffers before processing // Validate buffers before processing
if len(opusData) == 0 { if len(opusData) == 0 {
return 0, fmt.Errorf("empty opus data buffer") return 0, fmt.Errorf("empty opus data buffer")
@ -339,7 +320,7 @@ func (bap *BatchAudioProcessor) BatchDecodeWriteWithBuffers(opusData []byte, pcm
select { select {
case result := <-resultChan: case result := <-resultChan:
return result.length, result.err return result.length, result.err
case <-time.After(cache.BatchProcessingTimeout): case <-time.After(Config.BatchProcessorTimeout):
atomic.AddInt64(&bap.stats.SingleWrites, 1) atomic.AddInt64(&bap.stats.SingleWrites, 1)
atomic.AddInt64(&bap.stats.WriteFrames, 1) atomic.AddInt64(&bap.stats.WriteFrames, 1)
// Use the optimized function with separate buffers // Use the optimized function with separate buffers
@ -426,11 +407,9 @@ func (bap *BatchAudioProcessor) processBatchRead(batch []batchReadRequest) {
return return
} }
// Get cached config once - avoid repeated calls threadPinningThreshold := Config.BatchProcessorThreadPinningThreshold
cache := GetCachedConfig()
threadPinningThreshold := cache.BatchProcessorThreadPinningThreshold
if threadPinningThreshold == 0 { if threadPinningThreshold == 0 {
threadPinningThreshold = cache.MinBatchSizeForThreadPinning // Fallback threadPinningThreshold = Config.MinBatchSizeForThreadPinning // Fallback
} }
// Only pin to OS thread for large batches to reduce thread contention // Only pin to OS thread for large batches to reduce thread contention
@ -479,11 +458,9 @@ func (bap *BatchAudioProcessor) processBatchWrite(batch []batchWriteRequest) {
return return
} }
// Get cached config to avoid GetConfig() calls in hot path threadPinningThreshold := Config.BatchProcessorThreadPinningThreshold
cache := GetCachedConfig()
threadPinningThreshold := cache.BatchProcessorThreadPinningThreshold
if threadPinningThreshold == 0 { if threadPinningThreshold == 0 {
threadPinningThreshold = cache.MinBatchSizeForThreadPinning // Fallback threadPinningThreshold = Config.MinBatchSizeForThreadPinning // Fallback
} }
// Only pin to OS thread for large batches to reduce thread contention // Only pin to OS thread for large batches to reduce thread contention
@ -585,11 +562,7 @@ func GetBatchAudioProcessor() *BatchAudioProcessor {
// Initialize on first use // Initialize on first use
if atomic.CompareAndSwapInt32(&batchProcessorInitialized, 0, 1) { if atomic.CompareAndSwapInt32(&batchProcessorInitialized, 0, 1) {
// Get cached config to avoid GetConfig() calls processor := NewBatchAudioProcessor(Config.BatchProcessorFramesPerBatch, Config.BatchProcessorTimeout)
cache := GetCachedConfig()
cache.Update()
processor := NewBatchAudioProcessor(cache.BatchProcessorFramesPerBatch, cache.BatchProcessorTimeout)
atomic.StorePointer(&globalBatchProcessor, unsafe.Pointer(processor)) atomic.StorePointer(&globalBatchProcessor, unsafe.Pointer(processor))
return processor return processor
} }
@ -601,8 +574,7 @@ func GetBatchAudioProcessor() *BatchAudioProcessor {
} }
// Fallback: create a new processor (should rarely happen) // Fallback: create a new processor (should rarely happen)
config := GetConfig() return NewBatchAudioProcessor(Config.BatchProcessorFramesPerBatch, Config.BatchProcessorTimeout)
return NewBatchAudioProcessor(config.BatchProcessorFramesPerBatch, config.BatchProcessorTimeout)
} }
// EnableBatchAudioProcessing enables the global batch processor // EnableBatchAudioProcessing enables the global batch processor

View File

@ -0,0 +1,331 @@
//go:build cgo
package audio
import (
"errors"
"sync"
"sync/atomic"
"unsafe"
)
// BatchReferenceManager handles batch reference counting operations
// to reduce atomic operation overhead for high-frequency frame operations
type BatchReferenceManager struct {
// Batch operations queue
batchQueue chan batchRefOperation
workerPool chan struct{} // Worker pool semaphore
running int32
wg sync.WaitGroup
// Statistics
batchedOps int64
singleOps int64
batchSavings int64 // Number of atomic operations saved
}
type batchRefOperation struct {
frames []*ZeroCopyAudioFrame
operation refOperationType
resultCh chan batchRefResult
}
type refOperationType int
const (
refOpAddRef refOperationType = iota
refOpRelease
refOpMixed // For operations with mixed AddRef/Release
)
// Errors
var (
ErrUnsupportedOperation = errors.New("unsupported batch reference operation")
)
type batchRefResult struct {
finalReleases []bool // For Release operations, indicates which frames had final release
err error
}
// Global batch reference manager
var (
globalBatchRefManager *BatchReferenceManager
batchRefOnce sync.Once
)
// GetBatchReferenceManager returns the global batch reference manager
func GetBatchReferenceManager() *BatchReferenceManager {
batchRefOnce.Do(func() {
globalBatchRefManager = NewBatchReferenceManager()
globalBatchRefManager.Start()
})
return globalBatchRefManager
}
// NewBatchReferenceManager creates a new batch reference manager
func NewBatchReferenceManager() *BatchReferenceManager {
return &BatchReferenceManager{
batchQueue: make(chan batchRefOperation, 256), // Buffered for high throughput
workerPool: make(chan struct{}, 4), // 4 workers for parallel processing
}
}
// Start starts the batch reference manager workers
func (brm *BatchReferenceManager) Start() {
if !atomic.CompareAndSwapInt32(&brm.running, 0, 1) {
return // Already running
}
// Start worker goroutines
for i := 0; i < cap(brm.workerPool); i++ {
brm.wg.Add(1)
go brm.worker()
}
}
// Stop stops the batch reference manager
func (brm *BatchReferenceManager) Stop() {
if !atomic.CompareAndSwapInt32(&brm.running, 1, 0) {
return // Already stopped
}
close(brm.batchQueue)
brm.wg.Wait()
}
// worker processes batch reference operations
func (brm *BatchReferenceManager) worker() {
defer brm.wg.Done()
for op := range brm.batchQueue {
brm.processBatchOperation(op)
}
}
// processBatchOperation processes a batch of reference operations
func (brm *BatchReferenceManager) processBatchOperation(op batchRefOperation) {
result := batchRefResult{}
switch op.operation {
case refOpAddRef:
// Batch AddRef operations
for _, frame := range op.frames {
if frame != nil {
atomic.AddInt32(&frame.refCount, 1)
}
}
atomic.AddInt64(&brm.batchedOps, int64(len(op.frames)))
atomic.AddInt64(&brm.batchSavings, int64(len(op.frames)-1)) // Saved ops vs individual calls
case refOpRelease:
// Batch Release operations
result.finalReleases = make([]bool, len(op.frames))
for i, frame := range op.frames {
if frame != nil {
newCount := atomic.AddInt32(&frame.refCount, -1)
if newCount == 0 {
result.finalReleases[i] = true
// Return to pool if pooled
if frame.pooled {
globalZeroCopyPool.Put(frame)
}
}
}
}
atomic.AddInt64(&brm.batchedOps, int64(len(op.frames)))
atomic.AddInt64(&brm.batchSavings, int64(len(op.frames)-1))
case refOpMixed:
// Handle mixed operations (not implemented in this version)
result.err = ErrUnsupportedOperation
}
// Send result back
if op.resultCh != nil {
op.resultCh <- result
close(op.resultCh)
}
}
// BatchAddRef performs AddRef on multiple frames in a single batch
func (brm *BatchReferenceManager) BatchAddRef(frames []*ZeroCopyAudioFrame) error {
if len(frames) == 0 {
return nil
}
// For small batches, use direct operations to avoid overhead
if len(frames) <= 2 {
for _, frame := range frames {
if frame != nil {
frame.AddRef()
}
}
atomic.AddInt64(&brm.singleOps, int64(len(frames)))
return nil
}
// Use batch processing for larger sets
if atomic.LoadInt32(&brm.running) == 0 {
// Fallback to individual operations if batch manager not running
for _, frame := range frames {
if frame != nil {
frame.AddRef()
}
}
atomic.AddInt64(&brm.singleOps, int64(len(frames)))
return nil
}
resultCh := make(chan batchRefResult, 1)
op := batchRefOperation{
frames: frames,
operation: refOpAddRef,
resultCh: resultCh,
}
select {
case brm.batchQueue <- op:
// Wait for completion
<-resultCh
return nil
default:
// Queue full, fallback to individual operations
for _, frame := range frames {
if frame != nil {
frame.AddRef()
}
}
atomic.AddInt64(&brm.singleOps, int64(len(frames)))
return nil
}
}
// BatchRelease performs Release on multiple frames in a single batch
// Returns a slice indicating which frames had their final reference released
func (brm *BatchReferenceManager) BatchRelease(frames []*ZeroCopyAudioFrame) ([]bool, error) {
if len(frames) == 0 {
return nil, nil
}
// For small batches, use direct operations
if len(frames) <= 2 {
finalReleases := make([]bool, len(frames))
for i, frame := range frames {
if frame != nil {
finalReleases[i] = frame.Release()
}
}
atomic.AddInt64(&brm.singleOps, int64(len(frames)))
return finalReleases, nil
}
// Use batch processing for larger sets
if atomic.LoadInt32(&brm.running) == 0 {
// Fallback to individual operations
finalReleases := make([]bool, len(frames))
for i, frame := range frames {
if frame != nil {
finalReleases[i] = frame.Release()
}
}
atomic.AddInt64(&brm.singleOps, int64(len(frames)))
return finalReleases, nil
}
resultCh := make(chan batchRefResult, 1)
op := batchRefOperation{
frames: frames,
operation: refOpRelease,
resultCh: resultCh,
}
select {
case brm.batchQueue <- op:
// Wait for completion
result := <-resultCh
return result.finalReleases, result.err
default:
// Queue full, fallback to individual operations
finalReleases := make([]bool, len(frames))
for i, frame := range frames {
if frame != nil {
finalReleases[i] = frame.Release()
}
}
atomic.AddInt64(&brm.singleOps, int64(len(frames)))
return finalReleases, nil
}
}
// GetStats returns batch reference counting statistics
func (brm *BatchReferenceManager) GetStats() (batchedOps, singleOps, savings int64) {
return atomic.LoadInt64(&brm.batchedOps),
atomic.LoadInt64(&brm.singleOps),
atomic.LoadInt64(&brm.batchSavings)
}
// Convenience functions for global batch reference manager
// BatchAddRefFrames performs batch AddRef on multiple frames
func BatchAddRefFrames(frames []*ZeroCopyAudioFrame) error {
return GetBatchReferenceManager().BatchAddRef(frames)
}
// BatchReleaseFrames performs batch Release on multiple frames
func BatchReleaseFrames(frames []*ZeroCopyAudioFrame) ([]bool, error) {
return GetBatchReferenceManager().BatchRelease(frames)
}
// GetBatchReferenceStats returns global batch reference statistics
func GetBatchReferenceStats() (batchedOps, singleOps, savings int64) {
return GetBatchReferenceManager().GetStats()
}
// ZeroCopyFrameSlice provides utilities for working with slices of zero-copy frames
type ZeroCopyFrameSlice []*ZeroCopyAudioFrame
// AddRefAll performs batch AddRef on all frames in the slice
func (zfs ZeroCopyFrameSlice) AddRefAll() error {
return BatchAddRefFrames(zfs)
}
// ReleaseAll performs batch Release on all frames in the slice
func (zfs ZeroCopyFrameSlice) ReleaseAll() ([]bool, error) {
return BatchReleaseFrames(zfs)
}
// FilterNonNil returns a new slice with only non-nil frames
func (zfs ZeroCopyFrameSlice) FilterNonNil() ZeroCopyFrameSlice {
filtered := make(ZeroCopyFrameSlice, 0, len(zfs))
for _, frame := range zfs {
if frame != nil {
filtered = append(filtered, frame)
}
}
return filtered
}
// Len returns the number of frames in the slice
func (zfs ZeroCopyFrameSlice) Len() int {
return len(zfs)
}
// Get returns the frame at the specified index
func (zfs ZeroCopyFrameSlice) Get(index int) *ZeroCopyAudioFrame {
if index < 0 || index >= len(zfs) {
return nil
}
return zfs[index]
}
// UnsafePointers returns unsafe pointers for all frames (for CGO batch operations)
func (zfs ZeroCopyFrameSlice) UnsafePointers() []unsafe.Pointer {
pointers := make([]unsafe.Pointer, len(zfs))
for i, frame := range zfs {
if frame != nil {
pointers[i] = frame.UnsafePointer()
}
}
return pointers
}

View File

@ -0,0 +1,415 @@
//go:build cgo
package audio
import (
"sync"
"sync/atomic"
"time"
)
// BatchZeroCopyProcessor handles batch operations on zero-copy audio frames
// with optimized reference counting and memory management
type BatchZeroCopyProcessor struct {
// Configuration
maxBatchSize int
batchTimeout time.Duration
processingDelay time.Duration
adaptiveThreshold float64
// Processing queues
readEncodeQueue chan *batchZeroCopyRequest
decodeWriteQueue chan *batchZeroCopyRequest
// Worker management
workerPool chan struct{}
running int32
wg sync.WaitGroup
// Statistics
batchedFrames int64
singleFrames int64
batchSavings int64
processingTimeUs int64
adaptiveHits int64
adaptiveMisses int64
}
type batchZeroCopyRequest struct {
frames []*ZeroCopyAudioFrame
operation batchZeroCopyOperation
resultCh chan batchZeroCopyResult
timestamp time.Time
}
type batchZeroCopyOperation int
const (
batchOpReadEncode batchZeroCopyOperation = iota
batchOpDecodeWrite
batchOpMixed
)
type batchZeroCopyResult struct {
encodedData [][]byte // For read-encode operations
processedCount int // Number of successfully processed frames
err error
}
// Global batch zero-copy processor
var (
globalBatchZeroCopyProcessor *BatchZeroCopyProcessor
batchZeroCopyOnce sync.Once
)
// GetBatchZeroCopyProcessor returns the global batch zero-copy processor
func GetBatchZeroCopyProcessor() *BatchZeroCopyProcessor {
batchZeroCopyOnce.Do(func() {
globalBatchZeroCopyProcessor = NewBatchZeroCopyProcessor()
globalBatchZeroCopyProcessor.Start()
})
return globalBatchZeroCopyProcessor
}
// NewBatchZeroCopyProcessor creates a new batch zero-copy processor
func NewBatchZeroCopyProcessor() *BatchZeroCopyProcessor {
cache := Config
return &BatchZeroCopyProcessor{
maxBatchSize: cache.BatchProcessorFramesPerBatch,
batchTimeout: cache.BatchProcessorTimeout,
processingDelay: cache.BatchProcessingDelay,
adaptiveThreshold: cache.BatchProcessorAdaptiveThreshold,
readEncodeQueue: make(chan *batchZeroCopyRequest, cache.BatchProcessorMaxQueueSize),
decodeWriteQueue: make(chan *batchZeroCopyRequest, cache.BatchProcessorMaxQueueSize),
workerPool: make(chan struct{}, 4), // 4 workers for parallel processing
}
}
// Start starts the batch zero-copy processor workers
func (bzcp *BatchZeroCopyProcessor) Start() {
if !atomic.CompareAndSwapInt32(&bzcp.running, 0, 1) {
return // Already running
}
// Start worker goroutines for read-encode operations
for i := 0; i < cap(bzcp.workerPool)/2; i++ {
bzcp.wg.Add(1)
go bzcp.readEncodeWorker()
}
// Start worker goroutines for decode-write operations
for i := 0; i < cap(bzcp.workerPool)/2; i++ {
bzcp.wg.Add(1)
go bzcp.decodeWriteWorker()
}
}
// Stop stops the batch zero-copy processor
func (bzcp *BatchZeroCopyProcessor) Stop() {
if !atomic.CompareAndSwapInt32(&bzcp.running, 1, 0) {
return // Already stopped
}
close(bzcp.readEncodeQueue)
close(bzcp.decodeWriteQueue)
bzcp.wg.Wait()
}
// readEncodeWorker processes batch read-encode operations
func (bzcp *BatchZeroCopyProcessor) readEncodeWorker() {
defer bzcp.wg.Done()
for req := range bzcp.readEncodeQueue {
bzcp.processBatchReadEncode(req)
}
}
// decodeWriteWorker processes batch decode-write operations
func (bzcp *BatchZeroCopyProcessor) decodeWriteWorker() {
defer bzcp.wg.Done()
for req := range bzcp.decodeWriteQueue {
bzcp.processBatchDecodeWrite(req)
}
}
// processBatchReadEncode processes a batch of read-encode operations
func (bzcp *BatchZeroCopyProcessor) processBatchReadEncode(req *batchZeroCopyRequest) {
startTime := time.Now()
result := batchZeroCopyResult{}
// Batch AddRef all frames first
err := BatchAddRefFrames(req.frames)
if err != nil {
result.err = err
if req.resultCh != nil {
req.resultCh <- result
close(req.resultCh)
}
return
}
// Process frames using existing batch read-encode logic
encodedData, err := BatchReadEncode(len(req.frames))
if err != nil {
// Batch release frames on error
if _, releaseErr := BatchReleaseFrames(req.frames); releaseErr != nil {
// Log release error but preserve original error
_ = releaseErr
}
result.err = err
} else {
result.encodedData = encodedData
result.processedCount = len(encodedData)
// Batch release frames after successful processing
if _, releaseErr := BatchReleaseFrames(req.frames); releaseErr != nil {
// Log release error but don't fail the operation
_ = releaseErr
}
}
// Update statistics
atomic.AddInt64(&bzcp.batchedFrames, int64(len(req.frames)))
atomic.AddInt64(&bzcp.batchSavings, int64(len(req.frames)-1))
atomic.AddInt64(&bzcp.processingTimeUs, time.Since(startTime).Microseconds())
// Send result back
if req.resultCh != nil {
req.resultCh <- result
close(req.resultCh)
}
}
// processBatchDecodeWrite processes a batch of decode-write operations
func (bzcp *BatchZeroCopyProcessor) processBatchDecodeWrite(req *batchZeroCopyRequest) {
startTime := time.Now()
result := batchZeroCopyResult{}
// Batch AddRef all frames first
err := BatchAddRefFrames(req.frames)
if err != nil {
result.err = err
if req.resultCh != nil {
req.resultCh <- result
close(req.resultCh)
}
return
}
// Extract data from zero-copy frames for batch processing
frameData := make([][]byte, len(req.frames))
for i, frame := range req.frames {
if frame != nil {
// Get data from zero-copy frame
frameData[i] = frame.Data()[:frame.Length()]
}
}
// Process frames using existing batch decode-write logic
err = BatchDecodeWrite(frameData)
if err != nil {
result.err = err
} else {
result.processedCount = len(req.frames)
}
// Batch release frames
if _, releaseErr := BatchReleaseFrames(req.frames); releaseErr != nil {
// Log release error but don't override processing error
_ = releaseErr
}
// Update statistics
atomic.AddInt64(&bzcp.batchedFrames, int64(len(req.frames)))
atomic.AddInt64(&bzcp.batchSavings, int64(len(req.frames)-1))
atomic.AddInt64(&bzcp.processingTimeUs, time.Since(startTime).Microseconds())
// Send result back
if req.resultCh != nil {
req.resultCh <- result
close(req.resultCh)
}
}
// BatchReadEncodeZeroCopy performs batch read-encode on zero-copy frames
func (bzcp *BatchZeroCopyProcessor) BatchReadEncodeZeroCopy(frames []*ZeroCopyAudioFrame) ([][]byte, error) {
if len(frames) == 0 {
return nil, nil
}
// For small batches, use direct operations to avoid overhead
if len(frames) <= 2 {
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleReadEncode(frames)
}
// Use adaptive threshold to determine batch vs single processing
batchedFrames := atomic.LoadInt64(&bzcp.batchedFrames)
singleFrames := atomic.LoadInt64(&bzcp.singleFrames)
totalFrames := batchedFrames + singleFrames
if totalFrames > 100 { // Only apply adaptive logic after some samples
batchRatio := float64(batchedFrames) / float64(totalFrames)
if batchRatio < bzcp.adaptiveThreshold {
// Batch processing not effective, use single processing
atomic.AddInt64(&bzcp.adaptiveMisses, 1)
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleReadEncode(frames)
}
atomic.AddInt64(&bzcp.adaptiveHits, 1)
}
// Use batch processing
if atomic.LoadInt32(&bzcp.running) == 0 {
// Fallback to single processing if batch processor not running
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleReadEncode(frames)
}
resultCh := make(chan batchZeroCopyResult, 1)
req := &batchZeroCopyRequest{
frames: frames,
operation: batchOpReadEncode,
resultCh: resultCh,
timestamp: time.Now(),
}
select {
case bzcp.readEncodeQueue <- req:
// Wait for completion
result := <-resultCh
return result.encodedData, result.err
default:
// Queue full, fallback to single processing
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleReadEncode(frames)
}
}
// BatchDecodeWriteZeroCopy performs batch decode-write on zero-copy frames
func (bzcp *BatchZeroCopyProcessor) BatchDecodeWriteZeroCopy(frames []*ZeroCopyAudioFrame) error {
if len(frames) == 0 {
return nil
}
// For small batches, use direct operations
if len(frames) <= 2 {
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleDecodeWrite(frames)
}
// Use adaptive threshold
batchedFrames := atomic.LoadInt64(&bzcp.batchedFrames)
singleFrames := atomic.LoadInt64(&bzcp.singleFrames)
totalFrames := batchedFrames + singleFrames
if totalFrames > 100 {
batchRatio := float64(batchedFrames) / float64(totalFrames)
if batchRatio < bzcp.adaptiveThreshold {
atomic.AddInt64(&bzcp.adaptiveMisses, 1)
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleDecodeWrite(frames)
}
atomic.AddInt64(&bzcp.adaptiveHits, 1)
}
// Use batch processing
if atomic.LoadInt32(&bzcp.running) == 0 {
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleDecodeWrite(frames)
}
resultCh := make(chan batchZeroCopyResult, 1)
req := &batchZeroCopyRequest{
frames: frames,
operation: batchOpDecodeWrite,
resultCh: resultCh,
timestamp: time.Now(),
}
select {
case bzcp.decodeWriteQueue <- req:
// Wait for completion
result := <-resultCh
return result.err
default:
// Queue full, fallback to single processing
atomic.AddInt64(&bzcp.singleFrames, int64(len(frames)))
return bzcp.processSingleDecodeWrite(frames)
}
}
// processSingleReadEncode processes frames individually for read-encode
func (bzcp *BatchZeroCopyProcessor) processSingleReadEncode(frames []*ZeroCopyAudioFrame) ([][]byte, error) {
// Extract data and use existing batch processing
frameData := make([][]byte, 0, len(frames))
for _, frame := range frames {
if frame != nil {
frame.AddRef()
frameData = append(frameData, frame.Data()[:frame.Length()])
}
}
// Use existing batch read-encode
result, err := BatchReadEncode(len(frameData))
// Release frames
for _, frame := range frames {
if frame != nil {
frame.Release()
}
}
return result, err
}
// processSingleDecodeWrite processes frames individually for decode-write
func (bzcp *BatchZeroCopyProcessor) processSingleDecodeWrite(frames []*ZeroCopyAudioFrame) error {
// Extract data and use existing batch processing
frameData := make([][]byte, 0, len(frames))
for _, frame := range frames {
if frame != nil {
frame.AddRef()
frameData = append(frameData, frame.Data()[:frame.Length()])
}
}
// Use existing batch decode-write
err := BatchDecodeWrite(frameData)
// Release frames
for _, frame := range frames {
if frame != nil {
frame.Release()
}
}
return err
}
// GetBatchZeroCopyStats returns batch zero-copy processing statistics
func (bzcp *BatchZeroCopyProcessor) GetBatchZeroCopyStats() (batchedFrames, singleFrames, savings, processingTimeUs, adaptiveHits, adaptiveMisses int64) {
return atomic.LoadInt64(&bzcp.batchedFrames),
atomic.LoadInt64(&bzcp.singleFrames),
atomic.LoadInt64(&bzcp.batchSavings),
atomic.LoadInt64(&bzcp.processingTimeUs),
atomic.LoadInt64(&bzcp.adaptiveHits),
atomic.LoadInt64(&bzcp.adaptiveMisses)
}
// Convenience functions for global batch zero-copy processor
// BatchReadEncodeZeroCopyFrames performs batch read-encode on zero-copy frames
func BatchReadEncodeZeroCopyFrames(frames []*ZeroCopyAudioFrame) ([][]byte, error) {
return GetBatchZeroCopyProcessor().BatchReadEncodeZeroCopy(frames)
}
// BatchDecodeWriteZeroCopyFrames performs batch decode-write on zero-copy frames
func BatchDecodeWriteZeroCopyFrames(frames []*ZeroCopyAudioFrame) error {
return GetBatchZeroCopyProcessor().BatchDecodeWriteZeroCopy(frames)
}
// GetGlobalBatchZeroCopyStats returns global batch zero-copy processing statistics
func GetGlobalBatchZeroCopyStats() (batchedFrames, singleFrames, savings, processingTimeUs, adaptiveHits, adaptiveMisses int64) {
return GetBatchZeroCopyProcessor().GetBatchZeroCopyStats()
}

View File

@ -14,12 +14,15 @@ import (
/* /*
#cgo CFLAGS: -I$HOME/.jetkvm/audio-libs/alsa-lib-$ALSA_VERSION/include -I$HOME/.jetkvm/audio-libs/opus-$OPUS_VERSION/include -I$HOME/.jetkvm/audio-libs/opus-$OPUS_VERSION/celt #cgo CFLAGS: -I$HOME/.jetkvm/audio-libs/alsa-lib-$ALSA_VERSION/include -I$HOME/.jetkvm/audio-libs/opus-$OPUS_VERSION/include -I$HOME/.jetkvm/audio-libs/opus-$OPUS_VERSION/celt
#cgo LDFLAGS: -L$HOME/.jetkvm/audio-libs/alsa-lib-$ALSA_VERSION/src/.libs -lasound -L$HOME/.jetkvm/audio-libs/opus-$OPUS_VERSION/.libs -lopus -lm -ldl -static #cgo LDFLAGS: -L$HOME/.jetkvm/audio-libs/alsa-lib-$ALSA_VERSION/src/.libs -lasound -L$HOME/.jetkvm/audio-libs/opus-$OPUS_VERSION/.libs -lopus -lm -ldl -static
#include <alsa/asoundlib.h> #include <alsa/asoundlib.h>
#include <opus.h> #include <opus.h>
#include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include <errno.h>
#include <unistd.h> #include <unistd.h>
#include <errno.h>
#include <sys/mman.h>
// C state for ALSA/Opus with safety flags // C state for ALSA/Opus with safety flags
static snd_pcm_t *pcm_handle = NULL; static snd_pcm_t *pcm_handle = NULL;
@ -27,25 +30,33 @@ static snd_pcm_t *pcm_playback_handle = NULL;
static OpusEncoder *encoder = NULL; static OpusEncoder *encoder = NULL;
static OpusDecoder *decoder = NULL; static OpusDecoder *decoder = NULL;
// Opus encoder settings - initialized from Go configuration // Opus encoder settings - initialized from Go configuration
static int opus_bitrate = 96000; // Will be set from GetConfig().CGOOpusBitrate static int opus_bitrate = 96000; // Will be set from Config.CGOOpusBitrate
static int opus_complexity = 3; // Will be set from GetConfig().CGOOpusComplexity static int opus_complexity = 3; // Will be set from Config.CGOOpusComplexity
static int opus_vbr = 1; // Will be set from GetConfig().CGOOpusVBR static int opus_vbr = 1; // Will be set from Config.CGOOpusVBR
static int opus_vbr_constraint = 1; // Will be set from GetConfig().CGOOpusVBRConstraint static int opus_vbr_constraint = 1; // Will be set from Config.CGOOpusVBRConstraint
static int opus_signal_type = 3; // Will be set from GetConfig().CGOOpusSignalType static int opus_signal_type = 3; // Will be set from Config.CGOOpusSignalType
static int opus_bandwidth = 1105; // OPUS_BANDWIDTH_WIDEBAND for compatibility (was 1101) static int opus_bandwidth = 1105; // OPUS_BANDWIDTH_WIDEBAND for compatibility (was 1101)
static int opus_dtx = 0; // Will be set from GetConfig().CGOOpusDTX static int opus_dtx = 0; // Will be set from Config.CGOOpusDTX
static int opus_lsb_depth = 16; // LSB depth for improved bit allocation on constrained hardware static int opus_lsb_depth = 16; // LSB depth for improved bit allocation on constrained hardware
static int sample_rate = 48000; // Will be set from GetConfig().CGOSampleRate static int sample_rate = 48000; // Will be set from Config.CGOSampleRate
static int channels = 2; // Will be set from GetConfig().CGOChannels static int channels = 2; // Will be set from Config.CGOChannels
static int frame_size = 960; // Will be set from GetConfig().CGOFrameSize static int frame_size = 960; // Will be set from Config.CGOFrameSize
static int max_packet_size = 1500; // Will be set from GetConfig().CGOMaxPacketSize static int max_packet_size = 1500; // Will be set from Config.CGOMaxPacketSize
static int sleep_microseconds = 1000; // Will be set from GetConfig().CGOUsleepMicroseconds static int sleep_microseconds = 1000; // Will be set from Config.CGOUsleepMicroseconds
static int max_attempts_global = 5; // Will be set from GetConfig().CGOMaxAttempts static int max_attempts_global = 5; // Will be set from Config.CGOMaxAttempts
static int max_backoff_us_global = 500000; // Will be set from GetConfig().CGOMaxBackoffMicroseconds static int max_backoff_us_global = 500000; // Will be set from Config.CGOMaxBackoffMicroseconds
// Hardware optimization flags for constrained environments // Hardware optimization flags for constrained environments
static int use_mmap_access = 0; // Disable MMAP for compatibility (was 1) static int use_mmap_access = 0; // Disable MMAP for compatibility (was 1)
static int optimized_buffer_size = 0; // Disable optimized buffer sizing for stability (was 1) static int optimized_buffer_size = 0; // Disable optimized buffer sizing for stability (was 1)
// C function declarations (implementations are below)
int jetkvm_audio_init();
void jetkvm_audio_close();
int jetkvm_audio_read_encode(void *opus_buf);
int jetkvm_audio_decode_write(void *opus_buf, int opus_size);
int jetkvm_audio_playback_init();
void jetkvm_audio_playback_close();
// Function to update constants from Go configuration // Function to update constants from Go configuration
void update_audio_constants(int bitrate, int complexity, int vbr, int vbr_constraint, void update_audio_constants(int bitrate, int complexity, int vbr, int vbr_constraint,
int signal_type, int bandwidth, int dtx, int lsb_depth, int sr, int ch, int signal_type, int bandwidth, int dtx, int lsb_depth, int sr, int ch,
@ -76,8 +87,10 @@ static volatile int playback_initialized = 0;
// Function to dynamically update Opus encoder parameters // Function to dynamically update Opus encoder parameters
int update_opus_encoder_params(int bitrate, int complexity, int vbr, int vbr_constraint, int update_opus_encoder_params(int bitrate, int complexity, int vbr, int vbr_constraint,
int signal_type, int bandwidth, int dtx) { int signal_type, int bandwidth, int dtx) {
if (!encoder || !capture_initialized) { // This function works for both audio input and output encoder parameters
return -1; // Encoder not initialized // Require either capture (output) or playback (input) initialization
if (!encoder || (!capture_initialized && !playback_initialized)) {
return -1; // Audio encoder not initialized
} }
// Update the static variables // Update the static variables
@ -698,9 +711,9 @@ func cgoAudioInit() error {
C.int(cache.channels.Load()), C.int(cache.channels.Load()),
C.int(cache.frameSize.Load()), C.int(cache.frameSize.Load()),
C.int(cache.maxPacketSize.Load()), C.int(cache.maxPacketSize.Load()),
C.int(GetConfig().CGOUsleepMicroseconds), C.int(Config.CGOUsleepMicroseconds),
C.int(GetConfig().CGOMaxAttempts), C.int(Config.CGOMaxAttempts),
C.int(GetConfig().CGOMaxBackoffMicroseconds), C.int(Config.CGOMaxBackoffMicroseconds),
) )
result := C.jetkvm_audio_init() result := C.jetkvm_audio_init()
@ -715,7 +728,6 @@ func cgoAudioClose() {
} }
// AudioConfigCache provides a comprehensive caching system for audio configuration // AudioConfigCache provides a comprehensive caching system for audio configuration
// to minimize GetConfig() calls in the hot path
type AudioConfigCache struct { type AudioConfigCache struct {
// Atomic int64 fields MUST be first for ARM32 alignment (8-byte alignment required) // Atomic int64 fields MUST be first for ARM32 alignment (8-byte alignment required)
minFrameDuration atomic.Int64 // Store as nanoseconds minFrameDuration atomic.Int64 // Store as nanoseconds
@ -804,52 +816,50 @@ func (c *AudioConfigCache) Update() {
// Double-check after acquiring lock // Double-check after acquiring lock
if !c.initialized.Load() || time.Since(c.lastUpdate) > c.cacheExpiry { if !c.initialized.Load() || time.Since(c.lastUpdate) > c.cacheExpiry {
config := GetConfig() // Call GetConfig() only once
// Update atomic values for lock-free access - CGO values // Update atomic values for lock-free access - CGO values
c.minReadEncodeBuffer.Store(int32(config.MinReadEncodeBuffer)) c.minReadEncodeBuffer.Store(int32(Config.MinReadEncodeBuffer))
c.maxDecodeWriteBuffer.Store(int32(config.MaxDecodeWriteBuffer)) c.maxDecodeWriteBuffer.Store(int32(Config.MaxDecodeWriteBuffer))
c.maxPacketSize.Store(int32(config.CGOMaxPacketSize)) c.maxPacketSize.Store(int32(Config.CGOMaxPacketSize))
c.maxPCMBufferSize.Store(int32(config.MaxPCMBufferSize)) c.maxPCMBufferSize.Store(int32(Config.MaxPCMBufferSize))
c.opusBitrate.Store(int32(config.CGOOpusBitrate)) c.opusBitrate.Store(int32(Config.CGOOpusBitrate))
c.opusComplexity.Store(int32(config.CGOOpusComplexity)) c.opusComplexity.Store(int32(Config.CGOOpusComplexity))
c.opusVBR.Store(int32(config.CGOOpusVBR)) c.opusVBR.Store(int32(Config.CGOOpusVBR))
c.opusVBRConstraint.Store(int32(config.CGOOpusVBRConstraint)) c.opusVBRConstraint.Store(int32(Config.CGOOpusVBRConstraint))
c.opusSignalType.Store(int32(config.CGOOpusSignalType)) c.opusSignalType.Store(int32(Config.CGOOpusSignalType))
c.opusBandwidth.Store(int32(config.CGOOpusBandwidth)) c.opusBandwidth.Store(int32(Config.CGOOpusBandwidth))
c.opusDTX.Store(int32(config.CGOOpusDTX)) c.opusDTX.Store(int32(Config.CGOOpusDTX))
c.sampleRate.Store(int32(config.CGOSampleRate)) c.sampleRate.Store(int32(Config.CGOSampleRate))
c.channels.Store(int32(config.CGOChannels)) c.channels.Store(int32(Config.CGOChannels))
c.frameSize.Store(int32(config.CGOFrameSize)) c.frameSize.Store(int32(Config.CGOFrameSize))
// Update additional validation values // Update additional validation values
c.maxAudioFrameSize.Store(int32(config.MaxAudioFrameSize)) c.maxAudioFrameSize.Store(int32(Config.MaxAudioFrameSize))
c.maxChannels.Store(int32(config.MaxChannels)) c.maxChannels.Store(int32(Config.MaxChannels))
c.minFrameDuration.Store(int64(config.MinFrameDuration)) c.minFrameDuration.Store(int64(Config.MinFrameDuration))
c.maxFrameDuration.Store(int64(config.MaxFrameDuration)) c.maxFrameDuration.Store(int64(Config.MaxFrameDuration))
c.minOpusBitrate.Store(int32(config.MinOpusBitrate)) c.minOpusBitrate.Store(int32(Config.MinOpusBitrate))
c.maxOpusBitrate.Store(int32(config.MaxOpusBitrate)) c.maxOpusBitrate.Store(int32(Config.MaxOpusBitrate))
// Update batch processing related values // Update batch processing related values
c.BatchProcessingTimeout = 100 * time.Millisecond // Fixed timeout for batch processing c.BatchProcessingTimeout = 100 * time.Millisecond // Fixed timeout for batch processing
c.BatchProcessorFramesPerBatch = config.BatchProcessorFramesPerBatch c.BatchProcessorFramesPerBatch = Config.BatchProcessorFramesPerBatch
c.BatchProcessorTimeout = config.BatchProcessorTimeout c.BatchProcessorTimeout = Config.BatchProcessorTimeout
c.BatchProcessingDelay = config.BatchProcessingDelay c.BatchProcessingDelay = Config.BatchProcessingDelay
c.MinBatchSizeForThreadPinning = config.MinBatchSizeForThreadPinning c.MinBatchSizeForThreadPinning = Config.MinBatchSizeForThreadPinning
c.BatchProcessorMaxQueueSize = config.BatchProcessorMaxQueueSize c.BatchProcessorMaxQueueSize = Config.BatchProcessorMaxQueueSize
c.BatchProcessorAdaptiveThreshold = config.BatchProcessorAdaptiveThreshold c.BatchProcessorAdaptiveThreshold = Config.BatchProcessorAdaptiveThreshold
c.BatchProcessorThreadPinningThreshold = config.BatchProcessorThreadPinningThreshold c.BatchProcessorThreadPinningThreshold = Config.BatchProcessorThreadPinningThreshold
// Pre-allocate common errors // Pre-allocate common errors
c.bufferTooSmallReadEncode = newBufferTooSmallError(0, config.MinReadEncodeBuffer) c.bufferTooSmallReadEncode = newBufferTooSmallError(0, Config.MinReadEncodeBuffer)
c.bufferTooLargeDecodeWrite = newBufferTooLargeError(config.MaxDecodeWriteBuffer+1, config.MaxDecodeWriteBuffer) c.bufferTooLargeDecodeWrite = newBufferTooLargeError(Config.MaxDecodeWriteBuffer+1, Config.MaxDecodeWriteBuffer)
c.lastUpdate = time.Now() c.lastUpdate = time.Now()
c.initialized.Store(true) c.initialized.Store(true)
// Update the global validation cache as well // Update the global validation cache as well
if cachedMaxFrameSize != 0 { if cachedMaxFrameSize != 0 {
cachedMaxFrameSize = config.MaxAudioFrameSize cachedMaxFrameSize = Config.MaxAudioFrameSize
} }
} }
} }
@ -901,46 +911,28 @@ func updateCacheIfNeeded(cache *AudioConfigCache) {
} }
func cgoAudioReadEncode(buf []byte) (int, error) { func cgoAudioReadEncode(buf []byte) (int, error) {
cache := GetCachedConfig() // Minimal buffer validation - assume caller provides correct size
updateCacheIfNeeded(cache) if len(buf) == 0 {
return 0, errEmptyBuffer
// Fast validation with cached values - avoid lock with atomic access
minRequired := cache.GetMinReadEncodeBuffer()
// Buffer validation - use pre-allocated error for common case
if len(buf) < minRequired {
// Use pre-allocated error for common case, only create custom error for edge cases
if len(buf) > 0 {
return 0, newBufferTooSmallError(len(buf), minRequired)
}
return 0, cache.GetBufferTooSmallError()
} }
// Skip initialization check for now to avoid CGO compilation issues // Direct CGO call - hotpath optimization
// Direct CGO call with minimal overhead - unsafe.Pointer(&slice[0]) is safe for validated non-empty buffers
n := C.jetkvm_audio_read_encode(unsafe.Pointer(&buf[0])) n := C.jetkvm_audio_read_encode(unsafe.Pointer(&buf[0]))
// Fast path for success case // Fast path for success
if n > 0 { if n > 0 {
return int(n), nil return int(n), nil
} }
// Handle error cases - use static error codes to reduce allocations // Error handling with static errors
if n < 0 { if n < 0 {
// Common error cases if n == -1 {
switch n {
case -1:
return 0, errAudioInitFailed return 0, errAudioInitFailed
case -2:
return 0, errAudioReadEncode
default:
return 0, newAudioReadEncodeError(int(n))
} }
return 0, errAudioReadEncode
} }
// n == 0 case return 0, nil
return 0, nil // No data available
} }
// Audio playback functions // Audio playback functions
@ -962,58 +954,25 @@ func cgoAudioPlaybackClose() {
C.jetkvm_audio_playback_close() C.jetkvm_audio_playback_close()
} }
func cgoAudioDecodeWrite(buf []byte) (n int, err error) { func cgoAudioDecodeWrite(buf []byte) (int, error) {
// Fast validation with AudioConfigCache // Minimal validation - assume caller provides correct size
cache := GetCachedConfig()
// Only update cache if expired - avoid unnecessary overhead
// Use proper locking to avoid race condition
if cache.initialized.Load() {
cache.mutex.RLock()
cacheExpired := time.Since(cache.lastUpdate) > cache.cacheExpiry
cache.mutex.RUnlock()
if cacheExpired {
cache.Update()
}
} else {
cache.Update()
}
// Optimized buffer validation
if len(buf) == 0 { if len(buf) == 0 {
return 0, errEmptyBuffer return 0, errEmptyBuffer
} }
// Use cached max buffer size with atomic access // Direct CGO call - hotpath optimization
maxAllowed := cache.GetMaxDecodeWriteBuffer() n := int(C.jetkvm_audio_decode_write(unsafe.Pointer(&buf[0]), C.int(len(buf))))
if len(buf) > maxAllowed {
// Use pre-allocated error for common case
if len(buf) == maxAllowed+1 {
return 0, cache.GetBufferTooLargeError()
}
return 0, newBufferTooLargeError(len(buf), maxAllowed)
}
// Direct CGO call with minimal overhead - unsafe.Pointer(&slice[0]) is safe for validated non-empty buffers // Fast path for success
n = int(C.jetkvm_audio_decode_write(unsafe.Pointer(&buf[0]), C.int(len(buf))))
// Fast path for success case
if n >= 0 { if n >= 0 {
return n, nil return n, nil
} }
// Handle error cases with static error codes // Error handling with static errors
switch n { if n == -1 {
case -1: return 0, errAudioInitFailed
n = 0
err = errAudioInitFailed
case -2:
n = 0
err = errAudioDecodeWrite
default:
n = 0
err = newAudioDecodeWriteError(n)
} }
return return 0, errAudioDecodeWrite
} }
// updateOpusEncoderParams dynamically updates OPUS encoder parameters // updateOpusEncoderParams dynamically updates OPUS encoder parameters
@ -1036,7 +995,7 @@ func updateOpusEncoderParams(bitrate, complexity, vbr, vbrConstraint, signalType
// Buffer pool for reusing buffers in CGO functions // Buffer pool for reusing buffers in CGO functions
var ( var (
// Using SizedBufferPool for better memory management // Using SizedBufferPool for better memory management
// Track buffer pool usage for monitoring // Track buffer pool usage
cgoBufferPoolGets atomic.Int64 cgoBufferPoolGets atomic.Int64
cgoBufferPoolPuts atomic.Int64 cgoBufferPoolPuts atomic.Int64
// Batch processing statistics - only enabled in debug builds // Batch processing statistics - only enabled in debug builds
@ -1099,70 +1058,24 @@ func DecodeWriteWithPooledBuffer(data []byte) (int, error) {
} }
// BatchReadEncode reads and encodes multiple audio frames in a single batch // BatchReadEncode reads and encodes multiple audio frames in a single batch
// with optimized zero-copy frame management and batch reference counting
func BatchReadEncode(batchSize int) ([][]byte, error) { func BatchReadEncode(batchSize int) ([][]byte, error) {
cache := GetCachedConfig() // Simple batch processing without complex overhead
updateCacheIfNeeded(cache)
// Calculate total buffer size needed for batch
frameSize := cache.GetMinReadEncodeBuffer()
totalSize := frameSize * batchSize
// Get a single large buffer for all frames
batchBuffer := GetBufferFromPool(totalSize)
defer ReturnBufferToPool(batchBuffer)
// Pre-allocate frame result buffers from pool to avoid allocations in loop
frameBuffers := make([][]byte, 0, batchSize)
for i := 0; i < batchSize; i++ {
frameBuffers = append(frameBuffers, GetBufferFromPool(frameSize))
}
defer func() {
// Return all frame buffers to pool
for _, buf := range frameBuffers {
ReturnBufferToPool(buf)
}
}()
// Track batch processing statistics - only if enabled
var startTime time.Time
// Batch time tracking removed
trackTime := false
if trackTime {
startTime = time.Now()
}
batchProcessingCount.Add(1)
// Process frames in batch
frames := make([][]byte, 0, batchSize) frames := make([][]byte, 0, batchSize)
for i := 0; i < batchSize; i++ { frameSize := 4096 // Fixed frame size for performance
// Calculate offset for this frame in the batch buffer
offset := i * frameSize
frameBuf := batchBuffer[offset : offset+frameSize]
// Process this frame for i := 0; i < batchSize; i++ {
n, err := cgoAudioReadEncode(frameBuf) buf := make([]byte, frameSize)
n, err := cgoAudioReadEncode(buf)
if err != nil { if err != nil {
// Return partial batch on error
if i > 0 { if i > 0 {
batchFrameCount.Add(int64(i)) return frames, nil // Return partial batch
if trackTime {
batchProcessingTime.Add(time.Since(startTime).Microseconds())
}
return frames, nil
} }
return nil, err return nil, err
} }
if n > 0 {
// Reuse pre-allocated buffer instead of make([]byte, n) frames = append(frames, buf[:n])
frameCopy := frameBuffers[i][:n] // Slice to actual size
copy(frameCopy, frameBuf[:n])
frames = append(frames, frameCopy)
} }
// Update statistics
batchFrameCount.Add(int64(len(frames)))
if trackTime {
batchProcessingTime.Add(time.Since(startTime).Microseconds())
} }
return frames, nil return frames, nil
@ -1170,12 +1083,39 @@ func BatchReadEncode(batchSize int) ([][]byte, error) {
// BatchDecodeWrite decodes and writes multiple audio frames in a single batch // BatchDecodeWrite decodes and writes multiple audio frames in a single batch
// This reduces CGO call overhead by processing multiple frames at once // This reduces CGO call overhead by processing multiple frames at once
// with optimized zero-copy frame management and batch reference counting
func BatchDecodeWrite(frames [][]byte) error { func BatchDecodeWrite(frames [][]byte) error {
// Validate input // Validate input
if len(frames) == 0 { if len(frames) == 0 {
return nil return nil
} }
// Convert to zero-copy frames for optimized processing
zeroCopyFrames := make([]*ZeroCopyAudioFrame, 0, len(frames))
for _, frameData := range frames {
if len(frameData) > 0 {
frame := GetZeroCopyFrame()
frame.SetDataDirect(frameData) // Direct assignment without copy
zeroCopyFrames = append(zeroCopyFrames, frame)
}
}
// Use batch reference counting for efficient management
if len(zeroCopyFrames) > 0 {
// Batch AddRef all frames at once
err := BatchAddRefFrames(zeroCopyFrames)
if err != nil {
return err
}
// Ensure cleanup with batch release
defer func() {
if _, err := BatchReleaseFrames(zeroCopyFrames); err != nil {
// Log release error but don't fail the operation
_ = err
}
}()
}
// Get cached config // Get cached config
cache := GetCachedConfig() cache := GetCachedConfig()
// Only update cache if expired - avoid unnecessary overhead // Only update cache if expired - avoid unnecessary overhead
@ -1204,16 +1144,17 @@ func BatchDecodeWrite(frames [][]byte) error {
pcmBuffer := GetBufferFromPool(cache.GetMaxPCMBufferSize()) pcmBuffer := GetBufferFromPool(cache.GetMaxPCMBufferSize())
defer ReturnBufferToPool(pcmBuffer) defer ReturnBufferToPool(pcmBuffer)
// Process each frame // Process each zero-copy frame with optimized batch processing
frameCount := 0 frameCount := 0
for _, frame := range frames { for _, zcFrame := range zeroCopyFrames {
// Skip empty frames // Get frame data from zero-copy frame
if len(frame) == 0 { frameData := zcFrame.Data()[:zcFrame.Length()]
if len(frameData) == 0 {
continue continue
} }
// Process this frame using optimized implementation // Process this frame using optimized implementation
_, err := CGOAudioDecodeWrite(frame, pcmBuffer) _, err := CGOAudioDecodeWrite(frameData, pcmBuffer)
if err != nil { if err != nil {
// Update statistics before returning error // Update statistics before returning error
batchFrameCount.Add(int64(frameCount)) batchFrameCount.Add(int64(frameCount))

View File

@ -4,12 +4,12 @@ import "time"
// GetMetricsUpdateInterval returns the current metrics update interval from centralized config // GetMetricsUpdateInterval returns the current metrics update interval from centralized config
func GetMetricsUpdateInterval() time.Duration { func GetMetricsUpdateInterval() time.Duration {
return GetConfig().MetricsUpdateInterval return Config.MetricsUpdateInterval
} }
// SetMetricsUpdateInterval sets the metrics update interval in centralized config // SetMetricsUpdateInterval sets the metrics update interval in centralized config
func SetMetricsUpdateInterval(interval time.Duration) { func SetMetricsUpdateInterval(interval time.Duration) {
config := GetConfig() config := Config
config.MetricsUpdateInterval = interval config.MetricsUpdateInterval = interval
UpdateConfig(config) UpdateConfig(config)
} }

View File

@ -117,7 +117,6 @@ type AudioConfigConstants struct {
// Buffer Management // Buffer Management
PreallocSize int
MaxPoolSize int MaxPoolSize int
MessagePoolSize int MessagePoolSize int
OptimalSocketBuffer int OptimalSocketBuffer int
@ -131,7 +130,7 @@ type AudioConfigConstants struct {
MinReadEncodeBuffer int MinReadEncodeBuffer int
MaxDecodeWriteBuffer int MaxDecodeWriteBuffer int
MinBatchSizeForThreadPinning int MinBatchSizeForThreadPinning int
GoroutineMonitorInterval time.Duration
MagicNumber uint32 MagicNumber uint32
MaxFrameSize int MaxFrameSize int
WriteTimeout time.Duration WriteTimeout time.Duration
@ -172,9 +171,6 @@ type AudioConfigConstants struct {
OutputSupervisorTimeout time.Duration // 5s OutputSupervisorTimeout time.Duration // 5s
BatchProcessingDelay time.Duration // 10ms BatchProcessingDelay time.Duration // 10ms
AdaptiveOptimizerStability time.Duration // 10s
LatencyMonitorTarget time.Duration // 50ms
// Adaptive Buffer Configuration // Adaptive Buffer Configuration
// LowCPUThreshold defines CPU usage threshold for buffer size reduction. // LowCPUThreshold defines CPU usage threshold for buffer size reduction.
LowCPUThreshold float64 // 20% CPU threshold for buffer optimization LowCPUThreshold float64 // 20% CPU threshold for buffer optimization
@ -186,7 +182,7 @@ type AudioConfigConstants struct {
AdaptiveBufferTargetLatency time.Duration // 20ms target latency AdaptiveBufferTargetLatency time.Duration // 20ms target latency
CooldownPeriod time.Duration // 30s cooldown period CooldownPeriod time.Duration // 30s cooldown period
RollbackThreshold time.Duration // 300ms rollback threshold RollbackThreshold time.Duration // 300ms rollback threshold
AdaptiveOptimizerLatencyTarget time.Duration // 50ms latency target
MaxLatencyThreshold time.Duration // 200ms max latency MaxLatencyThreshold time.Duration // 200ms max latency
JitterThreshold time.Duration // 20ms jitter threshold JitterThreshold time.Duration // 20ms jitter threshold
LatencyOptimizationInterval time.Duration // 5s optimization interval LatencyOptimizationInterval time.Duration // 5s optimization interval
@ -216,27 +212,6 @@ type AudioConfigConstants struct {
CGOPCMBufferSize int // PCM buffer size for CGO audio processing CGOPCMBufferSize int // PCM buffer size for CGO audio processing
CGONanosecondsPerSecond float64 // Nanoseconds per second conversion CGONanosecondsPerSecond float64 // Nanoseconds per second conversion
FrontendOperationDebounceMS int // Frontend operation debounce delay
FrontendSyncDebounceMS int // Frontend sync debounce delay
FrontendSampleRate int // Frontend sample rate
FrontendRetryDelayMS int // Frontend retry delay
FrontendShortDelayMS int // Frontend short delay
FrontendLongDelayMS int // Frontend long delay
FrontendSyncDelayMS int // Frontend sync delay
FrontendMaxRetryAttempts int // Frontend max retry attempts
FrontendAudioLevelUpdateMS int // Frontend audio level update interval
FrontendFFTSize int // Frontend FFT size
FrontendAudioLevelMax int // Frontend max audio level
FrontendReconnectIntervalMS int // Frontend reconnect interval
FrontendSubscriptionDelayMS int // Frontend subscription delay
FrontendDebugIntervalMS int // Frontend debug interval
// Process Monitoring Constants
ProcessMonitorDefaultMemoryGB int // Default memory size for fallback (4GB)
ProcessMonitorKBToBytes int // KB to bytes conversion factor (1024)
ProcessMonitorDefaultClockHz float64 // Default system clock frequency (250.0 Hz)
ProcessMonitorFallbackClockHz float64 // Fallback clock frequency (1000.0 Hz)
ProcessMonitorTraditionalHz float64 // Traditional system clock frequency (100.0 Hz)
// Batch Processing Constants // Batch Processing Constants
BatchProcessorFramesPerBatch int // Frames processed per batch (4) BatchProcessorFramesPerBatch int // Frames processed per batch (4)
@ -272,7 +247,14 @@ type AudioConfigConstants struct {
LatencyPercentile50 int LatencyPercentile50 int
LatencyPercentile95 int LatencyPercentile95 int
LatencyPercentile99 int LatencyPercentile99 int
BufferPoolMaxOperations int
// Buffer Pool Configuration
BufferPoolDefaultSize int // Default buffer pool size when MaxPoolSize is invalid
BufferPoolControlSize int // Control buffer pool size
ZeroCopyPreallocSizeBytes int // Zero-copy frame pool preallocation size in bytes
ZeroCopyMinPreallocFrames int // Minimum preallocated frames for zero-copy pool
BufferPoolHitRateBase float64 // Base for hit rate percentage calculation
HitRateCalculationBase float64 HitRateCalculationBase float64
MaxLatency time.Duration MaxLatency time.Duration
MinMetricsUpdateInterval time.Duration MinMetricsUpdateInterval time.Duration
@ -313,6 +295,22 @@ type AudioConfigConstants struct {
AudioProcessorQueueSize int AudioProcessorQueueSize int
AudioReaderQueueSize int AudioReaderQueueSize int
WorkerMaxIdleTime time.Duration WorkerMaxIdleTime time.Duration
// Connection Retry Configuration
MaxConnectionAttempts int // Maximum connection retry attempts
ConnectionRetryDelay time.Duration // Initial connection retry delay
MaxConnectionRetryDelay time.Duration // Maximum connection retry delay
ConnectionBackoffFactor float64 // Connection retry backoff factor
ConnectionTimeoutDelay time.Duration // Connection timeout for each attempt
ReconnectionInterval time.Duration // Interval for automatic reconnection attempts
HealthCheckInterval time.Duration // Health check interval for connections
// Quality Change Timeout Configuration
QualityChangeSupervisorTimeout time.Duration // Timeout for supervisor stop during quality changes
QualityChangeTickerInterval time.Duration // Ticker interval for supervisor stop polling
QualityChangeSettleDelay time.Duration // Delay for quality change to settle
QualityChangeRecoveryDelay time.Duration // Delay before attempting recovery
} }
// DefaultAudioConfig returns the default configuration constants // DefaultAudioConfig returns the default configuration constants
@ -422,31 +420,31 @@ func DefaultAudioConfig() *AudioConfigConstants {
MaxRestartDelay: 30 * time.Second, // Maximum delay for exponential backoff MaxRestartDelay: 30 * time.Second, // Maximum delay for exponential backoff
// Buffer Management // Buffer Management
PreallocSize: 1024 * 1024, // 1MB buffer preallocation
MaxPoolSize: 100, // Maximum object pool size MaxPoolSize: 100, // Maximum object pool size
MessagePoolSize: 256, // Message pool size for IPC MessagePoolSize: 1024, // Significantly increased message pool for quality change bursts
OptimalSocketBuffer: 262144, // 256KB optimal socket buffer OptimalSocketBuffer: 262144, // 256KB optimal socket buffer
MaxSocketBuffer: 1048576, // 1MB maximum socket buffer MaxSocketBuffer: 1048576, // 1MB maximum socket buffer
MinSocketBuffer: 8192, // 8KB minimum socket buffer MinSocketBuffer: 8192, // 8KB minimum socket buffer
ChannelBufferSize: 500, // Inter-goroutine channel buffer size ChannelBufferSize: 2048, // Significantly increased channel buffer for quality change bursts
AudioFramePoolSize: 1500, // Audio frame object pool size AudioFramePoolSize: 1500, // Audio frame object pool size
PageSize: 4096, // Memory page size for alignment PageSize: 4096, // Memory page size for alignment
InitialBufferFrames: 500, // Initial buffer size during startup InitialBufferFrames: 1000, // Increased initial buffer size during startup
BytesToMBDivisor: 1024 * 1024, // Byte to megabyte conversion BytesToMBDivisor: 1024 * 1024, // Byte to megabyte conversion
MinReadEncodeBuffer: 1276, // Minimum CGO read/encode buffer MinReadEncodeBuffer: 1276, // Minimum CGO read/encode buffer
MaxDecodeWriteBuffer: 4096, // Maximum CGO decode/write buffer MaxDecodeWriteBuffer: 4096, // Maximum CGO decode/write buffer
// IPC Configuration // IPC Configuration - Balanced for stability
MagicNumber: 0xDEADBEEF, // IPC message validation header MagicNumber: 0xDEADBEEF, // IPC message validation header
MaxFrameSize: 4096, // Maximum audio frame size (4KB) MaxFrameSize: 4096, // Maximum audio frame size (4KB)
WriteTimeout: 100 * time.Millisecond, // IPC write operation timeout WriteTimeout: 1000 * time.Millisecond, // Further increased timeout to handle quality change bursts
HeaderSize: 8, // IPC message header size HeaderSize: 8, // IPC message header size
// Monitoring and Metrics // Monitoring and Metrics - Balanced for stability
MetricsUpdateInterval: 1000 * time.Millisecond, // Metrics collection frequency MetricsUpdateInterval: 1000 * time.Millisecond, // Stable metrics collection frequency
WarmupSamples: 10, // Warmup samples for metrics accuracy WarmupSamples: 10, // Adequate warmup samples for accuracy
MetricsChannelBuffer: 100, // Metrics data channel buffer size MetricsChannelBuffer: 100, // Adequate metrics data channel buffer
LatencyHistorySize: 100, // Number of latency measurements to keep LatencyHistorySize: 100, // Adequate latency measurements to keep
// Process Monitoring Constants // Process Monitoring Constants
MaxCPUPercent: 100.0, // Maximum CPU percentage MaxCPUPercent: 100.0, // Maximum CPU percentage
@ -470,41 +468,50 @@ func DefaultAudioConfig() *AudioConfigConstants {
BackoffMultiplier: 2.0, // Exponential backoff multiplier BackoffMultiplier: 2.0, // Exponential backoff multiplier
MaxConsecutiveErrors: 5, // Consecutive error threshold MaxConsecutiveErrors: 5, // Consecutive error threshold
// Timing Constants // Connection Retry Configuration
DefaultSleepDuration: 100 * time.Millisecond, // Standard polling interval MaxConnectionAttempts: 15, // Maximum connection retry attempts
ShortSleepDuration: 10 * time.Millisecond, // High-frequency polling ConnectionRetryDelay: 50 * time.Millisecond, // Initial connection retry delay
LongSleepDuration: 200 * time.Millisecond, // Background tasks MaxConnectionRetryDelay: 2 * time.Second, // Maximum connection retry delay
DefaultTickerInterval: 100 * time.Millisecond, // Periodic task interval ConnectionBackoffFactor: 1.5, // Connection retry backoff factor
BufferUpdateInterval: 500 * time.Millisecond, // Buffer status updates ConnectionTimeoutDelay: 5 * time.Second, // Connection timeout for each attempt
ReconnectionInterval: 30 * time.Second, // Interval for automatic reconnection attempts
HealthCheckInterval: 10 * time.Second, // Health check interval for connections
// Quality Change Timeout Configuration
QualityChangeSupervisorTimeout: 5 * time.Second, // Timeout for supervisor stop during quality changes
QualityChangeTickerInterval: 100 * time.Millisecond, // Ticker interval for supervisor stop polling
QualityChangeSettleDelay: 2 * time.Second, // Delay for quality change to settle
QualityChangeRecoveryDelay: 1 * time.Second, // Delay before attempting recovery
// Timing Constants - Optimized for quality change stability
DefaultSleepDuration: 100 * time.Millisecond, // Balanced polling interval
ShortSleepDuration: 10 * time.Millisecond, // Balanced high-frequency polling
LongSleepDuration: 200 * time.Millisecond, // Balanced background task delay
DefaultTickerInterval: 100 * time.Millisecond, // Balanced periodic task interval
BufferUpdateInterval: 250 * time.Millisecond, // Faster buffer size update frequency
InputSupervisorTimeout: 5 * time.Second, // Input monitoring timeout InputSupervisorTimeout: 5 * time.Second, // Input monitoring timeout
OutputSupervisorTimeout: 5 * time.Second, // Output monitoring timeout OutputSupervisorTimeout: 5 * time.Second, // Output monitoring timeout
BatchProcessingDelay: 10 * time.Millisecond, // Batch processing delay BatchProcessingDelay: 5 * time.Millisecond, // Reduced batch processing delay
AdaptiveOptimizerStability: 10 * time.Second, // Adaptive stability period
LatencyMonitorTarget: 50 * time.Millisecond, // Target latency for monitoring // Adaptive Buffer Configuration - Optimized for single-core RV1106G3
LowCPUThreshold: 0.40, // Adjusted for single-core ARM system
HighCPUThreshold: 0.75, // Adjusted for single-core RV1106G3 (current load ~64%)
LowMemoryThreshold: 0.60,
HighMemoryThreshold: 0.85, // Adjusted for 200MB total memory system
AdaptiveBufferTargetLatency: 10 * time.Millisecond, // Aggressive target latency for responsiveness
// Adaptive Buffer Configuration // Adaptive Buffer Size Configuration - Optimized for quality change bursts
LowCPUThreshold: 0.20, AdaptiveMinBufferSize: 256, // Further increased minimum to prevent emergency mode
HighCPUThreshold: 0.60, AdaptiveMaxBufferSize: 1024, // Much higher maximum for quality changes
LowMemoryThreshold: 0.50, AdaptiveDefaultBufferSize: 512, // Higher default for stability during bursts
HighMemoryThreshold: 0.75,
AdaptiveBufferTargetLatency: 20 * time.Millisecond,
// Adaptive Buffer Size Configuration CooldownPeriod: 15 * time.Second, // Reduced cooldown period
AdaptiveMinBufferSize: 3, // Minimum 3 frames for stability RollbackThreshold: 200 * time.Millisecond, // Lower rollback threshold
AdaptiveMaxBufferSize: 20, // Maximum 20 frames for high load
AdaptiveDefaultBufferSize: 6, // Balanced buffer size (6 frames)
// Adaptive Optimizer Configuration MaxLatencyThreshold: 150 * time.Millisecond, // Lower max latency threshold
CooldownPeriod: 30 * time.Second, JitterThreshold: 15 * time.Millisecond, // Reduced jitter threshold
RollbackThreshold: 300 * time.Millisecond, LatencyOptimizationInterval: 3 * time.Second, // More frequent optimization
AdaptiveOptimizerLatencyTarget: 50 * time.Millisecond, LatencyAdaptiveThreshold: 0.7, // More aggressive adaptive threshold
// Latency Monitor Configuration
MaxLatencyThreshold: 200 * time.Millisecond,
JitterThreshold: 20 * time.Millisecond,
LatencyOptimizationInterval: 5 * time.Second,
LatencyAdaptiveThreshold: 0.8,
// Microphone Contention Configuration // Microphone Contention Configuration
MicContentionTimeout: 200 * time.Millisecond, MicContentionTimeout: 200 * time.Millisecond,
@ -532,48 +539,25 @@ func DefaultAudioConfig() *AudioConfigConstants {
LatencyScalingFactor: 2.0, // Latency ratio scaling factor LatencyScalingFactor: 2.0, // Latency ratio scaling factor
OptimizerAggressiveness: 0.7, // Optimizer aggressiveness factor OptimizerAggressiveness: 0.7, // Optimizer aggressiveness factor
// CGO Audio Processing Constants // CGO Audio Processing Constants - Balanced for stability
CGOUsleepMicroseconds: 1000, // 1000 microseconds (1ms) for CGO usleep calls CGOUsleepMicroseconds: 1000, // 1000 microseconds (1ms) for stable CGO usleep calls
CGOPCMBufferSize: 1920, // 1920 samples for PCM buffer (max 2ch*960) CGOPCMBufferSize: 1920, // 1920 samples for PCM buffer (max 2ch*960)
CGONanosecondsPerSecond: 1000000000.0, // 1000000000.0 for nanosecond conversions CGONanosecondsPerSecond: 1000000000.0, // 1000000000.0 for nanosecond conversions
// Frontend Constants // Batch Processing Constants - Optimized for quality change bursts
FrontendOperationDebounceMS: 1000, // 1000ms debounce for frontend operations BatchProcessorFramesPerBatch: 16, // Larger batches for quality changes
FrontendSyncDebounceMS: 1000, // 1000ms debounce for sync operations BatchProcessorTimeout: 20 * time.Millisecond, // Longer timeout for bursts
FrontendSampleRate: 48000, // 48000Hz sample rate for frontend audio BatchProcessorMaxQueueSize: 64, // Larger queue for quality changes
FrontendRetryDelayMS: 500, // 500ms retry delay BatchProcessorAdaptiveThreshold: 0.6, // Lower threshold for faster adaptation
FrontendShortDelayMS: 200, // 200ms short delay BatchProcessorThreadPinningThreshold: 8, // Lower threshold for better performance
FrontendLongDelayMS: 300, // 300ms long delay
FrontendSyncDelayMS: 500, // 500ms sync delay
FrontendMaxRetryAttempts: 3, // 3 maximum retry attempts
FrontendAudioLevelUpdateMS: 100, // 100ms audio level update interval
FrontendFFTSize: 256, // 256 FFT size for audio analysis
FrontendAudioLevelMax: 100, // 100 maximum audio level
FrontendReconnectIntervalMS: 3000, // 3000ms reconnect interval
FrontendSubscriptionDelayMS: 100, // 100ms subscription delay
FrontendDebugIntervalMS: 5000, // 5000ms debug interval
// Process Monitor Constants // Output Streaming Constants - Balanced for stability
ProcessMonitorDefaultMemoryGB: 4, // 4GB default memory for fallback OutputStreamingFrameIntervalMS: 20, // 20ms frame interval (50 FPS) for stability
ProcessMonitorKBToBytes: 1024, // 1024 conversion factor
ProcessMonitorDefaultClockHz: 250.0, // 250.0 Hz default for ARM systems
ProcessMonitorFallbackClockHz: 1000.0, // 1000.0 Hz fallback clock
ProcessMonitorTraditionalHz: 100.0, // 100.0 Hz traditional clock
// Batch Processing Constants
BatchProcessorFramesPerBatch: 4, // 4 frames per batch
BatchProcessorTimeout: 5 * time.Millisecond, // 5ms timeout
BatchProcessorMaxQueueSize: 16, // 16 max queue size for balanced memory/performance
BatchProcessorAdaptiveThreshold: 0.8, // 0.8 threshold for adaptive batching (80% queue full)
BatchProcessorThreadPinningThreshold: 8, // 8 frames minimum for thread pinning optimization
// Output Streaming Constants
OutputStreamingFrameIntervalMS: 20, // 20ms frame interval (50 FPS)
// IPC Constants // IPC Constants
IPCInitialBufferFrames: 500, // 500 frames for initial buffer IPCInitialBufferFrames: 500, // 500 frames for initial buffer
// Event Constants // Event Constants - Balanced for stability
EventTimeoutSeconds: 2, // 2 seconds for event timeout EventTimeoutSeconds: 2, // 2 seconds for event timeout
EventTimeFormatString: "2006-01-02T15:04:05.000Z", // "2006-01-02T15:04:05.000Z" time format EventTimeFormatString: "2006-01-02T15:04:05.000Z", // "2006-01-02T15:04:05.000Z" time format
EventSubscriptionDelayMS: 100, // 100ms subscription delay EventSubscriptionDelayMS: 100, // 100ms subscription delay
@ -585,7 +569,7 @@ func DefaultAudioConfig() *AudioConfigConstants {
AudioReaderQueueSize: 32, // 32 tasks queue size for reader pool AudioReaderQueueSize: 32, // 32 tasks queue size for reader pool
WorkerMaxIdleTime: 60 * time.Second, // 60s maximum idle time before worker termination WorkerMaxIdleTime: 60 * time.Second, // 60s maximum idle time before worker termination
// Input Processing Constants // Input Processing Constants - Balanced for stability
InputProcessingTimeoutMS: 10, // 10ms processing timeout threshold InputProcessingTimeoutMS: 10, // 10ms processing timeout threshold
// Adaptive Buffer Constants // Adaptive Buffer Constants
@ -614,8 +598,14 @@ func DefaultAudioConfig() *AudioConfigConstants {
LatencyPercentile95: 95, // 95th percentile calculation factor LatencyPercentile95: 95, // 95th percentile calculation factor
LatencyPercentile99: 99, // 99th percentile calculation factor LatencyPercentile99: 99, // 99th percentile calculation factor
// Buffer Pool Configuration
BufferPoolDefaultSize: 64, // Default buffer pool size when MaxPoolSize is invalid
BufferPoolControlSize: 512, // Control buffer pool size
ZeroCopyPreallocSizeBytes: 1024 * 1024, // Zero-copy frame pool preallocation size in bytes (1MB)
ZeroCopyMinPreallocFrames: 1, // Minimum preallocated frames for zero-copy pool
BufferPoolHitRateBase: 100.0, // Base for hit rate percentage calculation
// Buffer Pool Efficiency Constants // Buffer Pool Efficiency Constants
BufferPoolMaxOperations: 1000, // 1000 operations for efficiency tracking
HitRateCalculationBase: 100.0, // 100.0 base for hit rate percentage calculation HitRateCalculationBase: 100.0, // 100.0 base for hit rate percentage calculation
// Validation Constants // Validation Constants
@ -646,8 +636,6 @@ func DefaultAudioConfig() *AudioConfigConstants {
MinFrameSize: 1, // 1 byte minimum frame size (allow small frames) MinFrameSize: 1, // 1 byte minimum frame size (allow small frames)
FrameSizeTolerance: 512, // 512 bytes frame size tolerance FrameSizeTolerance: 512, // 512 bytes frame size tolerance
// Removed device health monitoring configuration - functionality not used
// Latency Histogram Bucket Configuration // Latency Histogram Bucket Configuration
LatencyBucket10ms: 10 * time.Millisecond, // 10ms latency bucket LatencyBucket10ms: 10 * time.Millisecond, // 10ms latency bucket
LatencyBucket25ms: 25 * time.Millisecond, // 25ms latency bucket LatencyBucket25ms: 25 * time.Millisecond, // 25ms latency bucket
@ -661,16 +649,13 @@ func DefaultAudioConfig() *AudioConfigConstants {
// Batch Audio Processing Configuration // Batch Audio Processing Configuration
MinBatchSizeForThreadPinning: 5, // Minimum batch size to pin thread MinBatchSizeForThreadPinning: 5, // Minimum batch size to pin thread
// Goroutine Monitoring Configuration
GoroutineMonitorInterval: 30 * time.Second, // 30s monitoring interval
// Performance Configuration Flags - Production optimizations // Performance Configuration Flags - Production optimizations
} }
} }
// Global configuration instance // Global configuration instance
var audioConfigInstance = DefaultAudioConfig() var Config = DefaultAudioConfig()
// UpdateConfig allows runtime configuration updates // UpdateConfig allows runtime configuration updates
func UpdateConfig(newConfig *AudioConfigConstants) { func UpdateConfig(newConfig *AudioConfigConstants) {
@ -682,12 +667,12 @@ func UpdateConfig(newConfig *AudioConfigConstants) {
return return
} }
audioConfigInstance = newConfig Config = newConfig
logger := logging.GetDefaultLogger().With().Str("component", "AudioConfig").Logger() logger := logging.GetDefaultLogger().With().Str("component", "AudioConfig").Logger()
logger.Info().Msg("Audio configuration updated successfully") logger.Info().Msg("Audio configuration updated successfully")
} }
// GetConfig returns the current configuration // GetConfig returns the current configuration
func GetConfig() *AudioConfigConstants { func GetConfig() *AudioConfigConstants {
return audioConfigInstance return Config
} }

View File

@ -29,11 +29,9 @@ func (s *AudioControlService) MuteAudio(muted bool) error {
supervisor := GetAudioOutputSupervisor() supervisor := GetAudioOutputSupervisor()
if supervisor != nil { if supervisor != nil {
supervisor.Stop() supervisor.Stop()
s.logger.Info().Msg("audio output supervisor stopped")
} }
StopAudioRelay() StopAudioRelay()
SetAudioMuted(true) SetAudioMuted(true)
s.logger.Info().Msg("audio output muted (subprocess and relay stopped)")
} else { } else {
// Unmute: Start audio output subprocess and relay // Unmute: Start audio output subprocess and relay
if !s.sessionProvider.IsSessionActive() { if !s.sessionProvider.IsSessionActive() {
@ -44,10 +42,9 @@ func (s *AudioControlService) MuteAudio(muted bool) error {
if supervisor != nil { if supervisor != nil {
err := supervisor.Start() err := supervisor.Start()
if err != nil { if err != nil {
s.logger.Error().Err(err).Msg("failed to start audio output supervisor during unmute") s.logger.Debug().Err(err).Msg("failed to start audio output supervisor")
return err return err
} }
s.logger.Info().Msg("audio output supervisor started")
} }
// Start audio relay // Start audio relay

View File

@ -158,78 +158,6 @@ var (
}, },
) )
// Audio subprocess process metrics
audioProcessCpuPercent = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_audio_process_cpu_percent",
Help: "CPU usage percentage of audio output subprocess",
},
)
audioProcessMemoryPercent = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_audio_process_memory_percent",
Help: "Memory usage percentage of audio output subprocess",
},
)
audioProcessMemoryRssBytes = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_audio_process_memory_rss_bytes",
Help: "RSS memory usage in bytes of audio output subprocess",
},
)
audioProcessMemoryVmsBytes = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_audio_process_memory_vms_bytes",
Help: "VMS memory usage in bytes of audio output subprocess",
},
)
audioProcessRunning = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_audio_process_running",
Help: "Whether audio output subprocess is running (1=running, 0=stopped)",
},
)
// Microphone subprocess process metrics
microphoneProcessCpuPercent = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_microphone_process_cpu_percent",
Help: "CPU usage percentage of microphone input subprocess",
},
)
microphoneProcessMemoryPercent = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_microphone_process_memory_percent",
Help: "Memory usage percentage of microphone input subprocess",
},
)
microphoneProcessMemoryRssBytes = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_microphone_process_memory_rss_bytes",
Help: "RSS memory usage in bytes of microphone input subprocess",
},
)
microphoneProcessMemoryVmsBytes = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_microphone_process_memory_vms_bytes",
Help: "VMS memory usage in bytes of microphone input subprocess",
},
)
microphoneProcessRunning = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "jetkvm_microphone_process_running",
Help: "Whether microphone input subprocess is running (1=running, 0=stopped)",
},
)
// Device health metrics // Device health metrics
// Removed device health metrics - functionality not used // Removed device health metrics - functionality not used
@ -446,42 +374,6 @@ func UpdateMicrophoneMetrics(metrics UnifiedAudioMetrics) {
atomic.StoreInt64(&lastMetricsUpdate, time.Now().Unix()) atomic.StoreInt64(&lastMetricsUpdate, time.Now().Unix())
} }
// UpdateAudioProcessMetrics updates Prometheus metrics with audio subprocess data
func UpdateAudioProcessMetrics(metrics ProcessMetrics, isRunning bool) {
metricsUpdateMutex.Lock()
defer metricsUpdateMutex.Unlock()
audioProcessCpuPercent.Set(metrics.CPUPercent)
audioProcessMemoryPercent.Set(metrics.MemoryPercent)
audioProcessMemoryRssBytes.Set(float64(metrics.MemoryRSS))
audioProcessMemoryVmsBytes.Set(float64(metrics.MemoryVMS))
if isRunning {
audioProcessRunning.Set(1)
} else {
audioProcessRunning.Set(0)
}
atomic.StoreInt64(&lastMetricsUpdate, time.Now().Unix())
}
// UpdateMicrophoneProcessMetrics updates Prometheus metrics with microphone subprocess data
func UpdateMicrophoneProcessMetrics(metrics ProcessMetrics, isRunning bool) {
metricsUpdateMutex.Lock()
defer metricsUpdateMutex.Unlock()
microphoneProcessCpuPercent.Set(metrics.CPUPercent)
microphoneProcessMemoryPercent.Set(metrics.MemoryPercent)
microphoneProcessMemoryRssBytes.Set(float64(metrics.MemoryRSS))
microphoneProcessMemoryVmsBytes.Set(float64(metrics.MemoryVMS))
if isRunning {
microphoneProcessRunning.Set(1)
} else {
microphoneProcessRunning.Set(0)
}
atomic.StoreInt64(&lastMetricsUpdate, time.Now().Unix())
}
// UpdateAdaptiveBufferMetrics updates Prometheus metrics with adaptive buffer information // UpdateAdaptiveBufferMetrics updates Prometheus metrics with adaptive buffer information
func UpdateAdaptiveBufferMetrics(inputBufferSize, outputBufferSize int, cpuPercent, memoryPercent float64, adjustmentMade bool) { func UpdateAdaptiveBufferMetrics(inputBufferSize, outputBufferSize int, cpuPercent, memoryPercent float64, adjustmentMade bool) {
metricsUpdateMutex.Lock() metricsUpdateMutex.Lock()
@ -514,8 +406,7 @@ func UpdateSocketBufferMetrics(component, bufferType string, size, utilization f
atomic.StoreInt64(&lastMetricsUpdate, time.Now().Unix()) atomic.StoreInt64(&lastMetricsUpdate, time.Now().Unix())
} }
// UpdateDeviceHealthMetrics - Device health monitoring functionality has been removed // UpdateDeviceHealthMetrics - Placeholder for future device health metrics
// This function is no longer used as device health monitoring is not implemented
// UpdateMemoryMetrics updates memory metrics // UpdateMemoryMetrics updates memory metrics
func UpdateMemoryMetrics() { func UpdateMemoryMetrics() {

View File

@ -55,12 +55,11 @@ func ValidateZeroCopyFrame(frame *ZeroCopyAudioFrame) error {
maxFrameSize := cachedMaxFrameSize maxFrameSize := cachedMaxFrameSize
if maxFrameSize == 0 { if maxFrameSize == 0 {
// Fallback: get from cache // Fallback: get from cache
cache := GetCachedConfig() cache := Config
maxFrameSize = int(cache.maxAudioFrameSize.Load()) maxFrameSize = cache.MaxAudioFrameSize
if maxFrameSize == 0 { if maxFrameSize == 0 {
// Last resort: update cache // Last resort: use default
cache.Update() maxFrameSize = cache.MaxAudioFrameSize
maxFrameSize = int(cache.maxAudioFrameSize.Load())
} }
// Cache globally for next calls // Cache globally for next calls
cachedMaxFrameSize = maxFrameSize cachedMaxFrameSize = maxFrameSize
@ -73,28 +72,15 @@ func ValidateZeroCopyFrame(frame *ZeroCopyAudioFrame) error {
} }
// ValidateBufferSize validates buffer size parameters with enhanced boundary checks // ValidateBufferSize validates buffer size parameters with enhanced boundary checks
// Optimized to use AudioConfigCache for frequently accessed values // Optimized for minimal overhead in hotpath
func ValidateBufferSize(size int) error { func ValidateBufferSize(size int) error {
if size <= 0 { if size <= 0 {
return fmt.Errorf("%w: buffer size %d must be positive", ErrInvalidBufferSize, size) return fmt.Errorf("%w: buffer size %d must be positive", ErrInvalidBufferSize, size)
} }
// Single boundary check using pre-cached value
// Fast path: Check against cached max frame size if size > Config.SocketMaxBuffer {
cache := GetCachedConfig()
maxFrameSize := int(cache.maxAudioFrameSize.Load())
// Most common case: validating a buffer that's sized for audio frames
if maxFrameSize > 0 && size <= maxFrameSize {
return nil
}
// Slower path: full validation against SocketMaxBuffer
config := GetConfig()
// Use SocketMaxBuffer as the upper limit for general buffer validation
// This allows for socket buffers while still preventing extremely large allocations
if size > config.SocketMaxBuffer {
return fmt.Errorf("%w: buffer size %d exceeds maximum %d", return fmt.Errorf("%w: buffer size %d exceeds maximum %d",
ErrInvalidBufferSize, size, config.SocketMaxBuffer) ErrInvalidBufferSize, size, Config.SocketMaxBuffer)
} }
return nil return nil
} }
@ -107,8 +93,8 @@ func ValidateLatency(latency time.Duration) error {
} }
// Fast path: check against cached max latency // Fast path: check against cached max latency
cache := GetCachedConfig() cache := Config
maxLatency := time.Duration(cache.maxLatency.Load()) maxLatency := time.Duration(cache.MaxLatency)
// If we have a valid cached value, use it // If we have a valid cached value, use it
if maxLatency > 0 { if maxLatency > 0 {
@ -124,16 +110,14 @@ func ValidateLatency(latency time.Duration) error {
return nil return nil
} }
// Slower path: full validation with GetConfig()
config := GetConfig()
minLatency := time.Millisecond // Minimum reasonable latency minLatency := time.Millisecond // Minimum reasonable latency
if latency > 0 && latency < minLatency { if latency > 0 && latency < minLatency {
return fmt.Errorf("%w: latency %v below minimum %v", return fmt.Errorf("%w: latency %v below minimum %v",
ErrInvalidLatency, latency, minLatency) ErrInvalidLatency, latency, minLatency)
} }
if latency > config.MaxLatency { if latency > Config.MaxLatency {
return fmt.Errorf("%w: latency %v exceeds maximum %v", return fmt.Errorf("%w: latency %v exceeds maximum %v",
ErrInvalidLatency, latency, config.MaxLatency) ErrInvalidLatency, latency, Config.MaxLatency)
} }
return nil return nil
} }
@ -142,9 +126,9 @@ func ValidateLatency(latency time.Duration) error {
// Optimized to use AudioConfigCache for frequently accessed values // Optimized to use AudioConfigCache for frequently accessed values
func ValidateMetricsInterval(interval time.Duration) error { func ValidateMetricsInterval(interval time.Duration) error {
// Fast path: check against cached values // Fast path: check against cached values
cache := GetCachedConfig() cache := Config
minInterval := time.Duration(cache.minMetricsUpdateInterval.Load()) minInterval := time.Duration(cache.MinMetricsUpdateInterval)
maxInterval := time.Duration(cache.maxMetricsUpdateInterval.Load()) maxInterval := time.Duration(cache.MaxMetricsUpdateInterval)
// If we have valid cached values, use them // If we have valid cached values, use them
if minInterval > 0 && maxInterval > 0 { if minInterval > 0 && maxInterval > 0 {
@ -159,10 +143,8 @@ func ValidateMetricsInterval(interval time.Duration) error {
return nil return nil
} }
// Slower path: full validation with GetConfig() minInterval = Config.MinMetricsUpdateInterval
config := GetConfig() maxInterval = Config.MaxMetricsUpdateInterval
minInterval = config.MinMetricsUpdateInterval
maxInterval = config.MaxMetricsUpdateInterval
if interval < minInterval { if interval < minInterval {
return ErrInvalidMetricsInterval return ErrInvalidMetricsInterval
} }
@ -184,7 +166,7 @@ func ValidateAdaptiveBufferConfig(minSize, maxSize, defaultSize int) error {
return ErrInvalidBufferSize return ErrInvalidBufferSize
} }
// Validate against global limits // Validate against global limits
maxBuffer := GetConfig().SocketMaxBuffer maxBuffer := Config.SocketMaxBuffer
if maxSize > maxBuffer { if maxSize > maxBuffer {
return ErrInvalidBufferSize return ErrInvalidBufferSize
} }
@ -193,11 +175,9 @@ func ValidateAdaptiveBufferConfig(minSize, maxSize, defaultSize int) error {
// ValidateInputIPCConfig validates input IPC configuration // ValidateInputIPCConfig validates input IPC configuration
func ValidateInputIPCConfig(sampleRate, channels, frameSize int) error { func ValidateInputIPCConfig(sampleRate, channels, frameSize int) error {
// Use config values minSampleRate := Config.MinSampleRate
config := GetConfig() maxSampleRate := Config.MaxSampleRate
minSampleRate := config.MinSampleRate maxChannels := Config.MaxChannels
maxSampleRate := config.MaxSampleRate
maxChannels := config.MaxChannels
if sampleRate < minSampleRate || sampleRate > maxSampleRate { if sampleRate < minSampleRate || sampleRate > maxSampleRate {
return ErrInvalidSampleRate return ErrInvalidSampleRate
} }
@ -212,11 +192,9 @@ func ValidateInputIPCConfig(sampleRate, channels, frameSize int) error {
// ValidateOutputIPCConfig validates output IPC configuration // ValidateOutputIPCConfig validates output IPC configuration
func ValidateOutputIPCConfig(sampleRate, channels, frameSize int) error { func ValidateOutputIPCConfig(sampleRate, channels, frameSize int) error {
// Use config values minSampleRate := Config.MinSampleRate
config := GetConfig() maxSampleRate := Config.MaxSampleRate
minSampleRate := config.MinSampleRate maxChannels := Config.MaxChannels
maxSampleRate := config.MaxSampleRate
maxChannels := config.MaxChannels
if sampleRate < minSampleRate || sampleRate > maxSampleRate { if sampleRate < minSampleRate || sampleRate > maxSampleRate {
return ErrInvalidSampleRate return ErrInvalidSampleRate
} }
@ -229,130 +207,51 @@ func ValidateOutputIPCConfig(sampleRate, channels, frameSize int) error {
return nil return nil
} }
// ValidateLatencyConfig validates latency monitor configuration
func ValidateLatencyConfig(config LatencyConfig) error {
if err := ValidateLatency(config.TargetLatency); err != nil {
return err
}
if err := ValidateLatency(config.MaxLatency); err != nil {
return err
}
if config.TargetLatency >= config.MaxLatency {
return ErrInvalidLatency
}
if err := ValidateMetricsInterval(config.OptimizationInterval); err != nil {
return err
}
if config.HistorySize <= 0 {
return ErrInvalidBufferSize
}
if config.JitterThreshold < 0 {
return ErrInvalidLatency
}
if config.AdaptiveThreshold < 0 || config.AdaptiveThreshold > 1.0 {
return ErrInvalidConfiguration
}
return nil
}
// ValidateSampleRate validates audio sample rate values // ValidateSampleRate validates audio sample rate values
// Optimized to use AudioConfigCache for frequently accessed values // Optimized for minimal overhead in hotpath
func ValidateSampleRate(sampleRate int) error { func ValidateSampleRate(sampleRate int) error {
if sampleRate <= 0 { if sampleRate <= 0 {
return fmt.Errorf("%w: sample rate %d must be positive", ErrInvalidSampleRate, sampleRate) return fmt.Errorf("%w: sample rate %d must be positive", ErrInvalidSampleRate, sampleRate)
} }
// Direct validation against valid rates
// Fast path: Check against cached sample rate first for _, rate := range Config.ValidSampleRates {
cache := GetCachedConfig()
cachedRate := int(cache.sampleRate.Load())
// Most common case: validating against the current sample rate
if sampleRate == cachedRate {
return nil
}
// Slower path: check against all valid rates
config := GetConfig()
validRates := config.ValidSampleRates
for _, rate := range validRates {
if sampleRate == rate { if sampleRate == rate {
return nil return nil
} }
} }
return fmt.Errorf("%w: sample rate %d not in supported rates %v", return fmt.Errorf("%w: sample rate %d not in valid rates %v",
ErrInvalidSampleRate, sampleRate, validRates) ErrInvalidSampleRate, sampleRate, Config.ValidSampleRates)
} }
// ValidateChannelCount validates audio channel count // ValidateChannelCount validates audio channel count
// Optimized to use AudioConfigCache for frequently accessed values // Optimized for minimal overhead in hotpath
func ValidateChannelCount(channels int) error { func ValidateChannelCount(channels int) error {
if channels <= 0 { if channels <= 0 {
return fmt.Errorf("%w: channel count %d must be positive", ErrInvalidChannels, channels) return fmt.Errorf("%w: channel count %d must be positive", ErrInvalidChannels, channels)
} }
// Direct boundary check
// Fast path: Check against cached channels first if channels > Config.MaxChannels {
cache := GetCachedConfig()
cachedChannels := int(cache.channels.Load())
// Most common case: validating against the current channel count
if channels == cachedChannels {
return nil
}
// Fast path: Check against cached max channels
cachedMaxChannels := int(cache.maxChannels.Load())
if cachedMaxChannels > 0 && channels <= cachedMaxChannels {
return nil
}
// Slow path: Update cache and validate
cache.Update()
updatedMaxChannels := int(cache.maxChannels.Load())
if channels > updatedMaxChannels {
return fmt.Errorf("%w: channel count %d exceeds maximum %d", return fmt.Errorf("%w: channel count %d exceeds maximum %d",
ErrInvalidChannels, channels, updatedMaxChannels) ErrInvalidChannels, channels, Config.MaxChannels)
} }
return nil return nil
} }
// ValidateBitrate validates audio bitrate values (expects kbps) // ValidateBitrate validates audio bitrate values (expects kbps)
// Optimized to use AudioConfigCache for frequently accessed values // Optimized for minimal overhead in hotpath
func ValidateBitrate(bitrate int) error { func ValidateBitrate(bitrate int) error {
if bitrate <= 0 { if bitrate <= 0 {
return fmt.Errorf("%w: bitrate %d must be positive", ErrInvalidBitrate, bitrate) return fmt.Errorf("%w: bitrate %d must be positive", ErrInvalidBitrate, bitrate)
} }
// Direct boundary check with single conversion
// Fast path: Check against cached bitrate values
cache := GetCachedConfig()
minBitrate := int(cache.minOpusBitrate.Load())
maxBitrate := int(cache.maxOpusBitrate.Load())
// If we have valid cached values, use them
if minBitrate > 0 && maxBitrate > 0 {
// Convert kbps to bps for comparison with config limits
bitrateInBps := bitrate * 1000 bitrateInBps := bitrate * 1000
if bitrateInBps < minBitrate { if bitrateInBps < Config.MinOpusBitrate {
return fmt.Errorf("%w: bitrate %d kbps (%d bps) below minimum %d bps", return fmt.Errorf("%w: bitrate %d kbps (%d bps) below minimum %d bps",
ErrInvalidBitrate, bitrate, bitrateInBps, minBitrate) ErrInvalidBitrate, bitrate, bitrateInBps, Config.MinOpusBitrate)
} }
if bitrateInBps > maxBitrate { if bitrateInBps > Config.MaxOpusBitrate {
return fmt.Errorf("%w: bitrate %d kbps (%d bps) exceeds maximum %d bps", return fmt.Errorf("%w: bitrate %d kbps (%d bps) exceeds maximum %d bps",
ErrInvalidBitrate, bitrate, bitrateInBps, maxBitrate) ErrInvalidBitrate, bitrate, bitrateInBps, Config.MaxOpusBitrate)
}
return nil
}
// Slower path: full validation with GetConfig()
config := GetConfig()
// Convert kbps to bps for comparison with config limits
bitrateInBps := bitrate * 1000
if bitrateInBps < config.MinOpusBitrate {
return fmt.Errorf("%w: bitrate %d kbps (%d bps) below minimum %d bps",
ErrInvalidBitrate, bitrate, bitrateInBps, config.MinOpusBitrate)
}
if bitrateInBps > config.MaxOpusBitrate {
return fmt.Errorf("%w: bitrate %d kbps (%d bps) exceeds maximum %d bps",
ErrInvalidBitrate, bitrate, bitrateInBps, config.MaxOpusBitrate)
} }
return nil return nil
} }
@ -365,11 +264,11 @@ func ValidateFrameDuration(duration time.Duration) error {
} }
// Fast path: Check against cached frame size first // Fast path: Check against cached frame size first
cache := GetCachedConfig() cache := Config
// Convert frameSize (samples) to duration for comparison // Convert frameSize (samples) to duration for comparison
cachedFrameSize := int(cache.frameSize.Load()) cachedFrameSize := cache.FrameSize
cachedSampleRate := int(cache.sampleRate.Load()) cachedSampleRate := cache.SampleRate
// Only do this calculation if we have valid cached values // Only do this calculation if we have valid cached values
if cachedFrameSize > 0 && cachedSampleRate > 0 { if cachedFrameSize > 0 && cachedSampleRate > 0 {
@ -382,8 +281,8 @@ func ValidateFrameDuration(duration time.Duration) error {
} }
// Fast path: Check against cached min/max frame duration // Fast path: Check against cached min/max frame duration
cachedMinDuration := time.Duration(cache.minFrameDuration.Load()) cachedMinDuration := time.Duration(cache.MinFrameDuration)
cachedMaxDuration := time.Duration(cache.maxFrameDuration.Load()) cachedMaxDuration := time.Duration(cache.MaxFrameDuration)
if cachedMinDuration > 0 && cachedMaxDuration > 0 { if cachedMinDuration > 0 && cachedMaxDuration > 0 {
if duration < cachedMinDuration { if duration < cachedMinDuration {
@ -397,10 +296,9 @@ func ValidateFrameDuration(duration time.Duration) error {
return nil return nil
} }
// Slow path: Update cache and validate // Slow path: Use current config values
cache.Update() updatedMinDuration := time.Duration(cache.MinFrameDuration)
updatedMinDuration := time.Duration(cache.minFrameDuration.Load()) updatedMaxDuration := time.Duration(cache.MaxFrameDuration)
updatedMaxDuration := time.Duration(cache.maxFrameDuration.Load())
if duration < updatedMinDuration { if duration < updatedMinDuration {
return fmt.Errorf("%w: frame duration %v below minimum %v", return fmt.Errorf("%w: frame duration %v below minimum %v",
@ -417,11 +315,11 @@ func ValidateFrameDuration(duration time.Duration) error {
// Uses optimized validation functions that leverage AudioConfigCache // Uses optimized validation functions that leverage AudioConfigCache
func ValidateAudioConfigComplete(config AudioConfig) error { func ValidateAudioConfigComplete(config AudioConfig) error {
// Fast path: Check if all values match the current cached configuration // Fast path: Check if all values match the current cached configuration
cache := GetCachedConfig() cache := Config
cachedSampleRate := int(cache.sampleRate.Load()) cachedSampleRate := cache.SampleRate
cachedChannels := int(cache.channels.Load()) cachedChannels := cache.Channels
cachedBitrate := int(cache.opusBitrate.Load()) / 1000 // Convert from bps to kbps cachedBitrate := cache.OpusBitrate / 1000 // Convert from bps to kbps
cachedFrameSize := int(cache.frameSize.Load()) cachedFrameSize := cache.FrameSize
// Only do this calculation if we have valid cached values // Only do this calculation if we have valid cached values
if cachedSampleRate > 0 && cachedChannels > 0 && cachedBitrate > 0 && cachedFrameSize > 0 { if cachedSampleRate > 0 && cachedChannels > 0 && cachedBitrate > 0 && cachedFrameSize > 0 {
@ -465,11 +363,11 @@ func ValidateAudioConfigConstants(config *AudioConfigConstants) error {
} }
// Validate configuration values if config is provided // Validate configuration values if config is provided
if config != nil { if config != nil {
if config.MaxFrameSize <= 0 { if Config.MaxFrameSize <= 0 {
return fmt.Errorf("invalid MaxFrameSize: %d", config.MaxFrameSize) return fmt.Errorf("invalid MaxFrameSize: %d", Config.MaxFrameSize)
} }
if config.SampleRate <= 0 { if Config.SampleRate <= 0 {
return fmt.Errorf("invalid SampleRate: %d", config.SampleRate) return fmt.Errorf("invalid SampleRate: %d", Config.SampleRate)
} }
} }
return nil return nil
@ -481,11 +379,10 @@ var cachedMaxFrameSize int
// InitValidationCache initializes cached validation values with actual config // InitValidationCache initializes cached validation values with actual config
func InitValidationCache() { func InitValidationCache() {
// Initialize the global cache variable for backward compatibility // Initialize the global cache variable for backward compatibility
config := GetConfig() cachedMaxFrameSize = Config.MaxAudioFrameSize
cachedMaxFrameSize = config.MaxAudioFrameSize
// Update the global audio config cache // Initialize the global audio config cache
GetCachedConfig().Update() cachedMaxFrameSize = Config.MaxAudioFrameSize
} }
// ValidateAudioFrame validates audio frame data with cached max size for performance // ValidateAudioFrame validates audio frame data with cached max size for performance
@ -502,12 +399,11 @@ func ValidateAudioFrame(data []byte) error {
maxSize := cachedMaxFrameSize maxSize := cachedMaxFrameSize
if maxSize == 0 { if maxSize == 0 {
// Fallback: get from cache only if global cache not initialized // Fallback: get from cache only if global cache not initialized
cache := GetCachedConfig() cache := Config
maxSize = int(cache.maxAudioFrameSize.Load()) maxSize = cache.MaxAudioFrameSize
if maxSize == 0 { if maxSize == 0 {
// Last resort: update cache and get fresh value // Last resort: get fresh value
cache.Update() maxSize = cache.MaxAudioFrameSize
maxSize = int(cache.maxAudioFrameSize.Load())
} }
// Cache the value globally for next calls // Cache the value globally for next calls
cachedMaxFrameSize = maxSize cachedMaxFrameSize = maxSize

View File

@ -65,6 +65,42 @@ func (p *GoroutinePool) Submit(task Task) bool {
} }
} }
// SubmitWithBackpressure adds a task to the pool with backpressure handling
// Returns true if task was accepted, false if dropped due to backpressure
func (p *GoroutinePool) SubmitWithBackpressure(task Task) bool {
select {
case <-p.shutdown:
return false // Pool is shutting down
case p.taskQueue <- task:
// Task accepted, ensure we have a worker to process it
p.ensureWorkerAvailable()
return true
default:
// Queue is full - apply backpressure
// Check if we're in a high-load situation
queueLen := len(p.taskQueue)
queueCap := cap(p.taskQueue)
workerCount := atomic.LoadInt64(&p.workerCount)
// If queue is >90% full and we're at max workers, drop the task
if queueLen > int(float64(queueCap)*0.9) && workerCount >= int64(p.maxWorkers) {
p.logger.Warn().Int("queue_len", queueLen).Int("queue_cap", queueCap).Msg("Dropping task due to backpressure")
return false
}
// Try one more time with a short timeout
select {
case p.taskQueue <- task:
p.ensureWorkerAvailable()
return true
case <-time.After(1 * time.Millisecond):
// Still can't submit after timeout - drop task
p.logger.Debug().Msg("Task dropped after backpressure timeout")
return false
}
}
}
// ensureWorkerAvailable makes sure at least one worker is available to process tasks // ensureWorkerAvailable makes sure at least one worker is available to process tasks
func (p *GoroutinePool) ensureWorkerAvailable() { func (p *GoroutinePool) ensureWorkerAvailable() {
// Check if we already have enough workers // Check if we already have enough workers
@ -160,7 +196,7 @@ func (p *GoroutinePool) supervisor() {
tasks := atomic.LoadInt64(&p.taskCount) tasks := atomic.LoadInt64(&p.taskCount)
queueLen := len(p.taskQueue) queueLen := len(p.taskQueue)
p.logger.Info(). p.logger.Debug().
Int64("workers", workers). Int64("workers", workers).
Int64("tasks_processed", tasks). Int64("tasks_processed", tasks).
Int("queue_length", queueLen). Int("queue_length", queueLen).
@ -179,7 +215,7 @@ func (p *GoroutinePool) Shutdown(wait bool) {
if wait { if wait {
// Wait for all tasks to be processed // Wait for all tasks to be processed
if len(p.taskQueue) > 0 { if len(p.taskQueue) > 0 {
p.logger.Info().Int("remaining_tasks", len(p.taskQueue)).Msg("Waiting for tasks to complete") p.logger.Debug().Int("remaining_tasks", len(p.taskQueue)).Msg("Waiting for tasks to complete")
} }
// Close the task queue to signal no more tasks // Close the task queue to signal no more tasks
@ -219,7 +255,7 @@ func GetAudioProcessorPool() *GoroutinePool {
} }
globalAudioProcessorInitOnce.Do(func() { globalAudioProcessorInitOnce.Do(func() {
config := GetConfig() config := Config
newPool := NewGoroutinePool( newPool := NewGoroutinePool(
"audio-processor", "audio-processor",
config.MaxAudioProcessorWorkers, config.MaxAudioProcessorWorkers,
@ -241,7 +277,7 @@ func GetAudioReaderPool() *GoroutinePool {
} }
globalAudioReaderInitOnce.Do(func() { globalAudioReaderInitOnce.Do(func() {
config := GetConfig() config := Config
newPool := NewGoroutinePool( newPool := NewGoroutinePool(
"audio-reader", "audio-reader",
config.MaxAudioReaderWorkers, config.MaxAudioReaderWorkers,
@ -265,6 +301,16 @@ func SubmitAudioReaderTask(task Task) bool {
return GetAudioReaderPool().Submit(task) return GetAudioReaderPool().Submit(task)
} }
// SubmitAudioProcessorTaskWithBackpressure submits a task with backpressure handling
func SubmitAudioProcessorTaskWithBackpressure(task Task) bool {
return GetAudioProcessorPool().SubmitWithBackpressure(task)
}
// SubmitAudioReaderTaskWithBackpressure submits a task with backpressure handling
func SubmitAudioReaderTaskWithBackpressure(task Task) bool {
return GetAudioReaderPool().SubmitWithBackpressure(task)
}
// ShutdownAudioPools shuts down all audio goroutine pools // ShutdownAudioPools shuts down all audio goroutine pools
func ShutdownAudioPools(wait bool) { func ShutdownAudioPools(wait bool) {
logger := logging.GetDefaultLogger().With().Str("component", "audio-pools").Logger() logger := logging.GetDefaultLogger().With().Str("component", "audio-pools").Logger()

View File

@ -108,14 +108,13 @@ func (aim *AudioInputManager) WriteOpusFrame(frame []byte) error {
processingTime := time.Since(startTime) processingTime := time.Since(startTime)
// Log high latency warnings // Log high latency warnings
if processingTime > time.Duration(GetConfig().InputProcessingTimeoutMS)*time.Millisecond { if processingTime > time.Duration(Config.InputProcessingTimeoutMS)*time.Millisecond {
latencyMs := float64(processingTime.Milliseconds()) latencyMs := float64(processingTime.Milliseconds())
aim.logger.Warn(). aim.logger.Warn().
Float64("latency_ms", latencyMs). Float64("latency_ms", latencyMs).
Msg("High audio processing latency detected") Msg("High audio processing latency detected")
// Record latency for goroutine cleanup optimization // Record latency for goroutine cleanup optimization
RecordAudioLatency(latencyMs)
} }
if err != nil { if err != nil {
@ -149,14 +148,13 @@ func (aim *AudioInputManager) WriteOpusFrameZeroCopy(frame *ZeroCopyAudioFrame)
processingTime := time.Since(startTime) processingTime := time.Since(startTime)
// Log high latency warnings // Log high latency warnings
if processingTime > time.Duration(GetConfig().InputProcessingTimeoutMS)*time.Millisecond { if processingTime > time.Duration(Config.InputProcessingTimeoutMS)*time.Millisecond {
latencyMs := float64(processingTime.Milliseconds()) latencyMs := float64(processingTime.Milliseconds())
aim.logger.Warn(). aim.logger.Warn().
Float64("latency_ms", latencyMs). Float64("latency_ms", latencyMs).
Msg("High audio processing latency detected") Msg("High audio processing latency detected")
// Record latency for goroutine cleanup optimization // Record latency for goroutine cleanup optimization
RecordAudioLatency(latencyMs)
} }
if err != nil { if err != nil {

View File

@ -19,6 +19,28 @@ import (
"github.com/jetkvm/kvm/internal/logging" "github.com/jetkvm/kvm/internal/logging"
) )
// Global audio input server instance
var globalAudioInputServer *AudioInputServer
// GetGlobalAudioInputServer returns the global audio input server instance
func GetGlobalAudioInputServer() *AudioInputServer {
return globalAudioInputServer
}
// ResetGlobalAudioInputServerStats resets the global audio input server stats
func ResetGlobalAudioInputServerStats() {
if globalAudioInputServer != nil {
globalAudioInputServer.ResetServerStats()
}
}
// RecoverGlobalAudioInputServer attempts to recover from dropped frames
func RecoverGlobalAudioInputServer() {
if globalAudioInputServer != nil {
globalAudioInputServer.RecoverFromDroppedFrames()
}
}
// getEnvInt reads an integer from environment variable with a default value // getEnvInt reads an integer from environment variable with a default value
// RunAudioInputServer runs the audio input server subprocess // RunAudioInputServer runs the audio input server subprocess
@ -33,10 +55,6 @@ func RunAudioInputServer() error {
// Initialize validation cache for optimal performance // Initialize validation cache for optimal performance
InitValidationCache() InitValidationCache()
// Start adaptive buffer management for optimal performance
StartAdaptiveBuffering()
defer StopAdaptiveBuffering()
// Initialize CGO audio playback (optional for input server) // Initialize CGO audio playback (optional for input server)
// This is used for audio loopback/monitoring features // This is used for audio loopback/monitoring features
err := CGOAudioPlaybackInit() err := CGOAudioPlaybackInit()
@ -56,6 +74,9 @@ func RunAudioInputServer() error {
} }
defer server.Close() defer server.Close()
// Store globally for access by other functions
globalAudioInputServer = server
err = server.Start() err = server.Start()
if err != nil { if err != nil {
logger.Error().Err(err).Msg("failed to start audio input server") logger.Error().Err(err).Msg("failed to start audio input server")
@ -82,7 +103,7 @@ func RunAudioInputServer() error {
server.Stop() server.Stop()
// Give some time for cleanup // Give some time for cleanup
time.Sleep(GetConfig().DefaultSleepDuration) time.Sleep(Config.DefaultSleepDuration)
return nil return nil
} }

View File

@ -73,7 +73,7 @@ func (ais *AudioInputSupervisor) supervisionLoop() {
// Configure supervision parameters (no restart for input supervisor) // Configure supervision parameters (no restart for input supervisor)
config := SupervisionConfig{ config := SupervisionConfig{
ProcessType: "audio input server", ProcessType: "audio input server",
Timeout: GetConfig().InputSupervisorTimeout, Timeout: Config.InputSupervisorTimeout,
EnableRestart: false, // Input supervisor doesn't restart EnableRestart: false, // Input supervisor doesn't restart
MaxRestartAttempts: 0, MaxRestartAttempts: 0,
RestartWindow: 0, RestartWindow: 0,
@ -135,10 +135,9 @@ func (ais *AudioInputSupervisor) startProcess() error {
ais.logger.Info().Int("pid", ais.processPID).Strs("args", args).Strs("opus_env", ais.opusEnv).Msg("audio input server process started") ais.logger.Info().Int("pid", ais.processPID).Strs("args", args).Strs("opus_env", ais.opusEnv).Msg("audio input server process started")
// Add process to monitoring // Add process to monitoring
ais.processMonitor.AddProcess(ais.processPID, "audio-input-server")
// Connect client to the server // Connect client to the server synchronously to avoid race condition
go ais.connectClient() ais.connectClient()
return nil return nil
} }
@ -164,7 +163,7 @@ func (ais *AudioInputSupervisor) Stop() {
select { select {
case <-ais.processDone: case <-ais.processDone:
ais.logger.Info().Str("component", "audio-input-supervisor").Msg("component stopped gracefully") ais.logger.Info().Str("component", "audio-input-supervisor").Msg("component stopped gracefully")
case <-time.After(GetConfig().InputSupervisorTimeout): case <-time.After(Config.InputSupervisorTimeout):
ais.logger.Warn().Str("component", "audio-input-supervisor").Msg("component did not stop gracefully, forcing termination") ais.logger.Warn().Str("component", "audio-input-supervisor").Msg("component did not stop gracefully, forcing termination")
ais.forceKillProcess("audio input server") ais.forceKillProcess("audio input server")
} }
@ -190,7 +189,7 @@ func (ais *AudioInputSupervisor) GetClient() *AudioInputClient {
// connectClient attempts to connect the client to the server // connectClient attempts to connect the client to the server
func (ais *AudioInputSupervisor) connectClient() { func (ais *AudioInputSupervisor) connectClient() {
// Wait briefly for the server to start and create socket // Wait briefly for the server to start and create socket
time.Sleep(GetConfig().DefaultSleepDuration) time.Sleep(Config.DefaultSleepDuration)
// Additional small delay to ensure socket is ready after restart // Additional small delay to ensure socket is ready after restart
time.Sleep(20 * time.Millisecond) time.Sleep(20 * time.Millisecond)

View File

@ -49,7 +49,7 @@ func NewGenericMessagePool(size int) *GenericMessagePool {
pool.preallocated = make([]*OptimizedMessage, pool.preallocSize) pool.preallocated = make([]*OptimizedMessage, pool.preallocSize)
for i := 0; i < pool.preallocSize; i++ { for i := 0; i < pool.preallocSize; i++ {
pool.preallocated[i] = &OptimizedMessage{ pool.preallocated[i] = &OptimizedMessage{
data: make([]byte, 0, GetConfig().MaxFrameSize), data: make([]byte, 0, Config.MaxFrameSize),
} }
} }
@ -57,7 +57,7 @@ func NewGenericMessagePool(size int) *GenericMessagePool {
for i := 0; i < size-pool.preallocSize; i++ { for i := 0; i < size-pool.preallocSize; i++ {
select { select {
case pool.pool <- &OptimizedMessage{ case pool.pool <- &OptimizedMessage{
data: make([]byte, 0, GetConfig().MaxFrameSize), data: make([]byte, 0, Config.MaxFrameSize),
}: }:
default: default:
break break
@ -89,7 +89,7 @@ func (mp *GenericMessagePool) Get() *OptimizedMessage {
// Pool empty, create new message // Pool empty, create new message
atomic.AddInt64(&mp.missCount, 1) atomic.AddInt64(&mp.missCount, 1)
return &OptimizedMessage{ return &OptimizedMessage{
data: make([]byte, 0, GetConfig().MaxFrameSize), data: make([]byte, 0, Config.MaxFrameSize),
} }
} }
} }
@ -132,6 +132,42 @@ func (mp *GenericMessagePool) GetStats() (hitCount, missCount int64, hitRate flo
return hits, misses, hitRate return hits, misses, hitRate
} }
// Helper functions
// EncodeMessageHeader encodes a message header into a byte slice
func EncodeMessageHeader(magic uint32, msgType uint8, length uint32, timestamp int64) []byte {
header := make([]byte, 17)
binary.LittleEndian.PutUint32(header[0:4], magic)
header[4] = msgType
binary.LittleEndian.PutUint32(header[5:9], length)
binary.LittleEndian.PutUint64(header[9:17], uint64(timestamp))
return header
}
// EncodeAudioConfig encodes basic audio configuration to binary format
func EncodeAudioConfig(sampleRate, channels, frameSize int) []byte {
data := make([]byte, 12) // 3 * int32
binary.LittleEndian.PutUint32(data[0:4], uint32(sampleRate))
binary.LittleEndian.PutUint32(data[4:8], uint32(channels))
binary.LittleEndian.PutUint32(data[8:12], uint32(frameSize))
return data
}
// EncodeOpusConfig encodes complete Opus configuration to binary format
func EncodeOpusConfig(sampleRate, channels, frameSize, bitrate, complexity, vbr, signalType, bandwidth, dtx int) []byte {
data := make([]byte, 36) // 9 * int32
binary.LittleEndian.PutUint32(data[0:4], uint32(sampleRate))
binary.LittleEndian.PutUint32(data[4:8], uint32(channels))
binary.LittleEndian.PutUint32(data[8:12], uint32(frameSize))
binary.LittleEndian.PutUint32(data[12:16], uint32(bitrate))
binary.LittleEndian.PutUint32(data[16:20], uint32(complexity))
binary.LittleEndian.PutUint32(data[20:24], uint32(vbr))
binary.LittleEndian.PutUint32(data[24:28], uint32(signalType))
binary.LittleEndian.PutUint32(data[28:32], uint32(bandwidth))
binary.LittleEndian.PutUint32(data[32:36], uint32(dtx))
return data
}
// Common write message function // Common write message function
func WriteIPCMessage(conn net.Conn, msg IPCMessage, pool *GenericMessagePool, droppedFramesCounter *int64) error { func WriteIPCMessage(conn net.Conn, msg IPCMessage, pool *GenericMessagePool, droppedFramesCounter *int64) error {
if conn == nil { if conn == nil {
@ -143,13 +179,11 @@ func WriteIPCMessage(conn net.Conn, msg IPCMessage, pool *GenericMessagePool, dr
defer pool.Put(optMsg) defer pool.Put(optMsg)
// Prepare header in pre-allocated buffer // Prepare header in pre-allocated buffer
binary.LittleEndian.PutUint32(optMsg.header[0:4], msg.GetMagic()) header := EncodeMessageHeader(msg.GetMagic(), msg.GetType(), msg.GetLength(), msg.GetTimestamp())
optMsg.header[4] = msg.GetType() copy(optMsg.header[:], header)
binary.LittleEndian.PutUint32(optMsg.header[5:9], msg.GetLength())
binary.LittleEndian.PutUint64(optMsg.header[9:17], uint64(msg.GetTimestamp()))
// Set write deadline for timeout handling (more efficient than goroutines) // Set write deadline for timeout handling (more efficient than goroutines)
if deadline := time.Now().Add(GetConfig().WriteTimeout); deadline.After(time.Now()) { if deadline := time.Now().Add(Config.WriteTimeout); deadline.After(time.Now()) {
if err := conn.SetWriteDeadline(deadline); err != nil { if err := conn.SetWriteDeadline(deadline); err != nil {
// If we can't set deadline, proceed without it // If we can't set deadline, proceed without it
// This maintains compatibility with connections that don't support deadlines // This maintains compatibility with connections that don't support deadlines

View File

@ -23,8 +23,8 @@ const (
// Constants are now defined in unified_ipc.go // Constants are now defined in unified_ipc.go
var ( var (
maxFrameSize = GetConfig().MaxFrameSize // Maximum Opus frame size maxFrameSize = Config.MaxFrameSize // Maximum Opus frame size
messagePoolSize = GetConfig().MessagePoolSize // Pre-allocated message pool size messagePoolSize = Config.MessagePoolSize // Pre-allocated message pool size
) )
// Legacy aliases for backward compatibility // Legacy aliases for backward compatibility
@ -77,7 +77,7 @@ func initializeMessagePool() {
messagePoolInitOnce.Do(func() { messagePoolInitOnce.Do(func() {
preallocSize := messagePoolSize / 4 // 25% pre-allocated for immediate use preallocSize := messagePoolSize / 4 // 25% pre-allocated for immediate use
globalMessagePool.preallocSize = preallocSize globalMessagePool.preallocSize = preallocSize
globalMessagePool.maxPoolSize = messagePoolSize * GetConfig().PoolGrowthMultiplier // Allow growth up to 2x globalMessagePool.maxPoolSize = messagePoolSize * Config.PoolGrowthMultiplier // Allow growth up to 2x
globalMessagePool.preallocated = make([]*OptimizedIPCMessage, 0, preallocSize) globalMessagePool.preallocated = make([]*OptimizedIPCMessage, 0, preallocSize)
// Pre-allocate messages for immediate use // Pre-allocate messages for immediate use
@ -191,6 +191,10 @@ type AudioInputServer struct {
stopChan chan struct{} // Stop signal for all goroutines stopChan chan struct{} // Stop signal for all goroutines
wg sync.WaitGroup // Wait group for goroutine coordination wg sync.WaitGroup // Wait group for goroutine coordination
// Channel resizing support
channelMutex sync.RWMutex // Protects channel recreation
lastBufferSize int64 // Last known buffer size for change detection
// Socket buffer configuration // Socket buffer configuration
socketBufferConfig SocketBufferConfig socketBufferConfig SocketBufferConfig
} }
@ -227,9 +231,15 @@ func NewAudioInputServer() (*AudioInputServer, error) {
return nil, fmt.Errorf("failed to create unix socket after 3 attempts: %w", err) return nil, fmt.Errorf("failed to create unix socket after 3 attempts: %w", err)
} }
// Get initial buffer size from adaptive buffer manager // Get initial buffer size from config
adaptiveManager := GetAdaptiveBufferManager() initialBufferSize := int64(Config.AdaptiveDefaultBufferSize)
initialBufferSize := int64(adaptiveManager.GetInputBufferSize())
// Ensure minimum buffer size to prevent immediate overflow
// Use at least 50 frames to handle burst traffic
minBufferSize := int64(50)
if initialBufferSize < minBufferSize {
initialBufferSize = minBufferSize
}
// Initialize socket buffer configuration // Initialize socket buffer configuration
socketBufferConfig := DefaultSocketBufferConfig() socketBufferConfig := DefaultSocketBufferConfig()
@ -240,6 +250,7 @@ func NewAudioInputServer() (*AudioInputServer, error) {
processChan: make(chan *InputIPCMessage, initialBufferSize), processChan: make(chan *InputIPCMessage, initialBufferSize),
stopChan: make(chan struct{}), stopChan: make(chan struct{}),
bufferSize: initialBufferSize, bufferSize: initialBufferSize,
lastBufferSize: initialBufferSize,
socketBufferConfig: socketBufferConfig, socketBufferConfig: socketBufferConfig,
}, nil }, nil
} }
@ -366,7 +377,7 @@ func (ais *AudioInputServer) handleConnection(conn net.Conn) {
if ais.conn == nil { if ais.conn == nil {
return return
} }
time.Sleep(GetConfig().DefaultSleepDuration) time.Sleep(Config.DefaultSleepDuration)
} }
} }
} }
@ -487,11 +498,11 @@ func (ais *AudioInputServer) processOpusFrame(data []byte) error {
} }
// Get cached config once - avoid repeated calls and locking // Get cached config once - avoid repeated calls and locking
cache := GetCachedConfig() cache := Config
// Skip cache expiry check in hotpath - background updates handle this // Skip cache expiry check in hotpath - background updates handle this
// Get a PCM buffer from the pool for optimized decode-write // Get a PCM buffer from the pool for optimized decode-write
pcmBuffer := GetBufferFromPool(cache.GetMaxPCMBufferSize()) pcmBuffer := GetBufferFromPool(cache.MaxPCMBufferSize)
defer ReturnBufferToPool(pcmBuffer) defer ReturnBufferToPool(pcmBuffer)
// Direct CGO call - avoid wrapper function overhead // Direct CGO call - avoid wrapper function overhead
@ -634,9 +645,9 @@ func (aic *AudioInputClient) Connect() error {
return nil return nil
} }
// Exponential backoff starting from config // Exponential backoff starting from config
backoffStart := GetConfig().BackoffStart backoffStart := Config.BackoffStart
delay := time.Duration(backoffStart.Nanoseconds()*(1<<uint(i/3))) * time.Nanosecond delay := time.Duration(backoffStart.Nanoseconds()*(1<<uint(i/3))) * time.Nanosecond
maxDelay := GetConfig().MaxRetryDelay maxDelay := Config.MaxRetryDelay
if delay > maxDelay { if delay > maxDelay {
delay = maxDelay delay = maxDelay
} }
@ -677,32 +688,28 @@ func (aic *AudioInputClient) Disconnect() {
// SendFrame sends an Opus frame to the audio input server // SendFrame sends an Opus frame to the audio input server
func (aic *AudioInputClient) SendFrame(frame []byte) error { func (aic *AudioInputClient) SendFrame(frame []byte) error {
// Fast path validation
if len(frame) == 0 {
return nil
}
aic.mtx.Lock() aic.mtx.Lock()
defer aic.mtx.Unlock()
if !aic.running || aic.conn == nil { if !aic.running || aic.conn == nil {
return fmt.Errorf("not connected to audio input server") aic.mtx.Unlock()
} return fmt.Errorf("not connected")
frameLen := len(frame)
if frameLen == 0 {
return nil // Empty frame, ignore
}
// Inline frame validation to reduce function call overhead
if frameLen > maxFrameSize {
return ErrFrameDataTooLarge
} }
// Direct message creation without timestamp overhead
msg := &InputIPCMessage{ msg := &InputIPCMessage{
Magic: inputMagicNumber, Magic: inputMagicNumber,
Type: InputMessageTypeOpusFrame, Type: InputMessageTypeOpusFrame,
Length: uint32(frameLen), Length: uint32(len(frame)),
Timestamp: time.Now().UnixNano(),
Data: frame, Data: frame,
} }
return aic.writeMessage(msg) err := aic.writeMessage(msg)
aic.mtx.Unlock()
return err
} }
// SendFrameZeroCopy sends a zero-copy Opus frame to the audio input server // SendFrameZeroCopy sends a zero-copy Opus frame to the audio input server
@ -756,11 +763,8 @@ func (aic *AudioInputClient) SendConfig(config InputIPCConfig) error {
return fmt.Errorf("input configuration validation failed: %w", err) return fmt.Errorf("input configuration validation failed: %w", err)
} }
// Serialize config (simple binary format) // Serialize config using common function
data := make([]byte, 12) // 3 * int32 data := EncodeAudioConfig(config.SampleRate, config.Channels, config.FrameSize)
binary.LittleEndian.PutUint32(data[0:4], uint32(config.SampleRate))
binary.LittleEndian.PutUint32(data[4:8], uint32(config.Channels))
binary.LittleEndian.PutUint32(data[8:12], uint32(config.FrameSize))
msg := &InputIPCMessage{ msg := &InputIPCMessage{
Magic: inputMagicNumber, Magic: inputMagicNumber,
@ -788,17 +792,8 @@ func (aic *AudioInputClient) SendOpusConfig(config InputIPCOpusConfig) error {
config.SampleRate, config.Channels, config.FrameSize, config.Bitrate) config.SampleRate, config.Channels, config.FrameSize, config.Bitrate)
} }
// Serialize Opus configuration (9 * int32 = 36 bytes) // Serialize Opus configuration using common function
data := make([]byte, 36) data := EncodeOpusConfig(config.SampleRate, config.Channels, config.FrameSize, config.Bitrate, config.Complexity, config.VBR, config.SignalType, config.Bandwidth, config.DTX)
binary.LittleEndian.PutUint32(data[0:4], uint32(config.SampleRate))
binary.LittleEndian.PutUint32(data[4:8], uint32(config.Channels))
binary.LittleEndian.PutUint32(data[8:12], uint32(config.FrameSize))
binary.LittleEndian.PutUint32(data[12:16], uint32(config.Bitrate))
binary.LittleEndian.PutUint32(data[16:20], uint32(config.Complexity))
binary.LittleEndian.PutUint32(data[20:24], uint32(config.VBR))
binary.LittleEndian.PutUint32(data[24:28], uint32(config.SignalType))
binary.LittleEndian.PutUint32(data[28:32], uint32(config.Bandwidth))
binary.LittleEndian.PutUint32(data[32:36], uint32(config.DTX))
msg := &InputIPCMessage{ msg := &InputIPCMessage{
Magic: inputMagicNumber, Magic: inputMagicNumber,
@ -866,6 +861,28 @@ func (aic *AudioInputClient) ResetStats() {
ResetFrameStats(&aic.totalFrames, &aic.droppedFrames) ResetFrameStats(&aic.totalFrames, &aic.droppedFrames)
} }
// ResetServerStats resets server frame statistics
func (ais *AudioInputServer) ResetServerStats() {
atomic.StoreInt64(&ais.totalFrames, 0)
atomic.StoreInt64(&ais.droppedFrames, 0)
}
// RecoverFromDroppedFrames attempts to recover when too many frames are dropped
func (ais *AudioInputServer) RecoverFromDroppedFrames() {
total := atomic.LoadInt64(&ais.totalFrames)
dropped := atomic.LoadInt64(&ais.droppedFrames)
// If more than 50% of frames are dropped, attempt recovery
if total > 100 && dropped > total/2 {
logger := logging.GetDefaultLogger().With().Str("component", AudioInputServerComponent).Logger()
logger.Warn().Int64("total", total).Int64("dropped", dropped).Msg("high drop rate detected, attempting recovery")
// Reset stats and update buffer size from adaptive manager
ais.ResetServerStats()
ais.UpdateBufferSize()
}
}
// startReaderGoroutine starts the message reader using the goroutine pool // startReaderGoroutine starts the message reader using the goroutine pool
func (ais *AudioInputServer) startReaderGoroutine() { func (ais *AudioInputServer) startReaderGoroutine() {
ais.wg.Add(1) ais.wg.Add(1)
@ -877,10 +894,10 @@ func (ais *AudioInputServer) startReaderGoroutine() {
// Enhanced error tracking and recovery // Enhanced error tracking and recovery
var consecutiveErrors int var consecutiveErrors int
var lastErrorTime time.Time var lastErrorTime time.Time
maxConsecutiveErrors := GetConfig().MaxConsecutiveErrors maxConsecutiveErrors := Config.MaxConsecutiveErrors
errorResetWindow := GetConfig().RestartWindow // Use existing restart window errorResetWindow := Config.RestartWindow // Use existing restart window
baseBackoffDelay := GetConfig().RetryDelay baseBackoffDelay := Config.RetryDelay
maxBackoffDelay := GetConfig().MaxRetryDelay maxBackoffDelay := Config.MaxRetryDelay
logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger() logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger()
@ -950,9 +967,13 @@ func (ais *AudioInputServer) startReaderGoroutine() {
} }
} }
// Send to message channel with non-blocking write // Send to message channel with non-blocking write (use read lock for channel access)
ais.channelMutex.RLock()
messageChan := ais.messageChan
ais.channelMutex.RUnlock()
select { select {
case ais.messageChan <- msg: case messageChan <- msg:
atomic.AddInt64(&ais.totalFrames, 1) atomic.AddInt64(&ais.totalFrames, 1)
default: default:
// Channel full, drop message // Channel full, drop message
@ -966,16 +987,16 @@ func (ais *AudioInputServer) startReaderGoroutine() {
} }
} }
// Submit the reader task to the audio reader pool // Submit the reader task to the audio reader pool with backpressure
logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger() logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger()
if !SubmitAudioReaderTask(readerTask) { if !SubmitAudioReaderTaskWithBackpressure(readerTask) {
// If the pool is full or shutting down, fall back to direct goroutine creation // Task was dropped due to backpressure - this is expected under high load
// Only log if warn level enabled - avoid sampling logic in critical path // Log at debug level to avoid spam, but track the drop
if logger.GetLevel() <= zerolog.WarnLevel { logger.Debug().Msg("Audio reader task dropped due to backpressure")
logger.Warn().Msg("Audio reader pool full or shutting down, falling back to direct goroutine creation")
}
go readerTask() // Don't fall back to unlimited goroutine creation
// Instead, let the system recover naturally
ais.wg.Done() // Decrement the wait group since we're not starting the task
} }
} }
@ -987,7 +1008,7 @@ func (ais *AudioInputServer) startProcessorGoroutine() {
processorTask := func() { processorTask := func() {
// Only lock OS thread and set priority for high-load scenarios // Only lock OS thread and set priority for high-load scenarios
// This reduces interference with input processing threads // This reduces interference with input processing threads
config := GetConfig() config := Config
useThreadOptimizations := config.MaxAudioProcessorWorkers > 8 useThreadOptimizations := config.MaxAudioProcessorWorkers > 8
if useThreadOptimizations { if useThreadOptimizations {
@ -1011,7 +1032,7 @@ func (ais *AudioInputServer) startProcessorGoroutine() {
select { select {
case <-ais.stopChan: case <-ais.stopChan:
return return
case msg := <-ais.messageChan: case msg := <-ais.getMessageChan():
// Process message with error handling // Process message with error handling
start := time.Now() start := time.Now()
err := ais.processMessageWithRecovery(msg, logger) err := ais.processMessageWithRecovery(msg, logger)
@ -1032,9 +1053,10 @@ func (ais *AudioInputServer) startProcessorGoroutine() {
// If too many processing errors, drop frames more aggressively // If too many processing errors, drop frames more aggressively
if processingErrors >= maxProcessingErrors { if processingErrors >= maxProcessingErrors {
// Clear processing queue to recover // Clear processing queue to recover
for len(ais.processChan) > 0 { processChan := ais.getProcessChan()
for len(processChan) > 0 {
select { select {
case <-ais.processChan: case <-processChan:
atomic.AddInt64(&ais.droppedFrames, 1) atomic.AddInt64(&ais.droppedFrames, 1)
default: default:
break break
@ -1057,13 +1079,16 @@ func (ais *AudioInputServer) startProcessorGoroutine() {
} }
} }
// Submit the processor task to the audio processor pool // Submit the processor task to the audio processor pool with backpressure
logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger() logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger()
if !SubmitAudioProcessorTask(processorTask) { if !SubmitAudioProcessorTaskWithBackpressure(processorTask) {
// If the pool is full or shutting down, fall back to direct goroutine creation // Task was dropped due to backpressure - this is expected under high load
logger.Warn().Msg("Audio processor pool full or shutting down, falling back to direct goroutine creation") // Log at debug level to avoid spam, but track the drop
logger.Debug().Msg("Audio processor task dropped due to backpressure")
go processorTask() // Don't fall back to unlimited goroutine creation
// Instead, let the system recover naturally
ais.wg.Done() // Decrement the wait group since we're not starting the task
} }
} }
@ -1072,13 +1097,14 @@ func (ais *AudioInputServer) processMessageWithRecovery(msg *InputIPCMessage, lo
// Intelligent frame dropping: prioritize recent frames // Intelligent frame dropping: prioritize recent frames
if msg.Type == InputMessageTypeOpusFrame { if msg.Type == InputMessageTypeOpusFrame {
// Check if processing queue is getting full // Check if processing queue is getting full
queueLen := len(ais.processChan) processChan := ais.getProcessChan()
queueLen := len(processChan)
bufferSize := int(atomic.LoadInt64(&ais.bufferSize)) bufferSize := int(atomic.LoadInt64(&ais.bufferSize))
if queueLen > bufferSize*3/4 { if queueLen > bufferSize*3/4 {
// Drop oldest frames, keep newest // Drop oldest frames, keep newest
select { select {
case <-ais.processChan: // Remove oldest case <-processChan: // Remove oldest
atomic.AddInt64(&ais.droppedFrames, 1) atomic.AddInt64(&ais.droppedFrames, 1)
logger.Debug().Msg("Dropped oldest frame to make room") logger.Debug().Msg("Dropped oldest frame to make room")
default: default:
@ -1086,11 +1112,15 @@ func (ais *AudioInputServer) processMessageWithRecovery(msg *InputIPCMessage, lo
} }
} }
// Send to processing queue with timeout // Send to processing queue with timeout (use read lock for channel access)
ais.channelMutex.RLock()
processChan := ais.processChan
ais.channelMutex.RUnlock()
select { select {
case ais.processChan <- msg: case processChan <- msg:
return nil return nil
case <-time.After(GetConfig().WriteTimeout): case <-time.After(Config.WriteTimeout):
// Processing queue full and timeout reached, drop frame // Processing queue full and timeout reached, drop frame
atomic.AddInt64(&ais.droppedFrames, 1) atomic.AddInt64(&ais.droppedFrames, 1)
return fmt.Errorf("processing queue timeout") return fmt.Errorf("processing queue timeout")
@ -1109,7 +1139,7 @@ func (ais *AudioInputServer) startMonitorGoroutine() {
monitorTask := func() { monitorTask := func() {
// Monitor goroutine doesn't need thread locking for most scenarios // Monitor goroutine doesn't need thread locking for most scenarios
// Only use thread optimizations for high-throughput scenarios // Only use thread optimizations for high-throughput scenarios
config := GetConfig() config := Config
useThreadOptimizations := config.MaxAudioProcessorWorkers > 8 useThreadOptimizations := config.MaxAudioProcessorWorkers > 8
if useThreadOptimizations { if useThreadOptimizations {
@ -1120,11 +1150,11 @@ func (ais *AudioInputServer) startMonitorGoroutine() {
} }
defer ais.wg.Done() defer ais.wg.Done()
ticker := time.NewTicker(GetConfig().DefaultTickerInterval) ticker := time.NewTicker(Config.DefaultTickerInterval)
defer ticker.Stop() defer ticker.Stop()
// Buffer size update ticker (less frequent) // Buffer size update ticker (less frequent)
bufferUpdateTicker := time.NewTicker(GetConfig().BufferUpdateInterval) bufferUpdateTicker := time.NewTicker(Config.BufferUpdateInterval)
defer bufferUpdateTicker.Stop() defer bufferUpdateTicker.Stop()
for { for {
@ -1135,7 +1165,7 @@ func (ais *AudioInputServer) startMonitorGoroutine() {
// Process frames from processing queue // Process frames from processing queue
for { for {
select { select {
case msg := <-ais.processChan: case msg := <-ais.getProcessChan():
start := time.Now() start := time.Now()
err := ais.processMessage(msg) err := ais.processMessage(msg)
processingTime := time.Since(start) processingTime := time.Since(start)
@ -1174,8 +1204,7 @@ func (ais *AudioInputServer) startMonitorGoroutine() {
// Check if we need to update buffer size // Check if we need to update buffer size
select { select {
case <-bufferUpdateTicker.C: case <-bufferUpdateTicker.C:
// Update buffer size from adaptive buffer manager // Buffer size is now fixed from config
ais.UpdateBufferSize()
default: default:
// No buffer update needed // No buffer update needed
} }
@ -1183,13 +1212,16 @@ func (ais *AudioInputServer) startMonitorGoroutine() {
} }
} }
// Submit the monitor task to the audio processor pool // Submit the monitor task to the audio processor pool with backpressure
logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger() logger := logging.GetDefaultLogger().With().Str("component", AudioInputClientComponent).Logger()
if !SubmitAudioProcessorTask(monitorTask) { if !SubmitAudioProcessorTaskWithBackpressure(monitorTask) {
// If the pool is full or shutting down, fall back to direct goroutine creation // Task was dropped due to backpressure - this is expected under high load
logger.Warn().Msg("Audio processor pool full or shutting down, falling back to direct goroutine creation") // Log at debug level to avoid spam, but track the drop
logger.Debug().Msg("Audio monitor task dropped due to backpressure")
go monitorTask() // Don't fall back to unlimited goroutine creation
// Instead, let the system recover naturally
ais.wg.Done() // Decrement the wait group since we're not starting the task
} }
} }
@ -1201,17 +1233,16 @@ func (ais *AudioInputServer) GetServerStats() (total, dropped int64, avgProcessi
atomic.LoadInt64(&ais.bufferSize) atomic.LoadInt64(&ais.bufferSize)
} }
// UpdateBufferSize updates the buffer size from adaptive buffer manager // UpdateBufferSize updates the buffer size (now using fixed config values)
func (ais *AudioInputServer) UpdateBufferSize() { func (ais *AudioInputServer) UpdateBufferSize() {
adaptiveManager := GetAdaptiveBufferManager() // Buffer size is now fixed from config
newSize := int64(adaptiveManager.GetInputBufferSize()) newSize := int64(Config.AdaptiveDefaultBufferSize)
atomic.StoreInt64(&ais.bufferSize, newSize) atomic.StoreInt64(&ais.bufferSize, newSize)
} }
// ReportLatency reports processing latency to adaptive buffer manager // ReportLatency reports processing latency (now a no-op with fixed buffers)
func (ais *AudioInputServer) ReportLatency(latency time.Duration) { func (ais *AudioInputServer) ReportLatency(latency time.Duration) {
adaptiveManager := GetAdaptiveBufferManager() // Latency reporting is now a no-op with fixed buffer sizes
adaptiveManager.UpdateLatency(latency)
} }
// GetMessagePoolStats returns detailed statistics about the message pool // GetMessagePoolStats returns detailed statistics about the message pool
@ -1226,7 +1257,7 @@ func (mp *MessagePool) GetMessagePoolStats() MessagePoolStats {
var hitRate float64 var hitRate float64
if totalRequests > 0 { if totalRequests > 0 {
hitRate = float64(hitCount) / float64(totalRequests) * GetConfig().PercentageMultiplier hitRate = float64(hitCount) / float64(totalRequests) * Config.PercentageMultiplier
} }
// Calculate channel pool size // Calculate channel pool size
@ -1259,6 +1290,20 @@ func GetGlobalMessagePoolStats() MessagePoolStats {
return globalMessagePool.GetMessagePoolStats() return globalMessagePool.GetMessagePoolStats()
} }
// getMessageChan safely returns the current message channel
func (ais *AudioInputServer) getMessageChan() chan *InputIPCMessage {
ais.channelMutex.RLock()
defer ais.channelMutex.RUnlock()
return ais.messageChan
}
// getProcessChan safely returns the current process channel
func (ais *AudioInputServer) getProcessChan() chan *InputIPCMessage {
ais.channelMutex.RLock()
defer ais.channelMutex.RUnlock()
return ais.processChan
}
// Helper functions // Helper functions
// getInputSocketPath is now defined in unified_ipc.go // getInputSocketPath is now defined in unified_ipc.go

View File

@ -4,11 +4,18 @@ import (
"encoding/binary" "encoding/binary"
"fmt" "fmt"
"io" "io"
"net"
"sync"
"sync/atomic" "sync/atomic"
"time"
"github.com/jetkvm/kvm/internal/logging"
"github.com/rs/zerolog"
) )
// Legacy aliases for backward compatibility // Legacy aliases for backward compatibility
type OutputIPCConfig = UnifiedIPCConfig type OutputIPCConfig = UnifiedIPCConfig
type OutputIPCOpusConfig = UnifiedIPCOpusConfig
type OutputMessageType = UnifiedMessageType type OutputMessageType = UnifiedMessageType
type OutputIPCMessage = UnifiedIPCMessage type OutputIPCMessage = UnifiedIPCMessage
@ -16,6 +23,7 @@ type OutputIPCMessage = UnifiedIPCMessage
const ( const (
OutputMessageTypeOpusFrame = MessageTypeOpusFrame OutputMessageTypeOpusFrame = MessageTypeOpusFrame
OutputMessageTypeConfig = MessageTypeConfig OutputMessageTypeConfig = MessageTypeConfig
OutputMessageTypeOpusConfig = MessageTypeOpusConfig
OutputMessageTypeStop = MessageTypeStop OutputMessageTypeStop = MessageTypeStop
OutputMessageTypeHeartbeat = MessageTypeHeartbeat OutputMessageTypeHeartbeat = MessageTypeHeartbeat
OutputMessageTypeAck = MessageTypeAck OutputMessageTypeAck = MessageTypeAck
@ -24,47 +32,365 @@ const (
// Methods are now inherited from UnifiedIPCMessage // Methods are now inherited from UnifiedIPCMessage
// Global shared message pool for output IPC client header reading // Global shared message pool for output IPC client header reading
var globalOutputClientMessagePool = NewGenericMessagePool(GetConfig().OutputMessagePoolSize) var globalOutputClientMessagePool = NewGenericMessagePool(Config.OutputMessagePoolSize)
// AudioOutputServer is now an alias for UnifiedAudioServer // AudioOutputServer provides audio output IPC functionality
type AudioOutputServer = UnifiedAudioServer type AudioOutputServer struct {
// Atomic counters
bufferSize int64 // Current buffer size (atomic)
droppedFrames int64 // Dropped frames counter (atomic)
totalFrames int64 // Total frames counter (atomic)
listener net.Listener
conn net.Conn
mtx sync.Mutex
running bool
logger zerolog.Logger
// Message channels
messageChan chan *OutputIPCMessage // Buffered channel for incoming messages
processChan chan *OutputIPCMessage // Buffered channel for processing queue
wg sync.WaitGroup // Wait group for goroutine coordination
// Configuration
socketPath string
magicNumber uint32
}
func NewAudioOutputServer() (*AudioOutputServer, error) { func NewAudioOutputServer() (*AudioOutputServer, error) {
return NewUnifiedAudioServer(false) // false = output server socketPath := getOutputSocketPath()
logger := logging.GetDefaultLogger().With().Str("component", "audio-output-server").Logger()
server := &AudioOutputServer{
socketPath: socketPath,
magicNumber: Config.OutputMagicNumber,
logger: logger,
messageChan: make(chan *OutputIPCMessage, Config.ChannelBufferSize),
processChan: make(chan *OutputIPCMessage, Config.ChannelBufferSize),
}
return server, nil
} }
// Start method is now inherited from UnifiedAudioServer
// acceptConnections method is now inherited from UnifiedAudioServer
// startProcessorGoroutine method is now inherited from UnifiedAudioServer
// Stop method is now inherited from UnifiedAudioServer
// Close method is now inherited from UnifiedAudioServer
// SendFrame method is now inherited from UnifiedAudioServer
// GetServerStats returns server performance statistics // GetServerStats returns server performance statistics
func (s *AudioOutputServer) GetServerStats() (total, dropped int64, bufferSize int64) { // Start starts the audio output server
stats := GetFrameStats(&s.totalFrames, &s.droppedFrames) func (s *AudioOutputServer) Start() error {
return stats.Total, stats.Dropped, atomic.LoadInt64(&s.bufferSize) s.mtx.Lock()
defer s.mtx.Unlock()
if s.running {
return fmt.Errorf("audio output server is already running")
}
// Create Unix socket
listener, err := net.Listen("unix", s.socketPath)
if err != nil {
return fmt.Errorf("failed to create unix socket: %w", err)
}
s.listener = listener
s.running = true
// Start goroutines
s.wg.Add(1)
go s.acceptConnections()
s.logger.Info().Str("socket_path", s.socketPath).Msg("Audio output server started")
return nil
} }
// AudioOutputClient is now an alias for UnifiedAudioClient // Stop stops the audio output server
type AudioOutputClient = UnifiedAudioClient func (s *AudioOutputServer) Stop() {
s.mtx.Lock()
defer s.mtx.Unlock()
if !s.running {
return
}
s.running = false
if s.listener != nil {
s.listener.Close()
}
if s.conn != nil {
s.conn.Close()
}
// Close channels
close(s.messageChan)
close(s.processChan)
s.wg.Wait()
s.logger.Info().Msg("Audio output server stopped")
}
// acceptConnections handles incoming connections
func (s *AudioOutputServer) acceptConnections() {
defer s.wg.Done()
for s.running {
conn, err := s.listener.Accept()
if err != nil {
if s.running {
s.logger.Error().Err(err).Msg("Failed to accept connection")
}
return
}
s.mtx.Lock()
s.conn = conn
s.mtx.Unlock()
s.logger.Info().Msg("Client connected to audio output server")
// Start message processing for this connection
s.wg.Add(1)
go s.handleConnection(conn)
}
}
// handleConnection processes messages from a client connection
func (s *AudioOutputServer) handleConnection(conn net.Conn) {
defer s.wg.Done()
defer conn.Close()
for s.running {
msg, err := s.readMessage(conn)
if err != nil {
if s.running {
s.logger.Error().Err(err).Msg("Failed to read message from client")
}
return
}
if err := s.processMessage(msg); err != nil {
s.logger.Error().Err(err).Msg("Failed to process message")
}
}
}
// readMessage reads a message from the connection
func (s *AudioOutputServer) readMessage(conn net.Conn) (*OutputIPCMessage, error) {
header := make([]byte, 17)
if _, err := io.ReadFull(conn, header); err != nil {
return nil, fmt.Errorf("failed to read header: %w", err)
}
magic := binary.LittleEndian.Uint32(header[0:4])
if magic != s.magicNumber {
return nil, fmt.Errorf("invalid magic number: expected %d, got %d", s.magicNumber, magic)
}
msgType := OutputMessageType(header[4])
length := binary.LittleEndian.Uint32(header[5:9])
timestamp := int64(binary.LittleEndian.Uint64(header[9:17]))
var data []byte
if length > 0 {
data = make([]byte, length)
if _, err := io.ReadFull(conn, data); err != nil {
return nil, fmt.Errorf("failed to read data: %w", err)
}
}
return &OutputIPCMessage{
Magic: magic,
Type: msgType,
Length: length,
Timestamp: timestamp,
Data: data,
}, nil
}
// processMessage processes a received message
func (s *AudioOutputServer) processMessage(msg *OutputIPCMessage) error {
switch msg.Type {
case OutputMessageTypeOpusConfig:
return s.processOpusConfig(msg.Data)
case OutputMessageTypeStop:
s.logger.Info().Msg("Received stop message")
return nil
case OutputMessageTypeHeartbeat:
s.logger.Debug().Msg("Received heartbeat")
return nil
default:
s.logger.Warn().Int("type", int(msg.Type)).Msg("Unknown message type")
return nil
}
}
// processOpusConfig processes Opus configuration updates
func (s *AudioOutputServer) processOpusConfig(data []byte) error {
// Validate configuration data size (9 * int32 = 36 bytes)
if len(data) != 36 {
return fmt.Errorf("invalid Opus configuration data size: expected 36 bytes, got %d", len(data))
}
// Decode Opus configuration
config := OutputIPCOpusConfig{
SampleRate: int(binary.LittleEndian.Uint32(data[0:4])),
Channels: int(binary.LittleEndian.Uint32(data[4:8])),
FrameSize: int(binary.LittleEndian.Uint32(data[8:12])),
Bitrate: int(binary.LittleEndian.Uint32(data[12:16])),
Complexity: int(binary.LittleEndian.Uint32(data[16:20])),
VBR: int(binary.LittleEndian.Uint32(data[20:24])),
SignalType: int(binary.LittleEndian.Uint32(data[24:28])),
Bandwidth: int(binary.LittleEndian.Uint32(data[28:32])),
DTX: int(binary.LittleEndian.Uint32(data[32:36])),
}
s.logger.Info().Interface("config", config).Msg("Received Opus configuration update")
// Ensure we're running in the audio server subprocess
if !isAudioServerProcess() {
s.logger.Warn().Msg("Opus configuration update ignored - not running in audio server subprocess")
return nil
}
// Check if audio output streaming is currently active
if atomic.LoadInt32(&outputStreamingRunning) == 0 {
s.logger.Info().Msg("Audio output streaming not active, configuration will be applied when streaming starts")
return nil
}
// Ensure capture is initialized before updating encoder parameters
// The C function requires both encoder and capture_initialized to be true
if err := cgoAudioInit(); err != nil {
s.logger.Debug().Err(err).Msg("Audio capture already initialized or initialization failed")
// Continue anyway - capture may already be initialized
}
// Apply configuration using CGO function (only if audio system is running)
vbrConstraint := Config.CGOOpusVBRConstraint
if err := updateOpusEncoderParams(config.Bitrate, config.Complexity, config.VBR, vbrConstraint, config.SignalType, config.Bandwidth, config.DTX); err != nil {
s.logger.Error().Err(err).Msg("Failed to update Opus encoder parameters - encoder may not be initialized")
return err
}
s.logger.Info().Msg("Opus encoder parameters updated successfully")
return nil
}
// SendFrame sends an audio frame to the client
func (s *AudioOutputServer) SendFrame(frame []byte) error {
s.mtx.Lock()
conn := s.conn
s.mtx.Unlock()
if conn == nil {
return fmt.Errorf("no client connected")
}
msg := &OutputIPCMessage{
Magic: s.magicNumber,
Type: OutputMessageTypeOpusFrame,
Length: uint32(len(frame)),
Timestamp: time.Now().UnixNano(),
Data: frame,
}
return s.writeMessage(conn, msg)
}
// writeMessage writes a message to the connection
func (s *AudioOutputServer) writeMessage(conn net.Conn, msg *OutputIPCMessage) error {
header := EncodeMessageHeader(msg.Magic, uint8(msg.Type), msg.Length, msg.Timestamp)
if _, err := conn.Write(header); err != nil {
return fmt.Errorf("failed to write header: %w", err)
}
if msg.Length > 0 && msg.Data != nil {
if _, err := conn.Write(msg.Data); err != nil {
return fmt.Errorf("failed to write data: %w", err)
}
}
atomic.AddInt64(&s.totalFrames, 1)
return nil
}
func (s *AudioOutputServer) GetServerStats() (total, dropped int64, bufferSize int64) {
return atomic.LoadInt64(&s.totalFrames), atomic.LoadInt64(&s.droppedFrames), atomic.LoadInt64(&s.bufferSize)
}
// AudioOutputClient provides audio output IPC client functionality
type AudioOutputClient struct {
// Atomic counters
droppedFrames int64 // Atomic counter for dropped frames
totalFrames int64 // Atomic counter for total frames
conn net.Conn
mtx sync.Mutex
running bool
logger zerolog.Logger
socketPath string
magicNumber uint32
bufferPool *AudioBufferPool // Buffer pool for memory optimization
// Health monitoring
autoReconnect bool // Enable automatic reconnection
}
func NewAudioOutputClient() *AudioOutputClient { func NewAudioOutputClient() *AudioOutputClient {
return NewUnifiedAudioClient(false) // false = output client socketPath := getOutputSocketPath()
logger := logging.GetDefaultLogger().With().Str("component", "audio-output-client").Logger()
return &AudioOutputClient{
socketPath: socketPath,
magicNumber: Config.OutputMagicNumber,
logger: logger,
bufferPool: NewAudioBufferPool(Config.MaxFrameSize),
autoReconnect: true,
}
} }
// Connect method is now inherited from UnifiedAudioClient // Connect connects to the audio output server
func (c *AudioOutputClient) Connect() error {
c.mtx.Lock()
defer c.mtx.Unlock()
// Disconnect method is now inherited from UnifiedAudioClient if c.running {
return fmt.Errorf("audio output client is already connected")
}
// IsConnected method is now inherited from UnifiedAudioClient conn, err := net.Dial("unix", c.socketPath)
if err != nil {
return fmt.Errorf("failed to connect to audio output server: %w", err)
}
// Close method is now inherited from UnifiedAudioClient c.conn = conn
c.running = true
c.logger.Info().Str("socket_path", c.socketPath).Msg("Connected to audio output server")
return nil
}
// Disconnect disconnects from the audio output server
func (c *AudioOutputClient) Disconnect() {
c.mtx.Lock()
defer c.mtx.Unlock()
if !c.running {
return
}
c.running = false
if c.conn != nil {
c.conn.Close()
c.conn = nil
}
c.logger.Info().Msg("Disconnected from audio output server")
}
// IsConnected returns whether the client is connected
func (c *AudioOutputClient) IsConnected() bool {
c.mtx.Lock()
defer c.mtx.Unlock()
return c.running && c.conn != nil
}
func (c *AudioOutputClient) ReceiveFrame() ([]byte, error) { func (c *AudioOutputClient) ReceiveFrame() ([]byte, error) {
c.mtx.Lock() c.mtx.Lock()
@ -95,7 +421,7 @@ func (c *AudioOutputClient) ReceiveFrame() ([]byte, error) {
} }
size := binary.LittleEndian.Uint32(optMsg.header[5:9]) size := binary.LittleEndian.Uint32(optMsg.header[5:9])
maxFrameSize := GetConfig().OutputMaxFrameSize maxFrameSize := Config.OutputMaxFrameSize
if int(size) > maxFrameSize { if int(size) > maxFrameSize {
return nil, fmt.Errorf("received frame size validation failed: got %d bytes, maximum allowed %d bytes", size, maxFrameSize) return nil, fmt.Errorf("received frame size validation failed: got %d bytes, maximum allowed %d bytes", size, maxFrameSize)
} }
@ -116,6 +442,53 @@ func (c *AudioOutputClient) ReceiveFrame() ([]byte, error) {
return frame, nil return frame, nil
} }
// SendOpusConfig sends Opus configuration to the audio output server
func (c *AudioOutputClient) SendOpusConfig(config OutputIPCOpusConfig) error {
c.mtx.Lock()
defer c.mtx.Unlock()
if !c.running || c.conn == nil {
return fmt.Errorf("not connected to audio output server")
}
// Validate configuration parameters
if config.SampleRate <= 0 || config.Channels <= 0 || config.FrameSize <= 0 || config.Bitrate <= 0 {
return fmt.Errorf("invalid Opus configuration: SampleRate=%d, Channels=%d, FrameSize=%d, Bitrate=%d",
config.SampleRate, config.Channels, config.FrameSize, config.Bitrate)
}
// Serialize Opus configuration using common function
data := EncodeOpusConfig(config.SampleRate, config.Channels, config.FrameSize, config.Bitrate, config.Complexity, config.VBR, config.SignalType, config.Bandwidth, config.DTX)
msg := &OutputIPCMessage{
Magic: c.magicNumber,
Type: OutputMessageTypeOpusConfig,
Length: uint32(len(data)),
Timestamp: time.Now().UnixNano(),
Data: data,
}
return c.writeMessage(msg)
}
// writeMessage writes a message to the connection
func (c *AudioOutputClient) writeMessage(msg *OutputIPCMessage) error {
header := EncodeMessageHeader(msg.Magic, uint8(msg.Type), msg.Length, msg.Timestamp)
if _, err := c.conn.Write(header); err != nil {
return fmt.Errorf("failed to write header: %w", err)
}
if msg.Length > 0 && msg.Data != nil {
if _, err := c.conn.Write(msg.Data); err != nil {
return fmt.Errorf("failed to write data: %w", err)
}
}
atomic.AddInt64(&c.totalFrames, 1)
return nil
}
// GetClientStats returns client performance statistics // GetClientStats returns client performance statistics
func (c *AudioOutputClient) GetClientStats() (total, dropped int64) { func (c *AudioOutputClient) GetClientStats() (total, dropped int64) {
stats := GetFrameStats(&c.totalFrames, &c.droppedFrames) stats := GetFrameStats(&c.totalFrames, &c.droppedFrames)
@ -123,5 +496,4 @@ func (c *AudioOutputClient) GetClientStats() (total, dropped int64) {
} }
// Helper functions // Helper functions
// getOutputSocketPath is defined in ipc_unified.go
// getOutputSocketPath is now defined in unified_ipc.go

View File

@ -4,9 +4,11 @@ import (
"encoding/binary" "encoding/binary"
"fmt" "fmt"
"io" "io"
"math"
"net" "net"
"os" "os"
"path/filepath" "path/filepath"
"strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time" "time"
@ -17,13 +19,21 @@ import (
// Unified IPC constants // Unified IPC constants
var ( var (
outputMagicNumber uint32 = GetConfig().OutputMagicNumber // "JKOU" (JetKVM Output) outputMagicNumber uint32 = Config.OutputMagicNumber // "JKOU" (JetKVM Output)
inputMagicNumber uint32 = GetConfig().InputMagicNumber // "JKMI" (JetKVM Microphone Input) inputMagicNumber uint32 = Config.InputMagicNumber // "JKMI" (JetKVM Microphone Input)
outputSocketName = "audio_output.sock" outputSocketName = "audio_output.sock"
inputSocketName = "audio_input.sock" inputSocketName = "audio_input.sock"
headerSize = 17 // Fixed header size: 4+1+4+8 bytes headerSize = 17 // Fixed header size: 4+1+4+8 bytes
) )
// Header buffer pool to reduce allocation overhead
var headerBufferPool = sync.Pool{
New: func() interface{} {
buf := make([]byte, headerSize)
return &buf
},
}
// UnifiedMessageType represents the type of IPC message for both input and output // UnifiedMessageType represents the type of IPC message for both input and output
type UnifiedMessageType uint8 type UnifiedMessageType uint8
@ -89,7 +99,6 @@ type UnifiedIPCOpusConfig struct {
// UnifiedAudioServer provides common functionality for both input and output servers // UnifiedAudioServer provides common functionality for both input and output servers
type UnifiedAudioServer struct { type UnifiedAudioServer struct {
// Atomic counters for performance monitoring // Atomic counters for performance monitoring
bufferSize int64 // Current buffer size (atomic)
droppedFrames int64 // Dropped frames counter (atomic) droppedFrames int64 // Dropped frames counter (atomic)
totalFrames int64 // Total frames counter (atomic) totalFrames int64 // Total frames counter (atomic)
@ -108,10 +117,6 @@ type UnifiedAudioServer struct {
socketPath string socketPath string
magicNumber uint32 magicNumber uint32
socketBufferConfig SocketBufferConfig socketBufferConfig SocketBufferConfig
// Performance monitoring
latencyMonitor *LatencyMonitor
adaptiveOptimizer *AdaptiveOptimizer
} }
// NewUnifiedAudioServer creates a new unified audio server // NewUnifiedAudioServer creates a new unified audio server
@ -136,11 +141,9 @@ func NewUnifiedAudioServer(isInput bool) (*UnifiedAudioServer, error) {
logger: logger, logger: logger,
socketPath: socketPath, socketPath: socketPath,
magicNumber: magicNumber, magicNumber: magicNumber,
messageChan: make(chan *UnifiedIPCMessage, GetConfig().ChannelBufferSize), messageChan: make(chan *UnifiedIPCMessage, Config.ChannelBufferSize),
processChan: make(chan *UnifiedIPCMessage, GetConfig().ChannelBufferSize), processChan: make(chan *UnifiedIPCMessage, Config.ChannelBufferSize),
socketBufferConfig: DefaultSocketBufferConfig(), socketBufferConfig: DefaultSocketBufferConfig(),
latencyMonitor: nil,
adaptiveOptimizer: nil,
} }
return server, nil return server, nil
@ -155,15 +158,38 @@ func (s *UnifiedAudioServer) Start() error {
return fmt.Errorf("server already running") return fmt.Errorf("server already running")
} }
// Remove existing socket file // Remove existing socket file with retry logic
for i := 0; i < 3; i++ {
if err := os.Remove(s.socketPath); err != nil && !os.IsNotExist(err) { if err := os.Remove(s.socketPath); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("failed to remove existing socket: %w", err) s.logger.Warn().Err(err).Int("attempt", i+1).Msg("failed to remove existing socket file, retrying")
time.Sleep(100 * time.Millisecond)
continue
}
break
}
// Create listener with retry on address already in use
var listener net.Listener
var err error
for i := 0; i < 3; i++ {
listener, err = net.Listen("unix", s.socketPath)
if err == nil {
break
}
// If address is still in use, try to remove socket file again
if strings.Contains(err.Error(), "address already in use") {
s.logger.Warn().Err(err).Int("attempt", i+1).Msg("socket address in use, attempting cleanup and retry")
os.Remove(s.socketPath)
time.Sleep(200 * time.Millisecond)
continue
}
return fmt.Errorf("failed to create unix socket: %w", err)
} }
// Create listener
listener, err := net.Listen("unix", s.socketPath)
if err != nil { if err != nil {
return fmt.Errorf("failed to create listener: %w", err) return fmt.Errorf("failed to create unix socket after retries: %w", err)
} }
s.listener = listener s.listener = listener
@ -283,8 +309,11 @@ func (s *UnifiedAudioServer) startProcessorGoroutine() {
// readMessage reads a message from the connection // readMessage reads a message from the connection
func (s *UnifiedAudioServer) readMessage(conn net.Conn) (*UnifiedIPCMessage, error) { func (s *UnifiedAudioServer) readMessage(conn net.Conn) (*UnifiedIPCMessage, error) {
// Read header // Get header buffer from pool
header := make([]byte, headerSize) headerPtr := headerBufferPool.Get().(*[]byte)
header := *headerPtr
defer headerBufferPool.Put(headerPtr)
if _, err := io.ReadFull(conn, header); err != nil { if _, err := io.ReadFull(conn, header); err != nil {
return nil, fmt.Errorf("failed to read header: %w", err) return nil, fmt.Errorf("failed to read header: %w", err)
} }
@ -300,7 +329,7 @@ func (s *UnifiedAudioServer) readMessage(conn net.Conn) (*UnifiedIPCMessage, err
timestamp := int64(binary.LittleEndian.Uint64(header[9:17])) timestamp := int64(binary.LittleEndian.Uint64(header[9:17]))
// Validate length // Validate length
if length > uint32(GetConfig().MaxFrameSize) { if length > uint32(Config.MaxFrameSize) {
return nil, fmt.Errorf("message too large: %d bytes", length) return nil, fmt.Errorf("message too large: %d bytes", length)
} }
@ -328,7 +357,10 @@ func (s *UnifiedAudioServer) SendFrame(frame []byte) error {
defer s.mtx.Unlock() defer s.mtx.Unlock()
if !s.running || s.conn == nil { if !s.running || s.conn == nil {
return fmt.Errorf("no client connected") // Silently drop frames when no client is connected
// This prevents "no client connected" warnings during startup and quality changes
atomic.AddInt64(&s.droppedFrames, 1)
return nil // Return nil to avoid flooding logs with connection warnings
} }
start := time.Now() start := time.Now()
@ -350,10 +382,6 @@ func (s *UnifiedAudioServer) SendFrame(frame []byte) error {
} }
// Record latency for monitoring // Record latency for monitoring
if s.latencyMonitor != nil {
writeLatency := time.Since(start)
s.latencyMonitor.RecordLatency(writeLatency, "ipc_write")
}
atomic.AddInt64(&s.totalFrames, 1) atomic.AddInt64(&s.totalFrames, 1)
return nil return nil
@ -361,22 +389,21 @@ func (s *UnifiedAudioServer) SendFrame(frame []byte) error {
// writeMessage writes a message to the connection // writeMessage writes a message to the connection
func (s *UnifiedAudioServer) writeMessage(conn net.Conn, msg *UnifiedIPCMessage) error { func (s *UnifiedAudioServer) writeMessage(conn net.Conn, msg *UnifiedIPCMessage) error {
// Write header header := EncodeMessageHeader(msg.Magic, uint8(msg.Type), msg.Length, msg.Timestamp)
header := make([]byte, headerSize)
binary.LittleEndian.PutUint32(header[0:4], msg.Magic)
header[4] = uint8(msg.Type)
binary.LittleEndian.PutUint32(header[5:9], msg.Length)
binary.LittleEndian.PutUint64(header[9:17], uint64(msg.Timestamp))
// Optimize: Use single write for header+data to reduce system calls
if msg.Length > 0 && msg.Data != nil {
// Pre-allocate combined buffer to avoid copying
combined := make([]byte, len(header)+len(msg.Data))
copy(combined, header)
copy(combined[len(header):], msg.Data)
if _, err := conn.Write(combined); err != nil {
return fmt.Errorf("failed to write message: %w", err)
}
} else {
if _, err := conn.Write(header); err != nil { if _, err := conn.Write(header); err != nil {
return fmt.Errorf("failed to write header: %w", err) return fmt.Errorf("failed to write header: %w", err)
} }
// Write data if present
if msg.Length > 0 && msg.Data != nil {
if _, err := conn.Write(msg.Data); err != nil {
return fmt.Errorf("failed to write data: %w", err)
}
} }
return nil return nil
@ -384,7 +411,7 @@ func (s *UnifiedAudioServer) writeMessage(conn net.Conn, msg *UnifiedIPCMessage)
// UnifiedAudioClient provides common functionality for both input and output clients // UnifiedAudioClient provides common functionality for both input and output clients
type UnifiedAudioClient struct { type UnifiedAudioClient struct {
// Atomic fields first for ARM32 alignment // Atomic counters for frame statistics
droppedFrames int64 // Atomic counter for dropped frames droppedFrames int64 // Atomic counter for dropped frames
totalFrames int64 // Atomic counter for total frames totalFrames int64 // Atomic counter for total frames
@ -395,6 +422,13 @@ type UnifiedAudioClient struct {
socketPath string socketPath string
magicNumber uint32 magicNumber uint32
bufferPool *AudioBufferPool // Buffer pool for memory optimization bufferPool *AudioBufferPool // Buffer pool for memory optimization
// Connection health monitoring
lastHealthCheck time.Time
connectionErrors int64 // Atomic counter for connection errors
autoReconnect bool // Enable automatic reconnection
healthCheckTicker *time.Ticker
stopHealthCheck chan struct{}
} }
// NewUnifiedAudioClient creates a new unified audio client // NewUnifiedAudioClient creates a new unified audio client
@ -419,7 +453,9 @@ func NewUnifiedAudioClient(isInput bool) *UnifiedAudioClient {
logger: logger, logger: logger,
socketPath: socketPath, socketPath: socketPath,
magicNumber: magicNumber, magicNumber: magicNumber,
bufferPool: NewAudioBufferPool(GetConfig().MaxFrameSize), bufferPool: NewAudioBufferPool(Config.MaxFrameSize),
autoReconnect: true, // Enable automatic reconnection by default
stopHealthCheck: make(chan struct{}),
} }
} }
@ -439,32 +475,46 @@ func (c *UnifiedAudioClient) Connect() error {
} }
// Try connecting multiple times as the server might not be ready // Try connecting multiple times as the server might not be ready
// Reduced retry count and delay for faster startup // Use configurable retry parameters for better control
for i := 0; i < 10; i++ { maxAttempts := Config.MaxConnectionAttempts
conn, err := net.Dial("unix", c.socketPath) initialDelay := Config.ConnectionRetryDelay
maxDelay := Config.MaxConnectionRetryDelay
backoffFactor := Config.ConnectionBackoffFactor
for i := 0; i < maxAttempts; i++ {
// Set connection timeout for each attempt
conn, err := net.DialTimeout("unix", c.socketPath, Config.ConnectionTimeoutDelay)
if err == nil { if err == nil {
c.conn = conn c.conn = conn
c.running = true c.running = true
// Reset frame counters on successful connection // Reset frame counters on successful connection
atomic.StoreInt64(&c.totalFrames, 0) atomic.StoreInt64(&c.totalFrames, 0)
atomic.StoreInt64(&c.droppedFrames, 0) atomic.StoreInt64(&c.droppedFrames, 0)
c.logger.Info().Str("socket_path", c.socketPath).Msg("Connected to server") atomic.StoreInt64(&c.connectionErrors, 0)
c.lastHealthCheck = time.Now()
// Start health check monitoring if auto-reconnect is enabled
if c.autoReconnect {
c.startHealthCheck()
}
c.logger.Info().Str("socket_path", c.socketPath).Int("attempt", i+1).Msg("Connected to server")
return nil return nil
} }
// Exponential backoff starting from config
backoffStart := GetConfig().BackoffStart // Log connection attempt failure
delay := time.Duration(backoffStart.Nanoseconds()*(1<<uint(i/3))) * time.Nanosecond c.logger.Debug().Err(err).Str("socket_path", c.socketPath).Int("attempt", i+1).Int("max_attempts", maxAttempts).Msg("Connection attempt failed")
maxDelay := GetConfig().MaxRetryDelay
if delay > maxDelay { // Don't sleep after the last attempt
delay = maxDelay if i < maxAttempts-1 {
} // Calculate adaptive delay based on connection failure patterns
delay := c.calculateAdaptiveDelay(i, initialDelay, maxDelay, backoffFactor)
time.Sleep(delay) time.Sleep(delay)
} }
}
// Ensure clean state on connection failure // Ensure clean state on connection failure
c.conn = nil c.conn = nil
c.running = false c.running = false
return fmt.Errorf("failed to connect to audio server after 10 attempts") return fmt.Errorf("failed to connect to audio server after %d attempts", Config.MaxConnectionAttempts)
} }
// Disconnect disconnects the client from the server // Disconnect disconnects the client from the server
@ -478,6 +528,9 @@ func (c *UnifiedAudioClient) Disconnect() {
c.running = false c.running = false
// Stop health check monitoring
c.stopHealthCheckMonitoring()
if c.conn != nil { if c.conn != nil {
c.conn.Close() c.conn.Close()
c.conn = nil c.conn = nil
@ -497,14 +550,129 @@ func (c *UnifiedAudioClient) IsConnected() bool {
func (c *UnifiedAudioClient) GetFrameStats() (total, dropped int64) { func (c *UnifiedAudioClient) GetFrameStats() (total, dropped int64) {
total = atomic.LoadInt64(&c.totalFrames) total = atomic.LoadInt64(&c.totalFrames)
dropped = atomic.LoadInt64(&c.droppedFrames) dropped = atomic.LoadInt64(&c.droppedFrames)
return total, dropped return
}
// startHealthCheck starts the connection health monitoring
func (c *UnifiedAudioClient) startHealthCheck() {
if c.healthCheckTicker != nil {
c.healthCheckTicker.Stop()
}
c.healthCheckTicker = time.NewTicker(Config.HealthCheckInterval)
go func() {
for {
select {
case <-c.healthCheckTicker.C:
c.performHealthCheck()
case <-c.stopHealthCheck:
return
}
}
}()
}
// stopHealthCheckMonitoring stops the health check monitoring
func (c *UnifiedAudioClient) stopHealthCheckMonitoring() {
if c.healthCheckTicker != nil {
c.healthCheckTicker.Stop()
c.healthCheckTicker = nil
}
select {
case c.stopHealthCheck <- struct{}{}:
default:
}
}
// performHealthCheck checks the connection health and attempts reconnection if needed
func (c *UnifiedAudioClient) performHealthCheck() {
c.mtx.Lock()
defer c.mtx.Unlock()
if !c.running || c.conn == nil {
return
}
// Simple health check: try to get connection info
if tcpConn, ok := c.conn.(*net.UnixConn); ok {
if _, err := tcpConn.File(); err != nil {
// Connection is broken
atomic.AddInt64(&c.connectionErrors, 1)
c.logger.Warn().Err(err).Msg("Connection health check failed, attempting reconnection")
// Close the broken connection
c.conn.Close()
c.conn = nil
c.running = false
// Attempt reconnection
go func() {
time.Sleep(Config.ReconnectionInterval)
if err := c.Connect(); err != nil {
c.logger.Error().Err(err).Msg("Failed to reconnect during health check")
}
}()
}
}
c.lastHealthCheck = time.Now()
}
// SetAutoReconnect enables or disables automatic reconnection
func (c *UnifiedAudioClient) SetAutoReconnect(enabled bool) {
c.mtx.Lock()
defer c.mtx.Unlock()
c.autoReconnect = enabled
if !enabled {
c.stopHealthCheckMonitoring()
} else if c.running {
c.startHealthCheck()
}
}
// GetConnectionErrors returns the number of connection errors
func (c *UnifiedAudioClient) GetConnectionErrors() int64 {
return atomic.LoadInt64(&c.connectionErrors)
}
// calculateAdaptiveDelay calculates retry delay based on system load and failure patterns
func (c *UnifiedAudioClient) calculateAdaptiveDelay(attempt int, initialDelay, maxDelay time.Duration, backoffFactor float64) time.Duration {
// Base exponential backoff
baseDelay := time.Duration(float64(initialDelay.Nanoseconds()) * math.Pow(backoffFactor, float64(attempt)))
// Get connection error history for adaptive adjustment
errorCount := atomic.LoadInt64(&c.connectionErrors)
// Adjust delay based on recent connection errors
// More errors = longer delays to avoid overwhelming the server
adaptiveFactor := 1.0
if errorCount > 5 {
adaptiveFactor = 1.5 // 50% longer delays after many errors
} else if errorCount > 10 {
adaptiveFactor = 2.0 // Double delays after excessive errors
}
// Apply adaptive factor
adaptiveDelay := time.Duration(float64(baseDelay.Nanoseconds()) * adaptiveFactor)
// Ensure we don't exceed maximum delay
if adaptiveDelay > maxDelay {
adaptiveDelay = maxDelay
}
// Add small random jitter to avoid thundering herd
jitter := time.Duration(float64(adaptiveDelay.Nanoseconds()) * 0.1 * (0.5 + float64(attempt%3)/6.0))
adaptiveDelay += jitter
return adaptiveDelay
} }
// Helper functions for socket paths // Helper functions for socket paths
func getInputSocketPath() string { func getInputSocketPath() string {
return filepath.Join(os.TempDir(), inputSocketName) return filepath.Join("/var/run", inputSocketName)
} }
func getOutputSocketPath() string { func getOutputSocketPath() string {
return filepath.Join(os.TempDir(), outputSocketName) return filepath.Join("/var/run", outputSocketName)
} }

View File

@ -28,7 +28,6 @@ type BaseSupervisor struct {
processPID int processPID int
// Process monitoring // Process monitoring
processMonitor *ProcessMonitor
// Exit tracking // Exit tracking
lastExitCode int lastExitCode int
@ -46,7 +45,7 @@ func NewBaseSupervisor(componentName string) *BaseSupervisor {
logger := logging.GetDefaultLogger().With().Str("component", componentName).Logger() logger := logging.GetDefaultLogger().With().Str("component", componentName).Logger()
return &BaseSupervisor{ return &BaseSupervisor{
logger: &logger, logger: &logger,
processMonitor: GetProcessMonitor(),
stopChan: make(chan struct{}), stopChan: make(chan struct{}),
processDone: make(chan struct{}), processDone: make(chan struct{}),
} }
@ -211,7 +210,6 @@ func (bs *BaseSupervisor) waitForProcessExit(processType string) {
bs.mutex.Unlock() bs.mutex.Unlock()
// Remove process from monitoring // Remove process from monitoring
bs.processMonitor.RemoveProcess(pid)
if exitCode != 0 { if exitCode != 0 {
bs.logger.Error().Int("pid", pid).Int("exit_code", exitCode).Msgf("%s process exited with error", processType) bs.logger.Error().Int("pid", pid).Int("exit_code", exitCode).Msgf("%s process exited with error", processType)

View File

@ -32,7 +32,7 @@ type AudioInputIPCManager struct {
// NewAudioInputIPCManager creates a new IPC-based audio input manager // NewAudioInputIPCManager creates a new IPC-based audio input manager
func NewAudioInputIPCManager() *AudioInputIPCManager { func NewAudioInputIPCManager() *AudioInputIPCManager {
return &AudioInputIPCManager{ return &AudioInputIPCManager{
supervisor: NewAudioInputSupervisor(), supervisor: GetAudioInputSupervisor(), // Use global shared supervisor
logger: logging.GetDefaultLogger().With().Str("component", AudioInputIPCComponent).Logger(), logger: logging.GetDefaultLogger().With().Str("component", AudioInputIPCComponent).Logger(),
} }
} }
@ -63,9 +63,9 @@ func (aim *AudioInputIPCManager) Start() error {
} }
config := InputIPCConfig{ config := InputIPCConfig{
SampleRate: GetConfig().InputIPCSampleRate, SampleRate: Config.InputIPCSampleRate,
Channels: GetConfig().InputIPCChannels, Channels: Config.InputIPCChannels,
FrameSize: GetConfig().InputIPCFrameSize, FrameSize: Config.InputIPCFrameSize,
} }
// Validate configuration before using it // Validate configuration before using it
@ -80,7 +80,7 @@ func (aim *AudioInputIPCManager) Start() error {
} }
// Wait for subprocess readiness // Wait for subprocess readiness
time.Sleep(GetConfig().LongSleepDuration) time.Sleep(Config.LongSleepDuration)
err = aim.supervisor.SendConfig(config) err = aim.supervisor.SendConfig(config)
if err != nil { if err != nil {

View File

@ -57,9 +57,9 @@ func (aom *AudioOutputIPCManager) Start() error {
// Send initial configuration // Send initial configuration
config := OutputIPCConfig{ config := OutputIPCConfig{
SampleRate: GetConfig().SampleRate, SampleRate: Config.SampleRate,
Channels: GetConfig().Channels, Channels: Config.Channels,
FrameSize: int(GetConfig().AudioQualityMediumFrameSize.Milliseconds()), FrameSize: int(Config.AudioQualityMediumFrameSize.Milliseconds()),
} }
if err := aom.SendConfig(config); err != nil { if err := aom.SendConfig(config); err != nil {

View File

@ -105,7 +105,7 @@ func GetMicrophoneContentionManager() *MicrophoneContentionManager {
} }
if atomic.CompareAndSwapInt32(&micContentionInitialized, 0, 1) { if atomic.CompareAndSwapInt32(&micContentionInitialized, 0, 1) {
manager := NewMicrophoneContentionManager(GetConfig().MicContentionTimeout) manager := NewMicrophoneContentionManager(Config.MicContentionTimeout)
atomic.StorePointer(&globalMicContentionManager, unsafe.Pointer(manager)) atomic.StorePointer(&globalMicContentionManager, unsafe.Pointer(manager))
return manager return manager
} }
@ -115,7 +115,7 @@ func GetMicrophoneContentionManager() *MicrophoneContentionManager {
return (*MicrophoneContentionManager)(ptr) return (*MicrophoneContentionManager)(ptr)
} }
return NewMicrophoneContentionManager(GetConfig().MicContentionTimeout) return NewMicrophoneContentionManager(Config.MicContentionTimeout)
} }
func TryMicrophoneOperation() OperationResult { func TryMicrophoneOperation() OperationResult {

View File

@ -1,198 +0,0 @@
package audio
import (
"context"
"sync"
"sync/atomic"
"time"
"github.com/rs/zerolog"
)
// AdaptiveOptimizer automatically adjusts audio parameters based on latency metrics
type AdaptiveOptimizer struct {
// Atomic fields MUST be first for ARM32 alignment (int64 fields need 8-byte alignment)
optimizationCount int64 // Number of optimizations performed (atomic)
lastOptimization int64 // Timestamp of last optimization (atomic)
optimizationLevel int64 // Current optimization level (0-10) (atomic)
latencyMonitor *LatencyMonitor
bufferManager *AdaptiveBufferManager
logger zerolog.Logger
// Control channels
ctx context.Context
cancel context.CancelFunc
wg sync.WaitGroup
// Configuration
config OptimizerConfig
}
// OptimizerConfig holds configuration for the adaptive optimizer
type OptimizerConfig struct {
MaxOptimizationLevel int // Maximum optimization level (0-10)
CooldownPeriod time.Duration // Minimum time between optimizations
Aggressiveness float64 // How aggressively to optimize (0.0-1.0)
RollbackThreshold time.Duration // Latency threshold to rollback optimizations
StabilityPeriod time.Duration // Time to wait for stability after optimization
}
// DefaultOptimizerConfig returns a sensible default configuration
func DefaultOptimizerConfig() OptimizerConfig {
return OptimizerConfig{
MaxOptimizationLevel: 8,
CooldownPeriod: GetConfig().CooldownPeriod,
Aggressiveness: GetConfig().OptimizerAggressiveness,
RollbackThreshold: GetConfig().RollbackThreshold,
StabilityPeriod: GetConfig().AdaptiveOptimizerStability,
}
}
// NewAdaptiveOptimizer creates a new adaptive optimizer
func NewAdaptiveOptimizer(latencyMonitor *LatencyMonitor, bufferManager *AdaptiveBufferManager, config OptimizerConfig, logger zerolog.Logger) *AdaptiveOptimizer {
ctx, cancel := context.WithCancel(context.Background())
optimizer := &AdaptiveOptimizer{
latencyMonitor: latencyMonitor,
bufferManager: bufferManager,
config: config,
logger: logger.With().Str("component", "adaptive-optimizer").Logger(),
ctx: ctx,
cancel: cancel,
}
// Register as latency monitor callback
latencyMonitor.AddOptimizationCallback(optimizer.handleLatencyOptimization)
return optimizer
}
// Start begins the adaptive optimization process
func (ao *AdaptiveOptimizer) Start() {
ao.wg.Add(1)
go ao.optimizationLoop()
ao.logger.Debug().Msg("adaptive optimizer started")
}
// Stop stops the adaptive optimizer
func (ao *AdaptiveOptimizer) Stop() {
ao.cancel()
ao.wg.Wait()
ao.logger.Debug().Msg("adaptive optimizer stopped")
}
// initializeStrategies sets up the available optimization strategies
// handleLatencyOptimization is called when latency optimization is needed
func (ao *AdaptiveOptimizer) handleLatencyOptimization(metrics LatencyMetrics) error {
currentLevel := atomic.LoadInt64(&ao.optimizationLevel)
lastOpt := atomic.LoadInt64(&ao.lastOptimization)
// Check cooldown period
if time.Since(time.Unix(0, lastOpt)) < ao.config.CooldownPeriod {
return nil
}
// Determine if we need to increase or decrease optimization level
targetLevel := ao.calculateTargetOptimizationLevel(metrics)
if targetLevel > currentLevel {
return ao.increaseOptimization(int(targetLevel))
} else if targetLevel < currentLevel {
return ao.decreaseOptimization(int(targetLevel))
}
return nil
}
// calculateTargetOptimizationLevel determines the appropriate optimization level
func (ao *AdaptiveOptimizer) calculateTargetOptimizationLevel(metrics LatencyMetrics) int64 {
// Base calculation on current latency vs target
latencyRatio := float64(metrics.Current) / float64(GetConfig().AdaptiveOptimizerLatencyTarget) // 50ms target
// Adjust based on trend
switch metrics.Trend {
case LatencyTrendIncreasing:
latencyRatio *= 1.2 // Be more aggressive
case LatencyTrendDecreasing:
latencyRatio *= 0.8 // Be less aggressive
case LatencyTrendVolatile:
latencyRatio *= 1.1 // Slightly more aggressive
}
// Apply aggressiveness factor
latencyRatio *= ao.config.Aggressiveness
// Convert to optimization level
targetLevel := int64(latencyRatio * GetConfig().LatencyScalingFactor) // Scale to 0-10 range
if targetLevel > int64(ao.config.MaxOptimizationLevel) {
targetLevel = int64(ao.config.MaxOptimizationLevel)
}
if targetLevel < 0 {
targetLevel = 0
}
return targetLevel
}
// increaseOptimization applies optimization strategies up to the target level
func (ao *AdaptiveOptimizer) increaseOptimization(targetLevel int) error {
atomic.StoreInt64(&ao.optimizationLevel, int64(targetLevel))
atomic.StoreInt64(&ao.lastOptimization, time.Now().UnixNano())
atomic.AddInt64(&ao.optimizationCount, 1)
return nil
}
// decreaseOptimization rolls back optimization strategies to the target level
func (ao *AdaptiveOptimizer) decreaseOptimization(targetLevel int) error {
atomic.StoreInt64(&ao.optimizationLevel, int64(targetLevel))
atomic.StoreInt64(&ao.lastOptimization, time.Now().UnixNano())
return nil
}
// optimizationLoop runs the main optimization monitoring loop
func (ao *AdaptiveOptimizer) optimizationLoop() {
defer ao.wg.Done()
ticker := time.NewTicker(ao.config.StabilityPeriod)
defer ticker.Stop()
for {
select {
case <-ao.ctx.Done():
return
case <-ticker.C:
ao.checkStability()
}
}
}
// checkStability monitors system stability and rolls back if needed
func (ao *AdaptiveOptimizer) checkStability() {
metrics := ao.latencyMonitor.GetMetrics()
// Check if we need to rollback due to excessive latency
if metrics.Current > ao.config.RollbackThreshold {
currentLevel := int(atomic.LoadInt64(&ao.optimizationLevel))
if currentLevel > 0 {
ao.logger.Warn().Dur("current_latency", metrics.Current).Dur("threshold", ao.config.RollbackThreshold).Msg("rolling back optimizations due to excessive latency")
if err := ao.decreaseOptimization(currentLevel - 1); err != nil {
ao.logger.Error().Err(err).Msg("failed to decrease optimization level")
}
}
}
}
// GetOptimizationStats returns current optimization statistics
func (ao *AdaptiveOptimizer) GetOptimizationStats() map[string]interface{} {
return map[string]interface{}{
"optimization_level": atomic.LoadInt64(&ao.optimizationLevel),
"optimization_count": atomic.LoadInt64(&ao.optimizationCount),
"last_optimization": time.Unix(0, atomic.LoadInt64(&ao.lastOptimization)),
}
}
// Strategy implementation methods (stubs for now)

View File

@ -1,144 +0,0 @@
package audio
import (
"runtime"
"sync/atomic"
"time"
"github.com/jetkvm/kvm/internal/logging"
)
// GoroutineMonitor tracks goroutine count and provides cleanup mechanisms
type GoroutineMonitor struct {
baselineCount int
peakCount int
lastCount int
monitorInterval time.Duration
lastCheck time.Time
enabled int32
}
// Global goroutine monitor instance
var globalGoroutineMonitor *GoroutineMonitor
// NewGoroutineMonitor creates a new goroutine monitor
func NewGoroutineMonitor(monitorInterval time.Duration) *GoroutineMonitor {
if monitorInterval <= 0 {
monitorInterval = 30 * time.Second
}
// Get current goroutine count as baseline
baselineCount := runtime.NumGoroutine()
return &GoroutineMonitor{
baselineCount: baselineCount,
peakCount: baselineCount,
lastCount: baselineCount,
monitorInterval: monitorInterval,
lastCheck: time.Now(),
}
}
// Start begins goroutine monitoring
func (gm *GoroutineMonitor) Start() {
if !atomic.CompareAndSwapInt32(&gm.enabled, 0, 1) {
return // Already running
}
go gm.monitorLoop()
}
// Stop stops goroutine monitoring
func (gm *GoroutineMonitor) Stop() {
atomic.StoreInt32(&gm.enabled, 0)
}
// monitorLoop periodically checks goroutine count
func (gm *GoroutineMonitor) monitorLoop() {
logger := logging.GetDefaultLogger().With().Str("component", "goroutine-monitor").Logger()
logger.Info().Int("baseline", gm.baselineCount).Msg("goroutine monitor started")
for atomic.LoadInt32(&gm.enabled) == 1 {
time.Sleep(gm.monitorInterval)
gm.checkGoroutineCount()
}
logger.Info().Msg("goroutine monitor stopped")
}
// checkGoroutineCount checks current goroutine count and logs if it exceeds thresholds
func (gm *GoroutineMonitor) checkGoroutineCount() {
currentCount := runtime.NumGoroutine()
gm.lastCount = currentCount
// Update peak count if needed
if currentCount > gm.peakCount {
gm.peakCount = currentCount
}
// Calculate growth since baseline
growth := currentCount - gm.baselineCount
growthPercent := float64(growth) / float64(gm.baselineCount) * 100
// Log warning if growth exceeds thresholds
logger := logging.GetDefaultLogger().With().Str("component", "goroutine-monitor").Logger()
// Different log levels based on growth severity
if growthPercent > 30 {
// Severe growth - trigger cleanup
logger.Warn().Int("current", currentCount).Int("baseline", gm.baselineCount).
Int("growth", growth).Float64("growth_percent", growthPercent).
Msg("excessive goroutine growth detected - triggering cleanup")
// Force garbage collection to clean up unused resources
runtime.GC()
// Force cleanup of goroutine buffer cache
cleanupGoroutineCache()
} else if growthPercent > 20 {
// Moderate growth - just log warning
logger.Warn().Int("current", currentCount).Int("baseline", gm.baselineCount).
Int("growth", growth).Float64("growth_percent", growthPercent).
Msg("significant goroutine growth detected")
} else if growthPercent > 10 {
// Minor growth - log info
logger.Info().Int("current", currentCount).Int("baseline", gm.baselineCount).
Int("growth", growth).Float64("growth_percent", growthPercent).
Msg("goroutine growth detected")
}
// Update last check time
gm.lastCheck = time.Now()
}
// GetGoroutineStats returns current goroutine statistics
func (gm *GoroutineMonitor) GetGoroutineStats() map[string]interface{} {
return map[string]interface{}{
"current_count": gm.lastCount,
"baseline_count": gm.baselineCount,
"peak_count": gm.peakCount,
"growth": gm.lastCount - gm.baselineCount,
"growth_percent": float64(gm.lastCount-gm.baselineCount) / float64(gm.baselineCount) * 100,
"last_check": gm.lastCheck,
}
}
// GetGoroutineMonitor returns the global goroutine monitor instance
func GetGoroutineMonitor() *GoroutineMonitor {
if globalGoroutineMonitor == nil {
globalGoroutineMonitor = NewGoroutineMonitor(GetConfig().GoroutineMonitorInterval)
}
return globalGoroutineMonitor
}
// StartGoroutineMonitoring starts the global goroutine monitor
func StartGoroutineMonitoring() {
// Goroutine monitoring disabled
}
// StopGoroutineMonitoring stops the global goroutine monitor
func StopGoroutineMonitoring() {
if globalGoroutineMonitor != nil {
globalGoroutineMonitor.Stop()
}
}

View File

@ -1,333 +0,0 @@
package audio
import (
"context"
"sync"
"sync/atomic"
"time"
"github.com/rs/zerolog"
)
// LatencyMonitor tracks and optimizes audio latency in real-time
type LatencyMonitor struct {
// Atomic fields MUST be first for ARM32 alignment (int64 fields need 8-byte alignment)
currentLatency int64 // Current latency in nanoseconds (atomic)
averageLatency int64 // Rolling average latency in nanoseconds (atomic)
minLatency int64 // Minimum observed latency in nanoseconds (atomic)
maxLatency int64 // Maximum observed latency in nanoseconds (atomic)
latencySamples int64 // Number of latency samples collected (atomic)
jitterAccumulator int64 // Accumulated jitter for variance calculation (atomic)
lastOptimization int64 // Timestamp of last optimization in nanoseconds (atomic)
config LatencyConfig
logger zerolog.Logger
// Control channels
ctx context.Context
cancel context.CancelFunc
wg sync.WaitGroup
// Optimization callbacks
optimizationCallbacks []OptimizationCallback
mutex sync.RWMutex
// Performance tracking
latencyHistory []LatencyMeasurement
historyMutex sync.RWMutex
}
// LatencyConfig holds configuration for latency monitoring
type LatencyConfig struct {
TargetLatency time.Duration // Target latency to maintain
MaxLatency time.Duration // Maximum acceptable latency
OptimizationInterval time.Duration // How often to run optimization
HistorySize int // Number of latency measurements to keep
JitterThreshold time.Duration // Jitter threshold for optimization
AdaptiveThreshold float64 // Threshold for adaptive adjustments (0.0-1.0)
}
// LatencyMeasurement represents a single latency measurement
type LatencyMeasurement struct {
Timestamp time.Time
Latency time.Duration
Jitter time.Duration
Source string // Source of the measurement (e.g., "input", "output", "processing")
}
// OptimizationCallback is called when latency optimization is triggered
type OptimizationCallback func(metrics LatencyMetrics) error
// LatencyMetrics provides comprehensive latency statistics
type LatencyMetrics struct {
Current time.Duration
Average time.Duration
Min time.Duration
Max time.Duration
Jitter time.Duration
SampleCount int64
Trend LatencyTrend
}
// LatencyTrend indicates the direction of latency changes
type LatencyTrend int
const (
LatencyTrendStable LatencyTrend = iota
LatencyTrendIncreasing
LatencyTrendDecreasing
LatencyTrendVolatile
)
// DefaultLatencyConfig returns a sensible default configuration
func DefaultLatencyConfig() LatencyConfig {
config := GetConfig()
return LatencyConfig{
TargetLatency: config.LatencyMonitorTarget,
MaxLatency: config.MaxLatencyThreshold,
OptimizationInterval: config.LatencyOptimizationInterval,
HistorySize: config.LatencyHistorySize,
JitterThreshold: config.JitterThreshold,
AdaptiveThreshold: config.LatencyAdaptiveThreshold,
}
}
// NewLatencyMonitor creates a new latency monitoring system
func NewLatencyMonitor(config LatencyConfig, logger zerolog.Logger) *LatencyMonitor {
// Validate latency configuration
if err := ValidateLatencyConfig(config); err != nil {
// Log validation error and use default configuration
logger.Error().Err(err).Msg("Invalid latency configuration provided, using defaults")
config = DefaultLatencyConfig()
}
ctx, cancel := context.WithCancel(context.Background())
return &LatencyMonitor{
config: config,
logger: logger.With().Str("component", "latency-monitor").Logger(),
ctx: ctx,
cancel: cancel,
latencyHistory: make([]LatencyMeasurement, 0, config.HistorySize),
minLatency: int64(time.Hour), // Initialize to high value
}
}
// Start begins latency monitoring and optimization
func (lm *LatencyMonitor) Start() {
lm.wg.Add(1)
go lm.monitoringLoop()
}
// Stop stops the latency monitor
func (lm *LatencyMonitor) Stop() {
lm.cancel()
lm.wg.Wait()
}
// RecordLatency records a new latency measurement
func (lm *LatencyMonitor) RecordLatency(latency time.Duration, source string) {
now := time.Now()
latencyNanos := latency.Nanoseconds()
// Update atomic counters
atomic.StoreInt64(&lm.currentLatency, latencyNanos)
atomic.AddInt64(&lm.latencySamples, 1)
// Update min/max
for {
oldMin := atomic.LoadInt64(&lm.minLatency)
if latencyNanos >= oldMin || atomic.CompareAndSwapInt64(&lm.minLatency, oldMin, latencyNanos) {
break
}
}
for {
oldMax := atomic.LoadInt64(&lm.maxLatency)
if latencyNanos <= oldMax || atomic.CompareAndSwapInt64(&lm.maxLatency, oldMax, latencyNanos) {
break
}
}
// Update rolling average using exponential moving average
oldAvg := atomic.LoadInt64(&lm.averageLatency)
newAvg := oldAvg + (latencyNanos-oldAvg)/10 // Alpha = 0.1
atomic.StoreInt64(&lm.averageLatency, newAvg)
// Calculate jitter (difference from average)
jitter := latencyNanos - newAvg
if jitter < 0 {
jitter = -jitter
}
atomic.AddInt64(&lm.jitterAccumulator, jitter)
// Store in history
lm.historyMutex.Lock()
measurement := LatencyMeasurement{
Timestamp: now,
Latency: latency,
Jitter: time.Duration(jitter),
Source: source,
}
if len(lm.latencyHistory) >= lm.config.HistorySize {
// Remove oldest measurement
copy(lm.latencyHistory, lm.latencyHistory[1:])
lm.latencyHistory[len(lm.latencyHistory)-1] = measurement
} else {
lm.latencyHistory = append(lm.latencyHistory, measurement)
}
lm.historyMutex.Unlock()
}
// GetMetrics returns current latency metrics
func (lm *LatencyMonitor) GetMetrics() LatencyMetrics {
current := atomic.LoadInt64(&lm.currentLatency)
average := atomic.LoadInt64(&lm.averageLatency)
min := atomic.LoadInt64(&lm.minLatency)
max := atomic.LoadInt64(&lm.maxLatency)
samples := atomic.LoadInt64(&lm.latencySamples)
jitterSum := atomic.LoadInt64(&lm.jitterAccumulator)
var jitter time.Duration
if samples > 0 {
jitter = time.Duration(jitterSum / samples)
}
return LatencyMetrics{
Current: time.Duration(current),
Average: time.Duration(average),
Min: time.Duration(min),
Max: time.Duration(max),
Jitter: jitter,
SampleCount: samples,
Trend: lm.calculateTrend(),
}
}
// AddOptimizationCallback adds a callback for latency optimization
func (lm *LatencyMonitor) AddOptimizationCallback(callback OptimizationCallback) {
lm.mutex.Lock()
lm.optimizationCallbacks = append(lm.optimizationCallbacks, callback)
lm.mutex.Unlock()
}
// monitoringLoop runs the main monitoring and optimization loop
func (lm *LatencyMonitor) monitoringLoop() {
defer lm.wg.Done()
ticker := time.NewTicker(lm.config.OptimizationInterval)
defer ticker.Stop()
for {
select {
case <-lm.ctx.Done():
return
case <-ticker.C:
lm.runOptimization()
}
}
}
// runOptimization checks if optimization is needed and triggers callbacks with threshold validation.
//
// Validation Rules:
// - Current latency must not exceed MaxLatency (default: 200ms)
// - Average latency checked against adaptive threshold: TargetLatency * (1 + AdaptiveThreshold)
// - Jitter must not exceed JitterThreshold (default: 20ms)
// - All latency values must be non-negative durations
//
// Optimization Triggers:
// - Current latency > MaxLatency: Immediate optimization needed
// - Average latency > adaptive threshold: Gradual optimization needed
// - Jitter > JitterThreshold: Stability optimization needed
//
// Threshold Calculations:
// - Adaptive threshold = TargetLatency * (1.0 + AdaptiveThreshold)
// - Default: 50ms * (1.0 + 0.8) = 90ms adaptive threshold
// - Provides buffer above target before triggering optimization
//
// The function ensures real-time audio performance by monitoring multiple
// latency metrics and triggering optimization callbacks when thresholds are exceeded.
func (lm *LatencyMonitor) runOptimization() {
metrics := lm.GetMetrics()
// Check if optimization is needed
needsOptimization := false
// Check if current latency exceeds threshold
if metrics.Current > lm.config.MaxLatency {
needsOptimization = true
lm.logger.Warn().Dur("current_latency", metrics.Current).Dur("max_latency", lm.config.MaxLatency).Msg("latency exceeds maximum threshold")
}
// Check if average latency is above adaptive threshold
adaptiveThreshold := time.Duration(float64(lm.config.TargetLatency.Nanoseconds()) * (1.0 + lm.config.AdaptiveThreshold))
if metrics.Average > adaptiveThreshold {
needsOptimization = true
}
// Check if jitter is too high
if metrics.Jitter > lm.config.JitterThreshold {
needsOptimization = true
}
if needsOptimization {
atomic.StoreInt64(&lm.lastOptimization, time.Now().UnixNano())
// Run optimization callbacks
lm.mutex.RLock()
callbacks := make([]OptimizationCallback, len(lm.optimizationCallbacks))
copy(callbacks, lm.optimizationCallbacks)
lm.mutex.RUnlock()
for _, callback := range callbacks {
if err := callback(metrics); err != nil {
lm.logger.Error().Err(err).Msg("optimization callback failed")
}
}
}
}
// calculateTrend analyzes recent latency measurements to determine trend
func (lm *LatencyMonitor) calculateTrend() LatencyTrend {
lm.historyMutex.RLock()
defer lm.historyMutex.RUnlock()
if len(lm.latencyHistory) < 10 {
return LatencyTrendStable
}
// Analyze last 10 measurements
recentMeasurements := lm.latencyHistory[len(lm.latencyHistory)-10:]
var increasing, decreasing int
for i := 1; i < len(recentMeasurements); i++ {
if recentMeasurements[i].Latency > recentMeasurements[i-1].Latency {
increasing++
} else if recentMeasurements[i].Latency < recentMeasurements[i-1].Latency {
decreasing++
}
}
// Determine trend based on direction changes
if increasing > 6 {
return LatencyTrendIncreasing
} else if decreasing > 6 {
return LatencyTrendDecreasing
} else if increasing+decreasing > 7 {
return LatencyTrendVolatile
}
return LatencyTrendStable
}
// GetLatencyHistory returns a copy of recent latency measurements
func (lm *LatencyMonitor) GetLatencyHistory() []LatencyMeasurement {
lm.historyMutex.RLock()
defer lm.historyMutex.RUnlock()
history := make([]LatencyMeasurement, len(lm.latencyHistory))
copy(history, lm.latencyHistory)
return history
}

View File

@ -1,406 +0,0 @@
package audio
import (
"bufio"
"fmt"
"os"
"strconv"
"strings"
"sync"
"time"
"github.com/jetkvm/kvm/internal/logging"
"github.com/rs/zerolog"
)
// Variables for process monitoring (using configuration)
var (
// System constants
maxCPUPercent = GetConfig().MaxCPUPercent
minCPUPercent = GetConfig().MinCPUPercent
defaultClockTicks = GetConfig().DefaultClockTicks
defaultMemoryGB = GetConfig().DefaultMemoryGB
// Monitoring thresholds
maxWarmupSamples = GetConfig().MaxWarmupSamples
warmupCPUSamples = GetConfig().WarmupCPUSamples
// Channel buffer size
metricsChannelBuffer = GetConfig().MetricsChannelBuffer
// Clock tick detection ranges
minValidClockTicks = float64(GetConfig().MinValidClockTicks)
maxValidClockTicks = float64(GetConfig().MaxValidClockTicks)
)
// Variables for process monitoring
var (
pageSize = GetConfig().PageSize
)
// ProcessMetrics represents CPU and memory usage metrics for a process
type ProcessMetrics struct {
PID int `json:"pid"`
CPUPercent float64 `json:"cpu_percent"`
MemoryRSS int64 `json:"memory_rss_bytes"`
MemoryVMS int64 `json:"memory_vms_bytes"`
MemoryPercent float64 `json:"memory_percent"`
Timestamp time.Time `json:"timestamp"`
ProcessName string `json:"process_name"`
}
type ProcessMonitor struct {
logger zerolog.Logger
mutex sync.RWMutex
monitoredPIDs map[int]*processState
running bool
stopChan chan struct{}
metricsChan chan ProcessMetrics
updateInterval time.Duration
totalMemory int64
memoryOnce sync.Once
clockTicks float64
clockTicksOnce sync.Once
}
// processState tracks the state needed for CPU calculation
type processState struct {
name string
lastCPUTime int64
lastSysTime int64
lastUserTime int64
lastSample time.Time
warmupSamples int
}
// NewProcessMonitor creates a new process monitor
func NewProcessMonitor() *ProcessMonitor {
return &ProcessMonitor{
logger: logging.GetDefaultLogger().With().Str("component", "process-monitor").Logger(),
monitoredPIDs: make(map[int]*processState),
stopChan: make(chan struct{}),
metricsChan: make(chan ProcessMetrics, metricsChannelBuffer),
updateInterval: GetMetricsUpdateInterval(),
}
}
// Start begins monitoring processes
func (pm *ProcessMonitor) Start() {
pm.mutex.Lock()
defer pm.mutex.Unlock()
if pm.running {
return
}
pm.running = true
go pm.monitorLoop()
pm.logger.Debug().Msg("process monitor started")
}
// Stop stops monitoring processes
func (pm *ProcessMonitor) Stop() {
pm.mutex.Lock()
defer pm.mutex.Unlock()
if !pm.running {
return
}
pm.running = false
close(pm.stopChan)
pm.logger.Debug().Msg("process monitor stopped")
}
// AddProcess adds a process to monitor
func (pm *ProcessMonitor) AddProcess(pid int, name string) {
pm.mutex.Lock()
defer pm.mutex.Unlock()
pm.monitoredPIDs[pid] = &processState{
name: name,
lastSample: time.Now(),
}
pm.logger.Info().Int("pid", pid).Str("name", name).Msg("Added process to monitor")
}
// RemoveProcess removes a process from monitoring
func (pm *ProcessMonitor) RemoveProcess(pid int) {
pm.mutex.Lock()
defer pm.mutex.Unlock()
delete(pm.monitoredPIDs, pid)
pm.logger.Info().Int("pid", pid).Msg("Removed process from monitor")
}
// GetMetricsChan returns the channel for receiving metrics
func (pm *ProcessMonitor) GetMetricsChan() <-chan ProcessMetrics {
return pm.metricsChan
}
// GetCurrentMetrics returns current metrics for all monitored processes
func (pm *ProcessMonitor) GetCurrentMetrics() []ProcessMetrics {
pm.mutex.RLock()
defer pm.mutex.RUnlock()
var metrics []ProcessMetrics
for pid, state := range pm.monitoredPIDs {
if metric, err := pm.collectMetrics(pid, state); err == nil {
metrics = append(metrics, metric)
}
}
return metrics
}
// monitorLoop is the main monitoring loop
func (pm *ProcessMonitor) monitorLoop() {
ticker := time.NewTicker(pm.updateInterval)
defer ticker.Stop()
for {
select {
case <-pm.stopChan:
return
case <-ticker.C:
pm.collectAllMetrics()
}
}
}
func (pm *ProcessMonitor) collectAllMetrics() {
pm.mutex.RLock()
pidsToCheck := make([]int, 0, len(pm.monitoredPIDs))
states := make([]*processState, 0, len(pm.monitoredPIDs))
for pid, state := range pm.monitoredPIDs {
pidsToCheck = append(pidsToCheck, pid)
states = append(states, state)
}
pm.mutex.RUnlock()
deadPIDs := make([]int, 0)
for i, pid := range pidsToCheck {
if metric, err := pm.collectMetrics(pid, states[i]); err == nil {
select {
case pm.metricsChan <- metric:
default:
}
} else {
deadPIDs = append(deadPIDs, pid)
}
}
for _, pid := range deadPIDs {
pm.RemoveProcess(pid)
}
}
func (pm *ProcessMonitor) collectMetrics(pid int, state *processState) (ProcessMetrics, error) {
now := time.Now()
metric := ProcessMetrics{
PID: pid,
Timestamp: now,
ProcessName: state.name,
}
statPath := fmt.Sprintf("/proc/%d/stat", pid)
statData, err := os.ReadFile(statPath)
if err != nil {
return metric, fmt.Errorf("failed to read process statistics from /proc/%d/stat: %w", pid, err)
}
fields := strings.Fields(string(statData))
if len(fields) < 24 {
return metric, fmt.Errorf("invalid process stat format: expected at least 24 fields, got %d from /proc/%d/stat", len(fields), pid)
}
utime, _ := strconv.ParseInt(fields[13], 10, 64)
stime, _ := strconv.ParseInt(fields[14], 10, 64)
totalCPUTime := utime + stime
vsize, _ := strconv.ParseInt(fields[22], 10, 64)
rss, _ := strconv.ParseInt(fields[23], 10, 64)
metric.MemoryRSS = rss * int64(pageSize)
metric.MemoryVMS = vsize
// Calculate CPU percentage
metric.CPUPercent = pm.calculateCPUPercent(totalCPUTime, state, now)
// Increment warmup counter
if state.warmupSamples < maxWarmupSamples {
state.warmupSamples++
}
// Calculate memory percentage (RSS / total system memory)
if totalMem := pm.getTotalMemory(); totalMem > 0 {
metric.MemoryPercent = float64(metric.MemoryRSS) / float64(totalMem) * GetConfig().PercentageMultiplier
}
// Update state for next calculation
state.lastCPUTime = totalCPUTime
state.lastUserTime = utime
state.lastSysTime = stime
state.lastSample = now
return metric, nil
}
// calculateCPUPercent calculates CPU percentage for a process with validation and bounds checking.
//
// Validation Rules:
// - Returns 0.0 for first sample (no baseline for comparison)
// - Requires positive time delta between samples
// - Applies CPU percentage bounds: [MinCPUPercent, MaxCPUPercent]
// - Uses system clock ticks for accurate CPU time conversion
// - Validates clock ticks within range [MinValidClockTicks, MaxValidClockTicks]
//
// Bounds Applied:
// - CPU percentage clamped to [0.01%, 100.0%] (default values)
// - Clock ticks validated within [50, 1000] range (default values)
// - Time delta must be > 0 to prevent division by zero
//
// Warmup Behavior:
// - During warmup period (< WarmupCPUSamples), returns MinCPUPercent for idle processes
// - This indicates process is alive but not consuming significant CPU
//
// The function ensures accurate CPU percentage calculation while preventing
// invalid measurements that could affect system monitoring and adaptive algorithms.
func (pm *ProcessMonitor) calculateCPUPercent(totalCPUTime int64, state *processState, now time.Time) float64 {
if state.lastSample.IsZero() {
// First sample - initialize baseline
state.warmupSamples = 0
return 0.0
}
timeDelta := now.Sub(state.lastSample).Seconds()
cpuDelta := float64(totalCPUTime - state.lastCPUTime)
if timeDelta <= 0 {
return 0.0
}
if cpuDelta > 0 {
// Convert from clock ticks to seconds using actual system clock ticks
clockTicks := pm.getClockTicks()
cpuSeconds := cpuDelta / clockTicks
cpuPercent := (cpuSeconds / timeDelta) * GetConfig().PercentageMultiplier
// Apply bounds
if cpuPercent > maxCPUPercent {
cpuPercent = maxCPUPercent
}
if cpuPercent < minCPUPercent {
cpuPercent = minCPUPercent
}
return cpuPercent
}
// No CPU delta - process was idle
if state.warmupSamples < warmupCPUSamples {
// During warmup, provide a small non-zero value to indicate process is alive
return minCPUPercent
}
return 0.0
}
func (pm *ProcessMonitor) getClockTicks() float64 {
pm.clockTicksOnce.Do(func() {
// Try to detect actual clock ticks from kernel boot parameters or /proc/stat
if data, err := os.ReadFile("/proc/cmdline"); err == nil {
// Look for HZ parameter in kernel command line
cmdline := string(data)
if strings.Contains(cmdline, "HZ=") {
fields := strings.Fields(cmdline)
for _, field := range fields {
if strings.HasPrefix(field, "HZ=") {
if hz, err := strconv.ParseFloat(field[3:], 64); err == nil && hz > 0 {
pm.clockTicks = hz
return
}
}
}
}
}
// Try reading from /proc/timer_list for more accurate detection
if data, err := os.ReadFile("/proc/timer_list"); err == nil {
timer := string(data)
// Look for tick device frequency
lines := strings.Split(timer, "\n")
for _, line := range lines {
if strings.Contains(line, "tick_period:") {
fields := strings.Fields(line)
if len(fields) >= 2 {
if period, err := strconv.ParseInt(fields[1], 10, 64); err == nil && period > 0 {
// Convert nanoseconds to Hz
hz := GetConfig().CGONanosecondsPerSecond / float64(period)
if hz >= minValidClockTicks && hz <= maxValidClockTicks {
pm.clockTicks = hz
return
}
}
}
}
}
}
// Fallback: Most embedded ARM systems (like jetKVM) use 250 Hz or 1000 Hz
// rather than the traditional 100 Hz
pm.clockTicks = defaultClockTicks
pm.logger.Warn().Float64("clock_ticks", pm.clockTicks).Msg("Using fallback clock ticks value")
// Log successful detection for non-fallback values
if pm.clockTicks != defaultClockTicks {
pm.logger.Info().Float64("clock_ticks", pm.clockTicks).Msg("Detected system clock ticks")
}
})
return pm.clockTicks
}
func (pm *ProcessMonitor) getTotalMemory() int64 {
pm.memoryOnce.Do(func() {
file, err := os.Open("/proc/meminfo")
if err != nil {
pm.totalMemory = int64(defaultMemoryGB) * int64(GetConfig().ProcessMonitorKBToBytes) * int64(GetConfig().ProcessMonitorKBToBytes) * int64(GetConfig().ProcessMonitorKBToBytes)
return
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text()
if strings.HasPrefix(line, "MemTotal:") {
fields := strings.Fields(line)
if len(fields) >= 2 {
if kb, err := strconv.ParseInt(fields[1], 10, 64); err == nil {
pm.totalMemory = kb * int64(GetConfig().ProcessMonitorKBToBytes)
return
}
}
break
}
}
pm.totalMemory = int64(defaultMemoryGB) * int64(GetConfig().ProcessMonitorKBToBytes) * int64(GetConfig().ProcessMonitorKBToBytes) * int64(GetConfig().ProcessMonitorKBToBytes) // Fallback
})
return pm.totalMemory
}
// GetTotalMemory returns total system memory in bytes (public method)
func (pm *ProcessMonitor) GetTotalMemory() int64 {
return pm.getTotalMemory()
}
// Global process monitor instance
var globalProcessMonitor *ProcessMonitor
var processMonitorOnce sync.Once
// GetProcessMonitor returns the global process monitor instance
func GetProcessMonitor() *ProcessMonitor {
processMonitorOnce.Do(func() {
globalProcessMonitor = NewProcessMonitor()
globalProcessMonitor.Start()
})
return globalProcessMonitor
}

View File

@ -70,7 +70,7 @@ func RunAudioOutputServer() error {
StopNonBlockingAudioStreaming() StopNonBlockingAudioStreaming()
// Give some time for cleanup // Give some time for cleanup
time.Sleep(GetConfig().DefaultSleepDuration) time.Sleep(Config.DefaultSleepDuration)
return nil return nil
} }

View File

@ -48,9 +48,6 @@ func getOutputStreamingLogger() *zerolog.Logger {
// StartAudioOutputStreaming starts audio output streaming (capturing system audio) // StartAudioOutputStreaming starts audio output streaming (capturing system audio)
func StartAudioOutputStreaming(send func([]byte)) error { func StartAudioOutputStreaming(send func([]byte)) error {
// Initialize audio monitoring (latency tracking and cache cleanup)
InitializeAudioMonitoring()
if !atomic.CompareAndSwapInt32(&outputStreamingRunning, 0, 1) { if !atomic.CompareAndSwapInt32(&outputStreamingRunning, 0, 1) {
return ErrAudioAlreadyRunning return ErrAudioAlreadyRunning
} }
@ -84,9 +81,9 @@ func StartAudioOutputStreaming(send func([]byte)) error {
buffer := make([]byte, GetMaxAudioFrameSize()) buffer := make([]byte, GetMaxAudioFrameSize())
consecutiveErrors := 0 consecutiveErrors := 0
maxConsecutiveErrors := GetConfig().MaxConsecutiveErrors maxConsecutiveErrors := Config.MaxConsecutiveErrors
errorBackoffDelay := GetConfig().RetryDelay errorBackoffDelay := Config.RetryDelay
maxErrorBackoff := GetConfig().MaxRetryDelay maxErrorBackoff := Config.MaxRetryDelay
for { for {
select { select {
@ -123,18 +120,18 @@ func StartAudioOutputStreaming(send func([]byte)) error {
Err(initErr). Err(initErr).
Msg("Failed to reinitialize audio system") Msg("Failed to reinitialize audio system")
// Exponential backoff for reinitialization failures // Exponential backoff for reinitialization failures
errorBackoffDelay = time.Duration(float64(errorBackoffDelay) * GetConfig().BackoffMultiplier) errorBackoffDelay = time.Duration(float64(errorBackoffDelay) * Config.BackoffMultiplier)
if errorBackoffDelay > maxErrorBackoff { if errorBackoffDelay > maxErrorBackoff {
errorBackoffDelay = maxErrorBackoff errorBackoffDelay = maxErrorBackoff
} }
} else { } else {
getOutputStreamingLogger().Info().Msg("Audio system reinitialized successfully") getOutputStreamingLogger().Info().Msg("Audio system reinitialized successfully")
consecutiveErrors = 0 consecutiveErrors = 0
errorBackoffDelay = GetConfig().RetryDelay // Reset backoff errorBackoffDelay = Config.RetryDelay // Reset backoff
} }
} else { } else {
// Brief delay for transient errors // Brief delay for transient errors
time.Sleep(GetConfig().ShortSleepDuration) time.Sleep(Config.ShortSleepDuration)
} }
continue continue
} }
@ -142,7 +139,7 @@ func StartAudioOutputStreaming(send func([]byte)) error {
// Success - reset error counters // Success - reset error counters
if consecutiveErrors > 0 { if consecutiveErrors > 0 {
consecutiveErrors = 0 consecutiveErrors = 0
errorBackoffDelay = GetConfig().RetryDelay errorBackoffDelay = Config.RetryDelay
} }
if n > 0 { if n > 0 {
@ -164,7 +161,7 @@ func StartAudioOutputStreaming(send func([]byte)) error {
RecordFrameReceived(n) RecordFrameReceived(n)
} }
// Small delay to prevent busy waiting // Small delay to prevent busy waiting
time.Sleep(GetConfig().ShortSleepDuration) time.Sleep(Config.ShortSleepDuration)
} }
} }
}() }()
@ -185,6 +182,6 @@ func StopAudioOutputStreaming() {
// Wait for streaming to stop // Wait for streaming to stop
for atomic.LoadInt32(&outputStreamingRunning) == 1 { for atomic.LoadInt32(&outputStreamingRunning) == 1 {
time.Sleep(GetConfig().ShortSleepDuration) time.Sleep(Config.ShortSleepDuration)
} }
} }

View File

@ -19,19 +19,19 @@ const (
// Restart configuration is now retrieved from centralized config // Restart configuration is now retrieved from centralized config
func getMaxRestartAttempts() int { func getMaxRestartAttempts() int {
return GetConfig().MaxRestartAttempts return Config.MaxRestartAttempts
} }
func getRestartWindow() time.Duration { func getRestartWindow() time.Duration {
return GetConfig().RestartWindow return Config.RestartWindow
} }
func getRestartDelay() time.Duration { func getRestartDelay() time.Duration {
return GetConfig().RestartDelay return Config.RestartDelay
} }
func getMaxRestartDelay() time.Duration { func getMaxRestartDelay() time.Duration {
return GetConfig().MaxRestartDelay return Config.MaxRestartDelay
} }
// AudioOutputSupervisor manages the audio output server subprocess lifecycle // AudioOutputSupervisor manages the audio output server subprocess lifecycle
@ -125,6 +125,12 @@ func (s *AudioOutputSupervisor) Start() error {
// Start the supervision loop // Start the supervision loop
go s.supervisionLoop() go s.supervisionLoop()
// Establish IPC connection to subprocess after a brief delay
go func() {
time.Sleep(500 * time.Millisecond) // Wait for subprocess to start
s.connectClient()
}()
s.logger.Info().Str("component", AudioOutputSupervisorComponent).Msg("component started successfully") s.logger.Info().Str("component", AudioOutputSupervisorComponent).Msg("component started successfully")
return nil return nil
} }
@ -145,11 +151,20 @@ func (s *AudioOutputSupervisor) Stop() {
select { select {
case <-s.processDone: case <-s.processDone:
s.logger.Info().Str("component", AudioOutputSupervisorComponent).Msg("component stopped gracefully") s.logger.Info().Str("component", AudioOutputSupervisorComponent).Msg("component stopped gracefully")
case <-time.After(GetConfig().OutputSupervisorTimeout): case <-time.After(Config.OutputSupervisorTimeout):
s.logger.Warn().Str("component", AudioOutputSupervisorComponent).Msg("component did not stop gracefully, forcing termination") s.logger.Warn().Str("component", AudioOutputSupervisorComponent).Msg("component did not stop gracefully, forcing termination")
s.forceKillProcess("audio output server") s.forceKillProcess("audio output server")
} }
// Ensure socket file cleanup even if subprocess didn't clean up properly
// This prevents "address already in use" errors on restart
outputSocketPath := getOutputSocketPath()
if err := os.Remove(outputSocketPath); err != nil && !os.IsNotExist(err) {
s.logger.Warn().Err(err).Str("socket_path", outputSocketPath).Msg("failed to remove output socket file during supervisor stop")
} else if err == nil {
s.logger.Debug().Str("socket_path", outputSocketPath).Msg("cleaned up output socket file")
}
s.logger.Info().Str("component", AudioOutputSupervisorComponent).Msg("component stopped") s.logger.Info().Str("component", AudioOutputSupervisorComponent).Msg("component stopped")
} }
@ -158,7 +173,7 @@ func (s *AudioOutputSupervisor) supervisionLoop() {
// Configure supervision parameters // Configure supervision parameters
config := SupervisionConfig{ config := SupervisionConfig{
ProcessType: "audio output server", ProcessType: "audio output server",
Timeout: GetConfig().OutputSupervisorTimeout, Timeout: Config.OutputSupervisorTimeout,
EnableRestart: true, EnableRestart: true,
MaxRestartAttempts: getMaxRestartAttempts(), MaxRestartAttempts: getMaxRestartAttempts(),
RestartWindow: getRestartWindow(), RestartWindow: getRestartWindow(),
@ -213,7 +228,6 @@ func (s *AudioOutputSupervisor) startProcess() error {
s.logger.Info().Int("pid", s.processPID).Strs("args", args).Strs("opus_env", s.opusEnv).Msg("audio server process started") s.logger.Info().Int("pid", s.processPID).Strs("args", args).Strs("opus_env", s.opusEnv).Msg("audio server process started")
// Add process to monitoring // Add process to monitoring
s.processMonitor.AddProcess(s.processPID, "audio-output-server")
if s.onProcessStart != nil { if s.onProcessStart != nil {
s.onProcessStart(s.processPID) s.onProcessStart(s.processPID)
@ -275,3 +289,43 @@ func (s *AudioOutputSupervisor) calculateRestartDelay() time.Duration {
return delay return delay
} }
// client holds the IPC client for communicating with the subprocess
var outputClient *AudioOutputClient
// IsConnected returns whether the supervisor has an active connection to the subprocess
func (s *AudioOutputSupervisor) IsConnected() bool {
return outputClient != nil && outputClient.IsConnected()
}
// GetClient returns the IPC client for the subprocess
func (s *AudioOutputSupervisor) GetClient() *AudioOutputClient {
return outputClient
}
// connectClient establishes connection to the audio output subprocess
func (s *AudioOutputSupervisor) connectClient() {
if outputClient == nil {
outputClient = NewAudioOutputClient()
}
// Try to connect to the subprocess
if err := outputClient.Connect(); err != nil {
s.logger.Warn().Err(err).Msg("Failed to connect to audio output subprocess")
} else {
s.logger.Info().Msg("Connected to audio output subprocess")
}
}
// SendOpusConfig sends Opus configuration to the audio output subprocess
func (s *AudioOutputSupervisor) SendOpusConfig(config OutputIPCOpusConfig) error {
if outputClient == nil {
return fmt.Errorf("client not initialized")
}
if !outputClient.IsConnected() {
return fmt.Errorf("client not connected")
}
return outputClient.SendOpusConfig(config)
}

View File

@ -39,7 +39,7 @@ var (
// MaxAudioFrameSize is now retrieved from centralized config // MaxAudioFrameSize is now retrieved from centralized config
func GetMaxAudioFrameSize() int { func GetMaxAudioFrameSize() int {
return GetConfig().MaxAudioFrameSize return Config.MaxAudioFrameSize
} }
// AudioQuality represents different audio quality presets // AudioQuality represents different audio quality presets
@ -74,17 +74,17 @@ type AudioMetrics struct {
var ( var (
currentConfig = AudioConfig{ currentConfig = AudioConfig{
Quality: AudioQualityMedium, Quality: AudioQualityMedium,
Bitrate: GetConfig().AudioQualityMediumOutputBitrate, Bitrate: Config.AudioQualityMediumOutputBitrate,
SampleRate: GetConfig().SampleRate, SampleRate: Config.SampleRate,
Channels: GetConfig().Channels, Channels: Config.Channels,
FrameSize: GetConfig().AudioQualityMediumFrameSize, FrameSize: Config.AudioQualityMediumFrameSize,
} }
currentMicrophoneConfig = AudioConfig{ currentMicrophoneConfig = AudioConfig{
Quality: AudioQualityMedium, Quality: AudioQualityMedium,
Bitrate: GetConfig().AudioQualityMediumInputBitrate, Bitrate: Config.AudioQualityMediumInputBitrate,
SampleRate: GetConfig().SampleRate, SampleRate: Config.SampleRate,
Channels: 1, Channels: 1,
FrameSize: GetConfig().AudioQualityMediumFrameSize, FrameSize: Config.AudioQualityMediumFrameSize,
} }
metrics AudioMetrics metrics AudioMetrics
) )
@ -96,24 +96,24 @@ var qualityPresets = map[AudioQuality]struct {
frameSize time.Duration frameSize time.Duration
}{ }{
AudioQualityLow: { AudioQualityLow: {
outputBitrate: GetConfig().AudioQualityLowOutputBitrate, inputBitrate: GetConfig().AudioQualityLowInputBitrate, outputBitrate: Config.AudioQualityLowOutputBitrate, inputBitrate: Config.AudioQualityLowInputBitrate,
sampleRate: GetConfig().AudioQualityLowSampleRate, channels: GetConfig().AudioQualityLowChannels, sampleRate: Config.AudioQualityLowSampleRate, channels: Config.AudioQualityLowChannels,
frameSize: GetConfig().AudioQualityLowFrameSize, frameSize: Config.AudioQualityLowFrameSize,
}, },
AudioQualityMedium: { AudioQualityMedium: {
outputBitrate: GetConfig().AudioQualityMediumOutputBitrate, inputBitrate: GetConfig().AudioQualityMediumInputBitrate, outputBitrate: Config.AudioQualityMediumOutputBitrate, inputBitrate: Config.AudioQualityMediumInputBitrate,
sampleRate: GetConfig().AudioQualityMediumSampleRate, channels: GetConfig().AudioQualityMediumChannels, sampleRate: Config.AudioQualityMediumSampleRate, channels: Config.AudioQualityMediumChannels,
frameSize: GetConfig().AudioQualityMediumFrameSize, frameSize: Config.AudioQualityMediumFrameSize,
}, },
AudioQualityHigh: { AudioQualityHigh: {
outputBitrate: GetConfig().AudioQualityHighOutputBitrate, inputBitrate: GetConfig().AudioQualityHighInputBitrate, outputBitrate: Config.AudioQualityHighOutputBitrate, inputBitrate: Config.AudioQualityHighInputBitrate,
sampleRate: GetConfig().SampleRate, channels: GetConfig().AudioQualityHighChannels, sampleRate: Config.SampleRate, channels: Config.AudioQualityHighChannels,
frameSize: GetConfig().AudioQualityHighFrameSize, frameSize: Config.AudioQualityHighFrameSize,
}, },
AudioQualityUltra: { AudioQualityUltra: {
outputBitrate: GetConfig().AudioQualityUltraOutputBitrate, inputBitrate: GetConfig().AudioQualityUltraInputBitrate, outputBitrate: Config.AudioQualityUltraOutputBitrate, inputBitrate: Config.AudioQualityUltraInputBitrate,
sampleRate: GetConfig().SampleRate, channels: GetConfig().AudioQualityUltraChannels, sampleRate: Config.SampleRate, channels: Config.AudioQualityUltraChannels,
frameSize: GetConfig().AudioQualityUltraFrameSize, frameSize: Config.AudioQualityUltraFrameSize,
}, },
} }
@ -142,7 +142,7 @@ func GetMicrophoneQualityPresets() map[AudioQuality]AudioConfig {
Bitrate: preset.inputBitrate, Bitrate: preset.inputBitrate,
SampleRate: func() int { SampleRate: func() int {
if quality == AudioQualityLow { if quality == AudioQualityLow {
return GetConfig().AudioQualityMicLowSampleRate return Config.AudioQualityMicLowSampleRate
} }
return preset.sampleRate return preset.sampleRate
}(), }(),
@ -172,58 +172,84 @@ func SetAudioQuality(quality AudioQuality) {
var complexity, vbr, signalType, bandwidth, dtx int var complexity, vbr, signalType, bandwidth, dtx int
switch quality { switch quality {
case AudioQualityLow: case AudioQualityLow:
complexity = GetConfig().AudioQualityLowOpusComplexity complexity = Config.AudioQualityLowOpusComplexity
vbr = GetConfig().AudioQualityLowOpusVBR vbr = Config.AudioQualityLowOpusVBR
signalType = GetConfig().AudioQualityLowOpusSignalType signalType = Config.AudioQualityLowOpusSignalType
bandwidth = GetConfig().AudioQualityLowOpusBandwidth bandwidth = Config.AudioQualityLowOpusBandwidth
dtx = GetConfig().AudioQualityLowOpusDTX dtx = Config.AudioQualityLowOpusDTX
case AudioQualityMedium: case AudioQualityMedium:
complexity = GetConfig().AudioQualityMediumOpusComplexity complexity = Config.AudioQualityMediumOpusComplexity
vbr = GetConfig().AudioQualityMediumOpusVBR vbr = Config.AudioQualityMediumOpusVBR
signalType = GetConfig().AudioQualityMediumOpusSignalType signalType = Config.AudioQualityMediumOpusSignalType
bandwidth = GetConfig().AudioQualityMediumOpusBandwidth bandwidth = Config.AudioQualityMediumOpusBandwidth
dtx = GetConfig().AudioQualityMediumOpusDTX dtx = Config.AudioQualityMediumOpusDTX
case AudioQualityHigh: case AudioQualityHigh:
complexity = GetConfig().AudioQualityHighOpusComplexity complexity = Config.AudioQualityHighOpusComplexity
vbr = GetConfig().AudioQualityHighOpusVBR vbr = Config.AudioQualityHighOpusVBR
signalType = GetConfig().AudioQualityHighOpusSignalType signalType = Config.AudioQualityHighOpusSignalType
bandwidth = GetConfig().AudioQualityHighOpusBandwidth bandwidth = Config.AudioQualityHighOpusBandwidth
dtx = GetConfig().AudioQualityHighOpusDTX dtx = Config.AudioQualityHighOpusDTX
case AudioQualityUltra: case AudioQualityUltra:
complexity = GetConfig().AudioQualityUltraOpusComplexity complexity = Config.AudioQualityUltraOpusComplexity
vbr = GetConfig().AudioQualityUltraOpusVBR vbr = Config.AudioQualityUltraOpusVBR
signalType = GetConfig().AudioQualityUltraOpusSignalType signalType = Config.AudioQualityUltraOpusSignalType
bandwidth = GetConfig().AudioQualityUltraOpusBandwidth bandwidth = Config.AudioQualityUltraOpusBandwidth
dtx = GetConfig().AudioQualityUltraOpusDTX dtx = Config.AudioQualityUltraOpusDTX
default: default:
// Use medium quality as fallback // Use medium quality as fallback
complexity = GetConfig().AudioQualityMediumOpusComplexity complexity = Config.AudioQualityMediumOpusComplexity
vbr = GetConfig().AudioQualityMediumOpusVBR vbr = Config.AudioQualityMediumOpusVBR
signalType = GetConfig().AudioQualityMediumOpusSignalType signalType = Config.AudioQualityMediumOpusSignalType
bandwidth = GetConfig().AudioQualityMediumOpusBandwidth bandwidth = Config.AudioQualityMediumOpusBandwidth
dtx = GetConfig().AudioQualityMediumOpusDTX dtx = Config.AudioQualityMediumOpusDTX
} }
// Restart audio output subprocess with new OPUS configuration // Update audio output subprocess configuration dynamically without restart
if supervisor := GetAudioOutputSupervisor(); supervisor != nil {
logger := logging.GetDefaultLogger().With().Str("component", "audio").Logger() logger := logging.GetDefaultLogger().With().Str("component", "audio").Logger()
logger.Info().Int("quality", int(quality)).Msg("restarting audio output subprocess with new quality settings") logger.Info().Int("quality", int(quality)).Msg("updating audio output quality settings dynamically")
// Set new OPUS configuration // Set new OPUS configuration for future restarts
if supervisor := GetAudioOutputSupervisor(); supervisor != nil {
supervisor.SetOpusConfig(config.Bitrate*1000, complexity, vbr, signalType, bandwidth, dtx) supervisor.SetOpusConfig(config.Bitrate*1000, complexity, vbr, signalType, bandwidth, dtx)
// Stop current subprocess // Send dynamic configuration update to running subprocess via IPC
supervisor.Stop() if supervisor.IsConnected() {
// Convert AudioConfig to OutputIPCOpusConfig with complete Opus parameters
opusConfig := OutputIPCOpusConfig{
SampleRate: config.SampleRate,
Channels: config.Channels,
FrameSize: int(config.FrameSize.Milliseconds() * int64(config.SampleRate) / 1000), // Convert ms to samples
Bitrate: config.Bitrate * 1000, // Convert kbps to bps
Complexity: complexity,
VBR: vbr,
SignalType: signalType,
Bandwidth: bandwidth,
DTX: dtx,
}
// Start subprocess with new configuration logger.Info().Interface("opusConfig", opusConfig).Msg("sending Opus configuration to audio output subprocess")
if err := supervisor.SendOpusConfig(opusConfig); err != nil {
logger.Warn().Err(err).Msg("failed to send dynamic Opus config update via IPC, falling back to subprocess restart")
// Fallback to subprocess restart if IPC update fails
supervisor.Stop()
if err := supervisor.Start(); err != nil { if err := supervisor.Start(); err != nil {
logger.Error().Err(err).Msg("failed to restart audio output subprocess") logger.Error().Err(err).Msg("failed to restart audio output subprocess after IPC update failure")
} }
} else { } else {
// Fallback to dynamic update if supervisor is not available logger.Info().Msg("audio output quality updated dynamically via IPC")
vbrConstraint := GetConfig().CGOOpusVBRConstraint
if err := updateOpusEncoderParams(config.Bitrate*1000, complexity, vbr, vbrConstraint, signalType, bandwidth, dtx); err != nil { // Reset audio output stats after config update
logging.GetDefaultLogger().Error().Err(err).Msg("Failed to update OPUS encoder parameters") go func() {
time.Sleep(Config.QualityChangeSettleDelay) // Wait for quality change to settle
// Reset audio input server stats to clear persistent warnings
ResetGlobalAudioInputServerStats()
// Attempt recovery if there are still issues
time.Sleep(1 * time.Second)
RecoverGlobalAudioInputServer()
}()
}
} else {
logger.Info().Bool("supervisor_running", supervisor.IsRunning()).Msg("audio output subprocess not connected, configuration will apply on next start")
} }
} }
} }
@ -234,6 +260,16 @@ func GetAudioConfig() AudioConfig {
return currentConfig return currentConfig
} }
// Simplified OPUS parameter lookup table
var opusParams = map[AudioQuality]struct {
complexity, vbr, signalType, bandwidth, dtx int
}{
AudioQualityLow: {Config.AudioQualityLowOpusComplexity, Config.AudioQualityLowOpusVBR, Config.AudioQualityLowOpusSignalType, Config.AudioQualityLowOpusBandwidth, Config.AudioQualityLowOpusDTX},
AudioQualityMedium: {Config.AudioQualityMediumOpusComplexity, Config.AudioQualityMediumOpusVBR, Config.AudioQualityMediumOpusSignalType, Config.AudioQualityMediumOpusBandwidth, Config.AudioQualityMediumOpusDTX},
AudioQualityHigh: {Config.AudioQualityHighOpusComplexity, Config.AudioQualityHighOpusVBR, Config.AudioQualityHighOpusSignalType, Config.AudioQualityHighOpusBandwidth, Config.AudioQualityHighOpusDTX},
AudioQualityUltra: {Config.AudioQualityUltraOpusComplexity, Config.AudioQualityUltraOpusVBR, Config.AudioQualityUltraOpusSignalType, Config.AudioQualityUltraOpusBandwidth, Config.AudioQualityUltraOpusDTX},
}
// SetMicrophoneQuality updates the current microphone quality configuration // SetMicrophoneQuality updates the current microphone quality configuration
func SetMicrophoneQuality(quality AudioQuality) { func SetMicrophoneQuality(quality AudioQuality) {
// Validate audio quality parameter // Validate audio quality parameter
@ -248,51 +284,32 @@ func SetMicrophoneQuality(quality AudioQuality) {
if config, exists := presets[quality]; exists { if config, exists := presets[quality]; exists {
currentMicrophoneConfig = config currentMicrophoneConfig = config
// Get OPUS parameters for the selected quality // Get OPUS parameters using lookup table
var complexity, vbr, signalType, bandwidth, dtx int params, exists := opusParams[quality]
switch quality { if !exists {
case AudioQualityLow: // Fallback to medium quality
complexity = GetConfig().AudioQualityLowOpusComplexity params = opusParams[AudioQualityMedium]
vbr = GetConfig().AudioQualityLowOpusVBR
signalType = GetConfig().AudioQualityLowOpusSignalType
bandwidth = GetConfig().AudioQualityLowOpusBandwidth
dtx = GetConfig().AudioQualityLowOpusDTX
case AudioQualityMedium:
complexity = GetConfig().AudioQualityMediumOpusComplexity
vbr = GetConfig().AudioQualityMediumOpusVBR
signalType = GetConfig().AudioQualityMediumOpusSignalType
bandwidth = GetConfig().AudioQualityMediumOpusBandwidth
dtx = GetConfig().AudioQualityMediumOpusDTX
case AudioQualityHigh:
complexity = GetConfig().AudioQualityHighOpusComplexity
vbr = GetConfig().AudioQualityHighOpusVBR
signalType = GetConfig().AudioQualityHighOpusSignalType
bandwidth = GetConfig().AudioQualityHighOpusBandwidth
dtx = GetConfig().AudioQualityHighOpusDTX
case AudioQualityUltra:
complexity = GetConfig().AudioQualityUltraOpusComplexity
vbr = GetConfig().AudioQualityUltraOpusVBR
signalType = GetConfig().AudioQualityUltraOpusSignalType
bandwidth = GetConfig().AudioQualityUltraOpusBandwidth
dtx = GetConfig().AudioQualityUltraOpusDTX
default:
// Use medium quality as fallback
complexity = GetConfig().AudioQualityMediumOpusComplexity
vbr = GetConfig().AudioQualityMediumOpusVBR
signalType = GetConfig().AudioQualityMediumOpusSignalType
bandwidth = GetConfig().AudioQualityMediumOpusBandwidth
dtx = GetConfig().AudioQualityMediumOpusDTX
} }
// Update audio input subprocess configuration dynamically without restart // Update audio input subprocess configuration dynamically without restart
if supervisor := GetAudioInputSupervisor(); supervisor != nil {
logger := logging.GetDefaultLogger().With().Str("component", "audio").Logger() logger := logging.GetDefaultLogger().With().Str("component", "audio").Logger()
logger.Info().Int("quality", int(quality)).Msg("updating audio input subprocess quality settings dynamically")
// Set new OPUS configuration for future restarts // Set new OPUS configuration for future restarts
supervisor.SetOpusConfig(config.Bitrate*1000, complexity, vbr, signalType, bandwidth, dtx) if supervisor := GetAudioInputSupervisor(); supervisor != nil {
supervisor.SetOpusConfig(config.Bitrate*1000, params.complexity, params.vbr, params.signalType, params.bandwidth, params.dtx)
// Send dynamic configuration update to running subprocess // Check if microphone is active but IPC control is broken
inputManager := getAudioInputManager()
if inputManager.IsRunning() && !supervisor.IsConnected() {
// Reconnect the IPC control channel
supervisor.Stop()
time.Sleep(50 * time.Millisecond)
if err := supervisor.Start(); err != nil {
logger.Debug().Err(err).Msg("failed to reconnect IPC control channel")
}
}
// Send dynamic configuration update to running subprocess via IPC
if supervisor.IsConnected() { if supervisor.IsConnected() {
// Convert AudioConfig to InputIPCOpusConfig with complete Opus parameters // Convert AudioConfig to InputIPCOpusConfig with complete Opus parameters
opusConfig := InputIPCOpusConfig{ opusConfig := InputIPCOpusConfig{
@ -300,23 +317,32 @@ func SetMicrophoneQuality(quality AudioQuality) {
Channels: config.Channels, Channels: config.Channels,
FrameSize: int(config.FrameSize.Milliseconds() * int64(config.SampleRate) / 1000), // Convert ms to samples FrameSize: int(config.FrameSize.Milliseconds() * int64(config.SampleRate) / 1000), // Convert ms to samples
Bitrate: config.Bitrate * 1000, // Convert kbps to bps Bitrate: config.Bitrate * 1000, // Convert kbps to bps
Complexity: complexity, Complexity: params.complexity,
VBR: vbr, VBR: params.vbr,
SignalType: signalType, SignalType: params.signalType,
Bandwidth: bandwidth, Bandwidth: params.bandwidth,
DTX: dtx, DTX: params.dtx,
} }
logger.Info().Interface("opusConfig", opusConfig).Msg("sending Opus configuration to audio input subprocess")
if err := supervisor.SendOpusConfig(opusConfig); err != nil { if err := supervisor.SendOpusConfig(opusConfig); err != nil {
logger.Warn().Err(err).Msg("failed to send dynamic Opus config update, subprocess may need restart") logger.Debug().Err(err).Msg("failed to send dynamic Opus config update via IPC")
// Fallback to restart if dynamic update fails // Fallback to subprocess restart if IPC update fails
supervisor.Stop() supervisor.Stop()
if err := supervisor.Start(); err != nil { if err := supervisor.Start(); err != nil {
logger.Error().Err(err).Msg("failed to restart audio input subprocess after config update failure") logger.Error().Err(err).Msg("failed to restart audio input subprocess after IPC update failure")
} }
} else { } else {
logger.Info().Msg("audio input quality updated dynamically with complete Opus configuration") logger.Info().Msg("audio input quality updated dynamically via IPC")
// Reset audio input stats after config update
go func() {
time.Sleep(Config.QualityChangeSettleDelay) // Wait for quality change to settle
// Reset audio input server stats to clear persistent warnings
ResetGlobalAudioInputServerStats()
// Attempt recovery if microphone is still having issues
time.Sleep(1 * time.Second)
RecoverGlobalAudioInputServer()
}()
} }
} else { } else {
logger.Info().Bool("supervisor_running", supervisor.IsRunning()).Msg("audio input subprocess not connected, configuration will apply on next start") logger.Info().Bool("supervisor_running", supervisor.IsRunning()).Msg("audio input subprocess not connected, configuration will apply on next start")

View File

@ -2,7 +2,9 @@ package audio
import ( import (
"errors" "errors"
"fmt"
"sync" "sync"
"time"
) )
// Global relay instance for the main process // Global relay instance for the main process
@ -28,13 +30,34 @@ func StartAudioRelay(audioTrack AudioTrackWriter) error {
// Get current audio config // Get current audio config
config := GetAudioConfig() config := GetAudioConfig()
// Start the relay (audioTrack can be nil initially) // Retry starting the relay with exponential backoff
// This handles cases where the subprocess hasn't created its socket yet
maxAttempts := 5
baseDelay := 200 * time.Millisecond
maxDelay := 2 * time.Second
var lastErr error
for i := 0; i < maxAttempts; i++ {
if err := relay.Start(audioTrack, config); err != nil { if err := relay.Start(audioTrack, config); err != nil {
return err lastErr = err
if i < maxAttempts-1 {
// Calculate exponential backoff delay
delay := time.Duration(float64(baseDelay) * (1.5 * float64(i+1)))
if delay > maxDelay {
delay = maxDelay
}
time.Sleep(delay)
continue
}
return fmt.Errorf("failed to start audio relay after %d attempts: %w", maxAttempts, lastErr)
} }
// Success
globalRelay = relay globalRelay = relay
return nil return nil
}
return fmt.Errorf("failed to start audio relay after %d attempts: %w", maxAttempts, lastErr)
} }
// StopAudioRelay stops the audio relay system // StopAudioRelay stops the audio relay system
@ -89,37 +112,93 @@ func IsAudioRelayRunning() bool {
} }
// UpdateAudioRelayTrack updates the WebRTC audio track for the relay // UpdateAudioRelayTrack updates the WebRTC audio track for the relay
// This function is refactored to prevent mutex deadlocks during quality changes
func UpdateAudioRelayTrack(audioTrack AudioTrackWriter) error { func UpdateAudioRelayTrack(audioTrack AudioTrackWriter) error {
relayMutex.Lock() var needsCallback bool
defer relayMutex.Unlock() var callbackFunc TrackReplacementCallback
// Critical section: minimize time holding the mutex
relayMutex.Lock()
if globalRelay == nil { if globalRelay == nil {
// No relay running, start one with the provided track // No relay running, start one with the provided track
relay := NewAudioRelay() relay := NewAudioRelay()
config := GetAudioConfig() config := GetAudioConfig()
if err := relay.Start(audioTrack, config); err != nil { if err := relay.Start(audioTrack, config); err != nil {
relayMutex.Unlock()
return err return err
} }
globalRelay = relay globalRelay = relay
return nil } else {
}
// Update the track in the existing relay // Update the track in the existing relay
globalRelay.UpdateTrack(audioTrack) globalRelay.UpdateTrack(audioTrack)
}
// Capture callback state while holding mutex
needsCallback = trackReplacementCallback != nil
if needsCallback {
callbackFunc = trackReplacementCallback
}
relayMutex.Unlock()
// Execute callback outside of mutex to prevent deadlock
if needsCallback && callbackFunc != nil {
// Use goroutine with timeout to prevent blocking
done := make(chan error, 1)
go func() {
done <- callbackFunc(audioTrack)
}()
// Wait for callback with timeout
select {
case err := <-done:
if err != nil {
// Log error but don't fail the relay operation
// The relay can still work even if WebRTC track replacement fails
_ = err // Suppress linter warning
}
case <-time.After(5 * time.Second):
// Timeout: log warning but continue
// This prevents indefinite blocking during quality changes
_ = fmt.Errorf("track replacement callback timed out")
}
}
return nil return nil
} }
// CurrentSessionCallback is a function type for getting the current session's audio track // CurrentSessionCallback is a function type for getting the current session's audio track
type CurrentSessionCallback func() AudioTrackWriter type CurrentSessionCallback func() AudioTrackWriter
// TrackReplacementCallback is a function type for replacing the WebRTC audio track
type TrackReplacementCallback func(AudioTrackWriter) error
// currentSessionCallback holds the callback function to get the current session's audio track // currentSessionCallback holds the callback function to get the current session's audio track
var currentSessionCallback CurrentSessionCallback var currentSessionCallback CurrentSessionCallback
// trackReplacementCallback holds the callback function to replace the WebRTC audio track
var trackReplacementCallback TrackReplacementCallback
// SetCurrentSessionCallback sets the callback function to get the current session's audio track // SetCurrentSessionCallback sets the callback function to get the current session's audio track
func SetCurrentSessionCallback(callback CurrentSessionCallback) { func SetCurrentSessionCallback(callback CurrentSessionCallback) {
currentSessionCallback = callback currentSessionCallback = callback
} }
// SetTrackReplacementCallback sets the callback function to replace the WebRTC audio track
func SetTrackReplacementCallback(callback TrackReplacementCallback) {
trackReplacementCallback = callback
}
// UpdateAudioRelayTrackAsync performs async track update to prevent blocking
// This is used during WebRTC session creation to avoid deadlocks
func UpdateAudioRelayTrackAsync(audioTrack AudioTrackWriter) {
go func() {
if err := UpdateAudioRelayTrack(audioTrack); err != nil {
// Log error but don't block session creation
_ = err // Suppress linter warning
}
}()
}
// connectRelayToCurrentSession connects the audio relay to the current WebRTC session's audio track // connectRelayToCurrentSession connects the audio relay to the current WebRTC session's audio track
// This is used when restarting the relay during unmute operations // This is used when restarting the relay during unmute operations
func connectRelayToCurrentSession() error { func connectRelayToCurrentSession() error {

View File

@ -18,8 +18,8 @@ type SocketBufferConfig struct {
// DefaultSocketBufferConfig returns the default socket buffer configuration // DefaultSocketBufferConfig returns the default socket buffer configuration
func DefaultSocketBufferConfig() SocketBufferConfig { func DefaultSocketBufferConfig() SocketBufferConfig {
return SocketBufferConfig{ return SocketBufferConfig{
SendBufferSize: GetConfig().SocketOptimalBuffer, SendBufferSize: Config.SocketOptimalBuffer,
RecvBufferSize: GetConfig().SocketOptimalBuffer, RecvBufferSize: Config.SocketOptimalBuffer,
Enabled: true, Enabled: true,
} }
} }
@ -27,8 +27,8 @@ func DefaultSocketBufferConfig() SocketBufferConfig {
// HighLoadSocketBufferConfig returns configuration for high-load scenarios // HighLoadSocketBufferConfig returns configuration for high-load scenarios
func HighLoadSocketBufferConfig() SocketBufferConfig { func HighLoadSocketBufferConfig() SocketBufferConfig {
return SocketBufferConfig{ return SocketBufferConfig{
SendBufferSize: GetConfig().SocketMaxBuffer, SendBufferSize: Config.SocketMaxBuffer,
RecvBufferSize: GetConfig().SocketMaxBuffer, RecvBufferSize: Config.SocketMaxBuffer,
Enabled: true, Enabled: true,
} }
} }
@ -123,8 +123,8 @@ func ValidateSocketBufferConfig(config SocketBufferConfig) error {
return nil return nil
} }
minBuffer := GetConfig().SocketMinBuffer minBuffer := Config.SocketMinBuffer
maxBuffer := GetConfig().SocketMaxBuffer maxBuffer := Config.SocketMaxBuffer
if config.SendBufferSize < minBuffer { if config.SendBufferSize < minBuffer {
return fmt.Errorf("send buffer size validation failed: got %d bytes, minimum required %d bytes (configured range: %d-%d)", return fmt.Errorf("send buffer size validation failed: got %d bytes, minimum required %d bytes (configured range: %d-%d)",

View File

@ -4,765 +4,138 @@
package audio package audio
import ( import (
"runtime"
"sort"
"sync"
"sync/atomic" "sync/atomic"
"time"
"unsafe"
) )
// AudioLatencyInfo holds simplified latency information for cleanup decisions // AudioBufferPool provides a simple buffer pool for audio processing
type AudioLatencyInfo struct {
LatencyMs float64
Timestamp time.Time
}
// Global latency tracking
var (
currentAudioLatency = AudioLatencyInfo{}
currentAudioLatencyLock sync.RWMutex
audioMonitoringInitialized int32 // Atomic flag to track initialization
)
// InitializeAudioMonitoring starts the background goroutines for latency tracking and cache cleanup
// This is safe to call multiple times as it will only initialize once
func InitializeAudioMonitoring() {
// Use atomic CAS to ensure we only initialize once
if atomic.CompareAndSwapInt32(&audioMonitoringInitialized, 0, 1) {
// Start the latency recorder
startLatencyRecorder()
// Start the cleanup goroutine
startCleanupGoroutine()
}
}
// latencyChannel is used for non-blocking latency recording
var latencyChannel = make(chan float64, 10)
// startLatencyRecorder starts the latency recorder goroutine
// This should be called during package initialization
func startLatencyRecorder() {
go latencyRecorderLoop()
}
// latencyRecorderLoop processes latency recordings in the background
func latencyRecorderLoop() {
for latencyMs := range latencyChannel {
currentAudioLatencyLock.Lock()
currentAudioLatency = AudioLatencyInfo{
LatencyMs: latencyMs,
Timestamp: time.Now(),
}
currentAudioLatencyLock.Unlock()
}
}
// RecordAudioLatency records the current audio processing latency
// This is called from the audio input manager when latency is measured
// It is non-blocking to ensure zero overhead in the critical audio path
func RecordAudioLatency(latencyMs float64) {
// Non-blocking send - if channel is full, we drop the update
select {
case latencyChannel <- latencyMs:
// Successfully sent
default:
// Channel full, drop this update to avoid blocking the audio path
}
}
// GetAudioLatencyMetrics returns the current audio latency information
// Returns nil if no latency data is available or if it's too old
func GetAudioLatencyMetrics() *AudioLatencyInfo {
currentAudioLatencyLock.RLock()
defer currentAudioLatencyLock.RUnlock()
// Check if we have valid latency data
if currentAudioLatency.Timestamp.IsZero() {
return nil
}
// Check if the data is too old (more than 5 seconds)
if time.Since(currentAudioLatency.Timestamp) > 5*time.Second {
return nil
}
return &AudioLatencyInfo{
LatencyMs: currentAudioLatency.LatencyMs,
Timestamp: currentAudioLatency.Timestamp,
}
}
// Enhanced lock-free buffer cache for per-goroutine optimization
type lockFreeBufferCache struct {
buffers [8]*[]byte // Increased from 4 to 8 buffers per goroutine cache for better hit rates
}
const (
// Enhanced cache configuration for per-goroutine optimization
cacheSize = 8 // Increased from 4 to 8 buffers per goroutine cache for better hit rates
cacheTTL = 10 * time.Second // Increased from 5s to 10s for better cache retention
// Additional cache constants for enhanced performance
maxCacheEntries = 256 // Maximum number of goroutine cache entries to prevent memory bloat
cacheCleanupInterval = 30 * time.Second // How often to clean up stale cache entries
cacheWarmupThreshold = 50 // Number of requests before enabling cache warmup
cacheHitRateTarget = 0.85 // Target cache hit rate for optimization
)
// TTL tracking for goroutine cache entries
type cacheEntry struct {
cache *lockFreeBufferCache
lastAccess int64 // Unix timestamp of last access
gid int64 // Goroutine ID for better tracking
}
// Per-goroutine buffer cache using goroutine-local storage
var goroutineBufferCache = make(map[int64]*lockFreeBufferCache)
var goroutineCacheMutex sync.RWMutex
var lastCleanupTime int64 // Unix timestamp of last cleanup
const maxCacheSize = 500 // Maximum number of goroutine caches (reduced from 1000)
const cleanupInterval int64 = 30 // Cleanup interval in seconds (30 seconds, reduced from 60)
const bufferTTL int64 = 60 // Time-to-live for cached buffers in seconds (1 minute, reduced from 2)
// getGoroutineID extracts goroutine ID from runtime stack for cache key
func getGoroutineID() int64 {
b := make([]byte, 64)
b = b[:runtime.Stack(b, false)]
// Parse "goroutine 123 [running]:" format
for i := 10; i < len(b); i++ {
if b[i] == ' ' {
id := int64(0)
for j := 10; j < i; j++ {
if b[j] >= '0' && b[j] <= '9' {
id = id*10 + int64(b[j]-'0')
}
}
return id
}
}
return 0
}
// Map of goroutine ID to cache entry with TTL tracking
var goroutineCacheWithTTL = make(map[int64]*cacheEntry)
// cleanupChannel is used for asynchronous cleanup requests
var cleanupChannel = make(chan struct{}, 1)
// startCleanupGoroutine starts the cleanup goroutine
// This should be called during package initialization
func startCleanupGoroutine() {
go cleanupLoop()
}
// cleanupLoop processes cleanup requests in the background
func cleanupLoop() {
ticker := time.NewTicker(10 * time.Second)
defer ticker.Stop()
for {
select {
case <-cleanupChannel:
// Received explicit cleanup request
performCleanup(true)
case <-ticker.C:
// Regular cleanup check
performCleanup(false)
}
}
}
// requestCleanup signals the cleanup goroutine to perform a cleanup
// This is non-blocking and can be called from the critical path
func requestCleanup() {
select {
case cleanupChannel <- struct{}{}:
// Successfully requested cleanup
default:
// Channel full, cleanup already pending
}
}
// performCleanup does the actual cache cleanup work
// This runs in a dedicated goroutine, not in the critical path
func performCleanup(forced bool) {
now := time.Now().Unix()
lastCleanup := atomic.LoadInt64(&lastCleanupTime)
// Check if we're in a high-latency situation
isHighLatency := false
latencyMetrics := GetAudioLatencyMetrics()
if latencyMetrics != nil && latencyMetrics.LatencyMs > 10.0 {
// Under high latency, be more aggressive with cleanup
isHighLatency = true
}
// Only cleanup if enough time has passed (less time if high latency) or if forced
interval := cleanupInterval
if isHighLatency {
interval = cleanupInterval / 2 // More frequent cleanup under high latency
}
if !forced && now-lastCleanup < interval {
return
}
// Try to acquire cleanup lock atomically
if !atomic.CompareAndSwapInt64(&lastCleanupTime, lastCleanup, now) {
return // Another goroutine is already cleaning up
}
// Perform the actual cleanup
doCleanupGoroutineCache()
}
// cleanupGoroutineCache triggers an asynchronous cleanup of the goroutine cache
// This is safe to call from the critical path as it's non-blocking
func cleanupGoroutineCache() {
// Request asynchronous cleanup
requestCleanup()
}
// The actual cleanup implementation that runs in the background goroutine
func doCleanupGoroutineCache() {
// Get current time for TTL calculations
now := time.Now().Unix()
// Check if we're in a high-latency situation
isHighLatency := false
latencyMetrics := GetAudioLatencyMetrics()
if latencyMetrics != nil && latencyMetrics.LatencyMs > 10.0 {
// Under high latency, be more aggressive with cleanup
isHighLatency = true
}
goroutineCacheMutex.Lock()
defer goroutineCacheMutex.Unlock()
// Convert old cache format to new TTL-based format if needed
if len(goroutineCacheWithTTL) == 0 && len(goroutineBufferCache) > 0 {
for gid, cache := range goroutineBufferCache {
goroutineCacheWithTTL[gid] = &cacheEntry{
cache: cache,
lastAccess: now,
gid: gid,
}
}
// Clear old cache to free memory
goroutineBufferCache = make(map[int64]*lockFreeBufferCache)
}
// Enhanced cleanup with size limits and better TTL management
entriesToRemove := make([]int64, 0)
ttl := bufferTTL
if isHighLatency {
// Under high latency, use a much shorter TTL
ttl = bufferTTL / 4
}
// Remove entries older than enhanced TTL
for gid, entry := range goroutineCacheWithTTL {
// Both now and entry.lastAccess are int64, so this comparison is safe
if now-entry.lastAccess > ttl {
entriesToRemove = append(entriesToRemove, gid)
}
}
// If we have too many cache entries, remove the oldest ones
if len(goroutineCacheWithTTL) > maxCacheEntries {
// Sort by last access time and remove oldest entries
type cacheEntryWithGID struct {
gid int64
lastAccess int64
}
entries := make([]cacheEntryWithGID, 0, len(goroutineCacheWithTTL))
for gid, entry := range goroutineCacheWithTTL {
entries = append(entries, cacheEntryWithGID{gid: gid, lastAccess: entry.lastAccess})
}
// Sort by last access time (oldest first)
sort.Slice(entries, func(i, j int) bool {
return entries[i].lastAccess < entries[j].lastAccess
})
// Mark oldest entries for removal
excessCount := len(goroutineCacheWithTTL) - maxCacheEntries
for i := 0; i < excessCount && i < len(entries); i++ {
entriesToRemove = append(entriesToRemove, entries[i].gid)
}
}
// If cache is still too large after TTL cleanup, remove oldest entries
// Under high latency, use a more aggressive target size
targetSize := maxCacheSize
targetReduction := maxCacheSize / 2
if isHighLatency {
// Under high latency, target a much smaller cache size
targetSize = maxCacheSize / 4
targetReduction = maxCacheSize / 8
}
if len(goroutineCacheWithTTL) > targetSize {
// Find oldest entries
type ageEntry struct {
gid int64
lastAccess int64
}
oldestEntries := make([]ageEntry, 0, len(goroutineCacheWithTTL))
for gid, entry := range goroutineCacheWithTTL {
oldestEntries = append(oldestEntries, ageEntry{gid, entry.lastAccess})
}
// Sort by lastAccess (oldest first)
sort.Slice(oldestEntries, func(i, j int) bool {
return oldestEntries[i].lastAccess < oldestEntries[j].lastAccess
})
// Remove oldest entries to get down to target reduction size
toRemove := len(goroutineCacheWithTTL) - targetReduction
for i := 0; i < toRemove && i < len(oldestEntries); i++ {
entriesToRemove = append(entriesToRemove, oldestEntries[i].gid)
}
}
// Remove marked entries and return their buffers to the pool
for _, gid := range entriesToRemove {
if entry, exists := goroutineCacheWithTTL[gid]; exists {
// Return buffers to main pool before removing entry
for i, buf := range entry.cache.buffers {
if buf != nil {
// Clear the buffer slot atomically
entry.cache.buffers[i] = nil
}
}
delete(goroutineCacheWithTTL, gid)
}
}
}
type AudioBufferPool struct { type AudioBufferPool struct {
// Atomic fields MUST be first for ARM32 alignment (int64 fields need 8-byte alignment) // Atomic counters
currentSize int64 // Current pool size (atomic)
hitCount int64 // Pool hit counter (atomic) hitCount int64 // Pool hit counter (atomic)
missCount int64 // Pool miss counter (atomic) missCount int64 // Pool miss counter (atomic)
// Other fields // Pool configuration
pool sync.Pool
bufferSize int bufferSize int
maxPoolSize int pool chan []byte
mutex sync.RWMutex maxSize int
// Memory optimization fields
preallocated []*[]byte // Pre-allocated buffers for immediate use
preallocSize int // Number of pre-allocated buffers
} }
// NewAudioBufferPool creates a new simple audio buffer pool
func NewAudioBufferPool(bufferSize int) *AudioBufferPool { func NewAudioBufferPool(bufferSize int) *AudioBufferPool {
// Validate buffer size parameter maxSize := Config.MaxPoolSize
if err := ValidateBufferSize(bufferSize); err != nil { if maxSize <= 0 {
// Use default value on validation error maxSize = Config.BufferPoolDefaultSize
bufferSize = GetConfig().AudioFramePoolSize
} }
// Enhanced preallocation strategy based on buffer size and system capacity pool := &AudioBufferPool{
var preallocSize int
if bufferSize <= GetConfig().AudioFramePoolSize {
// For smaller pools, use enhanced preallocation (40% instead of 20%)
preallocSize = GetConfig().PreallocPercentage * 2
} else {
// For larger pools, use standard enhanced preallocation (30% instead of 10%)
preallocSize = (GetConfig().PreallocPercentage * 3) / 2
}
// Ensure minimum preallocation for better performance
minPrealloc := 50 // Minimum 50 buffers for startup performance
if preallocSize < minPrealloc {
preallocSize = minPrealloc
}
// Pre-allocate with exact capacity to avoid slice growth
preallocated := make([]*[]byte, 0, preallocSize)
// Pre-allocate buffers with optimized capacity
for i := 0; i < preallocSize; i++ {
// Use exact buffer size to prevent over-allocation
buf := make([]byte, 0, bufferSize)
preallocated = append(preallocated, &buf)
}
return &AudioBufferPool{
bufferSize: bufferSize, bufferSize: bufferSize,
maxPoolSize: GetConfig().MaxPoolSize * 2, // Double the max pool size for better buffering pool: make(chan []byte, maxSize),
preallocated: preallocated, maxSize: maxSize,
preallocSize: preallocSize,
pool: sync.Pool{
New: func() interface{} {
// Allocate exact size to minimize memory waste
buf := make([]byte, 0, bufferSize)
return &buf
},
},
} }
// Pre-populate the pool
for i := 0; i < maxSize/2; i++ {
buf := make([]byte, bufferSize)
select {
case pool.pool <- buf:
default:
break
}
}
return pool
} }
// Get retrieves a buffer from the pool
func (p *AudioBufferPool) Get() []byte { func (p *AudioBufferPool) Get() []byte {
// Skip cleanup trigger in hotpath - cleanup runs in background select {
// cleanupGoroutineCache() - moved to background goroutine case buf := <-p.pool:
// Fast path: Try lock-free per-goroutine cache first
gid := getGoroutineID()
goroutineCacheMutex.RLock()
cacheEntry, exists := goroutineCacheWithTTL[gid]
goroutineCacheMutex.RUnlock()
if exists && cacheEntry != nil && cacheEntry.cache != nil {
// Try to get buffer from lock-free cache
cache := cacheEntry.cache
for i := 0; i < len(cache.buffers); i++ {
bufPtr := (*unsafe.Pointer)(unsafe.Pointer(&cache.buffers[i]))
buf := (*[]byte)(atomic.LoadPointer(bufPtr))
if buf != nil && atomic.CompareAndSwapPointer(bufPtr, unsafe.Pointer(buf), nil) {
// Direct hit count update to avoid sampling complexity in critical path
atomic.AddInt64(&p.hitCount, 1) atomic.AddInt64(&p.hitCount, 1)
*buf = (*buf)[:0] return buf[:0] // Reset length but keep capacity
return *buf default:
}
}
// Update access time only after cache miss to reduce overhead
cacheEntry.lastAccess = time.Now().Unix()
}
// Fallback: Try pre-allocated pool with mutex
p.mutex.Lock()
if len(p.preallocated) > 0 {
lastIdx := len(p.preallocated) - 1
buf := p.preallocated[lastIdx]
p.preallocated = p.preallocated[:lastIdx]
p.mutex.Unlock()
// Direct hit count update to avoid sampling complexity in critical path
atomic.AddInt64(&p.hitCount, 1)
*buf = (*buf)[:0]
return *buf
}
p.mutex.Unlock()
// Try sync.Pool next
if poolBuf := p.pool.Get(); poolBuf != nil {
buf := poolBuf.(*[]byte)
// Direct hit count update to avoid sampling complexity in critical path
atomic.AddInt64(&p.hitCount, 1)
atomic.AddInt64(&p.currentSize, -1)
// Fast capacity check - most buffers should be correct size
if cap(*buf) >= p.bufferSize {
*buf = (*buf)[:0]
return *buf
}
// Buffer too small, fall through to allocation
}
// Pool miss - allocate new buffer with exact capacity
// Direct miss count update to avoid sampling complexity in critical path
atomic.AddInt64(&p.missCount, 1) atomic.AddInt64(&p.missCount, 1)
return make([]byte, 0, p.bufferSize) return make([]byte, 0, p.bufferSize)
}
} }
// Put returns a buffer to the pool
func (p *AudioBufferPool) Put(buf []byte) { func (p *AudioBufferPool) Put(buf []byte) {
// Fast validation - reject buffers that are too small or too large if buf == nil || cap(buf) != p.bufferSize {
bufCap := cap(buf) return // Invalid buffer
if bufCap < p.bufferSize || bufCap > p.bufferSize*2 {
return // Buffer size mismatch, don't pool it to prevent memory bloat
} }
// Enhanced buffer clearing - only clear if buffer contains sensitive data // Reset the buffer
// For audio buffers, we can skip clearing for performance unless needed buf = buf[:0]
// This reduces CPU overhead significantly
var resetBuf []byte
if cap(buf) > p.bufferSize {
// If capacity is larger than expected, create a new properly sized buffer
resetBuf = make([]byte, 0, p.bufferSize)
} else {
// Reset length but keep capacity for reuse efficiency
resetBuf = buf[:0]
}
// Fast path: Try to put in lock-free per-goroutine cache // Try to return to pool
gid := getGoroutineID() select {
goroutineCacheMutex.RLock() case p.pool <- buf:
entryWithTTL, exists := goroutineCacheWithTTL[gid] // Successfully returned to pool
goroutineCacheMutex.RUnlock() default:
// Pool is full, discard buffer
var cache *lockFreeBufferCache
if exists && entryWithTTL != nil {
cache = entryWithTTL.cache
// Update access time only when we successfully use the cache
} else {
// Create new cache for this goroutine
cache = &lockFreeBufferCache{}
now := time.Now().Unix()
goroutineCacheMutex.Lock()
goroutineCacheWithTTL[gid] = &cacheEntry{
cache: cache,
lastAccess: now,
gid: gid,
} }
goroutineCacheMutex.Unlock()
}
if cache != nil {
// Try to store in lock-free cache
for i := 0; i < len(cache.buffers); i++ {
bufPtr := (*unsafe.Pointer)(unsafe.Pointer(&cache.buffers[i]))
if atomic.CompareAndSwapPointer(bufPtr, nil, unsafe.Pointer(&resetBuf)) {
// Update access time only on successful cache
if exists && entryWithTTL != nil {
entryWithTTL.lastAccess = time.Now().Unix()
}
return // Successfully cached
}
}
}
// Fallback: Try to return to pre-allocated pool for fastest reuse
p.mutex.Lock()
if len(p.preallocated) < p.preallocSize {
p.preallocated = append(p.preallocated, &resetBuf)
p.mutex.Unlock()
return
}
p.mutex.Unlock()
// Check sync.Pool size limit to prevent excessive memory usage
if atomic.LoadInt64(&p.currentSize) >= int64(p.maxPoolSize) {
return // Pool is full, let GC handle this buffer
}
// Return to sync.Pool and update counter atomically
p.pool.Put(&resetBuf)
atomic.AddInt64(&p.currentSize, 1)
} }
// Enhanced global buffer pools for different audio frame types with improved sizing // GetStats returns pool statistics
var ( func (p *AudioBufferPool) GetStats() AudioBufferPoolStats {
// Main audio frame pool with enhanced capacity
audioFramePool = NewAudioBufferPool(GetConfig().AudioFramePoolSize)
// Control message pool with enhanced capacity for better throughput
audioControlPool = NewAudioBufferPool(512) // Increased from GetConfig().OutputHeaderSize to 512 for better control message handling
)
func GetAudioFrameBuffer() []byte {
return audioFramePool.Get()
}
func PutAudioFrameBuffer(buf []byte) {
audioFramePool.Put(buf)
}
func GetAudioControlBuffer() []byte {
return audioControlPool.Get()
}
func PutAudioControlBuffer(buf []byte) {
audioControlPool.Put(buf)
}
// GetPoolStats returns detailed statistics about this buffer pool
func (p *AudioBufferPool) GetPoolStats() AudioBufferPoolDetailedStats {
p.mutex.RLock()
preallocatedCount := len(p.preallocated)
currentSize := p.currentSize
p.mutex.RUnlock()
hitCount := atomic.LoadInt64(&p.hitCount) hitCount := atomic.LoadInt64(&p.hitCount)
missCount := atomic.LoadInt64(&p.missCount) missCount := atomic.LoadInt64(&p.missCount)
totalRequests := hitCount + missCount totalRequests := hitCount + missCount
var hitRate float64 var hitRate float64
if totalRequests > 0 { if totalRequests > 0 {
hitRate = float64(hitCount) / float64(totalRequests) * GetConfig().PercentageMultiplier hitRate = float64(hitCount) / float64(totalRequests) * Config.BufferPoolHitRateBase
} }
return AudioBufferPoolDetailedStats{ return AudioBufferPoolStats{
BufferSize: p.bufferSize, BufferSize: p.bufferSize,
MaxPoolSize: p.maxPoolSize, MaxPoolSize: p.maxSize,
CurrentPoolSize: currentSize, CurrentSize: int64(len(p.pool)),
PreallocatedCount: int64(preallocatedCount),
PreallocatedMax: int64(p.preallocSize),
HitCount: hitCount, HitCount: hitCount,
MissCount: missCount, MissCount: missCount,
HitRate: hitRate, HitRate: hitRate,
} }
} }
// AudioBufferPoolDetailedStats provides detailed pool statistics // AudioBufferPoolStats represents pool statistics
type AudioBufferPoolDetailedStats struct { type AudioBufferPoolStats struct {
BufferSize int BufferSize int
MaxPoolSize int MaxPoolSize int
CurrentPoolSize int64 CurrentSize int64
PreallocatedCount int64
PreallocatedMax int64
HitCount int64 HitCount int64
MissCount int64 MissCount int64
HitRate float64 // Percentage HitRate float64
TotalBytes int64 // Total memory usage in bytes
AverageBufferSize float64 // Average size of buffers in the pool
} }
// GetAudioBufferPoolStats returns statistics about the audio buffer pools // Global buffer pools
type AudioBufferPoolStats struct { var (
FramePoolSize int64 audioFramePool = NewAudioBufferPool(Config.AudioFramePoolSize)
FramePoolMax int audioControlPool = NewAudioBufferPool(Config.BufferPoolControlSize)
ControlPoolSize int64 )
ControlPoolMax int
// Enhanced statistics // GetAudioFrameBuffer gets a buffer for audio frames
FramePoolHitRate float64 func GetAudioFrameBuffer() []byte {
ControlPoolHitRate float64 return audioFramePool.Get()
FramePoolDetails AudioBufferPoolDetailedStats
ControlPoolDetails AudioBufferPoolDetailedStats
} }
func GetAudioBufferPoolStats() AudioBufferPoolStats { // PutAudioFrameBuffer returns a buffer to the frame pool
audioFramePool.mutex.RLock() func PutAudioFrameBuffer(buf []byte) {
frameSize := audioFramePool.currentSize audioFramePool.Put(buf)
frameMax := audioFramePool.maxPoolSize
audioFramePool.mutex.RUnlock()
audioControlPool.mutex.RLock()
controlSize := audioControlPool.currentSize
controlMax := audioControlPool.maxPoolSize
audioControlPool.mutex.RUnlock()
// Get detailed statistics
frameDetails := audioFramePool.GetPoolStats()
controlDetails := audioControlPool.GetPoolStats()
return AudioBufferPoolStats{
FramePoolSize: frameSize,
FramePoolMax: frameMax,
ControlPoolSize: controlSize,
ControlPoolMax: controlMax,
FramePoolHitRate: frameDetails.HitRate,
ControlPoolHitRate: controlDetails.HitRate,
FramePoolDetails: frameDetails,
ControlPoolDetails: controlDetails,
}
} }
// AdaptiveResize dynamically adjusts pool parameters based on performance metrics // GetAudioControlBuffer gets a buffer for control messages
func (p *AudioBufferPool) AdaptiveResize() { func GetAudioControlBuffer() []byte {
hitCount := atomic.LoadInt64(&p.hitCount) return audioControlPool.Get()
missCount := atomic.LoadInt64(&p.missCount)
totalRequests := hitCount + missCount
if totalRequests < 100 {
return // Not enough data for meaningful adaptation
}
hitRate := float64(hitCount) / float64(totalRequests)
currentSize := atomic.LoadInt64(&p.currentSize)
// If hit rate is low (< 80%), consider increasing pool size
if hitRate < 0.8 && currentSize < int64(p.maxPoolSize) {
// Increase preallocation by 25% up to max pool size
newPreallocSize := int(float64(len(p.preallocated)) * 1.25)
if newPreallocSize > p.maxPoolSize {
newPreallocSize = p.maxPoolSize
}
// Preallocate additional buffers
for len(p.preallocated) < newPreallocSize {
buf := make([]byte, p.bufferSize)
p.preallocated = append(p.preallocated, &buf)
}
}
// If hit rate is very high (> 95%) and pool is large, consider shrinking
if hitRate > 0.95 && len(p.preallocated) > p.preallocSize {
// Reduce preallocation by 10% but not below original size
newSize := int(float64(len(p.preallocated)) * 0.9)
if newSize < p.preallocSize {
newSize = p.preallocSize
}
// Remove excess preallocated buffers
if newSize < len(p.preallocated) {
p.preallocated = p.preallocated[:newSize]
}
}
} }
// WarmupCache pre-populates goroutine-local caches for better initial performance // PutAudioControlBuffer returns a buffer to the control pool
func (p *AudioBufferPool) WarmupCache() { func PutAudioControlBuffer(buf []byte) {
// Only warmup if we have sufficient request history audioControlPool.Put(buf)
hitCount := atomic.LoadInt64(&p.hitCount)
missCount := atomic.LoadInt64(&p.missCount)
totalRequests := hitCount + missCount
if totalRequests < int64(cacheWarmupThreshold) {
return
}
// Get or create cache for current goroutine
gid := getGoroutineID()
goroutineCacheMutex.RLock()
entryWithTTL, exists := goroutineCacheWithTTL[gid]
goroutineCacheMutex.RUnlock()
var cache *lockFreeBufferCache
if exists && entryWithTTL != nil {
cache = entryWithTTL.cache
} else {
// Create new cache for this goroutine
cache = &lockFreeBufferCache{}
now := time.Now().Unix()
goroutineCacheMutex.Lock()
goroutineCacheWithTTL[gid] = &cacheEntry{
cache: cache,
lastAccess: now,
gid: gid,
}
goroutineCacheMutex.Unlock()
}
if cache != nil {
// Fill cache to optimal level based on hit rate
hitRate := float64(hitCount) / float64(totalRequests)
optimalCacheSize := int(float64(cacheSize) * hitRate)
if optimalCacheSize < 2 {
optimalCacheSize = 2
}
// Pre-allocate buffers for cache
for i := 0; i < optimalCacheSize && i < len(cache.buffers); i++ {
if cache.buffers[i] == nil {
// Get buffer from main pool
buf := p.Get()
if len(buf) > 0 {
cache.buffers[i] = &buf
}
}
}
}
} }
// OptimizeCache performs periodic cache optimization based on usage patterns // GetAudioBufferPoolStats returns statistics for all pools
func (p *AudioBufferPool) OptimizeCache() { func GetAudioBufferPoolStats() map[string]AudioBufferPoolStats {
hitCount := atomic.LoadInt64(&p.hitCount) return map[string]AudioBufferPoolStats{
missCount := atomic.LoadInt64(&p.missCount) "frame_pool": audioFramePool.GetStats(),
totalRequests := hitCount + missCount "control_pool": audioControlPool.GetStats(),
if totalRequests < 100 {
return
}
hitRate := float64(hitCount) / float64(totalRequests)
// If hit rate is below target, trigger cache warmup
if hitRate < cacheHitRateTarget {
p.WarmupCache()
}
// Reset counters periodically to avoid overflow and get fresh metrics
if totalRequests > 10000 {
atomic.StoreInt64(&p.hitCount, hitCount/2)
atomic.StoreInt64(&p.missCount, missCount/2)
} }
} }

View File

@ -21,12 +21,12 @@ func getEnvInt(key string, defaultValue int) int {
// with fallback to default config values // with fallback to default config values
func parseOpusConfig() (bitrate, complexity, vbr, signalType, bandwidth, dtx int) { func parseOpusConfig() (bitrate, complexity, vbr, signalType, bandwidth, dtx int) {
// Read configuration from environment variables with config defaults // Read configuration from environment variables with config defaults
bitrate = getEnvInt("JETKVM_OPUS_BITRATE", GetConfig().CGOOpusBitrate) bitrate = getEnvInt("JETKVM_OPUS_BITRATE", Config.CGOOpusBitrate)
complexity = getEnvInt("JETKVM_OPUS_COMPLEXITY", GetConfig().CGOOpusComplexity) complexity = getEnvInt("JETKVM_OPUS_COMPLEXITY", Config.CGOOpusComplexity)
vbr = getEnvInt("JETKVM_OPUS_VBR", GetConfig().CGOOpusVBR) vbr = getEnvInt("JETKVM_OPUS_VBR", Config.CGOOpusVBR)
signalType = getEnvInt("JETKVM_OPUS_SIGNAL_TYPE", GetConfig().CGOOpusSignalType) signalType = getEnvInt("JETKVM_OPUS_SIGNAL_TYPE", Config.CGOOpusSignalType)
bandwidth = getEnvInt("JETKVM_OPUS_BANDWIDTH", GetConfig().CGOOpusBandwidth) bandwidth = getEnvInt("JETKVM_OPUS_BANDWIDTH", Config.CGOOpusBandwidth)
dtx = getEnvInt("JETKVM_OPUS_DTX", GetConfig().CGOOpusDTX) dtx = getEnvInt("JETKVM_OPUS_DTX", Config.CGOOpusDTX)
return bitrate, complexity, vbr, signalType, bandwidth, dtx return bitrate, complexity, vbr, signalType, bandwidth, dtx
} }
@ -34,7 +34,7 @@ func parseOpusConfig() (bitrate, complexity, vbr, signalType, bandwidth, dtx int
// applyOpusConfig applies OPUS configuration to the global config // applyOpusConfig applies OPUS configuration to the global config
// with optional logging for the specified component // with optional logging for the specified component
func applyOpusConfig(bitrate, complexity, vbr, signalType, bandwidth, dtx int, component string, enableLogging bool) { func applyOpusConfig(bitrate, complexity, vbr, signalType, bandwidth, dtx int, component string, enableLogging bool) {
config := GetConfig() config := Config
config.CGOOpusBitrate = bitrate config.CGOOpusBitrate = bitrate
config.CGOOpusComplexity = complexity config.CGOOpusComplexity = complexity
config.CGOOpusVBR = vbr config.CGOOpusVBR = vbr

View File

@ -5,6 +5,7 @@ import (
"fmt" "fmt"
"reflect" "reflect"
"sync" "sync"
"sync/atomic"
"time" "time"
"github.com/jetkvm/kvm/internal/logging" "github.com/jetkvm/kvm/internal/logging"
@ -118,9 +119,7 @@ func (r *AudioRelay) IsMuted() bool {
// GetStats returns relay statistics // GetStats returns relay statistics
func (r *AudioRelay) GetStats() (framesRelayed, framesDropped int64) { func (r *AudioRelay) GetStats() (framesRelayed, framesDropped int64) {
r.mutex.RLock() return atomic.LoadInt64(&r.framesRelayed), atomic.LoadInt64(&r.framesDropped)
defer r.mutex.RUnlock()
return r.framesRelayed, r.framesDropped
} }
// UpdateTrack updates the WebRTC audio track for the relay // UpdateTrack updates the WebRTC audio track for the relay
@ -132,34 +131,43 @@ func (r *AudioRelay) UpdateTrack(audioTrack AudioTrackWriter) {
func (r *AudioRelay) relayLoop() { func (r *AudioRelay) relayLoop() {
defer r.wg.Done() defer r.wg.Done()
r.logger.Debug().Msg("Audio relay loop started")
var maxConsecutiveErrors = GetConfig().MaxConsecutiveErrors var maxConsecutiveErrors = Config.MaxConsecutiveErrors
consecutiveErrors := 0 consecutiveErrors := 0
backoffDelay := time.Millisecond * 10
maxBackoff := time.Second * 5
for { for {
select { select {
case <-r.ctx.Done(): case <-r.ctx.Done():
r.logger.Debug().Msg("audio relay loop stopping")
return return
default: default:
frame, err := r.client.ReceiveFrame() frame, err := r.client.ReceiveFrame()
if err != nil { if err != nil {
consecutiveErrors++ consecutiveErrors++
r.logger.Error().Err(err).Int("consecutive_errors", consecutiveErrors).Msg("error reading frame from audio output server")
r.incrementDropped() r.incrementDropped()
// Exponential backoff for stability
if consecutiveErrors >= maxConsecutiveErrors { if consecutiveErrors >= maxConsecutiveErrors {
r.logger.Error().Int("consecutive_errors", consecutiveErrors).Int("max_errors", maxConsecutiveErrors).Msg("too many consecutive read errors, stopping audio relay") // Attempt reconnection
if r.attemptReconnection() {
consecutiveErrors = 0
backoffDelay = time.Millisecond * 10
continue
}
return return
} }
time.Sleep(GetConfig().ShortSleepDuration)
time.Sleep(backoffDelay)
if backoffDelay < maxBackoff {
backoffDelay *= 2
}
continue continue
} }
consecutiveErrors = 0 consecutiveErrors = 0
backoffDelay = time.Millisecond * 10
if err := r.forwardToWebRTC(frame); err != nil { if err := r.forwardToWebRTC(frame); err != nil {
r.logger.Warn().Err(err).Msg("failed to forward frame to webrtc")
r.incrementDropped() r.incrementDropped()
} else { } else {
r.incrementRelayed() r.incrementRelayed()
@ -218,14 +226,24 @@ func (r *AudioRelay) forwardToWebRTC(frame []byte) error {
// incrementRelayed atomically increments the relayed frames counter // incrementRelayed atomically increments the relayed frames counter
func (r *AudioRelay) incrementRelayed() { func (r *AudioRelay) incrementRelayed() {
r.mutex.Lock() atomic.AddInt64(&r.framesRelayed, 1)
r.framesRelayed++
r.mutex.Unlock()
} }
// incrementDropped atomically increments the dropped frames counter // incrementDropped atomically increments the dropped frames counter
func (r *AudioRelay) incrementDropped() { func (r *AudioRelay) incrementDropped() {
r.mutex.Lock() atomic.AddInt64(&r.framesDropped, 1)
r.framesDropped++ }
r.mutex.Unlock()
// attemptReconnection tries to reconnect the audio client for stability
func (r *AudioRelay) attemptReconnection() bool {
if r.client == nil {
return false
}
// Disconnect and reconnect
r.client.Disconnect()
time.Sleep(time.Millisecond * 100)
err := r.client.Connect()
return err == nil
} }

View File

@ -224,7 +224,7 @@ func (aeb *AudioEventBroadcaster) sendToSubscriber(subscriber *AudioEventSubscri
return false return false
} }
ctx, cancel := context.WithTimeout(subscriber.ctx, time.Duration(GetConfig().EventTimeoutSeconds)*time.Second) ctx, cancel := context.WithTimeout(subscriber.ctx, time.Duration(Config.EventTimeoutSeconds)*time.Second)
defer cancel() defer cancel()
err := wsjson.Write(ctx, subscriber.conn, event) err := wsjson.Write(ctx, subscriber.conn, event)

View File

@ -98,16 +98,16 @@ type ZeroCopyFramePool struct {
// NewZeroCopyFramePool creates a new zero-copy frame pool // NewZeroCopyFramePool creates a new zero-copy frame pool
func NewZeroCopyFramePool(maxFrameSize int) *ZeroCopyFramePool { func NewZeroCopyFramePool(maxFrameSize int) *ZeroCopyFramePool {
// Pre-allocate frames for immediate availability // Pre-allocate frames for immediate availability
preallocSizeBytes := GetConfig().PreallocSize preallocSizeBytes := Config.ZeroCopyPreallocSizeBytes
maxPoolSize := GetConfig().MaxPoolSize // Limit total pool size maxPoolSize := Config.MaxPoolSize // Limit total pool size
// Calculate number of frames based on memory budget, not frame count // Calculate number of frames based on memory budget, not frame count
preallocFrameCount := preallocSizeBytes / maxFrameSize preallocFrameCount := preallocSizeBytes / maxFrameSize
if preallocFrameCount > maxPoolSize { if preallocFrameCount > maxPoolSize {
preallocFrameCount = maxPoolSize preallocFrameCount = maxPoolSize
} }
if preallocFrameCount < 1 { if preallocFrameCount < Config.ZeroCopyMinPreallocFrames {
preallocFrameCount = 1 // Always preallocate at least one frame preallocFrameCount = Config.ZeroCopyMinPreallocFrames
} }
preallocated := make([]*ZeroCopyAudioFrame, 0, preallocFrameCount) preallocated := make([]*ZeroCopyAudioFrame, 0, preallocFrameCount)
@ -147,7 +147,7 @@ func (p *ZeroCopyFramePool) Get() *ZeroCopyAudioFrame {
// If we've allocated too many frames, force pool reuse // If we've allocated too many frames, force pool reuse
frame := p.pool.Get().(*ZeroCopyAudioFrame) frame := p.pool.Get().(*ZeroCopyAudioFrame)
frame.mutex.Lock() frame.mutex.Lock()
frame.refCount = 1 atomic.StoreInt32(&frame.refCount, 1)
frame.length = 0 frame.length = 0
frame.data = frame.data[:0] frame.data = frame.data[:0]
frame.mutex.Unlock() frame.mutex.Unlock()
@ -163,11 +163,12 @@ func (p *ZeroCopyFramePool) Get() *ZeroCopyAudioFrame {
p.mutex.Unlock() p.mutex.Unlock()
frame.mutex.Lock() frame.mutex.Lock()
frame.refCount = 1 atomic.StoreInt32(&frame.refCount, 1)
frame.length = 0 frame.length = 0
frame.data = frame.data[:0] frame.data = frame.data[:0]
frame.mutex.Unlock() frame.mutex.Unlock()
atomic.AddInt64(&p.hitCount, 1)
return frame return frame
} }
p.mutex.Unlock() p.mutex.Unlock()
@ -175,7 +176,7 @@ func (p *ZeroCopyFramePool) Get() *ZeroCopyAudioFrame {
// Try sync.Pool next and track allocation // Try sync.Pool next and track allocation
frame := p.pool.Get().(*ZeroCopyAudioFrame) frame := p.pool.Get().(*ZeroCopyAudioFrame)
frame.mutex.Lock() frame.mutex.Lock()
frame.refCount = 1 atomic.StoreInt32(&frame.refCount, 1)
frame.length = 0 frame.length = 0
frame.data = frame.data[:0] frame.data = frame.data[:0]
frame.mutex.Unlock() frame.mutex.Unlock()
@ -191,10 +192,9 @@ func (p *ZeroCopyFramePool) Put(frame *ZeroCopyAudioFrame) {
return return
} }
// Reset frame state for reuse
frame.mutex.Lock() frame.mutex.Lock()
frame.refCount-- atomic.StoreInt32(&frame.refCount, 0)
if frame.refCount <= 0 {
frame.refCount = 0
frame.length = 0 frame.length = 0
frame.data = frame.data[:0] frame.data = frame.data[:0]
frame.mutex.Unlock() frame.mutex.Unlock()
@ -219,15 +219,7 @@ func (p *ZeroCopyFramePool) Put(frame *ZeroCopyAudioFrame) {
// Return to sync.Pool // Return to sync.Pool
p.pool.Put(frame) p.pool.Put(frame)
// Metrics collection removed
if false {
atomic.AddInt64(&p.counter, 1) atomic.AddInt64(&p.counter, 1)
}
} else {
frame.mutex.Unlock()
}
// Metrics recording removed - granular metrics collector was unused
} }
// Data returns the frame data as a slice (zero-copy view) // Data returns the frame data as a slice (zero-copy view)
@ -271,18 +263,28 @@ func (f *ZeroCopyAudioFrame) SetDataDirect(data []byte) {
f.pooled = false // Direct assignment means we can't pool this frame f.pooled = false // Direct assignment means we can't pool this frame
} }
// AddRef increments the reference count for shared access // AddRef increments the reference count atomically
func (f *ZeroCopyAudioFrame) AddRef() { func (f *ZeroCopyAudioFrame) AddRef() {
f.mutex.Lock() atomic.AddInt32(&f.refCount, 1)
f.refCount++
f.mutex.Unlock()
} }
// Release decrements the reference count // Release decrements the reference count atomically
func (f *ZeroCopyAudioFrame) Release() { // Returns true if this was the final reference
f.mutex.Lock() func (f *ZeroCopyAudioFrame) Release() bool {
f.refCount-- newCount := atomic.AddInt32(&f.refCount, -1)
f.mutex.Unlock() if newCount == 0 {
// Final reference released, return to pool if pooled
if f.pooled {
globalZeroCopyPool.Put(f)
}
return true
}
return false
}
// RefCount returns the current reference count atomically
func (f *ZeroCopyAudioFrame) RefCount() int32 {
return atomic.LoadInt32(&f.refCount)
} }
// Length returns the current data length // Length returns the current data length
@ -325,7 +327,7 @@ func (p *ZeroCopyFramePool) GetZeroCopyPoolStats() ZeroCopyFramePoolStats {
var hitRate float64 var hitRate float64
if totalRequests > 0 { if totalRequests > 0 {
hitRate = float64(hitCount) / float64(totalRequests) * GetConfig().PercentageMultiplier hitRate = float64(hitCount) / float64(totalRequests) * Config.PercentageMultiplier
} }
return ZeroCopyFramePoolStats{ return ZeroCopyFramePoolStats{

26
main.go
View File

@ -35,12 +35,6 @@ func startAudioSubprocess() error {
// Initialize validation cache for optimal performance // Initialize validation cache for optimal performance
audio.InitValidationCache() audio.InitValidationCache()
// Start adaptive buffer management for optimal performance
audio.StartAdaptiveBuffering()
// Start goroutine monitoring to detect and prevent leaks
audio.StartGoroutineMonitoring()
// Enable batch audio processing to reduce CGO call overhead // Enable batch audio processing to reduce CGO call overhead
if err := audio.EnableBatchAudioProcessing(); err != nil { if err := audio.EnableBatchAudioProcessing(); err != nil {
logger.Warn().Err(err).Msg("failed to enable batch audio processing") logger.Warn().Err(err).Msg("failed to enable batch audio processing")
@ -58,7 +52,7 @@ func startAudioSubprocess() error {
audio.SetAudioInputSupervisor(audioInputSupervisor) audio.SetAudioInputSupervisor(audioInputSupervisor)
// Set default OPUS configuration for audio input supervisor (low quality for single-core RV1106) // Set default OPUS configuration for audio input supervisor (low quality for single-core RV1106)
config := audio.GetConfig() config := audio.Config
audioInputSupervisor.SetOpusConfig( audioInputSupervisor.SetOpusConfig(
config.AudioQualityLowInputBitrate*1000, // Convert kbps to bps config.AudioQualityLowInputBitrate*1000, // Convert kbps to bps
config.AudioQualityLowOpusComplexity, config.AudioQualityLowOpusComplexity,
@ -77,6 +71,13 @@ func startAudioSubprocess() error {
func(pid int) { func(pid int) {
logger.Info().Int("pid", pid).Msg("audio server process started") logger.Info().Int("pid", pid).Msg("audio server process started")
// Wait for audio output server to be fully ready before starting relay
// This prevents "no client connected" errors during quality changes
go func() {
// Give the audio output server time to initialize and start listening
// Increased delay to reduce frame drops during connection establishment
time.Sleep(1 * time.Second)
// Start audio relay system for main process // Start audio relay system for main process
// If there's an active WebRTC session, use its audio track // If there's an active WebRTC session, use its audio track
var audioTrack *webrtc.TrackLocalStaticSample var audioTrack *webrtc.TrackLocalStaticSample
@ -89,7 +90,13 @@ func startAudioSubprocess() error {
if err := audio.StartAudioRelay(audioTrack); err != nil { if err := audio.StartAudioRelay(audioTrack); err != nil {
logger.Error().Err(err).Msg("failed to start audio relay") logger.Error().Err(err).Msg("failed to start audio relay")
// Retry once after additional delay if initial attempt fails
time.Sleep(1 * time.Second)
if err := audio.StartAudioRelay(audioTrack); err != nil {
logger.Error().Err(err).Msg("failed to start audio relay after retry")
} }
}
}()
}, },
// onProcessExit // onProcessExit
func(pid int, exitCode int, crashed bool) { func(pid int, exitCode int, crashed bool) {
@ -101,10 +108,7 @@ func startAudioSubprocess() error {
// Stop audio relay when process exits // Stop audio relay when process exits
audio.StopAudioRelay() audio.StopAudioRelay()
// Stop adaptive buffering
audio.StopAdaptiveBuffering()
// Stop goroutine monitoring
audio.StopGoroutineMonitoring()
// Disable batch audio processing // Disable batch audio processing
audio.DisableBatchAudioProcessing() audio.DisableBatchAudioProcessing()
}, },

View File

@ -1,25 +1,25 @@
import Card from "@components/Card"; import Card from "@components/Card";
export interface CustomTooltipProps { export interface CustomTooltipProps {
payload: { payload: { date: number; stat: number }; unit: string }[]; payload: { payload: { date: number; metric: number }; unit: string }[];
} }
export default function CustomTooltip({ payload }: CustomTooltipProps) { export default function CustomTooltip({ payload }: CustomTooltipProps) {
if (payload?.length) { if (payload?.length) {
const toolTipData = payload[0]; const toolTipData = payload[0];
const { date, stat } = toolTipData.payload; const { date, metric } = toolTipData.payload;
return ( return (
<Card> <Card>
<div className="p-2 text-black dark:text-white"> <div className="px-2 py-1.5 text-black dark:text-white">
<div className="font-semibold"> <div className="text-[13px] font-semibold">
{new Date(date * 1000).toLocaleTimeString()} {new Date(date * 1000).toLocaleTimeString()}
</div> </div>
<div className="space-y-1"> <div className="space-y-1">
<div className="flex items-center gap-x-1"> <div className="flex items-center gap-x-1">
<div className="h-[2px] w-2 bg-blue-700" /> <div className="h-[2px] w-2 bg-blue-700" />
<span > <span className="text-[13px]">
{stat} {toolTipData?.unit} {metric} {toolTipData?.unit}
</span> </span>
</div> </div>
</div> </div>

View File

@ -103,7 +103,7 @@ export default function DashboardNavbar({
<hr className="h-[20px] w-px self-center border-none bg-slate-800/20 dark:bg-slate-300/20" /> <hr className="h-[20px] w-px self-center border-none bg-slate-800/20 dark:bg-slate-300/20" />
<div className="relative inline-block text-left"> <div className="relative inline-block text-left">
<Menu> <Menu>
<MenuButton className="h-full"> <MenuButton as="div" className="h-full">
<Button className="flex h-full items-center gap-x-3 rounded-md border border-slate-800/20 bg-white px-2 py-1.5 dark:border-slate-600 dark:bg-slate-800 dark:text-white"> <Button className="flex h-full items-center gap-x-3 rounded-md border border-slate-800/20 bg-white px-2 py-1.5 dark:border-slate-600 dark:bg-slate-800 dark:text-white">
{picture ? ( {picture ? (
<img <img

View File

@ -100,15 +100,12 @@ export default function KvmCard({
)} )}
</div> </div>
<Menu as="div" className="relative inline-block text-left"> <Menu as="div" className="relative inline-block text-left">
<div>
<MenuButton <MenuButton
as={Button} as={Button}
theme="light" theme="light"
TrailingIcon={LuEllipsisVertical} TrailingIcon={LuEllipsisVertical}
size="MD" size="MD"
></MenuButton> ></MenuButton>
</div>
<MenuItems <MenuItems
transition transition
className="data-closed:scale-95 data-closed:transform data-closed:opacity-0 data-enter:duration-100 data-leave:duration-75 data-enter:ease-out data-leave:ease-in" className="data-closed:scale-95 data-closed:transform data-closed:opacity-0 data-enter:duration-100 data-leave:duration-75 data-enter:ease-out data-leave:ease-in"

View File

@ -0,0 +1,180 @@
/* eslint-disable react-refresh/only-export-components */
import { ComponentProps } from "react";
import { cva, cx } from "cva";
import { someIterable } from "../utils";
import { GridCard } from "./Card";
import MetricsChart from "./MetricsChart";
interface ChartPoint {
date: number;
metric: number | null;
}
interface MetricProps<T, K extends keyof T> {
title: string;
description: string;
stream?: Map<number, T>;
metric?: K;
data?: ChartPoint[];
gate?: Map<number, unknown>;
supported?: boolean;
map?: (p: { date: number; metric: number | null }) => ChartPoint;
domain?: [number, number];
unit: string;
heightClassName?: string;
referenceValue?: number;
badge?: ComponentProps<typeof MetricHeader>["badge"];
badgeTheme?: ComponentProps<typeof MetricHeader>["badgeTheme"];
}
/**
* Creates a chart array from a metrics map and a metric name.
*
* @param metrics - Expected to be ordered from oldest to newest.
* @param metricName - Name of the metric to create a chart array for.
*/
export function createChartArray<T, K extends keyof T>(
metrics: Map<number, T>,
metricName: K,
) {
const result: { date: number; metric: number | null }[] = [];
const iter = metrics.entries();
let next = iter.next() as IteratorResult<[number, T]>;
const nowSeconds = Math.floor(Date.now() / 1000);
// We want 120 data points, in the chart.
const firstDate = Math.min(next.value?.[0] ?? nowSeconds, nowSeconds - 120);
for (let t = firstDate; t < nowSeconds; t++) {
while (!next.done && next.value[0] < t) next = iter.next();
const has = !next.done && next.value[0] === t;
let metric = null;
if (has) metric = next.value[1][metricName] as number;
result.push({ date: t, metric });
if (has) next = iter.next();
}
return result;
}
function computeReferenceValue(points: ChartPoint[]): number | undefined {
const values = points
.filter(p => p.metric != null && Number.isFinite(p.metric))
.map(p => Number(p.metric));
if (values.length === 0) return undefined;
const sum = values.reduce((acc, v) => acc + v, 0);
const mean = sum / values.length;
return Math.round(mean);
}
const theme = {
light:
"bg-white text-black border border-slate-800/20 dark:border dark:border-slate-700 dark:bg-slate-800 dark:text-slate-300",
danger: "bg-red-500 dark:border-red-700 dark:bg-red-800 dark:text-red-50",
primary: "bg-blue-500 dark:border-blue-700 dark:bg-blue-800 dark:text-blue-50",
};
interface SettingsItemProps {
readonly title: string;
readonly description: string | React.ReactNode;
readonly badge?: string;
readonly className?: string;
readonly children?: React.ReactNode;
readonly badgeTheme?: keyof typeof theme;
}
export function MetricHeader(props: SettingsItemProps) {
const { title, description, badge } = props;
const badgeVariants = cva({ variants: { theme: theme } });
return (
<div className="space-y-0.5">
<div className="flex items-center gap-x-2">
<div className="flex w-full items-center justify-between text-base font-semibold text-black dark:text-white">
{title}
{badge && (
<span
className={cx(
"ml-2 rounded-sm px-2 py-1 font-mono text-[10px] leading-none font-medium",
badgeVariants({ theme: props.badgeTheme ?? "light" }),
)}
>
{badge}
</span>
)}
</div>
</div>
<div className="text-sm text-slate-700 dark:text-slate-300">{description}</div>
</div>
);
}
export function Metric<T, K extends keyof T>({
title,
description,
stream,
metric,
data,
gate,
supported,
map,
domain = [0, 600],
unit = "",
heightClassName = "h-[127px]",
badge,
badgeTheme,
}: MetricProps<T, K>) {
const ready = gate ? gate.size > 0 : stream ? stream.size > 0 : true;
const supportedFinal =
supported ??
(stream && metric ? someIterable(stream, ([, s]) => s[metric] !== undefined) : true);
// Either we let the consumer provide their own chartArray, or we create one from the stream and metric.
const raw = data ?? ((stream && metric && createChartArray(stream, metric)) || []);
// If the consumer provides a map function, we apply it to the raw data.
const dataFinal: ChartPoint[] = map ? raw.map(map) : raw;
// Compute the average value of the metric.
const referenceValue = computeReferenceValue(dataFinal);
return (
<div className="space-y-2">
<MetricHeader
title={title}
description={description}
badge={badge}
badgeTheme={badgeTheme}
/>
<GridCard>
<div
className={`flex ${heightClassName} w-full items-center justify-center text-sm text-slate-500`}
>
{!ready ? (
<div className="flex flex-col items-center space-y-1">
<p className="text-slate-700">Waiting for data...</p>
</div>
) : supportedFinal ? (
<MetricsChart
data={dataFinal}
domain={domain}
unit={unit}
referenceValue={referenceValue}
/>
) : (
<div className="flex flex-col items-center space-y-1">
<p className="text-black">Metric not supported</p>
</div>
)}
</div>
</GridCard>
</div>
);
}

View File

@ -12,13 +12,13 @@ import {
import CustomTooltip, { CustomTooltipProps } from "@components/CustomTooltip"; import CustomTooltip, { CustomTooltipProps } from "@components/CustomTooltip";
export default function StatChart({ export default function MetricsChart({
data, data,
domain, domain,
unit, unit,
referenceValue, referenceValue,
}: { }: {
data: { date: number; stat: number | null | undefined }[]; data: { date: number; metric: number | null | undefined }[];
domain?: [string | number, string | number]; domain?: [string | number, string | number];
unit?: string; unit?: string;
referenceValue?: number; referenceValue?: number;
@ -33,7 +33,7 @@ export default function StatChart({
strokeLinecap="butt" strokeLinecap="butt"
stroke="rgba(30, 41, 59, 0.1)" stroke="rgba(30, 41, 59, 0.1)"
/> />
{referenceValue && ( {referenceValue !== undefined && (
<ReferenceLine <ReferenceLine
y={referenceValue} y={referenceValue}
strokeDasharray="3 3" strokeDasharray="3 3"
@ -64,7 +64,7 @@ export default function StatChart({
.map(x => x.date)} .map(x => x.date)}
/> />
<YAxis <YAxis
dataKey="stat" dataKey="metric"
axisLine={false} axisLine={false}
orientation="right" orientation="right"
tick={{ tick={{
@ -73,6 +73,7 @@ export default function StatChart({
fill: "rgba(107, 114, 128, 1)", fill: "rgba(107, 114, 128, 1)",
}} }}
padding={{ top: 0, bottom: 0 }} padding={{ top: 0, bottom: 0 }}
allowDecimals
tickLine={false} tickLine={false}
unit={unit} unit={unit}
domain={domain || ["auto", "auto"]} domain={domain || ["auto", "auto"]}
@ -87,7 +88,7 @@ export default function StatChart({
<Line <Line
type="monotone" type="monotone"
isAnimationActive={false} isAnimationActive={false}
dataKey="stat" dataKey="metric"
stroke="rgb(29 78 216)" stroke="rgb(29 78 216)"
strokeLinecap="round" strokeLinecap="round"
strokeWidth={2} strokeWidth={2}

View File

@ -345,8 +345,13 @@ export default function WebRTCVideo({ microphone }: WebRTCVideoProps) {
peerConnection.addEventListener( peerConnection.addEventListener(
"track", "track",
(e: RTCTrackEvent) => { (_e: RTCTrackEvent) => {
addStreamToVideoElm(e.streams[0]); // The combined MediaStream is now managed in the main component
// We'll use the mediaStream from the store instead of individual track streams
const { mediaStream } = useRTCStore.getState();
if (mediaStream) {
addStreamToVideoElm(mediaStream);
}
}, },
{ signal }, { signal },
); );

View File

@ -1,74 +1,40 @@
import { useInterval } from "usehooks-ts"; import { useInterval } from "usehooks-ts";
import SidebarHeader from "@/components/SidebarHeader"; import SidebarHeader from "@/components/SidebarHeader";
import { GridCard } from "@/components/Card";
import { useRTCStore, useUiStore } from "@/hooks/stores"; import { useRTCStore, useUiStore } from "@/hooks/stores";
import StatChart from "@/components/StatChart"; import { someIterable } from "@/utils";
function createChartArray<T, K extends keyof T>( import { createChartArray, Metric } from "../Metric";
stream: Map<number, T>, import { SettingsSectionHeader } from "../SettingsSectionHeader";
metric: K,
): { date: number; stat: T[K] | null }[] {
const stat = Array.from(stream).map(([key, stats]) => {
return { date: key, stat: stats[metric] };
});
// Sort the dates to ensure they are in chronological order
const sortedStat = stat.map(x => x.date).sort((a, b) => a - b);
// Determine the earliest statistic date
const earliestStat = sortedStat[0];
// Current time in seconds since the Unix epoch
const now = Math.floor(Date.now() / 1000);
// Determine the starting point for the chart data
const firstChartDate = earliestStat ? Math.min(earliestStat, now - 120) : now - 120;
// Generate the chart array for the range between 'firstChartDate' and 'now'
return Array.from({ length: now - firstChartDate }, (_, i) => {
const currentDate = firstChartDate + i;
return {
date: currentDate,
// Find the statistic for 'currentDate', or use the last known statistic if none exists for that date
stat: stat.find(x => x.date === currentDate)?.stat ?? null,
};
});
}
export default function ConnectionStatsSidebar() { export default function ConnectionStatsSidebar() {
const { sidebarView, setSidebarView } = useUiStore(); const { sidebarView, setSidebarView } = useUiStore();
const { const {
mediaStream, mediaStream,
peerConnection, peerConnection,
inboundRtpStats, inboundRtpStats: inboundVideoRtpStats,
appendInboundRtpStats, appendInboundRtpStats: appendInboundVideoRtpStats,
candidatePairStats, candidatePairStats: iceCandidatePairStats,
appendCandidatePairStats, appendCandidatePairStats,
appendLocalCandidateStats, appendLocalCandidateStats,
appendRemoteCandidateStats, appendRemoteCandidateStats,
appendDiskDataChannelStats, appendDiskDataChannelStats,
} = useRTCStore(); } = useRTCStore();
function isMetricSupported<T, K extends keyof T>(
stream: Map<number, T>,
metric: K,
): boolean {
return Array.from(stream).some(([, stat]) => stat[metric] !== undefined);
}
useInterval(function collectWebRTCStats() { useInterval(function collectWebRTCStats() {
(async () => { (async () => {
if (!mediaStream) return; if (!mediaStream) return;
const videoTrack = mediaStream.getVideoTracks()[0]; const videoTrack = mediaStream.getVideoTracks()[0];
if (!videoTrack) return; if (!videoTrack) return;
const stats = await peerConnection?.getStats(); const stats = await peerConnection?.getStats();
let successfulLocalCandidateId: string | null = null; let successfulLocalCandidateId: string | null = null;
let successfulRemoteCandidateId: string | null = null; let successfulRemoteCandidateId: string | null = null;
stats?.forEach(report => { stats?.forEach(report => {
if (report.type === "inbound-rtp") { if (report.type === "inbound-rtp" && report.kind === "video") {
appendInboundRtpStats(report); appendInboundVideoRtpStats(report);
} else if (report.type === "candidate-pair" && report.nominated) { } else if (report.type === "candidate-pair" && report.nominated) {
if (report.state === "succeeded") { if (report.state === "succeeded") {
successfulLocalCandidateId = report.localCandidateId; successfulLocalCandidateId = report.localCandidateId;
@ -91,144 +57,133 @@ export default function ConnectionStatsSidebar() {
})(); })();
}, 500); }, 500);
const jitterBufferDelay = createChartArray(inboundVideoRtpStats, "jitterBufferDelay");
const jitterBufferEmittedCount = createChartArray(
inboundVideoRtpStats,
"jitterBufferEmittedCount",
);
const jitterBufferAvgDelayData = jitterBufferDelay.map((d, idx) => {
if (idx === 0) return { date: d.date, metric: null };
const prevDelay = jitterBufferDelay[idx - 1]?.metric as number | null | undefined;
const currDelay = d.metric as number | null | undefined;
const prevCountEmitted =
(jitterBufferEmittedCount[idx - 1]?.metric as number | null | undefined) ?? null;
const currCountEmitted =
(jitterBufferEmittedCount[idx]?.metric as number | null | undefined) ?? null;
if (
prevDelay == null ||
currDelay == null ||
prevCountEmitted == null ||
currCountEmitted == null
) {
return { date: d.date, metric: null };
}
const deltaDelay = currDelay - prevDelay;
const deltaEmitted = currCountEmitted - prevCountEmitted;
// Guard counter resets or no emitted frames
if (deltaDelay < 0 || deltaEmitted <= 0) {
return { date: d.date, metric: null };
}
const valueMs = Math.round((deltaDelay / deltaEmitted) * 1000);
return { date: d.date, metric: valueMs };
});
return ( return (
<div className="grid h-full grid-rows-(--grid-headerBody) shadow-xs"> <div className="grid h-full grid-rows-(--grid-headerBody) shadow-xs">
<SidebarHeader title="Connection Stats" setSidebarView={setSidebarView} /> <SidebarHeader title="Connection Stats" setSidebarView={setSidebarView} />
<div className="h-full space-y-4 overflow-y-scroll bg-white px-4 py-2 pb-8 dark:bg-slate-900"> <div className="h-full space-y-4 overflow-y-scroll bg-white px-4 py-2 pb-8 dark:bg-slate-900">
<div className="space-y-4"> <div className="space-y-4">
{/*
The entire sidebar component is always rendered, with a display none when not visible
The charts below, need a height and width, otherwise they throw. So simply don't render them unless the thing is visible
*/}
{sidebarView === "connection-stats" && ( {sidebarView === "connection-stats" && (
<div className="space-y-4"> <div className="space-y-8">
<div className="space-y-2"> {/* Connection Group */}
<div> <div className="space-y-3">
<h2 className="text-lg font-semibold text-black dark:text-white"> <SettingsSectionHeader
Packets Lost title="Connection"
</h2> description="The connection between the client and the JetKVM."
<p className="text-sm text-slate-700 dark:text-slate-300">
Number of data packets lost during transmission.
</p>
</div>
<GridCard>
<div className="flex h-[127px] w-full items-center justify-center text-sm text-slate-500">
{inboundRtpStats.size === 0 ? (
<div className="flex flex-col items-center space-y-1">
<p className="text-slate-700">Waiting for data...</p>
</div>
) : isMetricSupported(inboundRtpStats, "packetsLost") ? (
<StatChart
data={createChartArray(inboundRtpStats, "packetsLost")}
domain={[0, 100]}
unit=" packets"
/> />
) : ( <Metric
<div className="flex flex-col items-center space-y-1"> title="Round-Trip Time"
<p className="text-black">Metric not supported</p> description="Round-trip time for the active ICE candidate pair between peers."
</div> stream={iceCandidatePairStats}
)} metric="currentRoundTripTime"
</div> map={x => ({
</GridCard>
</div>
<div className="space-y-2">
<div>
<h2 className="text-lg font-semibold text-black dark:text-white">
Round-Trip Time
</h2>
<p className="text-sm text-slate-700 dark:text-slate-300">
Time taken for data to travel from source to destination and back
</p>
</div>
<GridCard>
<div className="flex h-[127px] w-full items-center justify-center text-sm text-slate-500">
{inboundRtpStats.size === 0 ? (
<div className="flex flex-col items-center space-y-1">
<p className="text-slate-700">Waiting for data...</p>
</div>
) : isMetricSupported(candidatePairStats, "currentRoundTripTime") ? (
<StatChart
data={createChartArray(
candidatePairStats,
"currentRoundTripTime",
).map(x => {
return {
date: x.date, date: x.date,
stat: x.stat ? Math.round(x.stat * 1000) : null, metric: x.metric != null ? Math.round(x.metric * 1000) : null,
};
})} })}
domain={[0, 600]} domain={[0, 600]}
unit=" ms" unit=" ms"
/> />
) : (
<div className="flex flex-col items-center space-y-1">
<p className="text-black">Metric not supported</p>
</div> </div>
)}
</div> {/* Video Group */}
</GridCard> <div className="space-y-3">
</div> <SettingsSectionHeader
<div className="space-y-2"> title="Video"
<div> description="The video stream from the JetKVM to the client."
<h2 className="text-lg font-semibold text-black dark:text-white"> />
Jitter
</h2> {/* RTP Jitter */}
<p className="text-sm text-slate-700 dark:text-slate-300"> <Metric
Variation in packet delay, affecting video smoothness.{" "} title="Network Stability"
</p> badge="Jitter"
</div> badgeTheme="light"
<GridCard> description="How steady the flow of inbound video packets is across the network."
<div className="flex h-[127px] w-full items-center justify-center text-sm text-slate-500"> stream={inboundVideoRtpStats}
{inboundRtpStats.size === 0 ? ( metric="jitter"
<div className="flex flex-col items-center space-y-1"> map={x => ({
<p className="text-slate-700">Waiting for data...</p>
</div>
) : (
<StatChart
data={createChartArray(inboundRtpStats, "jitter").map(x => {
return {
date: x.date, date: x.date,
stat: x.stat ? Math.round(x.stat * 1000) : null, metric: x.metric != null ? Math.round(x.metric * 1000) : null,
};
})} })}
domain={[0, 300]} domain={[0, 10]}
unit=" ms" unit=" ms"
/> />
)}
</div> {/* Playback Delay */}
</GridCard> <Metric
</div> title="Playback Delay"
<div className="space-y-2"> description="Delay added by the jitter buffer to smooth playback when frames arrive unevenly."
<div> badge="Jitter Buffer Avg. Delay"
<h2 className="text-lg font-semibold text-black dark:text-white"> badgeTheme="light"
Frames per second data={jitterBufferAvgDelayData}
</h2> gate={inboundVideoRtpStats}
<p className="text-sm text-slate-700 dark:text-slate-300"> supported={
Number of video frames displayed per second. someIterable(
</p> inboundVideoRtpStats,
</div> ([, x]) => x.jitterBufferDelay != null,
<GridCard> ) &&
<div className="flex h-[127px] w-full items-center justify-center text-sm text-slate-500"> someIterable(
{inboundRtpStats.size === 0 ? ( inboundVideoRtpStats,
<div className="flex flex-col items-center space-y-1"> ([, x]) => x.jitterBufferEmittedCount != null,
<p className="text-slate-700">Waiting for data...</p> )
</div> }
) : ( domain={[0, 30]}
<StatChart unit=" ms"
data={createChartArray(inboundRtpStats, "framesPerSecond").map( />
x => {
return { {/* Packets Lost */}
date: x.date, <Metric
stat: x.stat ? x.stat : null, title="Packets Lost"
}; description="Count of lost inbound video RTP packets."
}, stream={inboundVideoRtpStats}
)} metric="packetsLost"
domain={[0, 100]}
unit=" packets"
/>
{/* Frames Per Second */}
<Metric
title="Frames per second"
description="Number of inbound video frames displayed per second."
stream={inboundVideoRtpStats}
metric="framesPerSecond"
domain={[0, 80]} domain={[0, 80]}
unit=" fps" unit=" fps"
/> />
)}
</div>
</GridCard>
</div> </div>
</div> </div>
)} )}

View File

@ -355,6 +355,10 @@ export interface SettingsState {
setVideoBrightness: (value: number) => void; setVideoBrightness: (value: number) => void;
videoContrast: number; videoContrast: number;
setVideoContrast: (value: number) => void; setVideoContrast: (value: number) => void;
// Microphone persistence settings
microphoneWasEnabled: boolean;
setMicrophoneWasEnabled: (enabled: boolean) => void;
} }
export const useSettingsStore = create( export const useSettingsStore = create(
@ -400,6 +404,10 @@ export const useSettingsStore = create(
setVideoBrightness: (value: number) => set({ videoBrightness: value }), setVideoBrightness: (value: number) => set({ videoBrightness: value }),
videoContrast: 1.0, videoContrast: 1.0,
setVideoContrast: (value: number) => set({ videoContrast: value }), setVideoContrast: (value: number) => set({ videoContrast: value }),
// Microphone persistence settings
microphoneWasEnabled: false,
setMicrophoneWasEnabled: (enabled: boolean) => set({ microphoneWasEnabled: enabled }),
}), }),
{ {
name: "settings", name: "settings",

View File

@ -1,6 +1,6 @@
import { useCallback, useEffect, useRef, useState } from "react"; import { useCallback, useEffect, useRef, useState } from "react";
import { useRTCStore } from "@/hooks/stores"; import { useRTCStore, useSettingsStore } from "@/hooks/stores";
import api from "@/api"; import api from "@/api";
import { devLog, devInfo, devWarn, devError, devOnly } from "@/utils/debug"; import { devLog, devInfo, devWarn, devError, devOnly } from "@/utils/debug";
import { AUDIO_CONFIG } from "@/config/constants"; import { AUDIO_CONFIG } from "@/config/constants";
@ -23,6 +23,8 @@ export function useMicrophone() {
setMicrophoneMuted, setMicrophoneMuted,
} = useRTCStore(); } = useRTCStore();
const { microphoneWasEnabled, setMicrophoneWasEnabled } = useSettingsStore();
const microphoneStreamRef = useRef<MediaStream | null>(null); const microphoneStreamRef = useRef<MediaStream | null>(null);
// Loading states // Loading states
@ -61,7 +63,7 @@ export function useMicrophone() {
// Cleaning up microphone stream // Cleaning up microphone stream
if (microphoneStreamRef.current) { if (microphoneStreamRef.current) {
microphoneStreamRef.current.getTracks().forEach(track => { microphoneStreamRef.current.getTracks().forEach((track: MediaStreamTrack) => {
track.stop(); track.stop();
}); });
microphoneStreamRef.current = null; microphoneStreamRef.current = null;
@ -193,7 +195,7 @@ export function useMicrophone() {
// Find the audio transceiver (should already exist with sendrecv direction) // Find the audio transceiver (should already exist with sendrecv direction)
const transceivers = peerConnection.getTransceivers(); const transceivers = peerConnection.getTransceivers();
devLog("Available transceivers:", transceivers.map(t => ({ devLog("Available transceivers:", transceivers.map((t: RTCRtpTransceiver) => ({
direction: t.direction, direction: t.direction,
mid: t.mid, mid: t.mid,
senderTrack: t.sender.track?.kind, senderTrack: t.sender.track?.kind,
@ -201,7 +203,7 @@ export function useMicrophone() {
}))); })));
// Look for an audio transceiver that can send (has sendrecv or sendonly direction) // Look for an audio transceiver that can send (has sendrecv or sendonly direction)
const audioTransceiver = transceivers.find(transceiver => { const audioTransceiver = transceivers.find((transceiver: RTCRtpTransceiver) => {
// Check if this transceiver is for audio and can send // Check if this transceiver is for audio and can send
const canSend = transceiver.direction === 'sendrecv' || transceiver.direction === 'sendonly'; const canSend = transceiver.direction === 'sendrecv' || transceiver.direction === 'sendonly';
@ -389,6 +391,9 @@ export function useMicrophone() {
setMicrophoneActive(true); setMicrophoneActive(true);
setMicrophoneMuted(false); setMicrophoneMuted(false);
// Save microphone enabled state for auto-restore on page reload
setMicrophoneWasEnabled(true);
devLog("Microphone state set to active. Verifying state:", { devLog("Microphone state set to active. Verifying state:", {
streamInRef: !!microphoneStreamRef.current, streamInRef: !!microphoneStreamRef.current,
streamInStore: !!microphoneStream, streamInStore: !!microphoneStream,
@ -447,7 +452,7 @@ export function useMicrophone() {
setIsStarting(false); setIsStarting(false);
return { success: false, error: micError }; return { success: false, error: micError };
} }
}, [peerConnection, setMicrophoneStream, setMicrophoneSender, setMicrophoneActive, setMicrophoneMuted, stopMicrophoneStream, isMicrophoneActive, isMicrophoneMuted, microphoneStream, isStarting, isStopping, isToggling]); }, [peerConnection, setMicrophoneStream, setMicrophoneSender, setMicrophoneActive, setMicrophoneMuted, setMicrophoneWasEnabled, stopMicrophoneStream, isMicrophoneActive, isMicrophoneMuted, microphoneStream, isStarting, isStopping, isToggling]);
@ -476,6 +481,9 @@ export function useMicrophone() {
setMicrophoneActive(false); setMicrophoneActive(false);
setMicrophoneMuted(false); setMicrophoneMuted(false);
// Save microphone disabled state for persistence
setMicrophoneWasEnabled(false);
// Sync state after stopping to ensure consistency (with longer delay) // Sync state after stopping to ensure consistency (with longer delay)
setTimeout(() => syncMicrophoneState(), 500); setTimeout(() => syncMicrophoneState(), 500);
@ -492,7 +500,7 @@ export function useMicrophone() {
} }
}; };
} }
}, [stopMicrophoneStream, syncMicrophoneState, setMicrophoneActive, setMicrophoneMuted, isStarting, isStopping, isToggling]); }, [stopMicrophoneStream, syncMicrophoneState, setMicrophoneActive, setMicrophoneMuted, setMicrophoneWasEnabled, isStarting, isStopping, isToggling]);
// Toggle microphone mute // Toggle microphone mute
const toggleMicrophoneMute = useCallback(async (): Promise<{ success: boolean; error?: MicrophoneError }> => { const toggleMicrophoneMute = useCallback(async (): Promise<{ success: boolean; error?: MicrophoneError }> => {
@ -560,7 +568,7 @@ export function useMicrophone() {
const newMutedState = !isMicrophoneMuted; const newMutedState = !isMicrophoneMuted;
// Mute/unmute the audio track // Mute/unmute the audio track
audioTracks.forEach(track => { audioTracks.forEach((track: MediaStreamTrack) => {
track.enabled = !newMutedState; track.enabled = !newMutedState;
devLog(`Audio track ${track.id} enabled: ${track.enabled}`); devLog(`Audio track ${track.id} enabled: ${track.enabled}`);
}); });
@ -607,10 +615,30 @@ export function useMicrophone() {
// Sync state on mount // Sync state on mount and auto-restore microphone if it was enabled before page reload
useEffect(() => { useEffect(() => {
syncMicrophoneState(); const autoRestoreMicrophone = async () => {
}, [syncMicrophoneState]); // First sync the current state
await syncMicrophoneState();
// If microphone was enabled before page reload and is not currently active, restore it
if (microphoneWasEnabled && !isMicrophoneActive && peerConnection) {
devLog("Auto-restoring microphone after page reload");
try {
const result = await startMicrophone();
if (result.success) {
devInfo("Microphone auto-restored successfully after page reload");
} else {
devWarn("Failed to auto-restore microphone:", result.error);
}
} catch (error) {
devWarn("Error during microphone auto-restoration:", error);
}
}
};
autoRestoreMicrophone();
}, [syncMicrophoneState, microphoneWasEnabled, isMicrophoneActive, peerConnection, startMicrophone]);
// Cleanup on unmount - use ref to avoid dependency on stopMicrophoneStream // Cleanup on unmount - use ref to avoid dependency on stopMicrophoneStream
useEffect(() => { useEffect(() => {
@ -619,7 +647,7 @@ export function useMicrophone() {
const stream = microphoneStreamRef.current; const stream = microphoneStreamRef.current;
if (stream) { if (stream) {
devLog("Cleanup: stopping microphone stream on unmount"); devLog("Cleanup: stopping microphone stream on unmount");
stream.getAudioTracks().forEach(track => { stream.getAudioTracks().forEach((track: MediaStreamTrack) => {
track.stop(); track.stop();
devLog(`Cleanup: stopped audio track ${track.id}`); devLog(`Cleanup: stopped audio track ${track.id}`);
}); });

View File

@ -116,6 +116,7 @@ if (isOnDevice) {
path: "/", path: "/",
errorElement: <ErrorBoundary />, errorElement: <ErrorBoundary />,
element: <DeviceRoute />, element: <DeviceRoute />,
HydrateFallback: () => <div className="p-4">Loading...</div>,
loader: DeviceRoute.loader, loader: DeviceRoute.loader,
children: [ children: [
{ {

View File

@ -355,7 +355,7 @@ function UrlView({
const popularImages = [ const popularImages = [
{ {
name: "Ubuntu 24.04 LTS", name: "Ubuntu 24.04 LTS",
url: "https://releases.ubuntu.com/24.04.2/ubuntu-24.04.2-desktop-amd64.iso", url: "https://releases.ubuntu.com/24.04.3/ubuntu-24.04.3-desktop-amd64.iso",
icon: UbuntuIcon, icon: UbuntuIcon,
}, },
{ {
@ -369,8 +369,8 @@ function UrlView({
icon: DebianIcon, icon: DebianIcon,
}, },
{ {
name: "Fedora 41", name: "Fedora 42",
url: "https://download.fedoraproject.org/pub/fedora/linux/releases/41/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-41-1.4.iso", url: "https://download.fedoraproject.org/pub/fedora/linux/releases/42/Workstation/x86_64/iso/Fedora-Workstation-Live-42-1.1.x86_64.iso",
icon: FedoraIcon, icon: FedoraIcon,
}, },
{ {
@ -385,7 +385,7 @@ function UrlView({
}, },
{ {
name: "Arch Linux", name: "Arch Linux",
url: "https://archlinux.doridian.net/iso/2025.02.01/archlinux-2025.02.01-x86_64.iso", url: "https://archlinux.doridian.net/iso/latest/archlinux-x86_64.iso",
icon: ArchIcon, icon: ArchIcon,
}, },
{ {

View File

@ -1,15 +1,16 @@
import { useState, useEffect } from "react"; import { useEffect, useState } from "react";
import { Button } from "@/components/Button"; import { Button } from "@/components/Button";
import { TextAreaWithLabel } from "@/components/TextArea"; import { TextAreaWithLabel } from "@/components/TextArea";
import { JsonRpcResponse, useJsonRpc } from "@/hooks/useJsonRpc"; import { JsonRpcResponse, useJsonRpc } from "@/hooks/useJsonRpc";
import { SettingsPageHeader } from "@components/SettingsPageheader"; import { SettingsPageHeader } from "@components/SettingsPageheader";
import { useSettingsStore } from "@/hooks/stores"; import { useSettingsStore } from "@/hooks/stores";
import { SelectMenuBasic } from "@components/SelectMenuBasic";
import notifications from "../notifications"; import Fieldset from "@components/Fieldset";
import { SelectMenuBasic } from "../components/SelectMenuBasic"; import notifications from "@/notifications";
import { SettingsItem } from "./devices.$id.settings"; import { SettingsItem } from "./devices.$id.settings";
const defaultEdid = const defaultEdid =
"00ffffffffffff0052620188008888881c150103800000780a0dc9a05747982712484c00000001010101010101010101010101010101023a801871382d40582c4500c48e2100001e011d007251d01e206e285500c48e2100001e000000fc00543734392d6648443732300a20000000fd00147801ff1d000a202020202020017b"; "00ffffffffffff0052620188008888881c150103800000780a0dc9a05747982712484c00000001010101010101010101010101010101023a801871382d40582c4500c48e2100001e011d007251d01e206e285500c48e2100001e000000fc00543734392d6648443732300a20000000fd00147801ff1d000a202020202020017b";
const edids = [ const edids = [
@ -50,21 +51,27 @@ export default function SettingsVideoRoute() {
const [streamQuality, setStreamQuality] = useState("1"); const [streamQuality, setStreamQuality] = useState("1");
const [customEdidValue, setCustomEdidValue] = useState<string | null>(null); const [customEdidValue, setCustomEdidValue] = useState<string | null>(null);
const [edid, setEdid] = useState<string | null>(null); const [edid, setEdid] = useState<string | null>(null);
const [edidLoading, setEdidLoading] = useState(false);
// Video enhancement settings from store // Video enhancement settings from store
const { const {
videoSaturation, setVideoSaturation, videoSaturation,
videoBrightness, setVideoBrightness, setVideoSaturation,
videoContrast, setVideoContrast videoBrightness,
setVideoBrightness,
videoContrast,
setVideoContrast,
} = useSettingsStore(); } = useSettingsStore();
useEffect(() => { useEffect(() => {
setEdidLoading(true);
send("getStreamQualityFactor", {}, (resp: JsonRpcResponse) => { send("getStreamQualityFactor", {}, (resp: JsonRpcResponse) => {
if ("error" in resp) return; if ("error" in resp) return;
setStreamQuality(String(resp.result)); setStreamQuality(String(resp.result));
}); });
send("getEDID", {}, (resp: JsonRpcResponse) => { send("getEDID", {}, (resp: JsonRpcResponse) => {
setEdidLoading(false);
if ("error" in resp) { if ("error" in resp) {
notifications.error(`Failed to get EDID: ${resp.error.data || "Unknown error"}`); notifications.error(`Failed to get EDID: ${resp.error.data || "Unknown error"}`);
return; return;
@ -89,7 +96,10 @@ export default function SettingsVideoRoute() {
}, [send]); }, [send]);
const handleStreamQualityChange = (factor: string) => { const handleStreamQualityChange = (factor: string) => {
send("setStreamQualityFactor", { factor: Number(factor) }, (resp: JsonRpcResponse) => { send(
"setStreamQualityFactor",
{ factor: Number(factor) },
(resp: JsonRpcResponse) => {
if ("error" in resp) { if ("error" in resp) {
notifications.error( notifications.error(
`Failed to set stream quality: ${resp.error.data || "Unknown error"}`, `Failed to set stream quality: ${resp.error.data || "Unknown error"}`,
@ -97,20 +107,25 @@ export default function SettingsVideoRoute() {
return; return;
} }
notifications.success(`Stream quality set to ${streamQualityOptions.find(x => x.value === factor)?.label}`); notifications.success(
`Stream quality set to ${streamQualityOptions.find(x => x.value === factor)?.label}`,
);
setStreamQuality(factor); setStreamQuality(factor);
}); },
);
}; };
const handleEDIDChange = (newEdid: string) => { const handleEDIDChange = (newEdid: string) => {
setEdidLoading(true);
send("setEDID", { edid: newEdid }, (resp: JsonRpcResponse) => { send("setEDID", { edid: newEdid }, (resp: JsonRpcResponse) => {
setEdidLoading(false);
if ("error" in resp) { if ("error" in resp) {
notifications.error(`Failed to set EDID: ${resp.error.data || "Unknown error"}`); notifications.error(`Failed to set EDID: ${resp.error.data || "Unknown error"}`);
return; return;
} }
notifications.success( notifications.success(
`EDID set successfully to ${edids.find(x => x.value === newEdid)?.label}`, `EDID set successfully to ${edids.find(x => x.value === newEdid)?.label ?? "the custom EDID"}`,
); );
// Update the EDID value in the UI // Update the EDID value in the UI
setEdid(newEdid); setEdid(newEdid);
@ -158,7 +173,7 @@ export default function SettingsVideoRoute() {
step="0.1" step="0.1"
value={videoSaturation} value={videoSaturation}
onChange={e => setVideoSaturation(parseFloat(e.target.value))} onChange={e => setVideoSaturation(parseFloat(e.target.value))}
className="w-32 h-2 bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700" className="h-2 w-32 cursor-pointer appearance-none rounded-lg bg-gray-200 dark:bg-gray-700"
/> />
</SettingsItem> </SettingsItem>
@ -173,7 +188,7 @@ export default function SettingsVideoRoute() {
step="0.1" step="0.1"
value={videoBrightness} value={videoBrightness}
onChange={e => setVideoBrightness(parseFloat(e.target.value))} onChange={e => setVideoBrightness(parseFloat(e.target.value))}
className="w-32 h-2 bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700" className="h-2 w-32 cursor-pointer appearance-none rounded-lg bg-gray-200 dark:bg-gray-700"
/> />
</SettingsItem> </SettingsItem>
@ -188,7 +203,7 @@ export default function SettingsVideoRoute() {
step="0.1" step="0.1"
value={videoContrast} value={videoContrast}
onChange={e => setVideoContrast(parseFloat(e.target.value))} onChange={e => setVideoContrast(parseFloat(e.target.value))}
className="w-32 h-2 bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700" className="h-2 w-32 cursor-pointer appearance-none rounded-lg bg-gray-200 dark:bg-gray-700"
/> />
</SettingsItem> </SettingsItem>
@ -205,10 +220,11 @@ export default function SettingsVideoRoute() {
/> />
</div> </div>
</div> </div>
<Fieldset disabled={edidLoading} className="space-y-2">
<SettingsItem <SettingsItem
title="EDID" title="EDID"
description="Adjust the EDID settings for the display" description="Adjust the EDID settings for the display"
loading={edidLoading}
> >
<SelectMenuBasic <SelectMenuBasic
size="SM" size="SM"
@ -245,12 +261,14 @@ export default function SettingsVideoRoute() {
size="SM" size="SM"
theme="primary" theme="primary"
text="Set Custom EDID" text="Set Custom EDID"
loading={edidLoading}
onClick={() => handleEDIDChange(customEdidValue)} onClick={() => handleEDIDChange(customEdidValue)}
/> />
<Button <Button
size="SM" size="SM"
theme="light" theme="light"
text="Restore to default" text="Restore to default"
loading={edidLoading}
onClick={() => { onClick={() => {
setCustomEdidValue(null); setCustomEdidValue(null);
handleEDIDChange(defaultEdid); handleEDIDChange(defaultEdid);
@ -259,6 +277,7 @@ export default function SettingsVideoRoute() {
</div> </div>
</> </>
)} )}
</Fieldset>
</div> </div>
</div> </div>
</div> </div>

View File

@ -54,6 +54,7 @@ import { useDeviceUiNavigation } from "@/hooks/useAppNavigation";
import { FeatureFlagProvider } from "@/providers/FeatureFlagProvider"; import { FeatureFlagProvider } from "@/providers/FeatureFlagProvider";
import { DeviceStatus } from "@routes/welcome-local"; import { DeviceStatus } from "@routes/welcome-local";
import { SystemVersionInfo } from "@routes/devices.$id.settings.general.update"; import { SystemVersionInfo } from "@routes/devices.$id.settings.general.update";
import audioQualityService from "@/services/audioQualityService";
interface LocalLoaderResp { interface LocalLoaderResp {
authMode: "password" | "noPassword" | null; authMode: "password" | "noPassword" | null;
@ -474,8 +475,27 @@ export default function KvmIdRoute() {
} }
}; };
pc.ontrack = function (event) { pc.ontrack = function (event: RTCTrackEvent) {
setMediaStream(event.streams[0]); // Handle separate MediaStreams for audio and video tracks
const track = event.track;
const streams = event.streams;
if (streams && streams.length > 0) {
// Get existing MediaStream or create a new one
const existingStream = useRTCStore.getState().mediaStream;
let combinedStream: MediaStream;
if (existingStream) {
combinedStream = existingStream;
// Add the new track to the existing stream
combinedStream.addTrack(track);
} else {
// Create a new MediaStream with the track
combinedStream = new MediaStream([track]);
}
setMediaStream(combinedStream);
}
}; };
setTransceiver(pc.addTransceiver("video", { direction: "recvonly" })); setTransceiver(pc.addTransceiver("video", { direction: "recvonly" }));
@ -533,6 +553,11 @@ export default function KvmIdRoute() {
}; };
}, [clearCandidatePairStats, clearInboundRtpStats, setPeerConnection, setSidebarView]); }, [clearCandidatePairStats, clearInboundRtpStats, setPeerConnection, setSidebarView]);
// Register callback with audioQualityService
useEffect(() => {
audioQualityService.setReconnectionCallback(setupPeerConnection);
}, [setupPeerConnection]);
// TURN server usage detection // TURN server usage detection
useEffect(() => { useEffect(() => {
if (peerConnectionState !== "connected") return; if (peerConnectionState !== "connected") return;

View File

@ -24,6 +24,7 @@ class AudioQualityService {
2: 'High', 2: 'High',
3: 'Ultra' 3: 'Ultra'
}; };
private reconnectionCallback: (() => Promise<void>) | null = null;
/** /**
* Fetch audio quality presets from the backend * Fetch audio quality presets from the backend
@ -96,12 +97,34 @@ class AudioQualityService {
} }
/** /**
* Set audio quality * Set reconnection callback for WebRTC reset
*/
setReconnectionCallback(callback: () => Promise<void>): void {
this.reconnectionCallback = callback;
}
/**
* Trigger audio track replacement using backend's track replacement mechanism
*/
private async replaceAudioTrack(): Promise<void> {
if (this.reconnectionCallback) {
await this.reconnectionCallback();
}
}
/**
* Set audio quality with track replacement
*/ */
async setAudioQuality(quality: number): Promise<boolean> { async setAudioQuality(quality: number): Promise<boolean> {
try { try {
const response = await api.POST('/audio/quality', { quality }); const response = await api.POST('/audio/quality', { quality });
return response.ok;
if (!response.ok) {
return false;
}
await this.replaceAudioTrack();
return true;
} catch (error) { } catch (error) {
console.error('Failed to set audio quality:', error); console.error('Failed to set audio quality:', error);
return false; return false;

View File

@ -94,6 +94,17 @@ export const formatters = {
}, },
}; };
export function someIterable<T>(
iterable: Iterable<T>,
predicate: (item: T) => boolean,
): boolean {
for (const item of iterable) {
if (predicate(item)) return true;
}
return false;
}
export const VIDEO = new Blob( export const VIDEO = new Blob(
[ [
new Uint8Array([ new Uint8Array([

View File

@ -4,6 +4,7 @@ import (
"context" "context"
"encoding/base64" "encoding/base64"
"encoding/json" "encoding/json"
"fmt"
"net" "net"
"runtime" "runtime"
"strings" "strings"
@ -24,6 +25,7 @@ type Session struct {
peerConnection *webrtc.PeerConnection peerConnection *webrtc.PeerConnection
VideoTrack *webrtc.TrackLocalStaticSample VideoTrack *webrtc.TrackLocalStaticSample
AudioTrack *webrtc.TrackLocalStaticSample AudioTrack *webrtc.TrackLocalStaticSample
AudioRtpSender *webrtc.RTPSender
ControlChannel *webrtc.DataChannel ControlChannel *webrtc.DataChannel
RPCChannel *webrtc.DataChannel RPCChannel *webrtc.DataChannel
HidChannel *webrtc.DataChannel HidChannel *webrtc.DataChannel
@ -231,22 +233,21 @@ func newSession(config SessionConfig) (*Session, error) {
} }
}) })
session.VideoTrack, err = webrtc.NewTrackLocalStaticSample(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeH264}, "video", "kvm") session.VideoTrack, err = webrtc.NewTrackLocalStaticSample(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeH264}, "video", "kvm-video")
if err != nil { if err != nil {
scopedLogger.Warn().Err(err).Msg("Failed to create VideoTrack") scopedLogger.Warn().Err(err).Msg("Failed to create VideoTrack")
return nil, err return nil, err
} }
session.AudioTrack, err = webrtc.NewTrackLocalStaticSample(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeOpus}, "audio", "kvm") session.AudioTrack, err = webrtc.NewTrackLocalStaticSample(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeOpus}, "audio", "kvm-audio")
if err != nil { if err != nil {
scopedLogger.Warn().Err(err).Msg("Failed to add VideoTrack to PeerConnection") scopedLogger.Warn().Err(err).Msg("Failed to add VideoTrack to PeerConnection")
return nil, err return nil, err
} }
// Update the audio relay with the new WebRTC audio track // Update the audio relay with the new WebRTC audio track asynchronously
if err := audio.UpdateAudioRelayTrack(session.AudioTrack); err != nil { // This prevents blocking during session creation and avoids mutex deadlocks
scopedLogger.Warn().Err(err).Msg("Failed to update audio relay track") audio.UpdateAudioRelayTrackAsync(session.AudioTrack)
}
videoRtpSender, err := peerConnection.AddTrack(session.VideoTrack) videoRtpSender, err := peerConnection.AddTrack(session.VideoTrack)
if err != nil { if err != nil {
@ -261,6 +262,7 @@ func newSession(config SessionConfig) (*Session, error) {
return nil, err return nil, err
} }
audioRtpSender := audioTransceiver.Sender() audioRtpSender := audioTransceiver.Sender()
session.AudioRtpSender = audioRtpSender
// Handle incoming audio track (microphone from browser) // Handle incoming audio track (microphone from browser)
peerConnection.OnTrack(func(track *webrtc.TrackRemote, receiver *webrtc.RTPReceiver) { peerConnection.OnTrack(func(track *webrtc.TrackRemote, receiver *webrtc.RTPReceiver) {
@ -410,6 +412,22 @@ func (s *Session) stopAudioProcessor() {
s.audioWg.Wait() s.audioWg.Wait()
} }
// ReplaceAudioTrack replaces the current audio track with a new one
func (s *Session) ReplaceAudioTrack(newTrack *webrtc.TrackLocalStaticSample) error {
if s.AudioRtpSender == nil {
return fmt.Errorf("audio RTP sender not available")
}
// Replace the track using the RTP sender
if err := s.AudioRtpSender.ReplaceTrack(newTrack); err != nil {
return fmt.Errorf("failed to replace audio track: %w", err)
}
// Update the session's audio track reference
s.AudioTrack = newTrack
return nil
}
func drainRtpSender(rtpSender *webrtc.RTPSender) { func drainRtpSender(rtpSender *webrtc.RTPSender) {
// Lock to OS thread to isolate RTCP processing // Lock to OS thread to isolate RTCP processing
runtime.LockOSThread() runtime.LockOSThread()