Chimera Audio Capture

The ChimeraAudioCaptureComponent captures audio from the microphone and provides WebRTC-based audio processing including echo cancellation, noise suppression, and automatic gain control.

Properties

Echo Cancellation

The component uses WebRTC's AEC3 algorithm to remove speaker echo from the microphone signal. This enables full-duplex conversations where the AI can speak while the user is also speaking.

How AEC Works

1. Render Audio: Audio played through speakers (AI voice) is fed to the AEC
2. Capture Audio: Microphone audio is captured
3. Processing: AEC removes the speaker audio from the mic signal
4. Clean Output: Echo-free audio is sent to the server

Feeding Render Audio

For AEC to work, you must feed the received audio (AI voice) to the audio capture component:

// In Blueprint: OnAudioDataReceived → FeedRenderAudioFloat
AudioCapture->FeedRenderAudioFloat(AudioData, FramesPerChannel, Channels);

Methods

StartCapture

Begin capturing audio from the microphone.

UFUNCTION(BlueprintCallable)
void StartCapture();

StopCapture

Stop audio capture.

UFUNCTION(BlueprintCallable)
void StopCapture();

FeedRenderAudioFloat

Feed speaker audio to the AEC for echo cancellation.

UFUNCTION(BlueprintCallable)
void FeedRenderAudioFloat(const TArray<float>& AudioData, int32 FramesPerChannel, int32 Channels);

Events

OnAudioCaptured

Fired when audio is captured and processed.

UPROPERTY(BlueprintAssignable)
FOnChimeraAudioCaptured OnAudioCaptured;

Usage with Publisher

The AudioCaptureComponent automatically connects to the Publisher if bAutoConnectMicCapture is enabled on the Publisher:

// Automatic connection (default)
Publisher->bAutoConnectMicCapture = true;
 
// Manual connection
AudioCapture->SetTargetPublisher(Publisher);

Blueprint Example

1. Add both Chimera Publisher and Chimera Audio Capture to your Actor
2. In Audio Capture details, enable Echo Cancellation
3. Bind OnAudioDataReceived from Publisher to FeedRenderAudioFloat on Audio Capture
4. Capture starts automatically when Publisher connects