Live streaming and off-screen rendering

When to use off-screen rendering

Use off-screen rendering when you need the processed frames rather than an on-screen preview:

• live streaming to platforms such as YouTube, Twitch or Facebook,
• recording to a file or feeding an encoder,
• background or batch image processing,
• custom output handling such as network transmission.

You apply beauty and filters exactly as you do for the on-screen preview. The effects are included in the off-screen output automatically.

iOS

Initialize off-screen processing with NosmaiCore, apply effects, then collect output.

#import <NosmaiSDK/NosmaiCore.h>

NosmaiCore *core = [[NosmaiCore alloc] init];
[core initializeWithLicenseKey:@"YOUR_LICENSE_KEY"];

// apply effects before processing
[core applyBeautyFilterWithSmooth:0.8 whitening:0.4];
[core applyFilter:@"filter_name.nosmai"];

Process a single image or pixel buffer.

[core processImage:inputImage completion:^(UIImage *processedImage, NSError *error) {
    // save or use the processed image
}];

[core processPixelBuffer:cameraPixelBuffer completion:^(CVPixelBufferRef processedBuffer, NSError *error) {
    // append to a video writer, or send to a stream
}];

Collect processed frames for streaming.

#import <NosmaiSDK/NosmaiSink.h>

NosmaiSink *sink = [[NosmaiSink alloc] init];
[sink setPixelBufferOutput:^(CVPixelBufferRef pixelBuffer) {
    // send to your RTMP client or streaming SDK
}];
[sink startOutput];

Android

Initialize NosmaiOffscreenSDK, choose an output method, feed camera frames, and apply effects.

NosmaiOffscreenSDK.setLicenseKey("YOUR_LICENSE_KEY");
NosmaiOffscreenSDK.initialize(context, new NosmaiOffscreenSDK.OffscreenEventListener() {
    @Override public void onInitialized() { }
    @Override public void onError(String error) { }
});

Option A: receive processed frames (for a custom video source).

NosmaiOffscreenSDK.setOffscreenRendering(1280, 720, NosmaiOffscreenSDK.PixelFormat.I420);
NosmaiOffscreenSDK.setFrameListener((data, width, height, format, timestamp) -> {
    // push I420 frames to your streaming SDK
});

Option B: render directly to an encoder surface.

Surface surface = streamingSdk.getInputSurface();
NosmaiOffscreenSDK.setRenderSurface(surface, 1280, 720);
NosmaiOffscreenSDK.setRenderRotation(0);
NosmaiOffscreenSDK.setMirrorX(true);   // for the front camera

Feed camera frames (YUV is preferred, fewer conversions).

NosmaiOffscreenSDK.receiveFrame(
    yBuffer, uBuffer, vBuffer,
    width, height,
    yStride, uStride, vStride,
    uPixelStride, vPixelStride,
    sensorOrientation,   // 0, 90, 180, 270
    isFrontCamera);

Clean up.

NosmaiOffscreenSDK.clearRenderSurface();  // if using a surface
NosmaiOffscreenSDK.cleanup();

Tips

• Prefer a YUV (YUV_420_888) input and I420 output for the widest compatibility.
• Reuse buffers and keep callbacks fast; offload heavy work to a background thread.
• Throttle to 24 to 30 FPS if your target does not need more.