What is an AI auto-zoom video API?

An AI auto-zoom video API is a programmable interface that automatically tracks faces and active speakers in a video conference and smoothly zooms the camera view to keep the current speaker or subject centered and framed. V100's implementation uses a Rust server-side brain with exponential smoothing and hysteresis to prevent jitter, combined with client-side face detection for low-latency framing adjustments.

How does V100 prevent auto-zoom jitter?

V100 uses two techniques to prevent jitter. Exponential smoothing applies a weighted average to face position data so the zoom viewport moves smoothly rather than jumping frame-to-frame. Hysteresis prevents the zoom from switching speakers too quickly -- a new speaker must be active for a configurable threshold (default 500ms) before the zoom transitions, preventing rapid switching during crosstalk.

Can I programmatically control the zoom?

Yes. V100 provides a zoomTo() API that lets developers programmatically set the zoom level and position for any participant tile. This enables custom zoom behaviors: zoom to a specific region during screen sharing, zoom to a presenter's face during a keynote, or implement custom auto-zoom logic using V100's face detection data.

NOBODY ELSE HAS THIS

AI Auto-Zoom Video API
with Face Tracking

An AI auto-zoom video API automatically tracks faces and active speakers in a video conference, smoothly zooming the camera viewport to keep the subject centered and professionally framed. V100 is the only video API that ships this feature. The system uses a Rust server-side brain for active speaker detection with exponential smoothing and hysteresis to prevent jitter, combined with client-side FaceDetector API (with MediaPipe and motion detection fallbacks) for low-latency face tracking. Developers also get a zoomTo() API for programmatic zoom control on any participant tile.

Get API Key View API Reference

Automatic face tracking

Active speaker detection

zoomTo() programmatic control

How V100 Auto-Zoom Works

V100's auto-zoom operates as a hybrid client-server system. The server-side Rust brain handles active speaker detection and zoom coordination across participants. The client-side handles face detection for immediate, low-latency framing. This architecture delivers smooth, professional camera behavior that no JavaScript-only solution can match.

Rust Server-Side Brain

The server-side component runs in V100's Rust media server. It receives audio energy levels from all participants and determines the active speaker using a combination of audio level thresholding and voice activity detection. The brain then coordinates zoom transitions across all client tiles.

Exponential smoothing: weighted average of audio energy over time window

Hysteresis: 500ms default threshold prevents crosstalk switching

Configurable smoothing factor and hysteresis duration

Sub-millisecond decision latency in Rust

Client-Side Face Detection

Face detection runs on the client for immediate framing adjustments. V100 uses the browser's native FaceDetector API where available (Chrome, Edge), falls back to MediaPipe face detection, and uses motion detection as a final fallback. This three-tier strategy ensures auto-zoom works on every browser and device.

FaceDetector API: native, zero-dependency (Chrome/Edge)

MediaPipe: WebAssembly fallback (Firefox, Safari)

Motion detection: canvas diff fallback (legacy browsers)

Per-tile independent processing

Exponential Smoothing

Raw face detection coordinates jitter frame-to-frame. V100 applies exponential smoothing to the detected face bounding box, producing a weighted average that gives more weight to recent frames while preserving stability. The result is a smooth, cinematic camera movement instead of a twitchy jump-cut experience.

Configurable alpha (smoothing factor): 0.0-1.0

Lower alpha = smoother but more delayed

Default 0.3 balances responsiveness and stability

zoomTo() Programmatic API

For custom zoom behaviors, the zoomTo() API gives developers direct control over the zoom viewport for any participant tile. Override auto-zoom temporarily, zoom to specific regions during screen sharing, or build entirely custom zoom logic using V100's face detection coordinates as input.

Set zoom level, x, y, and transition duration

Per-tile control (zoom one, not all)

Animated transitions with configurable easing

Resume auto-zoom after manual override

Why No Other Video API Has This

Auto-zoom requires real-time coordination between audio analysis, face detection, and viewport rendering with frame-level precision. Most video APIs are JavaScript-only or rely on third-party WebRTC libraries that provide no access to the rendering pipeline. V100's Rust media server and custom client SDK make this possible.

Twilio Video

No auto-zoom. No face detection. Active speaker events are available but viewport control is not exposed -- developers cannot programmatically zoom a participant tile.

Daily.co

No auto-zoom. Active speaker detection exists for layout switching but there is no face tracking, no viewport manipulation, and no API for zooming into a participant's feed.

LiveKit

No auto-zoom. Open-source SFU focused on WebRTC transport. Face detection and viewport manipulation are not part of the project scope. Active speaker is audio-level only.

V100 is the only video API with auto-zoom. Rust server-side brain, client-side face detection with three-tier fallback, exponential smoothing, hysteresis, per-tile independence, and a programmatic zoomTo() API. This is a differentiating feature that cannot be replicated with any other video API.

Integration Guide

Enable auto-zoom with a single flag. Customize behavior with smoothing and hysteresis parameters. Use zoomTo() for programmatic control.

auto-zoom.js

// npm install v100-sdk
import { V100 } from 'v100-sdk';

const v100 = new V100('YOUR_API_KEY');

// Create a meeting with auto-zoom enabled
const meeting = await v100.meetings.create({
  name: 'Product Demo',
  autoZoom: {
    enabled: true,
    mode: 'active-speaker',     // 'active-speaker' | 'face-track' | 'manual'
    smoothingAlpha: 0.3,       // Exponential smoothing factor (0.0-1.0)
    hysteresisMs: 500,         // Min ms before switching speakers
    zoomLevel: 1.5,            // Default zoom magnification (1.0-3.0)
    optIn: true                // Participants can toggle on/off
  }
});

// Join and initialize client with auto-zoom
const room = await v100.join(meeting.joinUrl, {
  autoZoom: true,
  faceDetection: 'auto'  // 'native' | 'mediapipe' | 'motion' | 'auto'
});

// Listen for auto-zoom events
room.on('autoZoom.speakerChanged', (event) => {
  console.log(`Active speaker: ${event.participantId}`);
  console.log(`Zoom: ${event.zoomLevel}x at (${event.x}, ${event.y})`);
});

room.on('autoZoom.faceDetected', (event) => {
  console.log(`Face bounds: ${JSON.stringify(event.boundingBox)}`);
});

// Programmatic zoom control via zoomTo()
room.zoomTo({
  participantId: 'user_abc',
  zoom: 2.0,                    // 2x magnification
  x: 0.5,                       // Center X (0.0-1.0)
  y: 0.3,                       // Center Y (0.0-1.0)
  transitionMs: 300,            // Smooth transition duration
  easing: 'ease-out'            // CSS easing function
});

// Resume auto-zoom after manual override
room.resumeAutoZoom('user_abc');

Use Cases

Auto-zoom transforms video quality from amateur webcam to professional broadcast without hardware changes.

Sales Demos

Keep the presenter's face perfectly framed regardless of camera position. When they lean in to emphasize a point, the zoom follows. Professional appearance builds buyer confidence.

Webinars and Keynotes

Active speaker detection zooms to the current speaker during panel discussions. No manual camera switching needed. Transitions are smooth and cinematic, matching broadcast quality.

Telehealth

Face tracking keeps patients centered even when they shift position. Clinicians can use zoomTo() to focus on specific areas during visual assessments without asking the patient to adjust their camera.

Hybrid Meetings

Conference room cameras with a wide field of view benefit from auto-zoom. The system tracks the active in-room speaker and zooms to their face, giving remote participants a focused view instead of a distant wide shot.

Recording and Content

Recordings with auto-zoom applied look significantly more polished. The active speaker is always framed well, creating content that is immediately shareable without post-production camera edits.

Education

Auto-zoom keeps the instructor centered while they move around. During student presentations, active speaker detection smoothly transitions focus. zoomTo() can focus on whiteboard regions.

Pricing

Auto-zoom is included on all plans at no additional cost. It is a core V100 feature, not an add-on.

Free Tier

$0/mo

Test auto-zoom in development.

Auto-zoom included

Face tracking + speaker detection

zoomTo() API

60 minutes/month

RECOMMENDED

Pro

Usage-based

Production auto-zoom.

$0.002/min video

Auto-zoom included (no add-on cost)

All modes and configurations

Per-tile independent control

Enterprise

Custom

Custom auto-zoom behaviors.

Custom face detection models

Server-side rendering with zoom

Volume discounts

Priority support

AI Auto-Zoom Video API
with Face Tracking

How V100 Auto-Zoom Works

Rust Server-Side Brain

Client-Side Face Detection

Exponential Smoothing

zoomTo() Programmatic API

Why No Other Video API Has This

Twilio Video

Daily.co

LiveKit

Integration Guide

Use Cases

Sales Demos

Webinars and Keynotes

Telehealth

Hybrid Meetings

Recording and Content

Education

Pricing

Ship Auto-Zoom in Your Video App Today

Related Resources

Video Conferencing API

Noise Suppression API

Virtual Backgrounds API

AI Auto-Zoom Video APIwith Face Tracking

How V100 Auto-Zoom Works

Rust Server-Side Brain

Client-Side Face Detection

Exponential Smoothing

zoomTo() Programmatic API

Why No Other Video API Has This

Twilio Video

Daily.co

LiveKit

Integration Guide

Use Cases

Sales Demos

Webinars and Keynotes

Telehealth

Hybrid Meetings

Recording and Content

Education

Pricing

Ship Auto-Zoom in Your Video App Today

Related Resources

Video Conferencing API

Noise Suppression API

Virtual Backgrounds API

AI Auto-Zoom Video API
with Face Tracking