Operator actions can be annotated via API: Pegasus 1.6 understands first-person video

According to TwelveLabs, the model annotates actions, checks behavior against an expected pattern, and predicts the operator's next action. For robotics developers, these tasks involve analyzing recordings in which a person shows the system what to do.

How to use the new model

Migrating from 1.5. Update the Python SDK with `pip install --upgrade twelvelabs` and set `model_name="pegasus1.6"`. No other request parameters need to change.

Previously, images required a separate tool. Now Pegasus analyzes them through the same API as video: a request can include 1 to 20 images, each up to 20 MB.

The `time_array` field extracts timestamped events within a segment. In its guide, TwelveLabs uses this to identify each line of dialogue within a scene.

Supported recordings and pricing

Pegasus 1.6 analyzes videos up to 2 hours long. For a source video up to 4 hours long, you need to select a clip between 1 second and 2 hours. Input and response share a limit of 261,120 tokens.

To segment long recordings, you can set `max_tokens="unlimited"`. The task returns extracted segments even when the result is incomplete and records a warning in the `error` field.

Annotation costs. Under the Developer plan as of Oct 6, analysis costs $1.75 per hour of input video. Images cost $3 per 1 million input tokens, and model responses cost $7.50 per 1 million output tokens.

In Segment, each specified segment type is billed separately. An hour of footage with 4 `segment definitions` is billed as 240 minutes of processing. The `start` and `end` parameters shorten the portion billed.

Batch analysis remains on Pegasus 1.5, and requests without `model_name` also use version 1.5.