October 1, 2026
AI agents create videos with code: HeyGen checks results against human-made references
On Sep 18, HeyGen released Code2Video with 168 human-made videos to compare against AI agents' work.
HeyGen
@heygen
In September, we released three tools for people building products. An open source integration for a real-time conversational avatar with OpenAI's GPT-Live-1 Code2Video Benchmark with Kaggle: testing whether AI actually makes good videos And Professional Voice Clone for the API Read more: https://www.heygen.com/blog/heygen-september-2026-release
· 1.1K views
An agent writes the code for a video, HyperFrames turns it into an MP4, and Code2Video compares the result with human-made work.
HeyGen collected human-made reference videos across 8 product video categories. Agents now receive the same briefs and the HyperFrames skill for building videos with code. In HeyGen's Sep 18 benchmark, every participating model fell short of the human-made work.
Video through an agent. Install the skill with `npx skills add heygen-com/hyperframes`. A local project requires Node.js 22+ and FFmpeg. The build sequence:
1. `npx hyperframes init my-video` 2. `cd my-video` 3. `npx hyperframes preview` 4. `npx hyperframes render`
Kaggle hosts Code2Video run traces and logs and maintains a model leaderboard. HeyGen and Kaggle published the reference compositions, briefs, and judge API. HeyGen described running your own model with the same evaluator as a future capability.
Talking avatar. In September, HeyGen also released an open source integration of LiveAvatar with OpenAI's GPT-Live-1. The ready-to-run demo acts as a Japanese tutor with voice and animated vocabulary cards. To adapt it to your own use case, edit `server/prompts/instructions.md` and `greeting.md`, then restart the server.
Run the demo from HeyGen's repository with `pnpm install`, `pnpm run setup`, and `pnpm dev`. It requires Node.js 20.12 or later, pnpm, and LiveAvatar and OpenAI keys with access to `gpt-live-1`. Once it is running at `localhost:5173`, click Start and allow microphone access.
Professional Voice Clone adds a trainable voice replica to the API. Training requires 1 to 10 recordings of one person, totaling at least 20 minutes. Upload the files through the Assets API, then call `POST /v3/models/audio/voices` with `mode: professional` and check the status until it reaches `ACTIVE`.
The trained voice returns WAV audio through `POST /v3/models/audio/tts` or streams audio through `POST /v3/models/audio/tts/stream`. According to HeyGen's Sep 9 documentation, each voice occupies a paid slot with 5 training runs per monthly billing period, including the first. Failed training runs do not count toward the limit, and synthesis costs 0.6 API credits per minute.
The official Professional Voice Clone guide includes a ready-to-use prompt for setup through Claude Code, Codex, or Cursor.
Source
