Video Generation API for AI Agents: From JSON to a Published Reel With MCP

Most video generation APIs were designed for a developer with an SDK and a weekend. The caller reads the docs, writes a client, handles webhooks, and ships. In 2026 a growing share of callers are not developers at all. They are AI agents: Claude, ChatGPT, Codex, a Cursor agent, an n8n workflow with an LLM node. An agent does not read your docs. It reads a tool schema, plans a call, and expects the response to tell it what to do next.

This post is about what a video generation API looks like when it is built for that caller. We use Shortzly's own server as the example, because we built it this way on purpose, but the ideas apply to any JSON to video service you might expose to an agent.

The shape of the problem

A short-form reel is a small structured document. It has a title, an aspect ratio, a list of segments, and a few global choices: the voice, the caption style, the transition. Each segment is one spoken line plus one visual. That is it. A 30-second reel is six to eight of those.

The creative work is producing that document. The mechanical work is rendering it: synthesizing or aligning audio, fetching or generating visuals, adding motion to stills, burning captions at the right words, cutting transitions, encoding an MP4 and uploading it somewhere. Language models are excellent at the first part and cannot do the second. A well-designed API draws the line exactly there.

JSON in, reel out

Here is the whole request for a reel, as an agent would send it to Shortzly's create_reel tool:

{
  "title": "Why the sky is blue",
  "aspect_ratio": "9:16",
  "language": "en",
  "segments": [
    { "text": "Why is the sky blue and not, say, violet?",
      "kind": "video", "media": { "search": "blue sky clouds timelapse" } },
    { "text": "Sunlight is every colour at once.",
      "kind": "image", "media": { "search": "prism rainbow light" } },
    { "text": "Air molecules scatter short wavelengths far more than long ones.",
      "kind": "image", "media": { "url": "https://example.com/scatter-diagram.png" } },
    { "text": "Violet scatters even more, but our eyes barely register it.",
      "kind": "image", "media": { "asset_ref": "uploads/01K.../eye-sensitivity.png" } },
    { "text": "So the sky lands on blue. Follow for more sixty-second science.",
      "kind": "video", "media": { "search": "sunset sky" } }
  ],
  "voiceover": { "mode": "tts", "voice": "nova" },
  "captions": { "mode": "auto", "style": "capcut",
                "config": { "words_per_chunk": 3, "position_bottom": 260 } },
  "transition": "dissolve",
  "render_quality": "preview",
  "idempotency_key": "sky-blue-v1"
}

Three things make this agent-friendly rather than merely valid.

1. Every enum is discoverable

A model should never guess a voice name or a caption style. Shortzly exposes get_capabilities, a free tool that returns the valid voices (Nova, Onyx, Shimmer, Echo, Fable, Alloy), the twelve caption styles, four aspect ratios, seven transitions, four languages, the segment limits and a plain-language note on which options cost AI. The tool schema itself carries the same enums, so a client that reads schemas can skip the call.

2. Media is a choice, not a requirement

Each segment takes exactly one of three media sources. A search phrase resolves against Pexels and Pixabay with no LLM involved. A public url is fetched through an SSRF guard. An asset_ref points at a file the agent uploaded through create_upload, which returns a presigned PUT URL. An agent that has product screenshots uses uploads; an agent writing an explainer uses search; either way the request shape is the same.

3. The response tells the agent what to do next

{
  "project_id": "01M1H45Z...",
  "job_id": "01M1H45Z...",
  "status": "queued",
  "estimated_seconds": 114,
  "retry_after_seconds": 15,
  "idempotent_replay": false,
  "next_step": "Call get_job with job_id every retry_after_seconds until status is completed or failed."
}

That next_step field is not decoration. Agents follow it. The same pattern continues through get_job (poll interval, then the clip list), get_clip (download URL with an expiry), upgrade_clip_hd and publish_clip (post ids to poll with get_social_post). Every step names the next tool.

Why MCP instead of a bespoke REST client

You could expose all of this as REST and write an OpenAPI spec. We did consider it. MCP won for three reasons that matter specifically when the caller is a model.

• Discovery is built in. An MCP client asks the server for its tool list and gets names, descriptions and JSON schemas. There is no "read the docs" step, which is the step an agent cannot do reliably.
• Authentication is standardized. OAuth 2.1 with PKCE and dynamic client registration means Claude and ChatGPT connect with a URL and a consent screen. Bearer tokens cover terminals and CI. We did not have to invent either flow.
• One integration, every client. The same endpoint serves Claude Desktop, claude.ai, ChatGPT, Claude Code, Codex, Cursor, Windsurf and VS Code today, and any client released tomorrow that speaks the protocol. A REST client would have needed a plugin per assistant.

The transport is stateless JSON-RPC over HTTPS, which means no sessions to expire, no SSE connections to keep alive, and nothing for a cloud agent to hold open between calls.

Cost design for agents

Agents are cheap to run and easy to loop, so a video API needs a cost model that is predictable under repetition. Shortzly's rule is that the caller's model does all the writing and the server never runs its own script AI on this path. On top of that, every use of the server's own AI is opt-in per call:

Quota is the same as the dashboard: a preview render is half a credit, an HD render one credit, charged on completion. Failed renders are not charged. Read-only tools are free. Starter ($19/month) includes 10 faceless reels a month, Pro ($49/month) includes 100.

Idempotency, polling and recovery

Agents crash, get interrupted and retry. Three features exist for that.

• Idempotency keys. create_reel with a repeated idempotency_key returns the original job with idempotent_replay: true instead of rendering again.
• Polling hints. Every in-progress response carries retry_after_seconds. The rate limit is 120 requests per minute per user, generous for a well-behaved poller and tight enough to stop a runaway loop.
• Listing. list_jobs and list_clips let a restarted agent find work it started earlier without any local state.

Text to video versus JSON to video

"Text to video" usually means a diffusion model generating footage from a prompt. That is a different product: impressive, expensive per second, and not what a marketing team needs for a 30-second product reel with real screenshots and a specific message. "JSON to video" is the assembly job: a structured plan becomes a finished, captioned, publishable clip. For agents, the assembly job is the useful one, because the agent already produced the plan and wants it executed exactly.

If you need generated footage inside a reel, nothing stops an agent from generating a clip elsewhere, uploading it through create_upload, and referencing it in a segment. The assembly layer does not care where the pixels came from.

Putting it to work

Three integrations we have seen in the first weeks:

• A Claude Code loop over a CSV. One row per product, one reel per row, previews rendered in a batch, HD only for the rows a human approved.
• A support bot that answers with a video. The agent writes a 20-second walkthrough for a common question, renders it once, and reuses the clip URL.
• A launch pipeline in Cursor. The same repository that holds the app generates its launch reel from the current screenshots on every release branch.

To connect an agent, see the MCP server page for the tool list and the per-client setup, or the guide on adding an MCP server to Claude Code, Codex, Cursor and Windsurf. For fully unattended topic discovery and scheduling without an agent, Autopilot covers that case.

Frequently asked questions

Is there a plain REST API?

The MCP server wraps the same services the dashboard uses. If you need a plain HTTP integration with no AI client in the loop, contact support and we will point you at the right endpoints for your case.

Can I generate the video from a webhook or n8n?

Yes, through any workflow tool that has an MCP client node or can run a command-line agent such as Claude Code or Codex with a bearer token. The idempotency key makes retries safe.

What formats come out?

H.264 MP4 at 1080x1920 for 9:16, with 4:5, 1:1 and 16:9 also available. Previews render at 720p, HD at full resolution with normalized loudness.

How long does a render take?

A 30 to 45 second reel typically renders in about two minutes for a preview and the same again for the HD upgrade. create_reel returns an estimate and every poll returns the next interval.