videoInference

Generate videos from text prompts and/or reference images using Runware's video generation API. Model Recommendations: - For Image-to-Video (I2V): Use 'klingai:5@2' if model is not provided, and use frameImages to guide the video generation, and do not use steps, CFGScale, or numberResults. - For Text-to-Video (T2V): Use 'google:3@1' if model is not provided, for pure text-based generation Args: positivePrompt: Text description of the video to generate model: Video generation model ID (recommended: klingai:5@2 ("width": 1920, "height": 1080) for I2V, google:3@1 ("width": 1280, "height": 720) for T2V) duration: The length of the generated video in seconds (min: 1, max: 10). This parameter directly affects the total number of frames produced based on the specified frame rate. Total frames are calculated as duration × fps. For example, a 5-second video at 24 fps will contain 120 frames. Longer durations require significantly more processing time and computational resources. Consider your specific use case when choosing duration length. width: Video width in pixels (optional, will be validated against model requirements) height: Video height in pixels (optional, will be validated against model requirements) outputType: Specifies the output type in which the video is returned. Currently, only URL delivery is supported for video outputs( default: "url") outputFormat: Specifies the format of the output video. Supported formats are: MP4 and WEBM (default: "mp4") outputQuality: Sets the compression quality of the output video. Higher values preserve more quality but increase file size, lower values reduce file size but decrease quality. (default: 95) uploadEndpoint: uploadEndpoint (str, optional): Specifies a URL where the generated content will be automatically uploaded using the HTTP PUT method such as Cloud storage, Webhook services, CDN integration. The content data will be sent as the request body, allowing your endpoint to receive and process the generated image or video immediately upon completion. includeCost: Whether to include cost information in response (default: False) negativePrompt: Text describing what NOT to include in the video. Common negative prompts for video include terms like "blurry", "low quality", "distorted", "static", "flickering", or specific content you want to exclude. frameImages: Array of frame objects that define key frames to guide video generation. Each object specifies an input image and optionally its position within the video timeline. This allows constraining specific frames within the video sequence, ensuring particular visual content appears at designated points (different from referenceImages which provide overall visual guidance without timeline constraints). Frame positioning options: - Omit frame parameter: Automatic distribution applies * 1 image: Used as first frame * 2 images: First and last frames * 3+ images: First, last, and evenly spaced intermediate frames - Named positions: "first" or "last" - Numeric positions: 0 (first frame) or any positive integer within frame count Example structures: - Single frame: [{"inputImage": "uuid_or_url"}] - First/last: [{"inputImage": "uuid1", "frame": "first"}, {"inputImage": "uuid2", "frame": "last"}] - Mixed: [{"inputImage": "uuid1", "frame": 0}, {"inputImage": "uuid2", "frame": 48}, {"inputImage": "uuid3", "frame": "last"}] inputImage accepts: UUID strings, data URIs, base64 data, or public URLs (PNG/JPG/WEBP) referenceImages: Array containing reference images used to condition the generation process. These images provide visual guidance to help the model generate content that aligns with the style, composition, or characteristics of the reference materials. Unlike frameImages which constrain specific timeline positions, reference images guide the general appearance that should appear consistently across the video. Reference images work in combination with your text prompt to provide both textual and visual guidance for the generation process. Each image can be specified in one of the following formats: - UUID v4 string of a previously uploaded image or generated image - Data URI string in format: data:<mediaType>;base64,<base64_data> - Base64 encoded image without data URI prefix - Public URL pointing to the image (PNG/JPG/WEBP supported) Example: ["aac49721-1964-481a-ae78-8a4e29b91402"] or ["https://example.com/image.jpg"] fps: The frame rate (frames per second) of the generated video. Higher frame rates create smoother motion but require more processing time and result in larger file sizes. example: 24 fps, 30 fps, 60 fps (default: 24 fps). Note: Using the same duration with higher frame rates creates smoother motion by generating more intermediate frames. The frame rate combines with duration to determine total frame count: duration × fps = total frames. steps: The number of denoising steps the model performs during video generation. More steps typically result in higher quality output but require longer processing time. seed: Random seed for reproducible results CFGScale: Controls how closely the video generation follows your prompt. Higher values make the model adhere more strictly to your text description, while lower values allow more creative freedom. range: 0 - 50.0 numberResults: Specifies how many videos to generate for the given parameters (default: 1) providerSettings: Additional provider-specific settings deliveryMethod: Determines how the video generation results are delivered. Currently, video inference only supports asynchronous processing due to the computational intensity of video generation. (default: "async") taskUUID: Custom task UUID (auto-generated if not provided) Returns: Dictionary containing video generation status and results Note: - Width and height will be automatically validated against the selected model's supported dimensions - For image-to-video generation, provide referenceImages or frameImages - For text-to-video generation, only positivePrompt is required - The tool automatically polls for completion when deliveryMethod is "async" . It is categorised as a Write tool in the Mcp Runware MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

Add a rule in your Intercept YAML policy under the tools section for videoInference. You can allow, deny, rate-limit, or validate arguments. Then run Intercept as a proxy in front of the Mcp Runware MCP server.

videoInference is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Yes. Add a rate_limit block to the videoInference rule in your Intercept policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

Set action: deny in the Intercept policy for videoInference. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

videoInference is provided by the Mcp Runware MCP server (elijahdev0/mcp-runware). Intercept sits as a proxy in front of this server to enforce policies before tool calls reach the server.