Agent Native

vlm-eval

This skill scores vision-language model evaluations of rollouts to determine task completion, then uses the score as a pipeline gate.

by NebiusOfficial publisherNot reviewed

Identity proven by GitHub's verified organisation badge (nebius)

Agents
Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, claude.ai and API
License
LicenseRef-Proprietary (restricted)
Pin
40bc4b8pinned Oct 9, 2026
Context cost
about 5,900 tokenswhen the skill loads
GitHub stars
34on nebius/nebius-physical-ai
Scripts
None found

What it does

“Use to score rollouts with a vision-language model and turn the score into a pipeline gate — single rollout, prefix-wide loop, rubric/threshold benchmark sweeps, backend selection (self-hosted, api, stub), and judging against a plan an earlier stage wrote.”
The author's description, from SKILL.md

Generated summary from the skill's own text

This skill scores vision-language model evaluations of rollouts to determine task completion, then uses the score as a pipeline gate. It supports single rollout evaluation, batch processing across rollout sets, configuration sweeps for threshold tuning, and multiple backend options including self-hosted, API, and stub inference.

When to use it

  • Assess whether robot rollouts completed their intended manipulation or navigation tasks
  • Compare vision-language model quality and select appropriate thresholds for task gates
  • Audit visual task completion against hardcoded or dynamically generated task specifications
  • Benchmark different models and rubrics on labeled datasets before production deployment

When not to

  • When you need independent human label verification or safety certification
  • If you require model outputs beyond numeric scores or when pure vision analysis is insufficient
  • When evaluating non-visual task completion or robotic safety without visual evidence review

Install

Restricted license: install from the publisher's repository and read its terms.

These commands install from the publisher's repository and need no account.

skills CLI

npx skills add nebius/nebius-physical-ai --skill vlm-eval

Installs from the repository's default branch, which may have changed since the commit we checked.

GitHub CLI 2.90 or later

gh skill install nebius/nebius-physical-ai vlm-eval@40bc4b8e5636debf3bcd06b5db5806e393355038

Installs the commit we checked (40bc4b8).

Copy by hand

Download the folder at 40bc4b8 and put it in .claude/skills/vlm-eval for Claude Code and in .agents/skills/vlm-eval for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode.

Skill folder on GitHub at 40bc4b8

Copy for your agent

A prompt for any coding agent. It installs this skill from GitHub at the pinned commit, copies the files as they are, and runs nothing unless you ask.

Prompt

Install the agent skill "vlm-eval" by Nebius.

Source, pinned to commit 40bc4b8e5636debf3bcd06b5db5806e393355038:
https://github.com/nebius/nebius-physical-ai/tree/40bc4b8e5636debf3bcd06b5db5806e393355038/skills/tools/vlm-eval

1. Download that folder at exactly this commit. Do not use the default branch.
2. Copy every file unchanged, keeping the folder structure, into .claude/skills/vlm-eval/ for Claude Code and .agents/skills/vlm-eval/ for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode. Create the folders if they do not exist.
3. Do not edit the files, and do not run any script in the folder unless I ask you to.
4. When you are done, list the files you copied and where you put them.

Its license is restricted or not stated. Show me the license terms from the repository before you copy anything.

Listing and checks: https://www.agentnative.dev/skills/nebius/vlm-eval

SKILL.md at the pin

The license of this skill does not let us show its files here.

View SKILL.md on GitHub at 40bc4b8

Files

1 file, 24 KB in total. Each file has a sha256 in the pin's manifest; the first 12 characters are shown.

Skill folder on GitHub at 40bc4b8
All 50 skills from Nebius

More from Nebius

  • agent-development

    This skill guides building the NPA chat agent backend using a grounded-first routing architecture that minimizes token spending.

    Claude Code, Codex, Cursor +4 · about 5.7K tokens · pinned 6 days ago

  • author-npa-workflow

    Helps authors write and validate NPA workflow YAML specifications, which are declarative state machines that orchestrate tools via SkyPilot.

    Claude Code, Codex, Cursor +4 · about 3.3K tokens · pinned 6 days ago

  • cosmos

    This skill enables the agent to work with Cosmos, a world model for synthetic data and video generation.

    Claude Code, Codex, Cursor +4 · about 1.6K tokens · pinned 6 days ago

  • cosmos3-npa-workflow

    This skill helps agents write, validate, and submit npa.workflow YAML specifications for Cosmos 3 image and video generation tasks.

    Claude Code, Codex, Cursor +4 · about 1.7K tokens · pinned 6 days ago

  • cosmos3-post-training

    This skill guides planning and understanding of Cosmos3 supervised fine-tuning workflows for robotic policy training on NPA infrastructure.

    Claude Code, Codex, Cursor +4 · about 2.5K tokens · pinned 6 days ago

  • curobo

    This skill enables agents to run NVIDIA cuRobo V2 motion planning for robotic arm pose planning and benchmarking on CUDA GPUs.

    Claude Code, Codex, Cursor +4 · about 1.5K tokens · pinned 6 days ago

All AI and agents skills

Related in AI and agents

  • agent-builder

    Agent Builder orchestrates the creation, modification, testing, and publishing of n8n Agents.

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +3 · about 2.3K tokens · pinned 6 days ago

  • intent-recognition

    Classifies automation requests by determining which primitive owns the top-level control flow (workflow, agent, needs clarification, or out-of-scope) and…

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +4 · about 6.9K tokens · pinned 6 days ago

  • planned-task-runtime

    This skill manages follow-up interactions with a background task execution system, handling task completion synthesis, failure replanning, workflow building…

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +3 · about 2.2K tokens · pinned 6 days ago

  • skill-creator

    This skill guides agents through creating new skills that extend Gemini CLI with specialized knowledge, workflows, and tool integrations.

    by GoogleOfficial publisher

    Contains scripts

    Gemini CLI · about 5K tokens · pinned 6 days ago

  • -21risk-automation

    This skill automates 21risk operations by connecting to the 21risk toolkit through Composio's Rube MCP server.

    by ComposioOfficial publisher

    Claude Code, Codex, Cursor +3 · about 690 tokens · pinned 6 days ago

  • -2chat-automation

    This skill automates 2chat tasks by integrating with Composio's 2chat toolkit through Rube MCP.

    by ComposioOfficial publisher

    Agents not labelled yet · about 680 tokens · pinned 6 days ago

Open roles at Nebius

5 of 83 open roles on the jobs board

See all 83 roles at Nebius on the jobs board

The skills listed here are third-party software, collected from public GitHub repositories as a catalog of recent skills and provided as is. Agent Native does not write or control them and is not responsible for any harm, loss or damage caused by installing or using them. Labels and checks describe what we saw at one commit and are not a guarantee. Read a skill's files before you run it. Terms of use