Agent Native

databricks-mlflow-evaluation

This skill teaches how to evaluate generative AI agents using MLflow 3 on Databricks, covering evaluation setup, custom scorer creation, production trace…

by DatabricksOfficial publisherNot reviewed

Identity proven by GitHub's verified organisation badge (databricks)

Agents
Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode
License
LicenseRef-Databricks (restricted)
Pin
e3df77bpinned Oct 9, 2026
Context cost
about 2,500 tokenswhen the skill loads
GitHub stars
345on databricks/databricks-agent-skills
Scripts
None found

What it does

“MLflow 3 GenAI agent evaluation. Use when writing mlflow.genai.evaluate() code, creating @scorer functions, using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), building eval datasets from traces, setting up trace ingestion and production monitoring, aligning judges with MemAlign from domain expert feedback, or running optimize_prompts() with GEPA for automated prompt improvement.”
The author's description, from SKILL.md

Generated summary from the skill's own text

This skill teaches how to evaluate generative AI agents using MLflow 3 on Databricks, covering evaluation setup, custom scorer creation, production trace ingestion into Unity Catalog, judge alignment via MemAlign, and automated prompt optimization with GEPA. It layers Databricks-specific patterns like UC-backed datasets, trace ingestion, and production monitoring on top of the core MLflow evaluation workflow.

When to use it

  • Building evaluation datasets from production traces in Unity Catalog
  • Creating custom scorers or aligning judges to domain expert feedback
  • Setting up automated prompt optimization with optimize_prompts() and GEPA
  • Ingesting and monitoring agent traces in production Databricks environments

When not to

  • Generic MLflow.genai.evaluate() setup without Databricks infrastructure
  • Evaluating non-Databricks agents or using external trace systems

Install

Restricted license: install from the publisher's repository and read its terms.

These commands install from the publisher's repository and need no account.

skills CLI

npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation

Installs from the repository's default branch, which may have changed since the commit we checked.

GitHub CLI 2.90 or later

gh skill install databricks/databricks-agent-skills databricks-mlflow-evaluation@e3df77b3f0eb5d70392eb35c6761ca0f2929a4f6

Installs the commit we checked (e3df77b).

Copy by hand

Download the folder at e3df77b and put it in .claude/skills/databricks-mlflow-evaluation for Claude Code and in .agents/skills/databricks-mlflow-evaluation for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode.

Skill folder on GitHub at e3df77b

Claude Code plugin

/plugin install databricks@claude-plugins-official

The publisher's plugin in Claude's official marketplace. It may hold more than this skill, and we have not pinned it.

Copy for your agent

A prompt for any coding agent. It installs this skill from GitHub at the pinned commit, copies the files as they are, and runs nothing unless you ask.

Prompt

Install the agent skill "databricks-mlflow-evaluation" by Databricks.

Source, pinned to commit e3df77b3f0eb5d70392eb35c6761ca0f2929a4f6:
https://github.com/databricks/databricks-agent-skills/tree/e3df77b3f0eb5d70392eb35c6761ca0f2929a4f6/plugins/databricks/claude/skills/databricks-mlflow-evaluation

1. Download that folder at exactly this commit. Do not use the default branch.
2. Copy every file unchanged, keeping the folder structure, into .claude/skills/databricks-mlflow-evaluation/ for Claude Code and .agents/skills/databricks-mlflow-evaluation/ for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode. Create the folders if they do not exist.
3. Do not edit the files, and do not run any script in the folder unless I ask you to.
4. When you are done, list the files you copied and where you put them.

Its license is restricted or not stated. Show me the license terms from the repository before you copy anything.

Listing and checks: https://www.agentnative.dev/skills/databricks/databricks-mlflow-evaluation

SKILL.md at the pin

The license of this skill does not let us show its files here.

View SKILL.md on GitHub at e3df77b

Files

15 files, 233 KB in total. Each file has a sha256 in the pin's manifest; the first 12 characters are shown.

Skill folder on GitHub at e3df77b

Also in

The same SKILL.md content appears in these places too. We list it once and credit the oldest public source.

All 37 skills from Databricks

More from Databricks

  • databricks-agent-bricks

    This skill enables agents to create and manage Knowledge Assistants for document question-answering via RAG on Volumes, and Supervisor Agents that orchestrate…

    Claude Code, Codex, Cursor +3 · about 1.7K tokens · pinned 6 days ago

  • databricks-ai-functions

    This skill provides built-in Databricks AI Functions that add AI capabilities directly to SQL and PySpark pipelines, including classification, extraction…

    Claude Code, Codex, Cursor +3 · about 3.8K tokens · pinned 6 days ago

  • databricks-ai-runtime

    This skill enables agents to submit and manage GPU training workloads on Databricks serverless compute using the `databricks air` CLI.

    Claude Code, Codex, Cursor +4 · about 2.9K tokens · pinned 6 days ago

  • databricks-aibi-dashboards

    Creates Databricks AI/BI dashboards using JSON specifications with support for multiple visualization types.

    Claude Code, Codex, Cursor +3 · about 9.5K tokens · pinned 6 days ago

  • databricks-app-design

    This skill helps design the user experience of custom-code Databricks Apps built with AppKit and React that display data.

    Claude Code, Codex, Cursor +3 · about 1.6K tokens · pinned 6 days ago

  • databricks-apps

    Build custom interactive data apps on Databricks using AppKit (TypeScript/React framework) or other frameworks like Streamlit or FastAPI.

    Claude Code, Codex, Cursor +3 · about 5.5K tokens · pinned 6 days ago

All AI and agents skills

Related in AI and agents

  • databricks-unity-gateway

    This skill manages Unity Gateway services on Databricks through the CLI, including model services, MCP services, and model provider services.

    by DatabricksOfficial publisher

    Claude Code, Codex, Cursor +3 · about 1.5K tokens · pinned 6 days ago

  • databricks-unstructured-pdf-generation

    This skill generates synthetic PDF documents from HTML and pairs them with test questions for RAG evaluation datasets on Databricks Unity Catalog.

    by DatabricksOfficial publisher

    Contains scripts

    Claude Code, Codex, Cursor +3 · about 1.6K tokens · pinned 6 days ago

  • neon-ai-gateway

    The Neon AI Gateway provides a single API endpoint and credential to access frontier and open-source LLMs from multiple providers (OpenAI, Anthropic, Google…

    by NeonOfficial publisher

    Claude Code, Codex, Cursor +4 · about 4.9K tokens · pinned 6 days ago

  • agent-builder

    Agent Builder orchestrates the creation, modification, testing, and publishing of n8n Agents.

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +3 · about 2.3K tokens · pinned 6 days ago

  • intent-recognition

    Classifies automation requests by determining which primitive owns the top-level control flow (workflow, agent, needs clarification, or out-of-scope) and…

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +4 · about 6.9K tokens · pinned 6 days ago

  • planned-task-runtime

    This skill manages follow-up interactions with a background task execution system, handling task completion synthesis, failure replanning, workflow building…

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +3 · about 2.2K tokens · pinned 6 days ago

Open roles at Databricks

5 of 136 open roles on the jobs board

See all 136 roles at Databricks on the jobs board

The skills listed here are third-party software, collected from public GitHub repositories as a catalog of recent skills and provided as is. Agent Native does not write or control them and is not responsible for any harm, loss or damage caused by installing or using them. Labels and checks describe what we saw at one commit and are not a guarantee. Read a skill's files before you run it. Terms of use