Agent Native

databricks-pipelines

Build batch and streaming data pipelines on Databricks using Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) in Python or SQL.

by DatabricksOfficial publisherNot reviewed

Identity proven by GitHub's verified organisation badge (databricks)

Agents
Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode
License
LicenseRef-Databricks (restricted)
Pin
e3df77bpinned Oct 9, 2026
Context cost
about 8,500 tokenswhen the skill loads
GitHub stars
345on databricks/databricks-agent-skills
Scripts
None found

What it does

“Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.”
The author's description, from SKILL.md

Generated summary from the skill's own text

Build batch and streaming data pipelines on Databricks using Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) in Python or SQL. The skill covers streaming tables, materialized views, Auto CDC for change data capture, Auto Loader for file ingestion, data quality expectations, and sinks for external outputs. Start here before implementing any pipeline logic.

When to use it

  • Building incremental or streaming data pipelines from files, databases, or message buses
  • Processing CDC feeds or periodic snapshots to track data changes
  • Ingesting files from cloud storage or volumes with Auto Loader
  • Applying data quality constraints to pipeline datasets

When not to

  • Migrating existing DBR 14 or earlier pipelines without updating runtime versions
  • Using batch-only aggregations on frequently changing source tables without serverless enabled

Install

Restricted license: install from the publisher's repository and read its terms.

These commands install from the publisher's repository and need no account.

skills CLI

npx skills add databricks/databricks-agent-skills --skill databricks-pipelines

Installs from the repository's default branch, which may have changed since the commit we checked.

GitHub CLI 2.90 or later

gh skill install databricks/databricks-agent-skills databricks-pipelines@e3df77b3f0eb5d70392eb35c6761ca0f2929a4f6

Installs the commit we checked (e3df77b).

Copy by hand

Download the folder at e3df77b and put it in .claude/skills/databricks-pipelines for Claude Code and in .agents/skills/databricks-pipelines for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode.

Skill folder on GitHub at e3df77b

Claude Code plugin

/plugin install databricks@claude-plugins-official

The publisher's plugin in Claude's official marketplace. It may hold more than this skill, and we have not pinned it.

Copy for your agent

A prompt for any coding agent. It installs this skill from GitHub at the pinned commit, copies the files as they are, and runs nothing unless you ask.

Prompt

Install the agent skill "databricks-pipelines" by Databricks.

Source, pinned to commit e3df77b3f0eb5d70392eb35c6761ca0f2929a4f6:
https://github.com/databricks/databricks-agent-skills/tree/e3df77b3f0eb5d70392eb35c6761ca0f2929a4f6/plugins/databricks/claude/skills/databricks-pipelines

1. Download that folder at exactly this commit. Do not use the default branch.
2. Copy every file unchanged, keeping the folder structure, into .claude/skills/databricks-pipelines/ for Claude Code and .agents/skills/databricks-pipelines/ for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode. Create the folders if they do not exist.
3. Do not edit the files, and do not run any script in the folder unless I ask you to.
4. When you are done, list the files you copied and where you put them.

Its license is restricted or not stated. Show me the license terms from the repository before you copy anything.

Listing and checks: https://www.agentnative.dev/skills/databricks/databricks-pipelines

SKILL.md at the pin

The license of this skill does not let us show its files here.

View SKILL.md on GitHub at e3df77b

Files

37 files, 197 KB in total. Each file has a sha256 in the pin's manifest; the first 12 characters are shown.

Skill folder on GitHub at e3df77b

Also in

The same SKILL.md content appears in these places too. We list it once and credit the oldest public source.

All 37 skills from Databricks

More from Databricks

  • databricks-agent-bricks

    This skill enables agents to create and manage Knowledge Assistants for document question-answering via RAG on Volumes, and Supervisor Agents that orchestrate…

    Claude Code, Codex, Cursor +3 · about 1.7K tokens · pinned 6 days ago

  • databricks-ai-functions

    This skill provides built-in Databricks AI Functions that add AI capabilities directly to SQL and PySpark pipelines, including classification, extraction…

    Claude Code, Codex, Cursor +3 · about 3.8K tokens · pinned 6 days ago

  • databricks-ai-runtime

    This skill enables agents to submit and manage GPU training workloads on Databricks serverless compute using the `databricks air` CLI.

    Claude Code, Codex, Cursor +4 · about 2.9K tokens · pinned 6 days ago

  • databricks-aibi-dashboards

    Creates Databricks AI/BI dashboards using JSON specifications with support for multiple visualization types.

    Claude Code, Codex, Cursor +3 · about 9.5K tokens · pinned 6 days ago

  • databricks-app-design

    This skill helps design the user experience of custom-code Databricks Apps built with AppKit and React that display data.

    Claude Code, Codex, Cursor +3 · about 1.6K tokens · pinned 6 days ago

  • databricks-apps

    Build custom interactive data apps on Databricks using AppKit (TypeScript/React framework) or other frameworks like Streamlit or FastAPI.

    Claude Code, Codex, Cursor +3 · about 5.5K tokens · pinned 6 days ago

All Data and databases skills

Related in Data and databases

  • databricks-execution-compute

    Execute Python, Scala, SQL, or R code on Databricks through three modes: Databricks Connect for local Spark jobs, serverless jobs for heavy processing, or…

    by DatabricksOfficial publisher

    Claude Code, Codex, Cursor +3 · about 2.2K tokens · pinned 6 days ago

  • databricks-jobs

    This skill helps develop and deploy Lakeflow Jobs on Databricks using Asset Bundles, Python SDK, or CLI.

    by DatabricksOfficial publisher

    Contains scripts

    Claude Code, Codex, Cursor +3 · about 3K tokens · pinned 6 days ago

  • databricks-python-sdk

    This skill guides agents in using the Databricks Python SDK, Databricks Connect, CLI, and REST API to interact with Databricks workspaces.

    by DatabricksOfficial publisher

    Contains scripts

    Claude Code, Codex, Cursor +3 · about 4.5K tokens · pinned 6 days ago

  • databricks-zerobus-ingest

    This skill enables agents to build gRPC-based data ingestion clients that write records directly into Databricks Delta tables using the Zerobus Ingest SDK.

    by DatabricksOfficial publisher

    Claude Code, Codex, Cursor +4 · about 2.9K tokens · pinned 6 days ago

  • alphagenome-single-variant-analysis

    Analyzes how genetic variants affect gene expression, chromatin accessibility, and splicing using the AlphaGenome API.

    by GoogleOfficial publisher

    Contains scripts

    Claude Code, Codex, Cursor +3 · about 2.8K tokens · pinned 6 days ago

  • alphagenome-variant-impact-score

    Score genetic variants using AlphaGenome Variant Impact models to assess functional impact through 18 biological feature attributions.

    by GoogleOfficial publisher

    Contains scripts

    Claude Code, Codex, Cursor +3 · about 4.1K tokens · pinned 6 days ago

Open roles at Databricks

5 of 136 open roles on the jobs board

See all 136 roles at Databricks on the jobs board

The skills listed here are third-party software, collected from public GitHub repositories as a catalog of recent skills and provided as is. Agent Native does not write or control them and is not responsible for any harm, loss or damage caused by installing or using them. Labels and checks describe what we saw at one commit and are not a guarantee. Read a skill's files before you run it. Terms of use