Evals for agent apps
An agent that ships LLM features should also write the tests that tell you whether they work. These skills cover eval datasets, graders, LLM-as-judge setups and the tracing tools that feed them.
24 listed skills in this collection, chosen from 2,274 in the directory.
Skills in Evals for agent apps
analyzing-offline-evaluations
This skill inspects and compares offline AI evaluation experiments stored in PostHog, diagnoses case-level regressions, and tracks scorer changes across…
by PostHogOfficial publisher
Claude Code, Codex, Cursor +4 · about 1.5K tokens · pinned 6 days ago
exploring-llm-evaluations
This skill enables agents to investigate, create, and manage AI observability evaluations in PostHog.
by PostHogOfficial publisher
Claude Code, Codex, Cursor +4 · about 5.6K tokens · pinned 6 days ago
huggingface-best
This skill helps users find the best AI models for their task by querying HuggingFace benchmark leaderboards, enriching results with model size and metadata…
by Hugging FaceOfficial publisher
Claude Code, Codex, Cursor +4 · about 1.3K tokens · pinned 6 days ago
huggingface-community-evals
This skill runs evaluations of Hugging Face Hub models on local hardware using inspect-ai or lighteval frameworks.
by Hugging FaceOfficial publisher
Contains scripts
Claude Code, Codex, Cursor +3 · about 1.6K tokens · pinned 6 days ago
eval-auth
This skill enables the agent to handle authentication evaluation requests by calling a specific MCP tool before responding.
by SentryOfficial publisher
Claude Code, Codex, Cursor +4 · about 230 tokens · pinned 6 days ago
eval-directory
This skill enables the agent to look up people in an evaluation directory using the eval-mcp tool provider.
by SentryOfficial publisher
Claude Code, Codex, Cursor +4 · about 39 tokens · pinned 6 days ago
eval-guardian-actions
This skill enables the agent to manage evaluation workspaces by performing cleanup operations and exporting credentials.
by SentryOfficial publisher
Claude Code, Codex, Cursor +4 · about 200 tokens · pinned 6 days ago
eval-mcp
This skill enables an agent to search and retrieve information from an eval MCP handbook by calling a dedicated handbook search tool with user queries.
by SentryOfficial publisher
Claude Code, Codex, Cursor +4 · about 120 tokens · pinned 6 days ago
databricks-mlflow-evaluation
This skill teaches how to evaluate generative AI agents using MLflow 3 on Databricks, covering evaluation setup, custom scorer creation, production trace…
by DatabricksOfficial publisher
Claude Code, Codex, Cursor +3 · about 2.5K tokens · pinned 6 days ago
databricks-unstructured-pdf-generation
This skill generates synthetic PDF documents from HTML and pairs them with test questions for RAG evaluation datasets on Databricks Unity Catalog.
by DatabricksOfficial publisher
Contains scripts
Claude Code, Codex, Cursor +3 · about 1.6K tokens · pinned 6 days ago
benchmark-agents
This skill runs advanced AI agent benchmarks by launching interactive Claude Code sessions through WezTerm, injecting a Vercel platform plugin, and verifying…
by VercelOfficial publisher
Claude Code · about 3.5K tokens · pinned 6 days ago
langfuse
This skill enables agents to interact with Langfuse for AI observability tasks including tracing, monitoring, and evaluation of AI applications.
by LangfuseOfficial publisher
Claude Code, Codex, Cursor +4 · about 1.9K tokens · pinned 6 days ago
migrate-to-langfuse
Migrate live tracing and historical data from another LLM observability platform (LangSmith, Arize AX, Phoenix, Braintrust, Helicone, Promptfoo, etc.) to…
by LangfuseOfficial publisher
Claude Code, Codex, Cursor +4 · about 1.6K tokens · pinned 6 days ago
skill-creator
This skill guides developers through creating effective skills that extend Claude's capabilities with specialized knowledge, workflows, and tool integrations.
by LangfuseOfficial publisher
Contains scripts
Claude Code, Codex, Cursor +3 · about 4.4K tokens · pinned 6 days ago
pwc-mcp
This skill connects an agent to the Papers With Code catalog to search papers by keywords or semantics, retrieve paper metadata and evaluations, inspect AI/ML…
by Hugging FaceOfficial publisher
Contains scripts
Claude Code, Codex, Cursor +3 · about 2.6K tokens · pinned 6 days ago
agent-observability-auto-experiment
Runs a local hill-climb optimization loop to improve code or prompts against real Datadog LLM-Obs evaluation data.
by DatadogOfficial publisher
Contains scripts
Claude Code · about 28.4K tokens · pinned 6 days ago
agent-observability-build-eval-from-annotations
Fits an LLM evaluator to human-labeled data by iteratively scoring a judge against the labels, reading its errors, making targeted improvements, and…
by DatadogOfficial publisher
Contains scripts
Claude Code, Codex, Cursor +3 · about 12.1K tokens · pinned 6 days ago
agent-observability-eval-bootstrap
Generates evaluators from production LLM traces by analyzing patterns and quality dimensions.
by DatadogOfficial publisher
Claude Code · about 24.5K tokens · pinned 6 days ago
score-eval
This skill evaluates code changes against a predefined rubric by analyzing diffs and comparing them to an original fixture application.
by NeonOfficial publisher
Claude Code · about 120 tokens · pinned 6 days ago
exploring-llm-evaluations
This skill investigates and manages AI observability evaluations in PostHog, covering three evaluation types: hog (code-based deterministic checks), llm_judge…
by PostHogOfficial publisher
Claude Code, Codex, Cursor +4 · about 5.5K tokens · pinned 6 days ago
vlm-eval
This skill scores vision-language model evaluations of rollouts to determine task completion, then uses the score as a pipeline gate.
by NebiusOfficial publisher
Claude Code, Codex, Cursor +4 · about 5.9K tokens · pinned 6 days ago
inngest-agent-evals
Builds production evaluation workflows for AI agents on Inngest using direct and deferred scoring, sessions for grouping related runs, step experiments for…
by InngestOfficial publisher
Claude Code, Codex, Cursor +4 · about 2.6K tokens · pinned 6 days ago
huggingface-best
This skill helps users find the best AI models for their specific tasks by querying HuggingFace benchmark leaderboards and filtering results based on device…
by Hugging FaceOfficial publisher
Claude Code, Codex, Cursor +4 · about 1.3K tokens · pinned 6 days ago
web-search-agent-evals
This skill provides development tools for evaluating web search capabilities across four CLI agents (Claude Code, Gemini, Droid, Codex) using two search…
by You.comOfficial publisher
Claude Code, Codex, Cursor +4 · about 3.4K tokens · pinned 6 days ago
Other collections
Payments with Stripe
Stripe's own skills and others for checkout, subscriptions and billing work with Stripe.
24 listed skills
Supabase, auth and row-level security
Skills for building on Supabase: Postgres, auth, row-level security policies and edge functions.
15 listed skills
Building and securing MCP servers
Skills for writing, testing and locking down Model Context Protocol servers.
24 listed skills
Browser automation
Skills that let an agent drive a browser: Playwright, Puppeteer, browser agents and scraping.
24 listed skills
Code review
Skills that make an agent review diffs and pull requests the way a careful senior engineer would.
13 listed skills
Testing and TDD
Skills for test-driven development, unit and end-to-end tests, and fixing flaky suites.
24 listed skills
Next.js and Vercel
Vercel's own skills and others for Next.js apps, React Server Components and deploys on Vercel.
24 listed skills
Observability and incidents
Skills for logs, traces, error tracking and incident response with Sentry, Datadog, Grafana and others.
24 listed skills
Security reviews
Skills for security review, vulnerability scanning, secrets and threat modelling.
24 listed skills