Agent Native

exploring-llm-evaluations

This skill investigates and manages AI observability evaluations in PostHog, covering three evaluation types: hog (code-based deterministic checks), llm_judge…

by PostHogOfficial publisherNot reviewed

Identity proven by GitHub's verified organisation badge (PostHog)

Agents
Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, claude.ai and API
License
No license stated
Pin
e0fbf5epinned Oct 9, 2026
Context cost
about 5,500 tokenswhen the skill loads
GitHub stars
88on posthog/ai-plugin
Scripts
None found

What it does

“Investigate AI observability evaluations — `hog` (deterministic code-based), `llm_judge` (LLM-prompt-based), and `sentiment` (user-message sentiment). Find existing evaluations, inspect their configuration, run them against specific generations, query individual results, and set up scheduled reports on an evaluation. Use when the user asks to debug why an evaluation is failing, surface common failure modes, compare results across filters, dry-run a Hog evaluator, prototype a new LLM-judge prompt, inspect sentiment classifications, or manage the evaluation lifecycle.”
The author's description, from SKILL.md

Generated summary from the skill's own text

This skill investigates and manages AI observability evaluations in PostHog, covering three evaluation types: hog (code-based deterministic checks), llm_judge (LLM-prompted subjective scoring), and sentiment (user message sentiment classification). It enables finding evaluations, inspecting configurations, running them against specific generations, querying results, and setting up scheduled reports.

When to use it

  • Debug why an evaluation is failing or producing unexpected results
  • Prototype and test new hog or llm_judge evaluators before deployment
  • Compare evaluation results across filters and time periods
  • Dry-run Hog code against sample generations without saving

When not to

  • When you need to modify the underlying generation model or prompt that produced the content being evaluated
  • If the organization lacks AI data processing approval for llm_judge evaluations

Install

No license stated: install from the publisher's repository, and ask the author before you reuse or share it.

These commands install from the publisher's repository and need no account.

skills CLI

npx skills add posthog/ai-plugin --skill exploring-llm-evaluations

Installs from the repository's default branch, which may have changed since the commit we checked.

GitHub CLI 2.90 or later

gh skill install posthog/ai-plugin exploring-llm-evaluations@e0fbf5e742fb4242c4ea509b395b2e343642f0b7

Installs the commit we checked (e0fbf5e).

Copy by hand

Download the folder at e0fbf5e and put it in .claude/skills/exploring-llm-evaluations for Claude Code and in .agents/skills/exploring-llm-evaluations for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode.

Skill folder on GitHub at e0fbf5e

Claude Code plugin

/plugin install posthog@claude-plugins-official

The publisher's plugin in Claude's official marketplace. It may hold more than this skill, and we have not pinned it.

Copy for your agent

A prompt for any coding agent. It installs this skill from GitHub at the pinned commit, copies the files as they are, and runs nothing unless you ask.

Prompt

Install the agent skill "exploring-llm-evaluations" by PostHog.

Source, pinned to commit e0fbf5e742fb4242c4ea509b395b2e343642f0b7:
https://github.com/posthog/ai-plugin/tree/e0fbf5e742fb4242c4ea509b395b2e343642f0b7/skills/exploring-llm-evaluations

1. Download that folder at exactly this commit. Do not use the default branch.
2. Copy every file unchanged, keeping the folder structure, into .claude/skills/exploring-llm-evaluations/ for Claude Code and .agents/skills/exploring-llm-evaluations/ for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode. Create the folders if they do not exist.
3. Do not edit the files, and do not run any script in the folder unless I ask you to.
4. When you are done, list the files you copied and where you put them.

Its license is restricted or not stated. Show me the license terms from the repository before you copy anything.

Listing and checks: https://www.agentnative.dev/skills/posthog/exploring-llm-evaluations

SKILL.md at the pin

The license of this skill does not let us show its files here.

View SKILL.md on GitHub at e0fbf5e

Files

1 file, 23 KB in total. Each file has a sha256 in the pin's manifest; the first 12 characters are shown.

Skill folder on GitHub at e0fbf5e
All 50 skills from PostHog

More from PostHog

  • analyzing-experiment-session-replays

    This skill enables agents to analyze session recordings from experiment variants to identify behavioral differences between control and test groups.

    Claude Code, Codex, Cursor +4 · about 2.3K tokens · pinned 6 days ago

  • analyzing-offline-evaluations

    This skill inspects and compares offline AI evaluation experiments stored in PostHog, diagnoses case-level regressions, and tracks scorer changes across…

    Claude Code, Codex, Cursor +4 · about 1.5K tokens · pinned 6 days ago

  • auditing-warehouse-view-health

    Audits materialized views in a PostHog data warehouse to identify failed views and flag unused ones that waste compute resources.

    Claude Code, Codex, Cursor +4 · about 1.4K tokens · pinned 6 days ago

  • authoring-data-quality-checks

    This skill adds data quality checks (like dbt tests) on warehouse tables, views, and HogQL metrics, supporting assertions such as not-null, uniqueness…

    Claude Code, Codex, Cursor +4 · about 2.5K tokens · pinned 6 days ago

  • authoring-scouts

    This skill helps engineers write, edit, and adapt PostHog Signals scouts, scheduled agents that scan projects and report findings to an inbox.

    Claude Code, Codex, Cursor +4 · about 10.6K tokens · pinned 6 days ago

  • building-a-dashboard

    This skill constructs or modifies PostHog dashboards by selecting and arranging insight tiles.

    Claude Code, Codex, Cursor +4 · about 2.1K tokens · pinned 6 days ago

All AI and agents skills

Related in AI and agents

  • agent-builder

    Agent Builder orchestrates the creation, modification, testing, and publishing of n8n Agents.

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +3 · about 2.3K tokens · pinned 6 days ago

  • intent-recognition

    Classifies automation requests by determining which primitive owns the top-level control flow (workflow, agent, needs clarification, or out-of-scope) and…

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +4 · about 6.9K tokens · pinned 6 days ago

  • planned-task-runtime

    This skill manages follow-up interactions with a background task execution system, handling task completion synthesis, failure replanning, workflow building…

    by n8nOfficial publisher

    Claude Code, Codex, Cursor +3 · about 2.2K tokens · pinned 6 days ago

  • skill-creator

    This skill guides agents through creating new skills that extend Gemini CLI with specialized knowledge, workflows, and tool integrations.

    by GoogleOfficial publisher

    Contains scripts

    Gemini CLI · about 5K tokens · pinned 6 days ago

  • -21risk-automation

    This skill automates 21risk operations by connecting to the 21risk toolkit through Composio's Rube MCP server.

    by ComposioOfficial publisher

    Claude Code, Codex, Cursor +3 · about 690 tokens · pinned 6 days ago

  • -2chat-automation

    This skill automates 2chat tasks by integrating with Composio's 2chat toolkit through Rube MCP.

    by ComposioOfficial publisher

    Agents not labelled yet · about 680 tokens · pinned 6 days ago

The skills listed here are third-party software, collected from public GitHub repositories as a catalog of recent skills and provided as is. Agent Native does not write or control them and is not responsible for any harm, loss or damage caused by installing or using them. Labels and checks describe what we saw at one commit and are not a guarantee. Read a skill's files before you run it. Terms of use