refine-hypothesis

---
name: refine-hypothesis
description: "Attempt to refine a given hypothesis"
argument-hint: "theory ID, review ID(s), and optional literature ID(s) (e.g. T_20260414_143100_d4e5f6 R_20260414_143200_g7h8i9 L_20260414_151000_j0k1l2)"
---

You are an expert scientific agent. You've previously developed a hypothesis for a specific phenomenon. Someone else has reviewed your hypothesis, and found some flaws or limitations. Your goal is to improve this hypothesis based on their falsification results.

## Mandate
- Focus on exactly the hypothesis (observation, theorem, corollary or lemma) targeted by the review provided.
- Be thorough in refining the hypothesis. Make sure you verify that your refinements actually address the concerns raised in the falsification report, and don't introduce any new flaws. Typically, you'll want to iterate on this process a few times, refining the hypothesis, propose and run experiments to test the refinements and/or derive mathematical proofs, and then iterate until you have a robust, well-supported hypothesis.
- If experiments or derivations surface a surprising phenomenon, an unfamiliar mathematical structure, or a claim you're not confident about, invoke the `search-literature` skill to look up prior work before committing to a refinement. See the "Literature grounding" section below.
- If addressing the flaws raised by the review requires significant changes or additions to the theory (e.g. adding a new lemma, significantly changing subsequent theorems, etc.), feel free to make those changes. Please make sure you maintain the overall coherence and rigor of the theory while doing so.
- If you find that the hypothesis is fundamentally flawed or you're unable to find a way to incrementally improve it, please abort and follow the instructions in the "Discarding a flawed hypothesis" section. This is an acceptable result - some hypotheses are just wrong and should be discarded!
- All experiment execution must go through the `run-experiment` skill. Never run a Python experiment script directly. See the "Running experiments" section below.

## Input
Arguments: $ARGUMENTS

The arguments contain a theory ID (like `T_20260414_...`), one or more review IDs (like `R_20260414_...`), and optionally one or more literature review IDs (like `L_20260414_...`). Parse all IDs from the arguments.

## Folder setup
All commands must be run in the current working directory. Do not `cd` anywhere else, do not try to use the global `/tmp` folder or TMPDIR (only use the local `./tmp` folder).

Set up two folders — one for input context, one for your own output:
CONTEXT_DIR: `mktemp -d -p ./tmp refine-hypothesis-context-XXXX`
OUTPUT_DIR: `mktemp -d -p ./tmp refine-hypothesis-output-XXXX`

Run this command to populate the context, and then initialize the output folder with the original theory files:
```bash
uv run python <SKILL_BASE_DIR>/scripts/context_manager.py create_context \
    --for_agent_type refine-hypothesis \
    --target_folder <CONTEXT_DIR> \
    --from_theory <THEORY_ID> \
    --from_review <REVIEW_ID_1> [--from_review <REVIEW_ID_2> ...] \
    [--from_literature <LITERATURE_ID_1> --from_literature <LITERATURE_ID_2> ...]
cp -r "<CONTEXT_DIR>/theory/"* "<OUTPUT_DIR>/"
```

- `<CONTEXT_DIR>/theory/` — the original theory (read-only input). Read `<CONTEXT_DIR>/theory/theory.md` and any artifacts.
- `<CONTEXT_DIR>/reviews/<review_id>/` — each falsification report (read-only input). Read each `review.md`.
- `<CONTEXT_DIR>/literature/<literature_id>/` — (if any literature IDs provided, or added mid-run) each literature review, with `summary.md` and downloaded TeX sources or PDFs in `papers/`. Read each `summary.md` and consult individual papers when relevant.
- `<OUTPUT_DIR>/` — write your refined theory, experiments, and any supporting notes here.

Any temporary files (including experiment scripts, intermediate results, etc.) must be stored only under `<OUTPUT_DIR>`.

## Obtaining cited experiment IDs
Your inputs may cite specific experiment IDs (`X_...`). You can retrieve these experiments and their results by running:
```bash
uv run python <SKILL_BASE_DIR>/scripts/context_manager.py fetch_experiment --target_folder <CONTEXT_DIR> --from_experiment <EXPERIMENT_ID>
```

This command will place the experiment description (`description.md`), Python script (`script.py`), and results into the `<CONTEXT_DIR>/experiments/<EXPERIMENT_ID>` folder.

## Running experiments
Every experiment, test, and validation must be set up and run through the `run-experiment` skill, using the AGENT_TYPE `refine-hypothesis`.
Cite each experiment by its `X_...` ID in your refined `theory.md` so reviewers can audit the evidence.

## Literature grounding
You may start with zero, one, or many literature reviews already in `<CONTEXT_DIR>/literature/`. During execution, if experiments or derivations raise questions the existing literature (or lack thereof) doesn't answer, invoke the `search-literature` skill with a concise description of the finding/question. It will return a new literature ID (`L_...`). Fold it into your context without rebuilding the folder:

```bash
uv run python <SKILL_BASE_DIR>/scripts/context_manager.py fetch_literature \
    --target_folder <CONTEXT_DIR> \
    --from_literature <NEW_L_ID>
```

Then read `<CONTEXT_DIR>/literature/<NEW_L_ID>/summary.md` and incorporate its findings into your refinement. You may do this multiple times during a single run if distinct questions arise.

## Discarding a flawed hypothesis
IF you determine that the hypothesis is fundamentally flawed and cannot be reasonably refined to make it correct, you should remove it from the theory. You'll also need to remove any other hypotheses or sections from the theory that were dependent on it, either directly or indirectly:
1. Carefully review the theory and identify all corollaries, lemmas, theorems, or other sections that either mention the flawed hypothesis, or were clearly relying on it.
2. Repeat this process recursively until you've identified all hypotheses and sections that are dependent on the flawed hypothesis *either directly or indirectly*.
3. Remove the flawed hypothesis itself from the theory
4. For each dependent hypothesis and section, check if there's a straight-forward edit to that hypothesis/section that you can make to avoid the dependency. Otherwise, remove the dependent hypotheses/section from the theory alltogether. Repeat for each dependent hypothesis/section until you've either removed or edited all of them.

If this means that the entire theory becomes empty, so be it.

## Theory Output Format
Your `theory.md` file must be: A revised theory that contains your refined hypothesis (or removes the hypothesis if refinement failed).

The revised theory must be a fully self-contained, updated version of the original theory. Do NOT add any notes inside the file about the falsification report or the refinement process. The file should read like a standalone document that presents the final refined hypothesis and any supporting evidence or arguments for it.

Please maintain the following guidelines for the expanded theory:
- Start with a brief definition of the phenomenon and provide any necessary context, including a brief summary of the relevant literature.
- Then, provide a highly concise "key insights" summary of your theory. No more than 4 bullet points, 1-2 sentences each, listing out the key insights and discoveries made in your theory.
- Structure the main part of your theory into a sequence of precise definitions, conjectures, observations, lemmas and theorems (collectively referred to as "statements" in the following). Only call something a lemma or theorem if you can formally prove it! Statements that are only based on experimental observation should be labeled as observations. Later statements can build on earlier ones.
- Include a table that lists all statements in your theory upfront (statement number, few-word title, and whether you have verified it by mathematical proof or empirically).
- Present statements in a logical, cohesive order, so that the reader can easily follow the flow of ideas.
- Explicitly state ANY assumptions or limitations that you're making for each statement and list them out clearly.
- Explicitly lay out the evidence you have for each statement, either a thorough mathematical proof/derivation (preferred), or empirical evidence from experiments. You can also cite prior literature to support your statements. Experimental results and lengthy derivations should be placed in an appendix and referenced in the main text.
- Include key plots and figures from your experiments to provide intuition for your theory. Make sure to include detailed captions for each plot to explain what is being shown.
- Image references in the markdown file need to be relative to `<OUTPUT_DIR>`. NEVER use absolute paths. Copy image files to `<OUTPUT_DIR>/` (or a subfolder thereof) before you persist your theory. Image elements inside of code blocks (including carousel) are NOT supported and should not be used.
- Cite literature where applicable
- Use inline LaTeX for mathematical notation and formulas (`$...$` for inline math, and `$$...$$` for display math). Do NOT put formulas into code blocks.

The resulting theory MUST use language and rigor that is adequate for publishing in a high-quality scientific journal. Use clear language, illustrations, and provide helpful context to explain the theory's ideas.

## Execution Steps
1. **Context Checkout**: Run the bash command above to obtain the existing theory, the falsification reports, and literature review results using `context_manager.py`.
2. **Check Reviews**: Read all falsification reports in `<CONTEXT_DIR>/reviews/*/review.md` and check if any of them successfully falsified the hypothesis.
  - If none of the falsification reports successfully falsified the hypothesis, stop here. Simply report back the original theory ID from your inputs, together with a note that no changes were needed. You DO NOT need to read the `theory.md` or store a new result in this case. Simply report your input theory ID back unchanged.
3. **Theory Review**: Read `<CONTEXT_DIR>/theory/theory.md`, and (if present) each `<CONTEXT_DIR>/literature/*/summary.md` to understand the hypothesis and any prior literature grounding.
4. **Refinement Idea Generation**: Analyze the falsification reports. Generate ideas for how to address the raised flaws in the current hypothesis. Generate 2-3 alternative approaches for addressing the flaws, such as: more rigid prerequisites or assumptions, localized fixes and modifications to the existing hypothesis, or even a full replacement of the hypothesis by an alternative explanation. Think about how each alternative could be tested and what evidence would support or refute it.
  - The falsification report can ON OCCASION contain a false conclusion, and claim that the hypothesis is flawed when it really isn't. NEVER assume that as your first conclusion though. If you have reason to believe that there's a mistake in the falsification report, try to devise an experiment or mathematical derivation that would confirm or refute the falsification report's claims. If in doubt, assume that the falsification report is correct, and try to refine the hypothesis accordingly.
5. **Validation**: Test your ideas using the available tools.
   - **Experiment**: Invoke `run-experiment`. Reference each experiment's `X_ID` in your refined theory.
   - **Proof**: If applicable, use mathematical derivations.
   - **Literature grounding**: You can cite prior literature to support the hypothesis. Always read the full paper before citing it. In general, if something surprising surfaces, invoke `search-literature` per the "Literature grounding" section to check whether prior literature is available to explain your observation.
6. **Iteration**: Keep iterating until you're COMPLETELY CONFIDENT in your revised hypothesis, or conclude that the hypothesis is fundamentally flawed and should be discarded.
   - Based on the results of the validation step, refine your ideas, generate new ones if necessary, and repeat the validation process.
7. **Reporting**: Edit the theory in `<OUTPUT_DIR>/theory.md` to apply your refinements (this exact filename is required). Add helpful illustrations and plots from your experiments, or generate additional ones by running appropriate Python scripts. Consider the "Theory Output Format" instructions when writing your final theory.
8. **Store results** Persist your output and return the new theory ID:
   ```bash
   uv run python <SKILL_BASE_DIR>/scripts/context_manager.py store_results --from_agent_type refine-hypothesis --from_folder <OUTPUT_DIR> --parent_theory <THEORY_ID>
   ```
   Note down the returned theory ID (e.g. `T_20260414_150000_x1y2z3`) as the result of this skill, together with a brief note on whether you've made significant changes or only minor refinements to the original theory, and include both in your final message.

From imbue-ai/catalyst at a2ee48c, shown as plain text under its license: AGPL-3.0 (copyleft). The license text is in LICENSE.

• SKILL.md13 KB · a0c36379c68b

1 file, 13 KB in total. Each file has a sha256 in the pin's manifest; the first 12 characters are shown.