vulnerability-validation

---
name: vulnerability-validation
description: Validate security findings from commit-security-scan by assessing exploitability, filtering false positives, and generating proof-of-concept exploits. Use after running commit-security-scan to confirm vulnerabilities.
version: 1.0.0
tags: [security, validation, exploitability]
---

# Vulnerability Validation

Validate security findings by assessing whether they are actually exploitable in the context of this codebase. This skill filters false positives, confirms real vulnerabilities, and generates proof-of-concept exploits.

## When to Use This Skill

- **After commit-security-scan** - Validate findings before creating issues or blocking PRs
- **HIGH/CRITICAL findings** - Prioritize validation of severe findings
- **Before patching** - Confirm vulnerability is real before investing in fixes
- **Security review** - Deep-dive validation of specific findings

## Prerequisites

- `.factory/threat-model.md` must exist (from `threat-model-generation` skill)
- `security-findings.json` must exist (from `commit-security-scan` skill)

## Inputs

| Input           | Description                                      | Required | Default                    |
| --------------- | ------------------------------------------------ | -------- | -------------------------- |
| Findings file   | Path to `security-findings.json`                 | Yes      | `security-findings.json`   |
| Threat model    | Path to threat model                             | No       | `.factory/threat-model.md` |
| Finding IDs     | Specific findings to validate (comma-separated)  | No       | All findings               |
| Severity filter | Only validate findings at or above this severity | No       | All severities             |

## Instructions

Follow these steps for each finding to validate:

### Step 1: Load Context

1. Read `security-findings.json` from `commit-security-scan`
2. Read `.factory/threat-model.md` for system context
3. Identify which findings to validate based on inputs

### Step 2: Reachability Analysis

For each finding, determine if the vulnerable code is reachable:

1. **Trace entry points**

   - Can external users reach this code path?
   - What HTTP endpoints, CLI commands, or event handlers lead here?
   - Is authentication required to reach this code?

2. **Map the call chain**

   - Starting from the entry point, trace the path to the vulnerable code
   - Document each function call in the chain
   - Note any branching conditions that must be satisfied

3. **Classify reachability**
   - `EXTERNAL` - Reachable from unauthenticated external input
   - `AUTHENTICATED` - Requires valid user session
   - `INTERNAL` - Only reachable from internal services
   - `UNREACHABLE` - Dead code or blocked by conditions

### Step 3: Control Flow Analysis

Determine if an attacker can control the vulnerable input:

1. **Identify the source**

   - Where does the tainted data originate?
   - HTTP parameter, file upload, database query, environment variable?

2. **Trace data flow**

   - Follow the data from source to sink (vulnerable function)
   - Document each transformation or validation step
   - Note any sanitization, encoding, or type conversion

3. **Assess attacker control**
   - Can the attacker fully control the input?
   - Are there length limits, character restrictions, or format validation?
   - Does the data pass through any sanitization?

### Step 4: Mitigation Assessment

Check if existing security controls prevent exploitation:

1. **Input validation**

   - Is the input validated before reaching the vulnerable code?
   - What validation rules are applied?

2. **Framework protections**

   - Does the framework provide automatic protection? (e.g., ORM parameterization, React XSS escaping)
   - Is the protection enabled and properly configured?

3. **Security middleware**

   - Are there WAF rules, rate limiting, or other controls?
   - Do CSP headers or other browser protections apply?

4. **Reference threat model**
   - Check the "Existing Mitigations" section for this threat type
   - Verify mitigations are actually in place

### Step 5: Exploitability Assessment

Determine how difficult it is to exploit:

| Rating            | Criteria                                                            |
| ----------------- | ------------------------------------------------------------------- |
| `EASY`            | No special conditions, standard tools, publicly known technique     |
| `MEDIUM`          | Requires specific conditions, timing, or chained vulnerabilities    |
| `HARD`            | Requires insider knowledge, rare conditions, or advanced techniques |
| `NOT_EXPLOITABLE` | Theoretical vulnerability but not practically exploitable           |

Consider:

- Attack complexity
- Required privileges
- User interaction needed
- Scope of impact

### Step 6: Generate Proof-of-Concept

For confirmed vulnerabilities, create a proof-of-concept:

1. **Craft exploit payload**

   - Create a minimal payload that demonstrates the vulnerability
   - Use benign payloads (no actual damage)

2. **Document the request**

   - HTTP method, URL, headers, body
   - Or CLI command, file input, etc.

3. **Describe expected vs actual behavior**
   - What should happen (secure behavior)
   - What actually happens (vulnerable behavior)

Example PoC structure:

```json
{
  "payload": "' OR '1'='1",
  "request": "GET /api/users?search=' OR '1'='1",
  "expected_behavior": "Returns users matching search term",
  "actual_behavior": "Returns all users due to SQL injection"
}
```

### Step 7: Calculate CVSS Score

Assign a CVSS 3.1 score based on:

| Metric                   | Options                                            |
| ------------------------ | -------------------------------------------------- |
| Attack Vector (AV)       | Network (N), Adjacent (A), Local (L), Physical (P) |
| Attack Complexity (AC)   | Low (L), High (H)                                  |
| Privileges Required (PR) | None (N), Low (L), High (H)                        |
| User Interaction (UI)    | None (N), Required (R)                             |
| Scope (S)                | Unchanged (U), Changed (C)                         |
| Confidentiality (C)      | None (N), Low (L), High (H)                        |
| Integrity (I)            | None (N), Low (L), High (H)                        |
| Availability (A)         | None (N), Low (L), High (H)                        |

Example: `CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N` = 9.1 (Critical)

### Step 8: Classify Finding

Based on analysis, classify each finding:

| Status                | Meaning                                        |
| --------------------- | ---------------------------------------------- |
| `CONFIRMED`           | Vulnerability is real and exploitable          |
| `LIKELY`              | Probably exploitable but couldn't fully verify |
| `FALSE_POSITIVE`      | Not actually a vulnerability (document why)    |
| `NEEDS_MANUAL_REVIEW` | Requires human security expert review          |

### Step 9: Generate Output

Create `validated-findings.json`:

```json
{
  "validation_id": "val-<timestamp>",
  "validation_date": "<ISO timestamp>",
  "scan_id": "<from security-findings.json>",
  "threat_model_version": "<from threat-model.md>",
  "validated_findings": [
    {
      "id": "VULN-001",
      "status": "CONFIRMED",
      "original_severity": "HIGH",
      "validated_severity": "HIGH",
      "exploitability": "EASY",
      "reachability": "EXTERNAL",
      "existing_mitigations": [],
      "exploitation_path": [
        "User submits search query via GET /api/users?search=<payload>",
        "Express router passes query to searchUsers() handler",
        "Handler passes unsanitized input to SQL template literal",
        "PostgreSQL executes malicious SQL"
      ],
      "proof_of_concept": {
        "payload": "' OR '1'='1",
        "request": "GET /api/users?search=' OR '1'='1",
        "expected_behavior": "Returns users matching search term",
        "actual_behavior": "Returns all users due to SQL injection"
      },
      "cvss_vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
      "cvss_score": 9.1,
      "validation_notes": "Confirmed via code tracing. No input validation or parameterization."
    }
  ],
  "false_positives": [
    {
      "id": "VULN-003",
      "original_severity": "MEDIUM",
      "reason": "Input is validated by Joi schema in middleware before reaching this code. Schema enforces UUID format which prevents injection.",
      "evidence": "See src/middleware/validation.js:45 - Joi.string().uuid()"
    }
  ],
  "needs_manual_review": [
    {
      "id": "VULN-005",
      "original_severity": "HIGH",
      "reason": "Complex data flow through message queue. Unable to fully trace if sanitization occurs in consumer service."
    }
  ],
  "summary": {
    "total_analyzed": 10,
    "confirmed": 5,
    "likely": 2,
    "false_positives": 2,
    "needs_manual_review": 1,
    "by_severity": {
      "CRITICAL": 1,
      "HIGH": 3,
      "MEDIUM": 1,
      "LOW": 0
    }
  }
}
```

## Success Criteria

The skill is complete when:

- [ ] All specified findings have been analyzed
- [ ] Each finding has a status (CONFIRMED, LIKELY, FALSE_POSITIVE, NEEDS_MANUAL_REVIEW)
- [ ] Confirmed findings have exploitation paths documented
- [ ] Confirmed findings have proof-of-concept exploits
- [ ] False positives have clear reasoning
- [ ] `validated-findings.json` is valid JSON
- [ ] CVSS scores are calculated for confirmed findings

## Verification

Run these checks before completing:

```bash
# Verify output exists and is valid JSON
cat validated-findings.json | jq . > /dev/null && echo "✓ Valid JSON"

# Check all findings have status
jq '.validated_findings | all(.status)' validated-findings.json

# Check confirmed findings have PoC
jq '.validated_findings | map(select(.status == "CONFIRMED")) | all(.proof_of_concept)' validated-findings.json

# Check false positives have reasoning
jq '.false_positives | all(.reason)' validated-findings.json
```

## Example Invocations

**Validate all findings:**

```
Validate the security findings from the last scan.
```

**Validate specific findings:**

```
Validate findings VULN-001 and VULN-002 from security-findings.json.
```

**Validate only HIGH/CRITICAL:**

```
Validate all HIGH and CRITICAL severity findings from the security scan.
```

**Validate with specific files:**

```
Validate findings in security-findings.json using threat model at .factory/threat-model.md.
```

## References

- CVSS 3.1 Calculator: https://www.first.org/cvss/calculator/3.1
- OWASP Testing Guide: https://owasp.org/www-project-web-security-testing-guide/
- Examples: `validation-examples.md` (in this skill directory)
- Upstream: `commit-security-scan` skill
- Downstream: `security-patch-generation` skill

From factory-ai/skills at 21a74de, shown as plain text under its license: MIT (permissive). The license text is in LICENSE.

• SKILL.md11 KB · 5d26d59e1909
• validation-examples.md14 KB · 86f6f384f0a6

2 files, 25 KB in total. Each file has a sha256 in the pin's manifest; the first 12 characters are shown.