Save as .claude/skills/incident-triage/SKILL.md in a repository, or under ~/.claude/skills/incident-triage/SKILL.md to have it everywhere. The copy button takes the whole file, frontmatter included.

---
name: incident-triage
description: Narrow a website or service outage to a layer, in order, before proposing a cause. Use when a site is down, unreachable, intermittently failing, slow for some users, showing a certificate error, or when someone asks why a domain stopped working.
---

# Incident triage

Your job in this skill is to locate the fault, not to fix it. Resist proposing
a cause until the layer is established.

Go in this order and stop at the first layer that fails. Each layer depends on
the one above, so a failure higher up makes everything below it meaningless.

## Requires

The toolhq MCP server, for `dns_lookup`, `dns_propagation`, `reverse_dns`,
`ip_lookup`, `ip_reputation`, `ssl_check` and `http_headers`:

    claude mcp add --transport http toolhq https://toolhq.io/api/mcp

## Layer 1: does the name resolve

`dns_lookup` for A and AAAA on the exact hostname the user reported, not the
apex, and not www unless that is what they said.

- No records: DNS is the fault. Check whether the zone answers at all and
  whether the records were recently changed.
- Records present but pointing somewhere unexpected: likely a recent change.
  Run `dns_propagation` to see whether resolvers disagree.
- Resolvers disagree: a change is in flight. Report the TTL and the expected
  settle time rather than treating it as broken.

An AAAA record pointing at an address that does not serve the site is a real
and easily missed fault. It breaks the site for IPv6 clients only, which
presents as "down for some people".

## Layer 2: is the address plausible

`ip_lookup` on the resolved address, and `reverse_dns`.

You cannot ping from here, so do not claim the host is up or down. What you can
establish is whether the address belongs to the hosting provider you expect. An
address in an unexpected network after a reported change is a strong signal.

`ip_reputation` is worth a look if the symptom is that some users are blocked
rather than all.

## Layer 3: does TLS complete

`ssl_check` on the host and port.

- Handshake fails entirely: the service is not listening, or not listening with
  TLS. This is the clearest "the host is the problem" signal available without
  shell access.
- Certificate expired: the fault, and the fix is a renewal.
- Chain incomplete: breaks some clients and not browsers, so it explains "works
  for me" reports.
- Hostname mismatch: often means traffic is reaching the wrong service, such as
  a default virtual host or the wrong load balancer target.

## Layer 4: what does HTTP say

`http_headers` on the full URL.

- 5xx: the application is reached and failing. The network is fine. Stop
  blaming DNS.
- 404 on a URL that should exist: routing or a deploy, not infrastructure.
- 401 or 403: access control, possibly a WAF rule or a new edge policy.
- A redirect loop, or a chain that does not terminate: configuration, commonly
  an apex and www rule fighting a platform level redirect.
- 200: the origin is serving. If users still report failure, the fault is
  client side, regional, or in a cache.

## Rules

- **Never report a symptom as a cause.** "The certificate is expired" is a
  cause. "The site shows an error" is not.
- **Say what you could not check.** You have no shell, no logs and no internal
  metrics. Name the gap instead of filling it with a guess.
- **Distinguish all users from some users.** Some users points at DNS
  propagation, IPv6, a regional edge, or an incomplete chain. All users points
  at the origin, the certificate or the record.
- **A recent change is the most likely cause.** Ask what changed before
  theorising. If a record changed, check propagation before anything else.
- **Do not change DNS while diagnosing.** Changes are slow to take effect and
  slow to undo. Establish the layer first.

## Report back

1. The layer that failed, and the layers you cleared.
2. The evidence, with the actual values returned.
3. The most likely cause, with your confidence.
4. The next check, naming who can run it if you cannot.