ksi-harness

Continuous control monitoring for FedRAMP 20x. It pins FedRAMP’s own machine-readable rules as the source of truth, collects evidence with population reconciliation, and will not mark an indicator automated until someone writes why the checks leave nothing material out. The headline number is currently zero. That is the point.

Run it in your browser — no install → A hosted demo runs the real monitoring pipeline against fixtures. See every KSI evaluated with full-population evidence, the hash chain verified, and 20x artifacts emitted — no credentials, nothing touching a real cloud.

What it does

A conventional tool would take 29 implemented checks and 25 indicators with passing fixture evidence and print a coverage percentage north of 50%. This one reports:

FedRAMP Consolidated Rules  ·  Class C
46 applicable indicators of 46 in the ruleset

automated      0
partial       25
manual        14
unaddressed    7

An indicator reaches automated only when a written sufficiency argument exists — and that argument is a property of a boundary, not of the indicator. The same routing map answers three different true numbers against three profiles. With no profile, the condition cannot be evaluated, so the headline stays zero.

Collectors reconcile populations before they grade. An incomplete population can never be a pass. Emitters read only control state and write schema-valid 20x artefacts (SDR, OCR, SCN, overview) plus an OSCAL projection for Rev5 customers. Evidence is hash-chained; the chain head is meant to live in a signed manifest and an anchor log in a different repository.

When and where to use it

Use it when

  • You are on FedRAMP 20x (Class C is the ceiling; Class D does not exist yet) and need continuous monitoring that FedRAMP’s own rules can invalidate.
  • You are tired of a collector that paginated badly and still reported “all users have MFA.”
  • You need preventive (Rego / conftest on IaC) and detective (collectors on deployed state) paired, with the gate’s own result folded back in as evidence.
  • You want a GRC engineer to practice writing the gap, not hiding it behind a percentage.

Do not use it when

  • You need CMMC / 800-171 Rev 2. That is cui-control-plane. CMMC is out of scope here on purpose.
  • You need a 3PAO authorization. This produces schema-valid artefacts. It does not produce an authorization, and it will not write ksiAssessment.
  • You want High / Rev5-only OSCAL as the system of record. 20x does not use OSCAL. An OSCAL emitter ships beside the 20x formats because customers still ask.

How to use it

Node 22+. The demo needs no credentials and contacts no real system.

0. Run the whole path against fixtures.

git clone https://github.com/RootCawsLLC/ksi-harness.git
cd ksi-harness
npm install
npm run demo

That verifies the ruleset pin, prints the Class C catalog, explains one indicator, validates the routing map, collects twice (so a chain exists), verifies hashes, diffs the two runs, prints the coverage report, and emits SDR, OCR, SCN, OSCAL assessment results, and a Markdown coverage file. Every fixture bundle is marked as fixture-derived. The interesting output is the coverage report, not the SDR.

1. Learn the catalog before you collect anything real.

npx ksi catalog --class c
npx ksi explain KSI-IAM-APM
npx ksi checks
npx ksi routes validate
npx ksi routes baseline --out routes.new.yaml

routes baseline is the first command on a new boundary. This repository’s own routes.yaml describes these collectors in this environment. Cloning it and editing the profile inherits those claims, and they are not true elsewhere. A baseline is 46 unaddressed: a program on its first day. The validate warnings are the backlog.

2. Declare the boundary. Do not discover it. The profile names accounts, projects, repositories, capabilities, third parties. If collection enumerated whatever it could see, a new account would join the authorization boundary without anyone deciding it should.

3. Collect, report, emit.

npx ksi collect --profile examples/northwind.profile.yaml --fixture fixtures/collectors --out .evidence
npx ksi coverage --evidence .evidence --md out/coverage.md
npx ksi verify --evidence .evidence --manifest .evidence/MANIFEST.json
npx ksi emit sdr --profile examples/northwind.profile.yaml --evidence .evidence --overview-uri https://example.invalid/overview.json

For a live AWS/GCP/GitHub boundary, drop --fixture and use the standard credential chains. A multi-account AWS run without aws.collector_role is refused: collecting several accounts on one ambient credential files the same account’s evidence under every id in the boundary.

4. Alert on transition, not on state. ksi notify names what moved. A control that has been failing for forty days is one piece of news and thirty-nine reasons to mute the channel.

Take it into your organization

  1. Pin the ruleset. Do not restate FedRAMP indicator text in your own files. vendor/fedramp/ is hashed. npm run drift tells you when upstream moved and which routes it invalidates.
  2. Write the gap. partial requires unautomated. automated requires a sufficiency argument and the boundary it holds for. If you cannot write the argument, the honest level is partial.
  3. Reconcile the population before you grade. expected comes from an enumeration made before grading, named in enumerated_from. A permission gap and a clean environment must not produce the same green tick.
  4. Keep the locker. A scheduled workflow that checks out fresh, collects into a gitignored directory, and uploads an artefact nobody reads back will report cadence_unmet: 0 forever. Restore the locker, verify it, then collect.
  5. Put the anchor somewhere you do not control. A hash beside its own data proves very little. ksi-anchors is the worked example: a different repository, a token that can write only that repository, branch protection that blocks force-push.
  6. Do not commit real evidence to the harness repo. A bundle is an inventory of where the boundary is weakest. Fixture runs are how you learn; evidence.store is how you keep the real thing.

Adding a collector: pure grade() separate from fetch, a fixture under fixtures/collectors/, a route that claims it. A check no route claims is evidence collected for nothing. See AGENTS.md.

What the fixture run shows

This page is not a hosted control plane — the tool is a CLI. The live part is that you can reproduce the numbers below on a laptop with no account.

0 automated
Twenty-five indicators have real passing fixture evidence. None has a sufficiency argument that holds for the default (no-profile) view.
Same map, three boundaries, three answers
examples/skylark.profile.yaml can resolve automated 1. northwind stays 0 because AWS is in scope and the gap is shown. No profile: 0, unresolved.
Two collections, one chain
The demo collects twice on purpose. One collection is a collection; two are a cadence. Edit a bundle and ksi verify names the break.
This repository is one of its own subjects
Against examples/self.profile.yaml, branch-protection can pass and pr-review can fail — settings say what is supposed to happen; commit history says what did. The failure is left standing.

What it is not

Not a certification. Not complete — unaddressed indicators each have a stated reason and a named next step. Not a substitute for judgment. Lula 2 said it: automated tests alone were insufficient for real compliance verification. That cuts against this project too. Completeness of a population is the invariant that matters; a hash stored beside its own data is not tamper detection. The full argument is the README and eleven ADRs.