Adversary
An open-source library that turns one schema into explained adversarial test inputs - boundary values, i18n, and injection, each with a plain-language reason it might break.
- TypeScript
- Zod
- OpenAPI
- Open Source
- QA
The CLI turning one Zod schema into explained adversarial inputs - each value labelled with its technique and reason.
Turning an OpenAPI spec into a per-endpoint hostile-input report - no Zod schema, from the spec alone.
Applied to a real FastAPI project (frameboard) - its Pydantic model exported via model_json_schema() and fed straight to adversary.
Setup
- Problem
Most people writing input-validation code can't recall every boundary, Unicode, and injection case by hand. After you write
z.string().min(3), building a table of the hostile inputs that field will actually accept - a value one character short of 3, a family emoji that is 11 UTF-16 code units, a URL starting withjavascript:- is a separate job, and usually it doesn't get done. That knowledge tends to live only in the head of someone who has done i18n, security, or QA for a long time. So validation creates the illusion that "it passed, so it's safe," while the genuinely dangerous values slip straight through the schema.- Context
The two things I reached for most in fourteen years of QA were boundary-value analysis (BVA), equivalence partitioning (EP), and the question "what will this field actually accept?" If that judgement could be pulled from the schema automatically instead of redone by hand each time, a developer without QA background could lay the same line of defense into CI. Generating nothing with AI was the load-bearing design constraint - a plausible-but-wrong failure hypothesis collapses trust in the whole tool. So every explanation is hand-curated, and each fact it rests on ('ß'.toUpperCase() is 'SS', z.url() really does accept javascript:) carries a test that asserts it. A wrong explanation breaks CI.
- Users
Developers validating untrusted input (a form body, an API request, a webhook, a config file) with Zod, and QA engineers who have an OpenAPI spec but no Zod schema.
- Hypothesis
Pull the boundary / i18n / injection catalog from the schema automatically and attach to each value "which technique produced it and why it might break" - and you can lay adversarial-input tests into CI in one line, without being an i18n or security expert.
Build
- What I did
- Generate explained adversarial inputs from a schema - each value labelled with its technique (BVA / EP / i18n / injection), family, failureHypothesis, and validity
- Four-technique catalog - boundary-value analysis, equivalence partitioning, i18n/Unicode (normalization, grapheme vs code unit, bidi, homoglyph), and injection (SQL, XSS, SSTI, CSV formula, OS command, path traversal, CRLF, NUL)
- Type coverage - string, number/integer, boolean, enum/literal, array, date/date-time, union, each with generators aimed at that type's traps
- Format-aware packs - email, url, uuid, hostname, ipv4, ipv6, base64, with extra inputs aimed at each parser. Several pass their z.*() validator yet stay dangerous
- Three entry points - Zod v4, plain JSON Schema, OpenAPI 3.x. With just a spec you get a per-endpoint report and no code to write
- CLI plus a risk-ranked Markdown report (injection first), published to npm (zero runtime deps, MIT), and a test suite that verifies every failure hypothesis
- Product decisions
- Hand-curated, no AI - runtime generation invents plausible-but-wrong explanations. Deterministic, offline, no API key is the condition for trust
- A data generator, not a scanner - it never runs your code or declares a bug; it hands you hostile inputs as a plain array and leaves the verdict to your assertions
- OpenAPI-first entry point - QA with no Zod schema still get a per-endpoint hostile-input report from the spec alone
- A verifying test behind every failure hypothesis - a wrong or overclaimed explanation breaks CI, so trust holds as the catalog grows
- QA considerations
- Does it label correctly whether a generated value satisfies the field's own constraints (validity: valid / invalid / unknown) - i18n and injection are left "unknown" honestly, because whether they're accepted is exactly the behaviour under test
- Is the fact each failureHypothesis rests on actually true - 217 tests pin facts one by one ('ß'.toUpperCase() is 'SS', the family emoji is 11 code units, z.url() really accepts javascript:)
- Do values JSON can't represent (absent, NaN, Infinity, -0) silently vanish from output - encoded as literal strings / null so nothing is lost
- Do invisible or bidi characters corrupt the Markdown report - non-ASCII is escaped to \uXXXX
- Is the public API (adversary, fromJsonSchema, fromOpenApi, toMarkdown, catalog, packs, Fixture) frozen under semver from 1.0, so only the catalog grows and nothing regresses
Outcome
- Metrics
Live on npm as adversary@1.1.0 (MIT, zero runtime deps). 217 tests across 13 files, CI green. Three entry points (Zod v4, JSON Schema, OpenAPI 3.x) and seven format packs.
- Result / Learning
The clearest outcome was proving "validation is not safety" as a tool. Feed z.url() a javascript:, a file://, or a cloud-metadata SSRF address and every one passes safeParse - showing, in front of you, that a validated value can still be dangerous carries the point that the code behind the field is the real line of defense, no explanation needed. What porting QA knowledge into code taught me is that the "why it breaks" is far harder, and far more valuable, than the list of hostile inputs. Values can be copied; reasons have to be verified - and pinning that verification as tests is exactly where this differs from a naughty-strings list.
- Retrospective
- I started thinking the core was "generating lots of hostile inputs"; the real value turned out to be the verified explanation attached to each. Values are common; a trustworthy reason is scarce.
- Composite types (nested objects, record, intersection) are still handled only as top-level presence probes - descending into deep nesting is left on the roadmap, and the current coverage is marked honestly.
- Tech stack
- TypeScript
- Zod v4
- JSON Schema
- OpenAPI 3.x
- vitest
- tsup
- npm