OpenGDD

The validator, and what a pass means

Conformance means meeting the OpenGDD specification's requirements. OpenGDD's validator checks the requirements that a program can check in a package. A package is one folder that contains a game's design. A person reviews the written design.

Run the checks

The validator runs inside OpenGDD's authoring tool as you write. You do not need to install anything.

Run the validator from the command line

With Node.js installed, replace <package-dir> with the path to the package folder:

npx opengdd validate <package-dir>

For an automated check, pin this release: npx opengdd@0.9 validate <package-dir>.

To check a build record, name the record and, if you have it, the package it was built from. --render-contract-tests prints the tests that the package's checked contract adoptions imply, as Markdown, without changing any file.

npx opengdd validate --build <opengdd-build.json> [<package-dir>]
npx opengdd validate --render-contract-tests <package-dir>

Exit codes

For a package check:

  • 0: the report has no errors. Warnings, hints, and safety notices never fail a run.
  • 1: the report has one or more conformance errors.
  • 2: the command has a usage error, such as an unknown option or a missing argument.

For a build-record check, 0 means conforming. No other outcome exits with 0. 1 means invalid, 2 means a command usage error, and 3 means incomplete, not verified, or not checked because no package was supplied.

JSON report

validate --json writes one JSON object to standard output for both passing and failing validation runs. It has these fields:

  • validator names the validation kind; format is the OpenGDD version checked, currently "0.9"; and validator_version is the npm package version that produced the report.
  • package, or build in build-record mode, contains id and path. A package id is null when the package directory could not be read.
  • valid is a Boolean, with one exception. It is null when a build record was checked without its package and has no errors of its own. A build record with errors has valid set to false, also when it was checked without its package. Package verdicts are PASS, PASS WITH WARNINGS, or FAIL. A build verdict is PASS or PASS WITH WARNINGS only when the outcome is conforming. Otherwise the build verdict is INCOMPLETE, NOT VERIFIED, NOT CHECKED, or FAIL.
  • In build-record mode, outcome is conforming, incomplete, not verified, invalid, or not checked.
  • summary contains the numeric counts errors, dependent, warnings, hints, safety, and findings.
  • findings is an array. Every finding has code, severity, spec_section, file, and message. line and check-specific data appear when known. dependent: true appears on an error that is waiting for required designer input.
  • hints and safety are separate arrays with the same location and message fields. All three arrays are present even when empty. Hints appear only when --review is requested; the safety scan runs by default. Neither array changes the verdict or warning count.

validate --review <package-dir> asks for optional review hints. The hint checks work only on English text. Three checks of the safety scan also read only English, so these checks do not find a hostile instruction in another language. A scan with no findings does not prove that a package is safe.

The source code of these checks is in the specification repository. It needs only Node.js and no other software. Every finding has a stable code, such as PROSE_CITATION_DANGLING. This code names the rule. The finding also names the specification section of that rule.

What the validator checks

The validator checks the files of the package and the connections between them:

  • Required files and fields are present.
  • References in the writing point to defined names.
  • Numbers fall within their declared limits and satisfy the rules in tuning.json.
  • Data files follow their required structure.
  • Acceptance tests, the checks a finished game must pass, contain the required information.
More detail by part of the package
  • the package: required files present, every declared path inside the package, the manifest valid against its schema;
  • the fantasy block: at least one fantasy line, no more than 280 characters across the lines, three to five feel entries, and a non-empty NOT: line;
  • values: finite shared numbers under declared keys;
  • ranges: inclusive limits that contain their current values;
  • rules: one comparison between two calculations, with resolvable keys and finite results;
  • personalization: question and answer shapes. Every sets entry targets a ranged values number or an open number, and stays inside the range when the number has one. A record field or a contract value is never a target;
  • collections: collection and record ids are well formed, every record is checked against its schema where one is declared, and every cited collection or record exists;
  • links: linked records exist, forbidden loops are absent, and declared back-pointers agree in both directions;
  • runtime values: each address in the unchanged.values list of a test has a declaration in a chapter or a clock;
  • clocks: every clock covers every time mode with one of the four behavior words, and supports any test claim that named runtime values stay unchanged;
  • acceptance tests: ordered headings, one structured test block per heading, and the fields that block requires;
  • art direction: palette and direction shape, dotted references, coverage of every measured promise, and declared WCAG 2.1 contrast floors;
  • contracts: the answers, numbers, rows and rules of each adoption agree with its contract, and the acceptance tests stored with the adoption match that contract.

Zero errors means the package passed the validator's checks. A warning marks something to read again. A warning does not fail validation. Passing validation does not prove that the design is good, or that a finished game follows it.

Build records

The builder is the person, team or AI that turns the package into a game. A build record is the builder's report of that game, stored in opengdd-build.json. It identifies the package, the choices made for that build, the final numbers and the test results.

If you also have the package, the validator checks that the record agrees with it. The final numbers include the builder's choices for ranged and open numbers. The test counts must add up, and every test that did not pass must be listed with failed, partial, or not-run and a reason. These checks read the record. They do not run the game or prove that the record is true.

The validator reports conforming only when the package has tests and all of them passed. A record that correctly lists the tests that did not pass is incomplete. A record for a package with no tests is not verified. Tests the builder added for their own process do not change these outcomes.

A record that has errors is invalid, also when it is checked without its package. Without its package, a record that has no errors of its own is not checked, because the package-dependent checks did not run. From the record alone, the validator still checks the acceptance sum: the number of passed tests plus the number of tests listed as not passed must equal the total.

The exit codes and the JSON verdicts for a build record are listed above, in the command-line part of the section Run the checks.

Auditing a build

An audit examines a finished game and its evidence to check whether it follows the package. OpenGDD's experimental certification protocol describes the process:

  1. Check the package. Run the validator and have a person review the applicable requirements in the writing.
  2. Build and test the game. The builder follows the design, runs every acceptance test the package declares and records the results.
  3. Review the game and evidence. A separate auditor checks whether the tests measure the intended behavior. They also check that the game uses the declared numbers and follows the design.
  4. Verify the report. The auditor checks identities, choices, test results and file hashes. A hash identifies the exact evidence file. The audit records its findings and the limits of the checks made.

The protocol defines experimental verdicts: certify, certify with notes, or do not certify. These are research outcomes. There is no official OpenGDD certification program. These verdicts grant no certification mark.

Limits of the checks

  • Taste. An audit asks whether a build follows its package. It does not decide whether the game is enjoyable. Builds may differ where the designer leaves choices open.
  • Audio. A package may describe audio in prose, but the current format has no checking rules for audio direction.
  • What a player feels or knows. The format can test what is on screen and in the game's state. Those observations do not prove a player's feelings or understanding. The known limitations page explains this limit.