This page says what is tested, by what, what stops a bad revision from shipping, and what is not evidence. It is deliberately plain about the last point: automated tests are engineering quality evidence; they are not a validated state (GAMP 5 and CSV).
What exists
| Layer | What it is | How many | Run by |
|---|---|---|---|
| Server, unit | Vitest, a database that cannot be reached | 430 files | npm run test:unit |
| Server, integration | Vitest against a real PostgreSQL, a throwaway database with every migration applied from zero | 137 files | npm run test:integration |
| Client | Vitest (components, the BFF, the permission and role tables, the vendored form engine) | 327 files | npm test in the client |
| Control plane, server | Vitest against a throwaway database | 21 files | scripts/ci/verify.sh |
| Control plane, console | Vitest | 9 files | npm test in the console |
| Installation qualification | A script that checks an installed environment and writes evidence | 1 script | npm run validate:iq |
| Load harness | Seeds a study of a stated size and drives virtual users through the real HTTP API | 4 sizes | npm run loadtest:run |
| Manual test sheet | A workbook of test cases written from industry workflows, run by hand through the screens, with evidence | 422 cases and 247 recorded runs on 21 October 2026 | The test team and review sessions |
The counts are the number of test files in the repositories at the builds named above; they change as the code does.
The server's tests
Two Vitest projects share one setup, tests/setup.ts:
- Unit tests run with
DATABASE_URLpointed at an address where nothing listens, and a second guard (tests/support/no-database.ts) records every connection attempt before a socket opens and fails the file that made one. "Unit tests never touch a database" is a property of the run, not a convention. - Integration tests build their database from the migrations (
migrations-from-zerois one of them) and run against the runtime role, so a test that needs a privilege the application lacks fails the way the application would. - A test cannot run against a real database.
tests/support/test-database.tsrefuses any database whose name does not end in_test, and a governance test (test-database-guard) holds the refusal itself in place. Never aim a test run at a development, staging or production database.
Tests that guard the rules themselves
Some tests do not exercise a feature; they fail the build when a promise the documentation makes stops being true:
Test (in tests/unit/governance unless stated) | What it holds |
|---|---|
audit-coverage | Every study module that changes regulated state writes to the audit ledger, directly or through the outbox; a new mutating module with no audit fails the build |
no-hard-delete | No DELETE FROM on a governed table outside a reviewed allowlist |
schema-baseline-current | db/schema.sql records exactly the migrations in db/migrations |
cross-repo-checks | The comparisons with the client and the control plane either ran, or the run opted out explicitly (CDMS_SERVER_ONLY=1); a silent skip is a failure |
test-database-guard, test-environment | A test can only run against a _test database in the test environment |
audit-restore-script-parity | The restore-verification script keeps its own copy of the audit hash (so it can check a dump taken by another version); this test runs both on events that set every field and fails when the digests differ |
least-privilege-contract (integration) | Every table and function the runtime role holds is on the expected list, and nothing else |
rls-tenant-isolation, rls-environment-wall, rls-identity-audit (integration) | A session of one organization sees nothing of another; a session in UAT sees nothing of Live and the reverse; the identity and audit tables answer to their own policies |
audit-immutability, audit-chain-roundtrip, audit-per-chain, audit-anchors, audit-outbox (integration) | The ledger refuses update, delete and truncate; every field the hash covers survives a round trip through the database; there is one gapless chain for each organization, study and workspace; a chain cut short of what was anchored fails verification; a bad envelope is charged alone |
evidence-actually-ran (integration) | The proofs that need a live database with the two-role model (ledger immutability, tenant isolation, the chain round trip) cannot vanish into "skipped": the run fails unless it acknowledges the skip on its command line (CDMS_ALLOW_SKIPPED_EVIDENCE=1) |
Other integration suites cover the study lifecycle (state and workspace rules, version release, go-live, lock and unlock), delegation, signatures, queries, safety events, exports and the dashboards' summary tables. The test names and their files are the index; read tests/integration for the list.
The client's, and the contract between the two
- The client's gate is
tsc --noEmit,eslint --max-warnings=0,vitest run, a production build and a dependency audit. - The typed API client is generated from the server's OpenAPI document (
npm run gen:api), and the permission, role, export-format and screen-permission tables are generated from the server's catalog (npm run gen:permissions). A change on one side that the other has not followed shows up as a type error or a failing parity test, not in production. - The server's tests that compare it with its two siblings run only when the siblings are checked out beside it. The client's own workflow (
.github/workflows/ci.yml) runs with the cross-repo comparisons switched off (CDMS_CLIENT_ONLY=1) and points at a "contracts" job that lived in the server repository; the server no longer has a workflow, so the comparisons run where all three repositories sit side by side (a developer machine, or a build project that checks out all three). See the gaps below.
What stops a revision shipping
| Repository | Gate | What it contains |
|---|---|---|
| Server | npm run verify | vendor:check (the vendored form engine is the one the client uses), typecheck, lint with no warnings, test (both projects). CodeDeploy deploys a commit that passed it; the deploy hooks do not run the tests (Running it) |
| Client | The workflow in the client repository | Typecheck, lint, tests, production build, no high or critical advisories in production dependencies |
| Control plane | server/scripts/ci/verify.sh | Creates a throwaway database, applies every migration from zero, typechecks, runs the suite, builds, checks the build produced the server and bootstrap programs, and fails on a high or critical advisory in production dependencies |
The control plane's gate needs a PostgreSQL server it may create a database on, as a superuser (the platform ledger tests switch session replication on to prove the guards hold), and drops the database again.
Coverage thresholds
The server's coverage run (npm run test:coverage, V8) fails below 70% of statements, branches, functions and lines, counted over src and the scripts, so an untested file counts against the total. This is a floor, not a statement about which behaviors are tested; the governance tests above are what hold the regulated behaviors.
Performance
server/loadtest is the harness of ADR 0017 (decision 18). It builds a database of a stated size, starts the API and the worker as the application's own entry points, and drives virtual users through the real HTTP API with a mix of form saves, lists, dashboards, trail pages and queries, measuring each operation's latency and how long an audit event takes to appear in the trail.
| Size | Subjects | Sites | Item values (at most) |
|---|---|---|---|
| smoke | 12 | 2 | 540 |
| small | 200 | 4 | 1,500,000 |
| medium | 1,000 | 10 | 7,500,000 |
| large | 5,000 | 50 | 37,500,000 |
The reports are in server/docs/perf; the target for a form save is 300 ms at the 95th percentile. The figures that matter:
| Run | Users | Form save p95 | Result |
|---|---|---|---|
| Medium, 5 October | 30 | 175 ms | every target met |
| Large (5,000 subjects, about 36 million audit events), 5 October | 30 | 209 ms | every target met |
| Large, 5 October | 40 | 383 ms | form-save target not met |
| Medium, 13 October | 30 | 122 ms | every target met |
| Medium, 13 October | 40 | 299 ms | every target met, just |
| Medium, 13 October | 60 | 636 ms | form-save target not met; the single machine is saturated |
In every run no request failed and every audit event reached the trail within a second (the slowest, 0.82 s). Said plainly in the harness itself: one machine runs the database, the server, the worker and the harness, so the figures show the shape of the cost as a study grows and not production concurrency, disk or network. Capacity for a given customer is a measurement on the hosting they will use, and a large database takes about 95 minutes to seed.
Installation qualification
npm run validate:iq runs automated installation checks against an installed environment and produces evidence that can be attached to a qualification record. It is the one part of a validation package a machine can answer honestly. No validation package exists in the repository: the validation plan, the user requirements, a risk assessment, a traceability matrix, executed OQ and PQ with a summary report, and the change-control and periodic-review procedures are all missing, and the validation directory says so on its first line. Those belong to the customer's quality system, written against the customer's protocol and configuration (GAMP 5 and CSV).
The manual test sheet
A shared workbook, TRIALION_TEST_SHEET.xlsx, holds test cases written from the way a trial is run (workflows from site start-up to lock and archive, access turnover, tenant isolation, security, resilience, mail and privacy), the runs of them with an evidence id and the build, the defects they found, and demo data. Its rules are in TEST_SHEET_PROTOCOL.md: a case is Pass only with evidence, "Pass with remark" says what was not checked, an automated test is linked and never counted as manual coverage, and runs are only appended. When the first full simulation ended (20 October 2026) it held 416 cases and 177 runs, of which 8 failed and 14 defects were open. After the second run, in which the fixes were retested (21 October 2026), it held 422 cases and 247 runs (139 Pass, 94 Pass with remark, 12 Fail, 2 not applicable) and 21 defects, of which 18 were fixed and retested, one was withdrawn and two were fixed without a retest on screen. The sheet keeps growing, so read the figures from the sheet itself; they are given here only to show the scale.
The sheet is evidence of what was done by hand on one build; it is not a validation record, because a validated state needs a plan, approval and a summary written by qualified staff.
Gaps to plan for
| Gap | Consequence |
|---|---|
| No validation package | A regulated trial needs one, written and executed by the customer; the tooling and the behaviors it will test exist |
| The cross-repo comparisons need all three repositories | The client's workflow skips them and refers to a workflow the server repository no longer has; they run on a machine that has the three side by side |
| No end-to-end browser test in the suites | The screens are exercised by unit tests and by the hand-run sheet; a browser-driven regression suite is not part of any gate |
| Load figures come from a single machine | Use them for shape, and measure on the target hosting |
| No mutation or fuzz testing, no static security scan beyond lint and the dependency audit | A penetration test and a code security review are the customer's to commission |
Compliance Q&A
Yes for engineering testing: the server, the client and the control plane each have an automated suite and a gate that a revision passes before CodeDeploy takes it, the server's integration tests run against a real database built from the migrations, and tests hold the audit-trail, least-privilege, tenant-isolation and no-delete rules in place.
Limits: these are the developer's tests, not a validation. They are not mapped to user requirements, there is no requirements traceability matrix, and no executed OQ or PQ exists in the repository; the customer's validation exercise is separate and is not provided.
Evidence: server tests (430 unit, 137 integration), client 327, control plane 21 and 9npm run verifyscripts/ci/verify.shthe client workflowgovernance teststest sheet
A repeatable harness and dated reports show how the cost of each operation grows with the size of the study (200 to 5,000 subjects) and where a single machine saturates.
Limits: every report comes from one machine running the database, the server, the worker and the harness, and two of the runs missed the form-save target once the user count passed what that machine sustained. Capacity on the customer's hosting has to be measured there.
Evidence: server/loadtestserver/docs/perf (2026-10-05, 2026-10-13)
Related
Running it · The database · Architecture decisions · GAMP 5 and CSV · Not built yet