The runtime topology. The platform is a Next.js app (the UI and the session-holding backend-for-frontend), an Express API and a background worker over PostgreSQL, S3 and a mail relay, plus a separate control plane and an optional AI provider. What is not yet in place is marked honestly below.
What you deploy
| Component | What it is | Notes |
|---|---|---|
| API | Node 24 process trialion-server (port 4000) | Behind a reverse proxy (nginx, with a request limit and a 2 MB body limit); a Dockerfile exists, but the path in use is a VM under PM2 |
| Worker | Node process trialion-worker, the same codebase | Not optional: it sends mail, writes the audit ledger, produces exports, keeps the dashboards' tables, and raises the alerts (Architecture) |
| Web application | Next.js on AWS Amplify | Also holds the session, so it is not a static export. Server-side settings: API_URL, SESSION_SECRET, BFF_CLIENT_ADDRESS_SECRET (the same value as the API's) and CLIENT_ADDRESS_TRUSTED_HOPS |
| PostgreSQL 18 | RDS, TLS with certificate verification | Two roles: an owner and a runtime role (The database) |
| Object storage | S3 | Documents, exports, audit anchors (Object Lock, compliance mode), backups; server-side encryption is required when the driver is S3 |
| SMTP | A mail relay | Invitations, notices, sign-in and signature codes, operator alerts |
| Control plane | Its own API and database, and the console | Provisions organizations and answers the feature-switch question (Platform console) |
Deployment as it stands
The API and worker deploy in place on a single instance through AWS CodeDeploy, and the revision is built by a pipeline that is defined in AWS, not in this repository. The repository has no GitHub Actions workflow: the gate is the test run that precedes a revision (npm run verify: vendored-engine check, typecheck, lint and the test suites, against a throwaway database), and CodeDeploy deploys only a commit that passed it. The hooks do:
| Hook | What it does |
|---|---|
| ApplicationStop | Stops the two PM2 processes |
| AfterInstall | Fetches the runtime settings from an SSM parameter (SecureString) into .env (mode 600), installs dependencies, applies the database migrations as the owner, builds, and prunes development dependencies. The tests do not run here |
| ApplicationStart | pm2 startOrReload of trialion-server and trialion-worker |
| ValidateService | Asks /healthz up to ten times, three seconds apart; a failure fails the deployment |
This is not blue/green: there is a brief gap between ApplicationStop and ApplicationStart with no server process running. Automatic rollback on failure should be switched on in the deployment group so a failed validation does not leave the instance stopped. Each PM2 process restarts at 500 MB. The web application builds and deploys on Amplify; its server-side settings are never exposed to the browser. The control plane follows the same pattern (CodeDeploy and PM2 for the API, Amplify for the console), and its migrations run as its own owner role.
Configuration
All configuration is environment variables, validated against a schema at boot. Invalid or missing required configuration stops the process rather than starting half-configured, and production and staging refuse to start on a list of unsafe combinations (for example a development sign-in code shortcut, no anchoring of the audit ledger without an explicit waiver, a single database role, a long idle timeout, a placeholder secret). Every setting, its default and what it does is on Configuration. The defaults a reviewer asks about first:
| Setting | Default |
|---|---|
| Access token lifetime / refresh token | 15 minutes / 7 days (a production check refuses a refresh token over 30 days) |
| Session idle timeout / absolute timeout | 15 minutes / 12 hours |
| One-time code | 6 digits, valid 5 minutes, 3 attempts, 60-second cooldown, 3 codes per 15 minutes |
| Presigned document link / ready export | 5 minutes / 7 days |
| Invitation link | 72 hours |
| Audit ledger anchoring | required in production: S3 with Object Lock |
| AI provider | none (the form generator and the role assistant are off) |
The two database roles
- Owner role: runs migrations; owns every table and bypasses row-level security.
- Runtime role: what the API and the worker connect as. Granted, table by table, exactly what the application does, and no more: read and append on the ledgers.
The application asserts this at boot, by reading its own effective privileges and the catalog against the ledger contract. In production and staging a violation is fatal: the process refuses to start rather than run with a mutable audit trail. If you deploy with one privileged role, you do not have an append-only audit trail, and the system tells you by not starting.
Health, alerts and lifecycle
- Probes.
/healthz(liveness) and/readyz(readiness) need no sign-in;/readyzanswers 503 only when the database is unreachable, and reports the state of the audit outbox, the export queue, the mail outbox and the backups for a person or a dashboard to read. Nothing polls it for escalation: nginx and CodeDeploy use/healthz. - Alerts. The worker's sweep (every minute) is what escalates. A condition raises an alert, delivered by e-mail to
ALERT_EMAILS(and logged), and clears itself when it recovers:audit.chain_integrity(the ledger no longer verifies),audit.anchor,audit.partitions,audit.outbox(lag or dead letters: events not in the ledger),mail.deliveryandmail.relay_idle(mail not leaving, usually no worker),summary.lagandsummary.drift,backup.stale, andexports.stalled. An alert with no recipient only logs, so settingALERT_EMAILSis part of going live (npm run alert:testsends a test). - Shutdown is idempotent: the probes flip to unavailable first, then the pool drains, with a force-exit timer.
- Diagnostics.
/api/v1/_internal/infoneedssystem.monitor, so the stack cannot be fingerprinted anonymously.
Backups and recovery
Two independent mechanisms cover different failures: RDS automated backups with point-in-time recovery (configured in AWS, up to 35 days) and nightly logical dumps (node scripts/backup/pg-backup.mjs, to a versioned S3 bucket, restorable anywhere), plus a weekly restore verification (pg-verify-restore.mjs), which is the half that is usually skipped. Both record their outcome in backup_runs, and the worker raises a critical backup.stale alert when the last backup is older than 26 hours, the last verified restore older than 8 days, or either has never happened. The cron entries, the AWS-side settings to verify (encryption with a customer-managed key, deletion protection, a copy to a second region, the bucket policy) and the recovery steps are in server/docs/operations/backup-and-recovery.md. The scheduled jobs must be installed: the tooling exists and nothing runs it until they are.
Operational gaps to plan for
| Gap | Consequence |
|---|---|
| One API process and one worker | No redundancy; the API's rate-limit counters are per process. Several API processes need a shared limiter store first |
| In-place deploy | A brief outage on every deployment |
| No metrics, no tracing | You need external observability; the logs, probes and alerts are what exists |
| The generic job queue is a stub | Nothing durable depends on it (mail, audit, exports and summaries use tables), but a new feature that queues work must use a table too |
| No per-request timeout | A slow upstream call can pin a connection; only the database statement timeout bounds database calls |
| Backup, restore, disaster recovery and retention | The tooling and the watch are provided; installing, operating and testing them is the hosting's |
| Validation | Installation qualification is automated (npm run validate:iq); IQ/OQ/PQ protocols and executed evidence are the customer's (GAMP 5 and CSV) |
None of these blocks a pilot. All of them belong on the plan before a production trial at scale.
Related
Architecture · The database · Configuration · Testing and quality · Security and hosting