What the E2E suite proves
The Playwright suite checks the deployed application through its public HTTP and browser boundaries. The security-boundary scenario verifies that, in SaaS mode:
- unauthenticated users are redirected away from dashboard and admin pages;
- enabled protected AI, organization, reporting, audit, and integration APIs return
401without a session; disabled AI/RAG surfaces return404; - public health, OpenAPI, changelog, landing, and blog surfaces remain reachable according to their feature flags;
- configured feature flags produce the documented
404behavior for disabled surfaces.
The boundary checks are in e2e/security-boundaries.spec.ts. They use requests without a storage state, so a passing result demonstrates the unauthenticated boundary rather than a test account shortcut.
Run the evidence
The E2E environment must provide the same database and auth configuration as the target deployment. The global setup validates the environment and waits for /api/health before the browser suite starts.
pnpm test:e2e --project=chromium e2e/security-boundaries.spec.tsRun the complete Chromium evidence set with:
pnpm test:e2e --project=chromiumEvidence beyond unauthenticated flows
Authenticated organization and role behavior is covered by route integration tests with explicit session and membership fixtures. This keeps owner/admin/member authorization deterministic while Playwright verifies the real HTTP boundary. The two layers are complementary:
| Boundary | Evidence |
|---|---|
| Browser navigation and redirects | Playwright |
| Unauthenticated API responses | Playwright |
| Organization membership and role checks | Route integration tests |
| Database/schema behavior | Unit and integration tests |
| Payment provider webhooks | Adapter and route integration tests |
When adding a protected surface, add its unauthenticated browser/API assertion and its authenticated organization-role test in the same change.
Enterprise provider boundaries
The repository tests provider boundaries with deterministic adapters and mocks:
| Flow | Automated evidence | Required staging evidence |
|---|---|---|
| SAML SSO | Authorize/callback route tests and SSO adapter tests | Complete a real IdP login with the deployed ACS URL and configured tenant |
| SCIM | Token, bearer authentication, organization-scoped lifecycle, and rate-limit tests | Run a real provider sync and verify deprovisioning does not affect another organization |
| Billing webhooks | Signature and normalized-event route tests for each configured adapter | Send a signed duplicate and out-of-order event against staging and verify reconciliation |
| Jobs and webhooks | Retry, backoff, dead-letter, and replay tests | Observe the worker, alert, dead-letter queue, and replay procedure with production-like Redis |
Provider mocks validate the application boundary and error handling. Pair them with a staging acceptance run against the provider that will own the production account to verify the complete deployment path.