Testing

This document describes how BatleHub is tested: the categories of automated tests, what each layer covers, how to run them, and — most importantly — what is currently exercised by the integration suite. It is a map, not a tutorial; for how to add a test when you add a registry, see adding-a-registry.md § Testing.

Test-function counts in this document are grep-derived snapshots (#[test] / #[tokio::test] / #[actix_web::test]) and drift as the suite grows. Treat them as orders of magnitude, not contract. The file lists and the shape of coverage are the stable part.

1. Test taxonomy

BatleHub's tests fall into six layers, in increasing order of infrastructure cost:

Layer	Where	Infra	Runner
Unit	`#[cfg(test)] mod tests` inline in each source file	none (HTTP upstreams mocked with `mockito`)	`cargo test --workspace --lib --bins`
In-process integration	`crates/web/tests/.rs`, `crates/examples/tests/.rs`	none — full actix app on in-memory backends	`cargo test -p batlehub-web --test '*'`
CLI subprocess integration	`cli/tests/integration.rs`	none — CLI binary vs. in-memory actix server	`task test:cli:integration`
External integration	`crates/adapters/tests/*.rs`	real Postgres / RustFS(S3) / Redis via Podman	`task test:pg-*`, `task test:s3`
Heavy client	`tests/heavy/*.sh`	real Postgres and a real client — VS Code, IntelliJ, Bundler, npm, pip, ovsx, micromamba, dotnet, composer, terraform, nvm, mise, cargo, go, mvn, apt/dnf, apk	`task test:heavy`, or one `task test:<ecosystem>-heavy`
Heavy authorization	`tests/heavy/authz.sh`	real Postgres, grants from a real config file, and the same clients	`task test:authz-heavy`, or `task test:authz-matrix-heavy` for the fast half
Soak / leak	`perf/k6/scenarios/10_soak.js`, `tests/heavy/soak.sh`	real Postgres, a mock or served upstream, and constant load for as long as you ask	`task perf:soak`, `task test:soak-heavy` — manual only
Fuzz	`fuzz/fuzz_targets/*.rs`	nightly toolchain to run, none to check	`task fuzz:check`, `task fuzz`
Editor patch	`patches/che-code/*.test.ts`	none — Node strips the types itself	`task test:patch`
API contract	`crates/web/tests/openapi_contract.rs`	none — walks the generated spec and the handler sources	`cargo test -p batlehub-web --test openapi_contract`

The in-process layer is the workhorse: every test there spins up a real actix-web application wired to InMemoryPackageRepository, InMemoryStorageBackend, InMemoryCacheStore, and FixedRegistry, so it exercises the true request lifecycle (auth middleware → handler → service → rules → storage) without any external dependency.

2. Running the tests

bash

# Everything that needs no external infra
cargo test --workspace

# One package / one filter
cargo test -p batlehub-web namespaces
cargo test -p batlehub-adapters --lib rbac

# In-process integration only
cargo test -p batlehub-web --test '*'
cargo test -p batlehub-cli --test integration

# batlehub-cli (`--bins`, not `--lib`: the crate is binary-only)
task test:cli                 # unit + subprocess integration
task test:cli:unit            # inline unit tests only, seconds
task test:cli:integration     # the built binary vs. an in-memory server
task test:cli:lint            # clippy -D warnings + fmt --check
task test:cli -- setup_detect # any of the above take a filter after `--`

# External integration (each starts its own container via Podman)
task test:pg-cache            # Postgres — PgCacheStore
task test:pg-local-registry   # Postgres — PostgresLocalRegistry
task test:pg-storage-router   # Postgres — StorageRouter
task test:pg-artifact-meta    # Postgres — PgArtifactMetaRepository
task test:pg-vulnerability    # Postgres — PgVulnerabilityRepository
task test:pg-air-gap          # Postgres — the miss log's upsert, cap and purge (RFC 0008)
task test:patch               # the editor credential patch (RFC 0011) — plain `node --test`
task test:s3                  # RustFS   — S3StorageBackend (feature storage-s3)

# Repo interop (real apt/dnf/pacman consume signed repos)
task test:repo-interop

# Heavy client integration (needs DATABASE_URL; each drives a real client)
task test:heavy               # every suite below except the marketplaces
task test:marketplace-heavy   # headless VS Code + IntelliJ install an extension
task test:bundler-heavy       # `bundle install` resolves through the compact index
task test:npm-heavy           # publish/install/whoami/dist-tags/search + `npm audit`
task test:pypi-heavy          # `twine upload`, `pip install`, PEP 658 metadata
task test:openvsx-heavy       # `ovsx publish` (query-parameter token) + `ovsx get`
task test:conda-heavy         # micromamba: the HEAD probe and a post-warm publish
task test:nuget-heavy         # `dotnet nuget push` / `package search` / `add package`
task test:composer-heavy      # local + proxy resolution with Packagist disabled
task test:terraform-heavy     # `terraform init` over TLS, host-routed discovery
task test:nvm-heavy           # `nvm ls-remote` / `nvm install` against a nodedist registry
task test:sdkman-heavy        # `sdk list` / `sdk install java` against an sdkman registry (RFC 0010)
task test:mise-heavy          # `mise install github:…` through a forge registry (RFC 0019),
                              # then the whole air gap: plan, seed, export, import into a
                              # second `[air_gap]` instance, install through it (RFC 0008)
task test:apk-heavy           # RFC 0026 §6.8: both apk generations (2.14.10 and 3.0.8, each as
                              # apk-tools-static so neither needs a rootfs) — the relayed index,
                              # a blocked .apk refused at the download, and apk's URL-embedded
                              # Basic credential proven denied *and* allowed
task test:cargo-heavy         # RFC 0018 §4.4: yanked mark, 403/404 refusal, recovery, `cargo publish`
task test:rustup-heavy        # RFC 0024 §6.10: a closed world — the compiler, the crates, the
                              # build and the run through one instance with egress denied
task test:closed-world-heavy  # egress denied to the client, kept for the server: 22 registry
                              # kinds each resolve, build and run against this instance
                              # alone (`-- go` for one phase)
task test:go-heavy            # …same axes for `go`, plus the GOPROXY `direct` fallback and the sumdb
task test:maven-heavy         # …same for `mvn`, incl. its cached failure and `deploy:deploy-file`
task test:pathproxy-heavy     # …same for `apt` and `dnf`, whose signed indexes cannot hide anything
task test:quarantine-heavy    # RFC 0018 from npm's side: a hold, the worker clearing it, a real OSV
                              # (and RFC 0002: a pushed flag refusing npm, then revoked)
                              # advisory refused, the listing agreeing, warn mode, the sandbox's egress
task test:upstream-audit-heavy # RFC 0014 from the receiving end: a served directory loses a package,
                              # two probes confirm it, a webhook receiver the suite runs gets the event
task test:authz-matrix-heavy  # every verb in the vocabulary, both directions, over curl
task test:authz-heavy         # …plus signed-URL expiry/rotation and each real client

# Coverage (starts Postgres + RustFS + Redis; HTML report in coverage/html/)
task coverage
task coverage-check           # fails if line coverage < 80%

# Fuzz
task fuzz:check               # every target still compiles — the per-PR gate
task fuzz TARGET=fuzz_deny_latest MAX_TIME=30   # actually fuzz one (nightly)

The Redis adapter tests (redis_cache, redis_rate_limit, redis_warm_coordinator) and pg_rate_limit / actions_oidc have no dedicated task test:* wrapper but run under task coverage and in CI.

The two suites that drive a forge registry — mise and airgap — read HEAVY_FORGE_TOKEN. Set, heavy_forge_auth_config (tests/heavy/lib.sh) copies the suite's config with [registries.upstream_auth] on its github registry and starts from the copy, the token expanded from the environment rather than written to the file. Unset, both stay anonymous and nothing changes — which is the point: neither suite may require a secret. What anonymous costs is that GitHub counts its 60 API requests an hour per source IP: a hosted runner shares that IP with every other job on the machine, so the budget is often already spent, the proxy refuses below its 10 % reserve (RFC 0019 §5.2) and the air-gap seed fails on a 502. CI therefore passes the workflow token (1 000 an hour, per repository, issued on forks too).

3. Unit tests

Unit tests live inline (#[cfg(test)] mod tests) in the file they cover. The notable convention is registry-client tests: they mock the upstream HTTP API with mockito::Server rather than hitting the real registry.

Registry adapters with test modules under crates/adapters/src/registry/:

Standalone <name>/tests.rs (used when tests span more than one sibling file): composer, conda, jetbrains_marketplace, pypi, rubygems, terraform.
Inline mod tests: cargo, npm, openvsx, goproxy, plus the shared infrastructure modules fanout, http_client, path_proxy, ssrf; and the directory clients forgejo/client.rs, github/client.rs, gitlab/client.rs, maven/models.rs, nuget/client.rs, vscode_marketplace/client.rs.

Every registry adapter has a test module. The ssrf and http_client modules additionally cover SSRF protection and the shared upstream HTTP client (auth forwarding, TLS, private-CA support).

4. In-process integration tests

crates/web/tests/*.rs — 88 files, 1 325 test functions (counted 2026-09-06). Shared app-factory infrastructure (make_app, make_local_svc, access_config*, LocalRegistryAppParts / build_local_registry_app) lives in crates/web/tests/common/mod.rs; every other file begins with mod common; use common::*;.

Feature areas covered (file → area):

File	Area
`proxy_basic.rs`	Core proxy: cache-first read, stale-on-error, streaming
`proxy_npm_edge_cases.rs`, `proxy_cargo_edge_cases.rs`	npm / cargo proxy edge cases
`proxy_openvsx_vscode_goproxy.rs`	OpenVSX / VS Code Marketplace / Go proxy
`cargo_and_downloads.rs`	Cargo proxy paths + download counting
`generic_proxy.rs`	Generic file-mirror `GET /proxy/{reg}/generic/{path}`
`repo_deb_rpm_pacman.rs`	Deb / RPM / Pacman path repositories
`terraform.rs`	Terraform modules + providers (v1 API)
`namespaces_and_visibility.rs`	Namespace claim/release + package visibility
`admin_packages.rs`, `admin_stats.rs`, `admin_health_and_bulk.rs`, `admin_access_check.rs`	Admin API: packages, stats, health, bulk ops, access-check
`bulk_and_quota_and_cache.rs`	Bulk operations, per-registry quotas, cache clear/warm
`tokens_and_pagination.rs`	API token CRUD + list pagination
`rate_limit.rs`	Rate-limiting middleware
`ip_blocks.rs`, `user_blocks.rs`	IP block enforcement, user block/unblock
`dynamic_groups.rs`	Dynamic group membership / RBAC groups
`beta_channel.rs`	Beta / pre-release channel gating
`banner_and_config_reload.rs`	Service banner + hot config-reload endpoint
`notifications.rs`	Notification channels / dispatch
`explore.rs`	Package Explorer discovery backend
`vuln_proxy_endpoints.rs`, `vuln_findings.rs`	Vulnerability proxy endpoints + findings store
`sbom_and_misc.rs`	SBOM read endpoints
`publish_traversal_guards.rs`, `upload_traversal_and_enforcement.rs`	Cross-registry publish/upload traversal guards + policy enforcement
`air_gap.rs`	RFC 0008: an instance that will not dial out, and what it answers instead
`security_registry.rs`, `flags.rs`	RFC 0018 quarantine end to end; RFC 0002 pushed flags on a registry with no `[security]`
`vsx_signing.rs`	RFC 0020: the VSIX signature asset the registry signs
`forge_refs.rs`, `forge_security.rs`, `forge_api_reads.rs`	RFC 0019 phases 1–3: ref resolution and commit keying, what a ref does to a request, the raw policy and typed reads
`authz_matrix.rs`, `authz_explain_oracle.rs`, `vocabulary_dead_ends.rs`	The route-by-route authorization matrix, `explain` agreeing with the decision, and RFC 0015 §11.5's no-dead-ends property
`grants_editor.rs`, `grants_shadow.rs`, `gate_exemptions.rs`	RFC 0017's grants editor, shadow mode, and the `gates:exempt` verb
`admin_policy.rs`, `admin_subjects.rs`, `tiered_versioning.rs`	The policy table's admin API, `GET /admin/subjects`, and `immutable` / `monotonic` on publish
`local_read_authorization.rs`	Per-package visibility on the artifact routes that read storage directly
`blocked_versions_hidden*.rs`	Twelve files, 91 tests. One property per ecosystem: a blocked version disappears from the listing, not only from the download, and whatever the protocol calls "newest" is repaired
`tombstones.rs`, `upstream_audit.rs`, `listing_audit.rs`	RFC 0016 coordinate reuse, RFC 0014 confirmed disappearance, and what a listing writes to the audit trail
`explore_fetch.rs`, `explore_upstream_detail.rs`	The Explorer's fetch-this-version button, and the package page for something this instance holds nothing of
`package_readmes.rs`, `readme_images.rs`, `readme_search.rs`, `search.rs`	README capture, the image endpoint's refusals, README search, and search across the five ecosystems that share one path
`host_routing.rs`, `spa_csp.rs`, `oidc_sso.rs`, `me_endpoints.rs`	RFC 0001 subdomain routing, the console's CSP, the browser sign-in flow, and the caller-scoped `/me` reads
`upstream_calls_are_cached.rs`, `document_cache_audience.rs`	Two invariants rather than features: every outbound call goes through the caching helper, and one caller's document is never replayed to another
`protocol_conformance.rs`, `vscode_gallery.rs`, `misc_standalone_endpoints.rs`	The paths clients actually send, BatleHub as an editor marketplace, and the endpoints that belong to no group

5. Per-registry local-registry tests

Each registry that supports local/hybrid mode has a dedicated local_<type>_registry.rs file with a make_local_<type>_app(mode) factory and a publish-payload helper. These assert the full private-registry lifecycle: publish (with anon-403 / proxy-404 / duplicate-409 rejections), download, the registry-specific metadata endpoints, yank/unyank/delete, and hybrid local-vs-proxy precedence.

File	Highlights
`local_cargo_registry.rs`	Sparse index, download, yank/unyank, unlist, admin-gated deprecate, owners, hybrid precedence
`local_npm_registry.rs`	Packument, version metadata, tarball download, name + version traversal guards
`local_maven_registry.rs`	PUT pom (version-mismatch-400 / dup-409), jar-before-pom, `maven-metadata.xml`, proxy-mode rejection
`local_nuget_registry.rs`	v3 service index (+ vulnerabilities resource), `X-NuGet-ApiKey` auth, flat-index, registration/catalog, search
`local_composer_registry.rs`	`packages.json`, p2 metadata (local/proxy/hybrid fallback), dist streaming, invalid-zip-422
`local_go_registry.rs`	`@v/list`, `.info`, `.mod` extraction, `.zip` download, `@latest`
`local_vsx_registry.rs`	VSIX publish + download-after-publish (shared by OpenVSX & VS Code Marketplace)
`local_jetbrains_marketplace_registry.rs`	Plugin publish (jar / nested-zip / descriptor validation), `updatePlugins.xml` build filtering, search, compatible-updates, offline/stale serving
`local_rubygems_proxy.rs`	Proxy-mode gem download, info, versions, specs (full/latest/prerelease); publish/yank return 404 in proxy mode
`local_rubygems_compact_index.rs`	The compact index (`/versions`, `/info/{gem}`) served from a local registry — what Bundler actually reads
`local_nodedist_registry.rs`	The `nodejs.org/dist` tree as a typed registry (RFC 0010 phases 2–3)
`local_sdkman_registry.rs`	SDKMAN as a typed registry (RFC 0010 phases 5–6): candidates, the broker, the post-install hook

Additional local-registry coverage (Deb, RPM, Pacman, Conda, PyPI, Terraform) lives in the feature-area files above (repo_deb_rpm_pacman.rs, terraform.rs, and the proxy/upload files).

6. Path-traversal guard tests

Rejecting .. and path separators in package coordinates before they reach a storage key is a hard requirement for every registry (see the security note in CLAUDE.md and adding-a-registry.md). The canonical regression is <type>_publish_traversal_version_returns_400, which publishes with version = "../../etc/x" and asserts 400.

Traversal guards are currently tested for: cargo, composer, conda, deb, generic, jetbrains-marketplace, maven, npm (name + version), nuget (id + version), openvsx, pacman, pypi, rpm, rubygems, terraform (provider artifact path) — plus a delete-cached-artifact traversal case. New registries must add the matching test.

7. External integration tests (real infra)

crates/adapters/tests/*.rs — these need real infrastructure, opt in via environment variables (DATABASE_URL, S3_TEST_ENDPOINT, REDIS_URL), and skip gracefully when the variable is unset.

File	Infra	Verifies	Task
`pg_cache.rs`	Postgres	`PgCacheStore`	`test:pg-cache`
`local_registry.rs`	Postgres	`PostgresLocalRegistry` (publish/yank/delete)	`test:pg-local-registry`
`artifact_meta.rs`	Postgres	`PgArtifactMetaRepository`	`test:pg-artifact-meta`
`storage_router.rs`	Postgres	`StorageRouter`	`test:pg-storage-router`
`pg_vulnerability.rs`	Postgres	`PgVulnerabilityRepository`	`test:pg-vulnerability`
`pg_rate_limit.rs`	Postgres	`PgRateLimitStore`	coverage / CI
`s3_storage.rs`	RustFS / S3 (`storage-s3`)	`S3StorageBackend`	`test:s3`
`redis_cache.rs`	Redis (`cache-redis`)	`RedisCacheStore`	coverage / CI
`redis_rate_limit.rs`	Redis (`cache-redis`)	`RedisRateLimitStore`	coverage / CI
`redis_warm_coordinator.rs`	Redis (`cache-redis`)	`RedisWarmCoordinator`	coverage / CI
`actions_oidc.rs`	none (mockito)	GitHub-Actions OIDC bootstrap / discovery / JWKS failures	CI
`selfhosted.rs`	none (mockito + in-test TLS)	private-CA HTTPS upstreams, `UpstreamHttpOptions` (bearer/basic auth, TLS)	CI
`repo_interop.rs`	none at test time (`#[ignore]`d generator)	writes a fully-signed Deb + RPM + Pacman repo with production signing code, consumed by `tests/interop/verify.sh`	`test:repo-interop`

The repo-interop flow is notable: repo_interop.rs generates signed repositories using the real signing code path, and tests/interop/verify.sh then has genuine apt, dnf, and pacman clients consume and verify them — end-to-end proof that the OS-package output is standards-compliant.

7-bis. Heavy client tests (a real package manager)

tests/heavy/*.sh — each one starts a real BatleHub against a real Postgres, puts a transparent logging proxy (http_tap.py) in front of it, drives that ecosystem's real client, and asserts on the wire transcript. Shared machinery is in tests/heavy/lib.sh.

Two of them are closed-world suites, and they are the ones that answer "can this instance be the only way out?". BatleHub keeps its egress; every client process gets none — HTTP(S)_PROXY points at a denial with only the loopback exempted — so anything a phase obtains, it obtained through the instance. rustup.sh points it at a closed port; closed_world.sh points it at closed_proxy.py, which relays the loopback and refuses every other host, because a closed port only denies a client that reads NO_PROXY and VS Code's CLI does not. tests/heavy/rustup.sh is Rust's, and proves the most, because there the compiler itself crosses the proxy. tests/heavy/closed_world.sh is every other kind, one phase each, selected by name:

Phase	Kind	Client	What ends the phase
`go`	`goproxy`	the Go tool	the built binary runs
`node`	`npm`	npm + tsc	the compiled program runs
`python`	`pypi`	pip	the built wheel's code runs
`java`	`maven`	Maven (plugins included)	the packaged jar runs
`ruby`	`rubygems`	Bundler	`bundle exec` runs
`dotnet`	`nuget`	the .NET SDK	the built program runs
`php`	`composer`	Composer	the program runs
`conda`	`conda`	micromamba	the env's own python runs
`terraform`	`terraform`	Terraform	`apply` executes the provider
`mise`	`github`	mise	the installed tool runs
`nvm`	`nodedist`	nvm	the runtime it installed runs
`sdkman`	`sdkman`	`sdk`	the JDK it installed compiles and runs
`ovsx`	`openvsx`	`ovsx` (itself installed through the npm registry)	the VSIX opens as the extension asked for
`helm`	`generic`	curl	the helm binary renders a chart
`apt`	`deb`	unprivileged `apt`	the packaged binary runs
`forgejo`	`forgejo`	mise's `forgejo:` backend	the installed tool runs
`gitlab`	`gitlab`	mise's `gitlab:` backend	the installed tool runs
`jbr`	`jetbrains`	curl	the JetBrains Runtime compiles and runs
`vscode`	`vscode-marketplace`	the editor, `product.json` pointed here	the editor lists the extension back
`jbplugin`	`jetbrains-marketplace`	IntelliJ headless	the plugin lands in the plugins directory
`dnf`	`rpm`	`dnf install`, inside an AlmaLinux 9 image	the installed program runs
`pacman`	`pacman`	`pacman -Sy`, inside an Arch image	the installed program runs

The last two run their client inside that client's own distribution image, on podman or docker (whichever answers info first). That is not convenience: a distribution package is dynamically linked against its own libraries, so a phase that downloaded an Arch package on Ubuntu and executed what was inside it could only ever pass on a host that already was Arch — which is no hosted runner. Running the client where the package was built is the only correct form of the claim, and it also means the runner no longer has to be persuaded to host a foreign package manager. deb stays on the host because the runner is Ubuntu.

The container is not the isolation. --network host is what keeps 127.0.0.1:<tap> meaning the same thing inside as out, so the container shares the host's network namespace and egress is denied by the same six proxy variables, passed in with -e. Each containerised phase re-proves that denial from inside against a real host before trusting anything else — §0 establishes it for a host process and cannot speak for a different process in a different filesystem. A phase whose image has no curl refuses rather than concluding the world is closed because the probe could not run.

Three rows can be left unmeasured: HEAVY_CW_SKIP_RPM=1 and HEAVY_CW_SKIP_PACMAN=1 for a machine with no container engine, and HEAVY_CW_SKIP_JB=1 for one that cannot spare the ~1.5 GB IntelliJ. Each prints a SKIPPED banner naming the row. Absent the variable a missing client is a failed run, not a skipped one — a heavy test that skips itself reports success for having done nothing.

The config generator's harness

The docs site's config generator (docs/.vitepress/components/ConfigGenerator.vue) writes TOML nobody used to parse. Its pure half — types, defaults, helpers and renderConfigToml(state) — lives in configToml.ts so it runs without Vue, and three things hold it:

node --test docs/build/config-generator.test.ts renders the scenarios of config-generator-scenarios.ts and asserts on the sections (plain Node, types stripped by Node itself);
docs/build/config-generator-fixtures.ts writes each rendering to crates/config/tests/fixtures/config-generator/*.toml, and cargo test -p batlehub-config --test config_generator_fixtures loads every one with load_from_str and validate() — the real parser saying the generator emits a config the server accepts;
task docs:generator:check (part of task docs:design) fails when the fixtures drift from the generator. task docs:generator regenerates them.

Add a scenario for every section the generator learns to emit.

They exist because the layers above them cannot fail on the defect that matters most here: a route that is present, tested, and answering 200 with something no client can use. RFC 0009 §5.2 lists the ways — a resource the client cannot select, a method the route does not accept, an auth boundary the client does not cross, a field whose digest is the wrong algorithm — and every one of them was found by running the client, by nothing else, twice over.

Suite	Client	What only this layer can prove
`marketplace.sh`	VS Code, IntelliJ	an extension that exists only here installs by id through the gallery
`bundler.sh`	Bundler 4.0.17	the compact index's `206`/`304` are answers Bundler accepts — the assertion is the absence of a re-fetch
`npm.sh`	npm	publish → install → `whoami`/`ping`/`dist-tag`/`search`, and `npm audit` on the path npm really sends
`pypi.sh`	twine, pip	the documented `twine upload` (HTTP Basic) works, and pip's PEP 658 `.metadata` sibling answers
`openvsx.sh`	ovsx	`ovsx publish` with its token in a query parameter, and `ovsx get` following the rewritten download URL
`vsx_login.sh`	VS Code 1.136.1 (the server build's CLI, headless)	RFC 0011 §4.4: `batlehub-cli proxy serve` in front of a registry whose `anonymous` holds no verb, with `product.json` repointed at the proxy. Unauthenticated, a search through the proxy is the one `batlehub.sign-in` entry with `Code.Engine`, an install by id fails as not found with no request reaching the registry, and the sign-in package is refused as `NotSigned` until `extensions.verifySignature` is off, then installs; after `auth write-token-file`, without restarting anything, the same editor installs the fixture by id and every registry request on the tap carries `Authorization: Bearer`
`vsx_view.sh`	VS Code 1.136.1 (the server build's workbench, in Chrome over the DevTools protocol — a workspace's sidecar via `CDP_URL`, or a headless `CHROME_BIN`)	RFC 0011 §4.4.2 and RFC 0020 in a real Extensions view: unauthenticated, browse and search list the one sign-in entry and opening it renders the sign-in page, with nothing reaching the registry; the entry's Install button is disabled and the editor says not signed (pinned — the entry is a page, deliberately unsigned). After `auth write-token-file`, the same page's Refresh lists the fixture with Install enabled (the registry signed it); the click passes the publisher-trust dialog, fetches package and signature through the proxy, and the editor's verifier refuses (`UnhandledException`, pinned by running the editor's own `vsce-sign` on the served archive, which `batlehub-cli vsx verify` accepts); the server's CLI refuses the same way and installs once `extensions.verifySignature` is off, and on a second look so does the view. A marketplace extension republished with its own archive attached (`PUT …/vsix/signature`) gets `Success` from `vsce-sign` and installs with the verifier on. Last, RFC 0007-bis §11 q1: an extension whose manifest names an `icon.svg` carrying a `<script>`, an `onload` and a `javascript:` link is served as `image/svg+xml` under the sandbox policy with all three gone and the drawing kept, and the gallery advertises that asset on its entry — which an `application/octet-stream` icon never is, since an entry with no usable icon gets the editor's `defaultIcon`. The real Extensions view lists the extension and shows that asset as its icon, which is why the step runs before anything is installed — a view with an installed extension opens on Installed and filters a typed query against it. Whether the browser paints the icon is logged and not asserted: it asks, and the fetch is aborted client-side for a reason not yet established. `tests/heavy/vsx_view.mjs` is the driver
`backends.sh`	npm, pip, on S3 (RustFS, locally and in CI alike), Redis and an OIDC issuer (dex)	the backends every deployment has and every other suite lacks — all 26 heavy configs are `filesystem` + `token` + no `[cache]`. Against an upstream the suite serves (so a cache hit is a count of zero upstream requests): anonymous and a tampered id_token are refused with nothing reaching the upstream; dex's id_token for `dev@example.com` installs with the upstream asked once for each of packument and tarball, the tarball an object in the bucket (`mc ls`) and the packument a Redis key naming the registry; a second install from a clean npm cache asks the upstream for nothing; the server is stopped and started again and a third install still asks for nothing — what the backends hold outlived the process, which the in-memory cache and a local directory never have to prove; pip the same way, a wheel through S3 and a simple page through Redis. Starts MinIO and Redis itself from cached binaries when `S3_TEST_ENDPOINT`/`REDIS_URL` are unset, and discovers dex on 9000 (the workspace sidecar; a container in CI)
`hybrid.sh`	npm, pip, on `mode = "hybrid"`	local-first, measured — sixteen in-process tests know the rule and no client had met it. One `npm install` of a private, an upstream-only and a shadowed name (held here at 2.0.0, upstream at 1.0.0) resolves all three: ours from here, the upstream's from the upstream once, the shadowed one ours — and `npm view … versions` lists our version alone, the upstream never asked. A blocked proxied version vanishes from the listing (npm: no matching version, the tarball never fetched again); a local package whose only version is blocked is refused `403` and the upstream's same-named 1.0.0 is never asked for — the one rule that turns a block into a fall-through if broken. pip: the same three shapes, published with the legacy upload
`vscode_patch.sh`	VS Code 1.136.2 (the server build's CLI, headless, with `patches/che-code/vsxRegistryAuth.ts` loaded into its process by `tests/heavy/editor_patch_preload.mjs`, which carries the glue of the validated diff)	RFC 0011 §4.2 on the consumer side, case A of §5.5: the editor core sends the credential itself, no proxy and no extension. `product.json` points straight at a registry whose `anonymous` holds no verb, through the tap, so what the tap records is the editor's own request. The patch's unit tests run first under the editor's own node; then, with nothing to send, an install by id fails and the registry saw the query refused `403` with no `Authorization` on any request; with `VSX_REGISTRY_AUTH_TOKEN` scoped by `VSX_REGISTRY_URL` — and the variable alone sending nothing — with the contract file `auth write-token-file` writes at the patch's default path (keyed by the origin, looked up by the request's own origin), and with that file beside a wrong variable, the install goes through and every registry request carries `Authorization: Bearer`; a `--from-file` token source entry is refused without the variable and installs with it; the credential is in none of the editor's output. Scenarios in `tests/heavy/editor_patch_scenarios.sh`
`che_code_patch.sh`	che-code (`quay.io/che-incubator/che-code` — the workspace's own build copied once, `CHE_CODE_DIR`, or `docker cp` out of the image), its CLI under its own node and `ld_libs`	the same scenarios on the editor Eclipse Che runs, with the gallery named through `OPENVSX_REGISTRY_URL` the way che-code's launcher names it — the variable the diff reads on that build
`vsx_extension.sh`	the `batlehub-vsx` extension (its own repository, §11 q6) in VS Code 1.136.2's workbench, in Chrome over CDP, both modes	RFC 0011 §6.5 and §14.11 as a gate on this side: the extension repository's `tests/heavy/view.sh`, run with `BATLEHUB_SRC` pointed here — marketplace mode (a stock build, `BATLEHUB_TOKEN` in the editor's environment: the BatleHub view lists what the registry shows this credential, its inline Install verifies RFC 0020's signature with the registry's key and installs through the editor's own command, the Extensions view lists it) and broker mode (`batlehub-cli proxy serve` as the gallery: sign-in offered, the sign-in entry alone, then the extension writes the §4.1 contract file `0600` keyed by the origin and the same workbench lists the registry's extension with no reload). The checkout is `../batlehub-vsx` when there is one, else a shallow clone; CI checks out its default branch, so this job is the canary for the two repositories drifting
`console_fetch.sh`	Chrome over the DevTools protocol (a workspace's sidecar via `CDP_URL`, or a headless `CHROME_BIN`), against the built console	RFC 0007-bis §11 q3: the catalogue's Fetch button pressed by a person. The SPA and the API are one origin (`static_dir`), which is the deployed shape and keeps the run from proving anything through a CORS arrangement nobody deploys. An anonymous reader — who can browse this registry, so the assertion is about the offer and not about visibility — sees the upstream row and no button. A signed-in reader sees a button whose label carries the version the upstream search returned, clicked by its accessible name so the control pressed is the one a keyboard user reaches. Afterwards the row has left the upstream half, the version is held according to the package API, its tarball comes back through npm's own path, the tap saw the console's `POST …/fetch`, and the audit attributes it to the reader who pressed it rather than to an administrator. `tests/heavy/console_fetch.mjs` is the driver
`conda.sh`	micromamba	the `HEAD` probe for `repodata.json.zst` reaches a handler, and a publish is visible in the compressed channel
`nuget.sh`	dotnet	the client can select the search resource, `skip` advances the page, and `push` hits the path it appends a slash to
`composer.sh`	composer	proxy-mode resolution with Packagist disabled, `dist.shasum` the client accepts, and `search.json` reached through the advertised template
`terraform.sh`	terraform	`init` over TLS: host-routed discovery, download document, shasums, signature and archive, all through the proxy
`upstream_audit.sh`	npm, `webhook_sink.py`	a disappearance is told: the suite serves a directory as an npm registry, `npm install` seeds the cache, the files are removed, two `recheck` probes confirm — one miss tells nobody — and `package_disappeared_upstream` lands at a receiver the suite runs, with RFC 0014 §4.5's payload; the restore delivers `package_reappeared_upstream`; then the same directory as a `generic` registry (RFC 0014 §13.5): a held file removed upstream is missed by a `HEAD` on its path, confirmed, refused `403` on its own route while its sibling stays `200`, and served again after the restore
`quarantine.sh`	npm, `batlehub wait`/`why`	a `[registries.security]` registry as a client meets it: first contact held with the reason code in npm's own output, the embedded worker clearing it and the same cache recovering, a real OSV advisory refused on the wire and named by `why`, the packument hiding the denied version, a reload to `warn` serving it with the verdict headers, a scanner under `bwrap` that tries to reach the tap and cannot, and the flip — a version scanned clean by an OSV the suite runs, installed, then refused after the scheduler's rescan, with the admin alert at a receiver the suite runs naming who pulled it; and RFC 0002's other producer — a SOC's signed `hard_block` on a version already installed, which refuses the pinned `npm ci` as `SOC_VERDICT`, shows up in `/api/v1/admin/exposure` as a pull that preceded the flag, and lifts on the revoke
`airgap.sh`	npm, pip, mise	RFC 0008-bis, both halves on one disconnected instance: one version of an npm package, a PyPI wheel and a release asset carried by bundle into a second `[air_gap]` instance. With `synthesise_listings = false` (the phase-0 measurement) `npm install` and `pip install` of a version string stop at the packument and the simple page (`503`, retried, the held artifact never asked for) and `mise install` with no lock at the release by tag and then the listing, while `npm ci` from a lock and a PEP 508 direct reference fetch the artifact and nothing else. Then the miss log is purged, the key flipped by a hot reload, and the same three commands complete off listings the instance composed — `X-BatleHub-Listing: synthesised` on the packument, the JSON simple page and the release by tag — with an unheld version failing in npm as `ETARGET` and never asked for, an unheld tag refused at the release by tag and recorded as requested v2.59.0, held v2.60.0, which `admin air-gap-missing` prints as two columns; then `cargo`, `go`, `mvn` and `dotnet` resolving a range or an unpinned request through the composed sparse index, `@v/list`, `maven-metadata.xml` (whose `.sha1` the instance answers for itself) and flat index (Maven's plugins warmed against the connected instance first, under the same mirror id); then `bundle install` through the composed compact index and `micromamba create` through the composed `repodata.json`, whose entries the import read out of the gem and the package; then `terraform init` through a composed `versions` and a composed provider `download` document, on a TLS tap of its own (8121, `localhost` bound to the Terraform registry) — the archive, checksum list and signature carried as three plan entries, the publisher's keys read off the connected instance's document by `mise export` and carried on the manifest as `facts`, and Terraform reporting the publisher's signature verified
`authz.sh`	all of the above	a caller who may not pull is stopped, the one who may is not stopped by accident, and RFC 0012's signed URLs let a closed Terraform registry install end to end

Conventions worth knowing before adding one:

A fresh registry name per run ($HEAVY_RUN). The database persists, and a package left by an earlier run changes what the client sees.
Never rewrite Host. The server builds its absolute URL templates from it, so a rewriting proxy sees only the first request and the transcript looks clean because nothing was observed.
Give each client phase its own cache. npm publish seeds the tarball into cacache, micromamba caches repodata, NuGet has a global package folder: reuse one and the test measures the client's cache, not the server.
A missing client is a failure, not a skip (heavy_need). A heavy test that skips itself reports success for having done nothing.
Assertions scope to a phase with heavy_mark / heavy_wire_after: the transcript accumulates, and an unscoped match can be satisfied by an earlier phase.

7-ter. The authorization suite

tests/heavy/authz.sh is a heavy suite in the same harness, aimed at RFC 0015's grants rather than at one ecosystem's protocol. It takes a target:

bash

tests/heavy/authz.sh matrix        # every verb, both directions, over curl
tests/heavy/authz.sh signing       # RFC 0012 capabilities: binding, expiry, rotation
tests/heavy/authz.sh reads         # the read boundary on the kinds no client drives
tests/heavy/authz.sh npm           # …and the boundary as npm meets it
tests/heavy/authz.sh <ecosystem>   # pypi nuget composer conda openvsx rubygems terraform
tests/heavy/authz.sh maven         # …hermetically: the run deploys its own fixture
tests/heavy/authz.sh cargo         # …and the regression test for `auth-required`
tests/heavy/authz.sh live:goproxy  # the same, for a kind with no local mode (nightly)

One target per invocation, because each starts its own server and tap and the Terraform one has to terminate TLS. task test:authz-heavy runs them in sequence; CI runs them as the heavy-authz matrix.

Four rules it is built on, each of which has a scar behind it:

Every assertion is a pair. The denial and the identical request working for somebody. RFC 0015 §13.16 records all three new ecosystem verbs shipping unreachable — 403 to the administrator — with passing tests, because each test granted its own verb and nothing asserted anyone could use it. A negative test cannot distinguish a correct denial from a denial for the wrong reason.
The negative arm is exactly 403; the positive arm is "not a refusal". A 404 where a 403 was meant hides a routing failure, and a holder can legitimately meet a 404, 400 or 409 past the gate — asserting 200 would make every positive control a hostage to its fixture.
The vocabulary is read out of the enum at run time, and the run fails on a verb it never exercised. Same shape as vocabulary_dead_ends.rs, asked of the wire.
explain is asked about the request the wire just answered. §11.6: "a diagnostic that can disagree with reality is worse than none, because it is trusted" — and it has disagreed twice, under a shadow and at the instance tier.

Which kinds are covered, and which are not

Eight ecosystems have a client phase. Everything else had no authenticated coverage of any kind until the reads target: closed_world.sh proves an anonymous client can reach a registry, and that says nothing about whether a caller who may not read is refused.

Coverage	Kinds
Client-level, hermetic — the real package manager carries a credential and is stopped, with no upstream	npm, pypi, nuget, composer, conda, openvsx, rubygems, terraform, maven, cargo
Client-level, live — the same, for a kind with no local mode (nightly)	goproxy, nodedist, sdkman, github, forgejo, gitlab, generic, deb, rpm, pacman, rustup
Route-level — `reads`, paired over curl	goproxy, maven, cargo, vscode-marketplace, deb, rpm, pacman, github, forgejo, gitlab, nodedist, sdkman, jetbrains, generic, rustup

reads publishes nothing and reaches no upstream. Both halves of that rest on one property — authorization is decided before any fetch:

Local kinds. read_gates fetches the version's row and, finding none, still calls authorize_unheld_read: a caller holding no read verb meets 403before the 404. The deb/rpm/pacman handler does the same one layer up, calling authorize_read before it touches storage.
Proxy-only kinds. handle_resolved evaluates grants in step 1 and resolves upstream metadata in step 2, so a registry pointed at example.invalid still answers 403 to the denied caller, while the reader gets the upstream's failure — not a refusal, therefore a pass.

The positive arm asserting "not a refusal" rather than 200 is what lets both work against a registry that can serve nothing.

Two server changes made this coverage possible

The authorization ordering. handle_resolved used to resolve upstream metadata before evaluating grants, so a proxy registry pointed at an unreachable host answered the denied caller and the reader identically. Grants now run first; the proxy rows in reads are that fix's regression test.

Cargo's auth-required. Cargo sends a credential on a read only when the sparse index's config.json declares it, and this server emitted dl and api and nothing else — so a cargo read could not be authenticated at all, in either direction. cargo_registry_config now derives the field from whether an anonymous caller can read the registry, and the bare-token normalisation in extractors.rs is scoped by registry type rather than by the /api/v1/crates path, which covered neither the index nor the download. The cargo target is the regression test for both halves.

That ordering is recent, and these rows are its regression test

Grants used to be evaluated after the resolve. Both callers then got the same upstream error, there was no boundary to observe, and the eight proxy-only kinds had no authenticated coverage anywhere as a direct result — besides which every refused request first fetched metadata from upstream and cached it, so an unauthenticated caller naming coordinates nobody had asked for could spend the upstream's rate limit one refusal at a time. The forge ref path already refused to do that; handle_resolved and resolve_metadata_for_inner now do too. Swap the two steps back and every proxy row in reads goes red.

The rule chain stays below the resolve, because it genuinely needs what the resolve produces: RuleContext.package is the metadata and the release-age gate reads published_at off it. A latest request is only known to be blocked once it has resolved to a version, which is why the block list cannot move up with the grants.

The live targets — a client boundary needs a real upstream

reads measures the route boundary everywhere, hermetically. The client boundary is a different claim and needs both arms: the denied caller stopped, and the identical request working for the reader — without that positive control a client refused for an unrelated reason reads as a correct denial. The positive arm can only succeed if the artifact exists, and nine kinds have no local mode to publish one into: a forge, a Node distribution, a JDK broker, a file tree and two OS archives have nothing to publish into.

So authz.sh live:<kind> exists, with its own config (tests/heavy/config.authz-live.toml) pointed at real upstreams. It is the one family in this suite that reaches the internet, which is why CI runs it nightly (heavy-authz-live) rather than per-PR, and why it is a separate file rather than more rows in the hermetic one. task test:authz-live-heavy runs all eleven.

Kind	Client	How the credential reaches the server
`goproxy`	the Go tool	userinfo in `GOPROXY` — not `~/.netrc`, which the Go tool reads for HTTPS only, and this tap is plain HTTP
`nodedist`	nvm	`NVM_AUTH_HEADER`
`sdkman`	`sdk`	userinfo in `SDKMAN_CANDIDATES_API` / `SDKMAN_BROKER_API`
`github`, `forgejo`, `gitlab`	mise	`~/.netrc` — mise's `netrc` setting defaults to `true`
`generic`	curl	`-u`, or a header
`deb`	apt	`/etc/apt/auth.conf.d/`, redirected into the run with `Dir::Etc::netrcparts`
`rpm`	dnf, in AlmaLinux 9	`username=` / `password=` in the `.repo`
`pacman`	pacman, in Arch	`XferCommand` — `pacman.conf(5)` documents no credential directive, so the documented custom downloader is the mechanism
`rustup`	rustup	userinfo in `RUSTUP_DIST_SERVER` — no token flag, no credential file, no `~/.netrc`

The token always travels in the password field: auth/token.rs accepts a Basic header and reads the token out of it, which is what the console's own withCredentials snippet writes.

Two of these were documented wrongly, and the phases are what found it

The console's nodedist and sdkman snippets both said "libcurl reads ~/.netrc without being asked". It does not: curl has never read ~/.netrc without -n. Worse for nvm, which downloads with curl -q — that also disables ~/.curlrc, so neither file can reach it, and the documented setup would have authenticated nothing while looking configured. nvm has a first-class NVM_AUTH_HEADER instead; sdk passes no -q, so a ~/.curlrc containing netrc does work for it, and userinfo in its two API URLs is simpler again. Both notes in ui/src/config/registryTypes.ts are corrected. The ~/.netrc tab still shown for nodedist should follow.

Where the client, not the harness, is the obstacle

Kind	The obstacle	Where the claim is made instead
`vscode-marketplace`	Stock VS Code has nowhere in `product.json` to put a token: the editor sends no credential to the gallery it is pointed at	Proven, in the other repository. `batlehub-vsx`'s `tests/heavy/view.sh` drives a real VS Code web build with the project's extension installed, and asserts `PROXY-OK` — the credential the extension writes authenticates the gallery — alongside `CONTRACT-OK` (written `0600`, refresh owned by the CLI, token never logged) and `BROKER-OK`. That is the answer to the obstacle, and it is the reason the fork and the extension exist (RFC 0023)
`openvsx`	Same, when the client is the editor. `ovsx` itself takes a `--pat`, but that is the publish credential	The `openvsx` target covers `ovsx`; the editor path goes through the same fork
`jetbrains-marketplace`	The IDE's `idea.plugins.host` has no documented credential field; the repo's own snippet authenticates with curl, not with IntelliJ	Route level only. An IDE-level test needs the mechanism to exist first

Where the coverage stands

Twenty-one of the twenty-four registry kinds are covered at both levels — the route, over curl and hermetically, and the client, with the real package manager carrying a real credential. jetbrains gets both from one row, because a path proxy has no protocol for a package manager to speak and its client is curl.

The three in the table above are the remainder, and none of them is blocked by this harness: stock VS Code has nowhere to put a token, and IntelliJ's plugin host has no documented credential field. vscode-marketplace is nonetheless proven at client level — in batlehub-vsx, by the extension that exists for exactly this.

Its config (tests/heavy/config.authz.toml) is deliberately inert in every respect but grants: every registry is local except Terraform, no rule gate is configured, and every namespace declares visibility = "public". A refusal under it is the grant hierarchy's, and a test whose denial has three possible causes proves none of them.

Two things it does not claim: group: subjects, which need an identity carrying groups and so belong to crates/web/tests/dynamic_groups.rs; and the expired half of shadow mode, which config load refuses to let anyone write (crates/web/tests/grants_shadow.rs holds it).

Terraform is the one ecosystem whose whole install is asserted against a closed registry. It authenticates the two JSON documents of a provider install and then fetches the archive, its SHA256SUMS and the .sig with no Authorization header, so signed_downloads = true and [server.signed_urls] are what let authz-tf grant anonymous nothing and still serve a complete terraform init — RFC 0012's entire claim, and covered by no heavy suite before this one. The phase asserts the install completes, that the artifact requests carried bh_sig, that the same URL with the signature stripped is 403 (or the install was served by an open registry and every signed-URL assertion is vacuous), and that the minted document is no-store — it is a bearer capability, and a shared cache would replay one caller's signature to the next.

The signing target covers the three properties a single terraform init cannot show, each needing a clock or a key change:

Binding. A capability minted for shasums presented on the shasums.sig route must be 403. The payload names art; this is where that name earns its place, and without it one minted URL opens every artifact of the version.
Expiry — both edges. A token is not dead at exp: verify_at refuses at exp + CLOCK_SKEW_SECS, a deliberate backward-clock allowance so a runner a minute behind the minter does not fail every install. The phase asserts it is still good just past exp and refused past the allowance, which is what makes the second assertion mean "expiry is enforced" rather than "something refused eventually". The skew is read out of signed_url.rs at run time rather than copied, so it cannot drift into a sleep.
Rotation, in both directions. Mint under secret A; rotate to B with previous_secrets = [A] and the old URL must still work — an operator who rotates and breaks every in-flight install has caused a worse outage than the one they were avoiding — and the new mint must differ, or B is not signing. Then retire A and the old URL must be refused, or retiring a key does not retire the capabilities it signed.

It drives the key changes through POST /config/from-content + pending/apply against a scratch config, never the file the server watches: that watcher polls every two seconds and only stages a pending, so editing the watched file races load_pending_from_content's byte-identical dedup — whichever load wins marks the content seen, the other returns pending_created: false, and the apply answers "no pending reload". One write also fires the watcher up to five times, which is its rate limit, so it disables itself mid-run. The phase asserts pending_created rather than trusting the 2xx, because the dedup path is a success that changed nothing.

Two clients cannot carry an identity on a read, and the phases say so rather than pretending otherwise. ovsx get sends no credential at all — not a header, not the ?token= it uses on publish. And dotnet restore attaches packageSourceCredentials to HttpClientHandler.Credentials, which .NET offers only in answer to a 401 carrying WWW-Authenticate; this server refuses an unauthenticated read with a bare 403, so the credential is never sent and a caller holding every read verb is refused exactly like one holding none. For both, the phase drives the client through a publish into a sealed namespace — which does carry a credential, and which a seal refuses to everybody including the administrator — and asserts the read boundary over curl beside it. The NuGet phase pins the missing challenge, so if the 403 ever grows a WWW-Authenticate the run says which arms to restore.

7-quater. Soak tests (leak detection)

Every layer above asks whether the server gives the right answer. This one asks whether it gives the memory back — and nothing else here can, because a leak is not a wrong answer. It is the same right answer, a few kilobytes more expensive each time, until a pod that served correctly for nine days is OOM-killed on the tenth with no failing test to point at.

Neither suite runs on a push or a pull request. A soak measures a slope, so running it per commit would cost every pull request an hour to answer a question no single commit changes the answer to. Run one when something suggests it: a report of memory that does not come back, a change to a streaming, caching or pooling path, or before a release.

The shape both suites use

warm-up load ──▶ quiesce ──▶ BASELINE ──▶ steady load ──▶ quiesce ──▶ FINAL

Both measurement windows are idle, and the baseline is taken after a warm-up rather than at startup. That ordering is the whole design:

measuring at startup reports every lazily-filled cache, connection pool and allocator arena as a leak — so the first run fails, and the threshold then gets raised until nothing can fail;
measuring under load compares a number that includes in-flight request buffers against one that does not, which is a different quantity in each window and cannot be subtracted;
quiescing before each window lets the server drain, and lets jemalloc (on by default, server/Cargo.toml) decay its dirty pages, which takes seconds and would otherwise read as growth.

What is left is what was still held with nothing in flight, which is what "leak" means.

The third bullet is a property of the build, not of the wait. jemalloc's decay runs on allocator activity: a page goes dirty, its deadline passes, and the purge happens on the next allocation that ticks that arena — so a process with nothing in flight purges nothing, however long you quiesce it. That is why server/Cargo.toml enables tikv-jemallocator's background_threads feature, which is not implied by the default background_threads_runtime_support: without it the crate compiles jemalloc with background_thread:false.

Measured on the same binary and the same 60-second quiesce, 10m at 100 req/s across 24 registry kinds:

	idle RSS	trend	verdict
purging off	272.5 → 300.5 MiB (+10.3 %)	1.27 MiB/min	FAILED
purging on	258.4 → 262.9 MiB (+1.7 %)	0.28 MiB/min	passed

Nothing about the server differed between those two runs. A build that drops it — --no-default-features, or _RJEM_MALLOC_CONF=background_thread:false — makes this gate measure retained arenas instead, and it fails by comparing a window that follows ten minutes of load against one that follows sixty seconds of it.

Five resources are compared, because they fail differently and a leak in one is invisible in the others:

Signal	Where it comes from	What it means when it grows
RSS at idle	`/proc/<pid>/status`	memory still held with nothing in flight
Live heap at idle	`batlehub_memory_allocated_bytes` from `/metrics`	the program is holding more objects — the row that separates a leak from an allocator keeping pages
Open descriptors	`/proc/<pid>/fd`	a socket or file not closed — ends in `EMFILE` on accept, which looks like a network fault
Threads	`/proc/<pid>/status`	a spawned worker that never joins
Connections held	`batlehub_db_pool_size` − `..._available_connections` from `/metrics`	a handler that took a pool connection and did not return it

The second row is the one to read first when the first row is red. RSS counts pages and an allocator may hold pages the program has already freed, so the two rows disagree exactly when the allocator is the answer: RSS up, live heap flat is fragmentation or decay, both up is a leak. That distinction cost four ten-minute runs and two seventeen-minute builds to establish once by A/B, which is why the process now reports it. It is limited at 5 % rather than RSS's 10 %, because it carries no bookkeeping — and on a build without the jemalloc feature it reads not measured, never zero.

Those series come from server/src/allocator.rs, which publishes jemalloc's allocated, active, resident, mapped and retained every five seconds. They are worth a dashboard panel beyond this gate: resident − allocated is what the allocator is holding on the program's behalf, and it is the difference between a memory limit that needs raising and a bug that needs fixing.

A sixth signal is not a growth comparison at all: the slope of RSS during the sustained load, by least squares. A leak slower than the run is long shows up as a line that never flattens, hours before the idle windows differ enough to fail. (Least squares rather than last-minus-first: RSS sawtooths with every cache sweep, and two endpoints landing on different teeth is a number with no relationship to the trend.)

Three bounds keep that signal honest. The fit skips the first third of the load, because the start is caches and pools filling rather than a trend; under five minutes of sustained load the slope is reported but not judged — measured at 10.5 MiB/min on a 30-second window whose idle comparison was +4.2 %, which was the fill and nothing else. A window that short cannot tell a leak from a cache warming up, and a gate that cannot tell should say so rather than guess.

The third is that the limit is compared against the lower bound of the fit's 95 % interval, and the row prints the interval (2.08 ±0.55 MiB/min). What the fit crosses is not a line with noise on it but a sawtooth: on the 24-kind mix the residual scatter is 9–14 MiB and the peak-to-peak swing 48–73 MiB, where the 2.00 MiB/min limit describes 13 MiB across the same window. Three runs of the same plateau fitted 0.28, 1.27 and 2.08 MiB/min purely by which tooth the window opened and closed on — and the last of those failed a gate the other two passed. Planting a synthetic leak on those same three runs measures what the interval costs: +2.5 MiB/min is caught in all three, +2.0 in two of three. A number the run cannot resolve is not a verdict.

`task perf:soak` — synthetic, fast, precise

perf/scripts/soak.sh starts its own mock upstream and server, seeds, and runs perf/k6/scenarios/10_soak.js twice — once to warm up, once to hold the load — sampling the process once a second throughout. perf/scripts/soak_verdict.py turns the samples into a table, an RSS sparkline and an exit code.

bash

task perf:soak                                  # 10 minutes at 100 req/s
task perf:soak DURATION=1h RATE=200             # overnight
task perf:soak PROFILE=debug DURATION=30s RATE=20   # smoke-test the harness itself

The scenario uses a constant arrival rate, not constant VUs: under constant-vus a server that slows down is offered less work, so the very degradation a soak looks for hides itself by reducing the load that causes it. The mix is rotated deterministically rather than sampled at random, so two runs of the same duration issue the same requests and their curves are comparable — 60 % artifact reads, 16 % listing documents, 10 % upstream misses, 6 % pooled API reads, 5 % per-request SBOM builds, 3 % publishes. Those are six different allocation paths, which is the point: a leak is a property of a code path, not of a request count.

Thresholds are environment variables, so a run can be made stricter without editing anything:

Variable	Default	Fails when
`SOAK_MAX_RSS_GROWTH_PCT`	`10`	idle RSS grew more than this, as a percentage
`SOAK_MAX_HEAP_GROWTH_PCT`	`5`	idle live heap grew more than this — jemalloc builds only
`SOAK_MAX_RSS_SLOPE_MIB_PER_MIN`	`2.0`	RSS trended upwards faster than this under load — compared against the fit's 95 % lower bound, not the fit
`SOAK_MAX_FD_GROWTH`	`16`	this many more descriptors are open at idle
`SOAK_MAX_THREAD_GROWTH`	`4`	this many more threads are running
`SOAK_MAX_POOL_GROWTH`	`2`	this many more pool connections are held at idle

Two further choices in perf/config.soak.toml are measurement decisions rather than deployment ones, and both were made after a run failed for the wrong reason:

max_connections = 10, against the perf config's 50. A large pool hides a connection leak for the length of any run anyone would sit through; a small one exhausts it while the run is still going, and the verdict names it.
[cache] type = "postgres", against the default in-memory cache. An in-memory metadata cache lives in the process's own RSS, so every cached document is memory the idle comparison sees — not wrong, since it is resident, but not a leak either, and it grows with the length of the run. Measured: +11.2 % idle RSS on a 40-second run whose only fault was having cached more than its 25-second warm-up did. A gate that fails on a correct cache teaches people to raise the threshold. With the cache in a table, what is left in RSS is what the process is holding — and it is also what a real deployment runs.

The workload is stationary for the same reason — and that has to hold for every arm, which is the part that is easy to get half-right. The cache-miss arm draws from a bounded set (BATLEHUB_SOAK_MISS_SPACE, 500 by default) and so does the publish arm (BATLEHUB_SOAK_PUBLISH_SPACE, 100); the publish arm was unbounded at first, and the first real ten-minute run failed on an RSS trend of 2.21 MiB/min against a 2.00 limit after some 1 800 brand-new package versions — a margin at which a workload-driven drift and a slow leak look exactly alike. Past its first pass that arm publishes duplicate coordinates and is answered 409, which is a weaker exercise than a fresh write but still runs auth, the parse, the quota check and the existence check.

The reasoning, for either arm: an unbounded miss space adds a cache entry, a row and a stored object per request, forever, so memory climbs for as long as the run lasts and every long soak "fails". That growth is the workload's, not the server's, and no threshold can tell the two apart. A leak test needs a workload whose steady state exists, so that anything still climbing after the caches have filled is the process's doing.

For the same reason the idle-RSS comparison carries the slope's bound: under five minutes of load it is reported and not judged, because the caches are still filling and "idle RSS grew" is then "the cache got bigger". Descriptors, threads and pool connections have no warm-up and are judged at any length.

What it reports

Three things, in the report and in the pull request comment:

The verdict table — the four idle-window comparisons and the slope, above.

A chart of the run. A text chart of RSS and open descriptors in the comment itself, because that renders in a terminal, a job summary and a comment alike; and an SVG (soak-chart.svg, beside the report in the run's artifacts) with all four curves and the phases shaded, so the two idle windows the verdict compares are visible as windows. Each curve is scaled to its own range and says so — RSS in hundreds of MiB and a descriptor count in tens share no axis, and drawing them on one flattens the smaller into the straight line a leak in it would live on.

Which registry cost the most. Ranked by seconds spent inside the handler, read from the server's own /metrics scraped at both ends of the load and subtracted. The server is the only party that knows what a request cost it: a load generator can say how many requests a registry took and how many bytes came back, and neither is how long the server spent nor how much it had to pull from upstream to answer them. Beside the ranking, per registry: requests, ms/req, bytes pulled from upstream, artifact and document misses, and the coordinates resolved from a listing already held — the upstream round trips it did not make.

That ranking is why perf/config.soak.toml declares more than one registry. "Which registry is the worst consumer" is not a question a single-registry run can answer, and the shapes have to differ or it ranks them by traffic rather than by cost: an npm read is a small document and a small artifact, a RubyGems compact index is the whole registry in one document, which is the expensive shape this project has measured before (RFC 0015 §11.7).

Every registry kind, and what that does and does not prove

The soak declares a registry for every kind in RegistryKind::ALL and drives each of them, because a leak lives in a code path and not in a request count: each kind brings its own client, its own document parser and its own rewriter, so a soak that drove npm alone would be as green against a kind that leaked a megabyte per listing.

Four files have to agree for that claim to hold, and three tests in crates/web/tests/soak_kind_coverage.rs make them:

File	What it holds
`perf/mock-upstream/src/protocols/`	one module per protocol — the documents the proxy's client parses
`perf/config.soak.toml`	a registry per kind, pointed at the mock
`perf/k6/soak_arms.js`	the arms: `{op, kind, registry, weight, expect, request(n)}`
`soak_kind_coverage.rs`	`NOT_SOAKED`, the kinds deliberately left out — currently empty

The tests refuse a kind that is neither driven nor excused, a registry no arm asks for (it would report as a registry that costs nothing), and — the one that makes the count mean anything — an arm whose kind is not the type of the registry it addresses. Without that last check an arm can claim conda while pointing at the npm registry and every file still reads correctly on its own.

The pre-flight is the gate that matters. perf/k6/scenarios/11_soak_arms.js asks for each arm once and requires the status the arm declares; soak.sh runs it before the warm-up and stops the run on a failure. It exists because the load's own check is "not 5xx", so an arm whose URL is wrong or whose upstream route was never written answers 404 and passes — and the report then shows that registry costing nothing, which reads exactly like a cheap registry. When the nineteen kinds beyond the original five were added, the pre-flight caught eight arms that looked right and were not: a dl_path that resolved to …/cargo/cargo/…, a wheel filename whose hyphen was parsed as the version boundary, three forge asset downloads that resolve the tag before the release, and a HEAD the mock answered with 404 because actix does not derive one from #[get].

What soaking a kind proves. That its client, parser, rewriter, cache and storage path hold up under constant load without the process growing. That is the question a soak asks, and it is the whole of it.

What it does not prove, and none of these are gaps in the soak — each has its own instrument:

That a real client would accept what is served. The mock answers what the proxy parses, not what apt, terraform or an editor would. That is tests/heavy/closed_world.sh, one phase per kind, driving the actual client.
The cost at real scale. Conda is the sharpest case: the mock's repodata.json is a few hundred entries, where conda-forge's is ~424 MiB and costs ~11.5 GB parsed as a Value — the measurement that drove the streaming filter and the sharded index. The arm exercises the code path; it says nothing about the memory profile at channel scale.
Two routes that cannot be driven through /proxy/{registry}/. Terraform's .well-known/terraform.json is served only on a host bound to a single registry (RFC 0001), and VS Code's /vscode/item is a 302 to the console's own package page rather than an upstream read. Both are routes left out, not kinds: the Terraform provider versions, its mirror index and the VS Code gallery query are all in the mix.
The write path, beyond npm. One local registry publishes; every other registry in the soak is a proxy. A publish arm per kind would be a different suite.

Two things the ranking does not count, and says so in its own footnote:

local-mode traffic. A local registry's publishes and reads go through LocalRegistryService, which emits neither metric, so a local registry is absent from the table rather than cheap.
nothing else. Listing reads are counted — they were not, until this work: batlehub_requests_total and batlehub_request_duration_seconds were emitted from the artifact path alone, so a registry serving nothing but documents recorded no requests at all and anything ranking registries by cost ranked it last for free. They are now recorded for the listing routes too, under outcome = listing | listing_denied | listing_error, and a path that declines (conda's byte route answering "not this path" before the parsed one runs) records nothing rather than counting the request twice.

Starting one, and where the result lands

.github/workflows/soak.yaml has two ways in, and neither runs by itself:

	how	when to use it
Label	add `soak` to a pull request	any branch, including one whose workflow file has not landed on `main`
Dispatch	Actions → Soak → Run workflow, or `gh workflow run soak.yaml --ref <branch>`	once the file is on the default branch; gives the full input form

The label exists because of a GitHub rule worth knowing: a workflow whose only trigger is workflow_dispatch is not registered at all — and cannot be dispatched on any branch — until the file has landed on the default branch. Measured on this repository while adding it: pr-checklist.yaml ran from its own branch, because a pull request is an event GitHub associates with that branch, while this workflow was absent from /actions/workflows entirely and gh workflow run had nothing to call. A label is a pull_request event, so it runs the branch's own copy of the file, on the branch's own code.

On the label path inputs.* is empty, so every input carries the same default the dispatch form offers — a threshold read as "" would parse as zero and fail on the first byte of noise. The label is removed when the run finishes, so adding it again re-runs the soak.

The third job collects both suites' reports, joins them under one heading and posts them as a single comment, edited in place on every re-run. A labelled run carries its own pull request; a dispatch resolves the open one whose head is the branch, or takes the pr input. With no pull request to find, the report is in the job summary and the run stays green: the verdict is the soak's to deliver, and a comment that could not be posted is not a leak.

`task test:soak-heavy` — a real client, in a loop

tests/heavy/soak.sh is the same shape driven by npm: a round is npm install from a proxied registry, npm publish to a local one, and npm view of what it just published, with npm's own cache removed each round so it asks the server rather than itself. It exists beside the k6 suite because k6 offers the requests this project thinks a client makes, and npm offers the ones it actually makes — every registry defect this project has shipped was found by a client and not by a test double (RFC 0009 §5.1), and there is no reason a leak would be different.

bash

task test:soak-heavy                            # 25 rounds
SOAK_ROUNDS=500 task test:soak-heavy            # a long one

The upstream is a directory the suite serves (upstream_dir.sh), never a real registry: a soak offers the same request thousands of times, and the served directory's access log also makes "the proxy answered from its cache" a count. After a warm-up that touches every package, the loop must reach the upstream exactly zero times; anything else is a cache that is not holding, and would also make the growth numbers meaningless, since the loop would then be measuring the upstream path rather than the served one.

The suite also pins what a first install costs upstream. The warm-up's own count is printed, and it is the number that caught a real defect: a first read of one npm package used to cost three packument fetches — the client's own, plus two more from the artifact route resolving the coordinate twice over. It is one now. See RegistryClient::fetch_artifact_resolved and ::resolve_metadata_from_document, which are the two halves of that fix, and note that both are default-None hooks: a kind that does not implement them is unchanged.

One trap is worth naming, because it cost a green run that measured nothing. heavy_start_server launches setsid cargo run, so $HEAVY_SERVER_PID is not the server — and /proc/<cargo>/status answers every question with a plausible number rather than an error. Both suites therefore resolve the process to measure, by executable name and config path, and refuse to run if they cannot find it.

8. CLI integration tests

cli/tests/integration.rs — a single file with ~83 test functions. It:

Starts a genuine actix-web HttpServer on in-memory backends (TestServer::start) and waits for the port.
Invokes the CLI as a subprocess via Command::new(env!("CARGO_BIN_EXE_batlehub-cli")), so cargo builds the binary automatically before the test run.
Seeds admin-visible packages directly with TestServer::seed_package().

In-memory store separation caveat. InMemoryLocalRegistry (used by LocalRegistryService for publish/yank/delete) and InMemoryPackageRepository (used by AdminService for package list/block) are separate stores in tests, though they share tables in Postgres. Consequence: a package published via the local-registry HTTP endpoint does not appear in GET /api/v1/packages. Tests use seed_package() for package list, and query registry-specific endpoints (e.g. the NuGet flat-index at /proxy/{reg}/nuget/v3/flat/{id}/index.json) to verify yank/unyank/delete state.

Flows exercised include: registry list/info/suggest (from mise.lock, Cargo.toml, …), auth whoami/token/login (kubernetes,oidc)/refresh, package list/versions, version yank/unyank/delete, CLI publish for nuget/npm/pypi/cargo (with a publish→proxy round-trip), admin (quota, ip-block, banner, cache clear/warm, config reload/changes, audit-log list/purge/export, stats, health, visibility, users, namespace, sbom, bulk ops, access-check, notifications), config show/set, completion, setup detect (per-manifest detection with depth / monorepo / hidden-dir handling), setup ide, and registry-types.

9. Example-project tests

crates/examples/tests/*.rs — in-memory backends, no external DB, run in CI via batlehub-examples --test '*':

local_registry.rs — local-mode upload/pull cycle through a real BatleHub app.
real_proxy.rs — the real proxy against public upstreams; each test skips gracefully (early return;) when the toolchain or network is unavailable.
smoke.rs — end-to-end smoke tests for every example project, with a recording HTTP server standing in for the proxy.
structure.rs — project-structure assertions.

10. Fuzz targets

fuzz/fuzz_targets/ (libfuzzer, task fuzz) — all fuzz batlehub-core domain logic:

fuzz_rbac_evaluate.rs — RbacRule against a reference model built from expand_patterns: the decision matches the model for all three roles at once, a higher role never loses a verb a lower one holds, and joining a group never revokes one.
fuzz_deny_latest.rs — DenyLatestRule; asserts only the exact string "latest" denies (unicode homoglyphs / whitespace must neither bypass nor over-block), and that a role in the bypass list is let through.
fuzz_release_age.rs — ReleaseAgeGateRule (durations capped at one year), with and without a publish date and under both deny_missing_timestamp settings. The rule reads the clock itself, so the oracle brackets the call with two readings and only decides when both agree.
fuzz_package_id_cache_key.rs — PackageId cache-key generation is deterministic and always contains the registry component.
fuzz_path_safe.rs — the traversal guard every storage key goes through (validate_coordinate, validate_package_name): whatever it accepts builds a key with no .. segment at any percent-decoding depth, no NUL, no backslash, no empty component; whatever an independently written decoder can turn into a traversal, it rejects. Differential, so a change to the guard's decoder is what this target notices.
fuzz_integrity_parse.rs — the advertised-checksum parser and both verifiers: a parse never yields a digest of the wrong length, SRI and hex spellings round-trip, and the buffered and streaming verifiers agree under arbitrary chunking.
fuzz_normalize_url.rs — the repository-URL normaliser the package page links to: whatever comes out is http(s):// with a host, or nothing.
fuzz_scanner_extract.rs — scanners::extract::extract_to, the archive reader that unpacks a published package before a scanner runs on it: raw bytes, or tar / tar.gz / zip archives assembled from fuzzed entries (.., absolute paths, symlinks and hard links pointing out, forty-deep names, a NUL) under a fuzzed budget — nothing lands outside the root or in a canary directory beside it, the bytes and entries on disk never exceed the policy, the report matches the disk, and a refusal leaves the tree within budget too.
fuzz_sbom_extract.rs — the ten SBOM extractors (RFC 0007 §5.2) over archives whose members carry the manifest names and bodies each parser looks for, cut and corrupted, plus a gem's tar-in-tar: for every registry kind the call returns a manifest, never panics, answers the same input the same way twice, and reports nothing longer than an archive could carry.
fuzz_readme_render.rs — the README pipeline: for arbitrary input, in any format and under either image policy, the output carries no <script, no on*= handler and no scheme outside the allow-list, and rendering is stable.
fuzz_svg_sanitize.rs — the SVG allow-list: output is well-formed XML and reaches outside its own document nowhere.
fuzz_grant_resolution.rs — grant resolution over arbitrary hierarchies: the tier order holds, a seal stops inheritance, and widening never narrows.
fuzz_signed_url.rs — the signed download URL (RFC 0012): a minted token round-trips to the same identity, is refused at any other coordinate (including a / moved between package and version, which the display string cannot see), under any other secret, and — once edited — either fails or still means exactly what was minted. The netstring encoding's injectivity, stated as a property rather than as the one attack its unit test pins.
fuzz_listing_filter.rs — every (kind, document) pair of the blocked-version listing filters, over token-soup text and vocabulary-driven JSON: an empty block set changes nothing, only blocked versions are reported removed, the body keeps its representation, and filtering is idempotent — a second pass removes nothing more and changes no byte. dispatch_multi (conda, RubyGems /versions) is held to the same.
fuzz_bundle_read.rs — the air-gap bundle container: whatever read_bundle accepts, manifest_bytes_of accepts too and yields the same manifest (the duplicate-manifest.json bypass, as an invariant); every kept blob hashes to its name; a written bundle reads back equal; a bundle with bytes flipped or truncated never panics.
fuzz_version_order.rs — newest_first is a total preorder over any three strings (a comparator that is not one makes sort_by panic since Rust 1.81), is_prerelease ignores a v prefix, and best_latest answers from its input with the highest stable strict-semver entry, cross-checked against an independent semver read and against newest_first.
fuzz_escaping.rs — the four small encoders, each checked with an independent decoder written in the target: escape_html decodes back to its input and leaves no markup character; percent_encode_path_segment decodes to the input bytes and cannot leave its segment; csv::field reads back under RFC 4180 as the input, apostrophe-guarded exactly when it led a formula; parse_cache_control reports only what the header said.
fuzz_release_coordinate.rs — coordinate_from_filename, constructively: a name and a version drawn from each ecosystem's legal alphabet, formatted the way its tooling names a file, must parse back to exactly that coordinate, for every convention (.nupkg, .whl, .gem, .tgz, .crate, .vsix, pacman, conda, .deb/.rpm).
fuzz_image_host.rs — image_host_allowed, the README image allow-list that gates an outbound fetch: the URL is built from a chosen host (user-info, port, path, query, fragment, case-shuffled scheme, embedded tabs) and the answer must match the rule restated over that host.

A target is only as good as its oracle. The nightly job that runs these had never got past cargo-fuzz's default build target until 2026-09-10 (see the comment on the job), and the first five seconds of fuzz_deny_latest under it reported a finding that was the target's own assertion contradicting the rule's unit tests. Two other targets asserted nothing and could only catch a panic. When you write one, state what the code claims — from its unit tests or its doc comment — and prefer an oracle that computes the expected answer from an independent source (a model, a second decoder, a second verifier) over one that spells out a handful of cases. Then run it for a couple of minutes locally before trusting a green nightly.

The targets are a separate workspace, and that is a trap. cargo check --workspace, cargo clippy --workspace and cargo test --workspace do not see fuzz/, so a target can stop compiling against a type it uses and nothing says so — the module docs go on naming a guard that is no longer running. Four of the seven had drifted that way before task fuzz:check existed. That check is a plain cargo check over fuzz/Cargo.toml, needs no nightly, and runs on every PR in the Fuzz targets job of test.yaml; the same job fuzzes each target for 60 seconds on the nightly schedule and uploads any crash artefact.

Running one locally needs the nightly toolchain and cargo-fuzz. If a run produces a 0-byte crash-da39a3ee… artefact, that is LeakSanitizer failing to start under a restricted ptrace_scope, not a finding — re-run with ASAN_OPTIONS=detect_leaks=0.

11. Coverage

Gate: 80% lines. task coverage-check runs cargo llvm-cov report --fail-under-lines 80.
task coverage aggregates the workspace tests plus each adapter integration test explicitly (pg_cache, pg_vulnerability, local_registry, artifact_meta, pg_rate_limit, storage_router, s3_storage [+storage-s3], actions_oidc, and the Redis suite [+cache-redis]).
Excluded paths live in the COVERAGE_EXCLUDE variable in .tasks/coverage.yaml (one variable, read by both coverage and coverage-check, which share their whole collection step and so can no longer drift). They are code that is either environment-glue or exercised only against real infra: server/src wiring (main, server_factory, setup, stores, watcher), the OIDC auth handlers, crates/adapters/src/db/, the Postgres/Redis adapter implementations, cli/src/tui/ (terminal UI), and a few others. When you add code that genuinely can't be unit-tested, add it to COVERAGE_EXCLUDE with the same reasoning — don't lower the gate.

12. CI wiring

.github/workflows/:

test.yaml — the main Rust matrix:
- lint: cargo clippy --workspace -- -D warnings + cargo fmt --all --check.
- unit: cargo llvm-cov --workspace --lib --bins + cargo test --workspace --doc.
- integration: web (-p batlehub-web --test '*'), CLI (-p batlehub-cli --test integration), examples (-p batlehub-examples --test '*'), adapters default + Postgres, S3 (--features storage-s3 --test s3_storage), Redis (--features cache-redis).
- heavy-marketplace: bash tests/heavy/marketplace.sh (headless VS Code — the server-linux-x64-web build, no Electron — + IntelliJ).
- heavy-browser: vsx_login.sh, vsx_view.sh and console_fetch.sh in sequence against the runner's Chrome over CDP (the headless-chrome action); the editor download is cached, and vsx_view.sh's evidence directory is uploaded on failure.
- heavy-backends: bash tests/heavy/backends.sh (S3 via RustFS, Redis, and dex as a container, under npm and pip).
- heavy-vsx-extension: bash tests/heavy/vsx_extension.sh (the batlehub-vsx repository's own heavy suite — both modes of the extension in the VS Code web build, in the runner's Chrome — against this checkout).
- heavy-bundler: bash tests/heavy/bundler.sh (a real bundle install against a local rubygems registry).
- heavy-client (matrix): one job per ecosystem — npm, pypi, openvsx, conda, nuget, composer, terraform, nvm, sdkman, mise, cargo, go, maven, pathproxy, vscode_patch, che_code_patch, hybrid, quarantine, upstream_audit, airgap — each running tests/heavy/<suite>.sh. A matrix rather than twenty jobs because only the toolchain setup differs; fail-fast: false, because one unhappy client says nothing about the others.
- heavy-authz (matrix): one job per target of tests/heavy/authz.sh — matrix, then npm, pypi, openvsx, conda, nuget, composer, rubygems, terraform. Its own job rather than more rows in heavy-client because every row runs the same script with an argument, and the matrix row — the one that matters most and costs least — needs no client toolchain at all, so it must not queue behind seven installs.
Every heavy job runs the server under cargo llvm-cov, so what the client exercised counts toward the merged coverage table — the compact-index paths, the Terraform provider chain and the conda HEAD probe are reached by no other job.
Two heavy suites are not in any of those jobs and are run by hand: vsx_view.sh and console_fetch.sh. Both drive a real browser over the DevTools protocol — one against the VS Code web build's workbench, the other against the built console — and both want a Chrome and a warm download cache that the matrix rows do not carry. They are the same exception heavy-marketplace is, for the same reason.
test.yaml also runs nightly (50 23 * * *). The heavy jobs are why: they drive clients fetched from outside this repository — Bundler and npm from their own registries, pinned VS Code, IntelliJ, Terraform, .NET and micromamba builds — so a new client release can break a tree that no commit touched, and only a scheduled run finds it.
Every job in it hangs off one gate job, and the gate has one rule: a pull request whose head branch starts with dependabot/ runs nothing unless it carries the full-test label. Dependabot opens a pull request per bump, daily, across four ecosystems, and each one used to buy the whole workflow — a Cargo.lock bump paying for a real ansible-galaxy, a real dotnet restore and a headless VS Code to prove it had not broken them. It almost never has; when a bump could (an actix-web, sqlx or aws-sdk-s3 line, anything the security constraints in CLAUDE.md name), label the pull request and the full suite runs on the spot.
Nothing else changes: a human branch, a push to main, the nightly schedule and workflow_dispatch all run the suite as before, because github.head_ref is empty outside a pull request and the dependabot test is then false. The gate writes the reason for a skip into the run summary, so fourteen skipped jobs are never unexplained.
The label has to be a trigger as well as a condition — pull_request declares labeled in its types, or adding full-test to an open pull request would change the answer with no run left to ask the question. That trigger fires on any label, so the gate's other clause drops a run whose label is not full-test: a soak or breaking-point label no longer re-runs a suite that already ran on the push.
A new job in this file must declare needs: gate and if: needs.gate.outputs.run == 'true' — a job that forgets it is a job dependabot still pays for, and nothing fails the build over it.
soak.yaml — the two soak suites, started by hand only: a workflow_dispatch, or the soak label on a pull request. No push, no cron, and no pull request without that label. Inputs for the duration, the arrival rate, the heavy suite's round count and all four growth thresholds, so a run can be made stricter from the dispatch form. Two jobs (k6-soak, heavy-soak), selectable, with a concurrency group so two soaks never share a runner's CPU — a resource curve measured beside another soak is a measurement of the runner. The samples and the report are uploaded on failure as well as on success, because that is when they are worth reading.
perf-report.yaml — on every release tag, runs scenarios 01–07 against a freshly built server, diffs the result against the previous published release's perf-report.json, and attaches the table and its machine-readable twin to the release. A record, not a gate, for the reason the soak is the other way round: a shared runner's noise is larger than most real regressions, so a threshold would fail honest releases — but peak RSS doubling between two releases is visible through any amount of noise, and nothing else was recording it. perf/README.md § The results table has the local equivalent (task perf:report).
pr-checklist.yaml — derives, from the paths a pull request changes, the obligations those paths carry (.github/scripts/pr_checklist.py), and posts them as one comment it edits in place. Items the diff suggests are missing — a migration with no mig! entry, a generated file that did not move with its source, a new RegistryKind with no entry in the UI's type table — are marked. It is advisory and never fails a build: a required check that turns green when somebody ticks a box measures nothing but the ticking.
front-test.yaml — frontend (ui/): install, regenerate the OpenAPI spec
- TS client, pnpm run coverage.
repo-interop.yaml — bash tests/interop/verify.sh (apt + dnf + pacman accept signed repos).
sonar.yaml — SonarCloud: rebuilds full Rust + frontend coverage and uploads lcov.info. Carries the same dependabot gate as test.yaml, for the same reason — rebuilding both coverage trees costs about what that whole workflow does — written inline on its single job rather than as a gate job, because one job needs no job to hang off.

The .github workflows start their own Postgres/RustFS/Redis service containers in YAML; the task test:* targets are the local-dev equivalents (Podman). .forgejo/workflows/ handles image builds, Helm, the website, and updatecli — the Rust test matrix runs on the GitHub side.

Testing ​

1. Test taxonomy ​

2. Running the tests ​

3. Unit tests ​

4. In-process integration tests ​

5. Per-registry local-registry tests ​

6. Path-traversal guard tests ​

7. External integration tests (real infra) ​

7-bis. Heavy client tests (a real package manager) ​

The config generator's harness ​

7-ter. The authorization suite ​

Which kinds are covered, and which are not ​

The live targets — a client boundary needs a real upstream ​

Where the client, not the harness, is the obstacle ​

Where the coverage stands ​

7-quater. Soak tests (leak detection) ​

The shape both suites use ​

task perf:soak — synthetic, fast, precise ​

What it reports ​

Every registry kind, and what that does and does not prove ​

Starting one, and where the result lands ​

task test:soak-heavy — a real client, in a loop ​

8. CLI integration tests ​

9. Example-project tests ​

10. Fuzz targets ​

11. Coverage ​

12. CI wiring ​

Testing

1. Test taxonomy

2. Running the tests

3. Unit tests

4. In-process integration tests

5. Per-registry local-registry tests

6. Path-traversal guard tests

7. External integration tests (real infra)

7-bis. Heavy client tests (a real package manager)

The config generator's harness

7-ter. The authorization suite

Which kinds are covered, and which are not

The live targets — a client boundary needs a real upstream

Where the client, not the harness, is the obstacle

Where the coverage stands

7-quater. Soak tests (leak detection)

The shape both suites use

`task perf:soak` — synthetic, fast, precise

What it reports

Every registry kind, and what that does and does not prove

Starting one, and where the result lands

`task test:soak-heavy` — a real client, in a loop

8. CLI integration tests

9. Example-project tests

10. Fuzz targets

11. Coverage

12. CI wiring