gal/constructionbot
Watch
2
0
Fork
You've already forked constructionbot
 0
Code Issues 8 Pull requests Projects Releases Packages Wiki Activity Actions

T-2 — Migrate constructionbot from dynamic-contained to dynamic-nix #11

New issue
Open
opened 2026-08-21 21:47:41 +00:00 by jsutter · 0 comments
jsutter commented 2026-08-21 21:47:41 +00:00
Owner
Copy link

Labels: infrastructure, deployment
Depends on: T-1; deepnet/dit-client#90; deepnet/dit-client#91; Arc 0
cutover accepted.

Rationale

The app runs on the legacy podman path by a one-day timing accident, not a
decision. First commit 2026-08-12 16:29; the dynamic-nix archetype merged to
dit-client main the same day (PR #45) and shipped in ditc v1.3.0 on
2026-08-13.

Spec 012 (deepnet/infra-template, specs/012-nix-native-app-execution) makes
nix-native the intended default for new apps and names two defects in the podman
model this app is living with:

  1. The podman bridge topology broke the ingress contract — every
    dynamic-contained app was unreachable until dit-client #31 added
    -p 127.0.0.1:PORT:PORT. This app carries the scar as a hand-maintained
    comment in ingress.yaml warning that upstream: 127.0.0.1:58402 and the
    deploy workflow's publish flag "move together."
  2. Double containerization — tenant apps already run inside an nspawn container,
    so podman adds a second, redundant isolation layer.

Two further wins specific to this repo:

  • The container build is the deploy tax. The dependency layer invalidates on
    every source edit; HEAD (ad91fb7) is a touch src/main.rs hack working
    around it. nix build .#default on a runner with a warm store is incremental
    by construction. This deletes the problem class rather than optimizing it.
  • Hardening becomes unweakenable. dit-nix-app-run injects
    DynamicUser=yes, PrivateTmp, ProtectSystem=strict, ProtectHome,
    ReadWritePaths=/data/apps/<slug> wrapper-side via systemd-run --property=.
    A regenerated workflow cannot strip them. Today's --read-only --cap-drop=ALL --tmpfs exist only because the workflow was hand-restored from commit
    6def9af; the current ditc podman generator emits no hardening flags at all,
    so the next ditc deploy would silently drop them again. That recurring
    gotcha dies with this migration.

Scope

  • Add flake.nix at the repo root exposing packages.<system>.default, built
    with rustPlatform.buildRustPackage. Reference implementation:
    deepnet/chess (symbio tenant) is a live Rust app on this exact path.
  • The derivation must emit exactly one binary at bin/constructionbot and the
    app must bind 127.0.0.1:58402. The port is fixed at build time — the
    wrapper passes no PORT env and no argv. config.rs's default_listen must
    reflect this.
  • poppler-utils (pdftotext) becomes a nix runtime dependency — wrap the
    binary so pdftotext is on PATH, replacing the apt-get install in the
    runtime stage.
  • Flip manifest.yaml dit.archetype to dynamic-nix and ingress.yaml
    type to dynamic-nix.
  • Regenerate the deploy workflow for the nix archetype. Re-apply the CI gates the
    current workflow carries and the ditc generator omits: cargo test --lib,
    cargo clippy --all-targets -- -D warnings,
    scripts/no_hardcoded_endpoints.sh, scripts/check_ingress_port.sh, and the
    secret-scrub step.
  • Delete Containerfile in the same commit as the flake lands (atomic
    migration — no intermediate state where both build paths exist).
  • Confirm the eight OpenBao secret refs resolve through the nix path's
    refs-file mechanism (root wrapper AppRole fetch) rather than the podman path's
    runner-side bao kv get + --env.

Acceptance criteria

  • nix build .#default succeeds on the self-hosted runner and produces
    bin/constructionbot.
  • The app serves constructionbot.apps.gallahercompanies.com behind Caddy
    forward-auth, and the X-Gal-Fwd-Proof header still validates
    (src/middleware.rs unchanged).
  • All eight secrets resolve; the app boots with Procore, Graph, sage, and
    FORWARD_AUTH_SECRET present.
  • /healthz returns 200; migrations applied on boot; pdftotext resolvable
    from the app's PATH — verify by processing a Morella fixture end to end.
  • dit-nix-app-run rollback returns the app to the previous store path and it
    serves 200.
  • The unit survives a tenant reboot via nix-app-runtime-restart.service.
  • Egress to Procore, Graph, sage, and the Entra token endpoint is enforced,
    not merely declared — depends on #90. Verify a disallowed destination is
    refused.
  • MemoryMax and TasksMax are set on the unit — depends on #91. Verify with
    systemctl show app-constructionbot. This bound matters: the app spawns
    pdftotext over vendor-supplied PDFs (untrusted input), and today
    --memory 512m --pids-limit 256 is the only thing stopping a hostile PDF from
    exhausting the tenant container and taking down co-tenant apps.

Rollback plan

Keep the dynamic-contained workflow on a branch until the nix path has served
production traffic for one full digest cycle (24h, so the 07:00 cron and the
mailbox poller both exercise). Reverting is: restore Containerfile + the
podman workflow, flip manifest.yaml and ingress.yaml back, push.

Do not start before

Arc 0 cutover is accepted. Do not change the deployment substrate under a system
Jake is mid-validation on, that writes legally binding records.

**Labels:** `infrastructure`, `deployment` **Depends on:** T-1; `deepnet/dit-client#90`; `deepnet/dit-client#91`; Arc 0 cutover accepted. ### Rationale The app runs on the legacy podman path by a one-day timing accident, not a decision. First commit 2026-08-12 16:29; the `dynamic-nix` archetype merged to `dit-client` main the same day (PR #45) and shipped in ditc v1.3.0 on 2026-08-13. Spec 012 (`deepnet/infra-template`, `specs/012-nix-native-app-execution`) makes nix-native the intended default for new apps and names two defects in the podman model this app is living with: 1. The podman bridge topology broke the ingress contract — every `dynamic-contained` app was unreachable until dit-client #31 added `-p 127.0.0.1:PORT:PORT`. This app carries the scar as a hand-maintained comment in `ingress.yaml` warning that `upstream: 127.0.0.1:58402` and the deploy workflow's publish flag "move together." 2. Double containerization — tenant apps already run inside an nspawn container, so podman adds a second, redundant isolation layer. Two further wins specific to this repo: - **The container build is the deploy tax.** The dependency layer invalidates on every source edit; HEAD (`ad91fb7`) is a `touch src/main.rs` hack working around it. `nix build .#default` on a runner with a warm store is incremental by construction. This deletes the problem class rather than optimizing it. - **Hardening becomes unweakenable.** `dit-nix-app-run` injects `DynamicUser=yes`, `PrivateTmp`, `ProtectSystem=strict`, `ProtectHome`, `ReadWritePaths=/data/apps/<slug>` wrapper-side via `systemd-run --property=`. A regenerated workflow cannot strip them. Today's `--read-only --cap-drop=ALL --tmpfs` exist only because the workflow was hand-restored from commit `6def9af`; the current ditc podman generator emits no hardening flags at all, so the next `ditc deploy` would silently drop them again. That recurring gotcha dies with this migration. ### Scope - Add `flake.nix` at the repo root exposing `packages.<system>.default`, built with `rustPlatform.buildRustPackage`. Reference implementation: `deepnet/chess` (symbio tenant) is a live Rust app on this exact path. - The derivation must emit exactly one binary at `bin/constructionbot` and the app must bind `127.0.0.1:58402`. **The port is fixed at build time** — the wrapper passes no `PORT` env and no argv. `config.rs`'s `default_listen` must reflect this. - `poppler-utils` (`pdftotext`) becomes a nix runtime dependency — wrap the binary so `pdftotext` is on `PATH`, replacing the `apt-get install` in the runtime stage. - Flip `manifest.yaml` `dit.archetype` to `dynamic-nix` and `ingress.yaml` `type` to `dynamic-nix`. - Regenerate the deploy workflow for the nix archetype. Re-apply the CI gates the current workflow carries and the ditc generator omits: `cargo test --lib`, `cargo clippy --all-targets -- -D warnings`, `scripts/no_hardcoded_endpoints.sh`, `scripts/check_ingress_port.sh`, and the secret-scrub step. - Delete `Containerfile` in the same commit as the flake lands (atomic migration — no intermediate state where both build paths exist). - Confirm the eight OpenBao secret refs resolve through the nix path's refs-file mechanism (root wrapper AppRole fetch) rather than the podman path's runner-side `bao kv get` + `--env`. ### Acceptance criteria - `nix build .#default` succeeds on the self-hosted runner and produces `bin/constructionbot`. - The app serves `constructionbot.apps.gallahercompanies.com` behind Caddy forward-auth, and the `X-Gal-Fwd-Proof` header still validates (`src/middleware.rs` unchanged). - All eight secrets resolve; the app boots with Procore, Graph, sage, and `FORWARD_AUTH_SECRET` present. - `/healthz` returns 200; migrations applied on boot; `pdftotext` resolvable from the app's `PATH` — verify by processing a Morella fixture end to end. - `dit-nix-app-run rollback` returns the app to the previous store path and it serves 200. - The unit survives a tenant reboot via `nix-app-runtime-restart.service`. - Egress to Procore, Graph, sage, and the Entra token endpoint is *enforced*, not merely declared — depends on #90. Verify a disallowed destination is refused. - `MemoryMax` and `TasksMax` are set on the unit — depends on #91. Verify with `systemctl show app-constructionbot`. This bound matters: the app spawns `pdftotext` over vendor-supplied PDFs (untrusted input), and today `--memory 512m --pids-limit 256` is the only thing stopping a hostile PDF from exhausting the tenant container and taking down co-tenant apps. ### Rollback plan Keep the `dynamic-contained` workflow on a branch until the nix path has served production traffic for one full digest cycle (24h, so the 07:00 cron and the mailbox poller both exercise). Reverting is: restore `Containerfile` + the podman workflow, flip `manifest.yaml` and `ingress.yaml` back, push. ### Do not start before Arc 0 cutover is accepted. Do not change the deployment substrate under a system Jake is mid-validation on, that writes legally binding records. ---
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gal/constructionbot#11
No description provided.