T-1 — Externalize the extraction prompt and business config out of the build #10

New issue

Open

opened 2026-08-21 21:47:41 +00:00 by jsutter · 0 comments

jsutter commented

2026-08-21 21:47:41 +00:00

Owner

Labels: enhancement, blocker:migration
Depends on: nothing. Can land during or immediately after cutover.
Blocks: T-2.

Problem

extraction_system_prompt is a ~1.5 KB string in config.toml:24. The
Containerfile copies config.toml into the image at
/etc/constructionbot/config.toml. Every prompt change therefore requires a
full release container build and redeploy.

Constitution rule V mandates re-running the Morella corpus on every change to
the prompt, model, or field mapping. So the highest-frequency loop in the
project — extraction calibration against the ≥80%/≤2-edits target — is coupled
to the slowest loop in the project.

This gets worse under dynamic-nix, not better. Nix derivations are hermetic
by design; a config file baked into the derivation is even harder to hot-swap
than one baked into an image layer. Externalizing config is therefore a
migration prerequisite, not a nice-to-have.

Scope

Move runtime-tunable values out of the image and into the app_settings table
(the mechanism already exists — migration 0012, plus the /settings page and
its frontend.rs handlers).

Move:

extraction_system_prompt
discipline_map
the cross-discipline vendor mappings (see open question 2 in the remaining-work
plan — this ticket should land after those mappings are settled, or carry
them as-is and let the settled values be a data edit rather than a deploy)
spec hints

Keep in config.toml (genuinely build-time / environment-shaped):

procore_base_url, procore_token_url, company_id, project_id
listen, timezone, mailbox address

Acceptance criteria

An operator changes the extraction prompt through the running app and the next
extraction uses the new prompt. No rebuild, no redeploy, no container restart.
sage::prompt_id() still derives from the effective prompt text, so
per-extraction prompt attribution survives — the prompt_id stored on an
extraction row identifies which prompt produced it.
The corpus suite (tests/extraction_corpus.rs) reads the effective prompt from
the same source the worker does, so a calibration run measures what production
will actually use.
Config precedence is explicit and documented: DB value wins; config.toml
supplies the seed/default on an empty table.
Fail-closed behaviour is preserved — an unparseable or absent DB value falls
back to the config default rather than sending an empty prompt.
Existing --check-config still validates.

Verification

Change the prompt in the UI, upload a Morella fixture, confirm the resulting
extraction row carries a new prompt_id.
Run the corpus suite before and after a prompt edit with no deploy in between;
confirm scores differ.

**Labels:** `enhancement`, `blocker:migration` **Depends on:** nothing. Can land during or immediately after cutover. **Blocks:** T-2. ### Problem `extraction_system_prompt` is a ~1.5 KB string in `config.toml:24`. The `Containerfile` copies `config.toml` into the image at `/etc/constructionbot/config.toml`. Every prompt change therefore requires a full release container build and redeploy. Constitution rule V mandates re-running the Morella corpus on *every* change to the prompt, model, or field mapping. So the highest-frequency loop in the project — extraction calibration against the ≥80%/≤2-edits target — is coupled to the slowest loop in the project. This gets **worse** under dynamic-nix, not better. Nix derivations are hermetic by design; a config file baked into the derivation is even harder to hot-swap than one baked into an image layer. Externalizing config is therefore a migration prerequisite, not a nice-to-have. ### Scope Move runtime-tunable values out of the image and into the `app_settings` table (the mechanism already exists — migration `0012`, plus the `/settings` page and its `frontend.rs` handlers). Move: - `extraction_system_prompt` - `discipline_map` - the cross-discipline vendor mappings (see open question 2 in the remaining-work plan — this ticket should land *after* those mappings are settled, or carry them as-is and let the settled values be a data edit rather than a deploy) - `spec hints` Keep in `config.toml` (genuinely build-time / environment-shaped): - `procore_base_url`, `procore_token_url`, `company_id`, `project_id` - `listen`, timezone, mailbox address ### Acceptance criteria - An operator changes the extraction prompt through the running app and the next extraction uses the new prompt. No rebuild, no redeploy, no container restart. - `sage::prompt_id()` still derives from the *effective* prompt text, so per-extraction prompt attribution survives — the `prompt_id` stored on an extraction row identifies which prompt produced it. - The corpus suite (`tests/extraction_corpus.rs`) reads the effective prompt from the same source the worker does, so a calibration run measures what production will actually use. - Config precedence is explicit and documented: DB value wins; `config.toml` supplies the seed/default on an empty table. - Fail-closed behaviour is preserved — an unparseable or absent DB value falls back to the config default rather than sending an empty prompt. - Existing `--check-config` still validates. ### Verification - Change the prompt in the UI, upload a Morella fixture, confirm the resulting extraction row carries a new `prompt_id`. - Run the corpus suite before and after a prompt edit with no deploy in between; confirm scores differ. ---

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gal/constructionbot#10

No description provided.

Rows
Columns