crewli/dev-docs/form-builder-migration-playbook.md

Form Builder — Migration Playbook

Operator runbook for migrating legacy registration_form_fields / person_field_values / registration_field_templates data into the new form_* structure. Companion to /dev-docs/ARCH-FORM-BUILDER.md §11.

---

Status (post-S1)

Legacy tables retained intentionally. The tables registration_form_fields, person_field_values, and registration_field_templates remain in the schema through end of S1. They are dropped atomically in the first S2 commit together with the removal of legacy controllers, services, requests, resources, policies, and routes. DevSeeder and FormBuilderDevSeeder no longer write to them; they hold zero rows in dev but the schema is preserved for environments with real legacy data that will be migrated via forms:migrate-legacy-data.

Until S2 lands, the legacy registration UI remains functional for any environment that depends on it.

---

When to use this playbook

Run this when promoting an environment that holds real legacy registration data into the new form-builder schema. For development machines starting from migrate:fresh --seed, no migration is needed — DevSeeder produces new-structure data directly.

---

Pre-flight audit

Before touching any data, confirm the working state matches expectations.

# 1. Working tree clean?
git status

# 2. On the expected branch?
git log -1 --oneline

# 3. Database backup exists? (production / staging only — your call for dev)

# 4. Verify the new tables are present and the legacy tables still hold data
php artisan db:show | grep -E "form_|registration_form_fields|person_field_values|registration_field_templates"

If the legacy tables are absent, the migration command will detect this and exit cleanly without doing work — there is nothing to migrate.

---

Step 1 — Dry run

Always start with a dry run. It writes nothing and prints the same summary table the real run would produce.

php artisan forms:migrate-legacy-data --dry-run

What to look for:

• The per-organisation log lines list every event that will get a new schema and the number of fields/submissions per event.
• The summary table shows projected counts for form_schemas, form_fields, form_submissions, form_values, form_templates.
• Exit code is 0.

If anything looks surprising — wildly off counts, organisations missing, templates duplicated — STOP and investigate. Do not proceed to step 2 until the dry-run output is what you expect.

---

Step 2 — Run the migration

php artisan forms:migrate-legacy-data

What happens:

• Per organisation, inside a transaction (one bad org rolls back only itself).
• Per event in that org: one form_schemas row + N form_fields rows.
• Per person with person_field_values for that event: one form_submissions row (status = submitted if person.status ∈ applied/approved/no_show else draft) + one form_values row per legacy value.
• FormValueObserver populates typed columns and the form_value_options pivot during the inserts (per ARCH §7.2).
• All registration_field_templates rows are wrapped in the ARCH §4.6.1 schema_snapshot shape and inserted as form_templates.
• The whole run is wrapped in App\Support\ActivityLog::suppressed(...) so the activity log isn't flooded by hundreds of logSchemaChange / logFieldChange entries from the bulk inserts.
• After migration completes, verification runs automatically. Final exit code is whatever forms:verify-data-integrity returns.

Idempotent: if a form_schemas row already exists for (org, event, event_registration), that event is skipped silently. Safe to re-run.

---

Step 3 — Verify

Verification runs automatically at the end of step 2. To re-run it standalone (or to verify state without touching data):

php artisan forms:verify-data-integrity

Or, equivalently:

php artisan forms:migrate-legacy-data --verify-only

Nine checks:

1. Schema coherence (purpose enum, custom_purpose_slug ⇄ purpose=custom, public_token uniqueness)
2. Field coherence (FK valid, field_type ∈ FormFieldType ∪ custom registry, is_filterable matches type, slug unique within schema, binding JSON valid against config/form_binding.php)
3. Submission coherence (subject_type/subject_id consistency, subject_type ∈ config/form_subjects.php, status ⇄ submitted_at)
4. Value coherence (orphans, unique pair, length, multi-value sanity, value_indexed only set when field.is_filterable)
5. User profile coherence (every user has one profile; reliability_score ∈ [0, 5])
6. Data migration counts (skipped when legacy tables absent)
7. Orphan records (pivot rows, section statuses)
8. Section/schema relation consistency
9. (--strict) Strict reachability: value.field.schema must equal value.submission.schema

Exit code 0 means all checks passed. Anything else is a real failure worth investigating before proceeding.

---

Step 4 — Spot-check a sample

After verification is green, eyeball a sample submission to confirm the shape is right.

php artisan tinker
>>> use App\Models\FormBuilder\FormSubmission;
>>> $sub = FormSubmission::with(['schema', 'values.field'])->latest()->first();
>>> $sub->schema->name;
>>> $sub->values->map(fn($v) => [$v->field->slug => $v->value]);

Compare against the legacy view of the same person's data in person_field_values to confirm nothing was silently lost.

---

Rollback

The data-migration command does not have a down() step. If a migration went wrong, the recovery path is:

1. Restore from backup (production / staging).
2. For dev: php artisan migrate:fresh --seed rebuilds everything from scratch.

The legacy tables are preserved during the migration — they are not modified. So even after running forms:migrate-legacy-data, the legacy data is still there (until S2 drops the tables). This means a production data-migration error can be recovered by truncating the new form_* tables and re-running with the underlying legacy data still intact.

---

Common gotchas

• registration_form_fields table absent: the command detects this and exits 0 silently. Expected behaviour for fresh environments.
• PII heuristic miss: the heuristic in §11.2.1 of the ARCH covers obvious patterns (email, phone, geboort, allergie, …) but is not exhaustive. After migration, organisers can fine-tune is_pii per field via the form-builder UI (S5).
• is_published derived from event status: the command considers an event "published" when its status is one of registration_open / buildup / showday / teardown / closed. If the legacy registration was open on events with other statuses, adjust is_published manually after migration.
• Custom field types: the command maps the 10 legacy field types (text/textarea/select/multiselect/checkbox/radio/boolean/number/tag_picker/heading) into the new uppercase enum values. If your legacy data has values outside this set, the command will set them to TEXT and the verifier will flag them as invalid field_type.