Commit Graph

3 Commits

Author SHA1 Message Date
3d323bf55f chore(test): switch test database from SQLite to MySQL (WS-6)
Test infrastructure now uses the same MySQL 8.0 engine as local dev
and production. SQLite is no longer used anywhere in the project.

Eliminates the SQLite "rebuild on FK add" quirk that forced session 2.5
to omit a foreign key on form_schemas.default_crowd_type_id (Task 2 of
this session restores it).

Configuration:
- phpunit.xml: DB_CONNECTION=sqlite (:memory:) replaced with mysql
  pointing at crewli_test database (127.0.0.1:3306, crewli/secret)
- Makefile: new test-db-create target creates crewli_test in the
  bm_mysql Docker container; make test ensures it exists before
  running suite

Latent-bug surfacing — fixes that MySQL exposed:

1. form_submissions.idempotency_key was declared `ulid()` (VARCHAR 26)
   while FormRequest validates `string|max:30`. SQLite ignored the cap;
   MySQL truncated and rejected. Column widened to string(30) to match
   validation.

2. FormFieldValidationRuleService / FormFieldConfigService /
   FormFieldBindingService::snapshotShapesFor — toJsonShape iterated
   collection in DB-default order (insertion-stable on SQLite, undefined
   on MySQL). Schema_snapshot bytes drifted across re-emits, breaking
   audit-replay. Added `->sortBy('id')` (ULID = insertion-order
   semantics, deterministic) on all three.

3. FormSubmissionObserverTest::test_denormalized_indexes_exist queried
   sqlite_master directly. Replaced with the cross-engine
   information_schema.STATISTICS query (the real production check is
   on MySQL anyway).

4. JSON column key order non-determinism: MySQL JSON columns may
   round-trip associative-array keys in a different order than they
   were inserted. assertSame on JSON-derived associative arrays now
   uses assertEquals (structural equality) where the test was previously
   over-asserting on key order:
   - ConditionalLogicActivityLogPayloadTest
   - ConditionalLogicBackfillTest::test_rollback_reconstructs_canonical_json
   - FormFieldBindingMigrationTest::test_rollback_reconstructs_json_and_drops_table
   - FormFieldOptionServiceAndScopeTest::test_replace_options_emits_activity_log_on_field_only
   - FormFieldOptionsActivityLogTest::test_field_updated_payload_contains_options_diff_when_options_change
   - FormFieldOptionsBackfillTest::test_forward_migration_backfills_rows_strips_translations_and_rewrites_snapshot
   - FormFieldOptionsSnapshotAndStrictRequestTest::test_submission_snapshot_embeds_rich_shape_options

5. Backfill / migration tests (4 classes, 21 tests) ran migrate:rollback
   then migrate inside RefreshDatabase's wrapping transaction. MySQL
   DDL implicit-commits the surrounding transaction, leaving Laravel
   unable to ROLLBACK TO SAVEPOINT at end-of-test (1305 SAVEPOINT
   does not exist). Replaced RefreshDatabase with a per-test
   migrate:fresh in setUp + RefreshDatabaseState::\$migrated = false to
   force the next RefreshDatabase test to re-migrate cleanly:
   - FormFieldBindingMigrationTest
   - ConditionalLogicBackfillTest
   - FormFieldOptionsBackfillTest
   - FormFieldValidationRuleBackfillTest

All 1386 tests now pass on MySQL. Larastan baseline unchanged.

Refs: WS-6 session 2.5 deviation #1 cleanup, RFC-WS-6.md v1.1

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:10:56 +02:00
e7c9482474 refactor(form-field): drop form_fields.options + form_field_library.options
Final WS-5d cleanup. The JSON columns that have been unread since
commit 3 are now physically dropped on both source tables. Their
canonical rich-shape lives in form_field_options, accessed
exclusively through the morphMany relation.

Defensive sweep: any lingering translations.{locale}.options key in
either source table's translations bag is stripped. Commit 2's
backfill should already have done so exhaustively; this is
belt-and-braces.

Rollback re-creates the columns as nullable JSON but leaves them
empty. Pair with commit 2's rollback to restore the pre-WS-5d data
shape on every owner row.

The commit-3 getOptionsAttribute accessor-bridge on FormField +
FormFieldLibrary is removed — Eloquent's getAttribute() resolution
now naturally falls through to the morphMany relation since there's
no underlying column to shadow it. New regression test
FormFieldOptionsAccessTest asserts $field->options resolves to an
Eloquent Collection of FormFieldOption instances and lazy-loads in
exactly 2 queries (1 parent + 1 lazy-load options) on a fresh fetch
without with() preload. Same trio for FormFieldLibrary.

Migration step-count tests in WS-5a/b/c bumped by 1 to account for
the new drop_form_field_options_json_columns migration on the
rollback stack.

Documentation:
  - SCHEMA.md v2.6: form_field_options table documented; options row
    removed from form_fields and form_field_library; morphMany
    relations updated; cross-references to ARCH-FORM-BUILDER §17.6
    and addendum §Q3 WS-5d Uitvoering added on both source-table
    docblocks.
  - ARCH-FORM-BUILDER.md v1.8: new §17.6 "Field options (relational)"
    mirrors the §17.4 / §17.5 relational-sibling structure with
    sub-sections 17.6.1 rationale, 17.6.2 table + catalogue, 17.6.3
    service / scope / cascade / activity log, 17.6.4 snapshot
    embedding, 17.6.5 external API contract. Existing Webhooks
    section renumbered from §17.6 to §17.7.
  - ARCH-CONSOLIDATION-ADDENDUM-2026-04-24.md: "Uitvoering — WS-5d
    (2026-04-27)" section added. Eight paragraphs covering the
    snapshot atomic rewrite, strict-fail backfill dispatch, dual
    activity-log emit, four-sibling base-class extraction warrant,
    commit 0 dead-code precondition, the temporary getOptionsAttribute
    accessor-bridge pattern (with reusability note for future
    JSON→relational refactors), the dev-seeder vergoedingstype RADIO
    normalisation (drift correction explicitly distinguished from the
    parallel apps/app RegistrationFieldTemplate description domain),
    and the WS-5 family completion note.
  - BACKLOG.md: FORM-BUILDER-LIBRARY-AUDIT-LOG entry extended to four
    services (adds library.options_replaced); new
    FORM-BUILDER-MORPH-SCOPE-BASE-CLASS entry added as the WS-5d
    follow-up now that all four concrete morph-scope siblings exist.

Tests: 1193 → 1208 green (+15 across commits 3+4+5; this commit alone:
+2 from the regression test).

This completes the WS-5 family.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 03:00:20 +02:00
15e4e49d8c feat(form-field): backfill form_fields.options to form_field_options
Atomic data migration. Every options datum in the database — in
form_fields and form_field_library, their translations bags, and the
form_submissions.schema_snapshot + form_templates.schema_snapshot JSON
blobs — is converted to the new relational rich-shape representation.

Strict dispatch per §17.4.4 / §8.7 convention:
  - Fail on field_type ∉ {RADIO, SELECT, MULTISELECT, CHECKBOX_LIST}
    carrying non-null options (post-WS-5b TAG_PICKER seed-bug indicator)
  - Fail on non-flat-string-array options shape
  - Fail on translations.{locale}.options[] length mismatch
  - Fail on non-string / >255-char translated labels
  - Fail on any residual translations.{locale}.options key after
    step C migration

Snapshot rewrite in-place: both form_submissions.schema_snapshot and
form_templates.schema_snapshot walk fields[*] and rewrite options to
the new rich-shape, strip per-locale options[] from the parallel
translations bag. Zero-compromise directive — no reader tolerance for
pre-WS-5d shape in commit 3 onwards.

Rollback reconstructs JSON column shapes plus translations bags.
Forward+back pair safe as a unit; partial rollback unsupported.

FormFieldService::insertFromLibrary switches from JSON-copy to
FormFieldOptionService::copyOptions row-clone per addendum Q3 row-copy
mandate. The field's own translations bag no longer carries
{locale}.options keys — those live on option rows now.

Seeders and factories switch to service-level option creation:
  - FormBuilderDevSeeder.canonicalFields keeps flat-string options as
    its data shape; FormField::create no longer receives an options
    key, the post-create FormFieldOptionService::replaceOptions call
    inserts the rich rows. The same applies to
    seedEventRegistrationShowcaseSchema. The vergoedingstype field's
    legacy {label, description} object shape (a pre-WS-5d seed-bug
    that the strict backfill would reject) is normalised to flat
    strings; the descriptions are dropped.
  - seedSystemTemplates embeds rich-shape options in the template
    snapshot — no flat-array snapshot data remains in newly-seeded
    rows.
  - FormFieldFactory + FormFieldLibraryFactory drop the options
    default; new ::withOptions() helper accepts either flat strings
    (each becomes value+label) or full spec arrays and routes through
    the service.

JSON columns (form_fields.options, form_field_library.options) remain
present and writable via fillable; column-drop lands in commit 5.
Reads from the JSON column still exist in resources, snapshot writer,
FormRequests, FormValueService, and FilterRegistryController — commit
3 switches those all atomically.

Migration step-count tests in WS-5a/b/c bumped by 1 to account for
the new backfill_form_field_options migration on the migration stack.

Tests: 1182 → 1193 green (+11 tests / +56 assertions).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 02:21:26 +02:00