Commit Graph

5 Commits

Author SHA1 Message Date
3d323bf55f chore(test): switch test database from SQLite to MySQL (WS-6)
Test infrastructure now uses the same MySQL 8.0 engine as local dev
and production. SQLite is no longer used anywhere in the project.

Eliminates the SQLite "rebuild on FK add" quirk that forced session 2.5
to omit a foreign key on form_schemas.default_crowd_type_id (Task 2 of
this session restores it).

Configuration:
- phpunit.xml: DB_CONNECTION=sqlite (:memory:) replaced with mysql
  pointing at crewli_test database (127.0.0.1:3306, crewli/secret)
- Makefile: new test-db-create target creates crewli_test in the
  bm_mysql Docker container; make test ensures it exists before
  running suite

Latent-bug surfacing — fixes that MySQL exposed:

1. form_submissions.idempotency_key was declared `ulid()` (VARCHAR 26)
   while FormRequest validates `string|max:30`. SQLite ignored the cap;
   MySQL truncated and rejected. Column widened to string(30) to match
   validation.

2. FormFieldValidationRuleService / FormFieldConfigService /
   FormFieldBindingService::snapshotShapesFor — toJsonShape iterated
   collection in DB-default order (insertion-stable on SQLite, undefined
   on MySQL). Schema_snapshot bytes drifted across re-emits, breaking
   audit-replay. Added `->sortBy('id')` (ULID = insertion-order
   semantics, deterministic) on all three.

3. FormSubmissionObserverTest::test_denormalized_indexes_exist queried
   sqlite_master directly. Replaced with the cross-engine
   information_schema.STATISTICS query (the real production check is
   on MySQL anyway).

4. JSON column key order non-determinism: MySQL JSON columns may
   round-trip associative-array keys in a different order than they
   were inserted. assertSame on JSON-derived associative arrays now
   uses assertEquals (structural equality) where the test was previously
   over-asserting on key order:
   - ConditionalLogicActivityLogPayloadTest
   - ConditionalLogicBackfillTest::test_rollback_reconstructs_canonical_json
   - FormFieldBindingMigrationTest::test_rollback_reconstructs_json_and_drops_table
   - FormFieldOptionServiceAndScopeTest::test_replace_options_emits_activity_log_on_field_only
   - FormFieldOptionsActivityLogTest::test_field_updated_payload_contains_options_diff_when_options_change
   - FormFieldOptionsBackfillTest::test_forward_migration_backfills_rows_strips_translations_and_rewrites_snapshot
   - FormFieldOptionsSnapshotAndStrictRequestTest::test_submission_snapshot_embeds_rich_shape_options

5. Backfill / migration tests (4 classes, 21 tests) ran migrate:rollback
   then migrate inside RefreshDatabase's wrapping transaction. MySQL
   DDL implicit-commits the surrounding transaction, leaving Laravel
   unable to ROLLBACK TO SAVEPOINT at end-of-test (1305 SAVEPOINT
   does not exist). Replaced RefreshDatabase with a per-test
   migrate:fresh in setUp + RefreshDatabaseState::\$migrated = false to
   force the next RefreshDatabase test to re-migrate cleanly:
   - FormFieldBindingMigrationTest
   - ConditionalLogicBackfillTest
   - FormFieldOptionsBackfillTest
   - FormFieldValidationRuleBackfillTest

All 1386 tests now pass on MySQL. Larastan baseline unchanged.

Refs: WS-6 session 2.5 deviation #1 cleanup, RFC-WS-6.md v1.1

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:10:56 +02:00
e7c9482474 refactor(form-field): drop form_fields.options + form_field_library.options
Final WS-5d cleanup. The JSON columns that have been unread since
commit 3 are now physically dropped on both source tables. Their
canonical rich-shape lives in form_field_options, accessed
exclusively through the morphMany relation.

Defensive sweep: any lingering translations.{locale}.options key in
either source table's translations bag is stripped. Commit 2's
backfill should already have done so exhaustively; this is
belt-and-braces.

Rollback re-creates the columns as nullable JSON but leaves them
empty. Pair with commit 2's rollback to restore the pre-WS-5d data
shape on every owner row.

The commit-3 getOptionsAttribute accessor-bridge on FormField +
FormFieldLibrary is removed — Eloquent's getAttribute() resolution
now naturally falls through to the morphMany relation since there's
no underlying column to shadow it. New regression test
FormFieldOptionsAccessTest asserts $field->options resolves to an
Eloquent Collection of FormFieldOption instances and lazy-loads in
exactly 2 queries (1 parent + 1 lazy-load options) on a fresh fetch
without with() preload. Same trio for FormFieldLibrary.

Migration step-count tests in WS-5a/b/c bumped by 1 to account for
the new drop_form_field_options_json_columns migration on the
rollback stack.

Documentation:
  - SCHEMA.md v2.6: form_field_options table documented; options row
    removed from form_fields and form_field_library; morphMany
    relations updated; cross-references to ARCH-FORM-BUILDER §17.6
    and addendum §Q3 WS-5d Uitvoering added on both source-table
    docblocks.
  - ARCH-FORM-BUILDER.md v1.8: new §17.6 "Field options (relational)"
    mirrors the §17.4 / §17.5 relational-sibling structure with
    sub-sections 17.6.1 rationale, 17.6.2 table + catalogue, 17.6.3
    service / scope / cascade / activity log, 17.6.4 snapshot
    embedding, 17.6.5 external API contract. Existing Webhooks
    section renumbered from §17.6 to §17.7.
  - ARCH-CONSOLIDATION-ADDENDUM-2026-04-24.md: "Uitvoering — WS-5d
    (2026-04-27)" section added. Eight paragraphs covering the
    snapshot atomic rewrite, strict-fail backfill dispatch, dual
    activity-log emit, four-sibling base-class extraction warrant,
    commit 0 dead-code precondition, the temporary getOptionsAttribute
    accessor-bridge pattern (with reusability note for future
    JSON→relational refactors), the dev-seeder vergoedingstype RADIO
    normalisation (drift correction explicitly distinguished from the
    parallel apps/app RegistrationFieldTemplate description domain),
    and the WS-5 family completion note.
  - BACKLOG.md: FORM-BUILDER-LIBRARY-AUDIT-LOG entry extended to four
    services (adds library.options_replaced); new
    FORM-BUILDER-MORPH-SCOPE-BASE-CLASS entry added as the WS-5d
    follow-up now that all four concrete morph-scope siblings exist.

Tests: 1193 → 1208 green (+15 across commits 3+4+5; this commit alone:
+2 from the regression test).

This completes the WS-5 family.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 03:00:20 +02:00
bb9242fd6e refactor(form-field): resources + snapshot + validator read form_field_options
Atomic reader switch. All call paths that previously read
form_fields.options / form_field_library.options from the JSON column
now read through FormFieldOptionService::toJsonShape() via the
morphMany relation:

  - FormFieldResource + FormFieldLibraryResource +
    PublicFormSchemaResource emit the rich-shape array
  - FilterRegistryController emits rich shape uniformly (no flat-array
    carve-out for filter-UI compatibility — preflight scan confirmed
    zero portal/app consumers, S5 territory)
  - FormFieldRuleBuilder plucks values from the relation for in:options
    rule construction
  - FormSubmissionService::buildSnapshot writes rich-shape options into
    snapshots and strips translations.{locale}.options from each field's
    translations bag (defensive — commit 2 backfill already did the
    bulk strip)
  - Four FormFieldRequest variants accept array-of-spec-objects,
    validate shape in after() via FormFieldOptionService::assertSpecsValid,
    and hand off to FormFieldOptionService::replaceOptions for writes
  - FormFieldService::create + update extract option specs from the
    request data and route through the service after the FormField row
    is persisted

FormField and FormFieldLibrary $casts no longer include 'options'; the
JSON column is no longer cast. Options removed from $fillable on both
models so ::create() / ::fill() / mass assignment can no longer touch
the legacy column. Both models gain a getOptionsAttribute() accessor
that resolves $model->options to the eager-loaded morphMany collection
— required because Eloquent's getAttribute() prefers a real DB column
over a relation method, and the JSON column lives on the table until
WS-5d commit 5 drops it.

Activity log — dual emit per §6.7 / §17.4.2 / §17.6.3:
  - field.updated carries old.options / new.options diff via
    toJsonShape() reconstruction, byte-equal JSON compare to avoid
    cosmetic false positives. Field updates that don't touch options
    omit the key entirely
  - field.options_replaced emits inside replaceOptions() on FormField
    subject only; library subject writes silent (mirrors the WS-5b /
    WS-5c convention)

JSON columns (form_fields.options, form_field_library.options) remain
present but unread — column drops land atomically in commit 5.

Two pre-existing test fixtures that seeded options via the JSON column
(FormFieldApiTest + PublicFormValidationTest) migrated to the
spec-array path: FormField::factory()->withOptions([...]) where the
options live on the field, or explicit spec-array request bodies for
HTTP tests.

Tests: 1193 → 1206 green (+13 tests / +28 assertions).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 02:33:21 +02:00
15e4e49d8c feat(form-field): backfill form_fields.options to form_field_options
Atomic data migration. Every options datum in the database — in
form_fields and form_field_library, their translations bags, and the
form_submissions.schema_snapshot + form_templates.schema_snapshot JSON
blobs — is converted to the new relational rich-shape representation.

Strict dispatch per §17.4.4 / §8.7 convention:
  - Fail on field_type ∉ {RADIO, SELECT, MULTISELECT, CHECKBOX_LIST}
    carrying non-null options (post-WS-5b TAG_PICKER seed-bug indicator)
  - Fail on non-flat-string-array options shape
  - Fail on translations.{locale}.options[] length mismatch
  - Fail on non-string / >255-char translated labels
  - Fail on any residual translations.{locale}.options key after
    step C migration

Snapshot rewrite in-place: both form_submissions.schema_snapshot and
form_templates.schema_snapshot walk fields[*] and rewrite options to
the new rich-shape, strip per-locale options[] from the parallel
translations bag. Zero-compromise directive — no reader tolerance for
pre-WS-5d shape in commit 3 onwards.

Rollback reconstructs JSON column shapes plus translations bags.
Forward+back pair safe as a unit; partial rollback unsupported.

FormFieldService::insertFromLibrary switches from JSON-copy to
FormFieldOptionService::copyOptions row-clone per addendum Q3 row-copy
mandate. The field's own translations bag no longer carries
{locale}.options keys — those live on option rows now.

Seeders and factories switch to service-level option creation:
  - FormBuilderDevSeeder.canonicalFields keeps flat-string options as
    its data shape; FormField::create no longer receives an options
    key, the post-create FormFieldOptionService::replaceOptions call
    inserts the rich rows. The same applies to
    seedEventRegistrationShowcaseSchema. The vergoedingstype field's
    legacy {label, description} object shape (a pre-WS-5d seed-bug
    that the strict backfill would reject) is normalised to flat
    strings; the descriptions are dropped.
  - seedSystemTemplates embeds rich-shape options in the template
    snapshot — no flat-array snapshot data remains in newly-seeded
    rows.
  - FormFieldFactory + FormFieldLibraryFactory drop the options
    default; new ::withOptions() helper accepts either flat strings
    (each becomes value+label) or full spec arrays and routes through
    the service.

JSON columns (form_fields.options, form_field_library.options) remain
present and writable via fillable; column-drop lands in commit 5.
Reads from the JSON column still exist in resources, snapshot writer,
FormRequests, FormValueService, and FilterRegistryController — commit
3 switches those all atomically.

Migration step-count tests in WS-5a/b/c bumped by 1 to account for
the new backfill_form_field_options migration on the migration stack.

Tests: 1182 → 1193 green (+11 tests / +56 assertions).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 02:21:26 +02:00
11588623c5 feat(form-field): add form_field_options table, service, scope, cascade
Fourth and final WS-5 sibling. Polymorphic morph-owned table for the
RADIO / SELECT / MULTISELECT / CHECKBOX_LIST option rows, shared
between form_fields and form_field_library via the owner_type
discriminator. Matches the WS-5a (bindings) / WS-5b (validation_rules
+ configs) pattern one-for-one: dedicated service as single writer,
UNION-over-two-owner-chains scope, shared cascade observer.

Row shape:
  - value         canonical storage value (string ≤255, UNIQUE per owner)
  - label         default-locale display label (string ≤255)
  - sort_order    int unsigned
  - translations  JSON { "<locale>": "<translated label>" }

The UNIQUE(owner_type, owner_id, value) index ffo_owner_value_unique
is the seed-bug guard — duplicate values per field have no semantic
meaning and must fail at both the service layer (assertSpecsValid)
and the DB level.

Activity log: field.options_replaced emits on FormField subject only,
per the §6.7 WS-5a / §17.4.2 WS-5b convention that library-level
changes are silent in activity log.

No production reads yet. The form_fields.options and
form_field_library.options JSON columns remain the active source of
truth until the commit-3 reader switch — accessing $field->options
still resolves through the JSON cast in commit 1, so model tests
exercise the new morphMany via $field->options() (explicit relation
call). Both FormField and FormFieldLibrary now carry an `options`
morphMany alongside `bindings`, `validation_rules`, and `configs`.

Cascade: FormFieldChildTablesCascadeObserver gains form_field_options
as the fourth child cleaned on owner delete (both FormField soft/
force-delete and FormFieldLibrary delete).

Migration step-count tests in WS-5a/b/c bumped by 1 to account for
the new create_form_field_options_table on the migration stack.

Base scope-class extraction across the four siblings — deliberately
deferred to a follow-up work package per addendum §17.4.3 / §17.5.3.
Now that all four concrete implementations exist, the "what actually
varies" question can be answered empirically.

Tests: 1158 → 1182 green (+24 tests / +42 assertions).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 02:07:53 +02:00