7d326720ab490db27ec7c1be306800fa224129a2
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| a791a276fa |
fix(form-builder): canonicalize JSON for byte-stable storage (WS-6)
MySQL 8.0 JSON columns may reorder associative-array keys on round-trip. For audit-immutable values (schema snapshots, webhook payloads, activity log diffs), this is corrupting: re-emits produce different byte sequences for the same logical content. Introduced JsonCanonicalizer (recursive ksort on associative arrays; numeric-indexed lists preserve order) and applied at every writer site that produces byte-stable JSON: - FormSubmissionService: canonicalize the schema_snapshot array before storage (audit-immutable per ARCH §4.3, RFC-WS-6 v1.1). - FormField::logFieldChange / FormSchema::logSchemaChange: canonicalize activity-log properties before withProperties() so old/new diffs read back byte-stable. - BindingActivityLogger: canonicalize both the pass-level and per-binding activity properties. - FormWebhookDispatcher: canonicalize payload_snapshot before storage (delivery-time HMAC re-encodes the same canonical bytes). - DeliverFormWebhookJob: switched json_encode to JsonCanonicalizer::encode for the HMAC-signed body, so the signature is byte-stable across re-deliveries and reproducible by receivers from the same logical payload. Sites NOT canonicalized (deliberate): - form_schemas.settings — opaque UI config; key order has no semantic meaning, no byte-stability requirement. - form_schemas.translations / form_fields.translations — read by display layer; key order doesn't matter. - form_templates.schema_snapshot — user-supplied input via store/ update; user is the source of truth, not audit-immutable in the same way as form_submissions.schema_snapshot. Reverted the 7 assertEquals workarounds from session 2.6: - ConditionalLogicActivityLogPayloadTest - ConditionalLogicBackfillTest::test_rollback_reconstructs_canonical_json - FormFieldBindingMigrationTest::test_rollback_reconstructs_json_and_drops_table - FormFieldOptionServiceAndScopeTest::test_replace_options_emits_activity_log_on_field_only - FormFieldOptionsActivityLogTest::test_field_updated_payload_contains_options_diff_when_options_change - FormFieldOptionsBackfillTest::test_forward_migration_backfills_rows_strips_translations_and_rewrites_snapshot - FormFieldOptionsSnapshotAndStrictRequestTest::test_submission_snapshot_embeds_rich_shape_options Each now uses assertSame on JsonCanonicalizer::encode of both sides — byte-stable comparison meaningful regardless of MySQL JSON storage behavior. New regression test SchemaSnapshotByteStableAcrossReemitsTest exercises the contract end-to-end: complex schema with bindings, validation rules, options, conditional logic, submitted; reads schema_snapshot via three roads (Eloquent cast, fresh model, raw bytes) and asserts the canonical encode is identical. ARCH-FORM-BUILDER.md §4.6.1 gets a "Byte-stability" sub-section explaining what's canonicalized and why. Test count: 1388 → 1400 (+11 JsonCanonicalizer unit, +1 snapshot regression). Larastan clean. Rector dry-run unchanged at 355. Refs: WS-6 session 2.6 deviation #4 cleanup, RFC-WS-6 v1.1 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 3d323bf55f |
chore(test): switch test database from SQLite to MySQL (WS-6)
Test infrastructure now uses the same MySQL 8.0 engine as local dev
and production. SQLite is no longer used anywhere in the project.
Eliminates the SQLite "rebuild on FK add" quirk that forced session 2.5
to omit a foreign key on form_schemas.default_crowd_type_id (Task 2 of
this session restores it).
Configuration:
- phpunit.xml: DB_CONNECTION=sqlite (:memory:) replaced with mysql
pointing at crewli_test database (127.0.0.1:3306, crewli/secret)
- Makefile: new test-db-create target creates crewli_test in the
bm_mysql Docker container; make test ensures it exists before
running suite
Latent-bug surfacing — fixes that MySQL exposed:
1. form_submissions.idempotency_key was declared `ulid()` (VARCHAR 26)
while FormRequest validates `string|max:30`. SQLite ignored the cap;
MySQL truncated and rejected. Column widened to string(30) to match
validation.
2. FormFieldValidationRuleService / FormFieldConfigService /
FormFieldBindingService::snapshotShapesFor — toJsonShape iterated
collection in DB-default order (insertion-stable on SQLite, undefined
on MySQL). Schema_snapshot bytes drifted across re-emits, breaking
audit-replay. Added `->sortBy('id')` (ULID = insertion-order
semantics, deterministic) on all three.
3. FormSubmissionObserverTest::test_denormalized_indexes_exist queried
sqlite_master directly. Replaced with the cross-engine
information_schema.STATISTICS query (the real production check is
on MySQL anyway).
4. JSON column key order non-determinism: MySQL JSON columns may
round-trip associative-array keys in a different order than they
were inserted. assertSame on JSON-derived associative arrays now
uses assertEquals (structural equality) where the test was previously
over-asserting on key order:
- ConditionalLogicActivityLogPayloadTest
- ConditionalLogicBackfillTest::test_rollback_reconstructs_canonical_json
- FormFieldBindingMigrationTest::test_rollback_reconstructs_json_and_drops_table
- FormFieldOptionServiceAndScopeTest::test_replace_options_emits_activity_log_on_field_only
- FormFieldOptionsActivityLogTest::test_field_updated_payload_contains_options_diff_when_options_change
- FormFieldOptionsBackfillTest::test_forward_migration_backfills_rows_strips_translations_and_rewrites_snapshot
- FormFieldOptionsSnapshotAndStrictRequestTest::test_submission_snapshot_embeds_rich_shape_options
5. Backfill / migration tests (4 classes, 21 tests) ran migrate:rollback
then migrate inside RefreshDatabase's wrapping transaction. MySQL
DDL implicit-commits the surrounding transaction, leaving Laravel
unable to ROLLBACK TO SAVEPOINT at end-of-test (1305 SAVEPOINT
does not exist). Replaced RefreshDatabase with a per-test
migrate:fresh in setUp + RefreshDatabaseState::\$migrated = false to
force the next RefreshDatabase test to re-migrate cleanly:
- FormFieldBindingMigrationTest
- ConditionalLogicBackfillTest
- FormFieldOptionsBackfillTest
- FormFieldValidationRuleBackfillTest
All 1386 tests now pass on MySQL. Larastan baseline unchanged.
Refs: WS-6 session 2.5 deviation #1 cleanup, RFC-WS-6.md v1.1
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| bb9242fd6e |
refactor(form-field): resources + snapshot + validator read form_field_options
Atomic reader switch. All call paths that previously read
form_fields.options / form_field_library.options from the JSON column
now read through FormFieldOptionService::toJsonShape() via the
morphMany relation:
- FormFieldResource + FormFieldLibraryResource +
PublicFormSchemaResource emit the rich-shape array
- FilterRegistryController emits rich shape uniformly (no flat-array
carve-out for filter-UI compatibility — preflight scan confirmed
zero portal/app consumers, S5 territory)
- FormFieldRuleBuilder plucks values from the relation for in:options
rule construction
- FormSubmissionService::buildSnapshot writes rich-shape options into
snapshots and strips translations.{locale}.options from each field's
translations bag (defensive — commit 2 backfill already did the
bulk strip)
- Four FormFieldRequest variants accept array-of-spec-objects,
validate shape in after() via FormFieldOptionService::assertSpecsValid,
and hand off to FormFieldOptionService::replaceOptions for writes
- FormFieldService::create + update extract option specs from the
request data and route through the service after the FormField row
is persisted
FormField and FormFieldLibrary $casts no longer include 'options'; the
JSON column is no longer cast. Options removed from $fillable on both
models so ::create() / ::fill() / mass assignment can no longer touch
the legacy column. Both models gain a getOptionsAttribute() accessor
that resolves $model->options to the eager-loaded morphMany collection
— required because Eloquent's getAttribute() prefers a real DB column
over a relation method, and the JSON column lives on the table until
WS-5d commit 5 drops it.
Activity log — dual emit per §6.7 / §17.4.2 / §17.6.3:
- field.updated carries old.options / new.options diff via
toJsonShape() reconstruction, byte-equal JSON compare to avoid
cosmetic false positives. Field updates that don't touch options
omit the key entirely
- field.options_replaced emits inside replaceOptions() on FormField
subject only; library subject writes silent (mirrors the WS-5b /
WS-5c convention)
JSON columns (form_fields.options, form_field_library.options) remain
present but unread — column drops land atomically in commit 5.
Two pre-existing test fixtures that seeded options via the JSON column
(FormFieldApiTest + PublicFormValidationTest) migrated to the
spec-array path: FormField::factory()->withOptions([...]) where the
options live on the field, or explicit spec-array request bodies for
HTTP tests.
Tests: 1193 → 1206 green (+13 tests / +28 assertions).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
