fix(form-builder): canonicalize JSON for byte-stable storage (WS-6)
MySQL 8.0 JSON columns may reorder associative-array keys on round-trip. For audit-immutable values (schema snapshots, webhook payloads, activity log diffs), this is corrupting: re-emits produce different byte sequences for the same logical content. Introduced JsonCanonicalizer (recursive ksort on associative arrays; numeric-indexed lists preserve order) and applied at every writer site that produces byte-stable JSON: - FormSubmissionService: canonicalize the schema_snapshot array before storage (audit-immutable per ARCH §4.3, RFC-WS-6 v1.1). - FormField::logFieldChange / FormSchema::logSchemaChange: canonicalize activity-log properties before withProperties() so old/new diffs read back byte-stable. - BindingActivityLogger: canonicalize both the pass-level and per-binding activity properties. - FormWebhookDispatcher: canonicalize payload_snapshot before storage (delivery-time HMAC re-encodes the same canonical bytes). - DeliverFormWebhookJob: switched json_encode to JsonCanonicalizer::encode for the HMAC-signed body, so the signature is byte-stable across re-deliveries and reproducible by receivers from the same logical payload. Sites NOT canonicalized (deliberate): - form_schemas.settings — opaque UI config; key order has no semantic meaning, no byte-stability requirement. - form_schemas.translations / form_fields.translations — read by display layer; key order doesn't matter. - form_templates.schema_snapshot — user-supplied input via store/ update; user is the source of truth, not audit-immutable in the same way as form_submissions.schema_snapshot. Reverted the 7 assertEquals workarounds from session 2.6: - ConditionalLogicActivityLogPayloadTest - ConditionalLogicBackfillTest::test_rollback_reconstructs_canonical_json - FormFieldBindingMigrationTest::test_rollback_reconstructs_json_and_drops_table - FormFieldOptionServiceAndScopeTest::test_replace_options_emits_activity_log_on_field_only - FormFieldOptionsActivityLogTest::test_field_updated_payload_contains_options_diff_when_options_change - FormFieldOptionsBackfillTest::test_forward_migration_backfills_rows_strips_translations_and_rewrites_snapshot - FormFieldOptionsSnapshotAndStrictRequestTest::test_submission_snapshot_embeds_rich_shape_options Each now uses assertSame on JsonCanonicalizer::encode of both sides — byte-stable comparison meaningful regardless of MySQL JSON storage behavior. New regression test SchemaSnapshotByteStableAcrossReemitsTest exercises the contract end-to-end: complex schema with bindings, validation rules, options, conditional logic, submitted; reads schema_snapshot via three roads (Eloquent cast, fresh model, raw bytes) and asserts the canonical encode is identical. ARCH-FORM-BUILDER.md §4.6.1 gets a "Byte-stability" sub-section explaining what's canonicalized and why. Test count: 1388 → 1400 (+11 JsonCanonicalizer unit, +1 snapshot regression). Larastan clean. Rector dry-run unchanged at 355. Refs: WS-6 session 2.6 deviation #4 cleanup, RFC-WS-6 v1.1 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -5,6 +5,7 @@ declare(strict_types=1);
|
||||
namespace App\FormBuilder\Bindings;
|
||||
|
||||
use App\Models\FormBuilder\FormSubmission;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Spatie\Activitylog\Models\Activity;
|
||||
|
||||
/**
|
||||
@@ -23,9 +24,12 @@ final class BindingActivityLogger
|
||||
{
|
||||
public function logPass(FormSubmission $submission, BindingPassResult $result): void
|
||||
{
|
||||
// RFC-WS-6 session 2.7 — canonicalize properties before they land
|
||||
// in activity_log.properties (MySQL JSON column round-trip would
|
||||
// otherwise reorder keys and break diff/regression assertions).
|
||||
$passActivity = activity()
|
||||
->performedOn($submission)
|
||||
->withProperties([
|
||||
->withProperties(JsonCanonicalizer::canonicalize([
|
||||
'binding_count' => count($result->applications),
|
||||
'succeeded' => $result->successCount(),
|
||||
'failed' => $result->failureCount(),
|
||||
@@ -33,7 +37,7 @@ final class BindingActivityLogger
|
||||
'person_provisioned' => $result->provisionedSubjectType === 'person',
|
||||
'subject_type' => $result->provisionedSubjectType,
|
||||
'subject_id' => $result->provisionedSubjectId,
|
||||
])
|
||||
]))
|
||||
->log('form_submission.bindings_pass_completed');
|
||||
|
||||
$parentActivityId = $passActivity instanceof Activity ? (string) $passActivity->id : null;
|
||||
@@ -56,7 +60,7 @@ final class BindingActivityLogger
|
||||
|
||||
activity()
|
||||
->performedOn($submission)
|
||||
->withProperties($properties)
|
||||
->withProperties(JsonCanonicalizer::canonicalize($properties))
|
||||
->log('form_submission.binding_applied');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ namespace App\Jobs\FormBuilder;
|
||||
|
||||
use App\Enums\FormBuilder\FormWebhookDeliveryStatus;
|
||||
use App\Models\FormBuilder\FormWebhookDelivery;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Bus\Queueable;
|
||||
use Illuminate\Contracts\Queue\ShouldQueue;
|
||||
use Illuminate\Foundation\Bus\Dispatchable;
|
||||
@@ -61,8 +62,13 @@ final class DeliverFormWebhookJob implements ShouldQueue
|
||||
return;
|
||||
}
|
||||
|
||||
// RFC-WS-6 session 2.7 — canonical JSON for HMAC signing.
|
||||
// payload_snapshot was read from a MySQL JSON column whose key
|
||||
// order may not match what we wrote. Canonicalize so the
|
||||
// signature is byte-stable across re-deliveries and matches what
|
||||
// a verifying receiver computes from the same logical payload.
|
||||
$payload = (array) ($delivery->payload_snapshot ?? []);
|
||||
$body = json_encode($payload, JSON_THROW_ON_ERROR);
|
||||
$body = JsonCanonicalizer::encode($payload);
|
||||
|
||||
$headers = ['Content-Type' => 'application/json'];
|
||||
if (! empty($webhook->secret)) {
|
||||
@@ -173,7 +179,7 @@ final class DeliverFormWebhookJob implements ShouldQueue
|
||||
}
|
||||
$maskLong = -1 << (32 - (int) $mask);
|
||||
|
||||
return (($ipLong & $maskLong) === ($subnetLong & $maskLong));
|
||||
return ($ipLong & $maskLong) === ($subnetLong & $maskLong);
|
||||
}
|
||||
|
||||
private function isRetriable(int $status): bool
|
||||
|
||||
@@ -7,6 +7,7 @@ namespace App\Models\FormBuilder;
|
||||
use App\Enums\FormBuilder\FormFieldDisplayWidth;
|
||||
use App\Enums\FormBuilder\FormValueStorageHint;
|
||||
use App\Models\Scopes\OrganisationScope;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Database\Eloquent\Concerns\HasUlids;
|
||||
use Illuminate\Database\Eloquent\Factories\HasFactory;
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
@@ -32,7 +33,7 @@ final class FormField extends Model
|
||||
|
||||
protected static function booted(): void
|
||||
{
|
||||
static::addGlobalScope(new OrganisationScope());
|
||||
self::addGlobalScope(new OrganisationScope);
|
||||
}
|
||||
|
||||
/** @return array{via: class-string, fk: string} */
|
||||
@@ -156,9 +157,13 @@ final class FormField extends Model
|
||||
*/
|
||||
public function logFieldChange(string $event, array $properties = []): void
|
||||
{
|
||||
// RFC-WS-6 session 2.7: properties land in activity_log.properties
|
||||
// (MySQL JSON column). Canonicalize so diff/regression assertions
|
||||
// and downstream consumers see byte-stable structure regardless of
|
||||
// MySQL key-order normalization on round-trip.
|
||||
activity()
|
||||
->performedOn($this)
|
||||
->withProperties($properties)
|
||||
->withProperties(JsonCanonicalizer::canonicalize($properties))
|
||||
->log($event);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ use App\Models\CrowdType;
|
||||
use App\Models\Organisation;
|
||||
use App\Models\Scopes\OrganisationScope;
|
||||
use App\Models\User;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Database\Eloquent\Concerns\HasUlids;
|
||||
use Illuminate\Database\Eloquent\Factories\HasFactory;
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
@@ -33,7 +34,7 @@ final class FormSchema extends Model
|
||||
|
||||
protected static function booted(): void
|
||||
{
|
||||
static::addGlobalScope(new OrganisationScope());
|
||||
self::addGlobalScope(new OrganisationScope);
|
||||
}
|
||||
|
||||
protected $fillable = [
|
||||
@@ -152,9 +153,13 @@ final class FormSchema extends Model
|
||||
*/
|
||||
public function logSchemaChange(string $event, array $properties = []): void
|
||||
{
|
||||
// RFC-WS-6 session 2.7: properties land in activity_log.properties
|
||||
// (MySQL JSON column). Canonicalize so diff/regression assertions
|
||||
// and downstream consumers see byte-stable structure regardless of
|
||||
// MySQL key-order normalization on round-trip.
|
||||
activity()
|
||||
->performedOn($this)
|
||||
->withProperties($properties)
|
||||
->withProperties(JsonCanonicalizer::canonicalize($properties))
|
||||
->log($event);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,9 +19,9 @@ use App\Models\FormBuilder\FormSubmission;
|
||||
use App\Models\FormBuilder\FormSubmissionDelegation;
|
||||
use App\Models\FormBuilder\FormValue;
|
||||
use App\Models\User;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
use Illuminate\Support\Str;
|
||||
|
||||
/**
|
||||
* Submission lifecycle: draft → submitted → reviewed per ARCH §4.3, §15.
|
||||
@@ -103,7 +103,7 @@ final class FormSubmissionService
|
||||
{
|
||||
$this->assertWritable($submission);
|
||||
|
||||
$result = DB::transaction(function () use ($submission, $actor): FormSubmission {
|
||||
$result = DB::transaction(function () use ($submission): FormSubmission {
|
||||
$schema = $submission->schema;
|
||||
|
||||
$submission->status = FormSubmissionStatus::SUBMITTED->value;
|
||||
@@ -111,7 +111,12 @@ final class FormSubmissionService
|
||||
$submission->schema_version_at_submit = $schema->version;
|
||||
|
||||
if ($schema->snapshot_mode !== FormSchemaSnapshotMode::NEVER) {
|
||||
$submission->schema_snapshot = $this->buildSnapshot($schema);
|
||||
// RFC-WS-6 session 2.7: schema_snapshot is audit-immutable;
|
||||
// canonicalize before storage so MySQL JSON-column round-trip
|
||||
// can never corrupt audit-replay diffs or webhook signing.
|
||||
$submission->schema_snapshot = JsonCanonicalizer::canonicalize(
|
||||
$this->buildSnapshot($schema),
|
||||
);
|
||||
}
|
||||
|
||||
if ($submission->opened_at !== null) {
|
||||
@@ -272,7 +277,6 @@ final class FormSubmissionService
|
||||
* any residual options key defensively (commit 2 backfill should
|
||||
* already have done so on existing rows).
|
||||
*
|
||||
* @param mixed $translations
|
||||
* @return array<string, mixed>|null
|
||||
*/
|
||||
private function stripOptionsFromTranslations(mixed $translations): ?array
|
||||
|
||||
@@ -9,6 +9,7 @@ use App\Jobs\FormBuilder\DeliverFormWebhookJob;
|
||||
use App\Models\FormBuilder\FormSchemaWebhook;
|
||||
use App\Models\FormBuilder\FormSubmission;
|
||||
use App\Models\FormBuilder\FormWebhookDelivery;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
|
||||
/**
|
||||
* Finds active webhooks for a submission's schema + trigger and queues a
|
||||
@@ -36,7 +37,12 @@ final class FormWebhookDispatcher
|
||||
'trigger_event' => $triggerEvent,
|
||||
'status' => FormWebhookDeliveryStatus::PENDING->value,
|
||||
'attempts' => 0,
|
||||
'payload_snapshot' => $this->buildPayload($submission, $triggerEvent),
|
||||
// RFC-WS-6 session 2.7 — canonicalize before storage; the
|
||||
// delivery job HMAC-signs the same canonical bytes after
|
||||
// re-encode, so signature is reproducible.
|
||||
'payload_snapshot' => JsonCanonicalizer::canonicalize(
|
||||
$this->buildPayload($submission, $triggerEvent),
|
||||
),
|
||||
]);
|
||||
|
||||
DeliverFormWebhookJob::dispatch($delivery->id)->onConnection('webhooks')->onQueue('webhooks');
|
||||
|
||||
75
api/app/Support/Json/JsonCanonicalizer.php
Normal file
75
api/app/Support/Json/JsonCanonicalizer.php
Normal file
@@ -0,0 +1,75 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Support\Json;
|
||||
|
||||
/**
|
||||
* Canonical JSON encoding for byte-stable storage.
|
||||
*
|
||||
* MySQL 8.0 JSON columns may reorder associative-array keys on
|
||||
* round-trip. For values that need byte-stability (schema snapshots,
|
||||
* webhook payloads signed via HMAC, audit-replay diffs), canonicalize
|
||||
* the structure before encode so re-emits produce identical bytes.
|
||||
*
|
||||
* Strategy:
|
||||
* - Associative arrays: recursively ksort
|
||||
* - Numeric-indexed lists (`array_is_list()`): preserve order
|
||||
* (semantically ordered)
|
||||
* - Scalars and non-arrays: passthrough
|
||||
*
|
||||
* Numeric vs associative detection follows array_is_list() — PHP 8.1+
|
||||
* convention. Mixed-key arrays are treated as associative (rare and
|
||||
* indicative of a data issue worth surfacing rather than papering over).
|
||||
*
|
||||
* RFC-WS-6 session 2.7 — see also CLAUDE.md "Database" section
|
||||
* (byte-stability rule for JSON columns).
|
||||
*/
|
||||
final class JsonCanonicalizer
|
||||
{
|
||||
/**
|
||||
* Canonicalize the structure recursively (sort associative keys).
|
||||
*
|
||||
* @template T
|
||||
*
|
||||
* @param T $value
|
||||
* @return T
|
||||
*/
|
||||
public static function canonicalize(mixed $value): mixed
|
||||
{
|
||||
if (! is_array($value)) {
|
||||
return $value;
|
||||
}
|
||||
|
||||
if ($value === []) {
|
||||
return $value;
|
||||
}
|
||||
|
||||
if (array_is_list($value)) {
|
||||
return array_map(self::canonicalize(...), $value);
|
||||
}
|
||||
|
||||
ksort($value);
|
||||
foreach ($value as $key => $child) {
|
||||
$value[$key] = self::canonicalize($child);
|
||||
}
|
||||
|
||||
return $value;
|
||||
}
|
||||
|
||||
/**
|
||||
* Encode a value as canonical JSON.
|
||||
*
|
||||
* Use for values stored in MySQL JSON columns where byte-stability
|
||||
* matters across reads/writes.
|
||||
*
|
||||
* @throws \JsonException
|
||||
*/
|
||||
public static function encode(mixed $value): string
|
||||
{
|
||||
return json_encode(
|
||||
self::canonicalize($value),
|
||||
JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ namespace Tests\Feature\FormBuilder\Bindings;
|
||||
|
||||
use App\Models\FormBuilder\FormSchema;
|
||||
use App\Models\Organisation;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabaseState;
|
||||
use Illuminate\Support\Facades\Artisan;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
@@ -150,23 +151,30 @@ final class FormFieldBindingMigrationTest extends TestCase
|
||||
$this->artisan('migrate:rollback', ['--step' => 1])->assertSuccessful();
|
||||
$this->assertFalse(Schema::hasTable('form_field_bindings'));
|
||||
|
||||
// assertEquals: MySQL JSON columns may reorder associative-array
|
||||
// keys on round-trip; structural equality is the contract here.
|
||||
// RFC-WS-6 session 2.7: rollback writes JSON directly from
|
||||
// migration code (not the canonicalizing service). Compare on
|
||||
// canonical form so the assertion is engine-agnostic.
|
||||
$field = DB::table('form_fields')->where('id', $fieldAId)->first();
|
||||
$this->assertNotNull($field->binding);
|
||||
$this->assertEquals([
|
||||
'mode' => 'entity_owned',
|
||||
'entity' => 'person',
|
||||
'column' => 'email',
|
||||
], json_decode((string) $field->binding, true));
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
'mode' => 'entity_owned',
|
||||
'entity' => 'person',
|
||||
'column' => 'email',
|
||||
]),
|
||||
JsonCanonicalizer::encode(json_decode((string) $field->binding, true)),
|
||||
);
|
||||
|
||||
$lib = DB::table('form_field_library')->where('id', $libAId)->first();
|
||||
$this->assertNotNull($lib->default_binding);
|
||||
$this->assertEquals([
|
||||
'mode' => 'entity_owned',
|
||||
'entity' => 'person',
|
||||
'column' => 'first_name',
|
||||
], json_decode((string) $lib->default_binding, true));
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
'mode' => 'entity_owned',
|
||||
'entity' => 'person',
|
||||
'column' => 'first_name',
|
||||
]),
|
||||
JsonCanonicalizer::encode(json_decode((string) $lib->default_binding, true)),
|
||||
);
|
||||
}
|
||||
|
||||
/** @return array{0:string,1:string,2:string} */
|
||||
|
||||
@@ -10,6 +10,7 @@ use App\Models\Organisation;
|
||||
use App\Models\User;
|
||||
use App\Services\FormBuilder\FormFieldConditionalLogicService;
|
||||
use App\Services\FormBuilder\FormFieldService;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabase;
|
||||
use Spatie\Activitylog\Models\Activity;
|
||||
use Tests\TestCase;
|
||||
@@ -75,18 +76,17 @@ final class ConditionalLogicActivityLogPayloadTest extends TestCase
|
||||
$this->assertNotNull($updated, 'field.updated row must exist');
|
||||
|
||||
$properties = $updated->properties;
|
||||
// Structural comparison (assertEquals): MySQL JSON columns may
|
||||
// return associative-array keys in a different order than they were
|
||||
// inserted; semantically the data is unchanged, so use loose
|
||||
// equality. Strict json_encode comparison would couple this test to
|
||||
// a specific DB engine's JSON key-order normalization.
|
||||
$this->assertEquals(
|
||||
$oldShape,
|
||||
$properties->get('old')['conditional_logic'] ?? null,
|
||||
// RFC-WS-6 session 2.7: canonicalized writes give byte-stable
|
||||
// round-trip; both sides go through JsonCanonicalizer::encode so
|
||||
// assertSame compares bytes regardless of MySQL key-order
|
||||
// normalization on the JSON column read.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode($oldShape),
|
||||
JsonCanonicalizer::encode($properties->get('old')['conditional_logic'] ?? null),
|
||||
);
|
||||
$this->assertEquals(
|
||||
$newShape,
|
||||
$properties->get('new')['conditional_logic'] ?? null,
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode($newShape),
|
||||
JsonCanonicalizer::encode($properties->get('new')['conditional_logic'] ?? null),
|
||||
);
|
||||
|
||||
$semantic = Activity::query()
|
||||
|
||||
@@ -6,6 +6,7 @@ namespace Tests\Feature\FormBuilder\ConditionalLogic;
|
||||
|
||||
use App\Models\FormBuilder\FormSchema;
|
||||
use App\Models\Organisation;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabaseState;
|
||||
use Illuminate\Support\Facades\Artisan;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
@@ -176,20 +177,24 @@ final class ConditionalLogicBackfillTest extends TestCase
|
||||
->value('conditional_logic');
|
||||
$this->assertNotNull($reconstructed);
|
||||
$json = json_decode((string) $reconstructed, true);
|
||||
// assertEquals: MySQL JSON columns may reorder associative-array
|
||||
// keys on round-trip; structural equality is the contract here.
|
||||
$this->assertEquals([
|
||||
'show_when' => [
|
||||
'all' => [
|
||||
['field_slug' => 'gate', 'operator' => 'equals', 'value' => 'yes'],
|
||||
[
|
||||
'any' => [
|
||||
['field_slug' => 'region', 'operator' => 'equals', 'value' => 'NL'],
|
||||
// RFC-WS-6 session 2.7: migration's down() reconstructs JSON via
|
||||
// raw DB writer (not the canonicalizing service). Compare on
|
||||
// canonical form so the assertion is engine-agnostic.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
'show_when' => [
|
||||
'all' => [
|
||||
['field_slug' => 'gate', 'operator' => 'equals', 'value' => 'yes'],
|
||||
[
|
||||
'any' => [
|
||||
['field_slug' => 'region', 'operator' => 'equals', 'value' => 'NL'],
|
||||
],
|
||||
],
|
||||
],
|
||||
],
|
||||
],
|
||||
], $json);
|
||||
]),
|
||||
JsonCanonicalizer::encode($json),
|
||||
);
|
||||
|
||||
// Relational tables cleared after reconstruction.
|
||||
$this->assertSame(0, DB::table('form_field_conditional_logic_groups')->count());
|
||||
|
||||
@@ -12,6 +12,7 @@ use App\Models\FormBuilder\FormSchema;
|
||||
use App\Models\Organisation;
|
||||
use App\Models\Scopes\FormFieldOptionScope;
|
||||
use App\Services\FormBuilder\FormFieldOptionService;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Database\QueryException;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabase;
|
||||
use Illuminate\Routing\Route;
|
||||
@@ -191,11 +192,12 @@ final class FormFieldOptionServiceAndScopeTest extends TestCase
|
||||
->where('description', 'field.options_replaced')
|
||||
->first();
|
||||
$this->assertNotNull($fieldEvent);
|
||||
// assertEquals: MySQL JSON columns may reorder associative-array
|
||||
// keys on round-trip; semantic content is what matters here.
|
||||
$this->assertEquals(
|
||||
[['value' => 'a', 'label' => 'A', 'sort_order' => 0]],
|
||||
$fieldEvent->properties->get('options'),
|
||||
// RFC-WS-6 session 2.7: activity log properties are canonicalized
|
||||
// at write; assertSame on canonical encodings of both sides is
|
||||
// byte-stable across MySQL JSON-column round-trip.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([['value' => 'a', 'label' => 'A', 'sort_order' => 0]]),
|
||||
JsonCanonicalizer::encode($fieldEvent->properties->get('options')),
|
||||
);
|
||||
|
||||
$this->assertNull(Activity::query()
|
||||
|
||||
@@ -11,6 +11,7 @@ use App\Models\FormBuilder\FormSchema;
|
||||
use App\Models\Organisation;
|
||||
use App\Services\FormBuilder\FormFieldOptionService;
|
||||
use App\Services\FormBuilder\FormFieldService;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabase;
|
||||
use Spatie\Activitylog\Models\Activity;
|
||||
use Tests\TestCase;
|
||||
@@ -61,22 +62,23 @@ final class FormFieldOptionsActivityLogTest extends TestCase
|
||||
$payload = $event->properties->toArray();
|
||||
$this->assertArrayHasKey('options', $payload['old']);
|
||||
$this->assertArrayHasKey('options', $payload['new']);
|
||||
// assertEquals: MySQL JSON columns may reorder associative-array
|
||||
// keys on round-trip; structural equality is the contract.
|
||||
$this->assertEquals(
|
||||
[
|
||||
// RFC-WS-6 session 2.7: activity log properties are canonicalized
|
||||
// at write; assertSame on canonical encodings of both sides is
|
||||
// byte-stable across MySQL JSON-column round-trip.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
['value' => 'a', 'label' => 'a', 'sort_order' => 0],
|
||||
['value' => 'b', 'label' => 'b', 'sort_order' => 1],
|
||||
],
|
||||
$payload['old']['options'],
|
||||
]),
|
||||
JsonCanonicalizer::encode($payload['old']['options']),
|
||||
);
|
||||
$this->assertEquals(
|
||||
[
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
['value' => 'a', 'label' => 'A', 'sort_order' => 0],
|
||||
['value' => 'b', 'label' => 'b', 'sort_order' => 1],
|
||||
['value' => 'c', 'label' => 'c', 'sort_order' => 2],
|
||||
],
|
||||
$payload['new']['options'],
|
||||
]),
|
||||
JsonCanonicalizer::encode($payload['new']['options']),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ namespace Tests\Feature\FormBuilder\Options;
|
||||
|
||||
use App\Models\FormBuilder\FormSchema;
|
||||
use App\Models\Organisation;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabaseState;
|
||||
use Illuminate\Support\Facades\Artisan;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
@@ -99,13 +100,18 @@ final class FormFieldOptionsBackfillTest extends TestCase
|
||||
$submission = DB::table('form_submissions')->where('id', $submissionId)->first();
|
||||
$snapshot = json_decode((string) $submission->schema_snapshot, true);
|
||||
$field = $snapshot['fields'][0];
|
||||
// assertEquals: MySQL JSON columns may reorder associative-array
|
||||
// keys on round-trip; structural equality is the contract here.
|
||||
$this->assertEquals([
|
||||
['value' => 'XS', 'label' => 'XS', 'sort_order' => 0, 'translations' => ['de' => 'Größe XS']],
|
||||
['value' => 'S', 'label' => 'S', 'sort_order' => 1, 'translations' => ['de' => 'Klein']],
|
||||
['value' => 'M', 'label' => 'M', 'sort_order' => 2, 'translations' => ['de' => 'Mittel']],
|
||||
], $field['options']);
|
||||
// RFC-WS-6 session 2.7: this snapshot was rewritten by the
|
||||
// migration's forward() (raw DB writer, not via the canonicalizing
|
||||
// service). Compare on canonical form so the assertion is
|
||||
// engine-agnostic.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
['value' => 'XS', 'label' => 'XS', 'sort_order' => 0, 'translations' => ['de' => 'Größe XS']],
|
||||
['value' => 'S', 'label' => 'S', 'sort_order' => 1, 'translations' => ['de' => 'Klein']],
|
||||
['value' => 'M', 'label' => 'M', 'sort_order' => 2, 'translations' => ['de' => 'Mittel']],
|
||||
]),
|
||||
JsonCanonicalizer::encode($field['options']),
|
||||
);
|
||||
// Field-level translations bag has the {locale}.options key
|
||||
// stripped.
|
||||
if (is_array($field['translations'] ?? null)) {
|
||||
@@ -117,14 +123,14 @@ final class FormFieldOptionsBackfillTest extends TestCase
|
||||
// Template snapshot rewritten the same way.
|
||||
$template = DB::table('form_templates')->where('id', $templateId)->first();
|
||||
$tplSnap = json_decode((string) $template->schema_snapshot, true);
|
||||
// assertEquals: MySQL JSON columns may reorder associative-array
|
||||
// keys on round-trip; structural equality is the contract here.
|
||||
$this->assertEquals(
|
||||
[
|
||||
// RFC-WS-6 session 2.7: snapshot rewritten by migration; compare
|
||||
// on canonical form to be engine-agnostic.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
['value' => 'A', 'label' => 'A', 'sort_order' => 0],
|
||||
['value' => 'B', 'label' => 'B', 'sort_order' => 1],
|
||||
],
|
||||
$tplSnap['fields'][0]['options'],
|
||||
]),
|
||||
JsonCanonicalizer::encode($tplSnap['fields'][0]['options']),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ use App\Models\FormBuilder\FormField;
|
||||
use App\Models\FormBuilder\FormSchema;
|
||||
use App\Models\Organisation;
|
||||
use App\Services\FormBuilder\FormSubmissionService;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabase;
|
||||
use Tests\TestCase;
|
||||
|
||||
@@ -46,15 +47,16 @@ final class FormFieldOptionsSnapshotAndStrictRequestTest extends TestCase
|
||||
$snapshot = $draft->fresh()->schema_snapshot;
|
||||
$this->assertIsArray($snapshot);
|
||||
$field = collect($snapshot['fields'])->firstWhere('slug', 'shirtmaat');
|
||||
// assertEquals: MySQL JSON columns may reorder associative-array
|
||||
// keys on round-trip; structural equality is the contract.
|
||||
$this->assertEquals(
|
||||
[
|
||||
// RFC-WS-6 session 2.7: schema_snapshot is canonicalized at write,
|
||||
// so byte equality holds when both sides go through
|
||||
// JsonCanonicalizer::encode.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode([
|
||||
['value' => 'XS', 'label' => 'XS', 'sort_order' => 0],
|
||||
['value' => 'S', 'label' => 'S', 'sort_order' => 1],
|
||||
['value' => 'M', 'label' => 'M', 'sort_order' => 2],
|
||||
],
|
||||
$field['options'],
|
||||
]),
|
||||
JsonCanonicalizer::encode($field['options']),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Feature\FormBuilder\Schema;
|
||||
|
||||
use App\Enums\FormBuilder\FormFieldType;
|
||||
use App\Enums\FormBuilder\FormFieldValidationRuleType;
|
||||
use App\Enums\FormBuilder\FormPurpose;
|
||||
use App\Models\FormBuilder\FormField;
|
||||
use App\Models\FormBuilder\FormSchema;
|
||||
use App\Models\FormBuilder\FormSubmission;
|
||||
use App\Models\Organisation;
|
||||
use App\Services\FormBuilder\FormSubmissionService;
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use Illuminate\Foundation\Testing\RefreshDatabase;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
use Tests\TestCase;
|
||||
|
||||
/**
|
||||
* Regression test for MySQL JSON key-order non-determinism.
|
||||
*
|
||||
* Session 2.6 surfaced that MySQL JSON columns may reorder
|
||||
* associative-array keys on round-trip. Session 2.7 introduced
|
||||
* JsonCanonicalizer to stabilize the writers. This test asserts
|
||||
* the contract end-to-end: a submission's schema_snapshot serialized
|
||||
* to canonical JSON must match the canonical serialization of the
|
||||
* raw bytes the DB returns on a separate read.
|
||||
*/
|
||||
final class SchemaSnapshotByteStableAcrossReemitsTest extends TestCase
|
||||
{
|
||||
use RefreshDatabase;
|
||||
|
||||
public function test_snapshot_bytes_are_stable_across_reads(): void
|
||||
{
|
||||
$org = Organisation::factory()->create();
|
||||
$schema = FormSchema::factory()->create([
|
||||
'organisation_id' => $org->id,
|
||||
'purpose' => FormPurpose::EVENT_REGISTRATION,
|
||||
'snapshot_mode' => 'on_submit',
|
||||
'is_published' => true,
|
||||
'public_token' => (string) \Illuminate\Support\Str::ulid(),
|
||||
]);
|
||||
|
||||
// Email field with entity binding + validation rule.
|
||||
FormField::factory()
|
||||
->withValidationRule(FormFieldValidationRuleType::MaxLength, ['value' => 100])
|
||||
->withEntityBinding('person', 'email')
|
||||
->create([
|
||||
'form_schema_id' => $schema->id,
|
||||
'field_type' => FormFieldType::EMAIL->value,
|
||||
'slug' => 'contact_email',
|
||||
'label' => 'E-mail',
|
||||
]);
|
||||
|
||||
// Number field with min/max validation + conditional logic.
|
||||
FormField::factory()
|
||||
->withValidationRule(FormFieldValidationRuleType::MinValue, ['value' => 18])
|
||||
->withValidationRule(FormFieldValidationRuleType::MaxValue, ['value' => 99])
|
||||
->withConditionalLogic([
|
||||
'operator' => 'all',
|
||||
'children' => [
|
||||
['field_slug' => 'contact_email', 'operator' => 'not_empty'],
|
||||
],
|
||||
])
|
||||
->create([
|
||||
'form_schema_id' => $schema->id,
|
||||
'field_type' => FormFieldType::NUMBER->value,
|
||||
'slug' => 'leeftijd',
|
||||
'label' => 'Leeftijd',
|
||||
]);
|
||||
|
||||
// Select field with options + translations.
|
||||
FormField::factory()
|
||||
->withOptions(['XS', 'S', 'M', 'L'])
|
||||
->create([
|
||||
'form_schema_id' => $schema->id,
|
||||
'field_type' => FormFieldType::SELECT->value,
|
||||
'slug' => 'shirtmaat',
|
||||
'label' => 'Shirtmaat',
|
||||
]);
|
||||
|
||||
// Submit so schema_snapshot materializes.
|
||||
$service = resolve(FormSubmissionService::class);
|
||||
$draft = $service->createDraft($schema, null, null, []);
|
||||
$service->submit($draft, null);
|
||||
|
||||
// First read: through Eloquent cast (decode → assoc array).
|
||||
$first = FormSubmission::query()->withoutGlobalScopes()->findOrFail($draft->id);
|
||||
$snapshotA = $first->schema_snapshot;
|
||||
|
||||
// Second read: a fresh model instance (no cached attributes).
|
||||
$second = FormSubmission::query()->withoutGlobalScopes()->findOrFail($draft->id);
|
||||
$snapshotB = $second->schema_snapshot;
|
||||
|
||||
// Third read: raw column bytes via the query builder, decoded once.
|
||||
$rawJson = (string) DB::table('form_submissions')
|
||||
->where('id', $draft->id)
|
||||
->value('schema_snapshot');
|
||||
$snapshotC = json_decode($rawJson, true);
|
||||
|
||||
// All three roads must produce byte-identical canonical JSON.
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode($snapshotA),
|
||||
JsonCanonicalizer::encode($snapshotB),
|
||||
);
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode($snapshotA),
|
||||
JsonCanonicalizer::encode($snapshotC),
|
||||
);
|
||||
|
||||
// And the canonical encode of every JSON-bearing nested fragment
|
||||
// must be byte-identical too — covers each field's options /
|
||||
// validation_rules / configs / bindings / conditional_logic
|
||||
// in one assertion via the whole-snapshot canonical encode.
|
||||
$this->assertNotEmpty($snapshotA['fields']);
|
||||
foreach ($snapshotA['fields'] as $idx => $fieldA) {
|
||||
$fieldC = $snapshotC['fields'][$idx];
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode($fieldA),
|
||||
JsonCanonicalizer::encode($fieldC),
|
||||
"field #{$idx} ({$fieldA['slug']}) drifted across reads",
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
128
api/tests/Unit/Support/Json/JsonCanonicalizerTest.php
Normal file
128
api/tests/Unit/Support/Json/JsonCanonicalizerTest.php
Normal file
@@ -0,0 +1,128 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Tests\Unit\Support\Json;
|
||||
|
||||
use App\Support\Json\JsonCanonicalizer;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
final class JsonCanonicalizerTest extends TestCase
|
||||
{
|
||||
public function test_scalar_passthrough_string(): void
|
||||
{
|
||||
$this->assertSame('hello', JsonCanonicalizer::canonicalize('hello'));
|
||||
}
|
||||
|
||||
/**
|
||||
* @return iterable<string, array{0: int|float|bool|null}>
|
||||
*/
|
||||
public static function scalarProvider(): iterable
|
||||
{
|
||||
yield 'int' => [42];
|
||||
yield 'float' => [3.14];
|
||||
yield 'bool true' => [true];
|
||||
yield 'bool false' => [false];
|
||||
yield 'null' => [null];
|
||||
}
|
||||
|
||||
/**
|
||||
* @dataProvider scalarProvider
|
||||
*/
|
||||
public function test_scalar_passthrough(int|float|bool|null $value): void
|
||||
{
|
||||
// assertSame across the provider keeps PHPStan from narrowing
|
||||
// both sides to a single literal type per call.
|
||||
$this->assertSame($value, JsonCanonicalizer::canonicalize($value));
|
||||
}
|
||||
|
||||
public function test_empty_array(): void
|
||||
{
|
||||
$this->assertSame([], JsonCanonicalizer::canonicalize([]));
|
||||
}
|
||||
|
||||
public function test_numeric_list_preserves_order(): void
|
||||
{
|
||||
$list = ['c', 'a', 'b'];
|
||||
$this->assertSame(['c', 'a', 'b'], JsonCanonicalizer::canonicalize($list));
|
||||
}
|
||||
|
||||
public function test_associative_array_keys_sorted(): void
|
||||
{
|
||||
$assoc = ['c' => 1, 'a' => 2, 'b' => 3];
|
||||
$result = JsonCanonicalizer::canonicalize($assoc);
|
||||
|
||||
$this->assertSame(['a' => 2, 'b' => 3, 'c' => 1], $result);
|
||||
$this->assertSame(['a', 'b', 'c'], array_keys($result));
|
||||
}
|
||||
|
||||
public function test_nested_associative_sorted_recursively(): void
|
||||
{
|
||||
$input = [
|
||||
'z' => ['c' => 1, 'a' => 2],
|
||||
'a' => 'first',
|
||||
];
|
||||
$result = JsonCanonicalizer::canonicalize($input);
|
||||
|
||||
$this->assertSame(['a', 'z'], array_keys($result));
|
||||
$this->assertSame(['a', 'c'], array_keys($result['z']));
|
||||
}
|
||||
|
||||
public function test_list_of_dicts_each_dict_sorted_list_order_preserved(): void
|
||||
{
|
||||
$input = [
|
||||
['c' => 1, 'a' => 2],
|
||||
['z' => 9, 'a' => 8],
|
||||
];
|
||||
$result = JsonCanonicalizer::canonicalize($input);
|
||||
|
||||
// List preserved
|
||||
$this->assertCount(2, $result);
|
||||
$this->assertSame(2, $result[0]['a']);
|
||||
$this->assertSame(8, $result[1]['a']);
|
||||
|
||||
// Each dict sorted
|
||||
$this->assertSame(['a', 'c'], array_keys($result[0]));
|
||||
$this->assertSame(['a', 'z'], array_keys($result[1]));
|
||||
}
|
||||
|
||||
public function test_encode_byte_identical_for_inputs_differing_only_in_key_order(): void
|
||||
{
|
||||
$a = ['name' => 'Alice', 'age' => 30, 'role' => 'admin'];
|
||||
$b = ['role' => 'admin', 'age' => 30, 'name' => 'Alice'];
|
||||
|
||||
$this->assertSame(
|
||||
JsonCanonicalizer::encode($a),
|
||||
JsonCanonicalizer::encode($b),
|
||||
);
|
||||
}
|
||||
|
||||
public function test_encode_uses_unescaped_unicode_and_slashes(): void
|
||||
{
|
||||
$value = ['url' => 'https://example.com/path', 'name' => 'café'];
|
||||
|
||||
$encoded = JsonCanonicalizer::encode($value);
|
||||
|
||||
$this->assertStringContainsString('café', $encoded, 'unicode must be unescaped');
|
||||
$this->assertStringNotContainsString('\\/', $encoded, 'slashes must be unescaped');
|
||||
}
|
||||
|
||||
public function test_mixed_key_array_treated_as_associative(): void
|
||||
{
|
||||
// Mixed keys: keys 0, 1 plus 'foo'. Not a list per array_is_list();
|
||||
// ksort applies, putting numeric-string keys before alphabetic.
|
||||
$input = [0 => 'first', 'foo' => 'bar', 1 => 'second'];
|
||||
$result = JsonCanonicalizer::canonicalize($input);
|
||||
|
||||
$this->assertSame([0, 1, 'foo'], array_keys($result));
|
||||
}
|
||||
|
||||
public function test_empty_dict_inside_canonical_structure(): void
|
||||
{
|
||||
$input = ['z' => [], 'a' => 'x'];
|
||||
$result = JsonCanonicalizer::canonicalize($input);
|
||||
|
||||
$this->assertSame(['a', 'z'], array_keys($result));
|
||||
$this->assertSame([], $result['z']);
|
||||
}
|
||||
}
|
||||
@@ -845,6 +845,28 @@ Rationale: using slugs (not IDs) in cross-references so snapshots are
|
||||
portable between orgs. library_field_id references are resolved to
|
||||
inline definitions at snapshot time (see §4.7).
|
||||
|
||||
##### Byte-stability (RFC-WS-6 v1.1, session 2.7)
|
||||
|
||||
The snapshot is audit-immutable. JSON content stored in
|
||||
`form_submissions.schema_snapshot` is canonicalized on write via
|
||||
`App\Support\Json\JsonCanonicalizer::canonicalize()` (recursive ksort
|
||||
on associative arrays; numeric-indexed lists preserve order). This
|
||||
guarantees that re-emits of the same logical content produce
|
||||
byte-identical JSON regardless of MySQL's JSON-column round-trip
|
||||
behavior. Critical for:
|
||||
|
||||
- Audit-replay diffs (otherwise key-reorder shows up as false positives)
|
||||
- Webhook payload signing (HMAC requires byte-stable input;
|
||||
`payload_snapshot` and the delivery-time `JsonCanonicalizer::encode`
|
||||
re-encode produce the same bytes)
|
||||
- Activity log diff regression tests (`field.updated` `old`/`new`
|
||||
payloads land via `FormField::logFieldChange` which canonicalizes
|
||||
before `withProperties()`)
|
||||
|
||||
Opaque-config columns (`form_schemas.settings`, `form_schemas.translations`)
|
||||
are NOT canonicalized — key order has no semantic meaning there. See
|
||||
`SchemaSnapshotByteStableAcrossReemitsTest` for the end-to-end contract.
|
||||
|
||||
### 4.7 `form_field_library` (cross-schema reusable definitions)
|
||||
|
||||
| Column | Type | Notes |
|
||||
|
||||
Reference in New Issue
Block a user