Files
crewli/api/app/Jobs/FormBuilder/DeliverFormWebhookJob.php
bert.hausmans a791a276fa fix(form-builder): canonicalize JSON for byte-stable storage (WS-6)
MySQL 8.0 JSON columns may reorder associative-array keys on
round-trip. For audit-immutable values (schema snapshots, webhook
payloads, activity log diffs), this is corrupting: re-emits produce
different byte sequences for the same logical content.

Introduced JsonCanonicalizer (recursive ksort on associative arrays;
numeric-indexed lists preserve order) and applied at every writer
site that produces byte-stable JSON:

- FormSubmissionService: canonicalize the schema_snapshot array
  before storage (audit-immutable per ARCH §4.3, RFC-WS-6 v1.1).
- FormField::logFieldChange / FormSchema::logSchemaChange: canonicalize
  activity-log properties before withProperties() so old/new diffs
  read back byte-stable.
- BindingActivityLogger: canonicalize both the pass-level and
  per-binding activity properties.
- FormWebhookDispatcher: canonicalize payload_snapshot before
  storage (delivery-time HMAC re-encodes the same canonical bytes).
- DeliverFormWebhookJob: switched json_encode to
  JsonCanonicalizer::encode for the HMAC-signed body, so the
  signature is byte-stable across re-deliveries and reproducible by
  receivers from the same logical payload.

Sites NOT canonicalized (deliberate):
- form_schemas.settings — opaque UI config; key order has no
  semantic meaning, no byte-stability requirement.
- form_schemas.translations / form_fields.translations — read by
  display layer; key order doesn't matter.
- form_templates.schema_snapshot — user-supplied input via store/
  update; user is the source of truth, not audit-immutable in the
  same way as form_submissions.schema_snapshot.

Reverted the 7 assertEquals workarounds from session 2.6:
- ConditionalLogicActivityLogPayloadTest
- ConditionalLogicBackfillTest::test_rollback_reconstructs_canonical_json
- FormFieldBindingMigrationTest::test_rollback_reconstructs_json_and_drops_table
- FormFieldOptionServiceAndScopeTest::test_replace_options_emits_activity_log_on_field_only
- FormFieldOptionsActivityLogTest::test_field_updated_payload_contains_options_diff_when_options_change
- FormFieldOptionsBackfillTest::test_forward_migration_backfills_rows_strips_translations_and_rewrites_snapshot
- FormFieldOptionsSnapshotAndStrictRequestTest::test_submission_snapshot_embeds_rich_shape_options

Each now uses assertSame on JsonCanonicalizer::encode of both sides —
byte-stable comparison meaningful regardless of MySQL JSON storage
behavior.

New regression test SchemaSnapshotByteStableAcrossReemitsTest
exercises the contract end-to-end: complex schema with bindings,
validation rules, options, conditional logic, submitted; reads
schema_snapshot via three roads (Eloquent cast, fresh model, raw
bytes) and asserts the canonical encode is identical.

ARCH-FORM-BUILDER.md §4.6.1 gets a "Byte-stability" sub-section
explaining what's canonicalized and why.

Test count: 1388 → 1400 (+11 JsonCanonicalizer unit, +1 snapshot
regression). Larastan clean. Rector dry-run unchanged at 355.

Refs: WS-6 session 2.6 deviation #4 cleanup, RFC-WS-6 v1.1

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:11:18 +02:00

194 lines
6.6 KiB
PHP
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<?php
declare(strict_types=1);
namespace App\Jobs\FormBuilder;
use App\Enums\FormBuilder\FormWebhookDeliveryStatus;
use App\Models\FormBuilder\FormWebhookDelivery;
use App\Support\Json\JsonCanonicalizer;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;
/**
* HTTP delivery with HMAC signing, SSRF protection, exponential backoff
* retries, and dead-letter on exhaustion (ARCH §17.5.3§17.5.4, §22.10).
*/
final class DeliverFormWebhookJob implements ShouldQueue
{
use Dispatchable;
use InteractsWithQueue;
use Queueable;
use SerializesModels;
/** @var array<int, int> Retry delays in seconds per attempt. */
public array $backoff = [60, 300, 1800, 7200, 28800];
public int $timeout = 30;
public int $tries = 5;
public function __construct(public readonly string $deliveryId) {}
public function handle(): void
{
/** @var FormWebhookDelivery|null $delivery */
$delivery = FormWebhookDelivery::query()->find($this->deliveryId);
if ($delivery === null) {
return;
}
$webhook = $delivery->webhook;
if ($webhook === null || ! $webhook->is_active) {
$delivery->status = FormWebhookDeliveryStatus::FAILED->value;
$delivery->failed_permanently_at = now();
$delivery->save();
return;
}
$url = (string) $webhook->url;
if (! $this->urlIsSafe($url)) {
$delivery->status = FormWebhookDeliveryStatus::FAILED->value;
$delivery->failed_permanently_at = now();
$delivery->response_body_excerpt = 'SSRF protection: URL rejected.';
$delivery->save();
return;
}
// RFC-WS-6 session 2.7 — canonical JSON for HMAC signing.
// payload_snapshot was read from a MySQL JSON column whose key
// order may not match what we wrote. Canonicalize so the
// signature is byte-stable across re-deliveries and matches what
// a verifying receiver computes from the same logical payload.
$payload = (array) ($delivery->payload_snapshot ?? []);
$body = JsonCanonicalizer::encode($payload);
$headers = ['Content-Type' => 'application/json'];
if (! empty($webhook->secret)) {
$headers['X-Crewli-Signature'] = 'sha256='.hash_hmac('sha256', $body, (string) $webhook->secret);
}
$delivery->attempts = (int) $delivery->attempts + 1;
$delivery->last_attempt_at = now();
try {
$response = Http::withHeaders($headers)
->timeout((int) config('form_builder.webhooks.timeout_seconds', 10))
->withBody($body, 'application/json')
->post($url);
$delivery->response_status = $response->status();
$delivery->response_body_excerpt = mb_substr((string) $response->body(), 0, 1000);
if ($response->successful()) {
$delivery->status = FormWebhookDeliveryStatus::DELIVERED->value;
$delivery->delivered_at = now();
$delivery->save();
return;
}
if ($this->isRetriable($response->status()) && $delivery->attempts < $this->tries) {
$delivery->status = FormWebhookDeliveryStatus::PENDING->value;
$delivery->next_retry_at = now()->addSeconds($this->backoff[$delivery->attempts - 1] ?? 28800);
$delivery->save();
$this->release($this->backoff[$delivery->attempts - 1] ?? 28800);
return;
}
$delivery->status = $delivery->attempts >= $this->tries
? FormWebhookDeliveryStatus::DEAD_LETTER->value
: FormWebhookDeliveryStatus::FAILED->value;
$delivery->failed_permanently_at = now();
$delivery->save();
} catch (\Throwable $e) {
Log::warning('form-webhook.delivery.exception', [
'delivery_id' => $delivery->id,
'message' => $e->getMessage(),
]);
if ($delivery->attempts < $this->tries) {
$delivery->status = FormWebhookDeliveryStatus::PENDING->value;
$delivery->next_retry_at = now()->addSeconds($this->backoff[$delivery->attempts - 1] ?? 28800);
$delivery->save();
$this->release($this->backoff[$delivery->attempts - 1] ?? 28800);
return;
}
$delivery->status = FormWebhookDeliveryStatus::DEAD_LETTER->value;
$delivery->failed_permanently_at = now();
$delivery->save();
}
}
private function urlIsSafe(string $url): bool
{
$parts = parse_url($url);
if ($parts === false || ! isset($parts['host'], $parts['scheme'])) {
return false;
}
if (! in_array(strtolower($parts['scheme']), ['http', 'https'], true)) {
return false;
}
$host = (string) $parts['host'];
$allowlist = (array) config('form_builder.webhooks.allowlist_domains', []);
if ($allowlist !== [] && ! in_array(strtolower($host), array_map('strtolower', $allowlist), true)) {
return false;
}
$ips = @gethostbynamel($host) ?: [$host];
$blocklist = (array) config('form_builder.webhooks.blocklist_ips', []);
foreach ($ips as $ip) {
if (! filter_var($ip, FILTER_VALIDATE_IP)) {
continue;
}
foreach ($blocklist as $cidr) {
if ($this->ipInCidr($ip, (string) $cidr)) {
return false;
}
}
if (! filter_var($ip, FILTER_VALIDATE_IP, FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE)) {
return false;
}
}
return true;
}
private function ipInCidr(string $ip, string $cidr): bool
{
if (! str_contains($cidr, '/')) {
return $ip === $cidr;
}
[$subnet, $mask] = explode('/', $cidr, 2);
$ipLong = ip2long($ip);
$subnetLong = ip2long($subnet);
if ($ipLong === false || $subnetLong === false) {
return false;
}
$maskLong = -1 << (32 - (int) $mask);
return ($ipLong & $maskLong) === ($subnetLong & $maskLong);
}
private function isRetriable(int $status): bool
{
if ($status >= 500) {
return true;
}
return in_array($status, [408, 425, 429], true);
}
}