cmdb-insight/docs/REFACTOR-PHASE-2B-3-STATUS.md

Refactor Phase 2B + 3: Implementation Status

Date: 2025-01-XX
Status: ✅ Phase 2B Complete - New Architecture Implemented
Next: Phase 3 - Migration & Cleanup

Summary

New refactored architecture has been fully implemented and wired behind feature flag USE_V2_API=true. All new services, repositories, and API controllers are in place.

---

✅ Completed Components

Infrastructure Layer (/infrastructure)

1. infrastructure/jira/JiraAssetsClient.ts ✅
   • Pure HTTP API client (no business logic)
   • Methods: getObject(), searchObjects(), updateObject(), getSchemas(), getObjectTypes(), getAttributes()
   • Token management (service account for reads, user PAT for writes)
   • Returns ObjectEntry from domain types

Domain Layer (/domain)

1. domain/jiraAssetsPayload.ts ✅ (Phase 2A)

   • Complete API payload contract
   • Type guards: isReferenceValue(), isSimpleValue(), hasAttributes()

2. domain/syncPolicy.ts ✅

   • SyncPolicy enum (ENABLED, REFERENCE_ONLY, SKIP)
   • Policy resolution logic

Repository Layer (/repositories)

1. repositories/SchemaRepository.ts ✅

   • Schema CRUD: upsertSchema(), getAllSchemas()
   • Object type CRUD: upsertObjectType(), getEnabledObjectTypes(), getObjectTypeByJiraId()
   • Attribute CRUD: upsertAttribute(), getAttributesForType(), getAttributeByFieldName()

2. repositories/ObjectCacheRepository.ts ✅

   • Object CRUD: upsertObject(), getObject(), getObjectByKey(), deleteObject()
   • Attribute value CRUD: upsertAttributeValue(), batchUpsertAttributeValues(), getAttributeValues(), deleteAttributeValues()
   • Relations: upsertRelation(), deleteRelations()
   • Queries: getObjectsByType(), countObjectsByType()

Service Layer (/services)

1. services/PayloadProcessor.ts ✅

   • Recursive reference processing with visited-set cycle detection
   • Processes ObjectEntry and ReferencedObject recursively (level2, level3, etc.)
   • CRITICAL: Only replaces attributes if attributes[] array is present
   • Extracts relations from references
   • Normalizes to EAV format

2. services/SchemaSyncService.ts ✅

   • Syncs schemas from Jira API: syncAllSchemas()
   • Discovers and stores object types and attributes
   • Returns enabled types for sync orchestration

3. services/ObjectSyncService.ts ✅

   • Full sync: syncObjectType() - syncs all objects of an enabled type
   • Incremental sync: syncIncremental() - syncs objects updated since timestamp
   • Single object sync: syncSingleObject() - for refresh operations
   • Recursive processing via PayloadProcessor
   • Respects SyncPolicy (ENABLED vs REFERENCE_ONLY)

4. services/QueryService.ts ✅

   • Universal query builder (DB → TypeScript)
   • getObject() - reconstruct single object
   • getObjects() - list objects of type
   • countObjects() - count by type
   • searchByLabel() - search by label

5. services/RefreshService.ts ✅

   • Force-refresh-on-read with deduplication
   • Locking mechanism prevents duplicate refresh operations
   • Timeout protection (30s)

6. services/WriteThroughService.ts ✅

   • Write-through updates: Jira API → DB cache
   • Builds Jira update payload from field updates
   • Uses same normalization logic as sync

7. services/ServiceFactory.ts ✅

   • Singleton factory for all services
   • Initializes all dependencies
   • Single entry point: getServices()

API Layer (/api)

1. api/controllers/ObjectsController.ts ✅

   • GET /api/v2/objects/:type - List objects
   • GET /api/v2/objects/:type/:id?refresh=true - Get object (with force refresh)
   • PUT /api/v2/objects/:type/:id - Update object

2. api/controllers/SyncController.ts ✅

   • POST /api/v2/sync/schemas - Sync all schemas
   • POST /api/v2/sync/objects - Sync all enabled types
   • POST /api/v2/sync/objects/:typeName - Sync single type

3. api/routes/v2.ts ✅

   • V2 routes mounted at /api/v2
   • Feature flag: USE_V2_API=true enables routes
   • All routes require authentication

Integration (/backend/src/index.ts)

✅ V2 routes wired with feature flag
✅ Token management for new JiraAssetsClient
✅ Backward compatible with old services

---

🔧 Key Features Implemented

1. Recursive Reference Processing ✅

• Cycle detection: Visited set using objectId:objectKey keys
• Recursive expansion: Processes referencedObject.attributes[] (level2, level3, etc.)
• Preserves shallow objects: Doesn't wipe attributes if attributes[] absent

2. Sync Policy Enforcement ✅

• ENABLED: Full sync with all attributes
• REFERENCE_ONLY: Cache minimal metadata for references
• SKIP: Unknown types skipped

3. Attribute Replacement Logic ✅

CRITICAL RULE: Only replaces attributes if attributes[] array is present in API response.

if (shouldCacheAttributes) {
  // attributes[] present - full replace
  await deleteAttributeValues(objectId);
  await batchUpsertAttributeValues(...);
}
// If attributes[] absent - keep existing attributes

4. Write-Through Updates ✅

1. Build Jira update payload
2. Send to Jira Assets API
3. Fetch fresh data
4. Update DB cache using same normalization

5. Force Refresh with Deduping ✅

• Lock mechanism prevents duplicate refreshes
• Timeout protection (30s)
• Concurrent reads allowed

---

📁 File Structure

backend/src/
├── domain/
│   ├── jiraAssetsPayload.ts        ✅ Phase 2A
│   └── syncPolicy.ts               ✅ New
├── infrastructure/
│   └── jira/
│       └── JiraAssetsClient.ts    ✅ New (pure API)
├── repositories/
│   ├── SchemaRepository.ts        ✅ New
│   └── ObjectCacheRepository.ts   ✅ New
├── services/
│   ├── PayloadProcessor.ts        ✅ New (recursive)
│   ├── SchemaSyncService.ts       ✅ New
│   ├── ObjectSyncService.ts       ✅ New
│   ├── QueryService.ts            ✅ New
│   ├── RefreshService.ts          ✅ New
│   ├── WriteThroughService.ts     ✅ New
│   └── ServiceFactory.ts          ✅ New
└── api/
    ├── controllers/
    │   ├── ObjectsController.ts   ✅ New
    │   └── SyncController.ts      ✅ New
    └── routes/
        └── v2.ts                  ✅ New

---

🚀 Usage (Feature Flag)

Enable V2 API

# .env
USE_V2_API=true

New Endpoints

GET    /api/v2/objects/:type                    # List objects
GET    /api/v2/objects/:type/:id?refresh=true   # Get object (with refresh)
PUT    /api/v2/objects/:type/:id                # Update object

POST   /api/v2/sync/schemas                     # Sync all schemas
POST   /api/v2/sync/objects                     # Sync all enabled types
POST   /api/v2/sync/objects/:typeName           # Sync single type

---

✅ API Payload Contract Compliance

All services correctly handle:

• ✅ objectEntries[] → ObjectEntry[]
• ✅ ObjectEntry.attributes[] → ObjectAttribute[] (optional)
• ✅ ObjectAttribute.objectAttributeValues[] → ObjectAttributeValue union
• ✅ ReferencedObject.attributes[] → Recursive (level2+)
• ✅ Cycle detection with visited sets
• ✅ CRITICAL: Don't wipe attributes if attributes[] absent on shallow objects

---

🧪 Testing Status

Compilation: ✅ New code compiles without errors (pre-existing TypeScript config issues unrelated)

Ready for Testing:

1. Enable USE_V2_API=true
2. Test new endpoints
3. Verify recursive reference processing
4. Verify attribute replacement logic
5. Verify write-through updates

---

📋 Next Steps (Phase 3)

Step 1: Test V2 API ✅ (Ready)

• Enable feature flag
• Test schema sync endpoint
• Test object sync endpoint
• Test object read endpoint
• Test object write endpoint
• Verify recursive references processed
• Verify attribute replacement logic

Step 2: Migrate Existing Endpoints

After V2 API is validated:

• Update routes/objects.ts to use new services
• Update routes/cache.ts to use new services
• Update routes/schema.ts to use new services

Step 3: Delete Old Code

After migration complete:

• Delete services/jiraAssets.ts (merge remaining business logic first)
• Delete services/jiraAssetsClient.ts (replaced by infrastructure client)
• Delete services/cacheStore.old.ts
• Delete services/normalizedCacheStore.ts (replace with repositories)
• Delete services/queryBuilder.ts (functionality in QueryService)
• Delete services/schemaDiscoveryService.ts (replaced by SchemaSyncService)
• Delete services/schemaCacheService.ts (merged into SchemaRepository)
• Delete services/schemaConfigurationService.ts (functionality moved to SchemaRepository)
• Delete services/schemaMappingService.ts (deprecated)
• Delete services/syncEngine.ts (replaced by ObjectSyncService)
• Delete services/cmdbService.ts (functionality split into QueryService + WriteThroughService + RefreshService)

---

⚠️ Important Notes

1. No Functional Changes Yet: Old code still runs in parallel
2. Feature Flag Required: V2 API only active when USE_V2_API=true
3. Token Management: New client receives tokens from middleware (same as old)
4. Database Schema: Uses existing normalized EAV schema (no migration needed)

---

End of Phase 2B + 3 Implementation Status