Bert Hausmans
cdee0e8819
- Restore blue PageHeader on Dashboard (/app-components) - Update homepage (/) with subtle header design without blue bar - Add uniform PageHeader styling to application edit page - Fix Rapporten link on homepage to point to /reports overview - Improve header descriptions spacing for better readability
284 lines
9.5 KiB
Markdown
284 lines
9.5 KiB
Markdown
# Refactor Phase 2B + 3: Implementation Status
|
|
|
|
**Date:** 2025-01-XX
|
|
**Status:** ✅ Phase 2B Complete - New Architecture Implemented
|
|
**Next:** Phase 3 - Migration & Cleanup
|
|
|
|
## Summary
|
|
|
|
New refactored architecture has been fully implemented and wired behind feature flag `USE_V2_API=true`. All new services, repositories, and API controllers are in place.
|
|
|
|
---
|
|
|
|
## ✅ Completed Components
|
|
|
|
### Infrastructure Layer (`/infrastructure`)
|
|
|
|
1. **`infrastructure/jira/JiraAssetsClient.ts`** ✅
|
|
- Pure HTTP API client (no business logic)
|
|
- Methods: `getObject()`, `searchObjects()`, `updateObject()`, `getSchemas()`, `getObjectTypes()`, `getAttributes()`
|
|
- Token management (service account for reads, user PAT for writes)
|
|
- Returns `ObjectEntry` from domain types
|
|
|
|
### Domain Layer (`/domain`)
|
|
|
|
1. **`domain/jiraAssetsPayload.ts`** ✅ (Phase 2A)
|
|
- Complete API payload contract
|
|
- Type guards: `isReferenceValue()`, `isSimpleValue()`, `hasAttributes()`
|
|
|
|
2. **`domain/syncPolicy.ts`** ✅
|
|
- `SyncPolicy` enum (ENABLED, REFERENCE_ONLY, SKIP)
|
|
- Policy resolution logic
|
|
|
|
### Repository Layer (`/repositories`)
|
|
|
|
1. **`repositories/SchemaRepository.ts`** ✅
|
|
- Schema CRUD: `upsertSchema()`, `getAllSchemas()`
|
|
- Object type CRUD: `upsertObjectType()`, `getEnabledObjectTypes()`, `getObjectTypeByJiraId()`
|
|
- Attribute CRUD: `upsertAttribute()`, `getAttributesForType()`, `getAttributeByFieldName()`
|
|
|
|
2. **`repositories/ObjectCacheRepository.ts`** ✅
|
|
- Object CRUD: `upsertObject()`, `getObject()`, `getObjectByKey()`, `deleteObject()`
|
|
- Attribute value CRUD: `upsertAttributeValue()`, `batchUpsertAttributeValues()`, `getAttributeValues()`, `deleteAttributeValues()`
|
|
- Relations: `upsertRelation()`, `deleteRelations()`
|
|
- Queries: `getObjectsByType()`, `countObjectsByType()`
|
|
|
|
### Service Layer (`/services`)
|
|
|
|
1. **`services/PayloadProcessor.ts`** ✅
|
|
- **Recursive reference processing** with visited-set cycle detection
|
|
- Processes `ObjectEntry` and `ReferencedObject` recursively (level2, level3, etc.)
|
|
- **CRITICAL**: Only replaces attributes if `attributes[]` array is present
|
|
- Extracts relations from references
|
|
- Normalizes to EAV format
|
|
|
|
2. **`services/SchemaSyncService.ts`** ✅
|
|
- Syncs schemas from Jira API: `syncAllSchemas()`
|
|
- Discovers and stores object types and attributes
|
|
- Returns enabled types for sync orchestration
|
|
|
|
3. **`services/ObjectSyncService.ts`** ✅
|
|
- Full sync: `syncObjectType()` - syncs all objects of an enabled type
|
|
- Incremental sync: `syncIncremental()` - syncs objects updated since timestamp
|
|
- Single object sync: `syncSingleObject()` - for refresh operations
|
|
- Recursive processing via `PayloadProcessor`
|
|
- Respects `SyncPolicy` (ENABLED vs REFERENCE_ONLY)
|
|
|
|
4. **`services/QueryService.ts`** ✅
|
|
- Universal query builder (DB → TypeScript)
|
|
- `getObject()` - reconstruct single object
|
|
- `getObjects()` - list objects of type
|
|
- `countObjects()` - count by type
|
|
- `searchByLabel()` - search by label
|
|
|
|
5. **`services/RefreshService.ts`** ✅
|
|
- Force-refresh-on-read with deduplication
|
|
- Locking mechanism prevents duplicate refresh operations
|
|
- Timeout protection (30s)
|
|
|
|
6. **`services/WriteThroughService.ts`** ✅
|
|
- Write-through updates: Jira API → DB cache
|
|
- Builds Jira update payload from field updates
|
|
- Uses same normalization logic as sync
|
|
|
|
7. **`services/ServiceFactory.ts`** ✅
|
|
- Singleton factory for all services
|
|
- Initializes all dependencies
|
|
- Single entry point: `getServices()`
|
|
|
|
### API Layer (`/api`)
|
|
|
|
1. **`api/controllers/ObjectsController.ts`** ✅
|
|
- `GET /api/v2/objects/:type` - List objects
|
|
- `GET /api/v2/objects/:type/:id?refresh=true` - Get object (with force refresh)
|
|
- `PUT /api/v2/objects/:type/:id` - Update object
|
|
|
|
2. **`api/controllers/SyncController.ts`** ✅
|
|
- `POST /api/v2/sync/schemas` - Sync all schemas
|
|
- `POST /api/v2/sync/objects` - Sync all enabled types
|
|
- `POST /api/v2/sync/objects/:typeName` - Sync single type
|
|
|
|
3. **`api/routes/v2.ts`** ✅
|
|
- V2 routes mounted at `/api/v2`
|
|
- Feature flag: `USE_V2_API=true` enables routes
|
|
- All routes require authentication
|
|
|
|
### Integration (`/backend/src/index.ts`)
|
|
|
|
✅ V2 routes wired with feature flag
|
|
✅ Token management for new `JiraAssetsClient`
|
|
✅ Backward compatible with old services
|
|
|
|
---
|
|
|
|
## 🔧 Key Features Implemented
|
|
|
|
### 1. Recursive Reference Processing ✅
|
|
|
|
- **Cycle detection**: Visited set using `objectId:objectKey` keys
|
|
- **Recursive expansion**: Processes `referencedObject.attributes[]` (level2, level3, etc.)
|
|
- **Preserves shallow objects**: Doesn't wipe attributes if `attributes[]` absent
|
|
|
|
### 2. Sync Policy Enforcement ✅
|
|
|
|
- **ENABLED**: Full sync with all attributes
|
|
- **REFERENCE_ONLY**: Cache minimal metadata for references
|
|
- **SKIP**: Unknown types skipped
|
|
|
|
### 3. Attribute Replacement Logic ✅
|
|
|
|
**CRITICAL RULE**: Only replaces attributes if `attributes[]` array is present in API response.
|
|
|
|
```typescript
|
|
if (shouldCacheAttributes) {
|
|
// attributes[] present - full replace
|
|
await deleteAttributeValues(objectId);
|
|
await batchUpsertAttributeValues(...);
|
|
}
|
|
// If attributes[] absent - keep existing attributes
|
|
```
|
|
|
|
### 4. Write-Through Updates ✅
|
|
|
|
1. Build Jira update payload
|
|
2. Send to Jira Assets API
|
|
3. Fetch fresh data
|
|
4. Update DB cache using same normalization
|
|
|
|
### 5. Force Refresh with Deduping ✅
|
|
|
|
- Lock mechanism prevents duplicate refreshes
|
|
- Timeout protection (30s)
|
|
- Concurrent reads allowed
|
|
|
|
---
|
|
|
|
## 📁 File Structure
|
|
|
|
```
|
|
backend/src/
|
|
├── domain/
|
|
│ ├── jiraAssetsPayload.ts ✅ Phase 2A
|
|
│ └── syncPolicy.ts ✅ New
|
|
├── infrastructure/
|
|
│ └── jira/
|
|
│ └── JiraAssetsClient.ts ✅ New (pure API)
|
|
├── repositories/
|
|
│ ├── SchemaRepository.ts ✅ New
|
|
│ └── ObjectCacheRepository.ts ✅ New
|
|
├── services/
|
|
│ ├── PayloadProcessor.ts ✅ New (recursive)
|
|
│ ├── SchemaSyncService.ts ✅ New
|
|
│ ├── ObjectSyncService.ts ✅ New
|
|
│ ├── QueryService.ts ✅ New
|
|
│ ├── RefreshService.ts ✅ New
|
|
│ ├── WriteThroughService.ts ✅ New
|
|
│ └── ServiceFactory.ts ✅ New
|
|
└── api/
|
|
├── controllers/
|
|
│ ├── ObjectsController.ts ✅ New
|
|
│ └── SyncController.ts ✅ New
|
|
└── routes/
|
|
└── v2.ts ✅ New
|
|
```
|
|
|
|
---
|
|
|
|
## 🚀 Usage (Feature Flag)
|
|
|
|
### Enable V2 API
|
|
|
|
```bash
|
|
# .env
|
|
USE_V2_API=true
|
|
```
|
|
|
|
### New Endpoints
|
|
|
|
```
|
|
GET /api/v2/objects/:type # List objects
|
|
GET /api/v2/objects/:type/:id?refresh=true # Get object (with refresh)
|
|
PUT /api/v2/objects/:type/:id # Update object
|
|
|
|
POST /api/v2/sync/schemas # Sync all schemas
|
|
POST /api/v2/sync/objects # Sync all enabled types
|
|
POST /api/v2/sync/objects/:typeName # Sync single type
|
|
```
|
|
|
|
---
|
|
|
|
## ✅ API Payload Contract Compliance
|
|
|
|
All services correctly handle:
|
|
|
|
- ✅ `objectEntries[]` → `ObjectEntry[]`
|
|
- ✅ `ObjectEntry.attributes[]` → `ObjectAttribute[]` (optional)
|
|
- ✅ `ObjectAttribute.objectAttributeValues[]` → `ObjectAttributeValue` union
|
|
- ✅ `ReferencedObject.attributes[]` → Recursive (level2+)
|
|
- ✅ Cycle detection with visited sets
|
|
- ✅ **CRITICAL**: Don't wipe attributes if `attributes[]` absent on shallow objects
|
|
|
|
---
|
|
|
|
## 🧪 Testing Status
|
|
|
|
**Compilation**: ✅ New code compiles without errors (pre-existing TypeScript config issues unrelated)
|
|
|
|
**Ready for Testing**:
|
|
1. Enable `USE_V2_API=true`
|
|
2. Test new endpoints
|
|
3. Verify recursive reference processing
|
|
4. Verify attribute replacement logic
|
|
5. Verify write-through updates
|
|
|
|
---
|
|
|
|
## 📋 Next Steps (Phase 3)
|
|
|
|
### Step 1: Test V2 API ✅ (Ready)
|
|
|
|
- [ ] Enable feature flag
|
|
- [ ] Test schema sync endpoint
|
|
- [ ] Test object sync endpoint
|
|
- [ ] Test object read endpoint
|
|
- [ ] Test object write endpoint
|
|
- [ ] Verify recursive references processed
|
|
- [ ] Verify attribute replacement logic
|
|
|
|
### Step 2: Migrate Existing Endpoints
|
|
|
|
After V2 API is validated:
|
|
|
|
- [ ] Update `routes/objects.ts` to use new services
|
|
- [ ] Update `routes/cache.ts` to use new services
|
|
- [ ] Update `routes/schema.ts` to use new services
|
|
|
|
### Step 3: Delete Old Code
|
|
|
|
After migration complete:
|
|
|
|
- [ ] Delete `services/jiraAssets.ts` (merge remaining business logic first)
|
|
- [ ] Delete `services/jiraAssetsClient.ts` (replaced by infrastructure client)
|
|
- [ ] Delete `services/cacheStore.old.ts`
|
|
- [ ] Delete `services/normalizedCacheStore.ts` (replace with repositories)
|
|
- [ ] Delete `services/queryBuilder.ts` (functionality in QueryService)
|
|
- [ ] Delete `services/schemaDiscoveryService.ts` (replaced by SchemaSyncService)
|
|
- [ ] Delete `services/schemaCacheService.ts` (merged into SchemaRepository)
|
|
- [ ] Delete `services/schemaConfigurationService.ts` (functionality moved to SchemaRepository)
|
|
- [ ] Delete `services/schemaMappingService.ts` (deprecated)
|
|
- [ ] Delete `services/syncEngine.ts` (replaced by ObjectSyncService)
|
|
- [ ] Delete `services/cmdbService.ts` (functionality split into QueryService + WriteThroughService + RefreshService)
|
|
|
|
---
|
|
|
|
## ⚠️ Important Notes
|
|
|
|
1. **No Functional Changes Yet**: Old code still runs in parallel
|
|
2. **Feature Flag Required**: V2 API only active when `USE_V2_API=true`
|
|
3. **Token Management**: New client receives tokens from middleware (same as old)
|
|
4. **Database Schema**: Uses existing normalized EAV schema (no migration needed)
|
|
|
|
---
|
|
|
|
**End of Phase 2B + 3 Implementation Status**
|