cmdb-insight/docs/JIRA-ASSETS-SYNC-REFACTOR-PLAN-2025-01-21

# Cursor AI Prompt: Jira Assets Schema Synchronization

## Context

This application syncs Jira Assets (Data Center) data to a local database with a generic structure. Your task is to review, implement, and/or modify the schema synchronization feature that fetches the complete Jira Assets configuration structure.

## Objective

Implement or verify the schema sync functionality that extracts the complete Jira Assets schema structure using the REST API. This includes:
- Object Schemas
- Object Types (with hierarchy)
- Object Type Attributes (field definitions)

**Note:** This task focuses on syncing the *structure/configuration* only, not the actual object data.

---

## API Reference

### Base URL
```
{JIRA_BASE_URL}/rest/assets/1.0
```

### Authentication
- HTTP Basic Authentication (username + password/API token)
- All requests require `Accept: application/json` header

---

## Required API Endpoints & Response Structures

### 1. List All Schemas
```
GET /rest/assets/1.0/objectschema/list
```

**Response Structure:**
```json
{
  "objectschemas": [
    {
      "id": 1,
      "name": "IT Assets",
      "objectSchemaKey": "IT",
      "status": "Ok",
      "description": "IT Asset Management Schema",
      "created": "2024-01-15T10:30:00.000Z",
      "updated": "2024-01-20T14:45:00.000Z",
      "objectCount": 1500,
      "objectTypeCount": 25
    }
  ]
}
```

**Fields to Store:**
| Field | Type | Description |
|-------|------|-------------|
| id | integer | Primary identifier |
| name | string | Schema name |
| objectSchemaKey | string | Unique key (e.g., "IT") |
| status | string | Schema status |
| description | string | Optional description |
| created | datetime | Creation timestamp |
| updated | datetime | Last modification |
| objectCount | integer | Total objects in schema |
| objectTypeCount | integer | Total object types |

---

### 2. Get Schema Details
```
GET /rest/assets/1.0/objectschema/:id
```

**Response Structure:**
```json
{
  "id": 1,
  "name": "IT Assets",
  "objectSchemaKey": "IT",
  "status": "Ok",
  "description": "IT Asset Management Schema",
  "created": "2024-01-15T10:30:00.000Z",
  "updated": "2024-01-20T14:45:00.000Z",
  "objectCount": 1500,
  "objectTypeCount": 25
}
```

---

### 3. Get Object Types (Flat List)
```
GET /rest/assets/1.0/objectschema/:id/objecttypes/flat
```

**Response Structure:**
```json
[
  {
    "id": 10,
    "name": "Hardware",
    "type": 0,
    "description": "Physical hardware assets",
    "icon": {
      "id": 1,
      "name": "Computer",
      "url16": "/rest/assets/1.0/icon/1/16",
      "url48": "/rest/assets/1.0/icon/1/48"
    },
    "position": 0,
    "created": "2024-01-15T10:30:00.000Z",
    "updated": "2024-01-20T14:45:00.000Z",
    "objectCount": 500,
    "parentObjectTypeId": null,
    "objectSchemaId": 1,
    "inherited": false,
    "abstractObjectType": false
  },
  {
    "id": 11,
    "name": "Computer",
    "type": 0,
    "description": "Desktop and laptop computers",
    "icon": {
      "id": 2,
      "name": "Laptop",
      "url16": "/rest/assets/1.0/icon/2/16",
      "url48": "/rest/assets/1.0/icon/2/48"
    },
    "position": 0,
    "created": "2024-01-15T10:35:00.000Z",
    "updated": "2024-01-20T14:50:00.000Z",
    "objectCount": 200,
    "parentObjectTypeId": 10,
    "objectSchemaId": 1,
    "inherited": true,
    "abstractObjectType": false
  }
]
```

**Fields to Store:**
| Field | Type | Description |
|-------|------|-------------|
| id | integer | Primary identifier |
| name | string | Object type name |
| type | integer | Type classification (0=normal) |
| description | string | Optional description |
| icon | object | Icon details (id, name, url16, url48) |
| position | integer | Display position in hierarchy |
| created | datetime | Creation timestamp |
| updated | datetime | Last modification |
| objectCount | integer | Number of objects of this type |
| parentObjectTypeId | integer/null | Parent type ID (null if root) |
| objectSchemaId | integer | Parent schema ID |
| inherited | boolean | Whether attributes are inherited |
| abstractObjectType | boolean | Whether type is abstract (no direct objects) |

---

### 4. Get Object Type Details
```
GET /rest/assets/1.0/objecttype/:id
```

**Response Structure:** Same as individual item in the flat list above.

---

### 5. Get Object Type Attributes
```
GET /rest/assets/1.0/objecttype/:id/attributes
```

**Response Structure:**
```json
[
  {
    "id": 100,
    "objectType": {
      "id": 11,
      "name": "Computer"
    },
    "name": "Name",
    "label": true,
    "type": 0,
    "description": "Asset name/label",
    "defaultType": {
      "id": 0,
      "name": "Text"
    },
    "typeValue": null,
    "typeValueMulti": [],
    "additionalValue": null,
    "referenceType": null,
    "referenceObjectTypeId": null,
    "referenceObjectType": null,
    "editable": true,
    "system": true,
    "sortable": true,
    "summable": false,
    "indexed": true,
    "minimumCardinality": 1,
    "maximumCardinality": 1,
    "suffix": "",
    "removable": false,
    "hidden": false,
    "includeChildObjectTypes": false,
    "uniqueAttribute": false,
    "regexValidation": null,
    "iql": null,
    "options": "",
    "position": 0
  },
  {
    "id": 101,
    "objectType": {
      "id": 11,
      "name": "Computer"
    },
    "name": "Serial Number",
    "label": false,
    "type": 0,
    "description": "Device serial number",
    "defaultType": {
      "id": 0,
      "name": "Text"
    },
    "typeValue": null,
    "typeValueMulti": [],
    "additionalValue": null,
    "referenceType": null,
    "referenceObjectTypeId": null,
    "referenceObjectType": null,
    "editable": true,
    "system": false,
    "sortable": true,
    "summable": false,
    "indexed": true,
    "minimumCardinality": 0,
    "maximumCardinality": 1,
    "suffix": "",
    "removable": true,
    "hidden": false,
    "includeChildObjectTypes": false,
    "uniqueAttribute": true,
    "regexValidation": "^[A-Z0-9]{10,20}$",
    "iql": null,
    "options": "",
    "position": 1
  },
  {
    "id": 102,
    "objectType": {
      "id": 11,
      "name": "Computer"
    },
    "name": "Assigned User",
    "label": false,
    "type": 2,
    "description": "User assigned to this asset",
    "defaultType": null,
    "typeValue": "SHOW_ON_ASSET",
    "typeValueMulti": [],
    "additionalValue": null,
    "referenceType": null,
    "referenceObjectTypeId": null,
    "referenceObjectType": null,
    "editable": true,
    "system": false,
    "sortable": true,
    "summable": false,
    "indexed": true,
    "minimumCardinality": 0,
    "maximumCardinality": 1,
    "suffix": "",
    "removable": true,
    "hidden": false,
    "includeChildObjectTypes": false,
    "uniqueAttribute": false,
    "regexValidation": null,
    "iql": null,
    "options": "",
    "position": 2
  },
  {
    "id": 103,
    "objectType": {
      "id": 11,
      "name": "Computer"
    },
    "name": "Location",
    "label": false,
    "type": 1,
    "description": "Physical location of the asset",
    "defaultType": null,
    "typeValue": null,
    "typeValueMulti": [],
    "additionalValue": null,
    "referenceType": {
      "id": 1,
      "name": "Reference",
      "description": "Standard reference",
      "color": "#0052CC",
      "url16": null,
      "removable": false,
      "objectSchemaId": 1
    },
    "referenceObjectTypeId": 20,
    "referenceObjectType": {
      "id": 20,
      "name": "Location",
      "objectSchemaId": 1
    },
    "editable": true,
    "system": false,
    "sortable": true,
    "summable": false,
    "indexed": true,
    "minimumCardinality": 0,
    "maximumCardinality": 1,
    "suffix": "",
    "removable": true,
    "hidden": false,
    "includeChildObjectTypes": true,
    "uniqueAttribute": false,
    "regexValidation": null,
    "iql": "objectType = Location",
    "options": "",
    "position": 3
  },
  {
    "id": 104,
    "objectType": {
      "id": 11,
      "name": "Computer"
    },
    "name": "Status",
    "label": false,
    "type": 7,
    "description": "Current asset status",
    "defaultType": null,
    "typeValue": "1",
    "typeValueMulti": ["1", "2", "3"],
    "additionalValue": null,
    "referenceType": null,
    "referenceObjectTypeId": null,
    "referenceObjectType": null,
    "editable": true,
    "system": false,
    "sortable": true,
    "summable": false,
    "indexed": true,
    "minimumCardinality": 1,
    "maximumCardinality": 1,
    "suffix": "",
    "removable": true,
    "hidden": false,
    "includeChildObjectTypes": false,
    "uniqueAttribute": false,
    "regexValidation": null,
    "iql": null,
    "options": "",
    "position": 4
  }
]
```

**Attribute Fields to Store:**
| Field | Type | Description |
|-------|------|-------------|
| id | integer | Attribute ID |
| objectType | object | Parent object type {id, name} |
| name | string | Attribute name |
| label | boolean | Is this the label/display attribute |
| type | integer | Attribute type (see type reference below) |
| description | string | Optional description |
| defaultType | object/null | Default type info {id, name} for type=0 |
| typeValue | string/null | Type-specific configuration |
| typeValueMulti | array | Multiple type values (e.g., allowed status IDs) |
| additionalValue | string/null | Additional configuration |
| referenceType | object/null | Reference type details for type=1 |
| referenceObjectTypeId | integer/null | Target object type ID for references |
| referenceObjectType | object/null | Target object type details |
| editable | boolean | Can values be edited |
| system | boolean | Is system attribute (Name, Key, Created, Updated) |
| sortable | boolean | Can sort by this attribute |
| summable | boolean | Can sum values (numeric types) |
| indexed | boolean | Is indexed for search |
| minimumCardinality | integer | Minimum required values (0=optional, 1=required) |
| maximumCardinality | integer | Maximum values (-1=unlimited, 1=single) |
| suffix | string | Display suffix (e.g., "GB", "USD") |
| removable | boolean | Can attribute be deleted |
| hidden | boolean | Is hidden from default view |
| includeChildObjectTypes | boolean | Include child types in reference selection |
| uniqueAttribute | boolean | Must values be unique |
| regexValidation | string/null | Validation regex pattern |
| iql | string/null | IQL/AQL filter for reference selection |
| options | string | Additional options (CSV for Select type) |
| position | integer | Display order position |

---

## Attribute Type Reference

### Main Types (type field)
| Type | Name | Description | Uses defaultType |
|------|------|-------------|------------------|
| 0 | Default | Uses defaultType for specific type | Yes |
| 1 | Object | Reference to another Assets object | No |
| 2 | User | Jira user reference | No |
| 3 | Confluence | Confluence page reference | No |
| 4 | Group | Jira group reference | No |
| 5 | Version | Jira version reference | No |
| 6 | Project | Jira project reference | No |
| 7 | Status | Status type reference | No |

### Default Types (defaultType.id when type=0)
| ID | Name | Description |
|----|------|-------------|
| 0 | Text | Single-line text |
| 1 | Integer | Whole number |
| 2 | Boolean | True/False checkbox |
| 3 | Double | Decimal number |
| 4 | Date | Date only (no time) |
| 5 | Time | Time only (no date) |
| 6 | DateTime | Date and time |
| 7 | URL | Web link |
| 8 | Email | Email address |
| 9 | Textarea | Multi-line text |
| 10 | Select | Dropdown selection (options in `options` field) |
| 11 | IP Address | IP address format |

---

## Implementation Requirements

### 1. Sync Flow

Implement the following synchronization flow:

```
┌─────────────────────────────────────────────────────────────┐
│                    Schema Sync Process                       │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  1. GET /objectschema/list                                   │
│     └── Store/Update all schemas in local DB                 │
│                                                              │
│  2. For each schema:                                         │
│     ├── GET /objectschema/:id                                │
│     │   └── Update schema details                            │
│     │                                                        │
│     └── GET /objectschema/:id/objecttypes/flat               │
│         └── Store/Update all object types                    │
│                                                              │
│  3. For each object type:                                    │
│     ├── GET /objecttype/:id (optional, for latest details)   │
│     │                                                        │
│     └── GET /objecttype/:id/attributes                       │
│         └── Store/Update all attributes                      │
│                                                              │
│  4. Clean up orphaned records (deleted in Jira)              │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

### 2. Database Operations

For each entity type, implement:
- **Upsert logic**: Insert new records, update existing ones based on Jira ID
- **Soft delete or cleanup**: Handle items that exist locally but not in Jira anymore
- **Relationship mapping**: Maintain foreign key relationships (schema → object types → attributes)

### 3. Rate Limiting

Implement rate limiting to avoid overloading the Jira server:
- Add 100-200ms delay between API requests
- Implement exponential backoff on 429 (Too Many Requests) responses
- Maximum 3-5 concurrent requests if using parallel processing

```typescript
// Example rate limiting implementation
const delay = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));

async function fetchWithRateLimit<T>(url: string): Promise<T> {
  await delay(150); // 150ms between requests
  
  const response = await fetch(url, { headers: getAuthHeaders() });
  
  if (response.status === 429) {
    const retryAfter = parseInt(response.headers.get('Retry-After') || '5');
    await delay(retryAfter * 1000);
    return fetchWithRateLimit(url);
  }
  
  return response.json();
}
```

### 4. Error Handling

Handle these scenarios:
- **401 Unauthorized**: Invalid credentials
- **403 Forbidden**: Insufficient permissions
- **404 Not Found**: Schema/Type deleted during sync
- **429 Too Many Requests**: Rate limited (implement backoff)
- **5xx Server Errors**: Retry with exponential backoff

### 5. Progress Tracking

Implement progress reporting:
- Total schemas to process
- Current schema being processed
- Total object types to process
- Current object type being processed
- Estimated time remaining (optional)

---

## Code Structure Suggestions

### Service/Repository Pattern

```
src/
├── services/
│   └── jira-assets/
│       ├── JiraAssetsApiClient.ts      # HTTP client with auth & rate limiting
│       ├── SchemaSyncService.ts        # Main sync orchestration
│       ├── ObjectTypeSyncService.ts    # Object type sync logic
│       └── AttributeSyncService.ts     # Attribute sync logic
├── repositories/
│   ├── SchemaRepository.ts             # Schema DB operations
│   ├── ObjectTypeRepository.ts         # Object type DB operations
│   └── AttributeRepository.ts          # Attribute DB operations
└── models/
    ├── Schema.ts                       # Schema entity/model
    ├── ObjectType.ts                   # Object type entity/model
    └── ObjectTypeAttribute.ts          # Attribute entity/model
```

### Sync Service Interface

```typescript
interface SchemaSyncService {
  /**
   * Sync all schemas and their complete structure
   * @returns Summary of sync operation
   */
  syncAll(): Promise<SyncResult>;
  
  /**
   * Sync a single schema by ID
   * @param schemaId - Jira schema ID
   */
  syncSchema(schemaId: number): Promise<SyncResult>;
  
  /**
   * Get sync status/progress
   */
  getProgress(): SyncProgress;
}

interface SyncResult {
  success: boolean;
  schemasProcessed: number;
  objectTypesProcessed: number;
  attributesProcessed: number;
  errors: SyncError[];
  duration: number; // milliseconds
}

interface SyncProgress {
  status: 'idle' | 'running' | 'completed' | 'failed';
  currentSchema?: string;
  currentObjectType?: string;
  schemasTotal: number;
  schemasCompleted: number;
  objectTypesTotal: number;
  objectTypesCompleted: number;
  startedAt?: Date;
  estimatedCompletion?: Date;
}
```

---

## Validation Checklist

After implementation, verify:

- [ ] All schemas are fetched from `/objectschema/list`
- [ ] Schema details are updated from `/objectschema/:id`
- [ ] All object types are fetched for each schema from `/objectschema/:id/objecttypes/flat`
- [ ] Object type hierarchy (parentObjectTypeId) is preserved
- [ ] All attributes are fetched for each object type from `/objecttype/:id/attributes`
- [ ] Attribute types are correctly mapped (type + defaultType)
- [ ] Reference attributes store referenceObjectTypeId and referenceType
- [ ] Status attributes store typeValueMulti (allowed status IDs)
- [ ] Rate limiting prevents 429 errors
- [ ] Error handling covers all failure scenarios
- [ ] Sync can be resumed after failure
- [ ] Orphaned local records are handled (deleted in Jira)
- [ ] Foreign key relationships are maintained
- [ ] Timestamps (created, updated) are stored correctly

---

## Testing Scenarios

1. **Initial sync**: Empty local database, full sync from Jira
2. **Incremental sync**: Existing data, detect changes
3. **Schema added**: New schema created in Jira
4. **Schema deleted**: Schema removed from Jira
5. **Object type added**: New type in existing schema
6. **Object type moved**: Parent changed in hierarchy
7. **Attribute added/modified/removed**: Changes to type attributes
8. **Large schema**: Schema with 50+ object types, 500+ attributes
9. **Network failure**: Handle timeouts and retries
10. **Rate limiting**: Handle 429 responses gracefully

---

## Notes

- The `/objectschema/:id/objecttypes/flat` endpoint returns ALL object types in one call, which is more efficient than fetching hierarchically
- The `label` field on attributes indicates which attribute is used as the display name for objects
- System attributes (system=true) are: Name, Key, Created, Updated - these exist on all object types
- The `iql` field on reference attributes contains the filter query for selecting valid reference targets
- The `options` field on Select type attributes (type=0, defaultType.id=10) contains comma-separated options