cmdb-insight/docs/SCHEMA-DISCOVERY-FLOW.md

Schema Discovery and Data Loading Flow

Overview

This document describes the logical flow for setting up and using CMDB Insight, from schema discovery to data viewing.

Implementation Verification

✅ Data Structure Alignment: The data structure from Jira Assets REST API matches the discovered schema structure. There are no fallbacks or guessing - the system uses the discovered schema exclusively.

How It Works

1. Schema Discovery (/api/schema-configuration/discover):

   • Discovers all schemas and object types from Jira Assets API
   • Stores them in the schemas and object_types database tables

2. Attribute Discovery (automatic after schema discovery):

   • schemaDiscoveryService.discoverAndStoreSchema() fetches attributes for each object type
   • Stores attributes in the attributes table with:
     • jira_attr_id - Jira's attribute ID
     • field_name - Our internal field name (camelCase)
     • attr_type - Data type (text, reference, integer, etc.)
     • is_multiple - Whether it's a multi-value field

3. Data Loading (when syncing):

   • jiraAssetsClient.parseObject() uses the discovered schema from the database
   • Maps Jira API attributes to our field names using jira_attr_id matching
   • No fallbacks - if a type is not discovered, the object is skipped (returns null)

Code Flow

Schema Discovery → Database (attributes table) → schemaCacheService.getSchema() 
→ OBJECT_TYPES_CACHE → jiraAssetsClient.parseObject() → CMDBObject

User Flow

Step 1: Schema Discovery & Configuration

Page: /settings/schema-configuration

1. Discover Schemas: Click "Ontdek Schemas & Object Types"

   • Fetches all schemas and object types from Jira Assets
   • Stores them in the database

2. Configure Object Types: Enable/disable which object types to sync

   • By default, all object types are disabled
   • Enable the object types you want to sync

3. Manual Sync: Click "Nu synchroniseren" (in CacheStatusIndicator)

   • Loads data from Jira Assets REST API
   • Uses the discovered schema structure to map attributes
   • Stores objects in the normalized database

Step 2: View Data Structure

Page: /settings/data-model

• View the discovered schema structure
• See object types, attributes, and relationships
• Verify that the structure matches your Jira Assets configuration

Step 3: Validate Data

Page: /settings/data-validation

• Validate data integrity
• Check for missing or invalid data
• Debug data loading issues

Step 4: Use Application Components

Pages: /application/overview, /app-components, etc.

• View and manage application components
• All data uses the discovered schema structure

Navigation Structure

The navigation menu follows this logical flow:

1. Setup - Initial configuration

   • Schema Configuratie (discover + configure + sync)
   • Datamodel Overzicht (view structure)

2. Data - Data management

   • Datamodel (view structure)
   • Data Validatie (validate data)

3. Application Component - Application management

   • Dashboard, Overzicht, FTE Calculator

4. Rapporten - Reports and analytics

5. Apps - Additional tools

   • BIA Sync

6. Instellingen - Advanced configuration

   • Data Completeness Config
   • FTE Config

7. Beheer - Administration

   • Users, Roles, Debug

Key Points

• ✅ No guessing: The system uses the discovered schema exclusively
• ✅ No fallbacks: If a type is not discovered, objects are skipped
• ✅ Schema-driven: All data mapping uses the discovered schema structure
• ✅ Database-driven: Schema is stored in the database, not hardcoded

Troubleshooting

If data is not loading correctly:

1. Check Schema Discovery: Ensure schemas and object types are discovered
2. Check Configuration: Ensure at least one object type is enabled
3. Check Attributes: Verify that attributes are discovered (check /settings/data-model)
4. Check Logs: Look for "Unknown object type" or "Type definition not found" warnings