Consolidate documentation and update backend services

- Reorganize docs into 'Core deployment guides' and 'Setup and configuration' subdirectories - Consolidate redundant documentation files (ACR, pipelines, deployment guides) - Add documentation consolidation plan - Update backend database factory and logger services - Update migration script and docker-compose configurations - Add PostgreSQL setup script
2026-01-22 22:45:54 +01:00
parent 18aec4ad80
commit f4399a8e4e
49 changed files with 1320 additions and 7243 deletions
--- a/guides/AZURE-POSTGRESQL-SETUP.md
+++ b/guides/AZURE-POSTGRESQL-SETUP.md
@@ -0,0 +1,371 @@
+# Azure PostgreSQL Setup for Production
+
+Complete guide for setting up Azure Database for PostgreSQL Flexible Server for CMDB Insight production deployment.
+
+## 🎯 Overview
+
+**Why PostgreSQL for Production?**
+- ✅ Better concurrency handling (multiple users)
+- ✅ Connection pooling support
+- ✅ Better performance for 20+ users
+- ✅ Production-ready database solution
+- ✅ Identical dev/prod stack
+
+**Cost:** ~€20-30/month (Basic B1ms tier)
+
+---
+
+## 📋 Prerequisites
+
+- Azure CLI installed and configured (`az login`)
+- Resource group created: `zdl-cmdb-insight-prd-euwe-rg`
+- Appropriate permissions to create Azure Database resources
+
+---
+
+## 🚀 Quick Setup (15 minutes)
+
+### Step 1: Create PostgreSQL Flexible Server
+
+```bash
+# Set variables
+RESOURCE_GROUP="zdl-cmdb-insight-prd-euwe-rg"
+SERVER_NAME="zdl-cmdb-insight-prd-psql"
+ADMIN_USER="cmdbadmin"
+ADMIN_PASSWORD="$(openssl rand -base64 32)"  # Generate secure password
+LOCATION="westeurope"
+
+# Create PostgreSQL Flexible Server
+az postgres flexible-server create \
+  --resource-group $RESOURCE_GROUP \
+  --name $SERVER_NAME \
+  --location $LOCATION \
+  --admin-user $ADMIN_USER \
+  --admin-password $ADMIN_PASSWORD \
+  --sku-name Standard_B1ms \
+  --tier Burstable \
+  --storage-size 32 \
+  --version 15 \
+  --public-access 0.0.0.0 \
+  --high-availability Disabled
+
+echo "PostgreSQL server created!"
+echo "Server: $SERVER_NAME.postgres.database.azure.com"
+echo "Admin User: $ADMIN_USER"
+echo "Password: $ADMIN_PASSWORD"
+echo ""
+echo "⚠️  Save the password securely!"
+```
+
+### Step 2: Create Database
+
+**Note:** The application uses a single database for all data. All tables (CMDB cache, classification history, and session state) are stored in the same database.
+
+```bash
+# Create main database (this is all you need)
+az postgres flexible-server db create \
+  --resource-group $RESOURCE_GROUP \
+  --server-name $SERVER_NAME \
+  --database-name cmdb_insight
+
+echo "✅ Database created"
+```
+
+
+### Step 3: Configure Firewall Rules
+
+Allow Azure App Service to connect:
+
+```bash
+# Get App Service outbound IPs
+BACKEND_IPS=$(az webapp show \
+  --name zdl-cmdb-insight-prd-backend-webapp \
+  --resource-group $RESOURCE_GROUP \
+  --query "outboundIpAddresses" -o tsv)
+
+# Add firewall rule for App Service (use first IP, or add all)
+az postgres flexible-server firewall-rule create \
+  --resource-group $RESOURCE_GROUP \
+  --name $SERVER_NAME \
+  --rule-name AllowAppService \
+  --start-ip-address 0.0.0.0 \
+  --end-ip-address 255.255.255.255
+
+# Or more secure: Allow Azure Services only
+az postgres flexible-server firewall-rule create \
+  --resource-group $RESOURCE_GROUP \
+  --name $SERVER_NAME \
+  --rule-name AllowAzureServices \
+  --start-ip-address 0.0.0.0 \
+  --end-ip-address 0.0.0.0
+```
+
+**Note:** `0.0.0.0` to `0.0.0.0` allows all Azure services. For production, consider using specific App Service outbound IPs.
+
+### Step 4: Store Credentials in Key Vault
+
+```bash
+KEY_VAULT="zdl-cmdb-insight-prd-kv"
+
+# Store database password
+az keyvault secret set \
+  --vault-name $KEY_VAULT \
+  --name DatabasePassword \
+  --value $ADMIN_PASSWORD
+
+# Store connection string (optional, can construct from components)
+CONNECTION_STRING="postgresql://${ADMIN_USER}:${ADMIN_PASSWORD}@${SERVER_NAME}.postgres.database.azure.com:5432/cmdb_insight?sslmode=require"
+az keyvault secret set \
+  --vault-name $KEY_VAULT \
+  --name DatabaseUrl \
+  --value $CONNECTION_STRING
+
+echo "✅ Credentials stored in Key Vault"
+```
+
+### Step 5: Configure App Service App Settings
+
+```bash
+# Get Key Vault URL
+KV_URL=$(az keyvault show --name $KEY_VAULT --query properties.vaultUri -o tsv)
+
+# Configure backend app settings
+az webapp config appsettings set \
+  --name zdl-cmdb-insight-prd-backend-webapp \
+  --resource-group $RESOURCE_GROUP \
+  --settings \
+    DATABASE_TYPE=postgres \
+    DATABASE_HOST="${SERVER_NAME}.postgres.database.azure.com" \
+    DATABASE_PORT=5432 \
+    DATABASE_NAME=cmdb_insight \
+    DATABASE_USER=$ADMIN_USER \
+    DATABASE_PASSWORD="@Microsoft.KeyVault(SecretUri=${KV_URL}secrets/DatabasePassword/)" \
+    DATABASE_SSL=true
+
+echo "✅ App settings configured"
+```
+
+**Alternative: Use DATABASE_URL directly**
+
+```bash
+az webapp config appsettings set \
+  --name zdl-cmdb-insight-prd-backend-webapp \
+  --resource-group $RESOURCE_GROUP \
+  --settings \
+    DATABASE_TYPE=postgres \
+    DATABASE_URL="@Microsoft.KeyVault(SecretUri=${KV_URL}secrets/DatabaseUrl/)"
+```
+
+---
+
+## 🔐 Security Best Practices
+
+### 1. Use Key Vault for Secrets
+
+✅ **Do:** Store database password in Key Vault  
+❌ **Don't:** Store password in app settings directly
+
+### 2. Enable SSL/TLS
+
+✅ **Do:** Always use `DATABASE_SSL=true` or `?sslmode=require` in connection string  
+❌ **Don't:** Connect without SSL in production
+
+### 3. Firewall Rules
+
+✅ **Do:** Restrict to specific IPs or Azure services  
+❌ **Don't:** Allow `0.0.0.0/0` (all IPs) unless necessary
+
+### 4. Use Managed Identity (Advanced)
+
+For even better security, use Managed Identity instead of passwords:
+
+```bash
+# Enable Managed Identity on PostgreSQL server
+az postgres flexible-server identity assign \
+  --resource-group $RESOURCE_GROUP \
+  --name $SERVER_NAME \
+  --identity /subscriptions/.../resourceGroups/.../providers/Microsoft.ManagedIdentity/userAssignedIdentities/...
+
+# Grant access
+az postgres flexible-server ad-admin create \
+  --resource-group $RESOURCE_GROUP \
+  --server-name $SERVER_NAME \
+  --display-name "App Service Identity" \
+  --object-id <principal-id>
+```
+
+---
+
+## 📊 Database Configuration
+
+### Connection Pooling
+
+The application uses connection pooling automatically via the `pg` library:
+- **Max connections:** 20 (configured in `PostgresAdapter`)
+- **Idle timeout:** 30 seconds
+- **Connection timeout:** 10 seconds
+
+### Database Sizes
+
+For 20 users:
+- **Database (cmdb_insight):** ~25-60MB total (includes CMDB cache, classification history, and session state)
+- **Total storage:** 32GB (plenty of room for growth)
+
+**Note:** All data (CMDB objects, classification history, and session state) is stored in a single database.
+
+---
+
+## 🔄 Migration from SQLite
+
+If you're migrating from SQLite to PostgreSQL:
+
+```bash
+# 1. Export data from SQLite (if needed)
+# The application will automatically sync from Jira, so migration may not be necessary
+
+# 2. Set DATABASE_TYPE=postgres in app settings
+
+# 3. Restart the app - it will create tables automatically on first run
+
+# 4. The app will sync data from Jira Assets on first sync
+```
+
+**Note:** Since the database is a cache layer that syncs from Jira, you typically don't need to migrate data - just let it sync fresh.
+
+---
+
+## 🧪 Testing Connection
+
+### Test from Local Machine
+
+```bash
+# Install psql if needed
+# macOS: brew install postgresql
+# Ubuntu: sudo apt-get install postgresql-client
+
+# Connect (replace with your values)
+psql "host=${SERVER_NAME}.postgres.database.azure.com port=5432 dbname=cmdb_insight user=${ADMIN_USER} password=${ADMIN_PASSWORD} sslmode=require"
+```
+
+### Test from App Service
+
+```bash
+# Check app logs
+az webapp log tail \
+  --name zdl-cmdb-insight-prd-backend-webapp \
+  --resource-group $RESOURCE_GROUP
+
+# Look for: "Creating PostgreSQL adapter" or connection errors
+```
+
+---
+
+## 📈 Monitoring
+
+### Check Database Status
+
+```bash
+az postgres flexible-server show \
+  --resource-group $RESOURCE_GROUP \
+  --name $SERVER_NAME \
+  --query "{state:state, version:version, sku:sku}"
+```
+
+### View Database Size
+
+```sql
+-- Connect to database
+SELECT 
+  pg_database.datname,
+  pg_size_pretty(pg_database_size(pg_database.datname)) AS size
+FROM pg_database
+WHERE datname = 'cmdb_insight';
+```
+
+### Monitor Connections
+
+```sql
+SELECT 
+  count(*) as total_connections,
+  state,
+  application_name
+FROM pg_stat_activity
+WHERE datname = 'cmdb_insight'
+GROUP BY state, application_name;
+```
+
+---
+
+## 💰 Cost Optimization
+
+### Current Setup (Recommended)
+
+- **Tier:** Burstable (B1ms)
+- **vCores:** 1
+- **RAM:** 2GB
+- **Storage:** 32GB
+- **Cost:** ~€20-30/month
+
+### If You Need More Performance
+
+- **Upgrade to:** Standard_B2s (2 vCores, 4GB RAM) - ~€40-50/month
+- **Or:** Standard_B1ms with more storage if needed
+
+### Cost Savings Tips
+
+1. **Use Burstable tier** - Perfect for 20 users
+2. **Start with 32GB storage** - Can scale up later
+3. **Disable high availability** - Not needed for small teams
+4. **Use same region** - Reduces latency and costs
+
+---
+
+## 🛠️ Troubleshooting
+
+### Connection Refused
+
+**Problem:** Can't connect to database
+
+**Solutions:**
+1. Check firewall rules: `az postgres flexible-server firewall-rule list --resource-group $RESOURCE_GROUP --name $SERVER_NAME`
+2. Verify SSL is enabled: `DATABASE_SSL=true`
+3. Check credentials in Key Vault
+
+### Authentication Failed
+
+**Problem:** Wrong username/password
+
+**Solutions:**
+1. Verify admin user: `az postgres flexible-server show --resource-group $RESOURCE_GROUP --name $SERVER_NAME --query administratorLogin`
+2. Reset password if needed: `az postgres flexible-server update --resource-group $RESOURCE_GROUP --name $SERVER_NAME --admin-password "new-password"`
+
+### SSL Required Error
+
+**Problem:** "SSL connection required"
+
+**Solution:** Add `DATABASE_SSL=true` or `?sslmode=require` to connection string
+
+---
+
+## 📚 Related Documentation
+
+- **`docs/AZURE-APP-SERVICE-DEPLOYMENT.md`** - Complete App Service deployment
+- **`docs/DATABASE-RECOMMENDATION.md`** - Database comparison and recommendations
+- **`docs/LOCAL-DEVELOPMENT-SETUP.md`** - Local PostgreSQL setup
+
+---
+
+## ✅ Checklist
+
+- [ ] PostgreSQL Flexible Server created
+- [ ] Database created (cmdb_insight)
+- [ ] Firewall rules configured
+- [ ] Credentials stored in Key Vault
+- [ ] App Service app settings configured
+- [ ] SSL enabled (`DATABASE_SSL=true`)
+- [ ] Connection tested
+- [ ] Monitoring configured
+
+---
+
+**🎉 Your PostgreSQL database is ready for production!**