rsyslog/MIGRATION_SUMMARY.md

# Migration Summary: Pushgateway → gitops-status-server

This document summarizes the changes made to migrate the rsyslog repository from Pushgateway metrics to JSON status updates for gitops-status-server.

## What Changed

### 1. New Files Created

#### `update-gitops-status.sh`
- Main script that orchestrates the status update flow
- Runs Ansible drift-check playbook
- Parses output to extract changed file names
- Builds structured JSON payload
- Sends JSON to gitops-status-server via HTTP POST
- Handles both SYNCED and OUT_OF_SYNC states

#### `GITOPS_STATUS_INTEGRATION.md`
- Documentation explaining the integration with gitops-status-server
- API endpoint specification
- JSON payload format examples
- Architecture diagram
- Error handling details

### 2. Modified Files

#### `.woodpecker.yml`
**Changes:**
- Updated header comments to reflect new JSON status flow
- Renamed step: `update-sync-metric` → `update-gitops-status`
- Removed Pushgateway environment variable (`PUSHGATEWAY_URL`)
- Added new environment variables:
  - `GITOPS_STATUS_SERVER_URL`
  - `REPO_NAME`
  - `SERVER_NAME`
- Added `jq` package installation for JSON formatting
- Added `bash` package to cron step (required by update-gitops-status.sh)
- Updated both `update-gitops-status` and `gitops_sync_check` steps to call new script
- Removed Pushgateway metric push logic
- Added JSON status update logic

**Step: `update-gitops-status` (formerly `update-sync-metric`)**
- Runs after successful deployment
- Calls `update-gitops-status.sh`
- Always succeeds (status sent regardless of drift)

**Step: `gitops_sync_check`**
- Runs on cron schedule (every 2 minutes)
- Calls `update-gitops-status.sh` to send JSON
- Then checks drift status to determine pipeline success/failure
- Pipeline fails if drift detected (for visibility in Woodpecker UI)
- JSON status always sent before checking drift

#### `README.md`
**Changes:**
- Updated pipeline flow diagrams
- Replaced "Prometheus" with "gitops-status-server" in diagrams
- Removed "sync metric" section
- Added "GitOps status JSON format" section with examples
- Updated pipeline step descriptions to mention JSON status
- Added `update-gitops-status.sh` to repository structure
- Added optional environment variables table
- Updated flow descriptions to explain file-level drift details

### 3. Unchanged Files

The following files remain unchanged and continue to work as before:
- `ansible/playbooks/drift-check.yml` - Still works as-is
- `ansible/playbooks/apply.yml` - Deploy logic unchanged
- `ansible/playbooks/validate.yml` - Validation logic unchanged
- `ansible/inventory/hosts.yml` - Inventory unchanged
- `ansible/inventory/group_vars/all.yml` - Variables unchanged
- `ansible.cfg` - Ansible config unchanged
- `apply.sh` - Local apply script unchanged
- `drift-check.sh` - Local drift check script unchanged
- `files/rsyslog.conf` - Config files unchanged
- `files/rsyslog.d/30-lab.conf` - Config files unchanged

## Behavior Changes

### Before (Pushgateway)

```
drift-check → calculate status (0 or 1) → push to Pushgateway → Prometheus scrapes
```

Output:
```
gitops_sync_status{repo="rsyslog",server="rsyslog-lab"} 1
```

### After (gitops-status-server)

```
drift-check → extract changed files → build JSON → POST to gitops-status-server → Grafana queries
```

Output:
```json
{
  "repo": "rsyslog",
  "server": "rsyslog-lab",
  "sync_status": "OUT_OF_SYNC",
  "drift_count": 2,
  "files": [
    { "name": "rsyslog.conf" },
    { "name": "rsyslog.d/30-lab.conf" }
  ],
  "last_check": "2026-04-21T10:32:15Z"
}
```

## Benefits

1. **Richer data**: File-level drift information instead of just a binary status
2. **Better visualization**: Grafana can display which specific files have drifted
3. **Detailed tracking**: Know exactly what changed, not just that something changed
4. **Timestamp tracking**: Last check time included in JSON
5. **Drift count**: Quick numeric indicator of severity
6. **Extensible**: JSON format can be easily extended with additional fields

## Migration Checklist

- [x] Create `update-gitops-status.sh` script
- [x] Update `.woodpecker.yml` pipeline
- [x] Update `README.md` documentation
- [x] Create `GITOPS_STATUS_INTEGRATION.md` integration docs
- [x] Remove Pushgateway environment variables
- [x] Add gitops-status-server environment variables
- [x] Update pipeline step names
- [x] Add required packages (jq, bash)
- [x] Test JSON generation logic
- [x] Update flow diagrams

## Testing Recommendations

1. **Test the script locally:**
   ```bash
   export GITOPS_STATUS_SERVER_URL="http://localhost:80"
   export REPO_NAME="rsyslog"
   export SERVER_NAME="rsyslog-lab"
   ./update-gitops-status.sh
   ```

2. **Test in Woodpecker:**
   - Trigger a push to master → check `update-gitops-status` step
   - Wait for cron run → check `gitops_sync_check` step
   - Manually edit a file on server → wait for next cron → verify OUT_OF_SYNC status

3. **Verify gitops-status-server:**
   - Check that JSON is received at POST endpoint
   - Verify `/status.json` serves the latest data
   - Confirm Grafana dashboard displays drift details

## Rollback Plan

If needed, the old Pushgateway approach can be restored by:
1. Reverting `.woodpecker.yml` to previous version
2. Removing `update-gitops-status.sh`
3. Restoring Pushgateway environment variables

All Ansible playbooks remain unchanged, so they will work with either approach.

## Notes

- The rsyslog repo now focuses only on status generation and sending
- gitops-status-server is responsible for serving data to Grafana
- No changes to observability-stack app are needed on the rsyslog side
- This migration is specific to rsyslog repo; other repos can follow same pattern