118 lines
4.1 KiB
Markdown
118 lines
4.1 KiB
Markdown
# GitOps Status Server Integration
|
|
|
|
This document explains how the rsyslog repository integrates with gitops-status-server for GitOps monitoring.
|
|
|
|
## Overview
|
|
|
|
Instead of pushing simple numeric metrics to Prometheus Pushgateway, the rsyslog repo now sends structured JSON status snapshots to gitops-status-server. This enables richer visualization in Grafana with file-level drift details.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────┐ ┌──────────────────────┐ ┌─────────────────────┐
|
|
│ Woodpecker CI │ │ gitops-status-server │ │ Grafana │
|
|
│ (rsyslog) │────────►│ (Kubernetes) │────────►│ Infinity Plugin │
|
|
│ │ POST │ │ GET │ │
|
|
│ drift-check │ JSON │ serves /status.json │ │ Dashboard shows │
|
|
│ every 2 min │ │ │ │ drift details │
|
|
└─────────────────┘ └──────────────────────┘ └─────────────────────┘
|
|
```
|
|
|
|
## API Endpoint
|
|
|
|
The rsyslog repo sends JSON status updates to:
|
|
|
|
```
|
|
POST http://gitops-status-server.observability-stack.svc.cluster.local:80/api/status
|
|
Content-Type: application/json
|
|
```
|
|
|
|
## JSON Payload Format
|
|
|
|
### When synced (no drift)
|
|
|
|
```json
|
|
{
|
|
"repo": "rsyslog",
|
|
"server": "rsyslog-lab",
|
|
"sync_status": "SYNCED",
|
|
"drift_count": 0,
|
|
"files": [],
|
|
"last_check": "2026-04-21T10:30:00Z"
|
|
}
|
|
```
|
|
|
|
### When drift detected
|
|
|
|
```json
|
|
{
|
|
"repo": "rsyslog",
|
|
"server": "rsyslog-lab",
|
|
"sync_status": "OUT_OF_SYNC",
|
|
"drift_count": 2,
|
|
"files": [
|
|
{ "name": "rsyslog.conf" },
|
|
{ "name": "rsyslog.d/30-lab.conf" }
|
|
],
|
|
"last_check": "2026-04-21T10:32:15Z"
|
|
}
|
|
```
|
|
|
|
## Field Definitions
|
|
|
|
| Field | Type | Description |
|
|
|----------------|----------|-------------------------------------------------------|
|
|
| `repo` | string | Repository name (e.g., "rsyslog") |
|
|
| `server` | string | Target server name (e.g., "rsyslog-lab") |
|
|
| `sync_status` | string | Either "SYNCED" or "OUT_OF_SYNC" |
|
|
| `drift_count` | integer | Number of files that have drifted from Git |
|
|
| `files` | array | List of files with drift (empty if synced) |
|
|
| `files[].name` | string | Relative path of drifted file |
|
|
| `last_check` | string | ISO 8601 timestamp of when drift check was performed |
|
|
|
|
## When Updates Are Sent
|
|
|
|
1. **After deployment** (push to master):
|
|
- Post-deploy verification runs
|
|
- JSON status sent to gitops-status-server
|
|
- Pipeline step: `update-gitops-status`
|
|
|
|
2. **Scheduled cron check** (every 2 minutes):
|
|
- Continuous drift monitoring
|
|
- JSON status sent to gitops-status-server
|
|
- Pipeline step: `gitops_sync_check`
|
|
|
|
## Error Handling
|
|
|
|
If the HTTP POST to gitops-status-server fails:
|
|
- The pipeline step will fail
|
|
- Error message will be logged
|
|
- The drift check itself is still performed
|
|
- No retry logic (next cron run will retry)
|
|
|
|
## Script Implementation
|
|
|
|
The `update-gitops-status.sh` script handles:
|
|
1. Running the Ansible drift-check playbook
|
|
2. Parsing the output to extract changed file names
|
|
3. Building the JSON payload
|
|
4. Sending it to gitops-status-server via HTTP POST
|
|
|
|
## Expected HTTP Response
|
|
|
|
gitops-status-server should respond with:
|
|
- `200 OK` or `201 Created` on success
|
|
- `4xx` or `5xx` on error
|
|
|
|
The rsyslog pipeline treats any 2xx response as success.
|
|
|
|
## Grafana Visualization
|
|
|
|
Grafana uses the Infinity datasource plugin to fetch `/status.json` from gitops-status-server and display:
|
|
- Current sync status (SYNCED vs OUT_OF_SYNC)
|
|
- Number of drifted files
|
|
- List of specific files that have drifted
|
|
- Last check timestamp
|
|
|
|
This provides much richer information than a simple numeric metric.
|