observability-stack/charts/gitops-status-server

GitOps Status Server Helm Chart

A dual-container HTTP server that receives GitOps status updates via POST API and serves status information as JSON for monitoring and observability purposes.

Overview

This chart deploys a two-container pod:

1. Nginx - Serves /status.json endpoint for monitoring tools and handles API routing
2. Flask API - Processes POST requests to /api/status and updates the status JSON

It's designed to be consumed by Grafana's Infinity datasource or other monitoring tools, and to receive updates from CI/CD pipelines like Woodpecker.

Architecture

CI/CD Pipeline (Woodpecker)
    ↓
    POST /api/status
    ↓
Kubernetes Service (port 80)
    ↓
Nginx (port 8080)
    ├─→ /api/status → Proxies to Flask (localhost:5000)
    └─→ /status.json → Serves static file
    ↓
Shared Volume (emptyDir)
    ├─→ status.json (updated by Flask API)
    └─→ Read by Nginx
    ↓
Grafana Infinity Datasource
    Reads /status.json

Features

• API-driven updates: POST endpoint for CI/CD pipelines to update status
• Read-only serving: Grafana-friendly JSON endpoint
• Minimal footprint: nginx-unprivileged + Python-Alpine with minimal resources
• Secure by default: Runs as non-root with restricted filesystems
• Internal only: ClusterIP service for cluster-internal access
• ArgoCD compatible: Init container auto-initializes status from ConfigMap
• Production-ready: Includes health checks, security contexts, and resource limits

Installation

Using Helm

# Install with default values
helm install gitops-status ./gitops-status-server

# Install with custom namespace
helm install gitops-status ./gitops-status-server -n observability-stack --create-namespace

# Install with custom values
helm install gitops-status ./gitops-status-server -f custom-values.yaml

Using ArgoCD

Create an Application manifest:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: gitops-status-server
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/observability-stack
    targetRevision: main
    path: gitops-status-server
    helm:
      values: |
        replicaCount: 1
        statusJson:
          repo: "rsyslog"
          server: "rsyslog-lab"
          sync_status: "UNKNOWN"

API Endpoints

GET /status.json

Returns the current status JSON

curl http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json

Response:

{
  "repo": "rsyslog",
  "server": "rsyslog-lab",
  "sync_status": "SYNCED",
  "drift_count": 0,
  "files": [],
  "last_check": "2026-04-21T10:30:00Z"
}

POST /api/status

Updates the status with new data

curl -X POST http://gitops-status-server.observability-stack.svc.cluster.local:80/api/status \
  -H "Content-Type: application/json" \
  -d '{
    "repo": "rsyslog",
    "server": "rsyslog-lab",
    "sync_status": "OUT_OF_SYNC",
    "drift_count": 2,
    "files": [
      {"name": "rsyslog.conf"},
      {"name": "rsyslog.d/30-lab.conf"}
    ],
    "last_check": "2026-04-21T10:30:00Z"
  }'

Response (HTTP 200):

{
  "success": true,
  "message": "Status updated successfully",
  "status": { ... }
}

GET /health

Health check endpoint (returns HTTP 200)

curl http://gitops-status-server.observability-stack.svc.cluster.local:80/health

GET /ready

Readiness check (verifies status file is readable)

curl http://gitops-status-server.observability-stack.svc.cluster.local:80/ready

Integration with Woodpecker

The rsyslog CI/CD pipeline can update status by POSTing to the /api/status endpoint:

#!/bin/bash

GITOPS_STATUS_SERVER_URL="http://gitops-status-server.observability-stack.svc.cluster.local:80"

STATUS_JSON='{
  "repo": "rsyslog",
  "server": "rsyslog-lab",
  "sync_status": "SYNCED",
  "drift_count": 0,
  "files": [],
  "last_check": "2026-04-21T10:30:00Z"
}'

curl -X POST "$GITOPS_STATUS_SERVER_URL/api/status" \
  -H "Content-Type: application/json" \
  -d "$STATUS_JSON"

Service Discovery

Internal Kubernetes URL

http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json

Port Forwarding (for local testing)

kubectl port-forward -n observability-stack svc/gitops-status-server 8080:80
# Then access at http://localhost:8080/status.json

NodePort (if service type is changed)

kubectl patch service -n observability-stack gitops-status-server -p '{"spec":{"type":"NodePort"}}'
# Then access at http://<node-ip>:<node-port>/status.json

Configuration

See values.yaml for all configuration options:

• replicaCount: Number of replicas
• image.repository: Container image
• image.tag: Image tag
• service.type: Service type (ClusterIP, NodePort, LoadBalancer)
• service.port: Service port (default 80)
• service.targetPort: Container port (default 8080)
• resources: CPU/memory limits and requests
• statusJson: Default status JSON values
• api.image.*: Python/Flask image configuration

Grafana Integration

Infinity Datasource Configuration

1. Install Infinity datasource plugin:

   grafana-cli plugins install yesoreyeram-infinity-datasource
   

2. Add datasource with URL:

   http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json
   

3. Create panels to visualize:

   • sync_status: Current synchronization state
   • drift_count: Number of drifted files
   • files[]: List of changed files
   • last_check: Timestamp of last check

Example Query

{
  "url": "http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json",
  "format": "json"
}

Security

• Runs as non-root user (UID 101)
• Read-only root filesystem (except for /tmp, /var/cache/nginx, /var/run)
• No privileged capabilities
• Network policies recommended for production
• Service Account with minimal RBAC

Troubleshooting

POST Request Returns 400 Error

Issue: "Invalid JSON" error

Solution: Verify JSON formatting with:

echo '{...}' | jq '.'

POST Updates Not Appearing in GET Response

Issue: Update endpoint returns 200 but status.json isn't updated

Possible causes:

• Shared volume permission issue
• API container crashed after POST
• Status file permissions

Debug:

# Check logs
kubectl logs -f deployment/gitops-status-server -c api
kubectl logs -f deployment/gitops-status-server -c nginx

# Check shared volume
kubectl exec deployment/gitops-status-server -c nginx -- ls -la /usr/share/nginx/html/

# Test API directly (port-forward to 5000 first)
kubectl port-forward deployment/gitops-status-server 5000:5000
curl -X POST http://localhost:5000/api/status -H "Content-Type: application/json" -d '{...}'

Connection Refused to gitops-status-server

Issue: Woodpecker can't reach the service

Possible causes:

• Service in different namespace
• Network policies blocking traffic
• Woodpecker outside cluster
• Service DNS name incorrect

Solutions:

• Verify service exists: kubectl get svc gitops-status-server -n observability-stack
• Use NodePort for external access (update service type in values)
• Use port-forward as a temporary solution
• Verify network policies allow traffic

Performance

• CPU: 150m limit (100m nginx + 100m API)
• Memory: 192Mi limit (64Mi nginx + 128Mi API)
• Startup time: ~5 seconds (Flask app install + startup)
• Update latency: <100ms (direct file write)
• Read performance: <10ms (static file serving)

License

Same as observability-stack repository statusJson: repo: "my-repo" server: "my-server" sync_status: "SYNCED" drift_count: 0 files: [] last_check: "2026-04-21T10:00:00Z" destination: server: https://kubernetes.default.svc namespace: monitoring syncPolicy: automated: prune: true selfHeal: true

## Configuration

### Key Values

| Parameter | Description | Default |
|-----------|-------------|---------|
| `replicaCount` | Number of replicas | `1` |
| `image.repository` | Container image repository | `nginxinc/nginx-unprivileged` |
| `image.tag` | Container image tag | `1.25-alpine` |
| `service.type` | Kubernetes service type | `ClusterIP` |
| `service.port` | Service port | `80` |
| `service.targetPort` | Container target port | `8080` |
| `resources.limits.cpu` | CPU limit | `100m` |
| `resources.limits.memory` | Memory limit | `64Mi` |
| `statusJson` | JSON content to serve | See values.yaml |

### Custom Status JSON

Override the status JSON content in your values:

```yaml
statusJson:
  repo: "production-apps"
  server: "prod-cluster-01"
  sync_status: "SYNCED"
  drift_count: 2
  files:
    - "deployment.yaml"
    - "service.yaml"
  last_check: "2026-04-21T12:30:00Z"

Usage

Access the Status Endpoint

From inside the cluster:

# Using the service DNS name
curl http://gitops-status-server/status.json

# With namespace
curl http://gitops-status-server.monitoring.svc.cluster.local/status.json

Grafana Infinity Datasource Configuration

1. Add an Infinity datasource in Grafana
2. Configure URL: http://gitops-status-server.monitoring.svc.cluster.local/status.json
3. Parser: JSON
4. Use fields from the JSON response in your dashboard

Example query fields:

• sync_status - Current sync status
• drift_count - Number of drifted resources
• files - List of changed files
• last_check - Timestamp of last check

Updating Status Data

Manual Update

Edit the ConfigMap directly:

kubectl edit configmap gitops-status-server -n monitoring

The deployment will automatically roll out with the new content due to the ConfigMap checksum annotation.

Automated Update via Pipeline

Use kubectl in your CI/CD pipeline:

kubectl create configmap gitops-status-server \
  --from-file=status.json=./status.json \
  --dry-run=client -o yaml | kubectl apply -f -

ArgoCD Hook (Advanced)

Create a PostSync hook that updates the ConfigMap with current sync status:

apiVersion: batch/v1
kind: Job
metadata:
  name: update-status
  annotations:
    argocd.argoproj.io/hook: PostSync
spec:
  template:
    spec:
      containers:
      - name: update
        image: bitnami/kubectl
        command:
        - /bin/sh
        - -c
        - |
          # Update status.json with current sync status
          kubectl patch configmap gitops-status-server \
            --patch '{"data":{"status.json":"..."}}'
      restartPolicy: Never

Security Considerations

• Runs as non-root user (UID 101)
• Read-only root filesystem
• No privilege escalation
• Minimal capabilities (all dropped)
• No external network access required
• ClusterIP only (no external exposure)

Resource Requirements

Minimal resource footprint suitable for small clusters:

• CPU: 50m request / 100m limit
• Memory: 32Mi request / 64Mi limit

Troubleshooting

Check pod status

kubectl get pods -l app.kubernetes.io/name=gitops-status-server

View logs

kubectl logs -l app.kubernetes.io/name=gitops-status-server

Test endpoint

kubectl run -it --rm curl --image=curlimages/curl --restart=Never -- \
  curl http://gitops-status-server/status.json

Common Issues

Pod not starting: Check security context compatibility with your cluster's PSP/PSA policies.

Empty response: Verify the ConfigMap is mounted correctly:

kubectl describe pod -l app.kubernetes.io/name=gitops-status-server

Service not accessible: Ensure you're accessing from within the cluster and using the correct namespace.

License

This chart is part of the observability-stack project.

Maintainers

• DevOps Team