Add simple api to the chart so woodpecker can update it

2026-04-21 12:57:15 +03:00 · 2026-04-21 12:57:15 +03:00 · 9af9b5b8c9
commit 9af9b5b8c9
parent 03830e34cb
7 changed files with 634 additions and 42 deletions
--- a/charts/gitops-status-server/README.md
+++ b/charts/gitops-status-server/README.md
@ -1,18 +1,44 @@
 # GitOps Status Server Helm Chart

-A minimal HTTP server that serves GitOps status information as JSON for monitoring and observability purposes.
+A dual-container HTTP server that receives GitOps status updates via POST API and serves status information as JSON for monitoring and observability purposes.

 ## Overview

-This chart deploys a lightweight nginx-based server that exposes a single endpoint (`/status.json`) containing GitOps synchronization status, drift information, and changed files. It's designed to be consumed by Grafana's Infinity datasource or other monitoring tools.
+This chart deploys a two-container pod:
+1. **Nginx** - Serves `/status.json` endpoint for monitoring tools and handles API routing
+2. **Flask API** - Processes POST requests to `/api/status` and updates the status JSON
+
+It's designed to be consumed by Grafana's Infinity datasource or other monitoring tools, and to receive updates from CI/CD pipelines like Woodpecker.
+
+## Architecture
+
+```
+CI/CD Pipeline (Woodpecker)
+    ↓
+    POST /api/status
+    ↓
+Kubernetes Service (port 80)
+    ↓
+Nginx (port 8080)
+    ├─→ /api/status → Proxies to Flask (localhost:5000)
+    └─→ /status.json → Serves static file
+    ↓
+Shared Volume (emptyDir)
+    ├─→ status.json (updated by Flask API)
+    └─→ Read by Nginx
+    ↓
+Grafana Infinity Datasource
+    Reads /status.json
+```

 ## Features

- **Minimal footprint**: Uses nginx-unprivileged with minimal resource requirements
- **Secure by default**: Runs as non-root with read-only root filesystem
+- **API-driven updates**: POST endpoint for CI/CD pipelines to update status
+- **Read-only serving**: Grafana-friendly JSON endpoint
+- **Minimal footprint**: nginx-unprivileged + Python-Alpine with minimal resources
+- **Secure by default**: Runs as non-root with restricted filesystems
 - **Internal only**: ClusterIP service for cluster-internal access
- **ConfigMap-based**: JSON content stored in ConfigMap for easy updates
- **ArgoCD compatible**: Automatically rolls deployment when ConfigMap changes
+- **ArgoCD compatible**: Init container auto-initializes status from ConfigMap
 - **Production-ready**: Includes health checks, security contexts, and resource limits

 ## Installation
@ -21,13 +47,13 @@ This chart deploys a lightweight nginx-based server that exposes a single endpoi

 ```bash
 # Install with default values
-helm install gitops-status ./charts/gitops-status-server
+helm install gitops-status ./gitops-status-server

 # Install with custom namespace
-helm install gitops-status ./charts/gitops-status-server -n monitoring --create-namespace
+helm install gitops-status ./gitops-status-server -n observability-stack --create-namespace

 # Install with custom values
-helm install gitops-status ./charts/gitops-status-server -f custom-values.yaml
+helm install gitops-status ./gitops-status-server -f custom-values.yaml
 ```

 ### Using ArgoCD
@ -45,9 +71,233 @@ spec:
  source:
    repoURL: https://github.com/your-org/observability-stack
    targetRevision: main
-    path: charts/gitops-status-server
+    path: gitops-status-server
    helm:
      values: |
+        replicaCount: 1
+        statusJson:
+          repo: "rsyslog"
+          server: "rsyslog-lab"
+          sync_status: "UNKNOWN"
+```
+
+## API Endpoints
+
+### GET /status.json
+Returns the current status JSON
+
+```bash
+curl http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json
+```
+
+Response:
+```json
+{
+  "repo": "rsyslog",
+  "server": "rsyslog-lab",
+  "sync_status": "SYNCED",
+  "drift_count": 0,
+  "files": [],
+  "last_check": "2026-04-21T10:30:00Z"
+}
+```
+
+### POST /api/status
+Updates the status with new data
+
+```bash
+curl -X POST http://gitops-status-server.observability-stack.svc.cluster.local:80/api/status \
+  -H "Content-Type: application/json" \
+  -d '{
+    "repo": "rsyslog",
+    "server": "rsyslog-lab",
+    "sync_status": "OUT_OF_SYNC",
+    "drift_count": 2,
+    "files": [
+      {"name": "rsyslog.conf"},
+      {"name": "rsyslog.d/30-lab.conf"}
+    ],
+    "last_check": "2026-04-21T10:30:00Z"
+  }'
+```
+
+Response (HTTP 200):
+```json
+{
+  "success": true,
+  "message": "Status updated successfully",
+  "status": { ... }
+}
+```
+
+### GET /health
+Health check endpoint (returns HTTP 200)
+
+```bash
+curl http://gitops-status-server.observability-stack.svc.cluster.local:80/health
+```
+
+### GET /ready
+Readiness check (verifies status file is readable)
+
+```bash
+curl http://gitops-status-server.observability-stack.svc.cluster.local:80/ready
+```
+
+## Integration with Woodpecker
+
+The rsyslog CI/CD pipeline can update status by POSTing to the `/api/status` endpoint:
+
+```bash
+#!/bin/bash
+
+GITOPS_STATUS_SERVER_URL="http://gitops-status-server.observability-stack.svc.cluster.local:80"
+
+STATUS_JSON='{
+  "repo": "rsyslog",
+  "server": "rsyslog-lab",
+  "sync_status": "SYNCED",
+  "drift_count": 0,
+  "files": [],
+  "last_check": "2026-04-21T10:30:00Z"
+}'
+
+curl -X POST "$GITOPS_STATUS_SERVER_URL/api/status" \
+  -H "Content-Type: application/json" \
+  -d "$STATUS_JSON"
+```
+
+## Service Discovery
+
+### Internal Kubernetes URL
+```
+http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json
+```
+
+### Port Forwarding (for local testing)
+```bash
+kubectl port-forward -n observability-stack svc/gitops-status-server 8080:80
+# Then access at http://localhost:8080/status.json
+```
+
+### NodePort (if service type is changed)
+```bash
+kubectl patch service -n observability-stack gitops-status-server -p '{"spec":{"type":"NodePort"}}'
+# Then access at http://<node-ip>:<node-port>/status.json
+```
+
+## Configuration
+
+See `values.yaml` for all configuration options:
+
+- `replicaCount`: Number of replicas
+- `image.repository`: Container image
+- `image.tag`: Image tag
+- `service.type`: Service type (ClusterIP, NodePort, LoadBalancer)
+- `service.port`: Service port (default 80)
+- `service.targetPort`: Container port (default 8080)
+- `resources`: CPU/memory limits and requests
+- `statusJson`: Default status JSON values
+- `api.image.*`: Python/Flask image configuration
+
+## Grafana Integration
+
+### Infinity Datasource Configuration
+
+1. Install Infinity datasource plugin:
+   ```bash
+   grafana-cli plugins install yesoreyeram-infinity-datasource
+   ```
+
+2. Add datasource with URL:
+   ```
+   http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json
+   ```
+
+3. Create panels to visualize:
+   - `sync_status`: Current synchronization state
+   - `drift_count`: Number of drifted files
+   - `files[]`: List of changed files
+   - `last_check`: Timestamp of last check
+
+### Example Query
+
+```json
+{
+  "url": "http://gitops-status-server.observability-stack.svc.cluster.local:80/status.json",
+  "format": "json"
+}
+```
+
+## Security
+
+- Runs as non-root user (UID 101)
+- Read-only root filesystem (except for /tmp, /var/cache/nginx, /var/run)
+- No privileged capabilities
+- Network policies recommended for production
+- Service Account with minimal RBAC
+
+## Troubleshooting
+
+### POST Request Returns 400 Error
+
+**Issue**: "Invalid JSON" error
+
+**Solution**: Verify JSON formatting with:
+```bash
+echo '{...}' | jq '.'
+```
+
+### POST Updates Not Appearing in GET Response
+
+**Issue**: Update endpoint returns 200 but status.json isn't updated
+
+**Possible causes**:
+- Shared volume permission issue
+- API container crashed after POST
+- Status file permissions
+
+**Debug**:
+```bash
+# Check logs
+kubectl logs -f deployment/gitops-status-server -c api
+kubectl logs -f deployment/gitops-status-server -c nginx
+
+# Check shared volume
+kubectl exec deployment/gitops-status-server -c nginx -- ls -la /usr/share/nginx/html/
+
+# Test API directly (port-forward to 5000 first)
+kubectl port-forward deployment/gitops-status-server 5000:5000
+curl -X POST http://localhost:5000/api/status -H "Content-Type: application/json" -d '{...}'
+```
+
+### Connection Refused to gitops-status-server
+
+**Issue**: Woodpecker can't reach the service
+
+**Possible causes**:
+- Service in different namespace
+- Network policies blocking traffic
+- Woodpecker outside cluster
+- Service DNS name incorrect
+
+**Solutions**:
+- Verify service exists: `kubectl get svc gitops-status-server -n observability-stack`
+- Use NodePort for external access (update service type in values)
+- Use port-forward as a temporary solution
+- Verify network policies allow traffic
+
+## Performance
+
+- **CPU**: 150m limit (100m nginx + 100m API)
+- **Memory**: 192Mi limit (64Mi nginx + 128Mi API)
+- **Startup time**: ~5 seconds (Flask app install + startup)
+- **Update latency**: <100ms (direct file write)
+- **Read performance**: <10ms (static file serving)
+
+## License
+
+Same as observability-stack repository
        statusJson:
          repo: "my-repo"
          server: "my-server"
--- a/charts/gitops-status-server/templates/api-app.yaml
+++ b/charts/gitops-status-server/templates/api-app.yaml
@ -0,0 +1,142 @@
+{{/*
+ConfigMap containing the API backend Python script
+Handles POST requests to /api/status and updates the status.json file
+*/}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "gitops-status-server.fullname" . }}-api
+  labels:
+    {{- include "gitops-status-server.labels" . | nindent 4 }}
+data:
+  app.py: |
+    #!/usr/bin/env python3
+    """
+    Simple Flask API for updating status.json
+    Listens on port 5000 and handles POST requests to /api/status
+    """
+    import os
+    import json
+    import logging
+    from flask import Flask, request, jsonify
+    from datetime import datetime
+
+    app = Flask(__name__)
+    
+    # Configuration
+    STATUS_FILE = '/usr/share/nginx/html/status.json'
+    API_PORT = int(os.environ.get('API_PORT', 5000))
+    API_HOST = os.environ.get('API_HOST', '127.0.0.1')
+    
+    # Setup logging
+    logging.basicConfig(
+        level=logging.INFO,
+        format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+    )
+    logger = logging.getLogger(__name__)
+
+    def load_status():
+        """Load the current status from file"""
+        try:
+            if os.path.exists(STATUS_FILE):
+                with open(STATUS_FILE, 'r') as f:
+                    return json.load(f)
+            else:
+                # Default status if file doesn't exist
+                return {
+                    "repo": "unknown",
+                    "server": "unknown",
+                    "sync_status": "UNKNOWN",
+                    "drift_count": 0,
+                    "files": [],
+                    "last_check": ""
+                }
+        except Exception as e:
+            logger.error(f"Error loading status: {e}")
+            return {}
+
+    def save_status(status):
+        """Save the status to file"""
+        try:
+            # Ensure directory exists (should already exist from mount)
+            os.makedirs(os.path.dirname(STATUS_FILE), exist_ok=True)
+            
+            # Write with proper formatting
+            with open(STATUS_FILE, 'w') as f:
+                json.dump(status, f, indent=2)
+            
+            logger.info(f"Status saved successfully: {status['repo']}/{status['server']} -> {status['sync_status']}")
+            return True
+        except Exception as e:
+            logger.error(f"Error saving status: {e}")
+            return False
+
+    @app.route('/api/status', methods=['GET', 'POST', 'OPTIONS'])
+    def api_status():
+        """
+        GET: Retrieve current status
+        POST: Update status with new data
+        """
+        if request.method == 'OPTIONS':
+            return '', 204
+        
+        if request.method == 'GET':
+            status = load_status()
+            return jsonify(status), 200
+
+        if request.method == 'POST':
+            try:
+                # Parse incoming JSON
+                incoming_data = request.get_json()
+                if not incoming_data:
+                    return jsonify({"error": "No JSON data provided"}), 400
+                
+                # Load current status
+                status = load_status()
+                
+                # Update with incoming data (merge)
+                status.update(incoming_data)
+                
+                # Ensure required fields exist
+                if 'last_check' not in status or not status['last_check']:
+                    status['last_check'] = datetime.utcnow().isoformat() + 'Z'
+                
+                # Save updated status
+                if save_status(status):
+                    return jsonify({
+                        "success": True,
+                        "message": "Status updated successfully",
+                        "status": status
+                    }), 200
+                else:
+                    return jsonify({
+                        "error": "Failed to save status"
+                    }), 500
+            
+            except json.JSONDecodeError:
+                return jsonify({"error": "Invalid JSON"}), 400
+            except Exception as e:
+                logger.error(f"Error processing POST request: {e}")
+                return jsonify({"error": str(e)}), 500
+
+    @app.route('/health', methods=['GET'])
+    def health():
+        """Health check endpoint"""
+        return jsonify({"status": "healthy"}), 200
+
+    @app.route('/ready', methods=['GET'])
+    def ready():
+        """Readiness check - verify status file is accessible"""
+        try:
+            status = load_status()
+            if status:
+                return jsonify({"status": "ready"}), 200
+            else:
+                return jsonify({"status": "not_ready", "reason": "status file empty"}), 503
+        except Exception as e:
+            return jsonify({"status": "not_ready", "error": str(e)}), 503
+
+    if __name__ == '__main__':
+        logger.info(f"Starting gitops-status-server API on {API_HOST}:{API_PORT}")
+        logger.info(f"Status file: {STATUS_FILE}")
+        app.run(host=API_HOST, port=API_PORT, debug=False)
--- a/charts/gitops-status-server/templates/configmap.yaml
+++ b/charts/gitops-status-server/templates/configmap.yaml
@ -1,6 +1,6 @@
 {{/*
-ConfigMap containing the status.json file
-This file will be mounted into the nginx container
+ConfigMap for default status.json values
+Used by init container to set up initial status if file doesn't exist
 */}}
 apiVersion: v1
 kind: ConfigMap
@ -13,7 +13,10 @@ metadata:
    {{- toYaml . | nindent 4 }}
  {{- end }}
 data:
-  # The status.json file that will be served by nginx
-  # This can be updated by your GitOps pipeline or ArgoCD hooks
+  # Default status.json values (used for initialization)
+  # This is not mounted directly; instead it's used by the init container
+  # to set up the initial status.json in the shared emptyDir volume.
+  # The actual status.json is stored on the emptyDir and updated via the API.
  status.json: |
    {{- .Values.statusJson | toJson | nindent 4 }}
+
--- a/charts/gitops-status-server/templates/deployment.yaml
+++ b/charts/gitops-status-server/templates/deployment.yaml
@ -35,7 +35,75 @@ spec:
      serviceAccountName: {{ include "gitops-status-server.serviceAccountName" . }}
      securityContext:
        {{- toYaml .Values.podSecurityContext | nindent 8 }}
+      
+      # Init container to set up initial status.json from ConfigMap
+      initContainers:
+      - name: init-status
+        image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
+        command:
+        - sh
+        - -c
+        - |
+          if [ ! -f /usr/share/nginx/html/status.json ]; then
+            cat > /usr/share/nginx/html/status.json <<'EOF'
+          {{- .Values.statusJson | toJson | nindent 10 }}
+          EOF
+          fi
+        volumeMounts:
+        - name: shared-data
+          mountPath: /usr/share/nginx/html
+      
      containers:
+      - name: api
+        image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag }}"
+        imagePullPolicy: {{ .Values.api.image.pullPolicy }}
+        command:
+        - sh
+        - -c
+        - |
+          pip install --no-cache-dir Flask==2.3.2 >/dev/null 2>&1
+          exec python3 /app/app.py
+        ports:
+        - name: api
+          containerPort: 5000
+          protocol: TCP
+        env:
+        - name: API_HOST
+          value: "127.0.0.1"
+        - name: API_PORT
+          value: "5000"
+        - name: FLASK_ENV
+          value: "production"
+        livenessProbe:
+          httpGet:
+            path: /health
+            port: api
+          initialDelaySeconds: 10
+          periodSeconds: 10
+          timeoutSeconds: 3
+          failureThreshold: 3
+        readinessProbe:
+          httpGet:
+            path: /ready
+            port: api
+          initialDelaySeconds: 5
+          periodSeconds: 5
+          timeoutSeconds: 2
+          failureThreshold: 2
+        resources:
+          limits:
+            cpu: 100m
+            memory: 128Mi
+          requests:
+            cpu: 50m
+            memory: 64Mi
+        volumeMounts:
+        - name: shared-data
+          mountPath: /usr/share/nginx/html
+        - name: api-code
+          mountPath: /app
+          readOnly: true
+      
      - name: nginx
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
        imagePullPolicy: {{ .Values.image.pullPolicy }}
@ -65,12 +133,14 @@ spec:
        resources:
          {{- toYaml .Values.resources | nindent 10 }}
        volumeMounts:
-        # Mount the status.json file from ConfigMap
-        # We mount it as a subPath to avoid overwriting the entire directory
-        - name: status-json
-          mountPath: /usr/share/nginx/html/status.json
-          subPath: status.json
+        # Mount the nginx config
+        - name: nginx-config
+          mountPath: /etc/nginx/nginx.conf
+          subPath: nginx.conf
          readOnly: true
+        # Mount the shared data directory (status.json is writable here)
+        - name: shared-data
+          mountPath: /usr/share/nginx/html
        # nginx-unprivileged needs writable directories for cache and run
        - name: cache
          mountPath: /var/cache/nginx
@ -80,13 +150,25 @@ spec:
        - name: tmp
          mountPath: /tmp
      volumes:
-      # ConfigMap volume containing the status.json
-      - name: status-json
+      # ConfigMap volume containing the nginx configuration
+      - name: nginx-config
        configMap:
-          name: {{ include "gitops-status-server.fullname" . }}
+          name: {{ include "gitops-status-server.fullname" . }}-nginx-config
          items:
-          - key: status.json
-            path: status.json
+          - key: nginx.conf
+            path: nginx.conf
+      # ConfigMap volume containing the API application code
+      - name: api-code
+        configMap:
+          name: {{ include "gitops-status-server.fullname" . }}-api
+          defaultMode: 0755
+          items:
+          - key: app.py
+            path: app.py
+      # Shared data volume for status.json (writable emptyDir)
+      - name: shared-data
+        emptyDir:
+          sizeLimit: 1Mi
      # Empty directories for nginx runtime
      - name: cache
        emptyDir: {}
--- a/charts/gitops-status-server/templates/nginx-config.yaml
+++ b/charts/gitops-status-server/templates/nginx-config.yaml
@ -0,0 +1,96 @@
+{{/*
+ConfigMap containing the nginx configuration
+Enables serving status.json via GET and updating via POST requests
+*/}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "gitops-status-server.fullname" . }}-nginx-config
+  labels:
+    {{- include "gitops-status-server.labels" . | nindent 4 }}
+data:
+  nginx.conf: |
+    # Minimal nginx config for serving and updating status.json
+    user nginx;
+    worker_processes auto;
+    error_log /var/log/nginx/error.log warn;
+    pid /var/run/nginx.pid;
+
+    events {
+      worker_connections 1024;
+    }
+
+    http {
+      include /etc/nginx/mime.types;
+      default_type application/octet-stream;
+
+      log_format main '$remote_addr - $remote_user [$time_local] "$request" '
+                      '$status $body_bytes_sent "$http_referer" '
+                      '"$http_user_agent" "$http_x_forwarded_for"';
+
+      access_log /var/log/nginx/access.log main;
+
+      sendfile on;
+      tcp_nopush on;
+      tcp_nodelay on;
+      keepalive_timeout 65;
+      types_hash_max_size 2048;
+      client_max_body_size 1M;
+
+      # Gzip compression
+      gzip on;
+      gzip_vary on;
+      gzip_types text/plain text/css text/xml text/javascript 
+                 application/x-javascript application/xml+rss 
+                 application/json;
+
+      upstream api_backend {
+        server 127.0.0.1:5000;
+        keepalive 32;
+      }
+
+      server {
+        listen 8080 default_server;
+        server_name _;
+
+        # Serve status.json as read-only
+        location /status.json {
+          alias /usr/share/nginx/html/status.json;
+          add_header Cache-Control "no-cache, no-store, must-revalidate";
+          add_header Pragma "no-cache";
+          add_header Expires "0";
+        }
+
+        # Health check endpoint
+        location /health {
+          access_log off;
+          return 200 "healthy\n";
+          add_header Content-Type text/plain;
+        }
+
+        # Proxy POST requests to the API backend (Python Flask)
+        location /api/ {
+          proxy_pass http://api_backend;
+          proxy_http_version 1.1;
+          proxy_set_header Connection "";
+          proxy_set_header Host $host;
+          proxy_set_header X-Real-IP $remote_addr;
+          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_set_header X-Forwarded-Proto $scheme;
+          
+          # Buffer settings for POST requests
+          proxy_request_buffering off;
+          proxy_buffering off;
+          
+          # Timeouts
+          proxy_connect_timeout 30s;
+          proxy_send_timeout 30s;
+          proxy_read_timeout 30s;
+        }
+
+        # Catch-all for root
+        location / {
+          return 301 /status.json;
+        }
+      }
+    }
--- a/charts/gitops-status-server/values.yaml
+++ b/charts/gitops-status-server/values.yaml
@ -13,6 +13,16 @@ image:
  # Overrides the image tag whose default is the chart appVersion
  tag: "1.25-alpine"

+# API backend container configuration (handles POST requests)
+api:
+  image:
+    # Python Flask API for handling status updates
+    repository: python
+    pullPolicy: IfNotPresent
+    tag: "3.11-alpine"
+  # Pre-install Flask via pip before running the app
+  pip_packages: "Flask==2.3.2"
+
 # Image pull secrets for private registries
 imagePullSecrets: []

--- a/manifests/gitops-status-server/values.yaml
+++ b/manifests/gitops-status-server/values.yaml
@ -1,8 +1,31 @@
-# Minimal values for gitops-status-server
-# Override default chart values as needed
+# Values for gitops-status-server Helm chart
+# Serves a static status.json file via nginx with an optional API for dynamic updates

-# Status JSON content
-# Update this with your actual GitOps status information
+# Number of replicas
+replicaCount: 1
+
+# Container image configuration
+image:
+  repository: nginxinc/nginx-unprivileged
+  tag: "1.25-alpine"
+  pullPolicy: IfNotPresent
+
+# API backend configuration (Flask server for status updates)
+api:
+  image:
+    repository: python
+    tag: "3.11-alpine"
+    pullPolicy: IfNotPresent
+  pip_packages: "Flask==2.3.2"
+
+# Service configuration
+service:
+  type: ClusterIP
+  port: 80
+  targetPort: 8080
+  annotations: {}
+
+# Status JSON content - customize with your actual GitOps status information
 statusJson:
  repo: "observability-stack"
  server: "rsyslog-lab"
@ -10,17 +33,3 @@ statusJson:
  drift_count: 0
  files: []
  last_check: ""
-
-# Resource limits (optional override)
-# resources:
-#   limits:
-#     cpu: 100m
-#     memory: 64Mi
-#   requests:
-#     cpu: 50m
-#     memory: 32Mi
-
-# Service configuration (optional override)
-# service:
-#   type: ClusterIP
-#   port: 80