open-meteo-service/README.md

# Open-Meteo Coordinates Service

A FastAPI-based microservice that queries the Open-Meteo Geocoding API to retrieve coordinates for various cities. The service includes caching, Prometheus metrics, and Grafana dashboards for monitoring.

## Features

- **RESTful API** for retrieving city coordinates
- **Intelligent caching** to reduce external API calls
- **Prometheus metrics** for observability
- **Pre-configured Grafana dashboards** with 5 panels
- **Docker Compose** setup for easy deployment

## Prerequisites

- Docker
- Docker Compose

## Quick Start

1. **Clone the repository**
   ```bash
   cd open-meteo-service
   ```

2. **Start all services**
   ```bash
   docker compose up --build
   ```

3. **Access the services**
   - API: http://localhost:8000/docs
   - Prometheus: http://localhost:9090
   - Grafana: http://localhost:3000 (admin/admin)

## Helm (Kubernetes)

The Helm chart in the `helm` folder deploys the FastAPI app and, by default, Prometheus and Grafana.

### What it deploys
- **App**: FastAPI service with `/healthz` probes and Prometheus scrape annotations
- **Prometheus**: Scrapes the app metrics at `/metrics`
- **Grafana**: Pre-provisioned datasource and dashboard for the app metrics

### Prerequisites
- Kubernetes cluster
- Helm v3

### Install
```bash
helm install open-meteo-service ./helm
```

### Upgrade
```bash
helm upgrade open-meteo-service ./helm
```

### Access the services
```bash
kubectl port-forward svc/open-meteo-service-open-meteo-service 8080:8000
kubectl port-forward svc/open-meteo-service-open-meteo-service-prometheus 9090:9090
kubectl port-forward svc/open-meteo-service-open-meteo-service-grafana 3000:3000
```

### Values overview
You can customize the chart via `helm/values.yaml` or `--set` flags.

**App values**
- `image.repository`, `image.tag`, `image.pullPolicy`
- `service.type`, `service.port`
- `env.cacheFile`
- `persistence.enabled`, `persistence.size`, `persistence.storageClassName`

**Prometheus values**
- `prometheus.enabled`
- `prometheus.image`
- `prometheus.service.type`, `prometheus.service.port`
- `prometheus.persistence.enabled`, `prometheus.persistence.size`, `prometheus.persistence.storageClassName`

**Grafana values**
- `grafana.enabled`
- `grafana.image`
- `grafana.service.type`, `grafana.service.port`
- `grafana.adminUser`, `grafana.adminPassword`
- `grafana.persistence.enabled`, `grafana.persistence.size`, `grafana.persistence.storageClassName`

### Example: disable Grafana and Prometheus
```bash
helm install open-meteo-service ./helm \
  --set grafana.enabled=false \
  --set prometheus.enabled=false
```

### Example: enable persistence for all components
```bash
helm install open-meteo-service ./helm \
  --set persistence.enabled=true \
  --set prometheus.persistence.enabled=true \
  --set grafana.persistence.enabled=true
```

## CI Pipeline

This project includes a GitLab CI/CD pipeline that automatically builds, deploys, and tests the application without requiring Docker-in-Docker.

### Pipeline Stages

1. **Build Stage (Kaniko)**
   - Builds Docker image using Kaniko (secure, daemonless container builder)
   - Pushes to GitLab Container Registry with commit-based tags
   - Tags `latest` for main branch builds
   - Uses layer caching for faster builds

2. **Deploy Stage (kind + Helm)**
   - Creates a local Kubernetes cluster using kind (Kubernetes in Docker)
   - Installs kubectl, kind, and Helm
   - Configures image pull secrets for GitLab registry access
   - Deploys app using Helm with the newly built image
   - Waits for deployment readiness with automatic rollback on failure

3. **Test Stage (Health Checks)**
   - Validates deployment with port-forwarding
   - Tests `/healthz` endpoint for service health
   - Tests `/coordinates` endpoint for data retrieval
   - Tests `/help` endpoint for API documentation
   - Provides detailed debugging info if tests fail
   - Cleans up kind cluster after tests

### Image Tagging Strategy
- **Commit builds**: `$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA`
- **Main branch**: Also tagged as `latest`

### Viewing Pipeline Status
1. Go to your GitLab project
2. Navigate to **CI/CD > Pipelines**
3. Click on a pipeline run to see individual job logs
4. Each stage shows real-time logs and can be expanded for details

### Debugging Failed Pipelines
If a pipeline fails:
- Check the **build** job logs for Docker build errors
- Check the **deploy** job logs for Kubernetes/Helm issues
  - Pod status and events are automatically logged
  - Container logs are shown if deployment fails
- Check the **test** job logs for endpoint test failures
  - Response bodies are printed for debugging

### Running Locally
To test the pipeline behavior locally:
```bash
# Build with Kaniko (requires Docker)
docker run --rm -v $(pwd):/workspace gcr.io/kaniko-project/executor:latest \
  --dockerfile /workspace/Dockerfile \
  --context /workspace \
  --no-push

# Test with kind
kind create cluster
kubectl create secret docker-registry gitlab-registry \
  --docker-server=your-registry \
  --docker-username=your-user \
  --docker-password=your-token
helm install open-meteo-service ./open-meteo-service \
  --set imagePullSecrets[0].name=gitlab-registry
```

## API Documentation

### Endpoints

#### `GET /coordinates`
Retrieve coordinates for all configured cities.

**Response:**
```json
{
  "source": "cache",
  "data": {
    "Tel Aviv": {
      "name": "Tel Aviv",
      "latitude": 32.08088,
      "longitude": 34.78057,
      "country": "Israel"
    },
    ...
  }
}
```

#### `GET /coordinates/{city}`
Retrieve coordinates for a specific city.

**Parameters:**
- `city` (path) - City name (e.g., "Ashkelon", "London")

**Example:**
```bash
curl http://localhost:8000/coordinates/Paris
```

**Response:**
```json
{
  "source": "open-meteo",
  "data": {
    "name": "Paris",
    "latitude": 48.85341,
    "longitude": 2.3488,
    "country": "France"
  }
}
```

#### `GET /metrics`
Prometheus metrics endpoint exposing service metrics.

**Example:**
```bash
curl http://localhost:8000/metrics
```

#### `GET /healthz`
Health check endpoint.

**Response:**
```json
{
  "status": "ok"
}
```

## Metrics

The service exposes the following Prometheus metrics:

### HTTP Metrics
- **`http_requests_total`** - Counter of total HTTP requests
  - Labels: `endpoint`, `method`, `status`
  
- **`http_request_duration_seconds`** - Histogram of request durations
  - Labels: `endpoint`, `method`

### Cache Metrics
- **`coordinates_cache_hits_total`** - Counter of cache hits
- **`coordinates_cache_misses_total`** - Counter of cache misses

### External API Metrics
- **`openmeteo_api_calls_total`** - Counter of calls to Open-Meteo Geocoding API
  - Labels: `city`

## Grafana Dashboard

The pre-configured dashboard includes 5 panels:

1. **Request Rate** - Requests per second by endpoint
2. **Request Duration p95** - 95th percentile latency
3. **Cache Hits vs Misses** - Cache effectiveness
4. **Open-Meteo Calls by City** - External API usage per city
5. **Requests by Status** - HTTP status code distribution

Access the dashboard at http://localhost:3000 after logging in with `admin/admin`.

## Caching

The service uses a local JSON file (`coordinates_cache.json`) to cache city coordinates:
- Reduces external API calls
- Shared across all API endpoints
- Persists between requests (not container restarts)
- Automatically updated when new cities are queried

## Development

### Project Structure
```
.
├── app/
│   ├── main.py          # FastAPI application
│   ├── service.py       # Business logic & caching
│   └── metrics.py       # Prometheus metrics definitions
├── grafana/
│   ├── provisioning/    # Auto-configured datasources & dashboards
│   └── dashboards/      # Dashboard JSON definitions
├── docker-compose.yml   # Service orchestration
├── Dockerfile           # Python app container
├── prometheus.yml       # Prometheus scrape configuration
└── requirements.txt     # Python dependencies
```

### Stop Services
```bash
docker compose down
```

### View Logs
```bash
docker compose logs -f open-meteo-service
```

### Rebuild After Code Changes
```bash
docker compose up --build
```

## Configuration

### Environment Variables
- `CACHE_FILE` - Path to cache file (default: `coordinates_cache.json`)

### Scrape Interval
Edit `prometheus.yml` to adjust the scrape interval (default: 15s).

## Testing

Generate test traffic to populate metrics:
```bash
# Test all endpoints
curl http://localhost:8000/coordinates
curl http://localhost:8000/coordinates/Paris
curl http://localhost:8000/coordinates/London

# Generate load
for i in {1..10}; do curl -s http://localhost:8000/coordinates > /dev/null; done
```

## License

MIT