Add offline Python wheels and air-gapped deployment solution

AIR_GAPPED_SOLUTION.md

@@ -0,0 +1,136 @@
+# Air-Gapped Cluster Build Solution
+
+## Problem
+Your Kubernetes cluster has **no external internet access**:
+- ❌ Cannot reach Docker Hub (`index.docker.io`)
+- ❌ Cannot reach Debian repositories (`deb.debian.org`)
+- ❌ Cannot reach PyPI (`pypi.org`)
+- ❌ TLS handshake failures
+- ❌ HTTP 530 errors
+
+## Solution
+Use Harbor as an internal registry for all base images.
+
+## Quick Fix (Implemented)
+
+### Backend Dockerfile Changes
+✅ **Removed `apt-get` commands** - Not needed since `psycopg2-binary` is pre-compiled
+✅ **Changed base image** to `harbor.dvirlabs.com/dockerhub/python:3.11-slim`
+
+### What You Need to Do NOW
+
+**1. Download Python Wheels (on a machine with internet)**
+
+```bash
+cd backend
+.\download-wheels.bat  # Windows
+# OR
+./download-wheels.sh   # Linux/Mac
+```
+
+This downloads all Python packages as wheel files to `backend/wheels/`.
+
+**2. Push Base Images to Harbor**
+
+Run the appropriate script for your OS:
+
+**Windows:**
+```bash
+.\push-base-images.bat
+```
+
+**Linux/Mac:**
+```bash
+chmod +x push-base-images.sh
+./push-base-images.sh
+```
+
+This pushes these images to `harbor.dvirlabs.com/dockerhub/`:
+- python:3.11-slim
+- node:18-alpine
+- nginx:alpine
+- postgres:16-alpine
+- alpCommit Wheels and Test the Build**
+
+```bash
+git add .
+git commit -m "fix: Bundle Python wheels for offline installation"
+git push
+```
+
+See [OFFLINE_PYPI_SOLUTION.md](OFFLINE_PYPI_SOLUTION.md) for details.e sure these projects exist in Harbor:
+- `dockerhub` - For public images from Docker Hub
+- `base-images` - For custom-built images (optional)
+- `my-apps` - For your application images
+
+**3. Test the Build**
+
+```bash
+git add .
+git commit -m "fix: Remove apt-get to avoid Debian repo access, use Harbor images"
+git push
+```
+
+## Advanced: Custom Base Image (Optional)
+
+If you need `gcc`, `postgresql-client`, or other build tools in the future:
+
+**1. Build Custom Image (on a machine with internet):**
+
+```bash
+# Windows
+.\build-custom-base-images.bat
+
+# Linux/Mac
+chmod +x build-custom-base-images.sh
+./build-custom-base-images.sh
+```
+
+**2. Update Dockerfile to use it:**
+
+```dockerfile
+FROM harbor.dvirlabs.com/base-images/python:3.11-slim-dev
+```
+
+## Project Structure
+
+```
+brand-master/
+├── backend/
+│   └── Dockerfile                    # ✅ Fixed - no apt-get needed
+├── frontend/  
+│   └── Dockerfile                    # ✅ Uses Harbor images
+├── base-images/
+│   ├── python-3.11-slim-dev/
+│   │   └── Dockerfile                # Custom image with build tools
+│   └── BUILD_INSTRUCTIONS.md
+├── push-base-images.sh               # Push Docker Hub images to Harbor
+├── push-base-images.bat
+├── build-custom-base-images.sh       # Build custom images with deps
+└── build-custom-base-images.bat
+```
+
+## Troubleshooting
+
+### Build still fails with "unable to reach..."
+- Verify images are in Harbor: `https://harbor.dvirlabs.com`
+- Check project permissions (public or pull secrets configured)
+- Verify Kubernetes nodes can resolve `harbor.dvirlabs.com`
+
+### Need to add more dependencies
+- Option 1: Use custom base image (see Advanced section)
+- Option 2: Pre-install in a custom image layer
+
+### Images not found in Harbor
+- Run `push-base-images.sh` or `.bat` script
+- Check Harbor UI to verify images uploaded
+- Ensure project names match exactly
+
+## Current Status
+
+✅ Backend Dockerfile - **FIXED** (no apt-get, uses Harbor)
+✅ Frontend Dockerfile - **FIXED** (uses Harbor)
+✅ Helm Chart - **FIXED** (uses Harbor)
+✅ CI Pipeline - **FIXED** (uses Harbor Kaniko)
+
+**Next Step:** Run `push-base-images.sh/.bat` and commit your changes!

OFFLINE_PYPI_SOLUTION.md

@@ -0,0 +1,111 @@
+# Offline PyPI Solution for Air-Gapped Cluster
+
+## Problem
+Your cluster blocks **ALL external internet access**:
+- ❌ Docker Hub (`index.docker.io`) - BLOCKED
+- ❌ Debian repositories (`deb.debian.org`) - BLOCKED  
+- ❌ **PyPI (`pypi.org`) - BLOCKED** ← NEW ISSUE
+
+## Solution: Bundle Python Packages
+
+Since `pip install` cannot reach PyPI, we pre-download all packages as wheel files and bundle them in the Docker image.
+
+## Quick Fix Steps
+
+### Step 1: Download Wheels (On a machine WITH internet)
+
+**Windows:**
+```bash
+cd backend
+.\download-wheels.bat
+```
+
+**Linux/Mac:**
+```bash
+cd backend
+chmod +x download-wheels.sh
+./download-wheels.sh
+```
+
+This downloads ~30-50 wheel files to `backend/wheels/` directory.
+
+### Step 2: Commit Wheels to Git
+
+```bash
+git add backend/wheels/
+git commit -m "Add offline Python wheels for air-gapped cluster"
+git push
+```
+
+### Step 3: Verify Dockerfile
+
+The Dockerfile now:
+1. ✅ Copies `wheels/` directory into image
+2. ✅ Installs using `pip install --no-index --find-links /tmp/wheels`
+3. ✅ No internet needed during build!
+
+### Step 4: Test Build
+
+Your CI pipeline will now build successfully without PyPI access.
+
+## How It Works
+
+### Before (FAILED):
+```dockerfile
+RUN pip install -r requirements.txt
+# ❌ Tries to reach pypi.org → SSL handshake failure
+```
+
+### After (SUCCESS):
+```dockerfile
+COPY wheels/ /tmp/wheels/
+RUN pip install --no-index --find-links /tmp/wheels -r requirements.txt
+# ✅ Installs from local files → No internet needed
+```
+
+## Updating Dependencies
+
+When you add new packages to `requirements.txt`:
+
+1. **Update wheels** (on internet-connected machine):
+   ```bash
+   cd backend
+   rm -rf wheels/  # Clear old wheels
+   ./download-wheels.sh  # Download new ones
+   ```
+
+2. **Commit and push**:
+   ```bash
+   git add wheels/ requirements.txt
+   git commit -m "Update dependencies and wheels"
+   git push
+   ```
+
+## Wheel Directory Size
+
+Expect `backend/wheels/` to be **~50-100MB** with all dependencies.
+
+**Tips:**
+- ✅ This is normal and necessary for air-gapped deployment
+- ✅ Git can handle it fine
+- ⚠️ If it grows >100MB, consider Git LFS
+
+## Alternative: PyPI Mirror in Harbor
+
+If Harbor supports PyPI proxy (check with your admin), you could:
+
+1. Configure Harbor as PyPI mirror
+2. Update pip to use it:
+   ```dockerfile
+   RUN pip install --index-url https://harbor.dvirlabs.com/api/pypi/simple -r requirements.txt
+   ```
+
+Ask your Harbor admin if PyPI proxy is available.
+
+## Current Status
+
+✅ Backend Dockerfile - **UPDATED** (installs from local wheels)  
+✅ Download scripts - **CREATED**  
+⏳ **ACTION NEEDED:** Run `download-wheels.bat` and commit wheels
+
+**Next:** Download wheels and push to Git!

backend/.gitignore

@@ -0,0 +1,30 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+env/
+venv/
+ENV/
+.venv
+
+# Environment
+.env
+.env.local
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# Uploads (images will be in PVC)
+uploads/
+
+# Database
+*.db
+*.sqlite
+
+# Note: We DO commit wheels/ directory for air-gapped deployment
+# If wheels/ gets too large, consider using Git LFS or external storage

backend/download-wheels.bat

@@ -0,0 +1,31 @@
+@echo off
+REM Download all Python packages as wheel files for offline installation
+REM Run this on a machine with internet access
+
+echo.
+echo 📦 Downloading Python wheels for offline installation...
+echo.
+
+REM Create wheels directory
+if not exist wheels mkdir wheels
+
+REM Download all dependencies including their dependencies
+pip download ^
+    --dest wheels ^
+    --platform manylinux2014_x86_64 ^
+    --only-binary=:all: ^
+    --python-version 3.11 ^
+    -r requirements.txt
+
+echo.
+echo ✅ Downloaded all wheels to ./wheels/
+echo.
+echo Files downloaded:
+dir wheels /b
+echo.
+echo Next steps:
+echo 1. Commit the wheels directory: git add wheels/ ^&^& git commit -m "Add offline Python wheels"
+echo 2. Push to Git: git push
+echo 3. The Dockerfile will install from local wheels (no PyPI access needed)
+echo.
+pause

backend/download-wheels.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+# Download all Python packages as wheel files for offline installation
+# Run this on a machine with internet access
+
+set -e
+
+echo "📦 Downloading Python wheels for offline installation..."
+echo ""
+
+# Create wheels directory
+mkdir -p wheels
+
+# Download all dependencies including their dependencies
+pip download \
+    --dest wheels \
+    --platform manylinux2014_x86_64 \
+    --only-binary=:all: \
+    --python-version 3.11 \
+    -r requirements.txt
+
+echo ""
+echo "✅ Downloaded all wheels to ./wheels/"
+echo ""
+echo "Files downloaded:"
+ls -lh wheels/
+echo ""
+echo "Next steps:"
+echo "1. Commit the wheels directory: git add wheels/ && git commit -m 'Add offline Python wheels'"
+echo "2. Push to Git: git push"
+echo "3. The Dockerfile will install from local wheels (no PyPI access needed)"

base-images/BUILD_INSTRUCTIONS.md

@@ -0,0 +1,10 @@
+# Build and push custom Python base image to Harbor
+# This image has all build dependencies pre-installed
+
+# 1. Build the image locally (from your machine with internet access)
+docker build -t harbor.dvirlabs.com/base-images/python:3.11-slim-dev ./base-images/python-3.11-slim-dev
+
+# 2. Push to Harbor
+docker push harbor.dvirlabs.com/base-images/python:3.11-slim-dev
+
+# Now your cluster can use this image without needing internet access

base-images/python-3.11-slim-dev/Dockerfile

@@ -0,0 +1,24 @@
+# Custom Python base image with build tools pre-installed
+# Build this once and push to Harbor to avoid apt-get during app builds
+
+FROM python:3.11-slim
+
+# Install system dependencies that might be needed for Python packages
+RUN apt-get update && apt-get install -y \
+    gcc \
+    g++ \
+    make \
+    postgresql-client \
+    libpq-dev \
+    curl \
+    git \
+    && rm -rf /var/lib/apt/lists/*
+
+# Upgrade pip
+RUN pip install --no-cache-dir --upgrade pip setuptools wheel
+
+# Set working directory
+WORKDIR /app
+
+# Default command
+CMD ["/bin/bash"]

build-custom-base-images.bat

@@ -0,0 +1,40 @@
+@echo off
+REM Build custom base images with dependencies pre-installed
+REM Run this on a machine with internet access, then push to Harbor
+
+set HARBOR_REGISTRY=harbor.dvirlabs.com
+set HARBOR_PROJECT=base-images
+
+echo.
+echo 🏗️  Building custom base images for air-gapped cluster...
+echo Registry: %HARBOR_REGISTRY%
+echo Project: %HARBOR_PROJECT%
+echo.
+
+REM Login to Harbor
+echo Please login to Harbor:
+docker login %HARBOR_REGISTRY%
+
+echo.
+echo 📦 Building Python base image with dev tools...
+echo.
+
+REM Build Python dev image
+cd base-images\python-3.11-slim-dev
+docker build -t %HARBOR_REGISTRY%/%HARBOR_PROJECT%/python:3.11-slim-dev .
+cd ..\..
+
+echo ✅ Built: python:3.11-slim-dev
+echo.
+
+REM Push to Harbor
+echo 🚀 Pushing to Harbor...
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/python:3.11-slim-dev
+
+echo.
+echo 🎉 Custom base image pushed successfully!
+echo.
+echo To use this image, update your Dockerfile:
+echo   FROM %HARBOR_REGISTRY%/%HARBOR_PROJECT%/python:3.11-slim-dev
+echo.
+pause

build-custom-base-images.sh

@@ -0,0 +1,40 @@
+#!/bin/bash
+# Build custom base images with dependencies pre-installed
+# Run this on a machine with internet access, then push to Harbor
+
+set -e
+
+HARBOR_REGISTRY="harbor.dvirlabs.com"
+HARBOR_PROJECT="base-images"
+
+echo "🏗️  Building custom base images for air-gapped cluster..."
+echo "Registry: $HARBOR_REGISTRY"
+echo "Project: $HARBOR_PROJECT"
+echo ""
+
+# Login to Harbor
+echo "Please login to Harbor:"
+docker login $HARBOR_REGISTRY
+
+echo ""
+echo "📦 Building Python base image with dev tools..."
+echo ""
+
+# Build Python dev image
+cd base-images/python-3.11-slim-dev
+docker build -t $HARBOR_REGISTRY/$HARBOR_PROJECT/python:3.11-slim-dev .
+cd ../..
+
+echo "✅ Built: python:3.11-slim-dev"
+echo ""
+
+# Push to Harbor
+echo "🚀 Pushing to Harbor..."
+docker push $HARBOR_REGISTRY/$HARBOR_PROJECT/python:3.11-slim-dev
+
+echo ""
+echo "🎉 Custom base image pushed successfully!"
+echo ""
+echo "To use this image, update your Dockerfile:"
+echo "  FROM $HARBOR_REGISTRY/$HARBOR_PROJECT/python:3.11-slim-dev"
+echo ""

push-base-images.bat

@@ -0,0 +1,82 @@
+@echo off
+REM Script to push base images to Harbor for offline/isolated cluster builds
+
+set HARBOR_REGISTRY=harbor.dvirlabs.com
+set HARBOR_PROJECT=dockerhub
+
+echo.
+echo 🐳 Pushing base images to Harbor...
+echo Registry: %HARBOR_REGISTRY%
+echo Project: %HARBOR_PROJECT%
+echo.
+
+REM Login to Harbor
+echo Please login to Harbor:
+docker login %HARBOR_REGISTRY%
+
+echo.
+echo 📦 Processing base images...
+echo.
+
+REM Python base image
+echo ➡️  Processing: python:3.11-slim
+docker pull python:3.11-slim
+docker tag python:3.11-slim %HARBOR_REGISTRY%/%HARBOR_PROJECT%/python:3.11-slim
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/python:3.11-slim
+echo    ✅ Done
+echo.
+
+REM Node base image
+echo ➡️  Processing: node:18-alpine
+docker pull node:18-alpine
+docker tag node:18-alpine %HARBOR_REGISTRY%/%HARBOR_PROJECT%/node:18-alpine
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/node:18-alpine
+echo    ✅ Done
+echo.
+
+REM Nginx base image
+echo ➡️  Processing: nginx:alpine
+docker pull nginx:alpine
+docker tag nginx:alpine %HARBOR_REGISTRY%/%HARBOR_PROJECT%/nginx:alpine
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/nginx:alpine
+echo    ✅ Done
+echo.
+
+REM Postgres base image
+echo ➡️  Processing: postgres:16-alpine
+docker pull postgres:16-alpine
+docker tag postgres:16-alpine %HARBOR_REGISTRY%/%HARBOR_PROJECT%/postgres:16-alpine
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/postgres:16-alpine
+echo    ✅ Done
+echo.
+
+REM Alpine base image
+echo ➡️  Processing: alpine:3.19
+docker pull alpine:3.19
+docker tag alpine:3.19 %HARBOR_REGISTRY%/%HARBOR_PROJECT%/alpine:3.19
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/alpine:3.19
+echo    ✅ Done
+echo.
+
+REM Busybox images
+echo ➡️  Processing: busybox:1.35
+docker pull busybox:1.35
+docker tag busybox:1.35 %HARBOR_REGISTRY%/%HARBOR_PROJECT%/busybox:1.35
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/busybox:1.35
+echo    ✅ Done
+echo.
+
+echo ➡️  Processing: busybox:latest
+docker pull busybox:latest
+docker tag busybox:latest %HARBOR_REGISTRY%/%HARBOR_PROJECT%/busybox:latest
+docker push %HARBOR_REGISTRY%/%HARBOR_PROJECT%/busybox:latest
+echo    ✅ Done
+echo.
+
+echo.
+echo 🎉 All base images pushed successfully!
+echo.
+echo Your Harbor now contains all required base images.
+echo You can now build your application in the cluster without Docker Hub access!
+echo.
+pause

push-base-images.sh

@@ -0,0 +1,60 @@
+#!/bin/bash
+# Script to push base images to Harbor for offline/isolated cluster builds
+
+set -e
+
+HARBOR_REGISTRY="harbor.dvirlabs.com"
+HARBOR_PROJECT="dockerhub"
+
+echo "🐳 Pushing base images to Harbor..."
+echo "Registry: $HARBOR_REGISTRY"
+echo "Project: $HARBOR_PROJECT"
+echo ""
+
+# Login to Harbor (you may need to provide credentials)
+echo "Please login to Harbor:"
+docker login $HARBOR_REGISTRY
+
+# Base images needed for the project
+BASE_IMAGES=(
+    "python:3.11-slim"
+    "node:18-alpine"
+    "nginx:alpine"
+    "postgres:16-alpine"
+    "alpine:3.19"
+    "busybox:1.35"
+    "busybox:latest"
+)
+
+echo ""
+echo "📦 Processing ${#BASE_IMAGES[@]} base images..."
+echo ""
+
+for IMAGE in "${BASE_IMAGES[@]}"; do
+    echo "➡️  Processing: $IMAGE"
+    
+    # Pull from Docker Hub
+    echo "   Pulling from Docker Hub..."
+    docker pull $IMAGE
+    
+    # Tag for Harbor
+    HARBOR_IMAGE="$HARBOR_REGISTRY/$HARBOR_PROJECT/$IMAGE"
+    echo "   Tagging as: $HARBOR_IMAGE"
+    docker tag $IMAGE $HARBOR_IMAGE
+    
+    # Push to Harbor
+    echo "   Pushing to Harbor..."
+    docker push $HARBOR_IMAGE
+    
+    echo "   ✅ Done: $IMAGE"
+    echo ""
+done
+
+echo "🎉 All base images pushed successfully!"
+echo ""
+echo "Your Harbor now contains:"
+for IMAGE in "${BASE_IMAGES[@]}"; do
+    echo "  - $HARBOR_REGISTRY/$HARBOR_PROJECT/$IMAGE"
+done
+echo ""
+echo "You can now build your application in the cluster without Docker Hub access!"