112 lines
2.6 KiB
Markdown
112 lines
2.6 KiB
Markdown
# Offline PyPI Solution for Air-Gapped Cluster
|
|
|
|
## Problem
|
|
Your cluster blocks **ALL external internet access**:
|
|
- ❌ Docker Hub (`index.docker.io`) - BLOCKED
|
|
- ❌ Debian repositories (`deb.debian.org`) - BLOCKED
|
|
- ❌ **PyPI (`pypi.org`) - BLOCKED** ← NEW ISSUE
|
|
|
|
## Solution: Bundle Python Packages
|
|
|
|
Since `pip install` cannot reach PyPI, we pre-download all packages as wheel files and bundle them in the Docker image.
|
|
|
|
## Quick Fix Steps
|
|
|
|
### Step 1: Download Wheels (On a machine WITH internet)
|
|
|
|
**Windows:**
|
|
```bash
|
|
cd backend
|
|
.\download-wheels.bat
|
|
```
|
|
|
|
**Linux/Mac:**
|
|
```bash
|
|
cd backend
|
|
chmod +x download-wheels.sh
|
|
./download-wheels.sh
|
|
```
|
|
|
|
This downloads ~30-50 wheel files to `backend/wheels/` directory.
|
|
|
|
### Step 2: Commit Wheels to Git
|
|
|
|
```bash
|
|
git add backend/wheels/
|
|
git commit -m "Add offline Python wheels for air-gapped cluster"
|
|
git push
|
|
```
|
|
|
|
### Step 3: Verify Dockerfile
|
|
|
|
The Dockerfile now:
|
|
1. ✅ Copies `wheels/` directory into image
|
|
2. ✅ Installs using `pip install --no-index --find-links /tmp/wheels`
|
|
3. ✅ No internet needed during build!
|
|
|
|
### Step 4: Test Build
|
|
|
|
Your CI pipeline will now build successfully without PyPI access.
|
|
|
|
## How It Works
|
|
|
|
### Before (FAILED):
|
|
```dockerfile
|
|
RUN pip install -r requirements.txt
|
|
# ❌ Tries to reach pypi.org → SSL handshake failure
|
|
```
|
|
|
|
### After (SUCCESS):
|
|
```dockerfile
|
|
COPY wheels/ /tmp/wheels/
|
|
RUN pip install --no-index --find-links /tmp/wheels -r requirements.txt
|
|
# ✅ Installs from local files → No internet needed
|
|
```
|
|
|
|
## Updating Dependencies
|
|
|
|
When you add new packages to `requirements.txt`:
|
|
|
|
1. **Update wheels** (on internet-connected machine):
|
|
```bash
|
|
cd backend
|
|
rm -rf wheels/ # Clear old wheels
|
|
./download-wheels.sh # Download new ones
|
|
```
|
|
|
|
2. **Commit and push**:
|
|
```bash
|
|
git add wheels/ requirements.txt
|
|
git commit -m "Update dependencies and wheels"
|
|
git push
|
|
```
|
|
|
|
## Wheel Directory Size
|
|
|
|
Expect `backend/wheels/` to be **~50-100MB** with all dependencies.
|
|
|
|
**Tips:**
|
|
- ✅ This is normal and necessary for air-gapped deployment
|
|
- ✅ Git can handle it fine
|
|
- ⚠️ If it grows >100MB, consider Git LFS
|
|
|
|
## Alternative: PyPI Mirror in Harbor
|
|
|
|
If Harbor supports PyPI proxy (check with your admin), you could:
|
|
|
|
1. Configure Harbor as PyPI mirror
|
|
2. Update pip to use it:
|
|
```dockerfile
|
|
RUN pip install --index-url https://harbor.dvirlabs.com/api/pypi/simple -r requirements.txt
|
|
```
|
|
|
|
Ask your Harbor admin if PyPI proxy is available.
|
|
|
|
## Current Status
|
|
|
|
✅ Backend Dockerfile - **UPDATED** (installs from local wheels)
|
|
✅ Download scripts - **CREATED**
|
|
⏳ **ACTION NEEDED:** Run `download-wheels.bat` and commit wheels
|
|
|
|
**Next:** Download wheels and push to Git!
|