mail-services/MAIL_STACK_README.md

13 KiB

Lightweight Mail Stack for k3s GitOps Lab

Complete deployment of Stalwart Mail Server + SnappyMail webmail using GitOps with ArgoCD.

📋 Overview

This repository deploys a lightweight, modern mail stack designed for self-hosting and lab environments:

  • Stalwart Mail Server: All-in-one mail server (SMTP, IMAP, Admin UI)
  • SnappyMail: Modern, lightweight webmail client
  • GitOps: Managed by ArgoCD
  • Storage: NFS-based persistent volumes
  • Ingress: Traefik for web UI access

🏗️ Architecture

┌─────────────────────────────────────────────────┐
│              Traefik Ingress                    │
│  mail.dvirlabs.com   webmail.dvirlabs.com      │
└───────────┬──────────────────────┬──────────────┘
            │                      │
            │                      │
    ┌───────▼────────┐    ┌────────▼────────┐
    │   Stalwart     │◄───│   SnappyMail    │
    │  Mail Server   │    │    Webmail      │
    │                │    │                 │
    │ • SMTP         │    │ Connects via:   │
    │ • IMAP         │    │ • IMAP:993      │
    │ • Admin UI     │    │ • SMTP:587      │
    └────────┬───────┘    └─────────────────┘
             │
             │
    ┌────────▼───────┐
    │  NFS Storage   │
    │  Mail Data     │
    └────────────────┘

📁 Repository Structure

mail-services/
├── argocd-apps/
│   ├── stalwart.yaml         # ArgoCD Application for Stalwart
│   └── snappymail.yaml       # ArgoCD Application for SnappyMail
├── charts/
│   ├── stalwart/             # Local Helm chart for Stalwart
│   │   ├── Chart.yaml
│   │   ├── values.yaml       # Default values
│   │   └── templates/
│   │       ├── namespace.yaml
│   │       ├── secret.yaml
│   │       ├── statefulset.yaml
│   │       ├── service.yaml
│   │       └── ingress.yaml
│   └── snappymail/           # Local Helm chart for SnappyMail
│       ├── Chart.yaml
│       ├── values.yaml       # Default values
│       └── templates/
│           ├── deployment.yaml
│           ├── pvc.yaml
│           ├── service.yaml
│           ├── ingress.yaml
│           └── configmap.yaml
└── manifests/
    ├── stalwart/
    │   └── values.yaml       # Custom values for dvirlabs.com
    └── snappymail/
        └── values.yaml       # Custom values for dvirlabs.com

🚀 Quick Start

Prerequisites

  • k3s cluster running
  • ArgoCD installed and configured
  • Traefik ingress controller
  • NFS storage class (nfs-client)
  • DNS records pointing to your cluster

Step 1: Update Configuration

  1. Update ArgoCD Application manifests with your Git repository URL:
# Edit both files and replace YOUR_USERNAME with your actual repo
vim argocd-apps/stalwart.yaml
vim argocd-apps/snappymail.yaml
  1. Change the Stalwart admin password:
# Edit and set a strong password
vim manifests/stalwart/values.yaml

Find this section and change CHANGE_ME_PLEASE_USE_STRONG_PASSWORD:

secret:
  create: true
  name: stalwart-credentials
  adminPassword: "YOUR_STRONG_PASSWORD_HERE"
  1. Update domain names (if not using dvirlabs.com):
# Update in both files
vim manifests/stalwart/values.yaml
vim manifests/snappymail/values.yaml

Step 2: Deploy with ArgoCD

# Apply ArgoCD Applications
kubectl apply -f argocd-apps/stalwart.yaml
kubectl apply -f argocd-apps/snappymail.yaml

# Check deployment status
kubectl get applications -n argocd

# Watch pods come up
kubectl get pods -n mail -w

Step 3: Verify Deployment

# Check all resources in mail namespace
kubectl get all -n mail

# Check PVCs
kubectl get pvc -n mail

# Check ingresses
kubectl get ingress -n mail

Expected output:

NAME                       READY   STATUS    RESTARTS   AGE
pod/stalwart-0             1/1     Running   0          2m
pod/snappymail-xxx-xxx     1/1     Running   0          2m

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)
service/stalwart     ClusterIP   10.43.x.x       <none>        8080/TCP,25/TCP,587/TCP,993/TCP
service/snappymail   ClusterIP   10.43.x.x       <none>        8888/TCP

NAME                                    CLASS     HOSTS
ingress.networking.k8s.io/stalwart      traefik   mail.dvirlabs.com
ingress.networking.k8s.io/snappymail    traefik   webmail.dvirlabs.com

🌐 Access the Services

Stalwart Admin UI

URL: https://mail.dvirlabs.com

Default credentials:

  • Username: admin@dvirlabs.com
  • Password: (the one you set in manifests/stalwart/values.yaml)

SnappyMail Webmail

URL: https://webmail.dvirlabs.com

First-time setup:

  1. Access the admin panel: https://webmail.dvirlabs.com/?admin
  2. Default admin password: 12345 (change immediately!)
  3. Configure mail server connection:
    • IMAP Server: stalwart.mail.svc.cluster.local
    • IMAP Port: 993
    • IMAP Security: SSL/TLS
    • SMTP Server: stalwart.mail.svc.cluster.local
    • SMTP Port: 587
    • SMTP Security: STARTTLS

📧 Configuring Real Mail Service

Important: Cloudflare Tunnel Limitations

⚠️ WARNING: While Cloudflare Tunnel works fine for web UIs (admin panel and webmail), it CANNOT be used for actual email protocols (SMTP/IMAP).

What works through Cloudflare Tunnel:

  • Stalwart admin UI (HTTPS)
  • SnappyMail webmail (HTTPS)

What does NOT work through Cloudflare Tunnel:

  • Receiving mail from other servers (SMTP port 25)
  • Sending mail to other servers (SMTP port 25)
  • External email clients (IMAP/SMTP)

Required for Real Email

To receive and send real email, you need:

1. DNS Records

; MX Record (Mail Exchange)
@           IN  MX  10  mail.dvirlabs.com.

; A Record (pointing to your public IP - NOT Cloudflare Tunnel)
mail        IN  A       YOUR_PUBLIC_IP

; SPF Record (Sender Policy Framework)
@           IN  TXT     "v=spf1 mx ~all"

; DMARC Record
_dmarc      IN  TXT     "v=DMARC1; p=quarantine; rua=mailto:admin@dvirlabs.com"

; DKIM Record (generated by Stalwart)
; Get this from Stalwart admin UI after setup
default._domainkey  IN  TXT  "v=DKIM1; k=rsa; p=YOUR_PUBLIC_KEY_HERE"

2. Port Forwarding

You need to expose these ports directly (NOT through Cloudflare):

Port 25   (SMTP)  - Required for receiving mail from other servers
Port 587  (SMTP)  - Required for sending mail (submission)
Port 465  (SMTPS) - Optional, secure SMTP submission
Port 993  (IMAPS) - Required for IMAP access
Port 143  (IMAP)  - Optional, plaintext IMAP

Option A: NodePort Service

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: stalwart-external
  namespace: mail
spec:
  type: NodePort
  ports:
  - name: smtp
    port: 25
    targetPort: 25
    nodePort: 30025
  - name: submission
    port: 587
    targetPort: 587
    nodePort: 30587
  - name: imaps
    port: 993
    targetPort: 993
    nodePort: 30993
  selector:
    app.kubernetes.io/name: stalwart
EOF

Then forward ports 25, 587, 993 from your router to your k3s node on ports 30025, 30587, 30993.

Option B: LoadBalancer with MetalLB

If you have MetalLB configured:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: stalwart-lb
  namespace: mail
spec:
  type: LoadBalancer
  loadBalancerIP: YOUR_LB_IP
  ports:
  - name: smtp
    port: 25
    targetPort: 25
  - name: submission
    port: 587
    targetPort: 587
  - name: imaps
    port: 993
    targetPort: 993
  selector:
    app.kubernetes.io/name: stalwart
EOF

3. PTR (Reverse DNS) Record

Contact your ISP or VPS provider to set a PTR record:

YOUR_PUBLIC_IP  ->  mail.dvirlabs.com

This is critical for email deliverability. Without it, many servers will reject your mail.

🔧 Configuration Management

Instead of storing passwords in Git, use External Secrets Operator:

  1. Install External Secrets Operator
  2. Create a secret in your secret backend (Vault, AWS Secrets Manager, etc.)
  3. Update manifests/stalwart/values.yaml:
secret:
  create: false  # Don't create the secret
  name: stalwart-credentials  # Reference external secret
  1. Create an ExternalSecret:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: stalwart-credentials
  namespace: mail
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: your-secret-store
    kind: SecretStore
  target:
    name: stalwart-credentials
  data:
  - secretKey: STALWART_ADMIN_PASSWORD
    remoteRef:
      key: mail/stalwart/admin-password

🛠️ Maintenance

View Stalwart Logs

kubectl logs -n mail stalwart-0 -f

View SnappyMail Logs

kubectl logs -n mail -l app.kubernetes.io/name=snappymail -f

Access Stalwart Shell

kubectl exec -it -n mail stalwart-0 -- /bin/sh

Backup Mail Data

# Backup Stalwart data
kubectl exec -n mail stalwart-0 -- tar czf /tmp/mail-backup.tar.gz /opt/stalwart-mail
kubectl cp mail/stalwart-0:/tmp/mail-backup.tar.gz ./mail-backup-$(date +%Y%m%d).tar.gz

# Backup SnappyMail config
kubectl exec -n mail -l app.kubernetes.io/name=snappymail -- tar czf /tmp/snappymail-backup.tar.gz /var/lib/snappymail
kubectl cp mail/snappymail-xxx:/tmp/snappymail-backup.tar.gz ./snappymail-backup-$(date +%Y%m%d).tar.gz

Restore from Backup

# Restore Stalwart
kubectl cp ./mail-backup.tar.gz mail/stalwart-0:/tmp/mail-backup.tar.gz
kubectl exec -n mail stalwart-0 -- tar xzf /tmp/mail-backup.tar.gz -C /
kubectl rollout restart statefulset -n mail stalwart

🔍 Troubleshooting

Pods Not Starting

# Check pod events
kubectl describe pod -n mail stalwart-0
kubectl describe pod -n mail -l app.kubernetes.io/name=snappymail

# Check PVC status
kubectl get pvc -n mail

Ingress Not Working

# Check ingress status
kubectl describe ingress -n mail

# Check Traefik logs
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik

# Test internal connectivity
kubectl run -it --rm debug --image=busybox -n mail -- wget -O- http://stalwart:8080

SnappyMail Can't Connect to Stalwart

# Test IMAP connectivity from SnappyMail pod
kubectl exec -it -n mail -l app.kubernetes.io/name=snappymail -- nc -zv stalwart.mail.svc.cluster.local 993

# Check Stalwart service
kubectl get svc -n mail stalwart

Email Not Being Delivered

Common issues:

  1. No PTR record: Check reverse DNS
  2. Port 25 blocked: Many ISPs block outbound port 25
  3. Missing SPF/DKIM/DMARC: Check DNS records
  4. IP on blacklist: Check https://mxtoolbox.com/blacklists.aspx

📊 Monitoring

Check Mail Queue

# Access Stalwart admin UI
# https://mail.dvirlabs.com
# Navigate to Queue section

Resource Usage

# Check pod resource usage
kubectl top pods -n mail

# Check PVC usage
kubectl exec -n mail stalwart-0 -- df -h /opt/stalwart-mail

🔐 Security Hardening

Recommended Post-Deployment Steps

  1. Change SnappyMail admin password immediately
  2. Enable fail2ban for brute force protection
  3. Set up TLS certificates with cert-manager (if not using Cloudflare)
  4. Enable DKIM signing in Stalwart
  5. Configure rate limiting in Stalwart
  6. Regular backups of mail data
  7. Monitor logs for suspicious activity

TLS Certificates with cert-manager

If you want Let's Encrypt certificates instead of Cloudflare:

# Add to ingress annotations
cert-manager.io/cluster-issuer: letsencrypt-prod

And configure TLS:

tls:
  - hosts:
      - mail.dvirlabs.com
    secretName: stalwart-tls

📚 Additional Resources

⚙️ Customization

Adjust Storage Size

Edit manifests/stalwart/values.yaml or manifests/snappymail/values.yaml:

persistence:
  size: 50Gi  # Increase as needed

Change Resource Limits

Edit manifests/*/values.yaml:

resources:
  requests:
    memory: "2Gi"
    cpu: "1000m"
  limits:
    memory: "8Gi"
    cpu: "4000m"

Add Multiple Domains

Configure in Stalwart admin UI after deployment.

🤝 Contributing

This is a personal lab setup, but feel free to fork and adapt for your needs!

📝 License

MIT License - Use at your own risk

⚠️ Disclaimer

This setup is designed for lab/self-hosting environments. For production use:

  • Use External Secrets for credentials
  • Set up proper TLS certificates
  • Configure backup automation
  • Enable monitoring and alerting
  • Review security best practices
  • Test email deliverability thoroughly