diff --git a/WHATSAPP_DEBUGGING.md b/WHATSAPP_DEBUGGING.md new file mode 100644 index 0000000..e7910b4 --- /dev/null +++ b/WHATSAPP_DEBUGGING.md @@ -0,0 +1,345 @@ +# WhatsApp Cloud API Debugging - Complete Implementation + +## Problem Summary + +WhatsApp Cloud API returns HTTP 200 with a wamid, but recipients don't receive messages. + +**Root Cause:** The initial API response (HTTP 200) only means Meta **accepted** the message, NOT that it was delivered. The actual delivery status comes from **webhook callbacks** which were not implemented. + +--- + +## Solution Implemented + +### 1. ✅ Message Status Tracking + +**File:** `models.py` + +Added `WhatsAppMessage` model to track: +- `wamid` - WhatsApp message ID from Meta +- `status` - sent, delivered, read, failed +- `event_id`, `guest_id` - Context for tracking +- `error_code`, `error_title`, `error_message` - Failure details +- Timestamps: `sent_at`, `delivered_at`, `read_at`, `failed_at` + +### 2. ✅ Webhook Endpoint + +**File:** `main.py` + +Added two webhook endpoints: + +```python +GET /whatsapp/webhook # Webhook verification +POST /whatsapp/webhook # Status updates from Meta +``` + +**What it does:** +- Receives delivery status updates from Meta +- Updates message status in database +- Logs all webhook payloads for debugging +- Handles: sent, delivered, read, failed statuses +- Captures error details when messages fail + +### 3. ✅ Enhanced WhatsApp Service + +**File:** `whatsapp.py` + +**Changes:** +1. Added `event_id` and `guest_id` parameters to `send_by_template_key()` +2. Saves messages to database after sending +3. Supports dynamic image headers via `header_handle_key` +4. Enhanced logging with full payload dumps + +### 4. ✅ Fixed hina_invitation Template + +**File:** `whatsapp_templates.py` + +**Configuration:** +```python +{ + "meta_name": "hina_invitation", + "language_code": "he", + "header_type": "IMAGE", + "header_handle_key": "invitation_image_url", # Dynamic image from params + "body_params": ["contact_name"], + "button_type": "URL", + "button_url": "https://invy.dvirlabs.com/guest/{{1}}", + "button_param_key": "event_id", +} +``` + +### 5. ✅ Comprehensive Test Script + +**File:** `test_whatsapp_debug.py` + +**Features:** +- Tests phone normalization +- Validates template configuration +- Sends test message with full logging +- Shows webhook setup instructions +- Provides clear debugging output + +### 6. ✅ Database Migration + +**File:** `add_whatsapp_messages_table.py` + +Creates the `whatsapp_messages` table. + +--- + +## Setup Instructions + +### Step 1: Run Database Migration + +```bash +cd backend +python add_whatsapp_messages_table.py +``` + +### Step 2: Configure Webhook in Meta Business Manager + +1. Go to **Meta Business Manager** → **WhatsApp** → **Configuration** → **Webhook** + +2. **Callback URL:** + ``` + https://invy.dvirlabs.com/whatsapp/webhook + ``` + +3. **Verify Token:** + - Set this in `.env` as `WHATSAPP_VERIFY_TOKEN=your_secret_token_here` + - Use the same value in Meta's webhook configuration + +4. Click **Verify and Save** + +5. **Subscribe to Fields:** + - ✅ messages + +### Step 3: Update Environment Variables + +Add to `.env`: +```bash +# Existing WhatsApp credentials +WHATSAPP_ACCESS_TOKEN=your_meta_access_token +WHATSAPP_PHONE_NUMBER_ID=your_phone_number_id +WHATSAPP_API_VERSION=v20.0 + +# NEW: Webhook verification token +WHATSAPP_VERIFY_TOKEN=your_webhook_secret_token_here +``` + +### Step 4: Test the Integration + +```bash +cd backend +python test_whatsapp_debug.py +``` + +**What to check:** +1. Phone normalization: ✓ All pass +2. Template configuration: ✓ All correct +3. Message send: ✓ Returns wamid +4. Webhook configured: ✓ Verify token set + +### Step 5: Send Test Message + +The script will prompt you to send a test message to: `0504370045` + +**Expected Result:** +```json +{ + "message_id": "wamid.HBgLOTcyNTA0MzcwMDQ1FQIAERgSQzI4QkE1RkQ5MzFFNTY1RTEwAA==", + "status": "sent", + "to": "+972504370045", + "template": "hina_invitation" +} +``` + +--- + +## How to Debug Delivery Issues + +### Check 1: Backend Logs + +Watch for webhook callbacks: +```bash +tail -f backend.log | grep "WhatsApp Webhook" +``` + +**Expected logs:** +``` +[WhatsApp Webhook] Status update: wamid=xxx, status=delivered +[WhatsApp Webhook] ✓ Updated message xxx to status: delivered +``` + +### Check 2: Database Status + +```sql +SELECT + wamid, + to_phone, + status, + template_name, + sent_at, + delivered_at, + failed_at, + error_code, + error_message +FROM whatsapp_messages +ORDER BY sent_at DESC +LIMIT 10; +``` + +**Status meanings:** +- `sent` - Meta accepted the message +- `delivered` - Recipient's phone received it +- `read` - Recipient opened the message +- `failed` - Delivery failed (check error_code/error_message) + +### Check 3: Common Failure Reasons + +**If status stays "sent" and never reaches "delivered":** + +1. **Phone number not on WhatsApp** + - Error code: 131026 + - Solution: Verify the recipient has WhatsApp + +2. **Template parameters mismatch** + - Error code: 132000 + - Solution: Check template in Meta matches your params + +3. **Image URL not accessible** + - Error code: 132015 + - Solution: Ensure image URL is publicly accessible + +4. **Phone number blocked your business** + - Error code: 131031 + - Solution: Recipient must unblock your business number + +5. **Rate limiting** + - Error code: 130429 + - Solution: Slow down sending rate + +### Check 4: Webhook Delivery Test + +Test webhook is receiving callbacks: +```bash +curl -X GET "https://invy.dvirlabs.com/whatsapp/webhook?hub.mode=subscribe&hub.verify_token=YOUR_TOKEN&hub.challenge=test123" +``` + +Expected: `test123` (echo challenge) + +--- + +## Example: Sending hina_invitation + +```python +from whatsapp import get_whatsapp_service + +service = get_whatsapp_service(db) + +params = { + "contact_name": "איילה חורב", + "event_id": "f3122a7d-1d7c-4cc1-955d-1c6b7358bd25", + "invitation_image_url": "https://api-invy.dvirlabs.com/uploads/1d32b5fbab0f494cae443b4188a83ca3.jpg", +} + +result = await service.send_by_template_key( + template_key="hina_invitation", + to_phone="0504370045", + params=params, + event_id="f3122a7d-1d7c-4cc1-955d-1c6b7358bd25", + guest_id="guest-uuid-here", +) + +print(f"Message ID: {result['message_id']}") +``` + +**Generated payload:** +```json +{ + "messaging_product": "whatsapp", + "to": "+972504370045", + "type": "template", + "template": { + "name": "hina_invitation", + "language": { "code": "he" }, + "components": [ + { + "type": "header", + "parameters": [{ + "type": "image", + "image": { + "link": "https://api-invy.dvirlabs.com/uploads/1d32b5fbab0f494cae443b4188a83ca3.jpg" + } + }] + }, + { + "type": "body", + "parameters": [ + { "type": "text", "text": "איילה חורב" } + ] + }, + { + "type": "button", + "sub_type": "url", + "index": "0", + "parameters": [ + { "type": "text", "text": "f3122a7d-1d7c-4cc1-955d-1c6b7358bd25" } + ] + } + ] + } +} +``` + +--- + +## Phone Normalization + +**Israeli numbers:** +- `0504370045` → `+972504370045` ✓ +- `050-437-0045` → `+972504370045` ✓ +- `+972504370045` → `+972504370045` ✓ +- `972504370045` → `+972504370045` ✓ + +--- + +## Files Modified + +1. ✅ `models.py` - Added WhatsAppMessage model +2. ✅ `main.py` - Added webhook endpoints, updated imports +3. ✅ `whatsapp.py` - Added event/guest tracking, dynamic image support +4. ✅ `whatsapp_templates.py` - Fixed hina_invitation template + +## Files Created + +1. ✅ `test_whatsapp_debug.py` - Comprehensive debugging script +2. ✅ `add_whatsapp_messages_table.py` - Database migration +3. ✅ `WHATSAPP_DEBUGGING.md` - This documentation + +--- + +## Next Steps + +1. **Run migration:** `python add_whatsapp_messages_table.py` +2. **Configure webhook in Meta Business Manager** +3. **Set WHATSAPP_VERIFY_TOKEN in .env** +4. **Run test script:** `python test_whatsapp_debug.py` +5. **Monitor webhook logs for delivery status** +6. **Check database for message status updates** + +--- + +## Critical Understanding + +**HTTP 200 ≠ Message Delivered** + +When Meta returns HTTP 200, it means: +- ✓ Request was valid +- ✓ Message was queued for delivery +- ✗ **NOT** that the recipient received it + +**Actual delivery confirmation comes from webhook callbacks.** + +Without webhooks, you'll never know if messages were delivered, read, or failed. + +**This is why webhook implementation is CRITICAL.** diff --git a/backend/add_whatsapp_messages_table.py b/backend/add_whatsapp_messages_table.py new file mode 100644 index 0000000..97247fe --- /dev/null +++ b/backend/add_whatsapp_messages_table.py @@ -0,0 +1,64 @@ +""" +Database Migration: Add WhatsApp Message Tracking +================================================== + +This migration adds the whatsapp_messages table for tracking +message delivery status from Meta WhatsApp Cloud API webhooks. + +Run this migration: + python add_whatsapp_messages_table.py +""" + +import sys +from pathlib import Path + +# Add backend to path +sys.path.insert(0, str(Path(__file__).parent)) + +from database import engine, Base, SessionLocal +from models import WhatsAppMessage +import logging + +logging.basicConfig(level=logging.INFO) +logger = logging.getLogger(__name__) + + +def run_migration(): + """Create the whatsapp_messages table""" + logger.info("Starting migration: Add whatsapp_messages table") + + try: + # Create only the WhatsAppMessage table + logger.info("Creating whatsapp_messages table...") + WhatsAppMessage.__table__.create(engine, checkfirst=True) + logger.info("✓ whatsapp_messages table created successfully") + + # Verify table exists + db = SessionLocal() + try: + result = db.execute("SELECT COUNT(*) FROM whatsapp_messages") + count = result.scalar() + logger.info(f"✓ Table verified: {count} messages in database") + finally: + db.close() + + logger.info("Migration completed successfully!") + + except Exception as e: + logger.error(f"✗ Migration failed: {e}") + raise + + +if __name__ == "__main__": + print("="*80) + print("WhatsApp Messages Table Migration") + print("="*80) + print("\nThis will create the whatsapp_messages table for tracking") + print("delivery status of WhatsApp messages sent through Meta Cloud API.") + print() + + response = input("Proceed with migration? (yes/no): ").strip().lower() + if response in ['yes', 'y']: + run_migration() + else: + print("Migration cancelled.") diff --git a/backend/test_whatsapp_debug.py b/backend/test_whatsapp_debug.py new file mode 100644 index 0000000..d514f70 --- /dev/null +++ b/backend/test_whatsapp_debug.py @@ -0,0 +1,313 @@ +""" +WhatsApp Cloud API Debug Test Script +===================================== + +This script tests WhatsApp message sending with full debugging to identify +why messages return HTTP 200 but are not received by recipients. + +Usage: + python test_whatsapp_debug.py + +Target: + Phone: 0504370045 → 972504370045 + Template: hina_invitation + Language: he +""" + +import asyncio +import sys +import os +import json +import logging +from pathlib import Path + +# Add backend to path +sys.path.insert(0, str(Path(__file__).parent)) + +# Configure detailed logging +logging.basicConfig( + level=logging.DEBUG, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' +) +logger = logging.getLogger(__name__) + +# Load environment variables +from dotenv import load_dotenv +load_dotenv() + +from whatsapp import WhatsAppService, WhatsAppError +from database import SessionLocal + + +async def test_phone_normalization(): + """Test that phone number normalization works correctly""" + print("\n" + "="*80) + print("STEP 1: Phone Number Normalization Test") + print("="*80) + + test_numbers = [ + ("0504370045", "+972504370045"), + ("050-437-0045", "+972504370045"), + ("+972504370045", "+972504370045"), + ("972504370045", "+972504370045"), + ("0506118707", "+972506118707"), + ] + + for input_phone, expected in test_numbers: + normalized = WhatsAppService.normalize_phone_to_e164(input_phone) + status = "✓" if normalized == expected else "✗" + print(f"{status} {input_phone:20} → {normalized:20} (expected: {expected})") + if normalized != expected: + print(f" ERROR: Normalization mismatch!") + return False + + print("\n✓ All phone normalizations correct") + return True + + +async def test_template_configuration(db): + """Test that hina_invitation template is configured correctly""" + print("\n" + "="*80) + print("STEP 2: Template Configuration Test") + print("="*80) + + try: + from whatsapp_templates import get_template + + template = get_template(db, "hina_invitation") + + print("\nTemplate Configuration:") + print(f" meta_name: {template['meta_name']}") + print(f" language_code: {template['language_code']}") + print(f" header_type: {template.get('header_type', 'TEXT')}") + print(f" header_params: {template.get('header_params', [])}") + print(f" header_handle_key:{template.get('header_handle_key', 'N/A')}") + print(f" body_params: {template.get('body_params', [])}") + print(f" button_type: {template.get('button_type', 'N/A')}") + print(f" button_url: {template.get('button_url', 'N/A')}") + print(f" button_param_key: {template.get('button_param_key', 'N/A')}") + + # Validate template structure + issues = [] + + if template.get('meta_name') != 'hina_invitation': + issues.append("meta_name must be 'hina_invitation'") + + if template.get('language_code') != 'he': + issues.append("language_code must be 'he'") + + if template.get('header_type') != 'IMAGE': + issues.append("header_type should be 'IMAGE' for this template") + + if not template.get('header_handle_key'): + issues.append("header_handle_key should be set for dynamic image") + + if template.get('body_params') != ['contact_name']: + issues.append("body_params should be ['contact_name']") + + if template.get('button_param_key') != 'event_id': + issues.append("button_param_key should be 'event_id'") + + if template.get('button_url') != 'https://invy.dvirlabs.com/guest/{{1}}': + issues.append("button_url should include dynamic parameter {{1}}") + + if issues: + print("\n✗ Template configuration issues found:") + for issue in issues: + print(f" - {issue}") + return False + + print("\n✓ Template configuration is correct") + return True + + except Exception as e: + print(f"\n✗ Failed to load template: {e}") + return False + + +async def test_send_message(db): + """Send a test WhatsApp message with full logging""" + print("\n" + "="*80) + print("STEP 3: Send Test WhatsApp Message") + print("="*80) + + # Test parameters + target_phone = "0504370045" + template_key = "hina_invitation" + + params = { + "contact_name": "בדיקה", + "event_id": "f3122a7d-1d7c-4cc1-955d-1c6b7358bd25", + "invitation_image_url": "https://api-invy.dvirlabs.com/uploads/1d32b5fbab0f494cae443b4188a83ca3.jpg", + } + + print(f"\nTest Parameters:") + print(f" Phone: {target_phone} → {WhatsAppService.normalize_phone_to_e164(target_phone)}") + print(f" Template: {template_key}") + print(f" Language: he") + print(f" Contact Name: {params['contact_name']}") + print(f" Event ID: {params['event_id']}") + print(f" Image URL: {params['invitation_image_url']}") + + print(f"\nMeta API Configuration:") + print(f" Access Token: {'***' + os.getenv('WHATSAPP_ACCESS_TOKEN', '')[-8:] if os.getenv('WHATSAPP_ACCESS_TOKEN') else 'NOT SET'}") + print(f" Phone Number ID: {os.getenv('WHATSAPP_PHONE_NUMBER_ID', 'NOT SET')}") + print(f" API Version: {os.getenv('WHATSAPP_API_VERSION', 'v20.0')}") + + if not os.getenv('WHATSAPP_ACCESS_TOKEN') or not os.getenv('WHATSAPP_PHONE_NUMBER_ID'): + print("\n✗ ERROR: WhatsApp credentials not configured in .env file") + return False + + try: + service = WhatsAppService(db=db) + + print("\n" + "-"*80) + print("Sending message to Meta WhatsApp Cloud API...") + print("-"*80) + + result = await service.send_by_template_key( + template_key=template_key, + to_phone=target_phone, + params=params, + event_id=params['event_id'], + guest_id=None, + ) + + print("\n" + "="*80) + print("✓ MESSAGE SENT SUCCESSFULLY!") + print("="*80) + print(f"\nResponse:") + print(json.dumps(result, indent=2, ensure_ascii=False)) + + wamid = result.get('message_id', 'N/A') + print(f"\n✓ WhatsApp Message ID (wamid): {wamid}") + print(f"✓ Sent to: {result.get('to', 'N/A')}") + print(f"✓ Template: {result.get('template', 'N/A')}") + print(f"✓ Status: {result.get('status', 'N/A')}") + + print("\n" + "="*80) + print("IMPORTANT: Check WhatsApp Webhook for Delivery Status") + print("="*80) + print(""" +The message was accepted by Meta (HTTP 200), but this does NOT mean +it was delivered. To confirm delivery: + +1. Configure webhook in Meta Business Manager: + URL: https://your-domain.com/whatsapp/webhook + Verify Token: (from WHATSAPP_VERIFY_TOKEN in .env) + +2. Subscribe to 'messages' webhook events + +3. Watch the backend logs for webhook callbacks: + - Status: 'sent' = Meta accepted + - Status: 'delivered' = Recipient received + - Status: 'read' = Recipient opened + - Status: 'failed' = Delivery failed (check error details) + +4. Query the database for message status: + SELECT * FROM whatsapp_messages WHERE wamid = '{wamid}'; + +If the message stays in 'sent' status and never reaches 'delivered', +the problem is with Meta's delivery, not your code. + +Common reasons for non-delivery: +- Phone number not registered with WhatsApp +- Phone number blocked your business account +- Template not approved or parameters mismatch +- Image URL not accessible to Meta servers +- Rate limiting or business account issues + """) + + return True + + except WhatsAppError as e: + print(f"\n✗ WhatsApp API Error: {e}") + return False + except Exception as e: + print(f"\n✗ Unexpected error: {e}") + logger.exception("Full traceback:") + return False + + +async def check_webhook_configuration(): + """Check if webhook is configured""" + print("\n" + "="*80) + print("STEP 4: Webhook Configuration Check") + print("="*80) + + verify_token = os.getenv('WHATSAPP_VERIFY_TOKEN', '') + + print(f"\nWebhook Settings:") + print(f" Verify Token: {'***' + verify_token[-8:] if verify_token else 'NOT SET'}") + print(f" Webhook URL: https://your-domain.com/whatsapp/webhook") + + if not verify_token: + print("\n⚠ Warning: WHATSAPP_VERIFY_TOKEN not set in .env") + print(" You need this to verify the webhook with Meta") + else: + print("\n✓ Verify token is configured") + + print("\nWebhook Endpoints Available:") + print(" GET /whatsapp/webhook - Webhook verification") + print(" POST /whatsapp/webhook - Status updates from Meta") + + print("\nTo configure webhook in Meta Business Manager:") + print(" 1. Go to WhatsApp > Configuration > Webhook") + print(" 2. Set Callback URL: https://your-domain.com/whatsapp/webhook") + print(f" 3. Set Verify Token: {verify_token if verify_token else '(SET THIS IN .env)'}") + print(" 4. Click 'Verify and Save'") + print(" 5. Subscribe to 'messages' webhook field") + + +async def main(): + """Run all debug tests""" + print("\n" + "="*80) + print("WhatsApp Cloud API Debugging Tool") + print("="*80) + print(""" +This script will: +1. Test phone number normalization +2. Validate template configuration +3. Send a test message with full logging +4. Show webhook configuration status + """) + + db = SessionLocal() + try: + # Run tests + test1 = await test_phone_normalization() + test2 = await test_template_configuration(db) + + if not (test1 and test2): + print("\n✗ Pre-flight checks failed. Fix errors above before sending.") + return + + # Ask user to confirm send + print("\n" + "="*80) + response = input("Ready to send test message? (yes/no): ").strip().lower() + if response not in ['yes', 'y']: + print("Test cancelled.") + return + + test3 = await test_send_message(db) + await check_webhook_configuration() + + print("\n" + "="*80) + print("Debug Session Complete") + print("="*80) + + if test3: + print("\n✓ Message sent successfully") + print("\nNext steps:") + print("1. Check if recipient received the message on WhatsApp") + print("2. Watch backend logs for webhook status updates") + print("3. Query database: SELECT * FROM whatsapp_messages;") + else: + print("\n✗ Message send failed - see errors above") + + finally: + db.close() + + +if __name__ == "__main__": + asyncio.run(main())