# Properstar Session Management

## Problem Solved

When running a Full Update, the progress bar could get stuck at "Starting... (0%)" if the Properstar login session had expired. This is now fixed with automatic session validation.

---

## New Features

### 1. Session Validation API

**GET** `/api/validate-session`

Check if your Properstar session is still valid before starting a full update.

**Request:**
```bash
curl http://localhost:5002/api/validate-session
```

**Response (Valid Session):**
```json
{
  "success": true,
  "session": {
    "exists": true,
    "valid": true,
    "age_days": 6.1,
    "age_hours": 147.4,
    "created": "2025-10-13T13:44:31",
    "message": "Session exists (6.1 days old)",
    "needs_refresh": false
  }
}
```

**Response (Expired Session):**
```json
{
  "success": true,
  "session": {
    "exists": true,
    "valid": false,
    "age_days": 8.5,
    "age_hours": 204.0,
    "created": "2025-10-11T10:15:22",
    "message": "Session is 8.5 days old - might be expired",
    "needs_refresh": true
  }
}
```

**Response (No Session):**
```json
{
  "success": true,
  "session": {
    "exists": false,
    "valid": false,
    "message": "No auth.json found - manual login required",
    "needs_refresh": true
  }
}
```

---

### 2. Session Invalidation API

**POST** `/api/invalidate-session`

Force a session refresh by removing auth.json (creates backup first).

**Request:**
```bash
curl -X POST http://localhost:5002/api/invalidate-session
```

**Response:**
```json
{
  "success": true,
  "message": "Session invalidated. Backup saved to auth_backup_20251019_171530.json",
  "backup_file": "auth_backup_20251019_171530.json"
}
```

---

## How It Works

### Session Lifecycle

1. **First Login**
   - User runs scraper without auth.json
   - Browser opens for manual login (10-minute window)
   - Session saved to auth.json

2. **Subsequent Runs**
   - Scraper uses auth.json (headless mode)
   - No manual login needed
   - Works for 7-14 days typically

3. **Session Expiry**
   - After ~7 days, Properstar may expire the session
   - Scraper detects this and exits with error
   - Progress bar shows error state
   - User needs to refresh session

4. **Session Refresh**
   - Call `/api/invalidate-session` to remove old auth.json
   - Run scraper again for fresh login
   - New session saved

---

## Improved Error Handling

### Before (Stuck Progress Bar):
```
Starting full update...
Progress: Starting... (0%)
[Hangs indefinitely - user doesn't know why]
```

### After (Clear Error):
```
Starting full update...
Progress: Starting... (0%)
Error: ❌ SESSIE VERLOPEN - handmatige login vereist

Actions:
1. Verwijder auth.json: rm auth.json
2. Of gebruik: python3 check_session_simple.py invalidate
3. Run opnieuw voor handmatige login
```

---

## Integration with UI

The Criteria Manager UI should:

1. **Check Session Before Full Update**
   ```javascript
   // Before starting full update
   const sessionCheck = await fetch('/api/validate-session');
   const {session} = await sessionCheck.json();

   if (!session.valid || session.needs_refresh) {
     // Show warning modal
     showSessionWarning(session);
     // Offer to invalidate and retry
   }
   ```

2. **Show Session Status**
   ```javascript
   // In system status panel
   GET /api/system-status
   // Returns: auth_status.session_age_days

   if (auth_status.session_age_days > 7) {
     showWarning('Session may be expired - test with /api/validate-session');
   }
   ```

3. **Handle Session Errors**
   ```javascript
   // When full update fails
   if (error.includes('SESSIE VERLOPEN')) {
     showSessionExpiredDialog({
       message: 'Properstar session has expired',
       actions: [
         {
           label: 'Refresh Session',
           action: async () => {
             await fetch('/api/invalidate-session', {method: 'POST'});
             // Then guide user to run scraper manually or show login UI
           }
         }
       ]
     });
   }
   ```

---

## Command Line Usage

### Check Session
```bash
# Simple check
python3 check_session_simple.py

# Output:
======================================================================
PROPERSTAR SESSION CHECK
======================================================================
✅ auth.json exists
   Created: 2025-10-13T13:44:31.020378
   Age: 6.1 days (147.4 hours)
   Status: ✅ Likely valid
======================================================================
```

### Invalidate Session
```bash
# Remove auth.json (with backup)
python3 check_session_simple.py invalidate

# Output:
{
  "success": true,
  "message": "Session invalidated. Backup saved to auth_backup_20251019_171530.json",
  "backup_file": "auth_backup_20251019_171530.json"
}
```

### Manual Login
```bash
# After invalidating, run scraper for fresh login
python3 favorites_scraper.py

# Browser will open for manual login
# Session saved to auth.json after successful login
```

---

## Troubleshooting

### Progress Bar Stuck at 0%

**Symptom:** Full update starts but progress never moves

**Diagnosis:**
```bash
curl http://localhost:5002/api/validate-session
```

**Solutions:**

1. **If session.valid = false:**
   ```bash
   # Invalidate session
   curl -X POST http://localhost:5002/api/invalidate-session

   # Run scraper for fresh login
   cd scraper
   python3 favorites_scraper.py
   # Browser opens - log in manually
   ```

2. **If session.exists = false:**
   ```bash
   # Need initial login
   cd scraper
   python3 favorites_scraper.py
   ```

3. **If other error:**
   ```bash
   # Check logs
   tail -f /tmp/farmmatch_job_<job_id>.log
   ```

---

### Session Age Warning

Sessions typically last 7-14 days. We warn after 7 days.

**Check session age:**
```bash
curl http://localhost:5002/api/system-status | grep session_age_days
```

**If age > 7 days:**
- Session might work, but could fail mid-scrape
- Recommended: Refresh session preemptively
- Best practice: Refresh every Sunday before weekly update

---

### Session Backup Files

When you invalidate a session, a backup is created:
```
auth_backup_YYYYMMDD_HHMMSS.json
```

**Restore a backup:**
```bash
cp auth_backup_20251019_171530.json auth.json
```

**Clean up old backups:**
```bash
# Remove backups older than 30 days
find . -name "auth_backup_*.json" -mtime +30 -delete
```

---

## Session Security Notes

**auth.json contains:**
- Cookies for Properstar authentication
- Session tokens
- Browser fingerprint

**Security best practices:**
1. ✅ File is in `.gitignore` (not committed to repo)
2. ✅ Backups are also ignored
3. ✅ Sessions expire after 7-14 days automatically
4. ✅ Only valid for Properstar.nl domain

**What to do if compromised:**
```bash
# Invalidate immediately
python3 check_session_simple.py invalidate

# Delete all backups
rm auth_backup_*.json

# Create fresh session
python3 favorites_scraper.py
```

---

## API Summary

| Endpoint | Method | Purpose | Response Time |
|----------|--------|---------|---------------|
| `/api/validate-session` | GET | Check if session valid | <1 second |
| `/api/invalidate-session` | POST | Remove session | <1 second |
| `/api/system-status` | GET | Includes session age | <1 second |

**Use before full update:**
```bash
# 1. Validate session
curl http://localhost:5002/api/validate-session

# 2. If valid, proceed with full update
curl -X POST http://localhost:5002/api/scrape-favorites \
  -H "Content-Type: application/json" \
  -d '{"full_pipeline": true}'
```

---

## Testing

### Test Session Validation
```bash
# Should show current session status
curl -s http://localhost:5002/api/validate-session | python3 -m json.tool
```

### Test Session Invalidation
```bash
# Invalidate (creates backup)
curl -X POST http://localhost:5002/api/invalidate-session | python3 -m json.tool

# Verify removed
ls -la auth.json  # Should not exist

# Verify backup created
ls -la auth_backup_*.json
```

### Test Full Update with Expired Session
```bash
# Remove auth.json to simulate expired session
rm auth.json

# Try full update - should fail with clear error
curl -X POST http://localhost:5002/api/scrape-favorites \
  -H "Content-Type: application/json" \
  -d '{"full_pipeline": true}'

# Check job status - should show session error
curl http://localhost:5002/api/job-status/<job_id>
```

---

## Recommended UI Flow

```
User clicks "Full Update"
↓
Call /api/validate-session
↓
┌─────────────────────────┐
│ Session valid?          │
└─────────┬───────────────┘
          │
    ┌─────┴─────┐
    │           │
   Yes         No
    │           │
    │           ↓
    │     Show Modal:
    │     "Session expired"
    │     [Refresh Session] [Cancel]
    │           │
    │     User clicks "Refresh"
    │           │
    │           ↓
    │     POST /api/invalidate-session
    │           │
    │           ↓
    │     Show instructions:
    │     "Run: python3 favorites_scraper.py"
    │     "Log in when browser opens"
    │           │
    ↓           ↓
Proceed    Wait for login
with       then retry
update
```

---

**Session management is now deployed and ready to use!** 🔐

Test it:
```bash
curl http://localhost:5002/api/validate-session
```