dropbox-backup-lib 0.3.0
A reusable library for backing up files to Dropbox and syncing from Dropbox with configurable patterns and exclusions
Requires Python: >=3.8
Keywords: dropbox, backup, sync, automation, cloud-storage, file-synchronization

# Dropbox Backup Library

A reusable Python library for backing up files to Dropbox and syncing files from Dropbox with configurable patterns and exclusions.

## Features

- **True bidirectional sync**: Automatically syncs changes in both directions with intelligent conflict resolution
- **Backup mode**: One-way upload from local to Dropbox
- OAuth2 refresh token authentication (tokens never expire)
- Configurable file patterns for inclusion and exclusion
- Smart conflict resolution (timestamp, local wins, remote wins, or prompt)
- Tracks sync state to detect changes since last sync
- Progress tracking and callbacks
- Easy integration into any Python project
- Command-line tools for authentication and syncing
- Dry-run mode to preview changes before applying
- Type hints and comprehensive documentation

## Installation

### For Development (Editable Install)

```bash
cd ~/projects/dropbox-backup-library
pip install -e .
```

This installs the package in "editable" mode, so changes to the source code are immediately available.

### For Production

```bash
pip install git+https://github.com/clayroach/dropbox-backup-library.git
```

Or if you publish to PyPI:

```bash
pip install dropbox-backup-lib
```

## Quick Start

### Interactive Tutorial (Recommended)

The fastest way to get started is to run the interactive getting started example:

```bash
cd examples
python3 getting_started.py
```

This will walk you through:
1. Setting up your Dropbox app credentials
2. Generating an OAuth refresh token
3. Testing backup functionality
4. Testing sync functionality
5. Verifying everything works

### Manual Setup

If you prefer to set things up manually:

### 1. Set Up Dropbox App

1. Go to [Dropbox App Console](https://www.dropbox.com/developers/apps)
2. Click "Create app"
3. Choose:
   - **API:** Scoped API
   - **Type:** Full Dropbox
   - **Name:** Your app name (e.g., "my-project-backup")
4. Go to "Permissions" tab and enable:
   - `files.metadata.read`
   - `files.content.read`
   - `files.content.write`
5. Go to "Settings" tab and note your App key and App secret

### 2. Generate Refresh Token

Create a `.env` file with your app credentials:

```bash
DROPBOX_APP_KEY=your_app_key_here
DROPBOX_APP_SECRET=your_app_secret_here
DROPBOX_REFRESH_TOKEN=
```

Then run the token generator:

```bash
dropbox-get-token
```

This will guide you through the OAuth flow and save the refresh token to your `.env` file.

### CLI Tool for Syncing

You can use the command-line tool for true bidirectional sync:

```bash
# Add sync configuration to .env
BACKUP_DROPBOX_PATH=/backups/my-project
BACKUP_SOURCE_DIR=./local-copy

# Run bidirectional sync
dropbox-sync

# Preview changes without applying them
dropbox-sync --dry-run
```

The CLI will:
- Detect changes in both local directory and Dropbox
- Upload local changes to Dropbox
- Download Dropbox changes to local
- Handle conflicts based on your chosen strategy
- Track sync state to know what changed since last sync
- Show progress and summary

### 3. Use in Your Code

**Backup to Dropbox:**

```python
from pathlib import Path
from dotenv import load_dotenv
from dropbox_backup_lib import BackupConfig, DropboxBackup

# Load environment variables
load_dotenv()

# Configure backup
config = BackupConfig.from_env(
    source_dir=Path.cwd(),
    dropbox_path='/backups/my-project',
    backup_patterns=[
        '*.md',
        '*.py',
        '*.xlsx',
        'data/**/*.csv',
    ],
    exclude_patterns=[
        '.git/*',
        '*.pyc',
        '.env*',
    ]
)

# Run backup
backup = DropboxBackup(config)
result = backup.run()

print(f"Backed up {result.success_count} files")
if result.failed_count > 0:
    print(f"Failed: {result.failed_count}")
```

**Bidirectional Sync (recommended):**

```python
from pathlib import Path
from dotenv import load_dotenv
from dropbox_backup_lib import BidirectionalSyncConfig, BidirectionalSync, ConflictStrategy

# Load environment variables
load_dotenv()

# Configure bidirectional sync
config = BidirectionalSyncConfig.from_env(
    local_dir=Path.cwd() / 'synced-files',
    dropbox_path='/backups/my-project',
    conflict_strategy=ConflictStrategy.NEWER_WINS
)

# Run sync
sync = BidirectionalSync(config)
result = sync.run()

print(f"Uploaded {len(result.uploaded)}, Downloaded {len(result.downloaded)}")
if result.conflicts:
    print(f"Conflicts: {len(result.conflicts)}")
```

**One-way download from Dropbox (legacy):**

```python
from pathlib import Path
from dotenv import load_dotenv
from dropbox_backup_lib import SyncConfig, DropboxSync, ConflictResolution

# Load environment variables
load_dotenv()

# Configure sync
config = SyncConfig.from_env(
    dropbox_path='/backups/my-project',
    local_dir=Path.cwd() / 'synced-files',
    conflict_resolution=ConflictResolution.TIMESTAMP
)

# Run sync
sync = DropboxSync(config)
result = sync.run()

print(f"Downloaded {result.downloaded_count} files")
if result.skipped_count > 0:
    print(f"Skipped {result.skipped_count} (local files newer)")
```

## API Reference

### BackupConfig

Configuration object for backup operations.

```python
from dropbox_backup_lib import BackupConfig

# Create from environment variables
config = BackupConfig.from_env(
    source_dir=Path('/path/to/source'),
    dropbox_path='/destination/in/dropbox'
)

# Or create manually
config = BackupConfig(
    app_key='your_app_key',
    app_secret='your_app_secret',
    refresh_token='your_refresh_token',
    source_dir=Path('/path/to/source'),
    dropbox_path='/destination/in/dropbox',
    backup_patterns=['*.md', '*.py'],
    exclude_patterns=['.git/*', '*.pyc']
)
```

**Parameters:**
- `app_key` (str): Dropbox app key
- `app_secret` (str): Dropbox app secret
- `refresh_token` (str): OAuth2 refresh token
- `source_dir` (Path): Directory to backup from
- `dropbox_path` (str): Destination path in Dropbox
- `backup_patterns` (List[str]): Glob patterns for files to include
- `exclude_patterns` (List[str]): Glob patterns for files to exclude

### DropboxBackup

Main backup class.

```python
from dropbox_backup_lib import DropboxBackup

backup = DropboxBackup(config)

# Run backup with default settings
result = backup.run()

# Run with custom progress callback
def on_progress(file_path, success, message):
    print(f"{'✓' if success else '✗'} {file_path}")

result = backup.run(progress_callback=on_progress, verbose=False)

# Access results
print(f"Success: {result.success_count}")
print(f"Failed: {result.failed_count}")
print(f"Total: {result.total_count}")
```

### SyncConfig

Configuration object for sync (download) operations.

```python
from dropbox_backup_lib import SyncConfig, ConflictResolution

# Create from environment variables
config = SyncConfig.from_env(
    dropbox_path='/backups/my-project',
    local_dir=Path('/path/to/local')
)

# Or create manually with all options
config = SyncConfig(
    app_key='your_app_key',
    app_secret='your_app_secret',
    refresh_token='your_refresh_token',
    dropbox_path='/backups/my-project',
    local_dir=Path('/path/to/local'),
    sync_patterns=['*.md', '*.py'],  # None = sync all files
    exclude_patterns=['.git/*', '*.pyc'],
    conflict_resolution=ConflictResolution.TIMESTAMP,
    track_modifications=True
)
```

**Parameters:**
- `app_key` (str): Dropbox app key
- `app_secret` (str): Dropbox app secret
- `refresh_token` (str): OAuth2 refresh token
- `dropbox_path` (str): Source path in Dropbox to sync from
- `local_dir` (Path): Local directory to sync to
- `sync_patterns` (List[str] | None): Glob patterns for files to include (None = all files)
- `exclude_patterns` (List[str]): Glob patterns for files to exclude
- `conflict_resolution` (ConflictResolution): Strategy for handling conflicts
  - `ConflictResolution.OVERWRITE`: Always overwrite local files
  - `ConflictResolution.SKIP`: Keep local files, skip download
  - `ConflictResolution.TIMESTAMP`: Download only if Dropbox file is newer
  - `ConflictResolution.PROMPT`: Ask user for each conflict
- `track_modifications` (bool): Track local file modifications since last sync

### BidirectionalSyncConfig

Configuration object for true bidirectional sync operations.

```python
from dropbox_backup_lib import BidirectionalSyncConfig, ConflictStrategy

# Create from environment variables
config = BidirectionalSyncConfig.from_env(
    local_dir=Path('/path/to/local'),
    dropbox_path='/backups/my-project'
)

# Or create manually with all options
config = BidirectionalSyncConfig(
    app_key='your_app_key',
    app_secret='your_app_secret',
    refresh_token='your_refresh_token',
    local_dir=Path('/path/to/local'),
    dropbox_path='/backups/my-project',
    sync_patterns=['*.md', '*.py'],  # None = sync all files
    exclude_patterns=['.git/*', '*.pyc'],
    conflict_strategy=ConflictStrategy.NEWER_WINS
)
```

**Parameters:**
- `app_key` (str): Dropbox app key
- `app_secret` (str): Dropbox app secret
- `refresh_token` (str): OAuth2 refresh token
- `local_dir` (Path): Local directory to sync
- `dropbox_path` (str): Dropbox path to sync
- `sync_patterns` (List[str] | None): Glob patterns for files to include (None = all files)
- `exclude_patterns` (List[str]): Glob patterns for files to exclude
- `conflict_strategy` (ConflictStrategy): Strategy for handling three-way conflicts
  - `ConflictStrategy.LOCAL_WINS`: Always prefer local version
  - `ConflictStrategy.REMOTE_WINS`: Always prefer Dropbox version
  - `ConflictStrategy.NEWER_WINS`: Use timestamp to decide (default)
  - `ConflictStrategy.PROMPT`: Ask user for each conflict
- `state_file` (Path): Path to file tracking sync state (default: local_dir/.sync_state.json)

### BidirectionalSync

Main class for true bidirectional sync between local directory and Dropbox.

```python
from dropbox_backup_lib import BidirectionalSync, ConflictStrategy, SyncDirection

sync = BidirectionalSync(config)

# Run sync with default settings
result = sync.run()

# Run with dry-run to preview changes
result = sync.run(dry_run=True)

# Run with custom conflict callback
def handle_conflict(path, local_mtime, remote_mtime):
    print(f"Conflict: {path}")
    return SyncDirection.UPLOAD  # or DOWNLOAD or SKIP

result = sync.run(
    conflict_callback=handle_conflict,
    verbose=True
)

# Access results
print(f"Uploaded: {len(result.uploaded)}")
print(f"Downloaded: {len(result.downloaded)}")
print(f"Conflicts: {len(result.conflicts)}")
print(f"Failed: {len(result.failed)}")
```

### DropboxSync

Main sync class for one-way download from Dropbox (legacy).

```python
from dropbox_backup_lib import DropboxSync, ConflictResolution

sync = DropboxSync(config)

# Run sync with default settings
result = sync.run()

# Run with custom conflict callback
def handle_conflict(local_path, local_mtime, dropbox_mtime):
    print(f"Conflict: {local_path}")
    return True  # True to download, False to skip

result = sync.run(
    conflict_callback=handle_conflict,
    verbose=True
)

# Access results
print(f"Downloaded: {result.downloaded_count}")
print(f"Skipped: {result.skipped_count}")
print(f"Failed: {result.failed_count}")
```

### DropboxAuthHelper

Helper for OAuth2 authentication flow.

```python
from dropbox_backup_lib import DropboxAuthHelper

# Create from environment
helper = DropboxAuthHelper.from_env()

# Get authorization URL
url = helper.get_authorization_url()
print(f"Visit: {url}")

# Get auth code from user
auth_code = input("Enter code: ")

# Get refresh token
refresh_token = helper.finish_auth(auth_code)

# Save to .env file
from pathlib import Path
helper.update_env_file(Path('.env'), refresh_token)
```

## Examples

### Basic Backup Script

Create a file `backup.py`:

```python
#!/usr/bin/env python3
from pathlib import Path
from dotenv import load_dotenv
from dropbox_backup_lib import BackupConfig, DropboxBackup

load_dotenv()

config = BackupConfig.from_env(
    source_dir=Path(__file__).parent,
    dropbox_path='/backups/my-project'
)

backup = DropboxBackup(config)
result = backup.run()

exit(0 if result.failed_count == 0 else 1)
```

Make it executable:
```bash
chmod +x backup.py
./backup.py
```

### Custom Progress Tracking

```python
from dropbox_backup_lib import DropboxBackup, BackupConfig

def progress_handler(file_path, success, message):
    if success:
        print(f"✓ Uploaded: {file_path}")
    else:
        print(f"✗ Failed: {message}")
        # Could log to file, send notification, etc.

config = BackupConfig.from_env(...)
backup = DropboxBackup(config)
result = backup.run(progress_callback=progress_handler)
```

### Backup Only Specific File Types

```python
config = BackupConfig.from_env(
    source_dir=Path.cwd(),
    dropbox_path='/backups/documents',
    backup_patterns=[
        '*.pdf',
        '*.docx',
        '*.xlsx',
        'reports/**/*.md',
    ],
    exclude_patterns=[
        '~$*',  # Temporary Office files
        '.git/*',
    ]
)
```

### Basic Sync from Dropbox

```python
from dropbox_backup_lib import DropboxSync, SyncConfig

# Sync all files from Dropbox to local
config = SyncConfig.from_env(
    dropbox_path='/backups/my-project',
    local_dir=Path.cwd() / 'local-copy'
)

sync = DropboxSync(config)
result = sync.run()
```

### Pattern-Based Sync

```python
from dropbox_backup_lib import DropboxSync, SyncConfig, ConflictResolution

# Sync only specific file types
config = SyncConfig.from_env(
    dropbox_path='/backups/my-project',
    local_dir=Path.cwd() / 'synced-files',
    sync_patterns=['*.md', '*.py', '*.xlsx'],  # Only these types
    exclude_patterns=['draft_*', '*.tmp'],
    conflict_resolution=ConflictResolution.TIMESTAMP
)

sync = DropboxSync(config)
result = sync.run()
```

### Interactive Conflict Resolution

```python
from dropbox_backup_lib import DropboxSync, SyncConfig, ConflictResolution

# Prompt user for conflicts
config = SyncConfig.from_env(
    dropbox_path='/backups/my-project',
    local_dir=Path.cwd(),
    conflict_resolution=ConflictResolution.PROMPT
)

def handle_conflict(local_path, local_mtime, dropbox_mtime):
    """Custom conflict handler"""
    print(f"\nConflict: {local_path}")
    print(f"  Local:   {local_mtime}")
    print(f"  Dropbox: {dropbox_mtime}")
    choice = input("  Download? (y/n): ")
    return choice.lower() == 'y'

sync = DropboxSync(config)
result = sync.run(conflict_callback=handle_conflict)
```

### Bidirectional Sync

```python
from pathlib import Path
from dropbox_backup_lib import (
    BidirectionalSync,
    BidirectionalSyncConfig,
    ConflictStrategy,
    SyncDirection
)

# Configure bidirectional sync
config = BidirectionalSyncConfig.from_env(
    local_dir=Path.cwd(),
    dropbox_path='/backups/my-project',
    conflict_strategy=ConflictStrategy.NEWER_WINS
)

# Custom conflict handler (optional)
def handle_conflict(path, local_mtime, remote_mtime):
    """Custom handler for when both sides changed"""
    print(f"Conflict: {path}")
    print(f"  Local: {local_mtime}")
    print(f"  Remote: {remote_mtime}")
    choice = input("Choose (l)ocal, (r)emote, or (s)kip: ")
    if choice == 'l':
        return SyncDirection.UPLOAD
    elif choice == 'r':
        return SyncDirection.DOWNLOAD
    return SyncDirection.SKIP

# Run sync
sync = BidirectionalSync(config)
result = sync.run(
    conflict_callback=handle_conflict,  # Only used for PROMPT strategy
    dry_run=False  # Set True to preview without changes
)

print(f"Uploaded: {len(result.uploaded)}")
print(f"Downloaded: {len(result.downloaded)}")
```

## File Patterns

The library uses glob patterns for file matching:

- `*.py` - All Python files in the source directory
- `**/*.py` - All Python files recursively
- `data/*.csv` - CSV files in the data directory
- `docs/**/*` - All files in docs and subdirectories

Default exclusion patterns:
- `~$*` - Temporary files
- `.git/*` - Git repository
- `.claude/*` - Claude Code files
- `*.pyc` - Python bytecode
- `__pycache__` - Python cache
- `.env*` - Environment files
- `venv/*` - Virtual environment
- `node_modules/*` - Node modules

## Security Best Practices

1. **Never commit credentials**: Always use `.env` files and add them to `.gitignore`
2. **Use refresh tokens**: They're more secure than access tokens and don't expire
3. **Limit app permissions**: Only enable the Dropbox permissions you need
4. **Exclude sensitive files**: Always exclude `.env*`, `*.pem`, `*.key`, etc.
5. **Review backup contents**: Check what's being uploaded before automating

## Automation

### Cron (Linux/macOS)

```bash
# Edit crontab
crontab -e

# Add daily backup at 9 AM
0 9 * * * cd /path/to/project && /usr/bin/python3 backup.py >> backup.log 2>&1
```

### Task Scheduler (Windows)

1. Open Task Scheduler
2. Create Basic Task
3. Set trigger: Daily at 9 AM
4. Set action: Run `python.exe` with argument `backup.py`

### GitHub Actions

Create `.github/workflows/backup.yml`:

```yaml
name: Backup to Dropbox
on:
  schedule:
    - cron: '0 9 * * *'  # Daily at 9 AM UTC
  workflow_dispatch:  # Allow manual trigger

jobs:
  backup:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - run: pip install dropbox-backup-lib python-dotenv
      - run: python backup.py
        env:
          DROPBOX_APP_KEY: ${{ secrets.DROPBOX_APP_KEY }}
          DROPBOX_APP_SECRET: ${{ secrets.DROPBOX_APP_SECRET }}
          DROPBOX_REFRESH_TOKEN: ${{ secrets.DROPBOX_REFRESH_TOKEN }}
```

## Troubleshooting

### "Missing required environment variables"
- Make sure `.env` file exists and contains all three variables
- Run `dropbox-get-token` to generate the refresh token

### "Failed to connect to Dropbox"
- Check your internet connection
- Verify your credentials are correct
- Make sure your app has the required permissions

### "Permission denied" or "Invalid access token"
- Go to your Dropbox app settings
- Check that permissions are enabled
- Try generating a new refresh token

### Files not uploading
- Check that files match your backup patterns
- Verify files aren't excluded by exclusion patterns
- Ensure you have space in your Dropbox account

## Development

### Running Tests

```bash
pip install -e ".[dev]"
pytest
```

### Code Formatting

```bash
black dropbox_backup_lib/
```

### Type Checking

```bash
mypy dropbox_backup_lib/
```

## License

MIT License - see LICENSE file for details

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.