django-project-backup 0.1.8
Add a short description here!
License: mit

# django-project-backup

Django project backup application.

(This is alpha software and is under heavy development)


## Setup

Add the following lines to *settings.py*

```python
# django
INSTALLED_APPS += [
    'django_project_backup'
]

# django-project-backup
SERIALIZATION_MODULES = {
    'dpb_couchdb_datastore': 'django_project_backup.utils.couchdb.serializers'
}

DJANGO_PROJECT_BACKUP_MODE = env('DJANGO_PROJECT_BACKUP_MODE', default='incremental')

DJANGO_PROJECT_BACKUP_PROJECT_NAME = env('DJANGO_PROJECT_BACKUP_PROJECT_NAME', default='django_project_backup')
DJANGO_PROJECT_BACKUP_SHARD_NAME = env('DJANGO_PROJECT_BACKUP_SHARD_NAME', default='1')

# realtime
DJANGO_PROJECT_BACKUP_REALTIME = env.bool('DJANGO_PROJECT_BACKUP_REALTIME', default=True)
DJANGO_PROJECT_BACKUP_DO_REALTIME_COUCHDB_BACKUP = env.bool('DJANGO_PROJECT_BACKUP_DO_REALTIME_COUCHDB_BACKUP',
                                                            default=True)
DJANGO_PROJECT_BACKUP_DO_REALTIME_LOG_BACKUP = env.bool('DJANGO_PROJECT_BACKUP_DO_REALTIME_LOG_BACKUP',
                                                        default=True)

DJANGO_PROJECT_BACKUP_EXCLUDED_MODELS = env.list('DJANGO_PROJECT_BACKUP_EXCLUDED_MODELS', default=[
    'sessions.session',
    'admin.logentry',
    'django_sso_app.passepartout',
    'django_sso_app.device',
    'easy_thumbnails.thumbnail'
])

DJANGO_PROJECT_BACKUP_PUBLIC_ASSETS_FOLDERS = env.list('DJANGO_PROJECT_BACKUP_PUBLIC_ASSETS_FOLDERS',
                                                        default=[str(PUBLIC_ROOT)])

DJANGO_PROJECT_BACKUP_PRIVATE_ASSETS_FOLDERS = env.list('DJANGO_PROJECT_BACKUP_PRIVATE_ASSETS_FOLDERS',
                                                        default=[str(PRIVATE_ROOT)])

DJANGO_PROJECT_BACKUP_DESTINATION_FOLDER = env('DJANGO_PROJECT_BACKUP_DESTINATION_FOLDER',
                                               default=os.path.join(ROOT_DIR, 'backups'))

DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_URL = env('DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_URL', default='http://127.0.0.1:5984')
DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_USER = env('DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_USER', default='admin')
DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_PASSWORD = env('DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_PASSWORD', default='couchdb')
# couchdb db index
DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_DATABASE_NAME = env('DJANGO_PROJECT_BACKUP_COUCHDB_DATASTORE_DATABASE_NAME',
                                                            default='django_project_backup')

DJANGO_PROJECT_BACKUP_DO_FAILSAFE_BACKUP = env.bool('DJANGO_PROJECT_BACKUP_DO_FAILSAFE_BACKUP',
                                                    default=True)
DJANGO_PROJECT_BACKUP_FAILSAFE_BACKUP_PATH = env('DJANGO_PROJECT_BACKUP_FAILSAFE_BACKUP_PATH',
                                                 default=os.path.join(DJANGO_PROJECT_BACKUP_DESTINATION_FOLDER, 'failed'))

# log realtime
LOGGING = {
    ...

    "formatters": {
        "dpb_serialized_model": {
            "format": "%(message)s"
        }
    },
    'handlers': {
        "dpb_realtime_log_backup_file": {
            "level": "INFO",
            "class": "logging.handlers.RotatingFileHandler",
            "filename": os.path.join(ROOT_DIR, "logs", "django_project_backup.realtime.log"),
            "maxBytes": 1024 * 1024 * 4000,  # 4GB
            "backupCount": 10,
            "formatter": "dpb_serialized_model",
        },
    },
    'loggers': {
        ...

        'django_project_backup.backup_logger': {
            'handlers': ['dpb_realtime_log_backup_file'],
            'level': 'INFO',
            'propagate': False,
        },
    },
}

# django-filer
# store files as payload
FILER_DUMP_PAYLOAD = True
```


## Usage

### Filesystem

- Backup DB

    ```
    $ python manage.py dpb_filesystem_dumpdata
    ```

- Backup assets

    ```
    $ python manage.py dpb_filesystem_assets_backup
    ```

### Couchdb

- Backup all

    ```
    $ python manage.py dpb_couchdb_dumpdata
    ```

- Restore all

    ```
    $ python manage.py dpb_couchdb_loaddata
    ```

## Sandbox

### Docker

#### Dependencies
* [Docker](https://docs.docker.com/engine/installation/)
* [Docker Compose](https://docs.docker.com/compose/install/)

### Installation
Run the following commands:

```bash
git clone https://bitbucket.org/pai/django-project-backup.git
cd django-project-backup
# build containers
docker-compose up --build -d
# start containers
docker-compose up
# load initial data
docker-compose run app /venv/bin/python manage.py load_initial_data
# perform first backup
docker-compose run app /venv/bin/python manage.py dbp_couchdb_dumpdata
```

The demo site will now be accessible at [http://localhost:8000/](http://localhost:8000/) and the django admin
interface at [http://localhost:8000/admin/](http://localhost:8000/admin/).

Log into the admin with the credentials ``admin / admin``.

**Important:** This `docker-compose.yml` is configured for local testing only, and is _not_ intended for production use.

### Debugging
To tail the logs from the Docker containers in realtime, run:

```bash
docker-compose logs -f
```

Setup with Virtualenv
---------------------
You can run the demo locally without setting up Docker and simply use Virtualenv, which is the [recommended installation approach](https://docs.djangoproject.com/en/2.2/topics/install/#install-the-django-code) for Django itself.

#### Dependencies
* Python 3.6, 3.7 or 3.8
* [Virtualenv](https://virtualenv.pypa.io/en/stable/installation/)
* [VirtualenvWrapper](https://virtualenvwrapper.readthedocs.io/en/latest/install.html) (optional)

### Installation

With [PIP](https://github.com/pypa/pip) and [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/)
installed, run:

    mkvirtualenv django_project_backup
    python --version

Confirm that this is showing a compatible version of Python 3.x. If not, and you have multiple versions of Python installed on your system, you may need to specify the appropriate version when creating the virtualenv:

    deactivate
    rmvirtualenv django_project_backup
    mkvirtualenv django_project_backup --python=python3.7
    python --version

Now we're ready to set up the sandbox demo project itself:

    cd ~/dev [or your preferred dev directory]
    git clone https://bitbucket.org/pai/django-project-backup.git
    cd django_project_backup/
    pip install -r requirements/base.txt

Next, we'll set up our local environment variables. We use [django-dotenv](https://github.com/jpadilla/django-dotenv)
to help with this. It reads environment variables located in a file name `.env` in the top level directory of the project.
The only variable we need to start is `DJANGO_SETTINGS_MODULE`:

    $ cp backend/settings/local.py.example backend/settings/local.py
    $ echo "DJANGO_SETTINGS_MODULE=backend.settings.local" > .env

To set up your database and load initial data, run the following commands:

    ./manage.py migrate
    ./manage.py load_initial_data
    ./manage.py runserver

Log into the admin with the credentials ``admin / admin``.



## Known issues

Since django-project-backup relies on natural primary keys when in **realtime** mode if one of those fields
is updated there is no efficient way to update related objects keys.



## Note

This project has been set up using PyScaffold 3.2.3. For details and usage
information on PyScaffold see https://pyscaffold.org/.