syncall 1.8.8
Versatile bi-directional synchronization tool
License: MIT
Requires Python: <=3.12.5,>=3.8.1
Keywords: asana, calendar, gkeep, google calendar, google keep, google tasks, google, notion, productivity, synchronization, task management, taskwarrior, caldav

# syncall

<p align="center">
  <img src="https://raw.githubusercontent.com/bergercookie/syncall/master/misc/meme.png"/>
</p>

<a href="https://github.com/bergercookie/syncall/actions" alt="master">
<img src="https://github.com/bergercookie/syncall/actions/workflows/tests.yml/badge.svg?branch=master" /></a>
<img src="https://github.com/bergercookie/syncall/actions/workflows/linters.yml/badge.svg?branch=master" /></a>
<a href='https://coveralls.io/github/bergercookie/syncall?branch=master'>
<img src='https://coveralls.io/repos/github/bergercookie/syncall/badge.svg?branch=master' alt='Coverage Status' /></a>
<a href="https://github.com/pre-commit/pre-commit">
<img src="https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white" alt="pre-commit"></a>
<a href="https://github.com/bergercookie/syncall/blob/master/LICENSE" alt="LICENSE">
<img src="https://img.shields.io/github/license/bergercookie/syncall.svg" /></a>
<a href="https://pypi.org/project/syncall" alt="PyPI">
<img src="https://img.shields.io/pypi/pyversions/syncall.svg" /></a>
<a href="https://badge.fury.io/py/syncall">
<img src="https://badge.fury.io/py/syncall.svg" alt="PyPI version" height="18"></a>
<a href="https://pepy.tech/project/syncall">
<img alt="Downloads" src="https://pepy.tech/badge/syncall"></a>
<a href="https://github.com/psf/black">
<img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>

## Description

`syncall` is your one-stop software to bi-directionally synchronize and keep in
sync the data from a variety of services. The framework is targeted towards, but
not limited to, the synchronization of note-taking and task management data.
Each synchronization comes with its own executable which handles the
synchronization services/sides at hand.

One of the main goals of `syncall` is to be extendable. Thus it should be easy
to introduce support for either a new service / synchronization side (e.g.,
[`ClickUp`](https://clickup.com/)) or a new synchronization altogether (e.g.,
ClickUp <-> Google Keep) given that you [implement the corresponding
synchronization sides and conversion
methods](docs/implement-a-new-synchronization.md). See also the
[CONTRIBUTING](CONTRIBUTING.md) guide to get started.

At the moment the list of supported synchronizations is the following:

<table style="undefined;table-layout: fixed; width: 823px">
<thead>
  <tr>
    <th></th>
    <th>Description</th>
    <th>Executable</th>
  </tr>
</thead>
<tbody>
  <tr>
    <td><a href="https://github.com/bergercookie/syncall/blob/master/docs/readme-tw-gtasks.md">README</a></td>
    <td> <a href="https://taskwarrior.org/">Taskwarrior</a> ⬄ <a href="https://support.google.com/tasks/answer/7675772">Google Tasks</a></td>
    <td><tt>tw-gtasks-sync</tt></td>
  </tr>
  <tr>
    <td><a href="https://github.com/bergercookie/syncall/blob/master/docs/readme-tw-gcal.md">README</a></td>
    <td> <a href="https://taskwarrior.org/">Taskwarrior</a> ⬄ <a href="https://calendar.google.com/">Google Calendar</a></td>
    <td><tt>tw-gcal-sync</tt></td>
  </tr>
  <tr>
    <td><a href="https://github.com/bergercookie/syncall/blob/master/docs/readme-tw-notion.md">README</a></td>
    <td> <a href="https://taskwarrior.org/">Taskwarrior</a> ⬄ <a href="https://notion.so">Notion Checkboxes</a></td>
    <td><tt>tw-notion-sync</tt></td>
  </tr>
  <tr>
    <td><a href="https://github.com/bergercookie/syncall/blob/master/docs/readme-tw-gkeep.md">README</a></td>
    <td> <a href="https://taskwarrior.org/">Taskwarrior</a> ⬄ <a href="https://www.google.com/keep/">Google Keep Checkboxes</a></td>
    <td><tt>tw-gkeep-sync</tt></td>
  </tr>
  <tr>
    <td><a href="https://github.com/bergercookie/syncall/blob/master/docs/readme-tw-asana.md">README</a></td>
    <td> <a href="https://taskwarrior.org/">Taskwarrior</a> ⬄ <a href="https://www.asana.com">Asana Tasks</a></td>
    <td><tt>tw-asana-sync</tt></td>
  </tr>
  <tr>
    <td><a href="https://github.com/bergercookie/syncall/blob/master/docs/readme-tw-caldav.md">README</a></td>
    <td> <a href="https://taskwarrior.org/">Taskwarrior</a> ⬄ Generic <a href="https://en.wikipedia.org/wiki/CalDAV">Caldav </a> server</td>
    <td><tt>tw-caldav-sync</tt></td>
  </tr>
  <tr>
    <td><a href="https://github.com/bergercookie/syncall/blob/master/docs/readme-fs-gkeep.md">README</a></td>
    <td>  Local Files ⬄  <a href="https://www.google.com/keep/">Google Keep Notes</a></td>
    <td><tt>fs-gkeep-sync</tt></td>
  </tr>
</tbody>
</table>

Each of the above should support _bi-directional_ synchronization between the
said services. This means that on an _addition_, _modification_, or _deletion_
of an item on one side, a corresponding addition, modification or deletion of
the counterpart item will occur on the other side so that the two sides are
eventually in sync. All synchronizations also support conflict resolution
meaning that it can successfully deal with item edits on both sides.

Currently unless the executable at hand specifies otherwise, the following
conflict resolution strategies are available:

<!-- START sniff-and-replace tw_gcal_sync --list-resolution-strategies START -->
<!-- OVERRIDES --no-collapsible --no-markdown OVERRIDES -->

1. MostRecentRS
2. LeastRecentRS
3. AlwaysFirstRS
4. AlwaysSecondRS

<!-- END sniff-and-replace -->

By the end of the run, it should show you a summary of what's been done, like
the following.

```
Google Keep
-----------
        * Items created: 3
        * Items updated: 2
        * Items deleted: 1

Taskwarrior
-----------
        * Items created: 1
        * Items updated: 0
        * Items deleted: 0
```

Refer to the corresponding README from the list above for instructions specific
to the synchronization with that particular service. Before jumping to that
though, please complete the installation instructions below.

## Installation instructions

### Requirements

- Python version >= `3.8`
- For the integrations that require Taskwarrior - ([Installation
  instructions](https://taskwarrior.org/download/)) version `>=2.6` is required.

### Installation Options

You have to specify at least one extra. To do so use the `[]` syntax in pip:

```sh
# for installing integration with google (e.g. Google Keep / Calendar) and Notion
pip3 install syncall[notion,google]
```

Here's some of the available options for installing it:

- From PyPI - e.g., Specify three extras for integrations - Google (Calendar/Tasks), Google Keep, Notion):

  ```sh
  pip3 install --user --upgrade syncall[notion,google,gkeep]
  ```

- From Github - e.g., Specify two extras:

  ```sh
  pip3 install --user "syncall[gkeep,fs] @ git+https://github.com/bergercookie/syncall"
  ```

- Download and install `devel` branch locally - bleeding edge

  ```sh
  git clone https://github.com/bergercookie/syncall
  cd syncall
  git checkout devel
  pip3 install --user --upgrade .[gkeep,fs,google,tw,caldav,asana]
  ```

- Setup using [poetry](https://python-poetry.org/) - handy for local
  development and for isolation of dependencies:

  ```sh
  git clone https://github.com/bergercookie/syncall
  poetry install --all-extras
  # get an interactive shell
  poetry shell

  # now the executables of all the services should be in your PATH for the
  # current shell and you can also edit the source code without further
  # re-installation ...
  ```

### Sample Usage Instructions

Here's the CLI help page for the synchronizations available.

<!-- START sniff-and-replace tw_gtasks_sync --help START -->

<details>
 <summary><tt>tw_gtasks_sync --help</tt></summary>

```
Usage: tw_gtasks_sync [OPTIONS]

  Synchronize lists from your Google Tasks with filters from Taskwarrior.

  The list of TW tasks is determined by a combination of TW tags and a TW
  project while the list in GTasks should be provided by their name. if it
  doesn't exist it will be crated

Options:
  -l, --gtasks-list TEXT          Name of the Google Tasks list to synchronize
                                  (will be created if not there)
  --google-secret FILE            Override the client secret used for the
                                  communication with the Google APIs
  --oauth-port INTEGER            Port to use for OAuth Authentication with
                                  Google Applications
  -t, --taskwarrior-tags TEXT     Taskwarrior tags to synchronize
  -p, --tw-project TEXT           Taskwarrior project to synchronize
  --list-combinations             List the available named TW<->Google Tasks
                                  combinations
  --list-resolution-strategies    List all the available resolution strategies
                                  and exit
  -r, --resolution-strategy [MostRecentRS|LeastRecentRS|AlwaysFirstRS|AlwaysSecondRS]
                                  Resolution strategy to use during conflicts
  -b, --combination TEXT          Name of an already saved TW<->Google Tasks
                                  combination
  -s, --save-as TEXT              Save the given TW<->Google Tasks filters
                                  combination using a specified custom name.
  --prefer-scheduled-date         Prefer using the "scheduled" date field
                                  instead of the "due" date if the former is
                                  available
  -v, --verbose
  --version                       Show the version and exit.
  --help                          Show this message and exit.
```

</details>

<!-- END sniff-and-replace -->
<!-- START sniff-and-replace tw_gcal_sync --help START -->

<details>
 <summary><tt>tw_gcal_sync --help</tt></summary>

```
Usage: tw_gcal_sync [OPTIONS]

  Synchronize calendars from your Google Calendar with filters from
  Taskwarrior.

  The list of TW tasks is determined by a combination of TW tags and a TW
  project while the calendar in GCal should be provided by their name. if it
  doesn't exist it will be crated

Options:
  -c, --gcal-calendar TEXT        Name of the Google Calendar to synchronize
                                  (will be created if not there)
  --google-secret FILE            Override the client secret used for the
                                  communication with the Google APIs
  --oauth-port INTEGER            Port to use for OAuth Authentication with
                                  Google Applications
  -t, --taskwarrior-tags TEXT     Taskwarrior tags to synchronize
  -p, --tw-project TEXT           Taskwarrior project to synchronize
  --list-combinations             List the available named TW<->Google
                                  Calendar combinations
  --list-resolution-strategies    List all the available resolution strategies
                                  and exit
  -r, --resolution-strategy [MostRecentRS|LeastRecentRS|AlwaysFirstRS|AlwaysSecondRS]
                                  Resolution strategy to use during conflicts
  -b, --combination TEXT          Name of an already saved TW<->Google
                                  Calendar combination
  -s, --save-as TEXT              Save the given TW<->Google Calendar filters
                                  combination using a specified custom name.
  --prefer-scheduled-date         Prefer using the "scheduled" date field
                                  instead of the "due" date if the former is
                                  available
  --default-event-duration-mins INTEGER
                                  The default duration of an event that is to
                                  be created on Google Calendar [in minutes]
  -v, --verbose
  --version                       Show the version and exit.
  --help                          Show this message and exit.

```

</details>

<!-- END sniff-and-replace -->
<!-- START sniff-and-replace tw_notion_sync --help START -->

<details>
 <summary><tt>tw_notion_sync --help</tt></summary>

```
Usage: tw_notion_sync [OPTIONS]

  Synchronise filters of TW tasks with the to_do items of Notion pages

  The list of TW tasks is determined by a combination of TW tags and TW
  project while the notion pages should be provided by their URLs.

Options:
  -n, --notion-page TEXT          Page ID of the Notion page to synchronize
  --token, --token-pass-path TEXT
                                  Path in the UNIX password manager to fetch
  -t, --taskwarrior-tags TEXT     Taskwarrior tags to synchronize
  -p, --tw-project TEXT           Taskwarrior project to synchronize
  -r, --resolution-strategy [MostRecentRS|LeastRecentRS|AlwaysFirstRS|AlwaysSecondRS]
                                  Resolution strategy to use during conflicts
  -b, --combination TEXT          Name of an already saved TW<->Notion
                                  combination
  --list-combinations             List the available named TW<->Notion
                                  combinations
  -s, --save-as TEXT              Save the given TW<->Notion filters
                                  combination using a specified custom name.
  -v, --verbose
  --version                       Show the version and exit.
  --help                          Show this message and exit.

```

</details>

<!-- END sniff-and-replace -->
<!-- START sniff-and-replace tw_gkeep_sync --help START -->

<details>
 <summary><tt>tw_gkeep_sync --help</tt></summary>

```
Usage: tw_gkeep_sync [OPTIONS]

  Synchronize Notes from your Google Keep with filters from Taskwarrior.

  The list of TW tasks is determined by a combination of TW tags and a TW
  project while the note in GKeep should be specified using their full name.
  if it doesn't exist it will be created.

  This service will create TaskWarrior tasks with the specified filter for
  each one of the checkboxed items in the specified Google Keep note and will
  create Google Keep items for each one of the tasks in the Taskwarrior
  filter. You have to first "Show checkboxes" in the Google Keep Note in order
  to use it with this service.

Options:
  -k, --gkeep-note TEXT           Full title of the Google Keep Note to
                                  synchronize - Make sure you enable the
                                  checkboxes
  --user, --user-pass-path TEXT   Path in the UNIX password manager to fetch
                                  the Google username from
  --passwd, --passwd-pass-path TEXT
                                  Path in the UNIX password manager to fetch
                                  the Google password from
  --token, --token-pass-path TEXT
                                  Path in the UNIX password manager to fetch
                                  the google keep token from
  -t, --taskwarrior-tags TEXT     Taskwarrior tags to synchronize
  -p, --tw-project TEXT           Taskwarrior project to synchronize
  --list-combinations             List the available named TW<->Google Keep
                                  combinations
  -r, --resolution-strategy [MostRecentRS|LeastRecentRS|AlwaysFirstRS|AlwaysSecondRS]
                                  Resolution strategy to use during conflicts
  -b, --combination TEXT          Name of an already saved TW<->Google Keep
                                  combination
  -s, --save-as TEXT              Save the given TW<->Google Keep filters
                                  combination using a specified custom name.
  -v, --verbose
  --version                       Show the version and exit.
  --help                          Show this message and exit.

```

</details>

<!-- END sniff-and-replace -->
<!-- START sniff-and-replace tw_asana_sync --help START -->

<details>
 <summary><tt>tw_asana_sync --help</tt></summary>

```
Usage: tw_asana_sync [OPTIONS]

Options:
  --token, --token-pass-path TEXT
                                  Path in the UNIX password manager to fetch
  -w, --asana-workspace-gid TEXT  Asana workspace GID used to filter tasks
  -W, --asana-workspace-name TEXT
                                  Asana workspace name used to filter tasks
  --list-asana-workspaces         List the available Asana workspaces
  -t, --taskwarrior-tags TEXT     Taskwarrior tags to synchronize
  -p, --tw-project TEXT           Taskwarrior project to synchronize
  -r, --resolution-strategy [MostRecentRS|LeastRecentRS|AlwaysFirstRS|AlwaysSecondRS]
                                  Resolution strategy to use during conflicts
  -b, --combination TEXT          Name of an already saved TW<->Asana
                                  combination
  --list-combinations             List the available named TW<->Asana
                                  combinations
  -s, --save-as TEXT              Save the given TW<->Asana filters
                                  combination using a specified custom name.
  -v, --verbose
  --version                       Show the version and exit.
  --help                          Show this message and exit.

```

</details>

<!-- END sniff-and-replace -->
<!-- START sniff-and-replace tw_caldav_sync --help START -->

<details>
 <summary><tt>tw_caldav_sync --help</tt></summary>

```
Usage: tw_caldav_sync [OPTIONS]

  Synchronize lists of tasks from your caldav Calendar with filters from
  Taskwarrior.

  The list of TW tasks is determined by a combination of TW tags and a TW
  project. Use `--all` to synchronize all tasks.

  The calendar in Caldav should be provided by their name. If it doesn't exist
  it will be created.

Options:
  --caldav-calendar TEXT          Name of the caldav Calendar to sync (will be
                                  created if not there)
  --caldav-url TEXT               URL where the caldav calendar is hosted at
                                  (including /dav if applicable)
  --caldav-user TEXT              The caldav username for the given caldav
                                  instance
  --caldav-passwd, --caldav-passwd-pass-path TEXT
                                  Path in the UNIX password manager to fetch
                                  the caldav password from
  --caldav-passwd-cmd CMD
                                  A command (which is run in a shell) that
                                  outputs the caldav password on stdout
  --all, --taskwarrior-all-tasks  Sync all taskwarrior tasks [potentially very
                                  slow]
  -t, --taskwarrior-tags TEXT     Taskwarrior tags to synchronize
  -p, --tw-project TEXT           Taskwarrior project to synchronize
  --30-days, --only-modified-last-30-days
                                  Only synchronize Taskwarrior tasks that have
                                  been modified in the last 30 days
  --list-combinations             List the available named TW<->Caldav
                                  combinations
  -r, --resolution-strategy [MostRecentRS|LeastRecentRS|AlwaysFirstRS|AlwaysSecondRS]
                                  Resolution strategy to use during conflicts
  -b, --combination TEXT          Name of an already saved TW<->Caldav
                                  combination
  -s, --save-as TEXT              Save the given TW<->Caldav filters
                                  combination using a specified custom name.
  -v, --verbose
  --version                       Show the version and exit.
  --help                          Show this message and exit.
```

</details>

<!-- END sniff-and-replace -->
<!-- START sniff-and-replace fs_gkeep_sync --help START -->

<details>
 <summary><tt>fs_gkeep_sync --help</tt></summary>

```
Usage: fs_gkeep_sync [OPTIONS]

  Synchronize Notes from your Google Keep with text files in a directory on
  your filesystem.

  You can only synchronize a subset of your Google Keep notes based on a set
  of provided labels and you can specify where to create the files by
  specifying the path to a local directory. If you don't specify Google Keep
  Labels it will synchronize all your Google Keep notes.

  For each Google Keep Note, fs_gkeep_sync will create a corresponding file
  under the specified root directory with a matching name. Any addition,
  deletion and modification of the files on the filesystem will result in the
  corresponding addition, deletion and modification of the corresponding
  Google Keep item. The same holds the other way around.

Options:
  -k, --gkeep-labels TEXT         Google Keep labels whose notes to
                                  synchronize
  -i, --gkeep-ignore-labels TEXT  Google Keep labels whose notes will be
                                  explicitly ignored
  --user, --user-pass-path TEXT   Path in the UNIX password manager to fetch
                                  the Google username from
  --passwd, --passwd-pass-path TEXT
                                  Path in the UNIX password manager to fetch
                                  the Google password from
  --token, --token-pass-path TEXT
                                  Path in the UNIX password manager to fetch
                                  the google keep token from
  --ext, --filename-extension TEXT
                                  Use this extension for locally created files
  --fs, --fs-root TEXT            Directory to consider as root for
                                  synchronization operations
  --list-combinations             List the available named Filesystem<->Google
                                  Keep combinations
  -r, --resolution-strategy [MostRecentRS|LeastRecentRS|AlwaysFirstRS|AlwaysSecondRS]
                                  Resolution strategy to use during conflicts
  -b, --combination TEXT          Name of an already saved Filesystem<->Google
                                  Keep combination
  -s, --save-as TEXT              Save the given Filesystem<->Google Keep
                                  filters combination using a specified custom
                                  name.
  -v, --verbose
  --version                       Show the version and exit.
  --help                          Show this message and exit.

```

</details>

<!-- END sniff-and-replace -->

## Mechanics / Automatic synchronization

To achieve synchronization between two arbitrary services, we use a push-pull
mechanism which is far easier and less troublesome than an automatic
synchronization solution. This means that you have to explicitly call the
executable for the synchronization you want to achieve. If you want to automate
this, consider adding a `cron` job or a `systemd` timer.

```sh
crontab -e
...

# Add the following to sync every 10' - modify the arguments according to your
# preferences and according to the instructions of the corresponding executable
# for example for `tw_gcal_sync`:
#
# See output and potential errors in your system logs (e.g., `/var/log/syslog`)
*/10 * * * * tw_gcal_sync -c "TW Reminders" -t "remindme"
```

## FAQ

<details>
  <summary>How do I reset the synchronization and start it from scratch?</summary>

1. Remove the combination file that corresponds to your synchronization. For
   example, if you're executing synchronization of `Google Keep` with
   `Taskwarrior`, like the following, your combination name is
   `testnote__None__test_tag`.

   ```sh
   tw_gkeep_sync -t test_tag -k "testnote"
   ```

   The executable also mentions the combination name at the end of the run.

   ```
   ...
   14:00:03.41 | INFO      | Flushing data to remote Google Keep...
   14:00:04.32 | SUCCESS   | Sync completed successfully. You can now use the -b/--combination option to refer to this particular combination

     tw_gkeep_sync --combination testnote__None__test_tag
   ```

   For this combination, on Linux, remove
   `~/.config/syncall/testnote__None__test_tag.yaml`.

1. Remove the section for your combination in the `<sideA_sideB_configs.yaml>`
   configuration file under the `~/.config/syncall/` directory.

   This section will have the same name as the combination file deleted in the
   earlier step and will look like this:

   ```yaml
   ---
   testnote__None__test_tag:
     gkeep_note: testnote
     tw_project: null
     tw_tags: !!python/tuple
       - test_tag
   ```

1. Remove the items of one of the sides. Keep e.g. the items from the Google Keep
   note and delete the tasks of the tag/project you are using for
   synchronization.
1. Rerun synchronization from scratch to populate the one side with the items of
   the other side.

</details>

<details>
  <summary>I want to synchronize more than a single (Notion page / TW filter),  (Google Calendar / TW filter) etc. combination.</summary>

See <a
href="https://github.com/bergercookie/syncall/blob/master/docs/combinations.md">combinations.md</a>.

</details>

## Tab Completion

All synchronization executables support tab auto-completion for their options
for `bash`, `zsh` and `fish`. You can find them under `completions/`

## Miscellaneous

- <a href="https://github.com/bergercookie/syncall/blob/master/docs/implement-a-new-synchronization.md">Implement a New Synchronization Service</a>.
- <a href="https://github.com/bergercookie/syncall/blob/master/docs/combinations.md">Using Multiple Combinations</a>.
- <a href="https://github.com/bergercookie/syncall/blob/master/CONTRIBUTING.md">CONTRIBUTING.md</a>.
- <a href="https://github.com/bergercookie/syncall/blob/master/docs/taskwarrior-filtering.md">Taskwarrior Filtering</a>.

## Self Promotion

If you find this tool useful, please [star it on
Github](https://github.com/bergercookie/syncall)

## TODO List

See [ISSUES list](https://github.com/bergercookie/syncall/issues) for
the things that I'm currently either working on or interested in implementing in
the near future. In case there's something you are interesting in working on,
don't hesitate to either ask for clarifications or just do it and directly make
a PR.