Package: datasette-insert
datasette-insertDatasette plugin for inserting and updating dataInstallationInstall this plugin in the same environment as Datasette.$ pip install datasette-insertThis plugin should always be deployed with additional configuration to prevent unauthenticated access, see notes below.If you are trying it out on your own local machine, you canpip installthedatasette-insert-unsafeplugin to allow access without needing to set up authentication or permissions separately.Inserting data and creating tablesStart datasette and make sure it has a writable SQLite database attached to it. If you have not yet created a database file you can use this:datasette data.db --createThe--createoption will create a new emptydata.dbdatabase file if it does not already exist.The plugin adds an endpoint that allows data to be inserted or updated and tables to be created by POSTing JSON data to the following URL:/-/insert/name-of-database/name-of-tableThe JSON should look like this:[{"id":1,"name":"Cleopaws","age":5},{"id":2,"name":"Pancakes","age":5}]The first time data is posted to the URL a table of that name will be created if it does not aready exist, with the desired columns.You can specify which column should be used as the primary key using the?pk=URL argument.Here's how to POST to a database and create a new table using the Pythonrequestslibrary:importrequestsrequests.post("http://localhost:8001/-/insert/data/dogs?pk=id",json=[{"id":1,"name":"Cleopaws","age":5},{"id":2,"name":"Pancakes","age":4}])And here's how to do the same thing usingcurl:curl --request POST \
  --data '[
      {
        "id": 1,
        "name": "Cleopaws",
        "age": 5
      },
      {
        "id": 2,
        "name": "Pancakes",
        "age": 4
      }
    ]' \
    'http://localhost:8001/-/insert/data/dogs?pk=id'Or by piping in JSON like so:cat dogs.json | curl --request POST -d @- \
    'http://localhost:8001/-/insert/data/dogs?pk=id'Inserting a single rowIf you are inserting a single row you can optionally send it as a dictionary rather than a list with a single item:curl --request POST \
  --data '{
      "id": 1,
      "name": "Cleopaws",
      "age": 5
    }' \
    'http://localhost:8001/-/insert/data/dogs?pk=id'Automatically adding new columnsIf you send data to an existing table with keys that are not reflected by the existing columns, you will get an HTTP 400 error with a JSON response like this:{"status":400,"error":"Unknown keys: 'foo'","error_code":"unknown_keys"}If you add?alter=1to the URL you are posting to any missing columns will be automatically added:curl --request POST \
  --data '[
      {
        "id": 3,
        "name": "Boris",
        "age": 1,
        "breed": "Husky"
      }
    ]' \
    'http://localhost:8001/-/insert/data/dogs?alter=1'Upserting dataAn "upsert" operation can be used to partially update a record. With upserts you can send a subset of the keys and, if the ID matches the specified primary key, they will be used to update an existing record.Upserts can be sent to the/-/upsertAPI endpoint.This example will update the dog with ID=1's age from 5 to 7:curl --request POST \
  --data '{
      "id": 1,
      "age": 7
    }' \
    'http://localhost:3322/-/upsert/data/dogs?pk=id'Like the/-/insertendpoint, the/-/upsertendpoint can accept an array of objects too. It also supports the?alter=1option.Permissions and authenticationThis plugin defaults to denying all access, to help ensure people don't accidentally deploy it on the open internet in an unsafe configuration.You can read aboutDatasette's approach to authenticationin the Datasette manual.You can install thedatasette-insert-unsafeplugin to run in unsafe mode, where all access is allowed by default.I recommend using this plugin in conjunction withdatasette-auth-tokens, which provides a mechanism for making authenticated calls using API tokens.You can then use"allow" blocksin thedatasette-insertplugin configuration to specify which authenticated tokens are allowed to make use of the API.Here's an examplemetadata.jsonfile which restricts access to the/-/insertAPI to an API token defined in anINSERT_TOKENenvironment variable:{"plugins":{"datasette-insert":{"allow":{"bot":"insert-bot"}},"datasette-auth-tokens":{"tokens":[{"token":{"$env":"INSERT_TOKEN"},"actor":{"bot":"insert-bot"}}]}}}With this configuration in place you can start Datasette like this:INSERT_TOKEN=abc123 datasette data.db -m metadata.jsonYou can now send data to the API usingcurllike this:curl --request POST \
  -H "Authorization: Bearer abc123" \
  --data '[
      {
        "id": 3,
        "name": "Boris",
        "age": 1,
        "breed": "Husky"
      }
    ]' \
    'http://localhost:8001/-/insert/data/dogs'Or using the Pythonrequestslibrary like so:requests.post("http://localhost:8001/-/insert/data/dogs",json={"id":1,"name":"Cleopaws","age":5},headers={"Authorization":"bearer abc123"},)Finely grained permissionsUsing an"allow"block as described above grants full permission to the features enabled by the API.The API implements several new Datasett permissions, which other plugins can use to make more finely grained decisions.The full set of permissions are as follows:insert:all- all permissions - this is used by the"allow"block described above. Argument:database_nameinsert:insert-update- the ability to insert data into an existing table, or to update data by its primary key. Arguments:(database_name, table_name)insert:create-table- the ability to create a new table. Argument:database_nameinsert:alter-table- the ability to add columns to an existing table (using?alter=1). Arguments:(database_name, table_name)You can use plugins likedatasette-permissions-sqlto hook into these more detailed permissions for finely grained control over what actions each authenticated actor can take.Plugins that implement thepermission_allowed()plugin hook can take full control over these permission decisions.CORSIf you start Datasette with thedatasette --corsoption the following HTTP headers will be added to resources served by this plugin:Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: content-type,authorization
Access-Control-Allow-Methods: POSTDevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cd datasette-insert
python3 -m venv venv
source venv/bin/activateNow install the dependencies and tests:pip install -e '.[test]'To run the tests:pytest