hiett commited on
Commit
d976565
1 Parent(s): 48432f0

Rename new readme

Browse files
Files changed (2) hide show
  1. README-UPDATED.md +0 -145
  2. README.md +116 -81
README-UPDATED.md DELETED
@@ -1,145 +0,0 @@
1
- # Serverless Redis HTTP (SRH)
2
-
3
- A Redis proxy and connection pooler that uses HTTP rather than the Redis binary protocol.\
4
- The aim of this project is to be entirely compatible with Upstash, and work with any Upstash supported Redis version.
5
-
6
- Use cases for SRH:
7
- - For usage in your CI pipelines, creating Upstash databases is tedious, or you have lots of parallel runs.
8
- - See [Using in GitHub Actions](#in-github-actions) on how to quickly get SRH setup for this context.
9
- - For usage inside of Kubernetes, or any network whereby the Redis server is not exposed to the internet.
10
- - See [Using in Docker Compose](#via-docker-compose) for the various setup options directly using the Docker Container.
11
- - For local development environments, where you have a local Redis server running, or require offline access.
12
- - See [Using the Docker Command](#via-docker-command), or [Using Docker Compose](#via-docker-compose).
13
-
14
- ## Differences between Upstash and Redis to note
15
- SRH tests are ran nightly against the `@upstash/redis` JavaScript package. However, there are some minor differences between Upstash's implementation of Redis and the official Redis code.
16
-
17
- - The `UNLINK` command will not throw an error when 0 keys are given to it. In Redis, and as such SRH, an error will be thrown.
18
- - In the `ZRANGE` command, in Upstash you are not required to provide `BYSCORE` or `BYLEX` in order to use the `LIMIT` argument. With Redis/SRH, this will throw an error if not provided.
19
- - The Upstash implementation of `RedisJSON` contains a number of subtle differences in what is returned in responses. For this reason, **it is not advisable to use SRH with Redis Stack if you are testing your Upstash implementation that uses JSON commands**. If you don't use any JSON commands, then all is good :)
20
- - **SRH does not implement commands via paths, or accepting the token via a query param**. Only the body method is implemented, which the `@upstash/redis` SDK uses.
21
-
22
- ### Similarities to note:
23
-
24
- Pipelines and Transaction endpoints are also implemented, also using the body data only. You can read more about the RestAPI here: [Upstash Docs on the Rest API](https://docs.upstash.com/redis/features/restapi)
25
-
26
- Response encoding is also fully implemented. This is enabled by default by the `@upstash/redis` SDK. You can read more about that here: [Upstash Docs on Hashed Responses](https://docs.upstash.com/redis/sdks/javascriptsdk/troubleshooting#hashed-response)
27
-
28
- ## How to use with the `@upstash/redis` SDK
29
- Simply set the REST URL and token to where the SRH instance is running. For example:
30
- ```ts
31
- import {Redis} from '@upstash/redis';
32
-
33
- export const redis = new Redis({
34
- url: "http://localhost:8079",
35
- token: "example_token",
36
- });
37
- ```
38
-
39
- # Setting up SRH
40
- ## Via Docker command
41
- If you have a locally running Redis server, you can simply start an SRH container that connects to it.
42
- In this example, SRH will be running on port `8080`.
43
-
44
- ```bash
45
- docker run \
46
- -it -d -p 8080:80 --name srh \
47
- -e SRH_MODE=env \
48
- -e SRH_TOKEN=your_token_here \
49
- -e SRH_CONNECTION_STRING="redis://your_server_here:6379" \
50
- hiett/serverless-redis-http:latest
51
- ```
52
-
53
- ## Via Docker Compose
54
- If you wish to run in Kubernetes, this should contain all the basics would need to set that up. However, be sure to read the Configuration Options, because you can create a setup whereby multiple Redis servers are proxied.
55
- ```yml
56
- version: '3'
57
- services:
58
- redis:
59
- image: redis
60
- ports:
61
- - '6379:6379'
62
- serverless-redis-http:
63
- ports:
64
- - '8079:80'
65
- image: hiett/serverless-redis-http:latest
66
- environment:
67
- SRH_MODE: env
68
- SRH_TOKEN: example_token
69
- SRH_CONNECTION_STRING: 'redis://redis:6379' # Using `redis` hostname since they're in the same Docker network.
70
- ```
71
-
72
- ## In GitHub Actions
73
-
74
- SRH works nicely in GitHub Actions because you can run it as a container in a job's services. Simply start a Redis server, and then
75
- SRH alongside it. You don't need to worry about a race condition of the Redis instance not being ready, because SRH doesn't create a Redis connection until the first command comes in.
76
-
77
- ```yml
78
- name: Test @upstash/redis compatability
79
- on:
80
- push:
81
- workflow_dispatch:
82
-
83
- env:
84
- SRH_TOKEN: example_token
85
-
86
- jobs:
87
- container-job:
88
- runs-on: ubuntu-latest
89
- container: denoland/deno
90
- services:
91
- redis:
92
- image: redis/redis-stack-server:6.2.6-v6 # 6.2 is the Upstash compatible Redis version
93
- srh:
94
- image: hiett/serverless-redis-http:latest
95
- env:
96
- SRH_MODE: env # We are using env mode because we are only connecting to one server.
97
- SRH_TOKEN: ${{ env.SRH_TOKEN }}
98
- SRH_CONNECTION_STRING: redis://redis:6379
99
-
100
- steps:
101
- # You can place your normal testing steps here. In this example, we are running SRH against the upstash/upstash-redis test suite.
102
- - name: Checkout code
103
- uses: actions/checkout@v3
104
- with:
105
- repository: upstash/upstash-redis
106
-
107
- - name: Run @upstash/redis Test Suite
108
- run: deno test -A ./pkg
109
- env:
110
- UPSTASH_REDIS_REST_URL: http://srh:80
111
- UPSTASH_REDIS_REST_TOKEN: ${{ env.SRH_TOKEN }}
112
- ```
113
-
114
- # Configuration Options
115
-
116
- SRH works with multiple Redis servers, and can pool however many connections you wish it to. It will shut down un-used pools after 15 minutes of inactivity. Upon the next command, it will re-build the pool.
117
-
118
- ## Connecting to multiple Redis servers at the same time
119
-
120
- The examples above use environment variables in order to tell SRH which Redis server to connect to. However, you can also use a configuration JSON file, which lets you create as many connections as you wish. The token provided in each request will decide which pool is used.
121
-
122
- Create a JSON file, in this example called `tokens.json`:
123
- ```json
124
- {
125
- "example_token": {
126
- "srh_id": "some_unique_identifier",
127
- "connection_string": "redis://localhost:6379",
128
- "max_connections": 3
129
- }
130
- }
131
- ```
132
- You can provide as many entries to the base object as you wish, and configure the number of max connections per pool. The `srh_id` is used internally to keep track of instances. It can be anything you want.
133
-
134
- Once you have created this, mount it to the docker container to the `/app/srh-config/tokens.json` file. Here is an example docker command:
135
-
136
- `docker run -it -d -p 8079:80 --name srh --mount type=bind,source=$(pwd)/tokens.json,target=/app/srh-config/tokens.json hiett/serverless-redis-http:latest`
137
-
138
- ## Environment Variables
139
-
140
- | Name | Default Value | Notes |
141
- | ---- | ------------- | ----- |
142
- | SRH_MODE | `file` | Can be `env` or `file`. If `file`, see [Connecting to multiple Redis servers](#connecting-to-multiple-redis-servers-at-the-same-time). If set to `env`, you are required to provide the following environment variables: |
143
- | SRH_TOKEN | `<required if SRH_MODE = env>` | Set the token that the Rest API will require |
144
- | SRH_CONNECTION_STRING | `<required if SRH_MODE = env>` | Sets the connection string to the Redis server. |
145
- | SRH_MAX_CONNECTIONS | `3` | Only used if `SRH_MODE=env`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
README.md CHANGED
@@ -1,25 +1,33 @@
1
- # SRH: Serverless Redis HTTP
2
 
3
- ---
 
4
 
5
- **TLDR: If you want to run a local Upstash-compatible HTTP layer in front of your Redis:**
 
 
 
 
 
 
6
 
7
- 0) Have a locally running Redis instance - in this example bound to the default port 6379
8
- 1) create a json file called tokens.json in a folder called srh-config (`srh-config/tokens.json`)
9
- 2) paste this in:
10
- ```json
11
- {
12
- "example_token": {
13
- "srh_id": "some_unique_identifier",
14
- "connection_string": "redis://localhost:6379",
15
- "max_connections": 3
16
- }
17
- }
18
- ```
19
- 3) Run this command:
20
- `docker run -it -d -p 8079:80 --name srh --mount type=bind,source=$(pwd)/srh-config/tokens.json,target=/app/srh-config/tokens.json hiett/serverless-redis-http:latest`
21
- 4) Set this as your Upstash configuration
22
- ```js
 
23
  import {Redis} from '@upstash/redis';
24
 
25
  export const redis = new Redis({
@@ -27,84 +35,111 @@ export const redis = new Redis({
27
  token: "example_token",
28
  });
29
  ```
30
- ---
31
 
32
- A Redis connection pooler for serverless applications. This allows your serverless functions to talk to Redis via HTTP,
33
- while also not having to worry about the Redis max connection limits.
34
-
35
- The idea is you host this alongside your Redis server, to minimise latency. Your serverless functions can then talk to
36
- this via HTTP.
37
-
38
- ## Features
39
- - Allows you to talk to redis via HTTP
40
- - Pools redis connections
41
- - Automatically kills redis connections after inactivity
42
- - Supports multiple redis instances, and you can configure unique tokens for each
43
- - Fully supports the `@upstash/redis` TypeScript library.
44
- - Works inside of GitHub Action's services, so you can run it alongside Redis in your tests.
45
 
46
- ## Client usage
47
- This will not work with regular Redis clients, as it is over HTTP and not the redis protocol.
48
- However, to try and keep the project as "standardised" as possible, you can use the `@upstash/redis` TypeScript library.
49
- You can read about it here: [Upstash Redis GitHub](https://github.com/upstash/upstash-redis)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
 
51
- Soon I will add specific documentation for the endpoints so you can implement clients in other languages.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
 
53
- ## Installation
54
- You have to options to run this:
55
- - Via docker: `docker pull hiett/serverless-redis-http:latest` [Docker Hub link](https://hub.docker.com/r/hiett/serverless-redis-http)
56
- - Via elixir: `(clone this repo)` -> `mix deps.get` -> `iex -S mix`
57
 
58
- If you are running via Docker, you will need to mount the configuration file to `/app/srh-config/tokens.json`.\
59
- An example of a run command is the following:
60
 
61
- `docker run -it -d -p 8080:80 --name srh --mount type=bind,source=$(pwd)/srh-config/tokens.json,target=/app/srh-config/tokens.json hiett/serverless-redis-http:latest`
62
 
63
- *Note that it is running on port 80*
64
 
65
- To configure Redis targets:\
66
- Create a file: `srh-config/tokens.json`
67
  ```json
68
  {
69
  "example_token": {
70
  "srh_id": "some_unique_identifier",
71
  "connection_string": "redis://localhost:6379",
72
  "max_connections": 3
73
- }
74
  }
75
  ```
 
76
 
77
- Alternatively, if you are connecting to just one Redis server, you can use environment variables to set the target.
78
- ```
79
- SRH_MODE="env"
80
- SRH_TOKEN="example_token"
81
- SRH_CONNECTION_STRING="redis://redis:6379"
82
- ```
83
- If you use this strategy, it's important that you set `SRH_MODE=env`. This is because SRH by default uses the file strategy for backwards-compatability.
84
 
85
- ### Docker Compose
86
- You'll want the above `tokens.json` file but use this as your connection string:
87
- ```json
88
- "connection_string": "redis://redis:6379"
89
- ```
90
- docker-compose.yaml
91
- ```yaml
92
- version: '3'
93
- services:
94
- redis:
95
- image: redis
96
- ports:
97
- - '6379:6379'
98
- serverless-redis-http:
99
- ports:
100
- - '8079:80'
101
- image: hiett/serverless-redis-http:latest
102
- volumes:
103
- - ./path/to/tokens.json:/app/srh-config/tokens.json
104
- ```
105
 
106
- Notes:
107
- - Srh_id can be anything you want, as long as it's a string, and unique.
108
- - `max_connections` is the maximum number of connections for the pool.
109
- - If there is inactivity, the pool will kill these connections. They will only be open while the pool is alive. The pool will re-create these connections when commands come in.
110
- - You can add more redis instances to connect to by adding more tokens and connection configurations. Based on the header in each request, the correct pool/connection info will be used.
 
 
1
+ # Serverless Redis HTTP (SRH)
2
 
3
+ A Redis proxy and connection pooler that uses HTTP rather than the Redis binary protocol.\
4
+ The aim of this project is to be entirely compatible with Upstash, and work with any Upstash supported Redis version.
5
 
6
+ Use cases for SRH:
7
+ - For usage in your CI pipelines, creating Upstash databases is tedious, or you have lots of parallel runs.
8
+ - See [Using in GitHub Actions](#in-github-actions) on how to quickly get SRH setup for this context.
9
+ - For usage inside of Kubernetes, or any network whereby the Redis server is not exposed to the internet.
10
+ - See [Using in Docker Compose](#via-docker-compose) for the various setup options directly using the Docker Container.
11
+ - For local development environments, where you have a local Redis server running, or require offline access.
12
+ - See [Using the Docker Command](#via-docker-command), or [Using Docker Compose](#via-docker-compose).
13
 
14
+ ## Differences between Upstash and Redis to note
15
+ SRH tests are ran nightly against the `@upstash/redis` JavaScript package. However, there are some minor differences between Upstash's implementation of Redis and the official Redis code.
16
+
17
+ - The `UNLINK` command will not throw an error when 0 keys are given to it. In Redis, and as such SRH, an error will be thrown.
18
+ - In the `ZRANGE` command, in Upstash you are not required to provide `BYSCORE` or `BYLEX` in order to use the `LIMIT` argument. With Redis/SRH, this will throw an error if not provided.
19
+ - The Upstash implementation of `RedisJSON` contains a number of subtle differences in what is returned in responses. For this reason, **it is not advisable to use SRH with Redis Stack if you are testing your Upstash implementation that uses JSON commands**. If you don't use any JSON commands, then all is good :)
20
+ - **SRH does not implement commands via paths, or accepting the token via a query param**. Only the body method is implemented, which the `@upstash/redis` SDK uses.
21
+
22
+ ### Similarities to note:
23
+
24
+ Pipelines and Transaction endpoints are also implemented, also using the body data only. You can read more about the RestAPI here: [Upstash Docs on the Rest API](https://docs.upstash.com/redis/features/restapi)
25
+
26
+ Response encoding is also fully implemented. This is enabled by default by the `@upstash/redis` SDK. You can read more about that here: [Upstash Docs on Hashed Responses](https://docs.upstash.com/redis/sdks/javascriptsdk/troubleshooting#hashed-response)
27
+
28
+ ## How to use with the `@upstash/redis` SDK
29
+ Simply set the REST URL and token to where the SRH instance is running. For example:
30
+ ```ts
31
  import {Redis} from '@upstash/redis';
32
 
33
  export const redis = new Redis({
 
35
  token: "example_token",
36
  });
37
  ```
 
38
 
39
+ # Setting up SRH
40
+ ## Via Docker command
41
+ If you have a locally running Redis server, you can simply start an SRH container that connects to it.
42
+ In this example, SRH will be running on port `8080`.
43
+
44
+ ```bash
45
+ docker run \
46
+ -it -d -p 8080:80 --name srh \
47
+ -e SRH_MODE=env \
48
+ -e SRH_TOKEN=your_token_here \
49
+ -e SRH_CONNECTION_STRING="redis://your_server_here:6379" \
50
+ hiett/serverless-redis-http:latest
51
+ ```
52
 
53
+ ## Via Docker Compose
54
+ If you wish to run in Kubernetes, this should contain all the basics would need to set that up. However, be sure to read the Configuration Options, because you can create a setup whereby multiple Redis servers are proxied.
55
+ ```yml
56
+ version: '3'
57
+ services:
58
+ redis:
59
+ image: redis
60
+ ports:
61
+ - '6379:6379'
62
+ serverless-redis-http:
63
+ ports:
64
+ - '8079:80'
65
+ image: hiett/serverless-redis-http:latest
66
+ environment:
67
+ SRH_MODE: env
68
+ SRH_TOKEN: example_token
69
+ SRH_CONNECTION_STRING: 'redis://redis:6379' # Using `redis` hostname since they're in the same Docker network.
70
+ ```
71
 
72
+ ## In GitHub Actions
73
+
74
+ SRH works nicely in GitHub Actions because you can run it as a container in a job's services. Simply start a Redis server, and then
75
+ SRH alongside it. You don't need to worry about a race condition of the Redis instance not being ready, because SRH doesn't create a Redis connection until the first command comes in.
76
+
77
+ ```yml
78
+ name: Test @upstash/redis compatability
79
+ on:
80
+ push:
81
+ workflow_dispatch:
82
+
83
+ env:
84
+ SRH_TOKEN: example_token
85
+
86
+ jobs:
87
+ container-job:
88
+ runs-on: ubuntu-latest
89
+ container: denoland/deno
90
+ services:
91
+ redis:
92
+ image: redis/redis-stack-server:6.2.6-v6 # 6.2 is the Upstash compatible Redis version
93
+ srh:
94
+ image: hiett/serverless-redis-http:latest
95
+ env:
96
+ SRH_MODE: env # We are using env mode because we are only connecting to one server.
97
+ SRH_TOKEN: ${{ env.SRH_TOKEN }}
98
+ SRH_CONNECTION_STRING: redis://redis:6379
99
+
100
+ steps:
101
+ # You can place your normal testing steps here. In this example, we are running SRH against the upstash/upstash-redis test suite.
102
+ - name: Checkout code
103
+ uses: actions/checkout@v3
104
+ with:
105
+ repository: upstash/upstash-redis
106
+
107
+ - name: Run @upstash/redis Test Suite
108
+ run: deno test -A ./pkg
109
+ env:
110
+ UPSTASH_REDIS_REST_URL: http://srh:80
111
+ UPSTASH_REDIS_REST_TOKEN: ${{ env.SRH_TOKEN }}
112
+ ```
113
 
114
+ # Configuration Options
 
 
 
115
 
116
+ SRH works with multiple Redis servers, and can pool however many connections you wish it to. It will shut down un-used pools after 15 minutes of inactivity. Upon the next command, it will re-build the pool.
 
117
 
118
+ ## Connecting to multiple Redis servers at the same time
119
 
120
+ The examples above use environment variables in order to tell SRH which Redis server to connect to. However, you can also use a configuration JSON file, which lets you create as many connections as you wish. The token provided in each request will decide which pool is used.
121
 
122
+ Create a JSON file, in this example called `tokens.json`:
 
123
  ```json
124
  {
125
  "example_token": {
126
  "srh_id": "some_unique_identifier",
127
  "connection_string": "redis://localhost:6379",
128
  "max_connections": 3
129
+ }
130
  }
131
  ```
132
+ You can provide as many entries to the base object as you wish, and configure the number of max connections per pool. The `srh_id` is used internally to keep track of instances. It can be anything you want.
133
 
134
+ Once you have created this, mount it to the docker container to the `/app/srh-config/tokens.json` file. Here is an example docker command:
 
 
 
 
 
 
135
 
136
+ `docker run -it -d -p 8079:80 --name srh --mount type=bind,source=$(pwd)/tokens.json,target=/app/srh-config/tokens.json hiett/serverless-redis-http:latest`
137
+
138
+ ## Environment Variables
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
139
 
140
+ | Name | Default Value | Notes |
141
+ | ---- | ------------- | ----- |
142
+ | SRH_MODE | `file` | Can be `env` or `file`. If `file`, see [Connecting to multiple Redis servers](#connecting-to-multiple-redis-servers-at-the-same-time). If set to `env`, you are required to provide the following environment variables: |
143
+ | SRH_TOKEN | `<required if SRH_MODE = env>` | Set the token that the Rest API will require |
144
+ | SRH_CONNECTION_STRING | `<required if SRH_MODE = env>` | Sets the connection string to the Redis server. |
145
+ | SRH_MAX_CONNECTIONS | `3` | Only used if `SRH_MODE=env`.