Repository: ClickHouse/ClickBench
Pull request #318: Update Hydra
Author: login: wuputah
Created: 2025-03-05T02:28:25Z | Merged: 2025-03-10T09:47:00Z

This updates the Hydra benchmark to the latest version of the Hydra cloud service.


Diff:
diff --git a/hydra/README.md b/hydra/README.md
index e4e3b9e8e..608f0d0cb 100644
--- a/hydra/README.md
+++ b/hydra/README.md
@@ -1,16 +1,59 @@
 # Hydra
 
-Hydra is an open source data warehouse built on Postgres.
+Hydra is a high-performance Postgres database with serverless
+online analytical processing (OLAP). It is designed for low-latency
+applications built on time series and event data.
 
-* [Homepage](https://hydras.io)
-* [GitHub](https://github.com/HydrasDB/hydra)
+* [Homepage](https://hydra.so)
 
 ## Running the benchmark
 
-The benchmark has been configured for a `c6a.4xlarge` running Ubuntu 22.04 and can be run without attendance.
+The benchmark itself requires `psql` to execute. You will need to install `psql`, e.g.:
 
 ```
-export FQDN=ec2-127-0-0-1.compute-1.amazonaws.com
-scp -i ~/.ssh/aws.pem *.sh *.sql ubuntu@$FQDN:~
-ssh -i ~/.ssh/aws.pem ubuntu@$FQDN ./benchmark.sh
+# Debian, Ubuntu, etc
+apt install postgresql-client
+# MacOS
+brew install postgresql
 ```
+
+Note that either option below will introduce additional latency depending on the location
+from which the benchmark is run.
+
+### Local Instance
+
+In this configuration, you will run a Postgres instance locally and it will connect
+to Hydra's serverless OLAP service. This will use the Hydra CLI which will manage
+a Docker container for you.
+
+1. Install Python, and if desired, your preferred Python management tools, like pipx, uv, etc.
+2. Install Hydra CLI using `pip install hydra-cli` (or via `pipx`, `uv`, etc)
+3. Install Docker.
+4. Sign up for Hydra at https://start.hydra.so/get-started and copy the token to your clipboard
+5. Run `hydra setup` and paste in your token
+6. Run `hydra start` to start the local instance
+7. Run `hydra config` to obtain the connection string and set it `DATABASE_URL` in your local environment,
+   or pass it in when you run the benchmark.
+8. Run the benchmark.
+
+    ```
+    DATABASE_URL="..." ./benchmark.sh
+    ```
+
+9.  Run `hydra stop` to stop the local instance
+
+### Cloud Instance
+
+In this configuration, you will connect to a Postgres instance already running in the cloud
+which is already configured to connect to Hydra's serverless OLAP service.
+
+1. Sign up for Hydra at https://start.hydra.so
+2. Request access and complete onboarding
+3. Create a project
+4. Obtain the connection string and set it as `DATABASE_URL` in your local environment,
+   or pass it in when you run the benchmark.
+5. Run the benchmark.
+
+    ```
+    DATABASE_URL="..." ./benchmark.sh
+    ```
diff --git a/hydra/benchmark.sh b/hydra/benchmark.sh
index 3f52eb7d7..1f76d6671 100755
--- a/hydra/benchmark.sh
+++ b/hydra/benchmark.sh
@@ -2,34 +2,15 @@
 set -euo pipefail
 IFS=$'\n\t'
 
-# sets up PG14, configs, and build deps on ubuntu 22.04
-sudo ./install.sh
+# install `psql` if necessary, e.g.:
+#sudo apt-get update
+#sudo apt-get install -y postgresql-client
 
-# download hits.tsv if we dont already have it
-if [ ! -e hits.tsv ]; then
-    wget --continue 'https://datasets.clickhouse.com/hits_compatible/hits.tsv.gz'
-    gzip -d hits.tsv.gz
-fi
-
-# ensure postgres user can access hits.tsv
-chmod a+rX . hits.tsv
-
-# set up db, user, table; load data.
-sudo -u postgres psql -t -c 'CREATE DATABASE test'
-sudo -u postgres psql test -t < create.sql
-sudo -u postgres psql test -t -c 'TRUNCATE hits'
-sudo -u postgres psql test -t -c '\timing' -c "\\copy hits FROM 'hits.tsv'"
-
-# COPY 99997497
-# Time: 1320024.670 ms (22:00.025)
+# load data
+psql "$DATABASE_URL" -c '\timing on' -t -f create.sql
 
 # run test
-./run.sh 2>&1 | tee log.txt
-
-# disk usage
-sudo du -bcs /var/lib/postgresql/14/main/
-
-# 18979994590
+DATABASE_URL="$DATABASE_URL" ./run.sh 2>&1 | tee log.txt
 
 # parse results for json file
 ./parse.sh < log.txt
diff --git a/hydra/create.sql b/hydra/create.sql
index 28aebe827..83c58d2f1 100644
--- a/hydra/create.sql
+++ b/hydra/create.sql
@@ -1,111 +1,124 @@
-CREATE EXTENSION IF NOT EXISTS columnar;
-CREATE TABLE IF NOT EXISTS hits
-(
-    WatchID BIGINT NOT NULL,
-    JavaEnable SMALLINT NOT NULL,
-    Title TEXT NOT NULL,
-    GoodEvent SMALLINT NOT NULL,
-    EventTime TIMESTAMP NOT NULL,
-    EventDate Date NOT NULL,
-    CounterID INTEGER NOT NULL,
-    ClientIP INTEGER NOT NULL,
-    RegionID INTEGER NOT NULL,
-    UserID BIGINT NOT NULL,
-    CounterClass SMALLINT NOT NULL,
-    OS SMALLINT NOT NULL,
-    UserAgent SMALLINT NOT NULL,
-    URL TEXT NOT NULL,
-    Referer TEXT NOT NULL,
-    IsRefresh SMALLINT NOT NULL,
-    RefererCategoryID SMALLINT NOT NULL,
-    RefererRegionID INTEGER NOT NULL,
-    URLCategoryID SMALLINT NOT NULL,
-    URLRegionID INTEGER NOT NULL,
-    ResolutionWidth SMALLINT NOT NULL,
-    ResolutionHeight SMALLINT NOT NULL,
-    ResolutionDepth SMALLINT NOT NULL,
-    FlashMajor SMALLINT NOT NULL,
-    FlashMinor SMALLINT NOT NULL,
-    FlashMinor2 TEXT NOT NULL,
-    NetMajor SMALLINT NOT NULL,
-    NetMinor SMALLINT NOT NULL,
-    UserAgentMajor SMALLINT NOT NULL,
-    UserAgentMinor VARCHAR(255) NOT NULL,
-    CookieEnable SMALLINT NOT NULL,
-    JavascriptEnable SMALLINT NOT NULL,
-    IsMobile SMALLINT NOT NULL,
-    MobilePhone SMALLINT NOT NULL,
-    MobilePhoneModel TEXT NOT NULL,
-    Params TEXT NOT NULL,
-    IPNetworkID INTEGER NOT NULL,
-    TraficSourceID SMALLINT NOT NULL,
-    SearchEngineID SMALLINT NOT NULL,
-    SearchPhrase TEXT NOT NULL,
-    AdvEngineID SMALLINT NOT NULL,
-    IsArtifical SMALLINT NOT NULL,
-    WindowClientWidth SMALLINT NOT NULL,
-    WindowClientHeight SMALLINT NOT NULL,
-    ClientTimeZone SMALLINT NOT NULL,
-    ClientEventTime TIMESTAMP NOT NULL,
-    SilverlightVersion1 SMALLINT NOT NULL,
-    SilverlightVersion2 SMALLINT NOT NULL,
-    SilverlightVersion3 INTEGER NOT NULL,
-    SilverlightVersion4 SMALLINT NOT NULL,
-    PageCharset TEXT NOT NULL,
-    CodeVersion INTEGER NOT NULL,
-    IsLink SMALLINT NOT NULL,
-    IsDownload SMALLINT NOT NULL,
-    IsNotBounce SMALLINT NOT NULL,
-    FUniqID BIGINT NOT NULL,
-    OriginalURL TEXT NOT NULL,
-    HID INTEGER NOT NULL,
-    IsOldCounter SMALLINT NOT NULL,
-    IsEvent SMALLINT NOT NULL,
-    IsParameter SMALLINT NOT NULL,
-    DontCountHits SMALLINT NOT NULL,
-    WithHash SMALLINT NOT NULL,
-    HitColor CHAR NOT NULL,
-    LocalEventTime TIMESTAMP NOT NULL,
-    Age SMALLINT NOT NULL,
-    Sex SMALLINT NOT NULL,
-    Income SMALLINT NOT NULL,
-    Interests SMALLINT NOT NULL,
-    Robotness SMALLINT NOT NULL,
-    RemoteIP INTEGER NOT NULL,
-    WindowName INTEGER NOT NULL,
-    OpenerName INTEGER NOT NULL,
-    HistoryLength SMALLINT NOT NULL,
-    BrowserLanguage TEXT NOT NULL,
-    BrowserCountry TEXT NOT NULL,
-    SocialNetwork TEXT NOT NULL,
-    SocialAction TEXT NOT NULL,
-    HTTPError SMALLINT NOT NULL,
-    SendTiming INTEGER NOT NULL,
-    DNSTiming INTEGER NOT NULL,
-    ConnectTiming INTEGER NOT NULL,
-    ResponseStartTiming INTEGER NOT NULL,
-    ResponseEndTiming INTEGER NOT NULL,
-    FetchTiming INTEGER NOT NULL,
-    SocialSourceNetworkID SMALLINT NOT NULL,
-    SocialSourcePage TEXT NOT NULL,
-    ParamPrice BIGINT NOT NULL,
-    ParamOrderID TEXT NOT NULL,
-    ParamCurrency TEXT NOT NULL,
-    ParamCurrencyID SMALLINT NOT NULL,
-    OpenstatServiceName TEXT NOT NULL,
-    OpenstatCampaignID TEXT NOT NULL,
-    OpenstatAdID TEXT NOT NULL,
-    OpenstatSourceID TEXT NOT NULL,
-    UTMSource TEXT NOT NULL,
-    UTMMedium TEXT NOT NULL,
-    UTMCampaign TEXT NOT NULL,
-    UTMContent TEXT NOT NULL,
-    UTMTerm TEXT NOT NULL,
-    FromTag TEXT NOT NULL,
-    HasGCLID SMALLINT NOT NULL,
-    RefererHash BIGINT NOT NULL,
-    URLHash BIGINT NOT NULL,
-    CLID INTEGER NOT NULL,
-    PRIMARY KEY (CounterID, EventDate, UserID, EventTime, WatchID)
-)
-USING columnar;
+CREATE EXTENSION IF NOT EXISTS pg_duckdb;
+
+CREATE SCHEMA IF NOT EXISTS clickbench;
+
+DROP VIEW IF EXISTS hits_parquet;
+
+CREATE VIEW hits_parquet AS
+SELECT
+  r['WatchID'] WatchID,
+  r['JavaEnable'] JavaEnable,
+  r['Title'] Title,
+  r['GoodEvent'] GoodEvent,
+  'epoch'::timestamp +(r['EventTime'] || 'second')::interval EventTime,
+  'epoch'::timestamp +(r['EventDate'] || 'day')::interval EventDate,
+  r['CounterID'] CounterID,
+  r['ClientIP'] ClientIP,
+  r['RegionID'] RegionID,
+  r['UserID'] UserID,
+  r['CounterClass'] CounterClass,
+  r['OS'] OS,
+  r['UserAgent'] UserAgent,
+  r['URL'] URL,
+  r['Referer'] Referer,
+  r['IsRefresh'] IsRefresh,
+  r['RefererCategoryID'] RefererCategoryID,
+  r['RefererRegionID'] RefererRegionID,
+  r['URLCategoryID'] URLCategoryID,
+  r['URLRegionID'] URLRegionID,
+  r['ResolutionWidth'] ResolutionWidth,
+  r['ResolutionHeight'] ResolutionHeight,
+  r['ResolutionDepth'] ResolutionDepth,
+  r['FlashMajor'] FlashMajor,
+  r['FlashMinor'] FlashMinor,
+  r['FlashMinor2'] FlashMinor2,
+  r['NetMajor'] NetMajor,
+  r['NetMinor'] NetMinor,
+  r['UserAgentMajor'] UserAgentMajor,
+  r['UserAgentMinor'] UserAgentMinor,
+  r['CookieEnable'] CookieEnable,
+  r['JavascriptEnable'] JavascriptEnable,
+  r['IsMobile'] IsMobile,
+  r['MobilePhone'] MobilePhone,
+  r['MobilePhoneModel'] MobilePhoneModel,
+  r['Params'] Params,
+  r['IPNetworkID'] IPNetworkID,
+  r['TraficSourceID'] TraficSourceID,
+  r['SearchEngineID'] SearchEngineID,
+  r['SearchPhrase'] SearchPhrase,
+  r['AdvEngineID'] AdvEngineID,
+  r['IsArtifical'] IsArtifical,
+  r['WindowClientWidth'] WindowClientWidth,
+  r['WindowClientHeight'] WindowClientHeight,
+  r['ClientTimeZone'] ClientTimeZone,
+  'epoch'::timestamp +(r['ClientEventTime'] || 'second')::interval ClientEventTime,
+  r['SilverlightVersion1'] SilverlightVersion1,
+  r['SilverlightVersion2'] SilverlightVersion2,
+  r['SilverlightVersion3'] SilverlightVersion3,
+  r['SilverlightVersion4'] SilverlightVersion4,
+  r['PageCharset'] PageCharset,
+  r['CodeVersion'] CodeVersion,
+  r['IsLink'] IsLink,
+  r['IsDownload'] IsDownload,
+  r['IsNotBounce'] IsNotBounce,
+  r['FUniqID'] FUniqID,
+  r['OriginalURL'] OriginalURL,
+  r['HID'] HID,
+  r['IsOldCounter'] IsOldCounter,
+  r['IsEvent'] IsEvent,
+  r['IsParameter'] IsParameter,
+  r['DontCountHits'] DontCountHits,
+  r['WithHash'] WithHash,
+  r['HitColor'] HitColor,
+  'epoch'::timestamp +(r['LocalEventTime'] || 'second')::interval LocalEventTime,
+  r['Age'] Age,
+  r['Sex'] Sex,
+  r['Income'] Income,
+  r['Interests'] Interests,
+  r['Robotness'] Robotness,
+  r['RemoteIP'] RemoteIP,
+  r['WindowName'] WindowName,
+  r['OpenerName'] OpenerName,
+  r['HistoryLength'] HistoryLength,
+  r['BrowserLanguage'] BrowserLanguage,
+  r['BrowserCountry'] BrowserCountry,
+  r['SocialNetwork'] SocialNetwork,
+  r['SocialAction'] SocialAction,
+  r['HTTPError'] HTTPError,
+  r['SendTiming'] SendTiming,
+  r['DNSTiming'] DNSTiming,
+  r['ConnectTiming'] ConnectTiming,
+  r['ResponseStartTiming'] ResponseStartTiming,
+  r['ResponseEndTiming'] ResponseEndTiming,
+  r['FetchTiming'] FetchTiming,
+  r['SocialSourceNetworkID'] SocialSourceNetworkID,
+  r['SocialSourcePage'] SocialSourcePage,
+  r['ParamPrice'] ParamPrice,
+  r['ParamOrderID'] ParamOrderID,
+  r['ParamCurrency'] ParamCurrency,
+  r['ParamCurrencyID'] ParamCurrencyID,
+  r['OpenstatServiceName'] OpenstatServiceName,
+  r['OpenstatCampaignID'] OpenstatCampaignID,
+  r['OpenstatAdID'] OpenstatAdID,
+  r['OpenstatSourceID'] OpenstatSourceID,
+  r['UTMSource'] UTMSource,
+  r['UTMMedium'] UTMMedium,
+  r['UTMCampaign'] UTMCampaign,
+  r['UTMContent'] UTMContent,
+  r['UTMTerm'] UTMTerm,
+  r['FromTag'] FromTag,
+  r['HasGCLID'] HasGCLID,
+  r['RefererHash'] RefererHash,
+  r['URLHash'] URLHash,
+  r['CLID'] CLID
+FROM
+  read_parquet('/tmp/hits.parquet') r;
+
+DROP TABLE IF EXISTS clickbench.hits;
+
+CREATE TABLE clickbench.hits
+USING duckdb AS
+SELECT
+  *
+FROM
+  hits_parquet;
diff --git a/hydra/install.sh b/hydra/install.sh
deleted file mode 100755
index 68c836454..000000000
--- a/hydra/install.sh
+++ /dev/null
@@ -1,59 +0,0 @@
-#!/bin/bash
-set -euo pipefail
-IFS=$'\n\t'
-
-config_file="/etc/postgresql/14/main/conf.d/hydra.conf"
-
-write_config_file() {
-  # config for a c6a.4xlarge
-  cat >${config_file} << CONF
-max_worker_processes = 32
-max_parallel_workers = 16
-work_mem = 64MB
-shared_buffers = 8GB
-effective_cache_size = 24GB
-effective_io_concurrency = 100
-hash_mem_multiplier = 8
-shared_preload_libraries = 'columnar'
-columnar.min_parallel_processes = 16
-CONF
-}
-
-if [ ! ${EUID} -eq 0 ]; then
-  echo "Please run as root or with sudo"
-  exit 1
-fi
-
-# already ran this setup file
-if [ -f "$config_file" ]; then
-  # rewrite the file with any tweaks
-  write_config_file
-  service postgresql restart
-  exit 0
-fi
-
-export DEBIAN_FRONTEND=noninteractive
-
-# install pg14 from pgdg, install build deps
-apt-get update
-apt-get install gnupg postgresql-common -y
-sh /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y
-apt-get update
-apt-get upgrade -y
-apt-get install lsb-release gcc make libssl-dev autoconf pkg-config \
-    postgresql-14 postgresql-server-dev-14 libcurl4-gnutls-dev \
-    liblz4-dev libzstd-dev -y
-
-# build and install hydra columnar extension
-if [ ! -d hydra ]; then
-    git clone --depth 1 https://github.com/HydrasDB/hydra.git
-fi
-cd hydra/columnar
-./configure
-make
-make install
-cd ../..
-
-write_config_file
-
-service postgresql restart
diff --git a/hydra/parse.sh b/hydra/parse.sh
index dd9a5b9a1..116ec2425 100755
--- a/hydra/parse.sh
+++ b/hydra/parse.sh
@@ -3,11 +3,11 @@
 # OS X: you need to use the gnu tools from homebrew
 GREP="grep"
 if command -v ggrep >/dev/null 2>&1; then
-    grep="ggrep"
+    GREP="ggrep"
 fi
 SED="sed"
 if command -v gsed >/dev/null 2>&1; then
-    sed="gsed"
+    SED="gsed"
 fi
 
 $GREP -oP 'Time: \d+\.\d+ ms' | $SED -r -e 's/Time: ([0-9]+\.[0-9]+) ms/\1/' |
diff --git a/hydra/results/c6a.4xlarge.json b/hydra/results/c6a.4xlarge.json
deleted file mode 100644
index 13d19b5df..000000000
--- a/hydra/results/c6a.4xlarge.json
+++ /dev/null
@@ -1,59 +0,0 @@
-{
-    "system": "Hydra",
-    "date": "2023-09-19",
-    "machine": "c6a.4xlarge, 500gb gp2",
-    "cluster_size": 1,
-    "comment": "",
-    "tags": [
-        "C",
-        "PostgreSQL compatible",
-        "column-oriented"
-    ],
-    "load_time": 1222,
-    "data_size": 18995669982,
-    "result": [
-[0.381752,0.249001,0.243783],
-[0.261088,0.211013,0.209277],
-[0.6692,0.634666,0.627473],
-[0.545165,0.431892,0.439854],
-[22.4371,21.8547,21.731],
-[34.8689,33.0467,32.5865],
-[0.410038,0.377206,0.376254],
-[0.22464,0.206133,0.204092],
-[27.1582,26.4292,26.5203],
-[30.1541,29.4784,30.2199],
-[3.31616,2.76618,2.75128],
-[3.55764,3.28978,3.27316],
-[7.74401,7.64012,7.6384],
-[10.703,10.2599,10.2098],
-[8.70712,8.33702,8.29268],
-[17.9937,17.5084,17.5291],
-[22.8406,22.2649,22.1176],
-[4.08958,3.59194,3.61345],
-[39.0156,37.7258,37.8926],
-[0.267125,0.233304,0.233594],
-[3.12914,2.74305,2.72563],
-[2.91522,2.89177,2.90416],
-[3.76881,3.62139,3.62348],
-[26.322,20.5805,20.3337],
-[1.69125,1.05602,1.06068],
-[1.06135,0.828339,0.831532],
-[1.26898,1.06035,1.05212],
-[4.17896,4.03193,4.03657],
-[43.0503,42.5091,42.3298],
-[7.89438,7.50181,7.39528],
-[9.33451,8.76728,8.79973],
-[15.0403,14.6597,14.3034],
-[58.8221,58.6056,58.6859],
-[34.2914,35.0405,34.2519],
-[36.3141,35.6205,36.5728],
-[14.2056,13.8882,13.8284],
-[0.367202,0.33666,0.33517],
-[0.22008,0.172653,0.167434],
-[0.15307,0.126572,0.126678],
-[1.11688,0.671288,0.696326],
-[0.160667,0.145181,0.141243],
-[0.137686,0.122734,0.118493],
-[0.197786,0.123078,0.122094]
-    ]
-}
diff --git a/hydra/results/hydra.json b/hydra/results/hydra.json
new file mode 100644
index 000000000..c1e56b399
--- /dev/null
+++ b/hydra/results/hydra.json
@@ -0,0 +1,58 @@
+{
+    "system": "Hydra",
+    "date": "2025-03-04",
+    "machine": "XL",
+    "cluster_size": "serverless",
+    "tags": [
+        "managed",
+        "PostgreSQL compatible",
+        "column-oriented"
+    ],
+    "load_time": 136,
+    "data_size": 30816390348,
+    "result": [
+        [0.192382,0.031411,0.0182],
+        [0.277037,0.017238,0.016364],
+        [0.245723,0.038173,0.037376],
+        [0.306797,0.042102,0.042718],
+        [0.418011,0.242568,0.23039],
+        [0.551909,0.282084,0.278836],
+        [0.180043,0.02662,0.025371],
+        [0.165209,0.024473,0.023097],
+        [0.453139,0.284364,0.283832],
+        [0.535552,0.393579,0.39756],
+        [0.30116,0.092113,0.091926],
+        [0.257973,0.100089,0.098468],
+        [0.398887,0.26349,0.253034],
+        [0.578082,0.426437,0.425838],
+        [0.434567,0.263478,0.266679],
+        [0.379118,0.262621,0.264961],
+        [0.729468,0.564334,0.562802],
+        [0.602923,0.464127,0.465615],
+        [1.33815,0.996799,0.993801],
+        [0.173591,0.013425,0.013638],
+        [1.68838,0.515917,0.516346],
+        [0.437349,0.294343,0.296138],
+        [1.99367,0.417726,0.417281],
+        [8.64109,1.77034,1.77321],
+        [0.264145,0.104584,0.105626],
+        [0.238236,0.095826,0.095679],
+        [0.287271,0.112243,0.108904],
+        [0.755408,0.612582,0.61124],
+        [6.75692,6.60809,6.60236],
+        [0.23476,0.07472,0.067433],
+        [0.382783,0.239876,0.239216],
+        [0.438111,0.296216,0.288612],
+        [1.5727,1.69792,1.40727],
+        [1.55361,1.28201,1.28065],
+        [1.49317,1.40405,1.38793],
+        [0.579766,0.41326,0.413215],
+        [0.226025,0.062452,0.059734],
+        [0.195866,0.024387,0.041084],
+        [0.198459,0.025349,0.026228],
+        [0.323418,0.091749,0.088929],
+        [0.172858,0.019531,0.019639],
+        [0.172619,0.019008,0.022273],
+        [0.194027,0.027197,0.024702]
+    ]
+}
diff --git a/hydra/run.sh b/hydra/run.sh
index d6b36853b..b8839776a 100755
--- a/hydra/run.sh
+++ b/hydra/run.sh
@@ -3,11 +3,10 @@
 TRIES=3
 
 cat queries.sql | while read -r query; do
-    sync
-    echo 3 | sudo tee /proc/sys/vm/drop_caches
-
-    echo "$query";
-    for i in $(seq 1 $TRIES); do
-        sudo -u postgres psql test -t -c '\timing' -c "$query" | grep 'Time'
-    done;
-done;
+    echo "$query"
+    (
+	    echo "set search_path=clickbench;"
+        echo '\timing on'
+        yes "$query" | head -n $TRIES
+    ) | psql $DATABASE_URL | grep 'Time'
+done


Reviews:
user: login: rschu1ze
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T09:26:40Z
user: login: wuputah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T18:53:33Z
user: login: wuputah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T18:55:34Z
user: login: wuputah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T18:57:33Z
user: login: rschu1ze
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-09T11:00:45Z
user: login: wuputah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-09T15:56:01Z
user: login: rschu1ze
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-09T17:03:22Z
user: login: wuputah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-09T21:04:13Z
user: login: rschu1ze
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-10T09:46:55Z

Review comments:
diff_hunk: @@ -2,34 +2,11 @@
 set -euo pipefail
 IFS=$'\n\t'
 
-# sets up PG14, configs, and build deps on ubuntu 22.04
-sudo ./install.sh
-
-# download hits.tsv if we dont already have it
-if [ ! -e hits.tsv ]; then
-    wget --no-verbose --continue 'https://datasets.clickhouse.com/hits_compatible/hits.tsv.gz'
-    gzip -d hits.tsv.gz
-fi
-
-# ensure postgres user can access hits.tsv
-chmod a+rX . hits.tsv
-
-# set up db, user, table; load data.
-sudo -u postgres psql -t -c 'CREATE DATABASE test'
-sudo -u postgres psql test -t < create.sql
-sudo -u postgres psql test -t -c 'TRUNCATE hits'
-sudo -u postgres psql test -t -c '\timing' -c "\\copy hits FROM 'hits.tsv'"
-
-# COPY 99997497
-# Time: 1320024.670 ms (22:00.025)
+# load data
path: hydra/benchmark.sh
user: login: rschu1ze
body: For this to work, the postgres client needs to be installed. Please grep `apt install` in the ClickBench repository for other databases which install stuff.
created_at: 2025-03-05T09:24:43Z
author_association: MEMBER
diff_hunk: @@ -432,7 +432,7 @@
 ,{"system":"GlareDB","date":"2024-02-02","machine":"c6a.metal, 500gb gp2","cluster_size":1,"comment":"","tags":["Rust","serverless"],"load_time":0,"data_size":14779976446,"result":[[0.213,0.070,0.071],[4.066,4.060,3.978],[3.994,3.812,4.046],[4.287,4.122,4.218],[6.016,6.482,6.210],[5.260,5.308,5.242],[3.968,4.054,4.054],[4.013,3.961,4.045],[7.124,6.742,6.934],[5.632,5.625,5.433],[4.546,4.463,4.577],[4.540,4.616,4.675],[5.438,5.302,5.243],[6.286,6.115,6.202],[5.460,5.259,5.531],[6.064,6.451,6.458],[7.461,7.655,7.337],[6.920,7.538,7.684],[11.688,11.362,11.378],[4.264,4.085,4.092],[12.973,4.585,4.449],[14.739,4.562,4.676],[25.511,5.088,5.323],[59.374,8.895,8.542],[6.076,4.352,4.324],[4.305,4.233,4.334],[6.083,4.450,4.398],[12.983,4.812,4.721],[11.651,5.245,5.281],[4.275,4.214,4.175],[5.561,5.378,5.630],[9.220,5.983,6.112],[17.147,15.575,15.934],[14.016,9.356,8.974],[14.274,9.454,9.371],[6.675,6.473,6.510],[13.348,4.684,4.587],[12.473,4.655,4.580],[13.153,4.471,4.637],[21.939,4.932,4.951],[6.033,4.320,4.282],[5.181,4.242,4.286],[4.503,4.312,4.215]],"source":"glaredb/results/c6a.metal.json"}
 ,{"system":"Greenplum","date":"2022-08-18","machine":"c6a.4xlarge, 500gb gp2","cluster_size":1,"comment":"","tags":["C","column-oriented","PostgreSQL compatible"],"load_time":1080,"data_size":42000000000,"result":[[2.19644,1.27581,1.07334],[1.36821,1.09289,1.00081],[1.84455,1.93433,1.9734],[1.38533,1.07306,0.997511],[10.4301,10.1377,10.0064],[3.24183,3.67479,3.23526],[1.81177,1.22447,1.72696],[1.28568,1.12341,1.02112],[3.5159,3.7946,3.51106],[5.78795,5.90487,5.95635],[1.50773,1.80496,1.26395],[1.68682,1.93425,1.54081],[2.29378,2.75715,2.22458],[3.41812,3.09848,3.0303],[2.96859,2.96836,2.9949],[3.04409,3.00935,2.98848],[4.10668,3.98626,4.00533],[3.51045,3.14617,3.02558],[9.31981,9.03239,9.00459],[0.563991,0.853628,0.971569],[8.40649,3.29997,3.82986],[9.52108,3.52917,3.86565],[18.4842,4.56174,4.67819],[50.449,16.4463,16.4923],[1.96056,1.59658,1.85278],[1.76567,1.51262,1.52955],[2.11338,1.81927,1.92422],[8.6392,5.18686,5.77986],[51.8749,52.0756,52.2289],[63.6736,63.868,63.3526],[2.92731,2.88967,2.96908],[4.83141,3.88826,3.41858],[42.2016,40.1165,39.4692],[14.5369,10.3602,10.4522],[14.7224,10.829,10.8651],[5.32702,5.79151,5.20852],[0.339799,0.158694,0.160555],[0.173487,0.112253,0.112419],[0.158136,0.102158,0.101214],[0.341324,0.263589,0.24586],[0.171119,0.109181,0.108795],[0.178617,0.117175,0.117738],[0.205935,0.137013,0.137252]],"source":"greenplum/results/c6a.4xlarge.json"}
 ,{"system":"HeavyAI","date":"2022-07-01","machine":"c6a.4xlarge, 500gb gp2","cluster_size":1,"comment":"Previous names: OmniSci, mapD. Many queries cannot run due to errors and limitations.","tags":["C++","column-oriented"],"load_time":572.633,"data_size":50887437386,"result":[[6.525,0.022,0.029],[0.301,0.042,0.04],[0.287,0.095,0.093],[2.572,0.039,0.04],[null,null,null],[null,null,null],[7.327,0.093,0.097],[0.244,0.043,0.038],[null,null,null],[null,null,null],[null,null,null],[null,null,null],[2.939,0.295,0.294],[null,null,null],[null,null,null],[null,null,null],[null,null,null],[4.716,3.91,3.955],[null,null,null],[0.154,0.083,0.106],[14.426,0.07,0.071],[null,null,null],[null,null,null],[null,null,null],[2.276,0.258,0.272],[null,null,null],[null,null,null],[null,null,null],[null,null,null],[1.832,1.64,1.602],[null,null,null],[null,null,null],[null,null,null],[14.811,0.494,0.497],[null,null,null],[null,null,null],[1.941,0.255,0.255],[5.457,0.172,0.283],[0.476,0.269,0.256],[14.239,0.179,0.178],[3.992,0.112,0.112],[1.031,0.116,0.116],[1.365,0.089,0.088]],"source":"heavyai/results/c6a.4xlarge.json"}
-,{"system":"Hydra","date":"2023-09-19","machine":"c6a.4xlarge, 500gb gp2","cluster_size":1,"comment":"","tags":["C","PostgreSQL compatible","column-oriented"],"load_time":1222,"data_size":18995669982,"result":[[0.381752,0.249001,0.243783],[0.261088,0.211013,0.209277],[0.6692,0.634666,0.627473],[0.545165,0.431892,0.439854],[22.4371,21.8547,21.731],[34.8689,33.0467,32.5865],[0.410038,0.377206,0.376254],[0.22464,0.206133,0.204092],[27.1582,26.4292,26.5203],[30.1541,29.4784,30.2199],[3.31616,2.76618,2.75128],[3.55764,3.28978,3.27316],[7.74401,7.64012,7.6384],[10.703,10.2599,10.2098],[8.70712,8.33702,8.29268],[17.9937,17.5084,17.5291],[22.8406,22.2649,22.1176],[4.08958,3.59194,3.61345],[39.0156,37.7258,37.8926],[0.267125,0.233304,0.233594],[3.12914,2.74305,2.72563],[2.91522,2.89177,2.90416],[3.76881,3.62139,3.62348],[26.322,20.5805,20.3337],[1.69125,1.05602,1.06068],[1.06135,0.828339,0.831532],[1.26898,1.06035,1.05212],[4.17896,4.03193,4.03657],[43.0503,42.5091,42.3298],[7.89438,7.50181,7.39528],[9.33451,8.76728,8.79973],[15.0403,14.6597,14.3034],[58.8221,58.6056,58.6859],[34.2914,35.0405,34.2519],[36.3141,35.6205,36.5728],[14.2056,13.8882,13.8284],[0.367202,0.33666,0.33517],[0.22008,0.172653,0.167434],[0.15307,0.126572,0.126678],[1.11688,0.671288,0.696326],[0.160667,0.145181,0.141243],[0.137686,0.122734,0.118493],[0.197786,0.123078,0.122094]],"source":"hydra/results/c6a.4xlarge.json"}
+,{"system":"Hydra","date":"2025-03-04","machine":"cloud","cluster_size":"serverless","tags":["managed","PostgreSQL compatible","column-oriented"],"load_time":136,"data_size":30816390348,"result":[[0.192382,0.031411,0.0182],[0.277037,0.017238,0.016364],[0.245723,0.038173,0.037376],[0.306797,0.042102,0.042718],[0.418011,0.242568,0.23039],[0.551909,0.282084,0.278836],[0.180043,0.02662,0.025371],[0.165209,0.024473,0.023097],[0.453139,0.284364,0.283832],[0.535552,0.393579,0.39756],[0.30116,0.092113,0.091926],[0.257973,0.100089,0.098468],[0.398887,0.26349,0.253034],[0.578082,0.426437,0.425838],[0.434567,0.263478,0.266679],[0.379118,0.262621,0.264961],[0.729468,0.564334,0.562802],[0.602923,0.464127,0.465615],[1.33815,0.996799,0.993801],[0.173591,0.013425,0.013638],[1.68838,0.515917,0.516346],[0.437349,0.294343,0.296138],[1.99367,0.417726,0.417281],[8.64109,1.77034,1.77321],[0.264145,0.104584,0.105626],[0.238236,0.095826,0.095679],[0.287271,0.112243,0.108904],[0.755408,0.612582,0.61124],[6.75692,6.60809,6.60236],[0.23476,0.07472,0.067433],[0.382783,0.239876,0.239216],[0.438111,0.296216,0.288612],[1.5727,1.69792,1.40727],[1.55361,1.28201,1.28065],[1.49317,1.40405,1.38793],[0.579766,0.41326,0.413215],[0.226025,0.062452,0.059734],[0.195866,0.024387,0.041084],[0.198459,0.025349,0.026228],[0.323418,0.091749,0.088929],[0.172858,0.019531,0.019639],[0.172619,0.019008,0.022273],[0.194027,0.027197,0.024702]],"source":"hydra/results/hydra.json"}
path: index.html
user: login: rschu1ze
body: No need to update index.html by hand, it is automatically updated by a CI job based on *.json files in `results/` subfolders.
created_at: 2025-03-05T09:25:33Z
author_association: MEMBER
diff_hunk: @@ -432,7 +432,7 @@
 ,{"system":"GlareDB","date":"2024-02-02","machine":"c6a.metal, 500gb gp2","cluster_size":1,"comment":"","tags":["Rust","serverless"],"load_time":0,"data_size":14779976446,"result":[[0.213,0.070,0.071],[4.066,4.060,3.978],[3.994,3.812,4.046],[4.287,4.122,4.218],[6.016,6.482,6.210],[5.260,5.308,5.242],[3.968,4.054,4.054],[4.013,3.961,4.045],[7.124,6.742,6.934],[5.632,5.625,5.433],[4.546,4.463,4.577],[4.540,4.616,4.675],[5.438,5.302,5.243],[6.286,6.115,6.202],[5.460,5.259,5.531],[6.064,6.451,6.458],[7.461,7.655,7.337],[6.920,7.538,7.684],[11.688,11.362,11.378],[4.264,4.085,4.092],[12.973,4.585,4.449],[14.739,4.562,4.676],[25.511,5.088,5.323],[59.374,8.895,8.542],[6.076,4.352,4.324],[4.305,4.233,4.334],[6.083,4.450,4.398],[12.983,4.812,4.721],[11.651,5.245,5.281],[4.275,4.214,4.175],[5.561,5.378,5.630],[9.220,5.983,6.112],[17.147,15.575,15.934],[14.016,9.356,8.974],[14.274,9.454,9.371],[6.675,6.473,6.510],[13.348,4.684,4.587],[12.473,4.655,4.580],[13.153,4.471,4.637],[21.939,4.932,4.951],[6.033,4.320,4.282],[5.181,4.242,4.286],[4.503,4.312,4.215]],"source":"glaredb/results/c6a.metal.json"}
 ,{"system":"Greenplum","date":"2022-08-18","machine":"c6a.4xlarge, 500gb gp2","cluster_size":1,"comment":"","tags":["C","column-oriented","PostgreSQL compatible"],"load_time":1080,"data_size":42000000000,"result":[[2.19644,1.27581,1.07334],[1.36821,1.09289,1.00081],[1.84455,1.93433,1.9734],[1.38533,1.07306,0.997511],[10.4301,10.1377,10.0064],[3.24183,3.67479,3.23526],[1.81177,1.22447,1.72696],[1.28568,1.12341,1.02112],[3.5159,3.7946,3.51106],[5.78795,5.90487,5.95635],[1.50773,1.80496,1.26395],[1.68682,1.93425,1.54081],[2.29378,2.75715,2.22458],[3.41812,3.09848,3.0303],[2.96859,2.96836,2.9949],[3.04409,3.00935,2.98848],[4.10668,3.98626,4.00533],[3.51045,3.14617,3.02558],[9.31981,9.03239,9.00459],[0.563991,0.853628,0.971569],[8.40649,3.29997,3.82986],[9.52108,3.52917,3.86565],[18.4842,4.56174,4.67819],[50.449,16.4463,16.4923],[1.96056,1.59658,1.85278],[1.76567,1.51262,1.52955],[2.11338,1.81927,1.92422],[8.6392,5.18686,5.77986],[51.8749,52.0756,52.2289],[63.6736,63.868,63.3526],[2.92731,2.88967,2.96908],[4.83141,3.88826,3.41858],[42.2016,40.1165,39.4692],[14.5369,10.3602,10.4522],[14.7224,10.829,10.8651],[5.32702,5.79151,5.20852],[0.339799,0.158694,0.160555],[0.173487,0.112253,0.112419],[0.158136,0.102158,0.101214],[0.341324,0.263589,0.24586],[0.171119,0.109181,0.108795],[0.178617,0.117175,0.117738],[0.205935,0.137013,0.137252]],"source":"greenplum/results/c6a.4xlarge.json"}
 ,{"system":"HeavyAI","date":"2022-07-01","machine":"c6a.4xlarge, 500gb gp2","cluster_size":1,"comment":"Previous names: OmniSci, mapD. Many queries cannot run due to errors and limitations.","tags":["C++","column-oriented"],"load_time":572.633,"data_size":50887437386,"result":[[6.525,0.022,0.029],[0.301,0.042,0.04],[0.287,0.095,0.093],[2.572,0.039,0.04],[null,null,null],[null,null,null],[7.327,0.093,0.097],[0.244,0.043,0.038],[null,null,null],[null,null,null],[null,null,null],[null,null,null],[2.939,0.295,0.294],[null,null,null],[null,null,null],[null,null,null],[null,null,null],[4.716,3.91,3.955],[null,null,null],[0.154,0.083,0.106],[14.426,0.07,0.071],[null,null,null],[null,null,null],[null,null,null],[2.276,0.258,0.272],[null,null,null],[null,null,null],[null,null,null],[null,null,null],[1.832,1.64,1.602],[null,null,null],[null,null,null],[null,null,null],[14.811,0.494,0.497],[null,null,null],[null,null,null],[1.941,0.255,0.255],[5.457,0.172,0.283],[0.476,0.269,0.256],[14.239,0.179,0.178],[3.992,0.112,0.112],[1.031,0.116,0.116],[1.365,0.089,0.088]],"source":"heavyai/results/c6a.4xlarge.json"}
-,{"system":"Hydra","date":"2023-09-19","machine":"c6a.4xlarge, 500gb gp2","cluster_size":1,"comment":"","tags":["C","PostgreSQL compatible","column-oriented"],"load_time":1222,"data_size":18995669982,"result":[[0.381752,0.249001,0.243783],[0.261088,0.211013,0.209277],[0.6692,0.634666,0.627473],[0.545165,0.431892,0.439854],[22.4371,21.8547,21.731],[34.8689,33.0467,32.5865],[0.410038,0.377206,0.376254],[0.22464,0.206133,0.204092],[27.1582,26.4292,26.5203],[30.1541,29.4784,30.2199],[3.31616,2.76618,2.75128],[3.55764,3.28978,3.27316],[7.74401,7.64012,7.6384],[10.703,10.2599,10.2098],[8.70712,8.33702,8.29268],[17.9937,17.5084,17.5291],[22.8406,22.2649,22.1176],[4.08958,3.59194,3.61345],[39.0156,37.7258,37.8926],[0.267125,0.233304,0.233594],[3.12914,2.74305,2.72563],[2.91522,2.89177,2.90416],[3.76881,3.62139,3.62348],[26.322,20.5805,20.3337],[1.69125,1.05602,1.06068],[1.06135,0.828339,0.831532],[1.26898,1.06035,1.05212],[4.17896,4.03193,4.03657],[43.0503,42.5091,42.3298],[7.89438,7.50181,7.39528],[9.33451,8.76728,8.79973],[15.0403,14.6597,14.3034],[58.8221,58.6056,58.6859],[34.2914,35.0405,34.2519],[36.3141,35.6205,36.5728],[14.2056,13.8882,13.8284],[0.367202,0.33666,0.33517],[0.22008,0.172653,0.167434],[0.15307,0.126572,0.126678],[1.11688,0.671288,0.696326],[0.160667,0.145181,0.141243],[0.137686,0.122734,0.118493],[0.197786,0.123078,0.122094]],"source":"hydra/results/c6a.4xlarge.json"}
+,{"system":"Hydra","date":"2025-03-04","machine":"cloud","cluster_size":"serverless","tags":["managed","PostgreSQL compatible","column-oriented"],"load_time":136,"data_size":30816390348,"result":[[0.192382,0.031411,0.0182],[0.277037,0.017238,0.016364],[0.245723,0.038173,0.037376],[0.306797,0.042102,0.042718],[0.418011,0.242568,0.23039],[0.551909,0.282084,0.278836],[0.180043,0.02662,0.025371],[0.165209,0.024473,0.023097],[0.453139,0.284364,0.283832],[0.535552,0.393579,0.39756],[0.30116,0.092113,0.091926],[0.257973,0.100089,0.098468],[0.398887,0.26349,0.253034],[0.578082,0.426437,0.425838],[0.434567,0.263478,0.266679],[0.379118,0.262621,0.264961],[0.729468,0.564334,0.562802],[0.602923,0.464127,0.465615],[1.33815,0.996799,0.993801],[0.173591,0.013425,0.013638],[1.68838,0.515917,0.516346],[0.437349,0.294343,0.296138],[1.99367,0.417726,0.417281],[8.64109,1.77034,1.77321],[0.264145,0.104584,0.105626],[0.238236,0.095826,0.095679],[0.287271,0.112243,0.108904],[0.755408,0.612582,0.61124],[6.75692,6.60809,6.60236],[0.23476,0.07472,0.067433],[0.382783,0.239876,0.239216],[0.438111,0.296216,0.288612],[1.5727,1.69792,1.40727],[1.55361,1.28201,1.28065],[1.49317,1.40405,1.38793],[0.579766,0.41326,0.413215],[0.226025,0.062452,0.059734],[0.195866,0.024387,0.041084],[0.198459,0.025349,0.026228],[0.323418,0.091749,0.088929],[0.172858,0.019531,0.019639],[0.172619,0.019008,0.022273],[0.194027,0.027197,0.024702]],"source":"hydra/results/hydra.json"}
path: index.html
user: login: wuputah
body: got it, I'll undo this change.
created_at: 2025-03-05T18:53:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,34 +2,11 @@
 set -euo pipefail
 IFS=$'\n\t'
 
-# sets up PG14, configs, and build deps on ubuntu 22.04
-sudo ./install.sh
-
-# download hits.tsv if we dont already have it
-if [ ! -e hits.tsv ]; then
-    wget --no-verbose --continue 'https://datasets.clickhouse.com/hits_compatible/hits.tsv.gz'
-    gzip -d hits.tsv.gz
-fi
-
-# ensure postgres user can access hits.tsv
-chmod a+rX . hits.tsv
-
-# set up db, user, table; load data.
-sudo -u postgres psql -t -c 'CREATE DATABASE test'
-sudo -u postgres psql test -t < create.sql
-sudo -u postgres psql test -t -c 'TRUNCATE hits'
-sudo -u postgres psql test -t -c '\timing' -c "\\copy hits FROM 'hits.tsv'"
-
-# COPY 99997497
-# Time: 1320024.670 ms (22:00.025)
+# load data
path: hydra/benchmark.sh
user: login: wuputah
body: hmm, ok it seems the pattern is to leave a commented-out command for these.
created_at: 2025-03-05T18:55:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,34 +2,11 @@
 set -euo pipefail
 IFS=$'\n\t'
 
-# sets up PG14, configs, and build deps on ubuntu 22.04
-sudo ./install.sh
-
-# download hits.tsv if we dont already have it
-if [ ! -e hits.tsv ]; then
-    wget --no-verbose --continue 'https://datasets.clickhouse.com/hits_compatible/hits.tsv.gz'
-    gzip -d hits.tsv.gz
-fi
-
-# ensure postgres user can access hits.tsv
-chmod a+rX . hits.tsv
-
-# set up db, user, table; load data.
-sudo -u postgres psql -t -c 'CREATE DATABASE test'
-sudo -u postgres psql test -t < create.sql
-sudo -u postgres psql test -t -c 'TRUNCATE hits'
-sudo -u postgres psql test -t -c '\timing' -c "\\copy hits FROM 'hits.tsv'"
-
-# COPY 99997497
-# Time: 1320024.670 ms (22:00.025)
+# load data
path: hydra/benchmark.sh
user: login: wuputah
body: updated.
created_at: 2025-03-05T18:57:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,58 @@
+{
+    "system": "Hydra",
+    "date": "2025-03-04",
+    "machine": "cloud",
path: hydra/results/hydra.json
user: login: rschu1ze
body: As far as I see, Hydra offers "Pro" plan and a "Basic" plan, each with different compute options ("small" - "xLarge"). Could we specify here, which on this was tested on?
created_at: 2025-03-09T10:55:30Z
author_association: MEMBER
diff_hunk: @@ -1,16 +1,59 @@
 # Hydra
 
-Hydra is an open source data warehouse built on Postgres.
+Hydra is a high-performance Postgres database with serverless
+online analytical processing (OLAP). It is designed for low-latency
+applications built on time series and event data.
 
-* [Homepage](https://hydras.io)
-* [GitHub](https://github.com/HydrasDB/hydra)
+* [Homepage](https://hydra.so)
 
 ## Running the benchmark
 
-The benchmark has been configured for a `c6a.4xlarge` running Ubuntu 22.04 and can be run without attendance.
+The benchmark itself requires `psql` to execute. You will need to install `psql`, e.g.:
 
 ```
-export FQDN=ec2-127-0-0-1.compute-1.amazonaws.com
-scp -i ~/.ssh/aws.pem *.sh *.sql ubuntu@$FQDN:~
-ssh -i ~/.ssh/aws.pem ubuntu@$FQDN ./benchmark.sh
+# Debian, Ubuntu, etc
+apt install postgresql-client
+# MacOS
+brew install postgresql
 ```
+
+Note that either option below will introduce additional latency depending on the location
+from which the benchmark is run.
+
+### Local Instance
+
+In this configuration, you will run a Postgres instance locally and it will connect
+to Hydra's serverless OLAP service. This will use the Hydra CLI which will manage
+a Docker container for you.
+
+1. Install Python, and if desired, your preferred Python management tools, like pipx, uv, etc.
+2. Install Hydra CLI using `pip install hydra-cli` (or via `pipx`, `uv`, etc)
+3. Install Docker.
+4. Sign up for Hydra at https://start.hydra.so/get-started and copy the token to your clipboard
+5. Run `hydra setup` and paste in your token
+6. Run `hydra start` to start the local instance
+7. Run `hydra config` to obtain the connection string and set it `DATABASE_URL` in your local environment,
+   or pass it in when you run the benchmark.
+8. Run the benchmark.
+
+    ```
+    DATABASE_URL="..." ./benchmark.sh
path: hydra/README.md
user: login: rschu1ze
body: I registered with a fresh new mail address (the third one by now), added my private credit card in the web UI, then went meticulously through steps 1-7 (on a standard Ubuntu, there were no error messages, the local Postgres was also running) and at step 8, I got:

```
psql: error: connection to server at "localhost" (127.0.0.1), port 52488 failed: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
```

I didn't have the time to debug this in detail and gave up. If someone else likes to double-check, please go ahead. (Please have a look at my other comment about the exact plan).
created_at: 2025-03-09T10:58:49Z
author_association: MEMBER
diff_hunk: @@ -1,16 +1,59 @@
 # Hydra
 
-Hydra is an open source data warehouse built on Postgres.
+Hydra is a high-performance Postgres database with serverless
+online analytical processing (OLAP). It is designed for low-latency
+applications built on time series and event data.
 
-* [Homepage](https://hydras.io)
-* [GitHub](https://github.com/HydrasDB/hydra)
+* [Homepage](https://hydra.so)
 
 ## Running the benchmark
 
-The benchmark has been configured for a `c6a.4xlarge` running Ubuntu 22.04 and can be run without attendance.
+The benchmark itself requires `psql` to execute. You will need to install `psql`, e.g.:
 
 ```
-export FQDN=ec2-127-0-0-1.compute-1.amazonaws.com
-scp -i ~/.ssh/aws.pem *.sh *.sql ubuntu@$FQDN:~
-ssh -i ~/.ssh/aws.pem ubuntu@$FQDN ./benchmark.sh
+# Debian, Ubuntu, etc
+apt install postgresql-client
+# MacOS
+brew install postgresql
 ```
+
+Note that either option below will introduce additional latency depending on the location
+from which the benchmark is run.
+
+### Local Instance
+
+In this configuration, you will run a Postgres instance locally and it will connect
+to Hydra's serverless OLAP service. This will use the Hydra CLI which will manage
+a Docker container for you.
+
+1. Install Python, and if desired, your preferred Python management tools, like pipx, uv, etc.
+2. Install Hydra CLI using `pip install hydra-cli` (or via `pipx`, `uv`, etc)
+3. Install Docker.
+4. Sign up for Hydra at https://start.hydra.so/get-started and copy the token to your clipboard
+5. Run `hydra setup` and paste in your token
+6. Run `hydra start` to start the local instance
+7. Run `hydra config` to obtain the connection string and set it `DATABASE_URL` in your local environment,
+   or pass it in when you run the benchmark.
+8. Run the benchmark.
+
+    ```
+    DATABASE_URL="..." ./benchmark.sh
path: hydra/README.md
user: login: wuputah
body: This seems like the same bug with the old CLI version, can you check `hydra --version`? If you have updated on the same machine, you'll then need to `hydra restart`, which should resolve the issue.
created_at: 2025-03-09T15:56:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,16 +1,59 @@
 # Hydra
 
-Hydra is an open source data warehouse built on Postgres.
+Hydra is a high-performance Postgres database with serverless
+online analytical processing (OLAP). It is designed for low-latency
+applications built on time series and event data.
 
-* [Homepage](https://hydras.io)
-* [GitHub](https://github.com/HydrasDB/hydra)
+* [Homepage](https://hydra.so)
 
 ## Running the benchmark
 
-The benchmark has been configured for a `c6a.4xlarge` running Ubuntu 22.04 and can be run without attendance.
+The benchmark itself requires `psql` to execute. You will need to install `psql`, e.g.:
 
 ```
-export FQDN=ec2-127-0-0-1.compute-1.amazonaws.com
-scp -i ~/.ssh/aws.pem *.sh *.sql ubuntu@$FQDN:~
-ssh -i ~/.ssh/aws.pem ubuntu@$FQDN ./benchmark.sh
+# Debian, Ubuntu, etc
+apt install postgresql-client
+# MacOS
+brew install postgresql
 ```
+
+Note that either option below will introduce additional latency depending on the location
+from which the benchmark is run.
+
+### Local Instance
+
+In this configuration, you will run a Postgres instance locally and it will connect
+to Hydra's serverless OLAP service. This will use the Hydra CLI which will manage
+a Docker container for you.
+
+1. Install Python, and if desired, your preferred Python management tools, like pipx, uv, etc.
+2. Install Hydra CLI using `pip install hydra-cli` (or via `pipx`, `uv`, etc)
+3. Install Docker.
+4. Sign up for Hydra at https://start.hydra.so/get-started and copy the token to your clipboard
+5. Run `hydra setup` and paste in your token
+6. Run `hydra start` to start the local instance
+7. Run `hydra config` to obtain the connection string and set it `DATABASE_URL` in your local environment,
+   or pass it in when you run the benchmark.
+8. Run the benchmark.
+
+    ```
+    DATABASE_URL="..." ./benchmark.sh
path: hydra/README.md
user: login: rschu1ze
body: I am fine with submitting the results, let's just make sure that the actually used compute option is specified in hydra.json (please see my other comment).
created_at: 2025-03-09T17:03:22Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,58 @@
+{
+    "system": "Hydra",
+    "date": "2025-03-04",
+    "machine": "cloud",
path: hydra/results/hydra.json
user: login: wuputah
body: all set
created_at: 2025-03-09T21:04:12Z
author_association: CONTRIBUTOR

PR comments:
user: login: rschu1ze
created_at: 2025-03-05T19:23:33Z
body: @wuputah I tried to create an account at hydra.so using Google single-sign on (`robert@clickhouse.com`). I didn't receive a confirmation code on this mail address (even after waiting 15 minutes).

I then tried to setup a second account with a freshly generated independent mail address. That succeeded.

"hydra/README.md" asks to provide a connection string. What I have is a token (that is equivalent to 5 hours of free compute). How does the connection string look like, is it printed somewhere on the website? Couldn't find something in the Hydra docs about it.
author_association: MEMBER
user: login: wuputah
created_at: 2025-03-05T19:30:37Z
body: You can run it locally and get a connection string using `hydra config`, but the benchmark used a paid cloud instance and was tested from the same cloud to eliminate effects of latency. A connection string is readily available on all paid instances.
author_association: CONTRIBUTOR
user: login: rschu1ze
created_at: 2025-03-05T19:39:40Z
body: One of the goals of ClickBench is reproducibility. If I read correctly, the trial token isn't sufficient to run the benchmark, one needs a paid instance. Fair enough.

So if I subscribe, it will show the connection string somewhere?

> You can run it locally and get a connection string using hydra config

I didn't understand if that applies to the free or the paid version. I also didn't understand what "hydra config" is, where to get it and how to run it.
author_association: MEMBER
user: login: wuputah
created_at: 2025-03-05T19:47:30Z
body: > So if I subscribe, it will show the connection string somewhere?

Yes, the connection string is displayed quite prominently on each project's dashboard, e.g.:

<img width="453" alt="image" src="https://github.com/user-attachments/assets/de10a3d1-6bf7-4fc6-839e-7b110942b757" />
 
> I didn't understand if that applies to the free or the paid version. I also didn't understand what "hydra config" is, where to get it and how to run it.

The `pip install hydra-cli` provides a local `hydra` command which will allow you to run things locally with the free trial. Running the benchmark isn't the intention here and `hydra config` isn't really in the happy path, but is there for those that need it.
author_association: CONTRIBUTOR
user: login: rschu1ze
created_at: 2025-03-05T20:55:19Z
body: Regarding the free version:

I installed `hydra-cli`, went through `hydra setup`, then `hydra config` to obtain the connection string but then hit

```
/data/ClickBench2/hydra (update-hydra =) $ DATABASE_URL="postgresql://postgres:esyD31pH@localhost:52488/postgres" ./benchmark.sh
psql: error: connection to server at "localhost" (127.0.0.1), port 52488 failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?
```

It obviously connects to localhost but I couldn't figure the remote URL out. I think that is what you meant that it won't work with the free version.

I then tried to register for a paid version. I signed out and in again with my other mail address (my work mail address from Google didn't work, see above). I then see this screen:

![grafik](https://github.com/user-attachments/assets/e71be4ae-910e-4a38-9d8d-1161f4d1afaa)

I then click "Register Cloud Access" and it states that it sent a mail with a confirmation code. This is the second time I am asked for a confirmation code (the first time, I entered it properly). I am waiting since 20 minutes now but didn't receive another code. The mail address is `robert.hydra@mailbox.org` in case you like to check.
author_association: MEMBER
user: login: wuputah
created_at: 2025-03-05T23:12:25Z
body: You'll need to run `hydra start` to have the local Postgres instance running. 
author_association: CONTRIBUTOR
user: login: rschu1ze
created_at: 2025-03-06T09:43:59Z
body: ```sh
/data/ClickBench2 (update-hydra =) $ hydra start
Updating image... done.
Starting service on port 52488... done.

/data/ClickBench2/hydra (update-hydra =) $ DATABASE_URL="postgresql://postgres:esyD31pH@localhost:52488/postgres" ./benchmark.sh
psql: error: connection to server at "localhost" (127.0.0.1), port 52488 failed: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
```

I see that postgres is running in the background after `hydra start`:

```sh
/data/ClickBench2 (update-hydra =) $ ps aux | grep postgres
dnsmasq     2249  0.2  0.1 281576 40704 ?        Ss   09:42   0:00 postgres -c config_file=/etc/postgresql/postgresql.conf
dnsmasq     2305  0.0  0.0 281708  7432 ?        Ss   09:42   0:00 postgres: checkpointer
dnsmasq     2306  0.0  0.0 281724  7944 ?        Ss   09:42   0:00 postgres: background writer
dnsmasq     2308  0.0  0.0 281576 11656 ?        Ss   09:42   0:00 postgres: walwriter
dnsmasq     2309  0.0  0.0 283144 10376 ?        Ss   09:42   0:00 postgres: autovacuum launcher
dnsmasq     2310  1.3  0.2 4210500 76360 ?       Ssl  09:42   0:00 postgres: pg_duckdb sync worker
dnsmasq     2311  0.0  0.0 283152  9736 ?        Ss   09:42   0:00 postgres: logical replication launcher
ubuntu      2589  0.0  0.0   6544  2304 pts/3    S+   09:43   0:00 grep postgres
```
author_association: MEMBER
user: login: rschu1ze
created_at: 2025-03-06T09:45:22Z
body: Maybe it helps if the README.md file could specify the steps to reproduce the benchmark at more detail. Basically write it for someone who has never used Hydra.
author_association: MEMBER
user: login: wuputah
created_at: 2025-03-06T20:07:51Z
body: Apologies, much of this is quite nascent so there appears to be a bug here with the local connection, I'll try to get a fix out for that ASAP.

The test itself was done on a paid instance where the credential would be readily available. Right now we are manually onboarding customers, so the product is not available without talking to us, which is what "Request Cloud Access" should have started. Happy to get you access if you wish to try it.
author_association: CONTRIBUTOR
user: login: wuputah
created_at: 2025-03-07T18:54:29Z
body: OK, that bug has been fixed. You can update with `pip install --upgrade hydra-cli` or as desired for your python toolchain.

I'll update the README with more instructions, both for local and cloud.
author_association: CONTRIBUTOR