Repository: pgexporter/pgexporter
Pull request #222: Update HTTP APIs
Author: login: bassamadnan
Created: 2025-05-12T13:37:00Z | Merged: 2025-05-27T11:10:50Z

This is a port from [pgmoneta](https://github.com/pgmoneta/pgmoneta/pull/620/) where we improvise our http functionality. I added a bridge and sent curl request to 5003 with this configuration. Addressing #226 

```
[pgexporter]
host = localhost
metrics = 5002
bridge = 5003
bridge_endpoints = localhost:5002
log_type = file
log_level = debug
log_path = /tmp/pgexporter.log
```

The logs remain the same what we obtained from previous implementation, and bridge works just fine. PTAL @jesperpedersen 


Diff:
diff --git a/src/include/http.h b/src/include/http.h
index 510a4fe4..5be2ba81 100644
--- a/src/include/http.h
+++ b/src/include/http.h
@@ -33,44 +33,117 @@
 extern "C" {
 #endif
 
+#include <pgexporter.h>
+
 #include <stdbool.h>
+#include <stdint.h>
 #include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+/* HTTP method definitions */
+#define PGEXPORTER_HTTP_GET  0
+#define PGEXPORTER_HTTP_POST 1
+#define PGEXPORTER_HTTP_PUT  2
 
 /** @struct http
  * Defines a HTTP interaction
  */
 struct http
 {
-   int endpoint;  /**< The endpoint */
-   int socket;    /**< The socket */
-   char* headers; /**< The HTTP headers */
-   char* body;    /**< The HTTP body */
+   int endpoint;            /**< The endpoint */
+   int socket;              /**< The socket descriptor */
+   char* body;              /**< The HTTP response body */
+   char* headers;           /**< The HTTP response headers */
+   char* request_headers;   /**< The HTTP request headers */
+   SSL* ssl;                /**< The SSL connection (NULL for non-secure) */
 };
 
 /**
- * Create a HTTP interaction
- * @param endpoint The endpoint
- * @param http The resulting HTTP interaction
- * @return 0 if success, otherwise 1
+ * Connect to an HTTP/HTTPS server
+ * @param hostname The host to connect to
+ * @param port The port number
+ * @param secure Use SSL if true
+ * @param result The resulting HTTP structure
+ * @return 0 upon success, otherwise 1
+ */
+int
+pgexporter_http_connect(char* hostname, int port, bool secure, struct http** result);
+
+/**
+ * Disconnect and clean up HTTP resources
+ * @param http The HTTP structure
+ * @return 0 upon success, otherwise 1
+ */
+int
+pgexporter_http_disconnect(struct http* http);
+
+/**
+ * Add a header to the HTTP request
+ * @param http The HTTP structure
+ * @param name The header name
+ * @param value The header value
+ * @return 0 upon success, otherwise 1
+ */
+int
+pgexporter_http_add_header(struct http* http, char* name, char* value);
+
+/**
+ * Read response data directly from a socket
+ * @param ssl SSL connection (can be NULL for non-secure connections)
+ * @param socket The socket to read from
+ * @param response_text Pointer to store the response text
+ * @return MESSAGE_STATUS_OK on success, MESSAGE_STATUS_ERROR otherwise
+ */
+int
+pgexporter_http_read(SSL* ssl, int socket, char** response_text);
+
+/**
+ * Perform HTTP GET request
+ * @param http The HTTP structure
+ * @param hostname The hostname for the Host header
+ * @param path The path for the request
+ * @return 0 upon success, otherwise 1
+ */
+int
+pgexporter_http_get(struct http* http, char* hostname, char* path);
+
+/**
+ * Perform HTTP POST request
+ * @param http The HTTP structure
+ * @param hostname The hostname for the Host header
+ * @param path The path for the request
+ * @param data The data to send
+ * @param length The length of the data
+ * @return 0 upon success, otherwise 1
  */
 int
-pgexporter_http_create(int endpoint, struct http** http);
+pgexporter_http_post(struct http* http, char* hostname, char* path, char* data, size_t length);
 
 /**
- * Execute GET request
- * @param http The HTTP interaction
- * @return 0 if success, otherwise 1
+ * Perform HTTP PUT request
+ * @param http The HTTP structure
+ * @param hostname The hostname for the Host header
+ * @param path The path for the request
+ * @param data The data to upload
+ * @param length The length of the data
+ * @return 0 upon success, otherwise 1
  */
 int
-pgexporter_http_get(struct http* http);
+pgexporter_http_put(struct http* http, char* hostname, char* path, const void* data, size_t length);
 
 /**
- * Destroy HTTP interaction
- * @param http The HTTP interaction
- * @return 0 if success, otherwise 1
+ * Perform HTTP PUT request with a file
+ * @param http The HTTP structure
+ * @param hostname The hostname for the Host header
+ * @param path The path for the request
+ * @param file The file to upload
+ * @param file_size The size of the file
+ * @param content_type The content type of the file (can be NULL for default)
+ * @return 0 upon success, otherwise 1
  */
 int
-pgexporter_http_destroy(struct http* http);
+pgexporter_http_put_file(struct http* http, char* hostname, char* path, FILE* file, size_t file_size, char* content_type);
 
 #ifdef __cplusplus
 }
diff --git a/src/libpgexporter/http.c b/src/libpgexporter/http.c
index d44be9da..cd7c061a 100644
--- a/src/libpgexporter/http.c
+++ b/src/libpgexporter/http.c
@@ -27,226 +27,900 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp = NULL;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      goto error;
+   }
+
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      goto error;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 
+error:
+   free(response_copy);
    return 1;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
    struct message* msg_response = NULL;
    int error = 0;
    int status;
    char* request = NULL;
+   char* full_request = NULL;
    char* response = NULL;
+   char* user_agent = NULL;
+   char* endpoint = (path != NULL) ? path : "/metrics";
 
-   if (build_get_request(http->endpoint, &request))
-   {
-      goto error;
-   }
+   memset(&msg_request, 0, sizeof(struct message));
 
-   msg_request = (struct message*)malloc(sizeof(struct message));
-   if (msg_request == NULL)
+   pgexporter_log_trace("Starting pgexporter_http_get");
+   if (http_build_header(PGEXPORTER_HTTP_GET, endpoint, &request))
    {
+      pgexporter_log_error("Failed to build HTTP header");
       goto error;
    }
 
-   memset(msg_request, 0, sizeof(struct message));
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Accept", "text/*");
+   pgexporter_http_add_header(http, "Connection", "close");
+
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
 
-   msg_request->data = request;
-   msg_request->length = strlen(request) + 1;
+   msg_request.data = full_request;
+   msg_request.length = strlen(full_request) + 1;
 
    error = 0;
 req:
    if (error < 5)
    {
-      status = pgexporter_write_message(NULL, http->socket, msg_request);
+      status = pgexporter_write_message(http->ssl, http->socket, &msg_request);
       if (status != MESSAGE_STATUS_OK)
       {
          error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
          goto req;
       }
    }
    else
    {
+      pgexporter_log_error("Failed to write after 5 attempts");
       goto error;
    }
 
 res:
-   status = pgexporter_read_block_message(NULL, http->socket, &msg_response);
+   // Get a pointer to the global message structure
+   status = pgexporter_read_block_message(http->ssl, http->socket, &msg_response);
    if (status != MESSAGE_STATUS_ZERO)
    {
       if (status == MESSAGE_STATUS_OK)
       {
-         response = pgexporter_append(response, (char*)msg_response->data);
+         // Get data from the response message
+         if (msg_response != NULL && msg_response->data != NULL)
+         {
+            response = pgexporter_append(response, (char*)msg_response->data);
+         }
          pgexporter_clear_message();
          goto res;
       }
       else
       {
+         pgexporter_log_error("Error reading response");
          goto error;
       }
    }
 
-   if (msg_response->length > 0)
+   if (msg_response != NULL && msg_response->length > 0 && msg_response->data != NULL)
    {
       response = pgexporter_append(response, (char*)msg_response->data);
    }
 
-   if (extract_headers_body(response, http))
+   if (http_extract_headers_body(response, http))
    {
+      pgexporter_log_error("Failed to extract headers and body");
       goto error;
    }
 
+   pgexporter_log_debug("HTTP Headers: %s", http->headers != NULL ? http->headers : "NULL");
+   pgexporter_log_debug("HTTP Body: %s", http->body != NULL ? http->body : "NULL");
+
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
 
-   free(msg_request);
+   free(http->request_headers);
+   http->request_headers = NULL;
 
    return 0;
 
 error:
-
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+   return 1;
+}
+
+int
+pgexporter_http_connect(char* hostname, int port, bool secure, struct http** result)
+{
+   struct http* h = NULL;
+   int socket_fd = -1;
+   SSL* ssl = NULL;
+   SSL_CTX* ctx = NULL;
+
+   pgexporter_log_debug("Connecting to %s:%d (secure: %d)", hostname, port, secure);
+   h = (struct http*) malloc(sizeof(struct http));
+   if (h == NULL)
+   {
+      pgexporter_log_error("Failed to allocate HTTP structure");
+      goto error;
+   }
+
+   memset(h, 0, sizeof(struct http));
+
+   if (pgexporter_connect(hostname, port, &socket_fd))
+   {
+      pgexporter_log_error("Failed to connect to %s:%d", hostname, port);
+      goto error;
+   }
+
+   h->socket = socket_fd;
+
+   if (secure)
+   {
+      if (pgexporter_create_ssl_ctx(true, &ctx))
+      {
+         pgexporter_log_error("Failed to create SSL context");
+         goto error;
+      }
+
+      if (SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION) == 0)
+      {
+         pgexporter_log_error("Failed to set minimum TLS version");
+         goto error;
+      }
+
+      ssl = SSL_new(ctx);
+      if (ssl == NULL)
+      {
+         pgexporter_log_error("Failed to create SSL structure");
+         goto error;
+      }
+
+      if (SSL_set_fd(ssl, socket_fd) == 0)
+      {
+         pgexporter_log_error("Failed to set SSL file descriptor");
+         goto error;
+      }
+
+      int connect_result;
+      do
+      {
+         connect_result = SSL_connect(ssl);
+
+         if (connect_result != 1)
+         {
+            int err = SSL_get_error(ssl, connect_result);
+            switch (err)
+            {
+               case SSL_ERROR_WANT_READ:
+               case SSL_ERROR_WANT_WRITE:
+                  continue;
+               default:
+                  pgexporter_log_error("SSL connection failed: %s", ERR_error_string(err, NULL));
+                  goto error;
+            }
+         }
+      }
+      while (connect_result != 1);
+
+      h->ssl = ssl;
+   }
 
-   free(msg_request);
+   *result = h;
 
+   return 0;
+
+error:
+   if (ssl != NULL)
+   {
+      SSL_free(ssl);
+   }
+   if (ctx != NULL)
+   {
+      SSL_CTX_free(ctx);
+   }
+   if (socket_fd != -1)
+   {
+      pgexporter_disconnect(socket_fd);
+   }
+   free(h);
    return 1;
 }
 
 int
-pgexporter_http_destroy(struct http* http)
+pgexporter_http_post(struct http* http, char* hostname, char* path, char* data, size_t length)
 {
-   if (http != NULL)
+   struct message msg_request;
+   struct message* msg_request_ptr = NULL;
+   int error = 0;
+   int status;
+   char* request = NULL;
+   char* full_request = NULL;
+   char* response = NULL;
+   char* user_agent = NULL;
+   char content_length[32];
+
+   memset(&msg_request, 0, sizeof(struct message));
+
+   pgexporter_log_trace("Starting pgexporter_http_post");
+   if (http_build_header(PGEXPORTER_HTTP_POST, path, &request))
+   {
+      pgexporter_log_error("Failed to build HTTP header");
+      goto error;
+   }
+
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Connection", "close");
+
+   sprintf(content_length, "%zu", length);
+   pgexporter_http_add_header(http, "Content-Length", content_length);
+   pgexporter_http_add_header(http, "Content-Type", "application/x-www-form-urlencoded");
+
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
+
+   if (data != NULL && length > 0)
+   {
+      full_request = pgexporter_append(full_request, data);
+   }
+
+   msg_request_ptr = (struct message*)malloc(sizeof(struct message));
+   if (msg_request_ptr == NULL)
+   {
+      pgexporter_log_error("Failed to allocate msg_request");
+      goto error;
+   }
+
+   memset(msg_request_ptr, 0, sizeof(struct message));
+
+   msg_request_ptr->data = full_request;
+   msg_request_ptr->length = strlen(full_request) + 1;
+
+   error = 0;
+req:
+   if (error < 5)
+   {
+      status = pgexporter_write_message(http->ssl, http->socket, msg_request_ptr);
+      if (status != MESSAGE_STATUS_OK)
+      {
+         error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
+         goto req;
+      }
+   }
+   else
    {
-      pgexporter_disconnect(http->socket);
+      pgexporter_log_error("Failed to write after 5 attempts");
+      goto error;
+   }
 
-      free(http->headers);
-      free(http->body);
+   status = pgexporter_http_read(http->ssl, http->socket, &response);
+
+   if (response == NULL)
+   {
+      pgexporter_log_error("No response data collected");
+      goto error;
    }
 
-   free(http);
-   http = NULL;
+   if (http_extract_headers_body(response, http))
+   {
+      pgexporter_log_error("Failed to extract headers and body");
+      goto error;
+   }
+
+   free(request);
+   free(full_request);
+   free(response);
+   free(msg_request_ptr);
+   free(user_agent);
+
+   free(http->request_headers);
+   http->request_headers = NULL;
 
    return 0;
+
+error:
+   free(request);
+   free(full_request);
+   free(response);
+   free(msg_request_ptr);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+   return 1;
 }
 
-static int
-build_get_request(int endpoint, char** request)
+int
+pgexporter_http_put(struct http* http, char* hostname, char* path, const void* data, size_t length)
 {
-   char* r = NULL;
-   struct configuration* config = NULL;
+   struct message msg_request;
+   struct message* msg_request_ptr = NULL;
+   int error = 0;
+   int status;
+   char* request = NULL;
+   char* full_request = NULL;
+   char* response = NULL;
+   char* user_agent = NULL;
+   char* complete_request = NULL;
+   char content_length[32];
 
-   config = (struct configuration*)shmem;
+   memset(&msg_request, 0, sizeof(struct message));
 
-   *request = NULL;
+   pgexporter_log_trace("Starting pgexporter_http_put");
+   if (http_build_header(PGEXPORTER_HTTP_PUT, path, &request))
+   {
+      pgexporter_log_error("Failed to build HTTP header");
+      goto error;
+   }
 
-   r = pgexporter_append(r, "GET /metrics HTTP/1.1\r\n");
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Connection", "close");
 
-   r = pgexporter_append(r, "Host: ");
-   r = pgexporter_append(r, config->endpoints[endpoint].host);
-   r = pgexporter_append(r, "\r\n");
+   sprintf(content_length, "%zu", length);
+   pgexporter_http_add_header(http, "Content-Length", content_length);
+   pgexporter_http_add_header(http, "Content-Type", "application/octet-stream");
 
-   r = pgexporter_append(r, "User-Agent: pgexporter/");
-   r = pgexporter_append(r, VERSION);
-   r = pgexporter_append(r, "\r\n");
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
 
-   r = pgexporter_append(r, "Accept: text/*\r\n");
+   size_t headers_len = strlen(full_request);
+   size_t total_len = headers_len + length;
 
-   r = pgexporter_append(r, "\r\n");
+   complete_request = malloc(total_len + 1);
+   if (complete_request == NULL)
+   {
+      pgexporter_log_error("Failed to allocate complete request");
+      goto error;
+   }
 
-   *request = r;
+   memcpy(complete_request, full_request, headers_len);
+
+   if (data != NULL && length > 0)
+   {
+      memcpy(complete_request + headers_len, data, length);
+   }
+
+   complete_request[total_len] = '\0';
+
+   msg_request_ptr = (struct message*)malloc(sizeof(struct message));
+   if (msg_request_ptr == NULL)
+   {
+      pgexporter_log_error("Failed to allocate msg_request");
+      goto error;
+   }
+
+   memset(msg_request_ptr, 0, sizeof(struct message));
+
+   msg_request_ptr->data = complete_request;
+   msg_request_ptr->length = total_len + 1;
+
+   error = 0;
+req:
+   if (error < 5)
+   {
+      status = pgexporter_write_message(http->ssl, http->socket, msg_request_ptr);
+      if (status != MESSAGE_STATUS_OK)
+      {
+         error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
+         goto req;
+      }
+   }
+   else
+   {
+      pgexporter_log_error("Failed to write after 5 attempts");
+      goto error;
+   }
+
+   status = pgexporter_http_read(http->ssl, http->socket, &response);
+
+   if (response == NULL)
+   {
+      pgexporter_log_error("No response data collected");
+      goto error;
+   }
+
+   if (http_extract_headers_body(response, http))
+   {
+      pgexporter_log_error("Failed to extract headers and body");
+      goto error;
+   }
+
+   free(request);
+   free(full_request);
+   free(response);
+   free(msg_request_ptr->data);
+   free(msg_request_ptr);
+   free(user_agent);
+
+   free(http->request_headers);
+   http->request_headers = NULL;
 
    return 0;
+
+error:
+   free(request);
+   free(full_request);
+   free(response);
+   free(complete_request);
+   free(msg_request_ptr);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+
+   return 1;
 }
 
-static int
-extract_headers_body(char* response, struct http* http)
+int
+pgexporter_http_put_file(struct http* http, char* hostname, char* path, FILE* file, size_t file_size, char* content_type)
 {
-   bool header = true;
-   char* p = NULL;
+   struct message msg_request;
+   struct message* msg_request_ptr = NULL;
+   int error = 0;
+   int status;
+   char* request = NULL;
+   char* header_part = NULL;
+   char* response = NULL;
+   char* user_agent = NULL;
+   char* full_request = NULL;
+   char content_length[32];
+   void* file_buffer = NULL;
 
-   p = strtok(response, "\n");
-   while (p != NULL)
+   memset(&msg_request, 0, sizeof(struct message));
+
+   pgexporter_log_trace("Starting pgexporter_http_put_file");
+   if (file == NULL)
    {
-      if (*p == '\r')
+      pgexporter_log_error("File is NULL");
+      goto error;
+   }
+
+   if (http_build_header(PGEXPORTER_HTTP_PUT, path, &request))
+   {
+      pgexporter_log_error("Failed to build HTTP header");
+      goto error;
+   }
+
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Connection", "close");
+
+   sprintf(content_length, "%zu", file_size);
+   pgexporter_http_add_header(http, "Content-Length", content_length);
+
+   /* default to application/octet-stream if not specified */
+   char* type = (content_type != NULL) ? content_type : "application/octet-stream";
+   pgexporter_http_add_header(http, "Content-Type", type);
+
+   header_part = pgexporter_append(NULL, request);
+   header_part = pgexporter_append(header_part, http->request_headers);
+   header_part = pgexporter_append(header_part, "\r\n");
+
+   pgexporter_log_trace("File size: %zu", file_size);
+
+   rewind(file);
+
+   file_buffer = malloc(file_size);
+   if (file_buffer == NULL)
+   {
+      pgexporter_log_error("Failed to allocate memory for file content: %zu bytes", file_size);
+      goto error;
+   }
+
+   size_t bytes_read = fread(file_buffer, 1, file_size, file);
+   if (bytes_read != file_size)
+   {
+      pgexporter_log_error("Failed to read entire file. Expected %zu bytes, got %zu", file_size, bytes_read);
+      goto error;
+   }
+
+   pgexporter_log_trace("Read %zu bytes from file", bytes_read);
+
+   msg_request_ptr = (struct message*)malloc(sizeof(struct message));
+   if (msg_request_ptr == NULL)
+   {
+      pgexporter_log_error("Failed to allocate msg_request");
+      goto error;
+   }
+
+   memset(msg_request_ptr, 0, sizeof(struct message));
+
+   size_t header_len = strlen(header_part);
+   size_t total_len = header_len + file_size;
+
+   full_request = malloc(total_len + 1);
+   if (full_request == NULL)
+   {
+      pgexporter_log_error("Failed to allocate memory for full request: %zu bytes", total_len + 1);
+      goto error;
+   }
+
+   memcpy(full_request, header_part, header_len);
+
+   memcpy(full_request + header_len, file_buffer, file_size);
+
+   full_request[total_len] = '\0';
+
+   pgexporter_log_trace("Setting msg_request data, total size: %zu", total_len);
+   msg_request_ptr->data = full_request;
+   msg_request_ptr->length = total_len;
+
+   error = 0;
+req:
+   if (error < 5)
+   {
+      status = pgexporter_write_message(http->ssl, http->socket, msg_request_ptr);
+      if (status != MESSAGE_STATUS_OK)
       {
-         header = false;
+         error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
+         goto req;
+      }
+   }
+   else
+   {
+      pgexporter_log_error("Failed to write after 5 attempts");
+      goto error;
+   }
+
+   status = pgexporter_http_read(http->ssl, http->socket, &response);
+
+   if (response == NULL)
+   {
+      pgexporter_log_error("No response data collected");
+      goto error;
+   }
+
+   if (http_extract_headers_body(response, http))
+   {
+      pgexporter_log_error("Failed to extract headers and body");
+      goto error;
+   }
+
+   int status_code = 0;
+   if (http->headers != NULL && sscanf(http->headers, "HTTP/1.1 %d", &status_code) == 1)
+   {
+      pgexporter_log_debug("HTTP status code: %d", status_code);
+      if (status_code >= 200 && status_code < 300)
+      {
+         pgexporter_log_debug("HTTP request successful");
       }
       else
       {
-         if (!pgexporter_is_number(p, 16))
+         pgexporter_log_error("HTTP request failed with status code: %d", status_code);
+      }
+   }
+
+   free(request);
+   free(header_part);
+   free(response);
+   free(file_buffer);
+   free(full_request);
+   free(msg_request_ptr);
+   free(user_agent);
+
+   free(http->request_headers);
+   http->request_headers = NULL;
+
+   return (status_code >= 200 && status_code < 300) ? 0 : 1;
+
+error:
+   free(request);
+   free(header_part);
+   free(response);
+   free(file_buffer);
+   free(full_request);
+   free(msg_request_ptr);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+
+   return 1;
+}
+
+int
+pgexporter_http_read(SSL* ssl, int socket, char** response_text)
+{
+   char buffer[8192];
+   ssize_t bytes_read;
+   int total_bytes = 0;
+   bool headers_complete = false;
+   bool chunked_encoding = false;
+
+   *response_text = NULL;
+
+   while (1)
+   {
+      if (ssl != NULL)
+      {
+         bytes_read = SSL_read(ssl, buffer, sizeof(buffer) - 1);
+         if (bytes_read <= 0)
          {
-            if (header)
+            int err = SSL_get_error(ssl, bytes_read);
+            if (err == SSL_ERROR_WANT_READ || err == SSL_ERROR_WANT_WRITE)
             {
-               http->headers = pgexporter_append(http->headers, p);
-               http->headers = pgexporter_append_char(http->headers, '\n');
+               continue;
             }
-            else
+            break;
+         }
+      }
+      else
+      {
+         bytes_read = read(socket, buffer, sizeof(buffer) - 1);
+         if (bytes_read < 0)
+         {
+            if (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK)
             {
-               http->body = pgexporter_append(http->body, p);
-               http->body = pgexporter_append_char(http->body, '\n');
+               continue;
             }
+            break;
+         }
+         else if (bytes_read == 0)
+         {
+            break;
          }
       }
 
-      p = strtok(NULL, "\n");
+      buffer[bytes_read] = '\0';
+
+      if (*response_text == NULL)
+      {
+         *response_text = malloc(bytes_read + 1);
+         if (*response_text == NULL)
+         {
+            return MESSAGE_STATUS_ERROR;
+         }
+         memcpy(*response_text, buffer, bytes_read + 1);
+      }
+      else
+      {
+         size_t current_len = strlen(*response_text);
+         char* new_response = realloc(*response_text, current_len + bytes_read + 1);
+         if (new_response == NULL)
+         {
+            free(*response_text);
+            *response_text = NULL;
+            return MESSAGE_STATUS_ERROR;
+         }
+         *response_text = new_response;
+         memcpy(*response_text + current_len, buffer, bytes_read + 1);
+      }
+
+      total_bytes += bytes_read;
+
+      if (!headers_complete)
+      {
+         if (strstr(*response_text, "\r\n\r\n") != NULL)
+         {
+            headers_complete = true;
+            if (strstr(*response_text, "Transfer-Encoding: chunked") != NULL)
+            {
+               chunked_encoding = true;
+            }
+         }
+      }
+
+      if (!chunked_encoding && bytes_read < sizeof(buffer) - 1)
+      {
+         break;
+      }
+
+      if (chunked_encoding && strstr(*response_text, "\r\n0\r\n\r\n") != NULL)
+      {
+         break;
+      }
+   }
+
+   pgexporter_log_debug("Read %d bytes from socket", total_bytes);
+
+   return total_bytes > 0 ? MESSAGE_STATUS_OK : MESSAGE_STATUS_ERROR;
+}
+
+int
+pgexporter_http_disconnect(struct http* http)
+{
+   int status = 0;
+
+   if (http != NULL)
+   {
+      if (http->ssl != NULL)
+      {
+         pgexporter_close_ssl(http->ssl);
+         http->ssl = NULL;
+      }
+
+      if (http->socket != -1)
+      {
+         if (pgexporter_disconnect(http->socket))
+         {
+            pgexporter_log_error("Failed to disconnect socket in pgexporter_http_disconnect");
+            status = 1;
+         }
+         http->socket = -1;
+      }
+
+      if (http->headers != NULL)
+      {
+         free(http->headers);
+         http->headers = NULL;
+      }
+
+      if (http->body != NULL)
+      {
+         free(http->body);
+         http->body = NULL;
+      }
+
+      if (http->request_headers != NULL)
+      {
+         free(http->request_headers);
+         http->request_headers = NULL;
+      }
+   }
+
+   if (status != 0)
+   {
+      goto error;
    }
 
    return 0;
+
+error:
+   return 1;
 }
diff --git a/src/libpgexporter/prometheus_client.c b/src/libpgexporter/prometheus_client.c
index bc86d255..f1113176 100644
--- a/src/libpgexporter/prometheus_client.c
+++ b/src/libpgexporter/prometheus_client.c
@@ -121,13 +121,18 @@ pgexporter_prometheus_client_get(int endpoint, struct prometheus_bridge* bridge)
 
    pgexporter_log_debug("Endpoint http://%s:%d/metrics", config->endpoints[endpoint].host, config->endpoints[endpoint].port);
 
-   if (pgexporter_http_create(endpoint, &http))
+   if (pgexporter_http_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, false, &http))
    {
-      pgexporter_log_error("Failed to create HTTP interaction for endpoint %d", endpoint);
+      pgexporter_log_error("Failed to connect to HTTP endpoint %d (%s:%d)",
+                           endpoint,
+                           config->endpoints[endpoint].host,
+                           config->endpoints[endpoint].port);
       goto error;
    }
 
-   if (pgexporter_http_get(http))
+   http->endpoint = endpoint;
+
+   if (pgexporter_http_get(http, config->endpoints[endpoint].host, "/metrics"))
    {
       pgexporter_log_error("Failed to execute HTTP/GET interaction with http://%s:%d/metrics",
                            config->endpoints[endpoint].host,
@@ -142,13 +147,16 @@ pgexporter_prometheus_client_get(int endpoint, struct prometheus_bridge* bridge)
       goto error;
    }
 
-   pgexporter_http_destroy(http);
+   pgexporter_http_disconnect(http);
 
    return 0;
 
 error:
 
-   pgexporter_http_destroy(http);
+   if (http != NULL)
+   {
+      pgexporter_http_disconnect(http);
+   }
 
    return 1;
 }
diff --git a/src/libpgexporter/utils.c b/src/libpgexporter/utils.c
index 24918943..557d7bf4 100644
--- a/src/libpgexporter/utils.c
+++ b/src/libpgexporter/utils.c
@@ -963,6 +963,11 @@ pgexporter_vappend(char* orig, unsigned int n_str, ...)
    }
 
    strings = (char**) malloc(n_str * sizeof(char*));
+   if (strings == NULL)
+   {
+      pgexporter_log_error("malloc failed for strings array");
+      return orig;
+   }
 
    va_start(args, n_str);
 
@@ -972,7 +977,16 @@ pgexporter_vappend(char* orig, unsigned int n_str, ...)
       fin_len += strlen(strings[i]);
    }
 
-   str = (char*) realloc(orig, fin_len + 1);
+   char* new_str = (char*) realloc(orig, fin_len + 1);
+   if (new_str == NULL)
+   {
+      pgexporter_log_error("realloc failed for appended string");
+      free(strings);
+      va_end(args);
+      return orig;
+   }
+
+   str = new_str;
    ptr = str + orig_len;
 
    for (unsigned int i = 0; i < n_str; i++)


Reviews:
user: login: resyfer
body: The changes look fine to me apart from the suggested ones. Please ensure it works well, I'll be testing it as well tomorrow morning.

There's however one thing that bugs me. The functions for various request types are fairly different from each other, while from what I understood as I skimmed through the HTTP 1.1 spec, there is virtually no difference apart from the meaning conveyed to the server. For example, a GET request _may_ have a body:

```
A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.
```

This is from the 2014 spec. Also, please do not refer the original HTTP 1.1 spec mentioned in the review for every specific think, I have added it as it spelled out clearly what I was referring to in that review comment. There is a new 2014 spec, you should refer that if needed.
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2025-05-12T18:09:15Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T04:57:19Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T04:57:27Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T04:57:51Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T05:03:11Z
user: login: jesperpedersen
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T17:05:20Z
user: login: jesperpedersen
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T17:13:26Z
user: login: resyfer
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T17:43:58Z
user: login: resyfer
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T17:47:10Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T05:11:59Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T05:23:35Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T05:26:31Z
user: login: resyfer
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T16:54:01Z
user: login: resyfer
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T19:52:21Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-16T04:54:22Z
user: login: resyfer
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-16T17:18:02Z
user: login: jesperpedersen
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-16T17:37:30Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-16T18:02:11Z
user: login: bassamadnan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-16T18:05:43Z
user: login: jesperpedersen
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-16T18:06:33Z
user: login: resyfer
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-17T05:15:13Z
user: login: jesperpedersen
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-17T09:13:12Z
user: login: jesperpedersen
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-25T12:55:06Z
user: login: jesperpedersen
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-25T14:46:25Z

Review comments:
diff_hunk: @@ -142,13 +147,16 @@ pgexporter_prometheus_client_get(int endpoint, struct prometheus_bridge* bridge)
       goto error;
    }
 
-   pgexporter_http_destroy(http);
+   pgexporter_http_disconnect(http);
path: src/libpgexporter/prometheus_client.c
user: login: resyfer
body: Please do not ignore return values.
created_at: 2025-05-12T17:43:48Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
path: src/libpgexporter/http.c
user: login: resyfer
body: Please check if `pgexporter_vappend` would make this task easier and more readable. Also on that note, please add a check for the return value of `realloc` inside `pgexporter_vappend`.
created_at: 2025-05-12T17:47:30Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: resyfer
body: According to [this](https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.3), the headers are of the form `*(message-header CRLF)`. Are you sure this takes care of the carriage return as well?
created_at: 2025-05-12T17:54:21Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
path: src/libpgexporter/http.c
user: login: resyfer
body: I am not too sure about the libev implementation, but @jesperpedersen is this ok being `strtok` instead of `strtok_r`?
created_at: 2025-05-12T17:56:47Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
    struct message* msg_response = NULL;
    int error = 0;
    int status;
    char* request = NULL;
+   char* full_request = NULL;
    char* response = NULL;
+   char* user_agent = NULL;
+   char* endpoint = path ? path : "/metrics";
 
-   if (build_get_request(http->endpoint, &request))
+   pgexporter_log_trace("Starting pgexporter_http_get");
+   if (http_build_header(PGEXPORTER_HTTP_GET, endpoint, &request))
    {
+      pgexporter_log_error("Failed to build HTTP header");
       goto error;
    }
 
-   msg_request = (struct message*)malloc(sizeof(struct message));
-   if (msg_request == NULL)
-   {
-      goto error;
-   }
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Accept", "text/*");
+   pgexporter_http_add_header(http, "Connection", "close");
 
-   memset(msg_request, 0, sizeof(struct message));
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
 
-   msg_request->data = request;
-   msg_request->length = strlen(request) + 1;
+   memset(&msg_request, 0, sizeof(struct message));
+   msg_request.data = full_request;
+   msg_request.length = strlen(full_request) + 1;
 
    error = 0;
 req:
    if (error < 5)
    {
-      status = pgexporter_write_message(NULL, http->socket, msg_request);
+      status = pgexporter_write_message(http->ssl, http->socket, &msg_request);
       if (status != MESSAGE_STATUS_OK)
       {
          error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
          goto req;
       }
    }
    else
    {
+      pgexporter_log_error("Failed to write after 5 attempts");
       goto error;
    }
 
 res:
-   status = pgexporter_read_block_message(NULL, http->socket, &msg_response);
+   // Get a pointer to the global message structure
+   status = pgexporter_read_block_message(http->ssl, http->socket, &msg_response);
    if (status != MESSAGE_STATUS_ZERO)
    {
       if (status == MESSAGE_STATUS_OK)
       {
-         response = pgexporter_append(response, (char*)msg_response->data);
+         // Get data from the response message
+         if (msg_response && msg_response->data)
+         {
+            response = pgexporter_append(response, (char*)msg_response->data);
+         }
          pgexporter_clear_message();
          goto res;
       }
       else
       {
+         pgexporter_log_error("Error reading response");
          goto error;
       }
    }
 
-   if (msg_response->length > 0)
+   if (msg_response && msg_response->length > 0 && msg_response->data)
    {
       response = pgexporter_append(response, (char*)msg_response->data);
    }
 
-   if (extract_headers_body(response, http))
+   if (http_extract_headers_body(response, http))
    {
+      pgexporter_log_error("Failed to extract headers and body");
       goto error;
    }
 
+   pgexporter_log_debug("HTTP Headers: %s", http->headers ? http->headers : "NULL");
+   pgexporter_log_debug("HTTP Body: %s", http->body ? http->body : "NULL");
+
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
 
-   free(msg_request);
+   free(http->request_headers);
+   http->request_headers = NULL;
 
    return 0;
 
 error:
-
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+   return 1;
+}
 
-   free(msg_request);
+int
+pgexporter_http_connect(char* hostname, int port, bool secure, struct http** result)
path: src/libpgexporter/http.c
user: login: resyfer
body: A small nitpick here. Since you're naming the functions like `pgexporter_http_<method>`, `CONNECT` is an actual HTTP method like `GET` or `POST` btw 😄 also, used a lot in corporates and institutes due to proxy.
created_at: 2025-05-12T18:01:04Z
author_association: COLLABORATOR
diff_hunk: @@ -142,13 +147,16 @@ pgexporter_prometheus_client_get(int endpoint, struct prometheus_bridge* bridge)
       goto error;
    }
 
-   pgexporter_http_destroy(http);
+   pgexporter_http_disconnect(http);
path: src/libpgexporter/prometheus_client.c
user: login: bassamadnan
body: hi, i tried to keep this the same as it was originally, if this reports an error , then inside the error block this would be triggered again (since we call disconnect there too). i think we'd need to refactor it abit do you have any suggestions? I might be missing something
created_at: 2025-05-13T04:57:19Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
path: src/libpgexporter/http.c
user: login: bassamadnan
body: yup, will do!
created_at: 2025-05-13T04:57:27Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: bassamadnan
body: i'll take a look and come back to this
created_at: 2025-05-13T04:57:51Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
    struct message* msg_response = NULL;
    int error = 0;
    int status;
    char* request = NULL;
+   char* full_request = NULL;
    char* response = NULL;
+   char* user_agent = NULL;
+   char* endpoint = path ? path : "/metrics";
 
-   if (build_get_request(http->endpoint, &request))
+   pgexporter_log_trace("Starting pgexporter_http_get");
+   if (http_build_header(PGEXPORTER_HTTP_GET, endpoint, &request))
    {
+      pgexporter_log_error("Failed to build HTTP header");
       goto error;
    }
 
-   msg_request = (struct message*)malloc(sizeof(struct message));
-   if (msg_request == NULL)
-   {
-      goto error;
-   }
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Accept", "text/*");
+   pgexporter_http_add_header(http, "Connection", "close");
 
-   memset(msg_request, 0, sizeof(struct message));
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
 
-   msg_request->data = request;
-   msg_request->length = strlen(request) + 1;
+   memset(&msg_request, 0, sizeof(struct message));
+   msg_request.data = full_request;
+   msg_request.length = strlen(full_request) + 1;
 
    error = 0;
 req:
    if (error < 5)
    {
-      status = pgexporter_write_message(NULL, http->socket, msg_request);
+      status = pgexporter_write_message(http->ssl, http->socket, &msg_request);
       if (status != MESSAGE_STATUS_OK)
       {
          error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
          goto req;
       }
    }
    else
    {
+      pgexporter_log_error("Failed to write after 5 attempts");
       goto error;
    }
 
 res:
-   status = pgexporter_read_block_message(NULL, http->socket, &msg_response);
+   // Get a pointer to the global message structure
+   status = pgexporter_read_block_message(http->ssl, http->socket, &msg_response);
    if (status != MESSAGE_STATUS_ZERO)
    {
       if (status == MESSAGE_STATUS_OK)
       {
-         response = pgexporter_append(response, (char*)msg_response->data);
+         // Get data from the response message
+         if (msg_response && msg_response->data)
+         {
+            response = pgexporter_append(response, (char*)msg_response->data);
+         }
          pgexporter_clear_message();
          goto res;
       }
       else
       {
+         pgexporter_log_error("Error reading response");
          goto error;
       }
    }
 
-   if (msg_response->length > 0)
+   if (msg_response && msg_response->length > 0 && msg_response->data)
    {
       response = pgexporter_append(response, (char*)msg_response->data);
    }
 
-   if (extract_headers_body(response, http))
+   if (http_extract_headers_body(response, http))
    {
+      pgexporter_log_error("Failed to extract headers and body");
       goto error;
    }
 
+   pgexporter_log_debug("HTTP Headers: %s", http->headers ? http->headers : "NULL");
+   pgexporter_log_debug("HTTP Body: %s", http->body ? http->body : "NULL");
+
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
 
-   free(msg_request);
+   free(http->request_headers);
+   http->request_headers = NULL;
 
    return 0;
 
 error:
-
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+   return 1;
+}
 
-   free(msg_request);
+int
+pgexporter_http_connect(char* hostname, int port, bool secure, struct http** result)
path: src/libpgexporter/http.c
user: login: bassamadnan
body: I was not aware of  connect being a method! thanks, very interesting. But I am also not sure - do we need to change something here ? 
created_at: 2025-05-13T05:03:10Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: Both are ok
created_at: 2025-05-13T17:05:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
    struct message* msg_response = NULL;
    int error = 0;
    int status;
    char* request = NULL;
+   char* full_request = NULL;
    char* response = NULL;
+   char* user_agent = NULL;
+   char* endpoint = path ? path : "/metrics";
 
-   if (build_get_request(http->endpoint, &request))
+   pgexporter_log_trace("Starting pgexporter_http_get");
+   if (http_build_header(PGEXPORTER_HTTP_GET, endpoint, &request))
    {
+      pgexporter_log_error("Failed to build HTTP header");
       goto error;
    }
 
-   msg_request = (struct message*)malloc(sizeof(struct message));
-   if (msg_request == NULL)
-   {
-      goto error;
-   }
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Accept", "text/*");
+   pgexporter_http_add_header(http, "Connection", "close");
 
-   memset(msg_request, 0, sizeof(struct message));
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
 
-   msg_request->data = request;
-   msg_request->length = strlen(request) + 1;
+   memset(&msg_request, 0, sizeof(struct message));
+   msg_request.data = full_request;
+   msg_request.length = strlen(full_request) + 1;
 
    error = 0;
 req:
    if (error < 5)
    {
-      status = pgexporter_write_message(NULL, http->socket, msg_request);
+      status = pgexporter_write_message(http->ssl, http->socket, &msg_request);
       if (status != MESSAGE_STATUS_OK)
       {
          error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
          goto req;
       }
    }
    else
    {
+      pgexporter_log_error("Failed to write after 5 attempts");
       goto error;
    }
 
 res:
-   status = pgexporter_read_block_message(NULL, http->socket, &msg_response);
+   // Get a pointer to the global message structure
+   status = pgexporter_read_block_message(http->ssl, http->socket, &msg_response);
    if (status != MESSAGE_STATUS_ZERO)
    {
       if (status == MESSAGE_STATUS_OK)
       {
-         response = pgexporter_append(response, (char*)msg_response->data);
+         // Get data from the response message
+         if (msg_response && msg_response->data)
+         {
+            response = pgexporter_append(response, (char*)msg_response->data);
+         }
          pgexporter_clear_message();
          goto res;
       }
       else
       {
+         pgexporter_log_error("Error reading response");
          goto error;
       }
    }
 
-   if (msg_response->length > 0)
+   if (msg_response && msg_response->length > 0 && msg_response->data)
    {
       response = pgexporter_append(response, (char*)msg_response->data);
    }
 
-   if (extract_headers_body(response, http))
+   if (http_extract_headers_body(response, http))
    {
+      pgexporter_log_error("Failed to extract headers and body");
       goto error;
    }
 
+   pgexporter_log_debug("HTTP Headers: %s", http->headers ? http->headers : "NULL");
+   pgexporter_log_debug("HTTP Body: %s", http->body ? http->body : "NULL");
+
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
 
-   free(msg_request);
+   free(http->request_headers);
+   http->request_headers = NULL;
 
    return 0;
 
 error:
-
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+   return 1;
+}
 
-   free(msg_request);
+int
+pgexporter_http_connect(char* hostname, int port, bool secure, struct http** result)
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: Just know that we don't need a full implementation - just what we need.

But, it wouldn't hurt to use the HTTP/1.1, HTTP/2 and HTTP/3 specifications as bed time reading
created_at: 2025-05-13T17:13:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -142,13 +147,16 @@ pgexporter_prometheus_client_get(int endpoint, struct prometheus_bridge* bridge)
       goto error;
    }
 
-   pgexporter_http_destroy(http);
+   pgexporter_http_disconnect(http);
path: src/libpgexporter/prometheus_client.c
user: login: resyfer
body: Hmm, if it were up to me, I would add retries for this. USUALLY, it should not be a problem though, so it's mostly a nitpick here. Just make sure you do use the returned values....sometimes not heeding them can cause memory related bugs where allocations are expected to take place within subroutines.
created_at: 2025-05-14T17:43:58Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
    struct message* msg_response = NULL;
    int error = 0;
    int status;
    char* request = NULL;
+   char* full_request = NULL;
    char* response = NULL;
+   char* user_agent = NULL;
+   char* endpoint = path ? path : "/metrics";
 
-   if (build_get_request(http->endpoint, &request))
+   pgexporter_log_trace("Starting pgexporter_http_get");
+   if (http_build_header(PGEXPORTER_HTTP_GET, endpoint, &request))
    {
+      pgexporter_log_error("Failed to build HTTP header");
       goto error;
    }
 
-   msg_request = (struct message*)malloc(sizeof(struct message));
-   if (msg_request == NULL)
-   {
-      goto error;
-   }
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Accept", "text/*");
+   pgexporter_http_add_header(http, "Connection", "close");
 
-   memset(msg_request, 0, sizeof(struct message));
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
 
-   msg_request->data = request;
-   msg_request->length = strlen(request) + 1;
+   memset(&msg_request, 0, sizeof(struct message));
+   msg_request.data = full_request;
+   msg_request.length = strlen(full_request) + 1;
 
    error = 0;
 req:
    if (error < 5)
    {
-      status = pgexporter_write_message(NULL, http->socket, msg_request);
+      status = pgexporter_write_message(http->ssl, http->socket, &msg_request);
       if (status != MESSAGE_STATUS_OK)
       {
          error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
          goto req;
       }
    }
    else
    {
+      pgexporter_log_error("Failed to write after 5 attempts");
       goto error;
    }
 
 res:
-   status = pgexporter_read_block_message(NULL, http->socket, &msg_response);
+   // Get a pointer to the global message structure
+   status = pgexporter_read_block_message(http->ssl, http->socket, &msg_response);
    if (status != MESSAGE_STATUS_ZERO)
    {
       if (status == MESSAGE_STATUS_OK)
       {
-         response = pgexporter_append(response, (char*)msg_response->data);
+         // Get data from the response message
+         if (msg_response && msg_response->data)
+         {
+            response = pgexporter_append(response, (char*)msg_response->data);
+         }
          pgexporter_clear_message();
          goto res;
       }
       else
       {
+         pgexporter_log_error("Error reading response");
          goto error;
       }
    }
 
-   if (msg_response->length > 0)
+   if (msg_response && msg_response->length > 0 && msg_response->data)
    {
       response = pgexporter_append(response, (char*)msg_response->data);
    }
 
-   if (extract_headers_body(response, http))
+   if (http_extract_headers_body(response, http))
    {
+      pgexporter_log_error("Failed to extract headers and body");
       goto error;
    }
 
+   pgexporter_log_debug("HTTP Headers: %s", http->headers ? http->headers : "NULL");
+   pgexporter_log_debug("HTTP Body: %s", http->body ? http->body : "NULL");
+
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
 
-   free(msg_request);
+   free(http->request_headers);
+   http->request_headers = NULL;
 
    return 0;
 
 error:
-
    free(request);
+   free(full_request);
    free(response);
+   free(user_agent);
+   free(http->request_headers);
+   http->request_headers = NULL;
+   return 1;
+}
 
-   free(msg_request);
+int
+pgexporter_http_connect(char* hostname, int port, bool secure, struct http** result)
path: src/libpgexporter/http.c
user: login: resyfer
body: @bassamadnan What I was suggesting was having a name that doesn't look like a pgexporter_http_<method> (init maybe?), but well, either way is fine I think.
created_at: 2025-05-14T17:47:10Z
author_association: COLLABORATOR
diff_hunk: @@ -142,13 +147,16 @@ pgexporter_prometheus_client_get(int endpoint, struct prometheus_bridge* bridge)
       goto error;
    }
 
-   pgexporter_http_destroy(http);
+   pgexporter_http_disconnect(http);
path: src/libpgexporter/prometheus_client.c
user: login: bassamadnan
body: yup makes sense, i will add some sort of retry logic while disconnecting just to be safe, or atleast have some sort of logging in place for reporting (error type log).  
created_at: 2025-05-15T05:11:59Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: bassamadnan
body: Yes, @resyfer  it is adding the CRLF (`\r\n`) wherever needed, by default in every http_append, after the entire request too (ctrl+f : `full_request = pgexporter_append(full_request, "\r\n");`) and also initially adding it inside build header function as `HTTP/1.1\r\n`. Just to be sure though we also have test cases for this in pgmoneta currently, so I am hoping its fine.
created_at: 2025-05-15T05:23:34Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
path: src/libpgexporter/http.c
user: login: bassamadnan
body: the problem is - due to this being a port from `pgmoneta` i dont think I should use vappend currently so that the code base is the same for the two projects, i just looked it up on `pgmoneta` and it dosnt seem to have a vappend in its util.h, maybe in the future at some later port i'll update this along with renaming the initial function to `pgmoneta_http_init` Let me know if you think otherwise though, i will however address the realloc issue you mentioned.
created_at: 2025-05-15T05:26:31Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
path: src/libpgexporter/http.c
user: login: resyfer
body: Alright, no issues.
created_at: 2025-05-15T16:54:00Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: resyfer
body: I saw that adding headers adds CRLF. What I was referring here was PARSING the received headers.
created_at: 2025-05-15T19:52:21Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: bassamadnan
body: oh i overlooked that. This is taken from existing implementation where we check if the headers end based on `\r` and starts adding remaining lines to the body, but yes currently the headers would contain the `\r` at the end (eg - `[HTTP/1.1 200 OK\r]`) at the end, should this be removed?  
created_at: 2025-05-16T04:54:22Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: resyfer
body: It would mess up a lot of the things tbh. Length, printing to console, etc. Basically it would be assumed the header ends where it actually should.
created_at: 2025-05-16T17:18:02Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: We should focus on a simple API, and add "utility" functions for `..._http_get_length`, `..._http_to_string`, .. and so on.

It is important that chunking is supported... 
created_at: 2025-05-16T17:37:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: bassamadnan
body: @jesperpedersen @resyfer  just to re-iterate, the current implementation is this (headers contain \r\n) 
```
HTTP/1.1 200 OK\r\nContent-Type: text/plain; version=0.0.1; charset=utf-8\r\nDate: Fri May 16 23:21:17 2025\r\nTransfer-Encoding: chunked\r\n
```

Is this the change you are expecting?
```
HTTP/1.1 200 OK Content-Type: text/plain; version=0.0.1; charset=utf-8 Date: Fri May 16 23:24:28 2025 Transfer-Encoding: chunked 
```

created_at: 2025-05-16T18:02:11Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: bassamadnan
body: Heres the function im passing `http->headers` to for checking this
```c
void print_with_control_chars(char* str)
{
   for (int i = 0; str[i] != '\0'; i++)
   {
      if (str[i] == '\r')
         printf("\\r");
      else if (str[i] == '\n')
         printf("\\n");
      else
         printf("%c", str[i]);
   }
   printf("\n");
}
```
created_at: 2025-05-16T18:05:43Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: @bassamadnan `\r\n` is part of the spec, so that is ok - it is ok that the `to_string` contains these chars as well.

We need a HTTP envelope that follows the spec(s)
created_at: 2025-05-16T18:06:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: resyfer
body: @bassamadnan I am talking about [this function](https://github.com/pgexporter/pgexporter/blob/1dfe139e4a944956e84b15d15e4919b005942393/src/libpgexporter/http.c#L114). You `strtok` at `\n`, which is fine, but that leaves a stray `\r` at the end of each token you get.

Taking your example, when you get headers from the request like:
```
HTTP/1.1 200 OK\r\nContent-Type: text/plain; version=0.0.1; charset=utf-8\r\nDate: Fri May 16 23:21:17 2025\r\nTransfer-Encoding: chunked\r\n
```

The current function will parse it like (at every `\n`):
```
["HTTP/1.1 200 OK\r",
"Content-Type: text/plain; version=0.0.1; charset=utf-8\r",
"Date: Fri May 16 23:21:17 2025\r",
"Transfer-Encoding: chunked\r"]
```

There is nothing wrong with [this function](https://github.com/pgexporter/pgexporter/blob/1dfe139e4a944956e84b15d15e4919b005942393/src/libpgexporter/http.c#L46), which I think you're confusing this with.
created_at: 2025-05-17T05:15:13Z
author_association: COLLABORATOR
diff_hunk: @@ -27,226 +27,874 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
    }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
+   }
+
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
 
-   *http = h;
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: https://github.com/pgexporter/pgexporter/blob/1dfe139e4a944956e84b15d15e4919b005942393/src/libpgexporter/http.c#L147 ?
created_at: 2025-05-17T09:13:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1978,33 +1978,33 @@ pgexporter_conf_set(SSL* ssl __attribute__((unused)), int client_fd, uint8_t com
       }
       else if (!strcmp(key, "metrics_ca_file"))
       {
-            max = strlen(config_value);
-            if (max > MISC_LENGTH - 1)
-            {
-               max = MISC_LENGTH - 1;
-            }
-            memcpy(config->metrics_ca_file, config_value, max);
-            pgexporter_json_put(response, key, (uintptr_t)config->metrics_ca_file, ValueString);
+         max = strlen(config_value);
+         if (max > MISC_LENGTH - 1)
path: src/libpgexporter/configuration.c
user: login: jesperpedersen
body: This need to be `MAX_PATH` instead - all the way through
created_at: 2025-05-25T12:48:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1978,33 +1978,33 @@ pgexporter_conf_set(SSL* ssl __attribute__((unused)), int client_fd, uint8_t com
       }
       else if (!strcmp(key, "metrics_ca_file"))
       {
-            max = strlen(config_value);
-            if (max > MISC_LENGTH - 1)
-            {
-               max = MISC_LENGTH - 1;
-            }
-            memcpy(config->metrics_ca_file, config_value, max);
-            pgexporter_json_put(response, key, (uintptr_t)config->metrics_ca_file, ValueString);
+         max = strlen(config_value);
+         if (max > MISC_LENGTH - 1)
+         {
+            max = MISC_LENGTH - 1;
+         }
+         memcpy(config->metrics_ca_file, config_value, max);
+         pgexporter_json_put(response, key, (uintptr_t)config->metrics_ca_file, ValueString);
       }
       else if (!strcmp(key, "metrics_cert_file"))
       {
-            max = strlen(config_value);
-            if (max > MISC_LENGTH - 1)
-            {
-               max = MISC_LENGTH - 1;
-            }
-            memcpy(config->metrics_cert_file, config_value, max);
-            pgexporter_json_put(response, key, (uintptr_t)config->metrics_cert_file, ValueString);
+         max = strlen(config_value);
+         if (max > MISC_LENGTH - 1)
path: src/libpgexporter/configuration.c
user: login: jesperpedersen
body: Same here -- all file based paths
created_at: 2025-05-25T12:48:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1978,33 +1978,33 @@ pgexporter_conf_set(SSL* ssl __attribute__((unused)), int client_fd, uint8_t com
       }
       else if (!strcmp(key, "metrics_ca_file"))
       {
-            max = strlen(config_value);
-            if (max > MISC_LENGTH - 1)
-            {
-               max = MISC_LENGTH - 1;
-            }
-            memcpy(config->metrics_ca_file, config_value, max);
-            pgexporter_json_put(response, key, (uintptr_t)config->metrics_ca_file, ValueString);
+         max = strlen(config_value);
+         if (max > MISC_LENGTH - 1)
+         {
+            max = MISC_LENGTH - 1;
+         }
+         memcpy(config->metrics_ca_file, config_value, max);
+         pgexporter_json_put(response, key, (uintptr_t)config->metrics_ca_file, ValueString);
       }
       else if (!strcmp(key, "metrics_cert_file"))
       {
-            max = strlen(config_value);
-            if (max > MISC_LENGTH - 1)
-            {
-               max = MISC_LENGTH - 1;
-            }
-            memcpy(config->metrics_cert_file, config_value, max);
-            pgexporter_json_put(response, key, (uintptr_t)config->metrics_cert_file, ValueString);
+         max = strlen(config_value);
+         if (max > MISC_LENGTH - 1)
+         {
+            max = MISC_LENGTH - 1;
+         }
+         memcpy(config->metrics_cert_file, config_value, max);
+         pgexporter_json_put(response, key, (uintptr_t)config->metrics_cert_file, ValueString);
       }
       else if (!strcmp(key, "metrics_key_file"))
       {
-            max = strlen(config_value);
-            if (max > MISC_LENGTH - 1)
-            {
-               max = MISC_LENGTH - 1;
-            }
-            memcpy(config->metrics_key_file, config_value, max);
-            pgexporter_json_put(response, key, (uintptr_t)config->metrics_key_file, ValueString);
+         max = strlen(config_value);
+         if (max > MISC_LENGTH - 1)
path: src/libpgexporter/configuration.c
user: login: jesperpedersen
body: Same
created_at: 2025-05-25T12:48:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -963,6 +963,11 @@ pgexporter_vappend(char* orig, unsigned int n_str, ...)
    }
 
    strings = (char**) malloc(n_str * sizeof(char*));
+   if (strings == NULL)
+   {
+      pgexporter_log_error("malloc fail for strings array");
path: src/libpgexporter/utils.c
user: login: jesperpedersen
body: failed
created_at: 2025-05-25T12:49:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -972,7 +977,16 @@ pgexporter_vappend(char* orig, unsigned int n_str, ...)
       fin_len += strlen(strings[i]);
    }
 
-   str = (char*) realloc(orig, fin_len + 1);
+   char* new_str = (char*) realloc(orig, fin_len + 1);
+   if (new_str == NULL)
+   {
+      pgexporter_log_error("realloc fail for appended string");
path: src/libpgexporter/utils.c
user: login: jesperpedersen
body: Same
created_at: 2025-05-25T12:49:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,886 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: Alway init
created_at: 2025-05-25T12:50:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,886 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
+   }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
    }
 
-   *http = h;
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
+
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: Always try to do an `error:` label at the end so we only have to update in one place in case of an error
created_at: 2025-05-25T12:52:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,886 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
+   }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
    }
 
-   *http = h;
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
+
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
    struct message* msg_response = NULL;
    int error = 0;
    int status;
    char* request = NULL;
+   char* full_request = NULL;
    char* response = NULL;
+   char* user_agent = NULL;
+   char* endpoint = path ? path : "/metrics";
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: Always add `!= NULL`
created_at: 2025-05-25T12:53:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,886 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
+   }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
    }
 
-   *http = h;
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
+
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: Init
created_at: 2025-05-25T12:53:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -27,226 +27,886 @@
  */
 
 /* pgexporter */
-#include "message.h"
 #include <pgexporter.h>
 #include <http.h>
 #include <logging.h>
-#include <memory.h>
 #include <network.h>
-#include <stdlib.h>
+#include <security.h>
 #include <utils.h>
 
-#include <stdio.h>
-#include <string.h>
+/* system */
+#include <errno.h>
+#include <unistd.h>
+#include <openssl/err.h>
 
-static int build_get_request(int endpoint, char** request);
-static int extract_headers_body(char* response, struct http* http);
+static int http_build_header(int method, char* path, char** request);
+static int http_extract_headers_body(char* response, struct http* http);
 
 int
-pgexporter_http_create(int endpoint, struct http** http)
+pgexporter_http_add_header(struct http* http, char* name, char* value)
 {
-   struct http* h = NULL;
-   struct configuration* config = NULL;
+   char* tmp;
 
-   config = (struct configuration*)shmem;
+   tmp = pgexporter_append(http->request_headers, name);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   *http = NULL;
+   tmp = pgexporter_append(http->request_headers, ": ");
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   h = (struct http*)malloc(sizeof(struct http));
+   tmp = pgexporter_append(http->request_headers, value);
+   if (tmp == NULL)
+   {
+      return 1;
+   }
+   http->request_headers = tmp;
 
-   if (h == NULL)
+   tmp = pgexporter_append(http->request_headers, "\r\n");
+   if (tmp == NULL)
    {
-      pgexporter_log_error("Failed to allocate to HTTP");
-      goto error;
+      return 1;
    }
+   http->request_headers = tmp;
 
-   /* Initialize */
-   memset(h, 0, sizeof(struct http));
+   return 0;
+}
 
-   h->endpoint = endpoint;
+static int
+http_build_header(int method, char* path, char** request)
+{
+   char* r = NULL;
+   *request = NULL;
 
-   if (pgexporter_connect(config->endpoints[endpoint].host, config->endpoints[endpoint].port, &h->socket))
+   if (method == PGEXPORTER_HTTP_GET)
    {
-      pgexporter_log_error("Failed to connect to %s:%d",
-                           config->endpoints[endpoint].host,
-                           config->endpoints[endpoint].port);
-      goto error;
+      r = pgexporter_append(r, "GET ");
+   }
+   else if (method == PGEXPORTER_HTTP_POST)
+   {
+      r = pgexporter_append(r, "POST ");
+   }
+   else if (method == PGEXPORTER_HTTP_PUT)
+   {
+      r = pgexporter_append(r, "PUT ");
+   }
+   else
+   {
+      pgexporter_log_error("Invalid HTTP method: %d", method);
+      return 1;
    }
 
-   *http = h;
+   r = pgexporter_append(r, path);
+   r = pgexporter_append(r, " HTTP/1.1\r\n");
+
+   *request = r;
 
    return 0;
+}
 
-error:
+static int
+http_extract_headers_body(char* response, struct http* http)
+{
+   bool header = true;
+   char* p = NULL;
+   char* response_copy = NULL;
 
-   free(h);
+   if (response == NULL)
+   {
+      pgexporter_log_error("Response is NULL");
+      return 1;
+   }
 
-   return 1;
+   response_copy = strdup(response);
+   if (response_copy == NULL)
+   {
+      pgexporter_log_error("Failed to duplicate response string");
+      return 1;
+   }
+
+   p = strtok(response_copy, "\n");
+   while (p != NULL)
+   {
+      if (*p == '\r')
+      {
+         header = false;
+      }
+      else
+      {
+         if (!pgexporter_is_number(p, 16))
+         {
+            if (header)
+            {
+               http->headers = pgexporter_append(http->headers, p);
+               http->headers = pgexporter_append_char(http->headers, '\n');
+            }
+            else
+            {
+               http->body = pgexporter_append(http->body, p);
+               http->body = pgexporter_append_char(http->body, '\n');
+            }
+         }
+      }
+
+      p = strtok(NULL, "\n");
+   }
+
+   free(response_copy);
+   return 0;
 }
 
 int
-pgexporter_http_get(struct http* http)
+pgexporter_http_get(struct http* http, char* hostname, char* path)
 {
-   struct message* msg_request = NULL;
+   struct message msg_request;
    struct message* msg_response = NULL;
    int error = 0;
    int status;
    char* request = NULL;
+   char* full_request = NULL;
    char* response = NULL;
+   char* user_agent = NULL;
+   char* endpoint = path ? path : "/metrics";
 
-   if (build_get_request(http->endpoint, &request))
+   pgexporter_log_trace("Starting pgexporter_http_get");
+   if (http_build_header(PGEXPORTER_HTTP_GET, endpoint, &request))
    {
+      pgexporter_log_error("Failed to build HTTP header");
       goto error;
    }
 
-   msg_request = (struct message*)malloc(sizeof(struct message));
-   if (msg_request == NULL)
-   {
-      goto error;
-   }
+   pgexporter_http_add_header(http, "Host", hostname);
+   user_agent = pgexporter_append(user_agent, "pgexporter/");
+   user_agent = pgexporter_append(user_agent, VERSION);
+   pgexporter_http_add_header(http, "User-Agent", user_agent);
+   pgexporter_http_add_header(http, "Accept", "text/*");
+   pgexporter_http_add_header(http, "Connection", "close");
 
-   memset(msg_request, 0, sizeof(struct message));
+   full_request = pgexporter_append(NULL, request);
+   full_request = pgexporter_append(full_request, http->request_headers);
+   full_request = pgexporter_append(full_request, "\r\n");
 
-   msg_request->data = request;
-   msg_request->length = strlen(request) + 1;
+   memset(&msg_request, 0, sizeof(struct message));
+   msg_request.data = full_request;
+   msg_request.length = strlen(full_request) + 1;
 
    error = 0;
 req:
    if (error < 5)
    {
-      status = pgexporter_write_message(NULL, http->socket, msg_request);
+      status = pgexporter_write_message(http->ssl, http->socket, &msg_request);
       if (status != MESSAGE_STATUS_OK)
       {
          error++;
+         pgexporter_log_debug("Write failed, retrying (%d/5)", error);
          goto req;
       }
    }
    else
    {
+      pgexporter_log_error("Failed to write after 5 attempts");
       goto error;
    }
 
 res:
-   status = pgexporter_read_block_message(NULL, http->socket, &msg_response);
+   // Get a pointer to the global message structure
+   status = pgexporter_read_block_message(http->ssl, http->socket, &msg_response);
    if (status != MESSAGE_STATUS_ZERO)
    {
       if (status == MESSAGE_STATUS_OK)
       {
-         response = pgexporter_append(response, (char*)msg_response->data);
+         // Get data from the response message
+         if (msg_response && msg_response->data)
path: src/libpgexporter/http.c
user: login: jesperpedersen
body: `!= NULL`
created_at: 2025-05-25T12:53:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1909,9 +1909,9 @@ pgexporter_conf_set(SSL* ssl __attribute__((unused)), int client_fd, uint8_t com
          if (strlen(section) > 0)
          {
             max = strlen(config_value);
-            if (max > MISC_LENGTH - 1)
+            if (max > MAX_PATH - 1)
path: src/libpgexporter/configuration.c
user: login: jesperpedersen
body: `pgexporter.h` changes ?
created_at: 2025-05-25T14:46:20Z
author_association: CONTRIBUTOR

PR comments:
user: login: bassamadnan
created_at: 2025-05-13T05:17:39Z
body: Hi,

> This is from the 2014 spec.

Yes, I ran into these issues earlier while using FastAPI for a separate project. But anyway, i dont think this is an issue right? Because we do not store anything in the body for our GET method, just a simple call to the endpoint. Do you want me to include a body as well? I am considering checking out libcurl implementation and see if i can grasp it and get an idea how they do things. This implementation tried to copy the previous one (since GET isn't really used in `pgmoneta`) so it was modified to mathc other signatures. [Here's](https://datatracker.ietf.org/doc/html/rfc7231) what I am reffering to and i think this is what you have cited as well.
edit: taking inspiration from curl.h would be abit of an over kill
author_association: COLLABORATOR
user: login: resyfer
created_at: 2025-05-14T17:49:34Z
body: @bassamadnan Yep, I was just stating that I don't think the implementations of the various methods NEED to be 4 different ones based on the need. It's fine as HTTP is itself a very loose standard, but it saves a lot of trouble on the dev and the maintainer.
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-14T18:29:55Z
body: @resyfer You have to explain a bit more - in pgmoneta we need the different HTTP methods for cloud stuff
author_association: CONTRIBUTOR
user: login: resyfer
created_at: 2025-05-14T19:35:12Z
body: For implementation purposes, the encoding is practically the same across the 4 methods except the method name. Everything else is semantics, and does not change encoding. So what I meant was we do not need 4 separate functions that are supposed to do 99% the same thing.
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-14T19:40:33Z
body: @resyfer But, then we delegate those HTTP methods to a static function with constants - if needed. The key is that it is very clear which HTTP method you are interacting with
author_association: CONTRIBUTOR
user: login: jesperpedersen
created_at: 2025-05-25T12:55:41Z
body: @bassamadnan Please, apply the review comments across the entire patch
author_association: CONTRIBUTOR
user: login: bassamadnan
created_at: 2025-05-25T12:59:55Z
body: @jesperpedersen Thanks ! working on it
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-25T15:11:25Z
body: Although it isn't really part of this patch, we need to align all the file paths to `MAX_PATH`, so you can decide if you want to do that first and rebase or make it a part of this patch -- we are ok either way
author_association: CONTRIBUTOR
user: login: bassamadnan
created_at: 2025-05-25T15:23:05Z
body: @jesperpedersen I just realized, there are two places where the variables for files are (in `server` and `configuration` struct), one uses MISC_LENGTH the other one uses the MAX_PATH, i'll change this one, along with some places in `security.c`  (for eg- `pgexporter_remote_management_scram_sha256`) , found this by `git grep -i MISC_LENGTH`, will see if i can catch others too. Will be happy to push it with this patch, working on it
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-25T15:28:32Z
body: @bassamadnan Yes, precisely, try and find all file based paths variables and fix them
author_association: CONTRIBUTOR
user: login: jesperpedersen
created_at: 2025-05-25T15:39:57Z
body: @bassamadnan Ideally you do this `MAX_PATH` patch first, and then rebase upon on it -- you know how it works, issue -> PR -> ...
author_association: CONTRIBUTOR
user: login: bassamadnan
created_at: 2025-05-25T15:41:36Z
body: @jesperpedersen  understood, lets go with that then :)
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-26T10:26:58Z
body: @bassamadnan You forgot the issue number in the commit subject
author_association: CONTRIBUTOR
user: login: bassamadnan
created_at: 2025-05-26T10:28:39Z
body: > @bassamadnan You forgot the issue number in the commit subject

Oh, im sorry. This does not have a issue created in pgexporter, I'll create one
author_association: COLLABORATOR
user: login: bassamadnan
created_at: 2025-05-26T10:33:00Z
body: @jesperpedersen done, was a fun experiencing rebasing on my own code
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-26T10:40:43Z
body: @bassamadnan Check the changes to `configuration.c` ... What do you see ?
author_association: CONTRIBUTOR
user: login: bassamadnan
created_at: 2025-05-26T10:57:54Z
body: @jesperpedersen I think my `--theirs` was set  to my origin main and not upstream, PTAL, other changes are from uncrustify and one place I mightve missed earlier (it probably got mixed up between different checkouts i had been doing)
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-26T11:04:16Z
body: @bassamadnan Make sure to read ProGit as referenced in the documentation
author_association: CONTRIBUTOR
user: login: jesperpedersen
created_at: 2025-05-26T11:07:29Z
body: @bassamadnan Why are there changes in `configuration.c` ? They should be part of #224 - if you need another pull request for that then send it
author_association: CONTRIBUTOR
user: login: bassamadnan
created_at: 2025-05-26T11:11:07Z
body: > @bassamadnan Why are there changes in `configuration.c` ? They should be part of #224 - if you need another pull request for that then send it

Yes I had missed it earlier, i'll send another PR.
author_association: COLLABORATOR
user: login: bassamadnan
created_at: 2025-05-27T06:26:59Z
body: @jesperpedersen PTAL, if theres nothing else here maybe we can close this?
author_association: COLLABORATOR
user: login: jesperpedersen
created_at: 2025-05-27T11:11:10Z
body: Merged.

Thanks for your contribution !
author_association: CONTRIBUTOR
user: login: jesperpedersen
created_at: 2025-05-27T11:12:04Z
body: @bassamadnan We can keep issue open until we are sure that the API is enough
author_association: CONTRIBUTOR
user: login: jesperpedersen
created_at: 2025-05-27T11:23:06Z
body: @bassamadnan Remember, if there are changes compared to pgmoneta then port them over
author_association: CONTRIBUTOR
user: login: bassamadnan
created_at: 2025-05-27T11:40:00Z
body: @jesperpedersen yes as @resyfer had suggested theres one place where we do use the return value of disconnect now to report error. This was not in place earlier, i will open a PR to port that (or any other change which transpires) after everything here is finalized
author_association: COLLABORATOR
user: login: bassamadnan
created_at: 2025-05-27T11:41:03Z
body: Oh im sorry i just realized this got merged! Checking through phone. I will port that change as well. Thanks for the reviews!
author_association: COLLABORATOR