Repository: mainmatter/reStations
Pull request #70: Distribute types between controller and db modules
Author: login: oliverbarnes
Created: 2025-03-09T12:41:20Z | Merged: 2025-03-10T22:38:49Z

closes #61

I'm not totally happy with the module layout (eg crate::db::db), but maybe this is a first iteration to get to a better place.

Also, my rust module fu isn't that great - let me know if I can do thing is a less convoluted way.

Linked issue #61: Move types to proper location
Currently, all types are in `src/types` which is a bit weird I'd say (and possibly because this was written by people with a Ruby background to whom types are not natural maybe?) I'd move them into the respective places where they _"belong"_ logically (e.g. data types that resemble db data likely belongs into `src/db.rs`, input and output types of HTTP endpoints belong into the respective controller files etc.)
Linked issue comments: user: login: oliverbarnes
created_at: 2025-03-08T23:47:20Z
body: Agree it's weird. I'd say it was the reverse though - it was actually added by people with a Rust background on 1b3e5d451643a54a28f0dbfa7cda33d7218136df.

It might have had something to do with the fact that the struct is both for mapping CSV and SQLite data (we started with the consumption of CSV data), and that it came before the Gerust project was generated. At the time of generation I think Gerust's db subsystem was deemed overkill. As with sqlx.

There was an effort to keep the project as lean as possible. You know, a Rust thing ;)

The db.rs abstraction was added later (7f403e557970df2a47d5f8a7f2370e6f7fd70330), by a person of Ruby background.

The railsy thing to do would have been to call this a model from the beginning, parallel to controllers. 

author_association: COLLABORATOR

Diff:
diff --git a/.gitignore b/.gitignore
index 9828f25..3ca2cd5 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,6 +3,7 @@
 **/debug
 **/*.pdb
 **/*.rs.bk
+.sqlx/
 
 # Editors
 .idea
diff --git a/web/src/controllers/places.rs b/web/src/controllers/places.rs
index e8174bf..e2f995e 100644
--- a/web/src/controllers/places.rs
+++ b/web/src/controllers/places.rs
@@ -1,12 +1,68 @@
-use crate::db;
+use crate::db::error::DbError;
+use crate::db::finders::*;
+use crate::db::station_record::StationRecord;
 use crate::state::SharedAppState;
-use crate::types::osdm::*;
-use crate::types::station_record::StationRecord;
 use axum::extract::{Path, State};
 use axum::http::StatusCode;
 use axum::response::{IntoResponse, Json, Response};
+use serde::{Deserialize, Serialize};
 use std::convert::From;
 
+#[derive(Deserialize, Serialize, Debug, Clone, PartialEq)]
+pub struct OsdmGeoPosition {
+    pub latitude: f64,
+    pub longitude: f64,
+}
+
+#[derive(Deserialize, Serialize)]
+pub struct OsdmLink {
+    rel: String,
+    href: String,
+    _type: String,
+    value: String,
+}
+
+#[derive(Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct OsdmPlace {
+    pub id: String,
+    pub object_type: String,
+    pub name: String,
+    pub alternative_ids: Vec<String>,
+    pub geo_position: Option<OsdmGeoPosition>,
+    pub _links: Vec<OsdmLink>,
+}
+
+//
+// Requests
+//
+
+#[derive(Deserialize, Serialize)]
+pub struct OsdmInitialPlaceInput {
+    pub name: Option<String>,
+}
+
+#[derive(Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct OsdmPlaceRequest {
+    pub place_input: Option<OsdmInitialPlaceInput>,
+}
+
+//
+// Responses
+//
+
+#[derive(Deserialize, Serialize)]
+pub struct OsdmPlaceResponse {
+    pub places: Vec<OsdmPlace>,
+}
+
+#[derive(Deserialize, Serialize)]
+pub struct OsdmProblem {
+    pub code: String,
+    pub title: String,
+}
+
 pub enum PlacesResponse {
     Ok(OsdmPlaceResponse),
     NotFound(OsdmProblem),
@@ -52,7 +108,7 @@ impl From<Vec<StationRecord>> for OsdmPlaceResponse {
 
 #[axum::debug_handler]
 pub async fn list(State(app_state): State<SharedAppState>) -> PlacesResponse {
-    let places = db::find_all_stations(&app_state.pool)
+    let places = find_all_stations(&app_state.pool)
         .await
         .expect("Unexpected error at places::list");
 
@@ -68,8 +124,8 @@ pub async fn search(
         None => None,
     };
     let query = match maybe_place_name {
-        Some(name) => db::search_all_stations(&app_state.pool, &name).await,
-        None => db::find_all_stations(&app_state.pool).await,
+        Some(name) => search_all_stations(&app_state.pool, &name).await,
+        None => find_all_stations(&app_state.pool).await,
     };
 
     let places = query.expect("Unexpected error at places::search");
@@ -82,9 +138,9 @@ pub async fn show(
     State(app_state): State<SharedAppState>,
     Path(place_id): Path<String>,
 ) -> PlacesResponse {
-    match db::find_station(&app_state.pool, &place_id).await {
+    match find_station(&app_state.pool, &place_id).await {
         Ok(station) => render_place_response(station),
-        Err(db::DbError::RecordNotFound(_msg)) => render_not_found(place_id),
+        Err(DbError::RecordNotFound(_msg)) => render_not_found(place_id),
         _ => todo!("Unexpected error at places::show"),
     }
 }
diff --git a/web/src/db/error.rs b/web/src/db/error.rs
new file mode 100644
index 0000000..c96416a
--- /dev/null
+++ b/web/src/db/error.rs
@@ -0,0 +1,17 @@
+#[derive(serde::Serialize, Debug, thiserror::Error)]
+pub enum DbError {
+    #[error("Unknown error")]
+    UnknownError,
+
+    #[error("Database error: {0}")]
+    Database(String),
+
+    #[error("RecordNotFound: {0}")]
+    RecordNotFound(String),
+}
+
+impl From<sqlx::Error> for DbError {
+    fn from(value: sqlx::Error) -> Self {
+        Self::Database(value.to_string())
+    }
+}
diff --git a/web/src/db.rs b/web/src/db/finders.rs
similarity index 71%
rename from web/src/db.rs
rename to web/src/db/finders.rs
index f36769c..a5133f4 100644
--- a/web/src/db.rs
+++ b/web/src/db/finders.rs
@@ -1,33 +1,6 @@
-use super::types::station_record::StationRecord;
-
-#[derive(serde::Serialize, Debug, thiserror::Error)]
-pub enum DbError {
-    #[error("Unknown error")]
-    UnknownError,
-
-    #[error("Database error: {0}")]
-    Database(String),
-
-    #[error("RecordNotFound: {0}")]
-    RecordNotFound(String),
-}
-
-impl From<sqlx::Error> for DbError {
-    fn from(value: sqlx::Error) -> Self {
-        Self::Database(value.to_string())
-    }
-}
-
-pub use sqlx::sqlite::SqlitePool as DbPool;
-
-pub async fn create_pool(db_file: &str) -> DbPool {
-    let options = sqlx::sqlite::SqliteConnectOptions::new()
-        .filename(db_file)
-        .create_if_missing(false);
-    sqlx::sqlite::SqlitePool::connect_with(options)
-        .await
-        .expect("Failed to create pool")
-}
+use crate::db::error::DbError;
+use crate::db::pool::DbPool;
+use crate::db::station_record::StationRecord;
 
 pub async fn find_station(db: &DbPool, place_id: &String) -> Result<StationRecord, DbError> {
     sqlx::query_as!(
diff --git a/web/src/db/mod.rs b/web/src/db/mod.rs
new file mode 100644
index 0000000..bd2a0bb
--- /dev/null
+++ b/web/src/db/mod.rs
@@ -0,0 +1,4 @@
+pub mod error;
+pub mod finders;
+pub mod pool;
+pub mod station_record;
diff --git a/web/src/db/pool.rs b/web/src/db/pool.rs
new file mode 100644
index 0000000..14c8b7b
--- /dev/null
+++ b/web/src/db/pool.rs
@@ -0,0 +1,10 @@
+pub use sqlx::sqlite::SqlitePool as DbPool;
+
+pub async fn create_pool(db_file: &str) -> DbPool {
+    let options = sqlx::sqlite::SqliteConnectOptions::new()
+        .filename(db_file)
+        .create_if_missing(false);
+    sqlx::sqlite::SqlitePool::connect_with(options)
+        .await
+        .expect("Failed to create pool")
+}
diff --git a/web/src/types/station_record.rs b/web/src/db/station_record.rs
similarity index 100%
rename from web/src/types/station_record.rs
rename to web/src/db/station_record.rs
diff --git a/web/src/error.rs b/web/src/error.rs
index 313be87..7937fc3 100644
--- a/web/src/error.rs
+++ b/web/src/error.rs
@@ -1,7 +1,7 @@
 use axum::{http::StatusCode, response::IntoResponse};
 use std::fmt::{Debug, Display};
 
-use super::db;
+use crate::db::error::DbError;
 
 /// Error type that encapsultes anything that can go wrong
 /// in this application. Implements [IntoResponse],
@@ -18,7 +18,7 @@ pub enum Error {
     Deserialization(#[from] csv_async::Error),
 
     #[error("SQLite error: {0}")]
-    Sqlite(#[from] db::DbError),
+    Sqlite(#[from] DbError),
 
     /// Any other error. Handled as an Internal Server Error.
     #[error("Error: {0}")]
diff --git a/web/src/lib.rs b/web/src/lib.rs
index 40b6ed3..4d8c279 100644
--- a/web/src/lib.rs
+++ b/web/src/lib.rs
@@ -19,8 +19,6 @@ pub mod middlewares;
 pub mod routes;
 /// Contains the application state definition and functionality to initialize it.
 pub mod state;
-/// Bunch of types we need
-pub mod types;
 
 /// Runs the application.
 ///
diff --git a/web/src/state.rs b/web/src/state.rs
index bb527c2..0b9fa5f 100644
--- a/web/src/state.rs
+++ b/web/src/state.rs
@@ -1,10 +1,10 @@
-use super::db;
+use crate::db::pool::{create_pool, DbPool};
 use restations_config::Config;
 use std::sync::Arc;
 
 /// The application's state that is available in [`crate::controllers`] and [`crate::middlewares`].
 pub struct AppState {
-    pub pool: db::DbPool,
+    pub pool: DbPool,
 }
 
 /// The application's state as it is shared across the application, e.g. in controllers and middlewares.
@@ -16,7 +16,7 @@ pub type SharedAppState = Arc<AppState>;
 ///
 /// This function creates an [`AppState`] based on the current [`restations_config::Config`].
 pub async fn init_app_state(_config: Config) -> AppState {
-    let pool = db::create_pool("stations.sqlite.db").await;
+    let pool = create_pool("stations.sqlite.db").await;
 
     AppState { pool }
 }
diff --git a/web/src/test_helpers/mod.rs b/web/src/test_helpers/mod.rs
index d376395..8cf8ca1 100644
--- a/web/src/test_helpers/mod.rs
+++ b/web/src/test_helpers/mod.rs
@@ -1,4 +1,4 @@
-use crate::db;
+use crate::db::pool::{create_pool, DbPool};
 use crate::routes::init_routes;
 use crate::state::AppState;
 use axum::{
@@ -180,7 +180,7 @@ impl BodyExt for Body {
 pub struct TestContext {
     /// The application that is being tested.
     pub app: Router,
-    pub pool: db::DbPool,
+    pub pool: DbPool,
 }
 
 /// Sets up a test and returns a [`TestContext`].
@@ -192,7 +192,7 @@ pub async fn setup() -> TestContext {
     let init_config: OnceCell<Config> = OnceCell::new();
     let _config = init_config.get_or_init(|| load_config(&Environment::Test).unwrap());
 
-    let pool = db::create_pool("../stations.sqlite.db").await;
+    let pool = create_pool("../stations.sqlite.db").await;
     let app = init_routes(AppState { pool: pool.clone() });
 
     TestContext { app, pool }
diff --git a/web/src/types/mod.rs b/web/src/types/mod.rs
deleted file mode 100644
index a5e7a07..0000000
--- a/web/src/types/mod.rs
+++ /dev/null
@@ -1,2 +0,0 @@
-pub mod osdm;
-pub mod station_record;
diff --git a/web/src/types/osdm.rs b/web/src/types/osdm.rs
deleted file mode 100644
index ac94af6..0000000
--- a/web/src/types/osdm.rs
+++ /dev/null
@@ -1,60 +0,0 @@
-use serde::{Deserialize, Serialize};
-
-//
-// Data
-//
-
-#[derive(Deserialize, Serialize, Debug, Clone, PartialEq)]
-pub struct OsdmGeoPosition {
-    pub latitude: f64,
-    pub longitude: f64,
-}
-
-#[derive(Deserialize, Serialize)]
-pub struct OsdmLink {
-    rel: String,
-    href: String,
-    _type: String,
-    value: String,
-}
-
-#[derive(Deserialize, Serialize)]
-#[serde(rename_all = "camelCase")]
-pub struct OsdmPlace {
-    pub id: String,
-    pub object_type: String,
-    pub name: String,
-    pub alternative_ids: Vec<String>,
-    pub geo_position: Option<OsdmGeoPosition>,
-    pub _links: Vec<OsdmLink>,
-}
-
-//
-// Requests
-//
-
-#[derive(Deserialize, Serialize)]
-pub struct OsdmInitialPlaceInput {
-    pub name: Option<String>,
-}
-
-#[derive(Deserialize, Serialize)]
-#[serde(rename_all = "camelCase")]
-pub struct OsdmPlaceRequest {
-    pub place_input: Option<OsdmInitialPlaceInput>,
-}
-
-//
-// Responses
-//
-
-#[derive(Deserialize, Serialize)]
-pub struct OsdmPlaceResponse {
-    pub places: Vec<OsdmPlace>,
-}
-
-#[derive(Deserialize, Serialize)]
-pub struct OsdmProblem {
-    pub code: String,
-    pub title: String,
-}
diff --git a/web/tests/api/places_test.rs b/web/tests/api/places_test.rs
index 7e9bebf..7953081 100644
--- a/web/tests/api/places_test.rs
+++ b/web/tests/api/places_test.rs
@@ -4,8 +4,8 @@ use axum::{
 };
 use googletest::prelude::{assert_that, eq, gt};
 use restations_macros::test;
+use restations_web::controllers::places::*;
 use restations_web::test_helpers::{BodyExt, RouterExt, TestContext};
-use restations_web::types::osdm::*;
 use serde_json::json;
 
 #[test]


Reviews:
user: login: marcoow
body: I think it's mostly fine although it seems like we're trying to avoiding having struct definitions in the same places as functions (or the structs' trait implementations) – I'm not a Rust guru but I'm not sure this might necessarily makes it easier to work with the code. I don't know what's idiomatic Rust in that regard though – I'm sure @LukeMathWalker and @hdoordt can give more input.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T07:09:27Z
user: login: oliverbarnes
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-10T11:54:03Z
user: login: pablobm
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-10T15:52:13Z
user: login: marcoow
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T16:01:05Z
user: login: pablobm
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-10T16:03:36Z
user: login: marcoow
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T16:30:13Z
user: login: oliverbarnes
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-10T22:27:53Z
user: login: oliverbarnes
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-10T22:31:12Z

Review comments:
path: web/src/controllers/osdm.rs
user: login: marcoow
body: I think it's a bit weird when this is a file in `controllers` but there's not actually a controller in it. I'd move the types defined in here into `controllers/places.rs` since that's where they are used (for inputs and outputs)
created_at: 2025-03-10T07:07:01Z
author_association: MEMBER
path: web/src/controllers/osdm.rs
user: login: oliverbarnes
body: Good points, will do
created_at: 2025-03-10T11:54:03Z
author_association: COLLABORATOR
path: web/src/controllers/osdm.rs
user: login: pablobm
body: Initially I put the types in a dedicated `osdm` file because I wanted to highlight that they are defined by an external entity (the OSDM spec). Is there place in the gerust hierarchy for a module like this? In a Rails app I would have put it in `/app/lib`.
created_at: 2025-03-10T15:52:13Z
author_association: COLLABORATOR
path: web/src/controllers/osdm.rs
user: login: marcoow
body: In this case I'd say they should be in `controllers/places.rs` since that's the (only) place where they are used. Otherwise I'd say more Rust-idiomatic would be to have sth. like `src/osdm/mod.rs` and put stuff there.
created_at: 2025-03-10T16:01:05Z
author_association: MEMBER
path: web/src/db/error.rs
user: login: pablobm
body: Should the `db` module live in the top level, alongside `web`, as described at https://gerust.rs/docs/architecture/the-db-crate?

On the other hand, we are not using the DB quite the Gerust way... but perhaps we should try to match it? (Not necessarily in this PR).
created_at: 2025-03-10T16:03:36Z
author_association: COLLABORATOR
path: web/src/db/error.rs
user: login: marcoow
body: I think eventually we want to switch to the full Gerust DB structure since that's also going to give us per-test DBs etc.
created_at: 2025-03-10T16:30:13Z
author_association: MEMBER
path: web/src/controllers/osdm.rs
user: login: oliverbarnes
body: Moved OSDM types into places.rs.

You know, I'm not sure we need to highlight what's OSDM or not, given the whole API is meant to be OSDM compliant. Initially I saw the distinction between the stations data we get from Trainline's CSV and the OSDM mappings, but really the whole thing is OSDM and we just happen to have an external data source (that perhaps we could namespace).

Come to think of it, it would be more descriptive to call the whole project OSDMPlacesAPI :) but that's a mouthful and a little boring as a name... I'll resolve this thread and we can discuss further on slack or another issue
created_at: 2025-03-10T22:27:53Z
author_association: COLLABORATOR
path: web/src/db/error.rs
user: login: oliverbarnes
body: Created an issue for it, to be tackled on a separate PR: https://github.com/mainmatter/reStations/issues/77
created_at: 2025-03-10T22:31:12Z
author_association: COLLABORATOR

PR comments:
user: login: oliverbarnes
created_at: 2025-03-09T12:54:03Z
body: Ok, clippy doesn't like `db::db` - will fix later today
author_association: COLLABORATOR
user: login: oliverbarnes
created_at: 2025-03-10T00:25:44Z
body: I think I found a better break down of the modules, easier to reason about. Thanks, clippy.  Let me know what you guys think.

(Sorry for the notifications spam, Marco - should've requested your review after the build passed)
author_association: COLLABORATOR
user: login: oliverbarnes
created_at: 2025-03-10T22:38:37Z
body: I'm going ahead and merging this as all comments were addressed. We can iterate afterwards if Henk or Luca have any suggestions for changes. Switching to Gerust DB structure is captured a new issue: #77 
author_association: COLLABORATOR