Repository: ls1intum/tum-apply
Pull request #152: `Development`: Improve usability of import script
Author: login: marc-fett
Labels: name: documentation
description: Improvements or additions to documentation
name: enhancement
description: New feature or request
Created: 2025-05-13T22:05:12Z | Merged: 2025-05-16T15:39:30Z

### Checklist

#### General

- [x] I tested **all** changes and their related features with **all** corresponding user types.
- [x] Language: I followed the [guidelines for inclusive, diversity-sensitive, and appreciative language](https://confluence.aet.cit.tum.de/spaces/AP/pages/257786006/Language+Guidelines+Client).
- [x] I chose a title conforming to the [naming conventions for pull requests](https://confluence.aet.cit.tum.de/spaces/AP/pages/256311714/PR+Guidelines).

#### Server

- [x] **Important**: I implemented the changes with a [very good performance](https://confluence.aet.cit.tum.de/spaces/AP/pages/256311718/Performance+Guidelines) and prevented too many (unnecessary) and too complex database calls.
- [x] I **strictly** followed the principle of **data economy** for all database calls.
- [x] I **strictly** followed the [server coding and design guidelines](https://confluence.aet.cit.tum.de/spaces/AP/pages/256311877/Server+Guidelines).
- [x] I documented the Java code using JavaDoc style.

### Motivation and Context

This PR improves the developer experience when importing test data by:
- Allowing users to choose whether they want to fully reset the database before running test data imports.
- Adding OS-specific instructions for running the import script on macOS, Linux, and Windows.

### Description

This PR includes the following changes:

- ➕ Added `00_drop_all_tables.sql` script that truncates all database tables and resets their auto-increment values.
- 🛠️ Modified `import-testdata.sh` to:
  - Prompt the user whether they want to reset the database.
  - Execute `00_drop_all_tables.sql` only if the user confirms.
  - Exclude `00_drop_all_tables.sql` from the general loop that executes all SQL files.
- 📘 Updated `README.md` to clarify instructions across platforms (macOS, Linux, Windows).

### Steps for Testing

Prerequisites:

1. MySQL database `tumapply` must be running. (Docker)
2. The `mysql` CLI must be installed and in `PATH`.

Steps:
1. Run the `import-testdata.sh` script as described in the `README.md`.
2. Answer `y` when asked about resetting the database.
   - ✅ All data should be deleted, and auto-increment values reset.
3. Run again and answer `n`.
   - ✅ Data should remain untouched; only other SQL files should be executed.
4. Confirm that the script works identically on macOS and Windows (Git Bash).

### Review Progress

#### Code Review

- [x] Code Review 1

#### Manual Tests

- [x] Test 1


Diff:
diff --git a/README.md b/README.md
index 93d06ee4ce..a7720db058 100644
--- a/README.md
+++ b/README.md
@@ -68,11 +68,19 @@ This starts a local MySQL instance with the database `tumapply`.
 
 #### Step 3: Import Example/Test Data
 
+This script automatically imports all SQL files from the `src/main/resources/testdata/` folder, sorted alphabetically.
+
 ```bash
-bash src/main/resources/testdata/import-testdata.sh
+bash ./src/main/resources/testdata/import-testdata.sh
 ```
 
-This script imports example users, roles, and research groups.
+💡 **Platform Notes:**
+
+- **macOS**: Requires MySQL CLI to be installed via Homebrew (`brew install mysql-client`) and
+  available in your `PATH`.
+- **Linux**: Requires `mysql` CLI to be installed (`sudo apt install mysql-client` or equivalent).
+- **Windows**: Use **Git Bash** to run this script. Make sure the MySQL CLI is installed (via MySQL Installer) and the
+  `bin` folder is added to your `PATH`.
 
 ---
 
diff --git a/src/main/resources/testdata/00_drop_all_tables.sql b/src/main/resources/testdata/00_drop_all_tables.sql
new file mode 100644
index 0000000000..e67282c044
--- /dev/null
+++ b/src/main/resources/testdata/00_drop_all_tables.sql
@@ -0,0 +1,23 @@
+-- =============================================
+-- 00_drop_all_tables.sql
+-- Resets the database by truncating all tables
+-- Preconditions:
+--   - All tables must already exist
+--   - This script removes all data and resets auto-increment values
+-- =============================================
+
+-- Disable foreign key checks
+SET FOREIGN_KEY_CHECKS = 0;
+
+-- Truncate all tables
+TRUNCATE TABLE applicants;
+TRUNCATE TABLE application_reviews;
+TRUNCATE TABLE applications;
+TRUNCATE TABLE custom_fields;
+TRUNCATE TABLE jobs;
+TRUNCATE TABLE research_groups;
+TRUNCATE TABLE user_research_group_roles;
+TRUNCATE TABLE users;
+
+-- Enable foreign key checks
+SET FOREIGN_KEY_CHECKS = 1;
diff --git a/src/main/resources/testdata/import-testdata.sh b/src/main/resources/testdata/import-testdata.sh
index 6991f01c99..7eb5901e20 100644
--- a/src/main/resources/testdata/import-testdata.sh
+++ b/src/main/resources/testdata/import-testdata.sh
@@ -1,16 +1,21 @@
 #!/bin/bash
+export MSYS_NO_PATHCONV=1
 
 ###############################################################################
-# 🧪 TumApply Testdata Import Script
+# 🧪 TumApply Test Data Import Script
 #
-# This script imports all test data SQL files located in:
-#   src/main/resources/testdata/ and its subfolders.
+# This script imports all SQL files located in:
+#   src/main/resources/testdata/ and its subdirectories, sorted alphabetically.
+#
+# ✅ Platform-independent:
+#   - Works on macOS, Linux, and Windows (via Git Bash).
 #
 # ⚙️ Usage:
-#   1. Place all your test SQL files inside src/main/resources/testdata/
+#   1. Add your test SQL files inside src/main/resources/testdata/
 #      → Example: src/main/resources/testdata/usermanagement/01_users.sql
 #
-#   2. Run this script: import-testdata.sh
+#   2. Run this script via terminal or Git Bash:
+#      ./import-testdata.sh
 #
 # 🔐 DB Connection:
 #   - Host:     127.0.0.1
@@ -36,30 +41,55 @@ DB_HOST="127.0.0.1"
 DB_PORT="3306"
 
 # Path to testdata SQL files
-SQL_PATH="./"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SQL_PATH="$SCRIPT_DIR"
 
-echo "🚀 Importing SQL test data into MySQL database '$DB_NAME'..."
+echo "Importing SQL test data into MySQL database '$DB_NAME'..."
+echo "Searching for SQL files in: $SQL_PATH"
 
 # Check for mysql CLI
 if ! command -v mysql &> /dev/null
 then
-  echo "❌ mysql CLI not found. Please install MySQL client."
+  echo "mysql CLI not found. Please install MySQL client."
   exit 1
 fi
 
+# Ask user if they want to reset the DB (run drop script)
+while true; do
+  echo "Do you want to reset the database (run 00_drop_all_tables.sql)? (y/n)"
+  read -p "Do you want to reset the database? (y/n): " RESET_DB
+  if [[ "$RESET_DB" == "y" || "$RESET_DB" == "Y" ]]; then
+    DROP_FILE="$SQL_PATH/00_drop_all_tables.sql"
+    if [ -f "$DROP_FILE" ]; then
+      echo "Resetting database..."
+      mysql -h "$DB_HOST" -P "$DB_PORT" -u "$DB_USER" --password="$DB_PASS" "$DB_NAME" < "$DROP_FILE"
+    else
+      echo "Reset script not found at: $DROP_FILE"
+      exit 1
+    fi
+    break
+  elif [[ "$RESET_DB" == "n" || "$RESET_DB" == "N" ]]; then
+    echo "Skipping database reset."
+    break
+  else
+    echo "Invalid input. Please enter y or n."
+  fi
+done
+
 # Find and run only SQL files under testdata folder (and subfolders)
-find "$SQL_PATH" -type f -name "*.sql" | sort | while read file; do
-  echo "📄 Attempting to run: $file"
+# Find and run all SQL files except the reset script
+find "$SQL_PATH" -type f -name "*.sql" ! -name "00_drop_all_tables.sql" | sort | while IFS= read -r file; do
+  echo "Attempting to run: $file"
   mysql -h "$DB_HOST" -P "$DB_PORT" -u "$DB_USER" --password="$DB_PASS" "$DB_NAME" < "$file"
 
   if [ ! -s "$file" ]; then
-    echo "⚠️  Warning: File is empty - $file"
+    echo "WARNING: File is empty - $file"
   fi
 
   if [ $? -ne 0 ]; then
-    echo "❌ Error while importing $file"
+    echo "ERROR while importing $file"
     exit 1
   fi
 done
 
-echo "✅ All test data imported successfully."
+echo "Success: All test data imported successfully."


Reviews:
user: login: Douglas-Sarah
body: Thanks for fixing this issue 👍 
I tested it on windows, where it works as expected.
I just added a nitpick to the part where the database can be reset.
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T22:07:46Z
user: login: marc-fett
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-14T22:11:11Z
user: login: Douglas-Sarah
body: Nicely done, LGTM 👍 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T23:27:51Z
user: login: ani-ib
body: Changes LGTM, script works with "n" and "y" as intended
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T10:56:46Z
user: login: az108
body: LGTM
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-16T15:39:20Z

Review comments:
diff_hunk: @@ -36,30 +41,49 @@ DB_HOST="127.0.0.1"
 DB_PORT="3306"
 
 # Path to testdata SQL files
-SQL_PATH="./"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SQL_PATH="$SCRIPT_DIR"
 
-echo "🚀 Importing SQL test data into MySQL database '$DB_NAME'..."
+echo "Importing SQL test data into MySQL database '$DB_NAME'..."
+echo "Searching for SQL files in: $SQL_PATH"
 
 # Check for mysql CLI
 if ! command -v mysql &> /dev/null
 then
-  echo "❌ mysql CLI not found. Please install MySQL client."
+  echo "mysql CLI not found. Please install MySQL client."
   exit 1
 fi
 
+# Ask user if they want to reset the DB (run drop script)
+echo "Do you want to reset the database (run 00_drop_all_tables.sql)? (y/n)"
+read -r RESET_DB
+if [[ "$RESET_DB" == "y" || "$RESET_DB" == "Y" ]]; then
+  DROP_FILE="$SQL_PATH/00_drop_all_tables.sql"
+  if [ -f "$DROP_FILE" ]; then
+    echo "Resetting database..."
+    mysql -h "$DB_HOST" -P "$DB_PORT" -u "$DB_USER" --password="$DB_PASS" "$DB_NAME" < "$DROP_FILE"
+  else
+    echo "Reset script not found at: $DROP_FILE"
+    exit 1
+  fi
+else
+  echo "Skipping database reset."
+fi
path: src/main/resources/testdata/import-testdata.sh
user: login: Douglas-Sarah
body: ```suggestion
while true; do
  echo "Do you want to reset the database (run 00_drop_all_tables.sql)? (y/n)"
  read -p "Do you want to reset the database? (y/n): " RESET_DB
  if [[ "$RESET_DB" == "y" || "$RESET_DB" == "Y" ]]; then
    DROP_FILE="$SQL_PATH/00_drop_all_tables.sql"
    if [ -f "$DROP_FILE" ]; then
      echo "Resetting database..."
      mysql -h "$DB_HOST" -P "$DB_PORT" -u "$DB_USER" --password="$DB_PASS" "$DB_NAME" < "$DROP_FILE"
    else
      echo "Reset script not found at: $DROP_FILE"
      exit 1
    fi
    break
  elif [[ "$RESET_DB" == "n" || "$RESET_DB" == "N" ]]; then
    echo "Skipping database reset."
    break
  else
    echo "Invalid input. Please enter y or n."
  fi
done
```

Suggestion: at the moment, if someone enters anything else except 'y' or 'Y', it is interpreted as 'n'/'N'.
Standard behaviour would dictate to ask again.
created_at: 2025-05-14T22:04:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -36,30 +41,49 @@ DB_HOST="127.0.0.1"
 DB_PORT="3306"
 
 # Path to testdata SQL files
-SQL_PATH="./"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SQL_PATH="$SCRIPT_DIR"
 
-echo "🚀 Importing SQL test data into MySQL database '$DB_NAME'..."
+echo "Importing SQL test data into MySQL database '$DB_NAME'..."
+echo "Searching for SQL files in: $SQL_PATH"
 
 # Check for mysql CLI
 if ! command -v mysql &> /dev/null
 then
-  echo "❌ mysql CLI not found. Please install MySQL client."
+  echo "mysql CLI not found. Please install MySQL client."
   exit 1
 fi
 
+# Ask user if they want to reset the DB (run drop script)
+echo "Do you want to reset the database (run 00_drop_all_tables.sql)? (y/n)"
+read -r RESET_DB
+if [[ "$RESET_DB" == "y" || "$RESET_DB" == "Y" ]]; then
+  DROP_FILE="$SQL_PATH/00_drop_all_tables.sql"
+  if [ -f "$DROP_FILE" ]; then
+    echo "Resetting database..."
+    mysql -h "$DB_HOST" -P "$DB_PORT" -u "$DB_USER" --password="$DB_PASS" "$DB_NAME" < "$DROP_FILE"
+  else
+    echo "Reset script not found at: $DROP_FILE"
+    exit 1
+  fi
+else
+  echo "Skipping database reset."
+fi
path: src/main/resources/testdata/import-testdata.sh
user: login: marc-fett
body: Thank you, fixed in the latest commit 👍🏼 
created_at: 2025-05-14T22:11:11Z
author_association: MEMBER

PR comments: