Repository: NTHU-SA/NTHU-Data-API
Pull request #168: feat: 改用異步方法來調用 data scraper 的資料
Author: login: iwtba4188
Labels: name: enhancement
description: New feature or request
Created: 2025-03-01T16:40:05Z | Merged: 2025-03-02T11:44:24Z

<!---
- **ACTION REQUIRED: Remove empty sections before submitting.**
- Briefly describe the purpose and changes introduced by this PR.
-->

<!--- Add or modify features.
      Including Commit Tags: feat, style, perf, test, revert.
      -->
### Features
- 新增 util functions，現在從 NTHU-Data-Scraper 獲取資料會使用異步方法，並使用更快速的 local cache。
- 修改 buses 和 courses 端口並使用新的異步方法，現在異常狀況發生時不會阻塞其他端口了。
- 新增 buses 和 courses 端口的 response headers，現在它們會同時回傳資料的版本（對應 NTHU-Data-Scraper 的 commit hash）。

<!--- Documentation changes.
      Including Commit Tags: docs.
      -->
### Documentations
- 新增 PR 和 Issue 中 Bug Report 和 Feature Request 的模板。


Diff:
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml
new file mode 100644
index 0000000..171f1d3
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.yml
@@ -0,0 +1,35 @@
+name: Bug Report
+description: File a bug report.
+title: "bug: "
+labels: ["bug"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to fill out this bug report!
+
+  - type: textarea
+    id: current-behavior
+    attributes:
+      label: Current Behavior
+      placeholder: Tell us what you see!
+    validations:
+      required: true
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: More description about this bug.
+      placeholder: Tell us more infomation!
+    validations:
+      required: true
+
+  - type: textarea
+    id: possible-solution
+    attributes:
+      label: Possible Solution
+      description: Any possible solution about this bug.
+      placeholder: Tell us your ideas about this bug!
+    validations:
+      required: false
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
new file mode 100644
index 0000000..3ba13e0
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -0,0 +1 @@
+blank_issues_enabled: false
diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml
new file mode 100644
index 0000000..d127ebf
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.yml
@@ -0,0 +1,35 @@
+name: Feature Request
+description: File a feature request.
+title: "feat: "
+labels: ["enhancement"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to fill out this feature request!
+
+  - type: textarea
+    id: what-feature
+    attributes:
+      label: What Feature You Want
+      placeholder: Tell us what you think!
+    validations:
+      required: true
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: More description about this feature request.
+      placeholder: Tell us more infomation!
+    validations:
+      required: false
+
+  - type: textarea
+    id: possible-solution
+    attributes:
+      label: Possible Solution
+      description: Any possible solution about this feature.
+      placeholder: Tell us your ideas about this feature!
+    validations:
+      required: false
diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md
new file mode 100644
index 0000000..d8828b9
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md
@@ -0,0 +1,38 @@
+<!---
+- **ACTION REQUIRED: Remove empty sections before submitting.**
+- Briefly describe the purpose and changes introduced by this PR.
+-->
+
+<!--- Add or modify features.
+      Including Commit Tags: feat, style, perf, test, revert.
+      -->
+### Features
+- <something>
+
+<!--- Bug fixes.
+      Including Commit Tags: fix.
+      -->
+### Fixes
+- <something>
+
+<!--- Code changes that neither fixes a bug nor adds a feature.
+      Including Commit Tags: refactor, update.
+      -->
+### Refactors
+- <something>
+
+<!--- Changes to the build process or auxiliary tools and libraries.
+      Including Commit Tags: chore.
+      -->
+### Internal
+- <something>
+
+<!--- Documentation changes.
+      Including Commit Tags: docs.
+      -->
+### Documentations
+- <something>
+
+<!--- Any other notes. -->
+### Notes
+- <something>
diff --git a/README.md b/README.md
index 37d93f3..f9b13c0 100644
--- a/README.md
+++ b/README.md
@@ -63,7 +63,7 @@ We follow certain guidelines for contributing. Here are the types of commits we
 - docs: Documentation changes.
 - style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi colons, etc).
 - refactor: Code changes that neither fixes a bug nor adds a feature.
-- update: Document or data changes that neither fixes a bug nor adds a feature.
+- update: Data changes that neither fixes a bug nor adds a feature.
 - perf: Code changes that improve performance.
 - test: Adding missing tests.
 - chore: Changes to the build process or auxiliary tools and libraries.
diff --git a/requirements.txt b/requirements.txt
index f58ccd1..9689391 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6,6 +6,7 @@ python-dotenv==1.0.1
 cachetools==5.5.1
 pyopenssl==24.2.1
 # Crawlers
+httpx==0.28.1
 requests==2.32.3
 beautifulsoup4==4.13.3
 xmltodict==0.13.0
diff --git a/src/api/models/buses.py b/src/api/models/buses.py
index f6a5635..2100ec1 100644
--- a/src/api/models/buses.py
+++ b/src/api/models/buses.py
@@ -1,7 +1,6 @@
 from __future__ import annotations
 
 import os
-import time
 from dataclasses import dataclass, field
 from datetime import datetime, timedelta
 from functools import reduce
@@ -9,8 +8,8 @@
 from typing import Any, Dict, List, Literal, Optional
 
 import pandas as pd
-import requests
 
+from src import utils
 from src.api import schemas
 
 # ---------------------------------------------------------------------------
@@ -290,60 +289,13 @@ def __init__(self) -> None:
                 [BUS_ROUTE_TYPE_WITHOUT_ALL, BUS_DIRECTION]
             ),
         )
+        self.last_commit_hash = None
+
         self._res_json: dict = {}  # 儲存原始 JSON 回應資料
         self._start_from_gen_2_bus_info: List[str] = []  # 記錄從綜二館發車的班次資訊
         self._last_updated_time: Optional[float] = None  # 上次資料更新時間戳記
-        self._initialize_data()
-
-    def _initialize_data(self) -> None:
-        """
-        初始化公車資料。
-
-        檢查快取資料是否過期，若過期或尚未初始化，則從遠端 JSON 端點獲取最新的公車時刻表資料。
-        資料獲取成功後，會進行資料轉換與處理，並更新上次更新時間戳記。
-        """
-        if not self._res_json or self._is_data_expired():
-            data = self._fetch_bus_data()
-            if data:
-                self._res_json = data
-                self._process_bus_data()
-                self._last_updated_time = time.time()  # 更新時間戳記
-
-    def _is_data_expired(self) -> bool:
-        """
-        檢查快取資料是否過期。
-
-        根據設定的 `DATA_TTL_HOURS` 判斷資料是否需要更新。
-
-        Returns:
-            bool: 若資料已過期或尚未初始化，則返回 True，否則返回 False。
-        """
-        if self._last_updated_time is None:
-            return True  # 尚未載入過資料
-        return time.time() - self._last_updated_time > DATA_TTL_HOURS * 3600
-
-    def _fetch_bus_data(self) -> Optional[dict]:
-        """
-        從遠端 JSON 端點獲取公車時刻表資料。
-
-        使用 `requests` 函式庫向 `JSON_URL` 發送 GET 請求，並處理可能的連線錯誤。
-
-        Returns:
-            Optional[dict]: 若成功獲取資料，則返回 JSON 格式的公車時刻表資料，否則返回 None。
-        """
-        try:
-            response = requests.get(
-                JSON_URL,
-                timeout=10,
-                headers={"User-Agent": "Mozilla/5.0"},
-            )
-            response.raise_for_status()  # 針對 HTTP 錯誤狀態碼拋出例外
-            return response.json()
-        except requests.RequestException as e:
-            print(f"Error fetching data from {JSON_URL}: {e}")
-            return None
 
-    def _process_bus_data(self) -> None:
+    async def _process_bus_data(self) -> None:
         """
         處理從 JSON API 獲取的公車時刻表資料。
 
@@ -353,7 +305,7 @@ def _process_bus_data(self) -> None:
         self._populate_info_data()
         self._populate_raw_schedule_data()
         self._add_fields_to_raw_schedule_data()
-        self.gen_bus_detailed_schedule_and_update_stops_data()
+        await self.gen_bus_detailed_schedule_and_update_stops_data()
 
     def _populate_info_data(self) -> None:
         """
@@ -396,7 +348,7 @@ def _classify_bus_type(
             description (str): 時刻表註解，用於判斷公車類型。
 
         Returns:
-            schemas.buses.BusType: 公車類型，包含 'route_83', 'tour_bus', 'middle_sized_bus'。
+            schemas.buses.BusType: 公車類型，包含 'route_83', 'large-sized_bus', 'middle-sized_bus'。
         """
         # 優先判斷是否為 83 路
         if route_type == "nanda" and "83" in description:
@@ -406,7 +358,7 @@ def _classify_bus_type(
         elif (route_type == "main" and "大" in description) or (
             route_type == "nanda" and day == "weekday"
         ):
-            return schemas.buses.BusType.tour_bus
+            return schemas.buses.BusType.large_sized_bus
 
         # 其他校內和校區區間公車為中型巴士
         else:
@@ -510,13 +462,19 @@ def _reset_stop_data(self) -> None:
                 index=schedule_index,
             )
 
-    def _update_data(self) -> None:
+    async def update_data(self) -> None:
         """更新公車時刻表資料，包含從 API 獲取最新資料並重新處理。"""
-        self._res_json = (
-            self._fetch_bus_data()
+        # asyncio.gather(self._init_task)  # 等待初始化任務完成
+
+        res_commit_hash, self._res_json = await utils.get(
+            "buses.json"
         )  # 直接更新 _res_json，後續處理會使用最新的 json 資料
-        if self._res_json:  # 只有成功獲取資料才需要重新處理
-            self._process_bus_data()
+
+        if (
+            self._res_json and res_commit_hash != self.last_commit_hash
+        ):  # 只有成功獲取資料且資料不一致時才需要重新處理
+            await self._process_bus_data()
+            self.last_commit_hash = res_commit_hash
         self._start_from_gen_2_bus_info.clear()  # 清空從綜二館發車的班次資訊快取
 
     def _add_on_time(self, start_time: str, time_delta: int) -> str:
@@ -800,7 +758,7 @@ def _update_stop_stopped_bus_data(
                 }
             )
 
-    def gen_bus_detailed_schedule_and_update_stops_data(self) -> None:
+    async def gen_bus_detailed_schedule_and_update_stops_data(self) -> None:
         """
         產生詳細公車時刻表並更新各站點的停靠公車資訊。
 
@@ -808,7 +766,7 @@ def gen_bus_detailed_schedule_and_update_stops_data(self) -> None:
         產生詳細時刻表的過程會計算每個班次在每個站點的預計抵達時間。
         """
         self._reset_stop_data()  # 清空站點的停靠公車資料，準備重新計算
-        # self._update_data()  # 資料更新移至 _process_bus_data 中處理
+        # await self.update_data()  # 又被移回來了：Ｄ ~~資料更新移至 _process_bus_data 中處理~~
 
         for route_type, day, direction in product(
             BUS_ROUTE_TYPE_WITHOUT_ALL, BUS_DAY_WITHOUT_ALL, BUS_DIRECTION
diff --git a/src/api/models/courses.py b/src/api/models/courses.py
index 76cfaba..13db792 100644
--- a/src/api/models/courses.py
+++ b/src/api/models/courses.py
@@ -1,12 +1,9 @@
-import json
 import operator
-import os
 import re
 from dataclasses import dataclass, field
 from typing import Any, List, Optional, Union
 
-import requests
-from cachetools import TTLCache, cached
+from src import utils
 
 
 # =============================================================================
@@ -203,32 +200,19 @@ def accept(self, course: CoursesData) -> bool:
 class Processor:
     """
     課程資料處理
-    資料來源：https://api-json.nthusa.tw/courses/lastest.json
     """
 
-    def __init__(self, json_path: Optional[str] = None) -> None:
-        self.NTHU_COURSE_DATA_URL = (
-            os.getenv("NTHU_DATA_URL", "https://data.nthusa.tw")
-            + "/courses/latest.json"
-        )
-        self.course_data: List[CoursesData] = self._get_course_data(json_path)
+    def __init__(self) -> None:
+        self.course_data: List[CoursesData] = []
+        self.last_commit_hash = None
 
-    @cached(cache=TTLCache(maxsize=1, ttl=60 * 60))
-    def _get_course_data(self, json_path: Optional[str] = None) -> List[CoursesData]:
-        """
-        取得課程資料，若提供 json_path 則由檔案讀取，
-        否則從遠端 URL 取得資料。
-        """
-        if json_path:
-            with open(json_path, "r", encoding="utf-8") as f:
-                course_data_dict_list = json.load(f)
-        else:
-            course_data_resp = requests.get(self.NTHU_COURSE_DATA_URL)
-            course_data_dict_list = course_data_resp.json()
-        return [CoursesData.from_dict(data) for data in course_data_dict_list]
+    async def update_data(self) -> None:
+        self.last_commit_hash, self.course_data = await utils.get("courses.json")
+
+        # 將 dict 轉換為 CoursesData 物件
+        self.course_data = list(map(CoursesData.from_dict, self.course_data))
 
-    def update(self, json_path: Optional[str] = None) -> None:
-        self.course_data = self._get_course_data(json_path)
+        print(self.last_commit_hash)
 
     def list_selected_fields(self, field: str) -> List[str]:
         """回傳所有課程中指定欄位的非空字串集合"""
@@ -290,7 +274,7 @@ def query(self, conditions: Conditions) -> List[CoursesData]:
     }
     # 可用 sample_data 測試 from_dict：
     course_sample = CoursesData.from_dict(sample_data)
-    logger.info("轉換後的課程資料: {}", course_sample)
+    print("轉換後的課程資料: {}", course_sample)
 
     # 以下為原有的查詢條件範例
     # 範例 1：中文課名為 "文化人類學專題" 且課號為 "11210ANTH651000"
@@ -298,21 +282,21 @@ def query(self, conditions: Conditions) -> List[CoursesData]:
         "id", "11210ANTH651000"
     )
     result1 = processor.query(condition1)
-    logger.info("中文課名 與 ID (有一堂課): {}", len(result1))
-    logger.info(result1)
+    print("中文課名 與 ID (有一堂課): {}", len(result1))
+    print(result1)
 
     # 範例 2：中文課名為 "化人類學專題" 或課號為 "11210ANTH651000"
     condition2 = Conditions("chinese_title", "化人類學專題") | Conditions(
         "id", "11210ANTH651000"
     )
     result2 = processor.query(condition2)
-    logger.info("中文課名 或 ID (有一堂課): {}", len(result2))
-    logger.info(result2)
+    print("中文課名 或 ID (有一堂課): {}", len(result2))
+    print(result2)
 
     # 範例 3：中文課名包含 "產業"
     condition3 = Conditions("chinese_title", "產業", regex_match=True)
     result3 = processor.query(condition3)
-    logger.info("中文課名包含 '產業' 的課程 (取前5筆): {}", result3[:5])
+    print("中文課名包含 '產業' 的課程 (取前5筆): {}", result3[:5])
 
     # 範例 4：中文課名包含 "產業" 且 credit 為 "2" 且課號包含 "GE"（通識課程）
     condition4 = (
@@ -321,20 +305,20 @@ def query(self, conditions: Conditions) -> List[CoursesData]:
         & Conditions("id", "GE", regex_match=True)
     )
     result4 = processor.query(condition4)
-    logger.info("符合複合條件的課程 (取前5筆): {}", result4[:5])
+    print("符合複合條件的課程 (取前5筆): {}", result4[:5])
 
-    logger.info("總開課數: {}", len(processor.course_data))
+    print("總開課數: {}", len(processor.course_data))
     # 範例 5：中文授課 或 英文授課
     condition_ce = Conditions("language", "中") | Conditions("language", "英")
     result_ce = processor.query(condition_ce)
-    logger.info("中文授課 或 英文授課 開課數量: {}", len(result_ce))
+    print("中文授課 或 英文授課 開課數量: {}", len(result_ce))
 
     # 範例 6：中文授課
     condition_c = Conditions("language", "中")
     result_c = processor.query(condition_c)
-    logger.info("中文授課 開課數量: {}", len(result_c))
+    print("中文授課 開課數量: {}", len(result_c))
 
     # 範例 7：英文授課
     condition_e = Conditions("language", "英")
     result_e = processor.query(condition_e)
-    logger.info("英文授課 開課數量: {}", len(result_e))
+    print("英文授課 開課數量: {}", len(result_e))
diff --git a/src/api/routers/buses.py b/src/api/routers/buses.py
index ec8edd9..6080e11 100644
--- a/src/api/routers/buses.py
+++ b/src/api/routers/buses.py
@@ -1,15 +1,40 @@
+from contextlib import asynccontextmanager
 from datetime import datetime
+from functools import wraps
 from typing import Literal
 
-from fastapi import APIRouter, Depends, HTTPException
+from fastapi import APIRouter, Depends, FastAPI, HTTPException, Response
 
 from src.api import constant, schemas
 from src.api.models.buses import Buses, after_specific_time, stops
 
-router = APIRouter()
 buses = Buses()
 
 
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    """FastAPI 的生命週期管理器，用於在應用啟動時更新公車資料。"""
+    global buses
+
+    # tasks when app starts
+    await buses.update_data()
+    yield
+    # tasks when app stops
+
+
+router = APIRouter(lifespan=lifespan)
+
+
+async def add_custom_header(response: Response):
+    """添加 X-Data-Commit-Hash 標頭。
+
+    Args:
+        response: FastAPI 的 Response 對象。
+    """
+
+    response.headers["X-Data-Commit-Hash"] = str(buses.last_commit_hash)
+
+
 def get_current_time_state():
     """
     取得目前時間狀態 (星期別, 時間)。
@@ -23,11 +48,17 @@ def get_current_time_state():
     return current_day, current_time
 
 
-@router.get("/main", response_model=schemas.buses.BusMainData, name="校本部公車資訊")
-def get_main_bus_data():
+@router.get(
+    "/main",
+    response_model=schemas.buses.BusMainData,
+    name="校本部公車資訊",
+    dependencies=[Depends(add_custom_header)],
+)
+async def get_main_bus_data():
     """
     取得校本部公車資訊。
     """
+    await buses.update_data()
     try:
         return buses.get_main_data()
     except Exception as e:
@@ -37,12 +68,16 @@ def get_main_bus_data():
 
 
 @router.get(
-    "/nanda", response_model=schemas.buses.BusNandaData, name="南大校區區間車資訊"
+    "/nanda",
+    response_model=schemas.buses.BusNandaData,
+    name="南大校區區間車資訊",
+    dependencies=[Depends(add_custom_header)],
 )
-def get_nanda_bus_data():
+async def get_nanda_bus_data():
     """
     取得南大校區區間車資訊。
     """
+    await buses.update_data()
     try:
         return buses.get_nanda_data()
     except Exception as e:
@@ -55,8 +90,9 @@ def get_nanda_bus_data():
     "/info/{bus_type}/{direction}",
     response_model=list[schemas.buses.BusInfo],
     name="公車路線資訊",
+    dependencies=[Depends(add_custom_header)],
 )
-def get_bus_route_info(
+async def get_bus_route_info(
     bus_type: Literal["main", "nanda"] = constant.buses.BUS_TYPE_PATH,
     direction: schemas.buses.BusDirection = constant.buses.BUS_DIRECTION_PATH,
 ):
@@ -73,6 +109,7 @@ def get_bus_route_info(
     Raises:
         HTTPException: 如果無法取得公車路線資訊，則拋出 500 錯誤。
     """
+    await buses.update_data()
     try:
         data = buses.info_data.loc[(bus_type, direction), "data"]
         return data
@@ -92,14 +129,16 @@ def get_bus_route_info(
     "/info/stops",
     response_model=list[schemas.buses.BusStopsInfo],
     name="停靠站資訊",
+    dependencies=[Depends(add_custom_header)],
 )
-def get_bus_stops_info():
+async def get_bus_stops_info():
     """
     取得所有公車停靠站點的資訊。
 
     Returns:
         list[schemas.buses.BusStopsInfo]: 公車停靠站點資訊列表。
     """
+    await buses.update_data()
     try:
         return buses.gen_bus_stops_info()
     except Exception as e:
@@ -114,8 +153,9 @@ def get_bus_stops_info():
         schemas.buses.BusMainSchedule | schemas.buses.BusNandaSchedule | None
     ],
     name="公車時刻表",
+    dependencies=[Depends(add_custom_header)],
 )
-def get_bus_schedule(
+async def get_bus_schedule(
     bus_type: schemas.buses.BusRouteType = constant.buses.BUS_TYPE_QUERY,
     day: schemas.buses.BusDayWithCurrent = constant.buses.BUS_DAY_QUERY,
     direction: schemas.buses.BusDirection = constant.buses.BUS_DIRECTION_QUERY,
@@ -134,6 +174,7 @@ def get_bus_schedule(
     Raises:
         HTTPException: 如果無法取得公車時刻表，則拋出 404 或 500 錯誤。
     """
+    await buses.update_data()
     try:
         if day != "current":
             schedule_data = buses.raw_schedule_data.loc[
@@ -166,8 +207,9 @@ def get_bus_schedule(
     "/stops/{stop_name}/",
     response_model=list[schemas.buses.BusStopsQueryResult | None],
     name="站點停靠公車資訊",
+    dependencies=[Depends(add_custom_header)],
 )
-def get_stop_bus_info(
+async def get_stop_bus_info(
     stop_name: schemas.buses.StopsName = constant.buses.STOPS_NAME_PATH,
     bus_type: schemas.buses.BusRouteType = constant.buses.BUS_TYPE_QUERY,
     day: schemas.buses.BusDayWithCurrent = constant.buses.BUS_DAY_QUERY,
@@ -190,6 +232,7 @@ def get_stop_bus_info(
     Raises:
         HTTPException: 如果無法取得站點停靠公車資訊，則拋出 404 或 500 錯誤。
     """
+    await buses.update_data()
     return_limit = (
         query.limits
         if day != "current"
@@ -231,8 +274,9 @@ def get_stop_bus_info(
         | None
     ],
     name="詳細公車時刻表",
+    dependencies=[Depends(add_custom_header)],
 )
-def get_bus_detailed_schedule(
+async def get_bus_detailed_schedule(
     bus_type: schemas.buses.BusRouteType = constant.buses.BUS_TYPE_QUERY,
     day: schemas.buses.BusDayWithCurrent = constant.buses.BUS_DAY_QUERY,
     direction: schemas.buses.BusDirection = constant.buses.BUS_DIRECTION_QUERY,
@@ -253,6 +297,7 @@ def get_bus_detailed_schedule(
     Raises:
         HTTPException: 如果無法取得詳細公車時刻表，則拋出 404 或 500 錯誤。
     """
+    await buses.update_data()
     return_limit = (
         query.limits
         if day != "current"
diff --git a/src/api/routers/courses.py b/src/api/routers/courses.py
index 6feb2e9..069f4b3 100644
--- a/src/api/routers/courses.py
+++ b/src/api/routers/courses.py
@@ -1,13 +1,51 @@
-from fastapi import APIRouter, Body, HTTPException, Path, Query, Response
+from contextlib import asynccontextmanager
+
+from fastapi import (
+    APIRouter,
+    Body,
+    Depends,
+    FastAPI,
+    HTTPException,
+    Path,
+    Query,
+    Response,
+)
 
 from src.api import constant, schemas
 from src.api.models.courses import Conditions, Processor
 
-router = APIRouter()
 courses = Processor()
 
 
-@router.get("/", response_model=list[schemas.courses.CourseData])
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    """FastAPI 的生命週期管理器，用於在應用啟動時更新公車資料。"""
+    global courses
+
+    # tasks when app starts
+    await courses.update_data()
+    yield
+    # tasks when app stops
+
+
+router = APIRouter(lifespan=lifespan)
+
+
+async def add_custom_header(response: Response):
+    """添加 X-Data-Commit-Hash 標頭。
+
+    Args:
+        response: FastAPI 的 Response 對象。
+    """
+
+    response.headers["X-Data-Commit-Hash"] = str(courses.last_commit_hash)
+
+
+@router.get(
+    "/",
+    response_model=list[schemas.courses.CourseData],
+    dependencies=[Depends(add_custom_header)],
+)
 async def get_all_courses_list(
     response: Response,
     limits: int = constant.general.LIMITS_QUERY,
@@ -17,10 +55,15 @@ async def get_all_courses_list(
     """
     result = courses.course_data[:limits]
     response.headers["X-Total-Count"] = str(len(result))
+    response.headers["X-Data-Commit-Hash"] = str(courses.last_commit_hash)
     return result
 
 
-@router.get("/fields/info", response_model=dict[str, str])
+@router.get(
+    "/fields/info",
+    response_model=dict[str, str],
+    dependencies=[Depends(add_custom_header)],
+)
 async def get_all_fields_list_info():
     """
     取得所有欄位的資訊。
@@ -48,7 +91,11 @@ async def get_all_fields_list_info():
     }
 
 
-@router.get("/fields/{field_name}", response_model=list[str])
+@router.get(
+    "/fields/{field_name}",
+    response_model=list[str],
+    dependencies=[Depends(add_custom_header)],
+)
 async def get_selected_fields_list(
     field_name: schemas.courses.CourseFieldName = Path(
         ..., example="id", description="欄位名稱"
@@ -63,7 +110,9 @@ async def get_selected_fields_list(
 
 
 @router.get(
-    "/fields/{field_name}/{value}", response_model=list[schemas.courses.CourseData]
+    "/fields/{field_name}/{value}",
+    response_model=list[schemas.courses.CourseData],
+    dependencies=[Depends(add_custom_header)],
 )
 async def get_selected_field_and_value_data(
     field_name: schemas.courses.CourseFieldName = Path(
@@ -80,7 +129,11 @@ async def get_selected_field_and_value_data(
     return result
 
 
-@router.get("/lists/{list_name}", response_model=list[schemas.courses.CourseData])
+@router.get(
+    "/lists/{list_name}",
+    response_model=list[schemas.courses.CourseData],
+    dependencies=[Depends(add_custom_header)],
+)
 async def get_courses_list(
     list_name: schemas.courses.CourseListName,
     response: Response,
@@ -102,7 +155,11 @@ async def get_courses_list(
     return result
 
 
-@router.get("/searches", response_model=list[schemas.courses.CourseData])
+@router.get(
+    "/searches",
+    response_model=list[schemas.courses.CourseData],
+    dependencies=[Depends(add_custom_header)],
+)
 async def search_by_field_and_value(
     field: schemas.courses.CourseFieldName = Query(
         ...,
@@ -122,7 +179,11 @@ async def search_by_field_and_value(
     return result
 
 
-@router.post("/searches", response_model=list[schemas.courses.CourseData])
+@router.post(
+    "/searches",
+    response_model=list[schemas.courses.CourseData],
+    dependencies=[Depends(add_custom_header)],
+)
 async def get_courses_by_condition(
     query_condition: (
         schemas.courses.CourseQueryCondition | schemas.courses.CourseCondition
diff --git a/src/api/schemas/buses.py b/src/api/schemas/buses.py
index ea6692a..47ca5dc 100644
--- a/src/api/schemas/buses.py
+++ b/src/api/schemas/buses.py
@@ -24,9 +24,9 @@ class BusRouteType(str, Enum):
 
 
 class BusType(str, Enum):
-    tour_bus = "tour_bus"
-    middle_sized_bus = "middle-sized_bus"
     route_83 = "route_83"
+    large_sized_bus = "large-sized_bus"
+    middle_sized_bus = "middle-sized_bus"
 
 
 class BusDirection(str, Enum):
diff --git a/src/utils/__init__.py b/src/utils/__init__.py
new file mode 100644
index 0000000..dec7348
--- /dev/null
+++ b/src/utils/__init__.py
@@ -0,0 +1 @@
+from src.utils.data_scraper.api import get, get_file_details
diff --git a/src/utils/data_scraper/__init__.py b/src/utils/data_scraper/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/utils/data_scraper/api.py b/src/utils/data_scraper/api.py
new file mode 100644
index 0000000..fc29069
--- /dev/null
+++ b/src/utils/data_scraper/api.py
@@ -0,0 +1,223 @@
+import asyncio
+import json
+import time
+
+import httpx
+
+_cache = {}  # 記憶體快取，儲存資料與 commit hash
+_file_details_url = "https://data.nthusa.tw/file_details.json"
+_base_url = "https://data.nthusa.tw"
+_file_details_cache = {
+    "data": None,
+    "last_updated": None,
+    "expiry": 60 * 5,
+}  # 快取 file_details.json, 預設 5 分鐘過期
+
+# 建立 httpx AsyncClient session，在模組初始化時建立
+_async_http_client = httpx.AsyncClient()
+
+
+async def _fetch_json(url: str) -> dict | None:
+    """
+    私有函式：使用 httpx AsyncClient session 從 URL 非同步獲取 JSON 資料。
+
+    Args:
+        url (str): 要獲取的 JSON 資料的 URL。
+
+    Returns:
+        dict or None: 如果成功獲取並解析 JSON 資料，則返回字典；如果發生錯誤，則返回 None。
+    """
+    try:
+        async with _async_http_client.stream(
+            "GET", url
+        ) as response:  # 使用 async with 確保資源釋放
+            response.raise_for_status()  # 檢查 HTTP 錯誤
+            data = await response.aread()  # 非同步讀取 response 內容
+            return json.loads(data)
+    except httpx.RequestError as e:
+        print(f"Error fetching {url}: {e}")
+        return None
+    except json.JSONDecodeError as e:
+        print(f"Error decoding JSON from {url}: {e}")
+        return None
+
+
+async def _format_file_details(file_details: dict) -> list:
+    """
+    私有函式：格式化 file_details.json 的資料結構。
+
+    原始的 file_details.json 結構較為巢狀，此函式將其轉換為扁平的列表結構，
+    方便後續查詢和使用。
+
+    Args:
+        file_details (dict): 從 file_details.json 獲取的原始 JSON 資料。
+
+    Returns:
+        list: 格式化後的檔案詳細資訊列表，每個元素都是一個字典，包含 'name', 'last_commit', 'last_updated' 鍵。
+            例如: [{'name': '/announcements.json', 'last_commit': '...', 'last_updated': '...'}, ...]
+    """
+    formatted_file_details = []
+    if file_details.get("file_details"):
+        file_details = file_details["file_details"]
+        for section, files in file_details.items():
+            if section == "/":
+                section = ""
+            for file_info in files:
+                formatted_file_details.append(
+                    {
+                        "name": section + "/" + file_info["name"],
+                        "last_commit": file_info["last_commit"],
+                        "last_updated": file_info["last_updated"],
+                    }
+                )
+    return formatted_file_details
+
+
+async def _update_file_details():
+    """
+    私有函式：非同步更新 file_details.json 快取。
+
+    此函式檢查 file_details.json 快取是否過期或不存在，如果需要更新，
+    則會從遠端伺服器獲取最新的 file_details.json 資料並更新快取。
+    """
+    global _file_details_cache
+    current_time = time.time()
+    if (
+        _file_details_cache["data"] is None
+        or _file_details_cache["last_updated"] is None
+        or (
+            current_time - _file_details_cache["last_updated"]
+            > _file_details_cache["expiry"]
+        )
+    ):
+        print("Updating file_details.json...")
+        file_details_data = await _fetch_json(
+            _file_details_url
+        )  # 使用 await 調用非同步函式
+        if file_details_data:  # 確保成功獲取資料才進行格式化和更新
+            formatted_file_details_data = await _format_file_details(file_details_data)
+            _file_details_cache["data"] = formatted_file_details_data
+            _file_details_cache["last_updated"] = current_time
+            print("file_details.json updated.")
+        else:
+            print("Failed to update file_details.json.")
+    else:
+        print("Using cached file_details.json.")
+
+
+async def get_file_details() -> list | None:
+    """
+    非同步取得 file_details.json 的資料，會自動檢查快取並更新。
+
+    公開函式，用於獲取格式化後的 file_details.json 資料。此函式會先檢查快取，
+    如果快取有效則直接返回快取資料，否則會非同步更新快取後再返回。
+
+    Returns:
+        list or None: 格式化後的檔案詳細資訊列表，如果獲取失敗則返回 None。
+                      例如: [{'name': '/announcements.json', 'last_commit': '...', 'last_updated': '...'}, ...]
+    """
+    await _update_file_details()  # 使用 await 調用非同步函式
+    return _file_details_cache["data"]
+
+
+async def get(endpoint_name: str) -> tuple[str, dict | list] | None:
+    """
+    公開函式：非同步取得指定 endpoint 的 JSON 資料。
+    會檢查快取和 commit hash 以判斷是否需要更新。
+
+    此函式首先會獲取 file_details.json，然後根據 endpoint_name 找到對應的檔案資訊。
+    接著檢查快取中是否有該 endpoint 的資料，並比對 commit hash。
+    如果快取有效且 commit hash 一致，則返回快取資料；否則從遠端伺服器獲取最新資料並更新快取。
+
+    Args:
+        endpoint_name (str): endpoint 名稱，例如 "buses.json" 或 "dining/shops.json"。
+
+    Returns:
+        dict or list or None: 指定 endpoint 的 JSON 資料，可以是字典或列表，如果獲取失敗或 endpoint 不存在則返回 None。
+    """
+    if not endpoint_name.startswith("/"):
+        endpoint_name = "/" + endpoint_name
+
+    file_details_sections = await get_file_details()  # 使用 await 調用非同步函式
+    if file_details_sections is None:
+        print("Cannot fetch data because file_details.json is unavailable or invalid.")
+        return None
+
+    expected_commit_hash = None
+    for file_info in file_details_sections:
+        if file_info["name"] == endpoint_name:
+            expected_commit_hash = file_info["last_commit"]
+            break  # 找到對應檔案後跳出迴圈
+
+    cached_data = _cache.get(endpoint_name)
+
+    if cached_data and cached_data.get("commit_hash") == expected_commit_hash:
+        print(f"Using cached data for '{endpoint_name}'.")
+        return (cached_data["commit_hash"], cached_data["data"])
+    else:
+        print(f"Fetching fresh data for '{endpoint_name}'...")
+        data_url = _base_url + endpoint_name
+        fresh_data = await _fetch_json(data_url)  # 使用 await 調用非同步函式
+        if fresh_data:
+            _cache[endpoint_name] = {
+                "data": fresh_data,
+                "commit_hash": expected_commit_hash,
+            }
+            print(f"Data for '{endpoint_name}' updated in cache.")
+            return (expected_commit_hash, fresh_data)
+        else:
+            print(f"Failed to fetch fresh data for '{endpoint_name}'.")
+            # 如果獲取失敗，可以選擇返回舊的快取資料 (如果有的話)，或者返回 None
+            return (
+                (cached_data["commit_hash"], cached_data.get("data"))
+                if cached_data
+                else None
+            )
+
+
+# # 在模組載入時，先更新一次 file_details.json 快取 (非同步初始化需要用 asyncio.run 在事件迴圈中執行)
+# async def _initialize_file_details():
+#     await get_file_details()
+# asyncio.get_event_loop().run_until_complete(_initialize_file_details())  # 改到各自 router 的 lifespan 執行
+
+
+if __name__ == "__main__":
+
+    async def main():  # 定義一個 async main 函式來包含所有的 await 操作
+        file_details = await get_file_details()
+        print(
+            "File Details Sections (first 2 keys):",
+            file_details[:2] if file_details else None,
+        )
+
+        endpoint1_data = await get("/buses.json")  # 測試根目錄檔案
+        if endpoint1_data:
+            print(endpoint1_data)
+
+        endpoint2_data = await get(
+            "buses/weekdayBusScheduleTowardMainCampus.json"
+        )  # 測試子目錄檔案
+        if endpoint2_data:
+            print(endpoint2_data)
+
+        endpoint1_data_cached = await get("buses.json")  # 再次請求，應該會使用快取
+        if endpoint1_data_cached:
+            print(endpoint1_data_cached)
+
+        await asyncio.sleep(6 * 60)  # 非同步的 sleep
+
+        endpoint1_data_after_wait = await get("buses.json")  # 等待快取過期後再次請求
+        if endpoint1_data_after_wait:
+            print(endpoint1_data_after_wait)
+
+    async def run_main_with_client():  # 包裹 main 函式在 async with 裡面
+        async with httpx.AsyncClient() as client:  # 使用 async with 建立並管理 AsyncClient
+            global _async_http_client  # 宣告使用全域變數
+            _async_http_client = (
+                client  # 將 client 賦值給全域變數，這樣 _fetch_json 等函式才能使用
+            )
+            await main()
+
+    asyncio.run(
+        run_main_with_client()
+    )  # 使用 asyncio.run() 執行 run_main_with_client 函式


Reviews:

Review comments:

PR comments:
user: login: sonarqubecloud[bot]
created_at: 2025-03-02T11:37:15Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=NTHU-SA_NTHU-Data-API&pullRequest=168) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=NTHU-SA_NTHU-Data-API&pullRequest=168&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=NTHU-SA_NTHU-Data-API&pullRequest=168&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=NTHU-SA_NTHU-Data-API&pullRequest=168&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=NTHU-SA_NTHU-Data-API&pullRequest=168&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=NTHU-SA_NTHU-Data-API&pullRequest=168&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=NTHU-SA_NTHU-Data-API&pullRequest=168)


author_association: NONE