Repository: reductstore/website
Pull request #177: KAN-156: Tutorial and example to analyse ros bags from boston dynamics
Author: login: AnthonyCvn
Created: 2025-04-28T13:32:50Z | Merged: 2025-04-30T13:55:59Z



Diff:
diff --git a/blog/2025-04-30-boston-dynamic-example/img/confusion_matrix.png b/blog/2025-04-30-boston-dynamic-example/img/confusion_matrix.png
new file mode 100644
index 000000000..3bbfe112b
Binary files /dev/null and b/blog/2025-04-30-boston-dynamic-example/img/confusion_matrix.png differ
diff --git a/blog/2025-04-30-boston-dynamic-example/img/feature_importance.png b/blog/2025-04-30-boston-dynamic-example/img/feature_importance.png
new file mode 100644
index 000000000..767aa27d7
Binary files /dev/null and b/blog/2025-04-30-boston-dynamic-example/img/feature_importance.png differ
diff --git a/blog/2025-04-30-boston-dynamic-example/img/forward_movement.png b/blog/2025-04-30-boston-dynamic-example/img/forward_movement.png
new file mode 100644
index 000000000..34d26e120
Binary files /dev/null and b/blog/2025-04-30-boston-dynamic-example/img/forward_movement.png differ
diff --git a/blog/2025-04-30-boston-dynamic-example/img/velocities.png b/blog/2025-04-30-boston-dynamic-example/img/velocities.png
new file mode 100644
index 000000000..fe034a7f6
Binary files /dev/null and b/blog/2025-04-30-boston-dynamic-example/img/velocities.png differ
diff --git a/blog/2025-04-30-boston-dynamic-example/index.mdx b/blog/2025-04-30-boston-dynamic-example/index.mdx
new file mode 100644
index 000000000..28e945a03
--- /dev/null
+++ b/blog/2025-04-30-boston-dynamic-example/index.mdx
@@ -0,0 +1,275 @@
+---
+title: "How to Analyze ROS Bag Files and Build a Dataset for Machine Learning"
+description: "In this tutorial, you will learn how to process ROS .bag files, extract relevant features, and build a dataset for machine learning. We will use Python, the bagpy library, and scikit-learn to train models that can classify different robot movements based on velocity data."
+authors: ekaterina
+tags: [robotics, ROS, tutorials]
+slug: boston-dynamic-example
+date: 2025-04-30
+image: ./img/velocities.png
+---
+
+![Linear and Angular Velocities over Time ](./img/velocities.png)
+
+Working with real-world robot data depends on how ROS (Robot Operating System) messages are stored. In the article [**3 Ways to Store ROS Topics**](/blog/store-ros-topics#method-2-store-rosbag-data-in-time-series-object-storage), we explored several approaches — including storing compressed Rosbag files in time-series storage and storing topics as separate records.
+
+In this tutorial, we'll focus on the most common format: `.bag` files recorded with Rosbag. These files contain valuable data on how a robot interacts with the world — such as odometry, camera frames, LiDAR, or IMU readings — and provide the foundation for analyzing the robot's behavior.
+
+You’ll learn how to:
+
+- Extract motion data from `.bag` files
+- Create basic velocity features
+- Train a classification model to recognize different types of robot movements
+
+We'll use the `bagpy` library to process `.bag` files and apply basic machine learning techniques for classification.
+
+Although the examples in this tutorial use [**data from a Boston Dynamics Spot robot**](http://ptak.felk.cvut.cz/darpa-subt/qualification_videos/spot/) (performing movements like moving forward, sideways, and rotating), you can adapt the code for your recordings.
+
+{/* truncate */}
+
+### Install Required Libraries
+
+```python
+!pip install numpy pandas matplotlib seaborn scikit-learn bagpy
+```
+
+## Loading and Preprocessing Bag Files
+
+Let's create a function to load a `.bag` file and extract velocity features.
+
+In our example, the odometry data is published under the `/spot/odometry` topic. Make sure to specify the correct topic where your robot's motion data is recorded. Depending on your use case, you might find other features, such as accelerations or additional sensor data, more relevant for recognizing your robot's movements. For this task, we'll primarily focus on linear and angular velocities.
+
+```python
+import numpy as np
+import pandas as pd
+from bagpy import bagreader
+
+def process_bag_to_dataframe(bag_path, topic='/spot/odometry', target_label=0):
+
+    # Load a .bag file and generate a DataFrame with velocity features and a target label
+
+    bag = bagreader(bag_path)
+    df = pd.read_csv(bag.message_by_topic(topic))
+
+    # Calculate linear and angular velocities
+    df['linear_velocity'] = np.sqrt(df['twist.twist.linear.x']**2 +
+                                    df['twist.twist.linear.y']**2 +
+                                    df['twist.twist.linear.z']**2)
+
+    df['angular_velocity'] = np.sqrt(df['twist.twist.angular.x']**2 +
+                                     df['twist.twist.angular.y']**2 +
+                                     df['twist.twist.angular.z']**2)
+
+    # Assign target label
+    df['target'] = target_label
+
+    # Keep only relevant columns
+    return df[['twist.twist.linear.x', 'twist.twist.linear.y', 'twist.twist.linear.z',
+               'twist.twist.angular.x', 'twist.twist.angular.y', 'twist.twist.angular.z',
+               'linear_velocity', 'angular_velocity', 'target']]
+```
+
+### Processing three different movement types
+
+Let's use the `process_bag_to_dataframe` function to load and process the data for each of the three movement types. Each movement type was recorded in a separate `.bag` file, so we'll apply the function to each file individually, and then merge the results into a single DataFrame.
+
+```python
+df_forward = process_bag_to_dataframe('linear_x.bag', target_label=1)   # moving forward
+df_sideways = process_bag_to_dataframe('linear_y.bag', target_label=2)  # moving sideways
+df_rotation = process_bag_to_dataframe('rotation.bag', target_label=3)  # rotating
+
+# Combine all samples
+df_all = pd.concat([df_forward, df_sideways, df_rotation], ignore_index=True)
+```
+
+### Visualizing velocities
+
+We can visualize the linear and angular velocities over time for each type of motion, as shown in the example for the forward movement. This will help us better understand how the velocities change during each specific motion.
+
+```python
+import matplotlib.pyplot as plt
+
+def plot_velocities(df, title=''):
+
+    plt.figure(figsize=(7, 3))
+
+    plt.subplot(1, 2, 1)
+    plt.plot(df['linear_velocity'], color='#4B0082')
+    plt.title('Linear Velocity')
+    plt.xlabel('Time Step')
+
+    plt.subplot(1, 2, 2)
+    plt.plot(df['angular_velocity'], color='#9A9E5E')
+    plt.title('Angular Velocity')
+    plt.xlabel('Time Step')
+
+    plt.suptitle(title)
+    plt.tight_layout()
+    plt.show()
+
+plot_velocities(df_forward, 'Forward Movement')
+```
+
+![Forward Movement](./img/forward_movement.png)
+
+## Training and Evaluating Classification Models
+
+We'll test several popular models, including Logistic Regression, Decision Tree, Random Forest, and Support Vector Machine, and tune their hyperparameters using `GridSearchCV`. You can also experiment with other hyperparameters to optimize the models based on your specific data and requirements.
+
+To evaluate the classifier, we'll use the **F1 Score** metric, which balances precision and recall and is especially useful for imbalanced datasets. However, you can also choose to evaluate using Accuracy, Precision, or Recall, depending on your needs.
+
+Now, let's prepare the data for training.
+
+```python
+X = df_all.drop('target', axis=1)
+y = df_all['target']
+```
+
+The features `X` consist of the velocity data, and the labels `y` represent the different movement types.
+
+Next, let’s define the scalers, models, and their respective hyperparameters.
+
+```python
+from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
+from sklearn.linear_model import LogisticRegression
+from sklearn.tree import DecisionTreeClassifier
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.svm import SVC
+
+# Scalers
+scalers = {
+    'Standard Scaler': StandardScaler(),
+    'MinMax Scaler': MinMaxScaler(),
+    'Robust Scaler': RobustScaler()
+}
+
+# Models
+models = {
+    'Logistic Regression': LogisticRegression(max_iter=10000),
+    'Decision Tree': DecisionTreeClassifier(),
+    'Random Forest': RandomForestClassifier(),
+    'SVM': SVC(probability=True)
+}
+
+# Hyperparameters for tuning
+parameters = {
+    'Logistic Regression': {'C': [0.01, 0.1, 1, 10, 100]},
+    'Decision Tree': {'max_depth': [3, 5, 10, None]},
+    'Random Forest': {'n_estimators': [50, 100]},
+    'SVM': {'C': [0.1, 1, 10], 'kernel': ['linear', 'rbf']}
+}
+```
+
+Let's train and test the models using the defined scalers, models, and hyperparameters.
+
+```python
+from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
+from sklearn.metrics import f1_score, confusion_matrix
+
+def run_classification(model_name, scaler_name, X, y):
+
+    # Split data into train and test sets
+    X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.2)
+
+    # Apply the chosen scaler to the data
+    scaler = scalers[scaler_name]
+    X_train = scaler.fit_transform(X_train)
+    X_test = scaler.transform(X_test)
+
+    # Set up hyperparameter grid and cross-validation
+    param_grid = parameters[model_name]
+    cv = StratifiedKFold(n_splits=5, shuffle=True)
+    grid = GridSearchCV(models[model_name], param_grid, cv=cv, scoring='f1_weighted')
+    grid.fit(X_train, y_train)
+
+    # Get the best model from the grid search
+    best_model = grid.best_estimator_
+
+    # Make predictions on the test set
+    y_pred = best_model.predict(X_test)
+
+    print(f'Best parameters: {grid.best_params_}')
+    print(f'F1 Score: {f1_score(y_test, y_pred, average="weighted"):.3f}')
+
+    return y_test, y_pred, best_model, X.columns
+```
+
+After training and testing the models, we'll plot the confusion matrix to visualize how well our model is performing by comparing the predicted labels with the actual labels.
+
+```python
+import seaborn as sns
+
+# Plot confusion matrix
+def plot_confusion_matrix(y_test, y_pred):
+
+    cm = confusion_matrix(y_test, y_pred)
+
+    sns.heatmap(cm, annot=True, fmt='d', cmap='Purples')
+    plt.title('Confusion Matrix')
+    plt.xlabel('Predicted')
+    plt.ylabel('Actual')
+    plt.show()
+```
+
+After evaluating the model's performance, we’ll visualize the feature importance for the Decision Tree and Random Forest models to understand which features contribute the most to the model’s predictions.
+
+```python
+# Feature importance for Decision Tree and Random Forest
+def plot_feature_importance(best_model, X_columns):
+
+    importances = best_model.feature_importances_
+    sorted_idx = importances.argsort()[::-1]
+
+    sns.barplot(x=importances[sorted_idx], y=X_columns[sorted_idx], color='#50208B')
+    plt.title('Feature Importances')
+    plt.xticks(rotation=90)
+    plt.tight_layout()
+    plt.show()
+```
+
+### Training a Random Forest classifier
+
+Finally, let's apply the Random Forest classifier to our data and evaluate its performance.
+
+```python
+y_test, y_pred, best_model, X_columns = run_classification('Random Forest', 'MinMax Scaler', X, y)
+```
+
+The optimal parameter for **n_estimators** is **100**, and the model achieved an **F1 score** of **0.976**.
+
+We’ll plot the **confusion matrix** to assess the classifier's performance across different movement types. The diagonal elements represent the correctly classified instances, while the off-diagonal elements indicate the misclassifications.
+
+```python
+plot_confusion_matrix(y_test, y_pred)
+```
+
+![Confusion Matrix](./img/confusion_matrix.png)
+
+After evaluating the Random Forest model, we can check the **Feature Importance** to see which velocity components were most important in distinguishing the movement types. This is especially useful for Decision Tree and Random Forest models, as they automatically rank features by their importance.
+
+```python
+plot_feature_importance(best_model, X_columns)
+```
+
+![Feature Importance](./img/feature_importance.png)
+
+### Best practices
+
+When working with `.bag` files and training machine learning models, these best practices can help you manage data more effectively and build better-performing models:
+
+- **Split large files:** If your `.bag` files are too large, divide them into smaller episodes. This helps avoid memory issues and makes the files easier to process.
+- **Separate topics by type:** If your `.bag` file includes both lightweight messages (like battery level) and large data streams (like images or LiDAR), store them in separate `.bag` files. This separation can optimize performance and make your workflow simpler.
+- **Use Randomized Search for tuning:** If your model’s accuracy isn’t good enough, try using RandomizedSearchCV instead of GridSearchCV. It can find the best hyperparameters faster and more efficiently.
+- **Try different algorithms:** Experiment with different algorithms to find what works best for your specific data.
+- **Consider ensemble methods:** Techniques like bagging or boosting can improve accuracy by combining multiple models and leveraging their strengths.
+- **Explore deep learning:** If you have a large dataset and enough computing power, deep learning models can capture complex patterns that simpler models may miss.
+- **Prevent overfitting:** Make sure that your model generalizes well by splitting your dataset into training, validation, and test sets. Use cross-validation to evaluate your model’s performance more reliably.
+
+## Conclusion
+
+In this tutorial, we walked through the process of handling robot movement data stored in `.bag` files. We extracted key velocity features and used them to train machine learning models for classifying different types of robot movements.
+
+As a next step, you can experiment with various models, hyperparameters, or additional features to improve classification performance. You can also explore advanced techniques such as deep learning for more complex tasks.
+
+---
+
+We hope this tutorial provided a clear starting point for processing robot data and building basic movement classification models. If you have any questions or comments, feel free to use the [**ReductStore Community Forum**](https://community.reduct.store/signup).
diff --git a/blog/authors.yml b/blog/authors.yml
index 829d4ebd7..a75036b19 100644
--- a/blog/authors.yml
+++ b/blog/authors.yml
@@ -41,3 +41,14 @@ gracija:
   socials:
     linkedin: gracija-nikolovska
     github: gracija
+
+ekaterina:
+  name: Ekaterina Marova
+  title: Data Scientist - ML, Python
+  url: https://github.com/EkaterinaMarova
+  image_url: https://github.com/EkaterinaMarova.png
+  description: An aspiring data scientist with a mathematical and research background.
+  page: true
+  socials:
+    linkedin: ekaterina-marova
+    github: EkaterinaMarova
diff --git a/docs/sdk/py/bucket/index.mdx b/docs/sdk/py/bucket/index.mdx
index 42251141e..6d5c8b0e6 100644
--- a/docs/sdk/py/bucket/index.mdx
+++ b/docs/sdk/py/bucket/index.mdx
@@ -319,16 +319,16 @@ Write a record to entry
 **Raises**:
 
 - `ReductError` - if there is an HTTP error
-
+  
 
 **Example**:
 ```python
       await bucket.write("entry-1", b"some_data",
          timestamp="2021-09-10T10:30:00")
-
+     
       # You can write data chunk-wise using an asynchronous iterator and the
       # size of the content:
-
+     
       async def sender():
           for chunk in [b"part1", b"part2", b"part3"]:
               yield chunk
@@ -381,7 +381,7 @@ If a label is empty, it will be removed.
 **Raises**:
 
 - `ReductError` - if there is an HTTP error
-
+  
 
 **Example**:
 ```python
@@ -415,7 +415,7 @@ If a label is empty, it will be removed.
 **Raises**:
 
 - `ReductError` - if there is an HTTP error
-
+  
 
 **Example**:
 ```python
@@ -466,7 +466,7 @@ float (UNIX timestamp in seconds) or str (ISO 8601 string).
 **Returns**:
 
 - `AsyncIterator[Record]` - iterator to the records
-
+  
 
 **Example**:
 ```python
@@ -528,7 +528,7 @@ float (UNIX timestamp in seconds) or str (ISO 8601 string).
 **Returns**:
 
 - `AsyncIterator[Record]` - iterator to the records
-
+  
 
 **Example**:
 ```python
@@ -538,3 +538,5 @@ float (UNIX timestamp in seconds) or str (ISO 8601 string).
           async for chunk in record.read(n=1024):
               print(chunk)
    ```
+
+
diff --git a/docs/sdk/py/msg/bucket/index.mdx b/docs/sdk/py/msg/bucket/index.mdx
index 06f10a2cb..36364732a 100644
--- a/docs/sdk/py/msg/bucket/index.mdx
+++ b/docs/sdk/py/msg/bucket/index.mdx
@@ -293,3 +293,5 @@ strict mode for when clause
 #### ext
 
 additional parameters for extensions
+
+


Reviews:
user: login: AnthonyCvn
body: @EkaterinaMarova the tutorial looks good overall. You can find my review in the comments for a few details.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-29T08:59:49Z

Review comments:
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
+description: "Learn how to process .bag files and train models to classify different robot movements"
+authors: ekaterina
+tags: [robotics, ROS, tutorials]
+slug: boston-dynamic-example
+date: 2025-04-30
+# image: ./img/demo_setup.png
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: an image is needed here for social media banners
created_at: 2025-04-29T08:41:39Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
+description: "Learn how to process .bag files and train models to classify different robot movements"
+authors: ekaterina
+tags: [robotics, ROS, tutorials]
+slug: boston-dynamic-example
+date: 2025-04-30
+# image: ./img/demo_setup.png
+---
+
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: here you can add the same image as the one you create for social media as an introduction image when you are browsing the page "reduct.store/blog"
created_at: 2025-04-29T08:42:25Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
+description: "Learn how to process .bag files and train models to classify different robot movements"
+authors: ekaterina
+tags: [robotics, ROS, tutorials]
+slug: boston-dynamic-example
+date: 2025-04-30
+# image: ./img/demo_setup.png
+---
+
+Working with real-world robot data often starts with `.bag` files — a standard format for recording ROS (Robot Operating System) messages. Whether you’re logging odometry, camera frames, LiDAR or IMU data, bag files contain valuable data on how a robot interacts with the world.
+
+In this tutorial, we’ll show you how to:
+
+- extract motion data from `.bag` files
+- create basic velocity features
+- train a classification model to recognize different types of robot movements
+
+We'll use the `bagpy` library to process `.bag` files and basic machine learning techniques for classification.
+
+Although the examples use a Boston Dynamics Spot robot (moving forward, sideways, and rotating), you can adapt the code for your own recordings. The original data can be found [here](http://ptak.felk.cvut.cz/darpa-subt/qualification_videos/spot/).
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: the link text "here" should be descriptive (that's an SEO thing to do) and you can make the link in bold. For example:

The [**original data from Boston Dynamics Spot robot**](http://ptak.felk.cvut.cz/darpa-subt/qualification_videos/spot/)..
created_at: 2025-04-29T08:44:32Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
+description: "Learn how to process .bag files and train models to classify different robot movements"
+authors: ekaterina
+tags: [robotics, ROS, tutorials]
+slug: boston-dynamic-example
+date: 2025-04-30
+# image: ./img/demo_setup.png
+---
+
+Working with real-world robot data often starts with `.bag` files — a standard format for recording ROS (Robot Operating System) messages. Whether you’re logging odometry, camera frames, LiDAR or IMU data, bag files contain valuable data on how a robot interacts with the world.
+
+In this tutorial, we’ll show you how to:
+
+- extract motion data from `.bag` files
+- create basic velocity features
+- train a classification model to recognize different types of robot movements
+
+We'll use the `bagpy` library to process `.bag` files and basic machine learning techniques for classification.
+
+Although the examples use a Boston Dynamics Spot robot (moving forward, sideways, and rotating), you can adapt the code for your own recordings. The original data can be found [here](http://ptak.felk.cvut.cz/darpa-subt/qualification_videos/spot/).
+
+{/* truncate */}
+
+### **Install Required Libraries**
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: you don't need to add bold after `###`
created_at: 2025-04-29T08:45:00Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
+description: "Learn how to process .bag files and train models to classify different robot movements"
+authors: ekaterina
+tags: [robotics, ROS, tutorials]
+slug: boston-dynamic-example
+date: 2025-04-30
+# image: ./img/demo_setup.png
+---
+
+Working with real-world robot data often starts with `.bag` files — a standard format for recording ROS (Robot Operating System) messages. Whether you’re logging odometry, camera frames, LiDAR or IMU data, bag files contain valuable data on how a robot interacts with the world.
+
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: you can also explain shortly that we saw different ways to store ROS data in the article [**3 Ways to Store ROS Topics**](https://www.reduct.store/blog/store-ros-topics) and here we assume that it's stored in rosbags and we want to analyse it and train an ML model in python.
created_at: 2025-04-29T08:49:55Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: "classify robot movements" is the example used in the tutorial, but would make the title more about learning to work with ROS bag files and use sklearn to train an ML model.
created_at: 2025-04-29T08:52:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
+description: "Learn how to process .bag files and train models to classify different robot movements"
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: description is quite important for google. you can make it a bit longer. Like "In this tutorial you will lear... We will use ..." 
created_at: 2025-04-29T08:53:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,266 @@
+---
+title: "How to work with ROS bag files and classify robot movements using Python"
+description: "Learn how to process .bag files and train models to classify different robot movements"
+authors: ekaterina
+tags: [robotics, ROS, tutorials]
+slug: boston-dynamic-example
+date: 2025-04-30
+# image: ./img/demo_setup.png
+---
+
+Working with real-world robot data often starts with `.bag` files — a standard format for recording ROS (Robot Operating System) messages. Whether you’re logging odometry, camera frames, LiDAR or IMU data, bag files contain valuable data on how a robot interacts with the world.
+
+In this tutorial, we’ll show you how to:
+
+- extract motion data from `.bag` files
+- create basic velocity features
+- train a classification model to recognize different types of robot movements
+
+We'll use the `bagpy` library to process `.bag` files and basic machine learning techniques for classification.
+
+Although the examples use a Boston Dynamics Spot robot (moving forward, sideways, and rotating), you can adapt the code for your own recordings. The original data can be found [here](http://ptak.felk.cvut.cz/darpa-subt/qualification_videos/spot/).
+
+{/* truncate */}
+
+### **Install Required Libraries**
+
+```python
+!pip install numpy pandas matplotlib seaborn scikit-learn bagpy
+```
+
+# Loading and Preprocessing Bag Files
+
+Let's create a function to load a `.bag` file and extract velocity features.
+
+In our example, odometry data is published under the `/spot/odometry` topic. Make sure to specify the correct topic where your robot's motion data is recorded. Depending on your specific use case, you might find other features, such as accelerations or additional sensor data, more relevant for recognizing your robot's movements. For our task, we focus primarily on linear and angular velocities.
+
+```python
+import numpy as np
+import pandas as pd
+from bagpy import bagreader
+
+def process_bag_to_dataframe(bag_path, topic='/spot/odometry', target_label=0):
+
+    # Load a ROS bag file and generate a DataFrame with velocity features and a target label.
+
+    bag = bagreader(bag_path)
+    df = pd.read_csv(bag.message_by_topic(topic))
+
+    # Calculate linear and angular velocities
+    df['linear_velocity'] = np.sqrt(df['twist.twist.linear.x']**2 +
+                                    df['twist.twist.linear.y']**2 +
+                                    df['twist.twist.linear.z']**2)
+
+    df['angular_velocity'] = np.sqrt(df['twist.twist.angular.x']**2 +
+                                     df['twist.twist.angular.y']**2 +
+                                     df['twist.twist.angular.z']**2)
+
+    # Assign target label
+    df['target'] = target_label
+
+    # Keep only relevant columns
+    return df[['twist.twist.linear.x', 'twist.twist.linear.y', 'twist.twist.linear.z',
+               'twist.twist.angular.x', 'twist.twist.angular.y', 'twist.twist.angular.z',
+               'linear_velocity', 'angular_velocity', 'target']]
+```
+
+### **Processing three different movement types**
+
+Let's apply the `process_bag_to_dataframe` function to load and process the data for each of the three movement types. Each movement was recorded in a separate `.bag` file.
+
+```python
+df_forward = process_bag_to_dataframe('linear_x.bag', target_label=1)   # moving forward
+df_sideways = process_bag_to_dataframe('linear_y.bag', target_label=2)  # moving sideways
+df_rotation = process_bag_to_dataframe('rotation.bag', target_label=3)  # rotating
+
+# Combine all samples
+df_all = pd.concat([df_forward, df_sideways, df_rotation], ignore_index=True)
+```
+
+### Visualizing velocities
+
+We can plot linear and angular velocities over time for each kind of motion, such as the forward movement.
+
+```python
+import matplotlib.pyplot as plt
+
+def plot_velocities(df, title=''):
+
+    plt.figure(figsize=(7, 3))
+
+    plt.subplot(1, 2, 1)
+    plt.plot(df['linear_velocity'], color='#4B0082')
+    plt.title('Linear Velocity')
+    plt.xlabel('Time Step')
+
+    plt.subplot(1, 2, 2)
+    plt.plot(df['angular_velocity'], color='#9A9E5E')
+    plt.title('Angular Velocity')
+    plt.xlabel('Time Step')
+
+    plt.suptitle(title)
+    plt.tight_layout()
+    plt.show()
+
+plot_velocities(df_forward, 'Forward Movement')
+```
+
+![Forward Movement](./img/forward_movement.png)
+
+# Training and Evaluating Classification Models
+
+We will test several popular models: Logistic Regression, Decision Tree, Random Forest, and Support Vector Machine.
+
+We'll also use `GridSearchCV` for hyperparameter tuning. You can experiment with other hyperparameters to fine-tune the models based on your specific data and requirements.
+
+We evaluate the classifier using **F1 Score**, as it balances precision and recall. This is particularly useful for imbalanced datasets. However, you can also use Accuracy, Precision, and Recall.
+
+Let's prepare the data for training.
+
+```python
+X = df_all.drop('target', axis=1)
+y = df_all['target']
+```
+
+The features `X` consist of the velocity data, while the labels `y` represent the movement types.
+
+Let’s define the scalers, models and hyperparameters.
+
+```python
+# Scalers
+scalers = {
+    'Standard Scaler': StandardScaler(),
+    'MinMax Scaler': MinMaxScaler(),
+    'Robust Scaler': RobustScaler()
+}
+
+# Models
+models = {
+    'Logistic Regression': LogisticRegression(max_iter=10000),
+    'Decision Tree': DecisionTreeClassifier(),
+    'Random Forest': RandomForestClassifier(),
+    'SVM': SVC(probability=True)
+}
+
+# Hyperparameters for tuning
+parameters = {
+    'Logistic Regression': {'C': [0.01, 0.1, 1, 10, 100]},
+    'Decision Tree': {'max_depth': [3, 5, 10, None]},
+    'Random Forest': {'n_estimators': [50, 100]},
+    'SVM': {'C': [0.1, 1, 10], 'kernel': ['linear', 'rbf']}
+}
+```
+
+Next, let’s train and test the models.
+
+```python
+from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
+from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
+from sklearn.metrics import f1_score, confusion_matrix
+from sklearn.linear_model import LogisticRegression
+from sklearn.tree import DecisionTreeClassifier
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.svm import SVC
+
+def run_classification(model_name, scaler_name, X, y):
+
+    # Split data into train and test sets
+    X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.2)
+
+    # Apply chosen scaler to the data
+    scaler = scalers[scaler_name]
+    X_train = scaler.fit_transform(X_train)
+    X_test = scaler.transform(X_test)
+
+    # Set up hyperparameter grid and cross-validation
+    param_grid = parameters[model_name]
+    cv = StratifiedKFold(n_splits=5, shuffle=True)
+    grid = GridSearchCV(models[model_name], param_grid, cv=cv, scoring='f1_weighted')
+    grid.fit(X_train, y_train)
+
+    # Get best model from grid search
+    best_model = grid.best_estimator_
+
+    # Make predictions on test set
+    y_pred = best_model.predict(X_test)
+
+    print(f'Best parameters: {grid.best_params_}')
+    print(f'F1 Score: {f1_score(y_test, y_pred, average="weighted"):.3f}')
+
+    return y_test, y_pred, best_model, X.columns
+```
+
+We'll plot the confusion matrix to visualize how well our model is performing by comparing the predicted values with the actual labels.
+
+```python
+import seaborn as sns
+
+# Plot confusion matrix
+def plot_confusion_matrix(y_test, y_pred):
+
+    cm = confusion_matrix(y_test, y_pred)
+
+    sns.heatmap(cm, annot=True, fmt='d', cmap='Purples')
+    plt.title('Confusion Matrix')
+    plt.xlabel('Predicted')
+    plt.ylabel('Actual')
+    plt.show()
+```
+
+We’ll visualize the feature importance for Decision Tree and Random Forest models to understand which features contribute the most to the model’s predictions.
+
+```python
+
+# Feature importance for Decision Tree and Random Forest
+def plot_feature_importance(best_model, X_columns):
+
+    importances = best_model.feature_importances_
+    sorted_idx = importances.argsort()[::-1]
+
+    sns.barplot(x=importances[sorted_idx], y=X_columns[sorted_idx], color='#50208B')
+    plt.title('Feature Importances')
+    plt.xticks(rotation=90)
+    plt.tight_layout()
+    plt.show()
+```
+
+### **Training a Random Forest classifier**
+
+Finally, let's apply the Random Forest classifier to our data and see how well it performs.
+
+```python
+y_test, y_pred, best_model, X_columns = run_classification('Random Forest', 'MinMax Scaler', X, y)
+```
+
+**Best parameters**: {`{'n_estimators': 100}`}
+
+**F1 Score**: 0.976
+
+The confusion matrix provides a breakdown of how the classifier performed on each movement type. The diagonal elements represent the correctly classified instances, while the off-diagonal elements indicate misclassifications.
+
+```python
+plot_confusion_matrix(y_test, y_pred)
+```
+
+![Confusion Matrix](./img/confusion_matrix.png)
+
+With an F1 score of 0.976, this classifier performs well, but you can always experiment to optimize the model based on your own data.
+
+After evaluating the Random Forest model, we can look at **Feature Importance** to see velocity components contributed most to distinguishing the movement types. This is useful for Decision Tree and Random Forest models, which automatically rank features based on their importance.
+
+```python
+plot_feature_importance(best_model, X_columns)
+```
+
+![Feature Importance](./img/feature_importance.png)
+
path: blog/2025-04-30-boston-dynamic-example/index.mdx
user: login: AnthonyCvn
body: you can add here a small section "Best practices" and explain in a few bullet points what to be careful about. For example if ROS bag files is too big, you can split it in 1-5 minutes episodes; split topics with lightweight topics in on BAG and images or LiDAR in another one if any. Also best practices about the use of sklearn (for example to use randomize search instead of grid search if accuracy is not so good ; try different methods; consider ensemble methods or deep-learning models; avoid overfitting and split dataset into train/val/test)
created_at: 2025-04-29T08:58:35Z
author_association: MEMBER

PR comments:
user: login: github-actions[bot]
created_at: 2025-04-28T13:36:11Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-28T19:46:43Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T08:07:25Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T08:25:46Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T10:05:57Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T11:07:00Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T12:50:32Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T13:18:18Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T13:28:45Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T21:04:29Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-29T21:33:21Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-30T13:01:16Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-30T13:52:13Z
body: Azure Static Web Apps: Your stage site is ready! Visit it here: https://salmon-bush-0fb136503-177.westeurope.5.azurestaticapps.net
author_association: NONE