cv4beamanalysis 1.0.7
A package to analyze handwritten beam diagrams
Requires Python: >=3.10

# Computer Vision for Beam Analysis

This repository contains the code for an AI-based system capable of recognizing and analyzing handwritten sketches of engineering beam diagrams.

## Setup

All files implementing the core functionality of the system (namely `analyze.py`, `beam.py`, `main.py`, `number.py`, `relationships.py`, and `yolo.py`) are in the base directory.

* The `yolo.py` file implements the object detection stage of the workflow (stage 1) via a wrapper to the YOLOv5 programs contained in the `yolov5/` directory.
* The `number.py` file implements the number reading stage of the workflow (stage 2) via a wrapper to the SimpleHTR programs contained in the `numberhtr/` directory.
* The `beam.py`, `relationships.py`, `analyze.py` files implement the feature association stage of the workflow (stage 3).
* The `analyze.py` file also implements the structural analysis stage of the workflow (stage 4).
* The `main.py` file utilizes functions in these programs to complete the overall workflow.

Training and testing datasets are contained in the `data/` directory and the machine-learned models to be used are contained in the `models/` directory. Each of these contains subfolders named `features/`, `number/`, and `relationships/` which contain the relevant files for that model stage. Running `pip install -r requirements.txt` downloads and installs all required packages for the system to work.

## Results

Using the baseline models included, 45% of the images in the testing dataset are analyzed entirely correctly, an impressive figure considering how many model inferences are required for an image to be entirely correct.

## Usage

Proper usage of the end-to-end model is the following:

```
python3 main.py [-h] --image-name IMAGE_NAME [--features-path FEATURES_PATH] [--number-path NUMBER_PATH] [--relationships-path RELATIONSHIPS_PATH]
```

A path to an image to analyze must be provided following the `--image-name` (or `--i`) prefix. This produces structural analysis diagrams in a folder named after the image in the `runs/` directory. The path to the models to be used in the object detection, number reading, and feature association stages can be specified if they differ from the baseline models. For example, `python3 main.py --i data/test/IMG-8287.jpg` analyzes the beam system in the first testing image.

To train a new MLP, ensure the number of parameters is set in the `relationships.py` file, and run:

```
python3 relationships.py --mode create --source data/relationships/preprocessed/<FILE> --preprocess no --epochs 30 --name models/relationships/<NAME>
```

## Citation

```
@article{joffe2024cv,
    AUTHOR = {Joffe, Isaac and Qian, Yuchen and Talebi-Kalaleh, Mohammad and Mei, Qipei},
    TITLE = {A Computer Vision Framework for Structural Analysis of Hand-Drawn Engineering Sketches},
    JOURNAL = {Sensors},
    VOLUME = {24},
    YEAR = {2024},
    NUMBER = {9},
    ARTICLE-NUMBER = {2923},
    URL = {https://www.mdpi.com/1424-8220/24/9/2923},
    PubMedID = {38733029},
    ISSN = {1424-8220},
    ABSTRACT = {Structural engineers are often required to draw two-dimensional engineering sketches for quick structural analysis, either by hand calculation or using analysis software. However, calculation by hand is slow and error-prone, and the manual conversion of a hand-drawn sketch into a virtual model is tedious and time-consuming. This paper presents a complete and autonomous framework for converting a hand-drawn engineering sketch into an analyzed structural model using a camera and computer vision. In this framework, a computer vision object detection stage initially extracts information about the raw features in the image of the beam diagram. Next, a computer vision number-reading model transcribes any handwritten numerals appearing in the image. Then, feature association models are applied to characterize the relationships among the detected features in order to build a comprehensive structural model. Finally, the structural model generated is analyzed using OpenSees. In the system presented, the object detection model achieves a mean average precision of 99.1%, the number-reading model achieves an accuracy of 99.0%, and the models in the feature association stage achieve accuracies ranging from 95.1% to 99.5%. Overall, the tool analyzes 45.0% of images entirely correctly and the remaining 55.0% of images partially correctly. The proposed framework holds promise for other types of structural sketches, such as trusses and frames. Moreover, it can be a valuable tool for structural engineers that is capable of improving the efficiency, safety, and sustainability of future construction projects.},
    DOI = {10.3390/s24092923}
}
```