# Visual Object Detection with DETR to Support Video-Diagnosis Using Conference Tools
## 2 Materials and Methods
### Pre-Processing  
Nowadays, the Visual Transformers are the subject of dedicated focus because of their ability to comprehensively interpret the image's information. These model types have high fine tunability and higher accuracy than traditional image processing neural networks. \"DETR approaches object detection as a direct set prediction problem. The model uses a set-based global loss, which forces unique predictions via bipartite matching, and a Transformer encoder-decoder architecture\" [30]. Each experiment and validation research (see Sections 3.1 and 3.2) in the same research phase was conducted on the same training- and validation dataset, application categories, and object categories, resulting in: (i) Phase 1: 248 images with 55 validation, 7 application categories, and 3 object categories, (ii) Phase 2: 512 annotated images, with 11 validation, 10 application categories, and 3 object categories. In addition, the prepared dataset contains manually prepared, multi-oriented scene text with quadrangle bounding box annotation. We tested the model's limits using a minimum viable set of learned object queries (see Section 2.1) on minimalist test environment (see Section 2.2). We tested the most well-known algorithms (See Tables 1 and 2) and their **variations** (See Table 3) to get the best performance index and to prove that if we choose the optimal setup, **we will not face a well-known drawback: to run until infinity the training process**. Our goal was to run some relatively limited experiment rounds and summarize the results. We made result elaborations based on average precision (AP) and mean average precision (mAP) metrics for evaluation. Furthermore, we also inspected some labeled validation examples visually for a better human evaluation. We used the following variations:  
1. Different number of epochs: 50, 150, and 1000. 2. Different categorizations: all 4 predefined categories, or 2 categories: text + others (static, no translate, container categories merged to one category). \begin{table}
\begin{tabular}{l r r r r r r r} \hline
**Model** & **Iterations** & **AP\({}_{1}\)** & **AP\({}_{2}\)** & **AP\({}_{3}\)** & **mAP** & **Total Loss** \\ \hline faster\_rcnn\_R\_50\_C4\_3x & 500 & 0.07034 & 0.07367 & 0.09235 & 0.06877 & 1.320 \\ faster\_rcnn\_R\_50\_DC5\_3x & 1000 & 0.17659 & 0.23342 & 0.20921 & 0.17706 & 1.253 \\ faster\_rcnn\_R\_50\_FPN\_3x & 700 & 0.30985 & 0.44342 & 0.36016 & 0.34255 & 1.154 \\ faster\_rcnn\_R\_50\_FPN\_3x & 1000 & 0.30560 & 0.42302 & 0.34745 & 0.34160 & 1.183 \\ faster\_rcnn\_R\_101\_FPN\_3x & 1000 & 0.29764 & 0.47187 & 0.43785 & 0.39341 & 1.129 \\ faster\_rcnn\_X\_101\_32\_8\_FPN\_3x & 3000 & 0.42799 & 0.56096 & 0.56331 & 0.49369 & 0.458 \\ faster\_rcnn\_X\_101\_32\_8\_FPN\_3x & 10,000 & 0.45223 & 0.57821 & 0.55734 & 0.48758 & 0.343 \\ \hline retainnetnet\_R\_50\_FPN\_1x & 1000 & 0.31027 & 0.45998 & 0.47357 & 0.40872 & 0.495 \\ retainnetnet\_R\_50\_FPN\_3x & 1000 & 0.32150 & 0.46114 & 0.46953 & 0.41103 & 0.477 \\ retainnetnet\_R\_101\_FPN\_3x & 3000 & 0.36923 & 0.55230 & 0.55267 & 0.49589 & 0.250 \\ retainnetnet\_R\_101\_FPN\_3x & 10,000 & 0.37968 & 0.55938 & 0.57270 & 0.50513 & 0.189 \\ \hline rpn\_R\_50\_C4\_1x & 1000 & & & & & 0.320 \\ rpn\_R\_50\_FPN\_1x & 3000 & & & & & 0.180 \\ \hline \end{tabular}
\end{table} 
Table 1: Detectron Phase 1 Experiments and Results.