Stagnant zone segmentation with U-net
Authors: Selam Waktola
Laurent Babout
Krzysztof Grudzien
Venue:  (2020)

Abstract: Silo discharging and monitoring the process for
industrial or research application depend on computerized
segmentation of different parts of images such as stagnant and
flowing zones which is the toughest task. X-ray Computed
Tomography (CT) is one of a powerful non-destructive technique
for cross-sectional images of a 3D object based on X-ray
absorption. CT is the most proficient for investigating different
granular flow phenomena and segmentation of the stagnant zone
as compared to other imaging techniques. In any case, manual
segmentation is tiresome and erroneous for further investigations.
Hence, automatic and precise strategies are required. In the
present work, a U-net architecture is used for segmenting the
stagnant zone during silo discharging process. This proposed
image segmentation method provides fast and effective outcomes
by exploiting a convolutional neural networks technique with an
accuracy of 97 percent

Stagnant zone segmentation with U-net
Selam Waktola*
Krzysztof Grudzien
Institute of A pplied Computer Science
Institute of A pplied Computer Science Lodz University of T'echnology Lodz, Poland
Lodz University of Technology Lodz, Poland selam. waktola@ gmail.com
Laurent Babout
Institute of Applied Computer Science
Lodz University of Technology Lodz, Poland
Abstract- Silo discharging and monitoring the process for industrial or research application depend on computerized segmentation of different parts of images such as stagnant and flowing zones which is the toughest task. X-ray Computed Tomography (CT) is one of a powerful non-destructive technique for cross-sectional images of a 3D object based X-ray absorption. CT the most proficient for investigating different granular flow phenomena and segmentation of stagnant zone as compared to other imaging techniques. In any case, manual segmentation is tiresome and erroneous for further investigations. Hence, automatic and precise strategles are required. In the present work, a U-net architecture is used for segmenting the stagmant zone during silo discharging process. This proposed image segmentation method provides fast and effective outcomes by exploiting a convolutional neural networks technique with an accuracy 97 percent.
X-ray Computed tomography (CT) is one of the most powerful 3D imaging techniques among available tomographic techniques due to its high- resolution capability. For the last two decades CT has been used as a non-destruetive method to characterize objects, visualize flows analyze concentration changes of the bulk solid during silo discharging process [5 7,81. Particle Image V Velocimetry (PIV) [9,10] and Electrical Capacitance Tomography (ECT) [11-14] are also some of the techniques used to visualize concentration changes during flows of granular materials.
Even though convolutional neural network (CNN) has recently become popular and has increasingly been used as an alternative to many traditional pattern recognition problems, its application for segmenting stagnant zones of X X-ray CT images is not common. For processing and analyzing process tomography data, artificial neural network algorithms were applied in electrical impedance tomography images [15-17]. This paper proposed a deep neural networks tochnique for segmenting the stagnant zone automatically. The main advantage of the proposed approach is effective segmentation for acquiring the desired characteristics of flow parameters without prior image processing expert guidance.
Keywords- U-net segmentation; deep neural networks; stagnant zone; X-ray tomography
I. INTRODUCTION
In many industries, like in mining, agriculture, civil engineering, and pharmaceutical manufacturing: silo is used for protecting, storing and loading granular materials into process machinery [1]. D During the silo discharging process, there are two major types of flow: namely, "funnel" and "mass" flows [2]. In the case of mass flow, all granulates discharge with a uniform downward velocity across the entire cross-section area. W Whereas, in the case of funnel flow, in which this paper focuses on and characterized by granular is flowing only in the center of the silo and creating a stagnant zone at the walls of the container [2,3].
II. EXPERIMENTAL SET-UP USING X-RAY TOMOGRA PHY
An especially designed model silo, with rectangular bin, allowed carrying out in situ experiments. The bin part is 10 cm wide, 5 cm deep and 20 cm high. The left and right hopper angles can be independently set to generate different types of flows, i.e. mass and funnel flows, with concentric/eccenttic discharying modes. The silo model design was easily customizable to shift the outlet position and the angle between the hopper and the silo with respect to the vertical. The silo material is polycarbonate with 5 mm thickness. The outlet width is manually set, which allows controlling the discharge velocity. During concentric flow, both hoppers' angles are set to the same value. In order to observe the eccentric flow, these angles are set to different value. The non- symmerri silo construction causes shift silo outlet to the left or right side, in the direction of the larger angle.
The hopper geometry, internal friction between particles and wall all have a direct impact on the flow type [4]. In order to understand and describe the flow behavior and evaluating the silo wall pressures, the knowledge of the density distribution within the bulk solid is very important aspect [5]. Furthermore, during funnel flow, the shape and size of the stagnant zone depend on different factors including the granular material, the bin wall roughness, the initial packing density and filling level [6].

---

The time-lapse studies were performed at INSA-Lyon (France) using GE Phoenix v/tomelx device (see Fig. 1). The device is equipped with high energy X-ray microfocus sowrce (up to 160 kV) with a um spot size. The detector equipped with a 1500X 1920 pixels (with pixel size of 127X127 μm"). Fig. presented a picture of the silo model inside X -ray tomography hutch, During measurements, the distance between the --ray source and the detector was equal to 577 mm and voxel size 160 um. Soryhum and rice have been used a granular material during the experimental campaign of this study.
absolute difference between two successive scans has been computed and it shows the formation of the stagmant zone in the case of funnel flow. Fig. 3c presents concentration changes during eccentric discharging mode (angle between the hopper and the silo with respect to the vertical axis was 30' and 20') and it reveals the evolution of the stagnant zone at the right side of the image with a good contrast which the hopper angle was 20'.
(c)
Detector
Sillo model
Sample scan
X-ray source
Fig. 3. Silo hopper angle: 30 the left and 202 the right side view in the middle of the silo where 200 pixel. (a) Second scan. (b) Third scan. (c) The absolute difference between scan three and four of sorghum.
Fig. 1. X-ray Computed Tomography system with silo model.
The last step consists in segmenting the stagmant zones. By using dine--immesiioal Otsu threshold method it was possible compute the stagnant zone mask as presented in Fig.4.
THREE- DIMENSIONAL SEGMENTATION
Cross-sectional views of the X- X-ray tomography of initially loosely packed rice with smooth silo bin walls are presented in Fig. 2.
(a)
(b)
(c)
Z
(a)
Fig. (a) Ground x=240 (b) segmentation using Otsu x=240 (c) superimposing ground truth and Otsu segmentation at x=240 with red color and ground truth with white.
Although the Otsu method was able to segment the stagnant zone, is hard get 100% accurate segmentation out of it. Since some particles in the flow zone may shift in the position of another particle, giving impression of no movement based on the absolute difference. Thus, such kind effect could hinder the segmentation result. As a result, a deep neural networks approach has been applied for effective segmentation and explained hereafter.
(b)
Fig. (a) front side at x200 voxel. (b) top side 2=00 voxel.
The main aim of this study was to find effective way of segmenting the stagnant zone during eccentric discharying modes. After the initial packing density scan was performed, the outlet of the silo was opened for about 2 second and the next scan was carried out till all granulate are discharyed out from the silo. Thus, to analyze the concentration changes, the
IV. U-NET BASED DEEP CONVOLUTIONAL NETWORKS
The main advantage of the proposed approach an effective segmentation for acquiring the desired characteristics

---

flow parameters without prior image processing or expert guidance. From several deep neural networks, u-net is one of the successful architecture which is used in different image segmentation tasks. Originally u-net neural network architecture was built for performing semantic segmentation on a small bio- medical data-set set [18). The architecture is computationally efficient and trainable with a small dataset, which is core advantage for datasets like in material science where the little amount of labelled data is available. Despite it might be good and well-fitted in
positives + false negatives + false positives). Table presents the result of the experiment for segmenting stagnant zone.
Network architecture
IoU
Original U-Net
0.95
Modified U-Net
0.97
tasks, needs to be tuned again for fitting the granular material X-ray CT image segmentation task investigated in this study. Thus, the u-net neural network architecture and parameeetss values are adjusted in order to get good segmentation results.
Table Model perfomances on segmenting stagnant zone.
Once after having trained model using the CNN method, the end and-to-end 3D automatic segmentation offers an effective and fast segmentation of stagnant zone during silo discharging process. The key advantage of this method could be used for deep investigation of flow characteristics without utilizing any prior image processing methods or expert guidance. For both the Otsu and the trained model execution time (on CPU) were compared for generating one segmented image. The Otsu segmentation took around sec per one 2D image. Where else the trained model took less than a second for generating its prediction for a given input image.
The network has been trained on 2D segmentation and then it can find the whole 3D volume segmentation of new scan in which the network has never seen before. As each of the CT images already contain repetitive structures with the corresponding variation, only very few images are required to train a network that generalizes reasonably well. One of the major modifications is that. the original u- u-net used stochastic gradient descent optimizer (18), but this modified u-net architecture used Adam optimizer [19] to minimize the categorical cross- entropy objective.
In order to prove that the trained model (which used the sorghum grains flow as training dataset) could generate the stagnant zone segmentation for completely different scan and grain material, it was tested by pre- processing two 3D scans of rice grains flow. The result shows that the trained model was able to generate predicted segmentation successfully. Fig. 6 presents two major steps for generating the stagnant zone prediction of new scans having similar flow property (in this case rice grains flow). The first step was computing the absolute difference of two successive scans and then this new scan was given an input the trained model. Finally, the trained model generates the stagnant zone segmentation as shown in Fig. 6d. Fig. 7 illustrates 3D dense segmentation of rice grains flow by superimposing the predicted mask into the original scan.
During training, 40 images were selected for the training and test dataset. The dataset was first divided into two subsets, which
train, and test. The first subset contains 30 images 80% were used for training and 20% for validation. The trained model was next tested on the second subset which contains the remaining 10 images. During training. 30 manually annotated ground truth segmentations were used to train the network to recogmize the stagnant zone bortlers Since the available dataset is small, an extensive amount of data augmentations has been applied to improve the performance of the network. The testing datasets were used for the evaluation of the network performance.
Fig. 5 illustrates the comparison between the ground truth, segmentation by using Otsu and predicted segmentation by the modified u-net trained model for the same 2D image. As the result shows, Otsu segmentation is too sensitive for gradient changes and the trained model predicted a smooth segmentation like the ground truth.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
Fig. 6. (a) Rice scan (b) Rice scan 4 (c) the absolute difference between
scan and scan 4 (d) stagnant zone prediction.
Fig. 5. (a) Original Image (b) Ground truth (c) segmented by using Otsu (d) modified u-net model prediction.
Intersection over Union (IoU) is used as an accuracy measure to compare dropped out ground truth slices to the predicted results. The IoU score is standard performance measure for the object category segmentation problem. Given a set of images, the IoU measure gives the similarity between the predicted region and the ground-truth region for an object present in the set of images and is defined as true posttives/(truee

---

(a)
Flows of Dense Granular Avalanches. Springer, New Y ork. (2007) Babout, L., Grudzien, K., Maire, E., Withers, PJ:: Influence of wall roughness and packing density on stagnant zone formation during funnel flow discharge from silo: ray imaging study, Chem. Eng. Sci. 97, 210-224 (2013)
Brown, C.J., Nielsen, J:: Silos: Fundamentals Theory, Rehavior, and Design. In: E & FN Spon, London. London (1998)
[7]
Grudzich, Niedostarkiewicz, M., Adrien, Tejehman, J., Maire, E.: Q Quantitative estimation of volume changes of granular matenials during silo flow using X-ray tomography. Chem. Eng. Process. Process Intensif. 50, 59-67 (2011).
(c)
(d)
Flowing zone
Grudzien, K., Niedostafkiewlcz, M., Adrien, J.. Maire, Babout, L.: A Analysis the bulk solid flow during gravitational silo emptying using X-may and ECT tomogruphy. Powder Technol. 224, 196-208 (2012)
[9]
Slominski, C., Niedostatkewwczz M., Tejchman, Jaz Application particle image velocimetry (PIV) for deformation measurement during granular silo flow. Powder Technol. 173, 1 (2007).
Stagnent
Fig. 7. Original rice grains flow scan 4 (a) front side view (b) Jeft side view; the network predicted dense segmentation front side view (d) Jeft side view
[10]
Steingart, D.A., Evans, J.W.: Measurements granular flows two-dimensional hoppers by particle image velocimetry. Part I: experimental method results. Chem. Eng. 60, 1043-1051 (2005).
V. CONCLUSION
[11]
Grudzien, K., Chaniecki, Z., Romanowski, A., Niedostatklewicz M., Sankowski, D.: ECT image analysis methods for shear zone measurements during silo discharging process. Chinese J. Chem. Eng. 20, 337-345 (2012)
In order to analyse the X-ray CT data that has been acquired from different experimental campaigns and complex structure of granular material flows, it can be tedious and extremely time- consuming for manual analysis. This paper presents new approach for automatic segmentation of the stagnant zone in an elfective way by exploiting the CNN technique. The main advantages of the proposed approach are the speed and effective segmentation for acquiring the desired characteristic flow parameters. Once having the trained model, it was tested that the model could generate a predicted segmentation in less than a second for a given completely new granular material flow image with an accuracy of 97%. The accuracy of the CNN approach could also probably be further improved if the delineations of the ground truth were acquired from different experts and more number of dataset was used. Moreover, the architecture of the model could be modified accommodate 3D volumes of images as an input for processing them with corresponding 3D operations.
[12]
Niedostatkiewier, M., Tejehman, J., Chaniecki, Z., Grudzien, K.: Determination of bulk solid concentraiion changes during granular flow in model silo with ECT sensors. Chem. Eng. Sci. 64, 20 30 (2009), 1oi:10.1016/J 1tt.1111//........... wC8...00..0880.33.
[13]
Chaniecki, Z., Dyakowski, T., Niedostatkiewicz, M., Sankowski, D.: Application of Electrical Capacitanoe Tomography for Bulk Solids Flow A Analysis in Silos. Part. Part. Syst. Charact. 23, 306 312 (2006). Grudzieñ, Krzysztof Sankowski, gravitational flow silos using lomography image processing. TAPGOS. 7, 24 24-29 (2017). doi:D01:
[14]
Methods monitoring
[15]
Stasiak, M., Sikora, J., Filipowicz, S., Nita, K.: Principal component analysis and artificial neural network approcch electrical impedance tomography problems approximated by multi region boundary element method. Eng. Anal. Bound. Elem. 31, 713-720 (2007)
A ACKNOWLEDGMENT
The work is funded by the National Science Centre in Poland (grant number: 2015/19/B/ST8/02777
[16) Rymarezyk, Т., Klosowski, G., Cieplak, T., Kozlowski, E.: Industrial processes control with the use of neural tomographic algorithm. Prz. ELEKTROTECHNICZNY R. 95 NR 2, (2019)
REFERENCES
[17]
Klosowski, Rymarezyk, T.: Using leaming algorithms in electrical impodance tomography. IAPGOS.
networks and deep
Chaniecki, Z., Grudzien, Romanowski, A.: Tomographic Visualization of Dynamic Industrial Solid Transporting and Storage Systems. In: Computer Vision in Robotics and Industrial Applications, pp. 257-280 (2014)
99 102 (2017)
[18]
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Medical Image Computing and Computer-A ssistedd Intervention (MICCAT). pp. 234-241. Springr (2015)
Schulze, D.: Silo design for flow. In: Powders and Bulk Solids: Bebavior, Characterization Storage and Flow. pp. 291 319 (2007) Drescher, A.:
critetia for mass flow hoppers. Powder Technol. 73, 251-260 (1992)
[19]
Kingma, D.P., ., Ba, Adam: A Method for Stochastic Optimization. Int. Conf. Learn. Repreeet (2014)
Pudasaini, S.P., K., H.: Avalanche Dynamics: Dynamics of Rapid


Reviews:
[{"review_id": "SklcjZbP5r", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper propose a modified U-net architecture to segment the stagnant zone during silo discharging process. It lacks novelty and the improvement is marginal.  More importantly than all of that, this paper violates the double blind review rule and is same with [1]. So I think this paper is not suitable for acception.\n\n[1]Waktola S, Grudzien K, Babout L. Stagnant zone segmentation with U-net[C]//2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE). IEEE, 2019: 277-280.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "Hyxln7Sotr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I support desk rejection since violating double blind rule, wrong format and insufficient length.\n\n                                                                                                                                                                                                                                                                                              \n                                                                                                                                                                                                                                                                                              \n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}, {"review_id": "BygUatwYOB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- This paper simply proposes to use UNet for the segmentation of stagnant zones in X-ray CTs. While the applicability of this model may represent an advance in the particular field of the authors, the technical contribution of this paper is far from the level expected in this conference. \n\n- As the paper reads, the main contribution of the paper is the modified version of UNet 'proposed' by the authors, which major modification consists on replacing SGD by Adam. Nevertheless, this cannot be considered a contribution, as changing the optimizer in a deep model is a marginal change, from a methodological point of view. \n\n- Overall, the quality of the paper is below the standards of ICLR (content, technical contribution, length).\n\n- The submission is not anonymized (authors included their names and affiliations). ", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposed U-net for segmentation of stagnant zones in computed tomography. Technical contribution of the paper is severely limited, and is not of the quality expected of publications in this venue. The paper is not anonymized and violates the double blind review rule. I'm thus recommending rejection.
Decision: Reject