Hugging Face's logo

Worldcoin
/
iris-semantic-segmentation

ONNX
Model card Files Community
1
wiktorlazarski commited on
Commit
366cb20
•
1 Parent(s): 1b7c7c6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -37
README.md CHANGED
@@ -14,16 +14,26 @@ The decoder is enhanced through the incorporation of Spatial and Channel "Squeez
14
 
15
  ## Dataset
16
 
17
- Importantly, no Worldcoin user data was used to train or fine-tune the IRIS pipeline. Rather, a research dataset from the University of Notre Dame du Lac (ND-IRIS-0405) [4] was used, with the University’s permission. This dataset was enhanced with manual labels, which themselves may be made available for research purposes.
18
-
19
- The experimental dataset contained a total of 9 957 manually annotated IR images comming from 676 different people. All images were captured using LG 4000 device. Table below presents dataset split used during training semantic segmentation model.
20
-
21
-
22
- | **Dataset type**| **Number of images** | **Number of subject** |
23
- |-----------------------------|-----------|-----------|
24
- | train | 7933 | 541 |
25
- | validation | 1032 | 67 |
26
- | test | 1030 | 68 |
 
 
 
 
 
 
 
 
 
 
27
 
28
  ## Performance
29
 
@@ -33,17 +43,33 @@ The experimental dataset contained a total of 9 957 manually annotated IR images
33
 
34
  | **Metric** | **Value** |
35
  |-----------------------------|-----------|
36
- | _eyeball_IoU_ | 0.981 |
37
- | _iris_IoU_ | 0.977 |
38
- | _pupil_IoU_ | 0.976 |
39
- | _eyelashes_IoU_ | 0.755 |
40
- | _mIoU_ | 0.918 |
41
- | _eyelashes_inside_iris_IoU_ | 0.683 |
42
- | _boundary_confidence_pupil_ | 0.971 |
43
- | _boundary_confidence_iris_ | 0.921 |
44
- | _chamfer_eyeball_ | 1.667 |
45
- | _chamfer_iris_ | 1.632 |
46
- | _chamfer_pupil_ | 0.686 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  ### Time Performance
49
 
@@ -76,36 +102,25 @@ The model yields a tensor characterized by dimensions of Nx4x640x480, denoting b
76
 
77
  Within the ambit of each class, the model formulates probability estimates pertaining to the likelihood of a given pixel being attributed to a particular class.
78
 
79
- ### Examplary inference results
80
 
81
  **Note**: The provided input image has been subjected to the processing methodology described earlier, prior to its introduction into the model. Moreover, for the intent of visualization, the IR image presented has been anonymized to safeguard the identity of the user. It is also worth to note that the inference process was conducted on the original, non-anonymized version of the image.
82
 
83
  #### Input image
84
 
85
- <img src="https://github.com/worldcoin/open-iris/blob/main/docs/model_card/anonymized.png?raw=true" width=640 height=480>
86
 
87
  #### Output masks
88
 
89
  **Note**: A threshold of 0.5 was applied on every resultant probability matrix to generate a definitive binary mask for each individual class. Also, every outputted segmentation map was resized to the input image resolution.
90
 
91
- <img src="https://github.com/worldcoin/open-iris/blob/main/docs/model_card/overlayed_segmaps.png?raw=true">
92
 
93
  ## Limitations
94
 
95
  Thorough examination of the results enabled us to pinpoint situations where the segmentation model experiences declines in performance. These instances are as follows:
96
- - Segmenting images that were not captured by LG4400 sensor may not always produce smooth segmentation maps. The segmention performance depends on how similar images to be segmented are to the images captured by LG4400 sensor.
97
- - Segmenting images with high specular reflection comming usually from glasses may lead to bad segmentation map predictions.
98
- - Data based on which the model was trained were captured in the constrained environment with cooperative users. Therefore, in practise model is expected to produce poor segmentation maps for cases like: offgazes, misaligned eyes, blurry images etc.
99
-
100
- ## License
101
- Unless otherwise specified, the contents of this repository are dual-licensed under either:
102
-
103
- - MIT License (LICENSE-MIT)
104
- - Apache License, Version 2.0 (LICENSE-APACHE)
105
-
106
- at your option. This means you may select the license you prefer to use.
107
-
108
- Any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
109
 
110
  ## Further reading
111
 
@@ -113,3 +128,9 @@ Any contribution intentionally submitted for inclusion in the work by you, as de
113
  2. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. (https://arxiv.org/abs/1801.04381)
114
  3. Roy, A. G., Navab, N., & Wachinger, C. (2018). Recalibrating Fully Convolutional Networks with Spatial and Channel “Squeeze and Excitation” Blocks. (https://arxiv.org/abs/1808.08127v1)
115
  4. Bowyer, K. , Flynn, P. (2016), The ND-IRIS-0405 Iris Image Dataset (https://arxiv.org/abs/1606.04853)
 
 
 
 
 
 
 
14
 
15
  ## Dataset
16
 
17
+ Importantly, no Worldcoin user data was used to train or fine-tune the IRIS pipeline. Rather, a research datasets were used to train model. Datasets used during training comes from:
18
+ - ND-IRIS-0405 [4]
19
+ - CASIA-Iris-Interval [5]
20
+ - CASIA-Iris-Lamp [5]
21
+ - CASIA-Iris-Thousand [5]
22
+ - CASIA-Iris-Twins [5]
23
+ - IIT Delhi Iris Database (IITD) [6]
24
+ - LivDet2013-Iris [7]
25
+ - LivDet2015-Iris [8]
26
+ - LivDet2017-Iris [9]
27
+ - Multimedia University Iris Database (MMU) [10]
28
+
29
+ The experimental dataset contained a total of 18 431 annotated IR images. Table below presents dataset split used during training semantic segmentation model.
30
+
31
+
32
+ | **Dataset type**| **Number of images** |
33
+ |-----------------------------|-----------|
34
+ | train | 14685 |
35
+ | validation | 1880 |
36
+ | test | 1866 |
37
 
38
  ## Performance
39
 
 
43
 
44
  | **Metric** | **Value** |
45
  |-----------------------------|-----------|
46
+ | _eyeball_IoU_ | 0.986 |
47
+ | _iris_IoU_ | 0.978 |
48
+ | _pupil_IoU_ | 0.978 |
49
+ | _eyelashes_IoU_ | 0.798 |
50
+ | _mIoU_ | 0.943 |
51
+ | _eyelashes_inside_iris_IoU_ | 0.791 |
52
+ | _boundary_confidence_pupil_ | 0.965 |
53
+ | _boundary_confidence_iris_ | 0.907 |
54
+ | _chamfer_eyeball_ | 1.689 |
55
+ | _chamfer_iris_ | 1.868 |
56
+ | _chamfer_pupil_ | 0.680 |
57
+
58
+ #### Measured on particular dataset
59
+
60
+ |Name |test_IoU_eyeball|test_IoU_eyelashes|test_IoU_iris|test_IoU_pupil|test_boundary_confidence_iris|test_boundary_confidence_pupil|test_chamfer_dist_eyeball|test_chamfer_dist_iris|test_chamfer_dist_pupil|test_mIoU |test_mIoU_eyelashes_inside_iris|
61
+ |---------------------------------------------------------------|------------------------|--------------------------|---------------------|----------------------|-----------------------------|------------------------------|---------------------------------|------------------------------|-------------------------------|------------------|---------------------------------------|
62
+ |CASIA-Iris-Interval |0.992 |0.885 |0.977 |0.983 |0.868 |0.959 |1.230 |3.132 |0.673 |0.970 |0.876 |
63
+ |CASIA-Iris-Lamp |0.987 |0.844 |0.988 |0.983 |0.979 |0.988 |0.683 |0.482 |0.295 |0.953 |0.836 |
64
+ |CASIA-Iris-Thousand |0.987 |0.823 |0.989 |0.981 |0.981 |0.988 |0.637 |0.430 |0.252 |0.956 |0.767 |
65
+ |CASIA-Iris-Twins |0.986 |0.837 |0.986 |0.981 |0.966 |0.984 |0.998 |0.865 |0.710 |0.949 |0.840 |
66
+ |IITD |0.990 |0.868 |0.975 |0.979 |0.843 |0.944 |3.424 |3.564 |0.939 |0.966 |0.861 |
67
+ |LivDet2013-Iris |0.980 |0.853 |0.985 |0.977 |0.973 |0.977 |1.242 |0.518 |0.393 |0.945 |0.851 |
68
+ |LivDet2015-Iris |0.978 |0.844 |0.984 |0.974 |0.955 |0.958 |1.524 |0.794 |0.608 |0.938 |0.870 |
69
+ |LivDet2017-Iris |0.978 |0.791 |0.981 |0.956 |0.956 |0.973 |1.920 |1.274 |1.204 |0.921 |0.738 |
70
+ |MMU |0.983 |0.845 |0.985 |0.980 |0.960 |0.980 |1.405 |0.821 |0.500 |0.946 |0.765 |
71
+ |ND-IRIS-0405 |0.981 |0.750 |0.977 |0.975 |0.918 |0.969 |1.757 |1.653 |0.739 |0.917 |0.676 |
72
+
73
 
74
  ### Time Performance
75
 
 
102
 
103
  Within the ambit of each class, the model formulates probability estimates pertaining to the likelihood of a given pixel being attributed to a particular class.
104
 
105
+ ### Example inference results
106
 
107
  **Note**: The provided input image has been subjected to the processing methodology described earlier, prior to its introduction into the model. Moreover, for the intent of visualization, the IR image presented has been anonymized to safeguard the identity of the user. It is also worth to note that the inference process was conducted on the original, non-anonymized version of the image.
108
 
109
  #### Input image
110
 
111
+ <img src="https://github.com/worldcoin/open-iris/blob/main/docs/model_card/anonymized.png" alt="anonymized input image" width=640 height=480>
112
 
113
  #### Output masks
114
 
115
  **Note**: A threshold of 0.5 was applied on every resultant probability matrix to generate a definitive binary mask for each individual class. Also, every outputted segmentation map was resized to the input image resolution.
116
 
117
+ <img src="https://github.com/worldcoin/open-iris/blob/main/docs/model_card/overlayed_segmaps.png" alt="segmaps">
118
 
119
  ## Limitations
120
 
121
  Thorough examination of the results enabled us to pinpoint situations where the segmentation model experiences declines in performance. These instances are as follows:
122
+ - Segmenting images with high specular reflection coming usually from glasses may lead to bad segmentation map predictions.
123
+ - Data based on which the model was trained were captured in the constrained environment with cooperative users. Therefore, in practice model is expected to produce poor segmentation maps for cases like: offgaze, misaligned eyes, blurry images etc.
 
 
 
 
 
 
 
 
 
 
 
124
 
125
  ## Further reading
126
 
 
128
  2. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. (https://arxiv.org/abs/1801.04381)
129
  3. Roy, A. G., Navab, N., & Wachinger, C. (2018). Recalibrating Fully Convolutional Networks with Spatial and Channel “Squeeze and Excitation” Blocks. (https://arxiv.org/abs/1808.08127v1)
130
  4. Bowyer, K. , Flynn, P. (2016), The ND-IRIS-0405 Iris Image Dataset (https://arxiv.org/abs/1606.04853)
131
+ 5. http://biometrics.idealtest.org/
132
+ 6. http://www4.comp.polyu.edu.hk/csajaykr/IITD/Database_Iris.htm
133
+ 7. D. Yambay, J.S. Doyle, K.W. Bowyer, A. Czajka, S. Schuckers Livdet-iris 2013 - iris liveness detection competition 2013
134
+ 8. D. Yambay, B. Walczak, S. Schuckers, A. Czajka Livdet-iris 2015 - iris liveness detection competition 2015
135
+ 9. D. Yambay, B. Becker, N. Kohli, D. Yadav, A. Czajka, K.W. Bowyer, S. Schuckers, R. Singh, M.Vatsa, A. Noore, D. Gragnaniello, C. Sansone, L. Verdoliva, L. He, Y. Ru, H. Li, N. Liu, Z. Sun, T. Tan Livdet iris 2017 - iris liveness detection competition 2017
136
+ 10. https://mmuexpert.mmu.edu.my/ccteo