# A New Approach to Image-Based Estimation of Food Volume
## 2 Method
### Size/Ground Reference Localization  
Volume can be estimated from images using several different procedures, but results are computed only up to a scale factor. Therefore, in many studies, an object of known size is used as a reference for volume estimation. In [8], for instance, the user puts his/her finger besides the dish. In [6], a specific pattern of known size printed on a card is used as a reference. Many studies have followed the same idea using a checkerboard as a reference [13; 22] because of the simplicity of the checkerboard pattern and the existence of effective algorithms to detect it. Nevertheless, off-the-shelf checkerboard detection algorithms are usually designed to be means for camera calibration or pose-detection processes. They are usually tuned for specific situations like: flat checkerboards, a big checkerboard that is often the only object in the image, etc. Thus, for applications that do not satisfy these requirements, different algorithms, or modified versions of available ones, are needed. Checkerboards that are used as size references usually consist of few squares and occupy a relatively small region of the image. This makes detecting them hard for'standard' checkerboard detectors. Figure 4 shows three examples in which the checkerboard detection algorithms provided by the open software package OpenCV 3.1.0 [23] and Matlab R2014b (The MathWorks Inc., Natick, MA, USA) fail. In [12], we have introduced a method to locate a small checkerboard in an image. We used a \(5\times 5\) checkerboard, printed on a Polyvinyl Chloride (PVC) card, as a size reference. The method consists of two main steps: locating the checkerboard in the image and then detecting the checkerboard corners in the located area. The procedure that we adopt estimates the pose of an object based on a 3D model and can be utilized with any projection system and any general object model. Figure 5 shows the model that we use to detect the checkerboard. It consists of 73 key points corresponding to the center, edges and corners of each square. In this method, a hypothesis of checkerboard location can be evaluated by rigidly transforming a model of the checkerboard and then projecting it onto the image according to a perspective transform. The likelihood of the hypothesis is evaluated using a similarity measure between the projected model and the actual content of the image region onto which the pattern is projected. Figure 4: First row: typical images used for calibration. Second row: examples of images in which a checkerboard is used as size reference for food volume estimation, on which both OpenCV and Matlab functions for detecting checkerboards fail, while our model-based algorithm correctly locates them (green areas). This procedure allows one to turn checkerboard detection into an optimization problem, in which the parameters to be optimized are the coefficients of the rigid transformation and of the projection. In this work, Differential Evolution (DE) [24] is employed for optimization as described in [25]. Algorithm 3 reports the pseudo-code of the fitness function used to evaluate each hypothesis. In the algorithm, each step presented in Figure 5 is implemented as a separate function. One of the advantages of using a stochastic algorithm like DE is having a chance of success in the next try if an attempt to find the checkerboard fails. ```
functionFitnessFunction(PoseVector)  Calc Rotation & Translation matrices from PoseVector \(Score\gets Score+FirstLevelCenterCheck()\) ifScore\(>6\)then \(Score\gets Score+SecondLevelCenterCheck()\) endif ifScore\(>20\)&& the center is blackthen \(Score\gets Score+PoseCheck()\) endif ifScore\(>23\)then \(Score\gets Score+EdgesCheck()\) \(Score\gets Score+VerticesCheck()\) endif returnScore endfunction
```  
**Algorithm 3** Fitness Function  
Based on this estimated checkerboard location, the corners of the checkerboard are then precisely detected as described in [12].