Patent Description:
As known, collection of anthropometric measures may be of interest in many fields, ranging from commerce to healthcare. For example, e-commerce systems dealing in garment may assist customers by providing suggestion on size selection based on personal data set including relevant body measures. Personal data may be fed to an automatic categorization system and size may be selected based on generally best fit criteria. This kind of service is not only convenient to the customers, but can dramatically reduce the rate of returned articles and is therefore beneficial also for the providers in terms of cost saving. As another example, in dietetics monitoring of body measures may help patient follow up and tailoring of treatments based on response.

It is apparent that advantages of use of anthropometric data would be greatly increased if data could be collected on the field directly by customers or patients using simple tools that do not require special training or skill to provide sufficiently accurate results. Anthropometric data are generally gathered through image acquisition and analysis and data collection may take advantage from the widespread diffusion of extremely versatile devices such as smartphones. Image capture tools have become so friendly that end users are nearly always expected to be able to frame and acquire images of sufficient quality to perform desired analysis procedures.

Rather, delegating the task of image acquisition to end users involves other issues, that need to be taken into consideration. On the one side, image processing, which is in itself demanding in terms of requirement of resources, must also cope with the fact that acquisition on the field normally suffers from poor conditions in terms of lighting, contrast against background, user posture and the like. Such conditions must be compensated by refining processing techniques, that normally result in increased computational load. In order to provide sufficient processing capacity, only basic capturing functions are devoted to end user devices image analysis, whereas main analysis procedures are carried out at remote sites reached through internet connections. On the other side, sharing images with remote sites leads to concerns regarding privacy and protection of personal data, because sensitive information is involved and law requirements are becoming increasingly strict. As a matter of fact, information relating to personal aspect is sent out of the control of the end user. Clearly, treatment of information is critical especially for medical applications.

Eugenio Alessandro Canepa, "ISizeYou - human measure capture by <NUM> mobile pictures + size suggestion for e-commerce", (<NUM>), Youtube, URL: https://www. com/watch?v=s5S0GzTE0ns discloses an anthropometric data portable acquisition device comprising an image sensor and a processing unit. The processing unit comprises an acquisition module, configured to receive first images of a person from the image sensor; an image processor, configured to provide second images from respective first images by applying edge detection procedure; a communication module, configured to send out the second images.

Other examples of known anthropometric data portable acquisition devices are disclosed in <CIT> and in<NPL>.

It is thus an object of the present invention to provide an anthropometric data portable acquisition device which allows to overcome or at least mitigate the limitations described.

According to the present invention, there are provided an anthropometric data portable acquisition device as defined in claims <NUM>.

The present invention will now be described with reference to the accompanying drawings, which illustrate some non-limitative embodiments thereof, in which:.

In <FIG>, an anthropometric data acquisition system is indicated as a whole by reference number <NUM> and comprises one or more anthropometric data portable acquisition devices <NUM> in accordance with an embodiment of the present invention and a server <NUM>. The portable acquisition devices <NUM> and the server <NUM> are coupled in communication over a wide area network <NUM>, e.g. the internet. The portable acquisition devices <NUM> are configured to capture images of individuals and to perform preliminary image processing steps, which are identified as frontend processing in what follows, whereby data sent to the server for completing extraction of relevant features (backend processing) are cleared of identity information.

One exemplary portable acquisition device <NUM> is shown in <FIG> and will be referred to hereinafter, being it understood that the other portable acquisition devices <NUM> include the same components. The portable acquisition device <NUM> comprises a processing unit <NUM>, a display <NUM> and an image sensor <NUM>. In one embodiment, the portable acquisition device <NUM> may be integrated in a smartphone, a tablet computer or a laptop computer, advantageously, with integrated touchscreen as the display <NUM> and photo camera as the image sensor <NUM>. The display <NUM> need not be a touchscreen, though.

The processing unit <NUM> is configured to implement an acquisition module <NUM>, an image processor <NUM> and a communication module <NUM>.

The acquisition module <NUM> activates the display <NUM> in live view mode during acquisition and captures images from the image sensor <NUM> in response to shooting commands sent by a user through an interface, which may include virtual or hard buttons (not shown). In live view mode, output signals of the image sensor <NUM> are continuously displayed on the display <NUM>. The acquisition module <NUM> also sends information to the display <NUM> for visualization to the user, for the purpose of aiding correct acquisition.

Images captured by the acquisition module <NUM> are supplied to the image processor <NUM>. The acquisition module <NUM> may be provided with some processing capability to perform basic actions such as white balance, brightness and contrast adjustment and conversion into compressed format. As an alternative, images in raw format are sent to the image processor <NUM>, that carries out all desired processing steps.

The image processor <NUM> extracts a modified frontend images from received images. The extracted frontend images are sent to the server <NUM> through the communication module <NUM> for backend processing, that may therefore rely on higher level resources.

Image acquisition is assisted by the acquisition module <NUM>, that provides information to lead the framed person to assume a posture in a range of admissible postures, while an operator takes a photo from the back and a photo on one side. Standard posture remarkably reduces computational load associated with image processing. More specifically (<FIG>), the acquisition module <NUM> starts acquisition of a back image by superimposing a back mask <NUM> to images shown on the display <NUM> in live view. The person can thus be guided to take a posture that fits into the back mask <NUM>, which in one embodiment may require a standing position with legs apart and arms away from the torso. When the posture is correct, shooting commands are activated by the operator and a back image IMGB (<FIG>) is captured by the acquisition module <NUM> and sent to the image processor <NUM>. The back image IMGB may be temporarily stored in a memory unit incorporated either in the acquisition module <NUM> or in the image processor <NUM> or in a general purpose memory unit of the portable acquisition device <NUM>.

Once the back image IMGB has been captured and stored or sent to the image processor <NUM>, the acquisition module <NUM> starts acquisition of a side image and superimposes a side mask <NUM> (<FIG>) to the images shown on the display <NUM> in live view. Again, the person is guided to take a posture that fits into the side mask <NUM>, which requires a standing position with arms lying along the torso. When the posture is correct, the shooting command is activated by the operator and a side image IMGs is captured and sent to the image processor <NUM> or stored like the back image IMGB.

Each of the back image IMGB and side image IMGs is then processed by the image processor <NUM> for edge detection and contour reconstruction substantially through the same step. For the sake of simplicity, reference will be made in what follows to the back image IMGB and it will be understood that the same applies also to the side image IMGs, unless otherwise specified.

With reference to <FIG>, after the side image IMGs has been received in a format suitable for processing, the image processor <NUM> clears portions of the side image IMGs corresponding to locations outside the back mask <NUM> (block <NUM>), thereby reducing noise. Then, edge detection (block <NUM>) and contour reconstruction (block <NUM>) are performed. A frontend back image FIMGB including only body contour and possibly residual background noise is thus obtained and transferred to the server <NUM>. Examples of a frontend back image FIMGB and of a frontend side image FIMGs are shown in <FIG>, respectively. During the step of edge detection, body features which may possibly allow identification of the portrayed person are lost.

Edge detection may be carried out in several manners, but it is generally preferred that contour thickness is fairly constant, because it may be a critical parameter in subsequent backend processing. The image processor <NUM> may be configured to combine different edge detection procedures, in order to enhance performance.

In one embodiment (<FIG>), the image processor <NUM> preliminary applies a noise reduction process to the back image IMGB to mitigate effects of high-frequency noise (block <NUM>). The noise reduction process may be median filtering. It has been found that a size of the kernel of <NUM> pixel in the median filtering operator provides acceptable filtering performance without affecting precision of subsequent edge detection.

Then (block <NUM>), contour of the back image IMGB is first extracted using a Canny edge detector process, which is described in <NPL>. The Canny edge detector is very sensitive to image regions of low contrast which may be actually present. This aspect may be critical, because in image acquisition on the field it is not always possible to choose background providing adequate contrast. In order to avoid loss of information, contour of the back image IMGB is second extracted by the image processor <NUM> using another edge detector process (block <NUM>) and the result of second contour extraction is combined with the result of the first contour extraction by the Canny edge detector in block <NUM> of <FIG>. In one embodiment, the image processor <NUM> uses a structured random forest for edge detection, which is also referred to as structured edge detector. The structured edge detector is less sensitive to local regions of low contrast and is also more robust to noise than the Canny edge detector, thus curing weaknesses of the latter.

<FIG> shows an exemplary structure of the image detector <NUM> implementing the process of <FIG>. In the embodiment illustrated therein, the image detector <NUM> comprises a noise reduction filter <NUM>, receiving the back image IMGB (and the side image IMGs) and feeding in parallel the Canny edge detector and the structured edge detector, here indicated by <NUM> and <NUM>, respectively. A first output I<NUM> of the Canny edge detector <NUM> and a second output I<NUM> of the structured edge detector <NUM> are then combined in a contour reconstruction module <NUM> which combines the results of the first and second contour extraction and supplies the frontend back image FIMGB and the frontend side image FIMGs.

Probabilistic models may be exploited, when sufficient computational capacity is available in the portable acquisition devices <NUM> to have acceptable response delay for users.

A label l∈[L1, L2]IMGB that minimizes an energy function E(l) is assigned to each location in the image IMGB (or IMGs). The energy function has a first component and a second component as follows: <MAT>.

The first component EUNARY(l) is a function which tends to assign to each pixel a value corresponding to the output of the edge detectors with relative probabilities w<NUM> and w<NUM>, respectively <MAT> where <MAT> and w<NUM> and w<NUM> are weight parameters that determine to what extent the Canny edge detector <NUM> and the structured edge detector <NUM> will affect the result.

The second component EPAIRWISE(l) seeks to make neighboring pixels agree on a label, adding a constant penalty for disagreement, and is defined by: <MAT> where <MAT>.

and PAIRS(I) includes all pairs of neighboring image points.

The weight parameter wP determines how strong the agreement between neighboring pixels should be. The optimal solution <MAT> is approximated using the tree reweighted belief propagation algorithm.

In this manner, frontend back image FIMGB and frontend side image FIMGs are obtained from the back image IMGB and from the side image IMGs, respectively, and are sent to the server <NUM> for backend processing. Frontend back image FIMGB and frontend side image FIMGs include complete and reliable body contour of the framed person and no personal identity information is transferred out of user's control. Personal information (e.g. face features) is in fact deleted during the edge detection and contour reconstruction steps without the need of any special dedicated processing and body contour is substantially anonymous. As already mentioned, exemplary frontend back image FIMGB and frontend side image FIMGs are shown in <FIG>, respectively, where also the back mask <NUM> and the side mask <NUM> are depicted to intuitively suggest how image acquisition and the frontend processing are carried out. However the back mask <NUM> and the side mask <NUM> in themselves need not be part of the frontend back image FIMGB and frontend side image FIMGs.

As illustrated in <FIG>, the server <NUM> comprises a communication module <NUM>, which receives the frontend back image FIMGB and the frontend side image FIMGs, and components for backend processing, including a key point detector <NUM>, a projection measure extractor <NUM> and a 3D measure predictor <NUM>.

The key point detector <NUM> is configured to identify key points based on contour line thickness, since frontend images may contain noise, i.e. points that do not correspond to body contour. More precisely, a scan by horizontal lines (e.g. from left to right) is firstly carried out on the frontend back image FIMGB and pairs of subsequent lines of equal thickness are considered as left and right borders of a respective body portion. For example, if the horizontal scan reveals three pairs of subsequent lines of equal thickness, three body portions are identified, which may be left and right border of the left arm for the first pair, left and right border of the torso for the second pair and left and right border of the right arm for the third pair.

By the scan of the frontend back image FIMGB, the key point detector <NUM> identifies head tip, heels, armpits and crotch. Then head tip and heels are identified by a second scan of the frontend side image FIMGs.

The projection measure extractor <NUM> is configured to identify the contour points belonging to the torso, based on armpits and crotch positions as determined by key point detector <NUM> (windows <NUM>, <NUM> in <FIG>). In order to reduce the unevenness which may be introduced by the previous processing steps, the projection measure extractor <NUM> approximates torso lines (<NUM>, <FIG>) by polynomial functions (<NUM>, <FIG>). Given the nature of the torso side, and in particular the expected number of maxima and minima, a fourth degree polynomial is sufficient and at the same time not too complex for the fitting. Also possible missing parts of the contour may be estimated by the projection measure extractor <NUM> by use of polynomial functions, such as Bezier polynomials.

Then, the projection measure extractor <NUM> takes measures of the projections of main body dimensions on the image plane or on plane perpendicular thereto. The main body dimensions may include, for example, neck, chest, waist, hips, inseam. Measures of chest, waist and hips may be taken at fixed and predetermined vertical coordinates (Y-coordinate), e.g. as follows: <MAT> <MAT> <MAT> where CA is the distance between crotch and armpits.

In another embodiment, Ychest is taken at the widest part of upper third of the torso as derived from the frontend side image FIMGs; Ywaist is taken at the smallest part of the central third of the torso as derived from the frontend back image FIMGB; and Yhips is taken at the widest part of the lower third of the torso as derived from the frontend side image FIMGs. Other measures may be added according to design preferences.

The 3D measure predictor <NUM> is configured to provide estimates or predictions of actual body measures starting from the measures o projection measure provided by the projection measure extractor <NUM>. To this end, 3D measure predictor <NUM> may use either a simplified model such as a linear model or a probabilistic model, e.g. a Gaussian Process model. While linear models are more simple and yield widely satisfactory results in line with industry standards in terms of errors, probabilistic models allow to complement the prediction for each measurement with a measure of its uncertainty. Uncertainty can be used to predict estimation errors and trigger appropriate corrective actions to avoid providing the users with wrong output and predictions affected by very high variance.

Gaussian Processes can effectively deal with noise and this is important in order to allow collection on data form images taken on the filed by users without specific training.

Typically, Gaussian Processes may indicated as <MAT> where m(x)=E[f(x)] is the expected value and k(x,x')=E[(f(x)-m(x))(f(x')-m(x'))] is the covariance function.

The function modelled by the Gaussian Process is the one predicting 3D body measurements starting from the length of their planar projections as provided by the projection measure extractor <NUM>. Covariance function allows us to obtain a non-linear mapping and give flexibility in the specific form of non-linearity which is allowed. Due to their mathematical properties, covariance functions can be combined together to obtain new ones showing the desired characteristics. In the one embodiment, the sum of <NUM> different covariance functions:.

On its output, the 3D measure predictor <NUM> provides a data set DS which includes estimates of the actual (3D) size of relevant parts of the body of the framed person.

Claim 1:
An anthropometric data portable acquisition device comprising an image sensor (<NUM>) and a processing unit (<NUM>), the processing unit (<NUM>) comprising:
an acquisition module (<NUM>), configured to receive first images (IMGB, IMGs) of a person from the image sensor (<NUM>) ;
an image processor (<NUM>), configured to provide second images (FIMGB, FIMGS) from respective first images (IMGB, IMGs) by applying edge detection procedure;
a communication module (<NUM>), configured to send out the second images (FIMGB, FIMGS);
characterized in that the image processor (<NUM>) comprises a first edge detector (<NUM>), configured to perform a first contour extraction, a second edge detector (<NUM>), configured to perform a second contour extraction, and a contour reconstruction module (<NUM>), configured to combine results of the first contour extraction performed by the first edge detector (<NUM>) and of the second contour extraction performed by the second edge detector (<NUM>) and to provide the second images (FIMGB, FIMGs);
wherein the first edge detector (<NUM>) is a Canny edge detector and the second edge detector (<NUM>) is a structured edge detector;
wherein the first images (IMGB, IMGs) are fed in parallel to the first edge detector (<NUM>) and to the second edge detector (<NUM>);
and wherein the contour reconstruction module (<NUM>) is configured to assign a label (<NUM>) that minimizes an energy function E(l) to each location in the first images (IMGB, IMGs) and wherein the energy function has a first component EUNARY(l) and a second component EPAIRWISE(l) and is defined as follows: <MAT> <MAT>
where <MAT>
I<NUM> is a first output of the first edge detector (<NUM>), I<NUM> is a second output of the second edge detector (<NUM>) and w<NUM> and w<NUM> are weight parameters <MAT>
where <MAT>
And wP is a weight parameter.