COMPUTER GRAPHICS forum

Pacific Graphics 2023
R. Chaine, Z. Deng, and M. H. Kim(Guest Editors)

Volume 42 (2023), Number 7

Refinement of Hair Geometry by Strand Integration
Ryota Maeda1†, Kenshi Takayama2, Takafumi Taketomi2

1 University of Hyogo, Japan 2 CyberAgent, Japan

StrandcIntegration

Initial point cloud from LPMVSResult of refinement

Figure 1: Example result of shape refinement. Our strand integration technique refines the inaccurate hair strand by integrating the gradientalong the strand. Left: Initial hair geometry from LPMVS [NWKS19]. Each point has 3D direction information illustrated as black arrows(while the red lines are for the purpose of illustration only and are not generated). Right: Refined hair geometry using our method. Our
method improves the accuracy of the hair geometry without any additional data.
Abstract
Reconstructing 3D hair is challenging due to its complex micro-scale geometry, and is of essential importance for the efficientcreation of high-fidelity virtual humans. Existing hair capture methods based on multi-view stereo tend to generate resultsthat are noisy and inaccurate. In this study, we propose a refinement method for hair geometry by incorporating the gradientof strands into the computation of their position. We formulate a gradient integration strategy for hair strands. We evaluate
the performance of our method using a synthetic multi-view dataset containing four hairstyles, and show that our refinement
produces more accurate hair geometry. Furthermore, we tested our method with a real image input. Our method produces aplausible result. Our source code is publicly available at https://github.com/elerac/strand_integration.
CCS Concepts• Computing methodologies → Reconstruction; Shape modeling;

1. Introduction
Hair is an important aspect of our appearance, and it has a significant impact on how we look and feel. Modeling realistic and† Work done during an internship at CyberAgent AI Lab.
© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

natural-looking hair is essential to creating virtual humans for many
applications such as VR, AR, and online commerce. However, hairmodeling is a laborious and time-consuming task that requires technical expertise and artistic skill, especially when trying to achievea specific hairstyle.
Automatic hair generation using deep neural networks has2 of 9Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integration

widely been investigated in graphics and vision [RSW∗ 22,
KCF∗ 22]. 3D hair reconstruction from a single portrait can in particular drastically improve a hair modeling workflow [WYY∗ 22,
YZM∗ 23]. These approaches are, however, difficult to generalize
due to diverse variations in hairstyles.
On the other hand, directly capturing actual hair strands using a multi-camera system has also been explored. Although special equipment is needed, various hairstyles can be reconstructed.
To reconstruct hair strands from multiple-view images, the multiview stereo (MVS) algorithm is specialized by assuming 3D local lines instead of using 3D local planes [PCK∗ 08, HMLL14,
NWKS19, SNA∗ 21]. In these methods, reconstructed points areassociated with direction information. In particular, Nam et al.
proposed an algorithm for accurately reconstructing hair strands
named line-based PatchMatch MVS (LPMVS) [NWKS19]. While
their method achieves remarkable hair reconstruction results, theintermediate raw output of reconstructed strands still exhibits significant amount of noise. In the original paper, they address thisissue by eliminating the inconsistent 3D lines and selecting only a
few reliable ones. Consequently, this approach reduces the overall
density of the 3D lines, resulting in sparse reconstruction results.
In the 3D vision research field, normal integration is a classical technique for surface reconstruction that determines shape froma set of normal vectors [HB86]. This technique is beneficial forreconstructing the detailed shape, whereas the MVS technique issuited to the overall shape. The basic idea is that the gradient equals
the differentiation of shape. As a result, we can reconstruct the object shape by integrating normals. Although numerous normal integration approaches have been proposed [QDA18], all of them stillneed to be modified to be applied to hair strands because existing
approaches assume smooth surfaces.
In this study, we propose a refinement method for hair strands by
integrating gradients with a line assumption, as illustrated in Fig. 1.
We start with a noisy 3D line map (depth and direction) obtained
through the LPMVS technique and refine it for each view. The refined lines are merged to reconstruct the final hair geometry. Unlike
the previous approach, our refinement process can restore the noisy
3D lines instead of eliminating them. It can preserve 3D lines andreconstruct dense and less noisy hair shapes. To accomplish this,inspired by the normal integration method, we leverage the gradient for correcting the noisy lines. Compared to the traditional normal integration, our method extracts the gradient from the directionof the 3D line map rather than the normal vector of a plane. Furthermore, in our method, the directions of integration are definedby the 2D hair directions estimated on the input images. Additionally, as an option to further improve the quality, our method canintegrate normal vectors estimated by the photometric stereo technique [Woo80]. Because our method adapts the concept of normal
integration to the task of strand reconstruction, we call our method
Strand Integration.

2. Related Work
There are several approaches for reconstructing the 3D shape of
hair from a captured image input. 3D hair reconstruction from asingle portrait is actively studied because it can drastically reducethe modeling cost. As 3D reconstruction from a single portrait isan ill-posed problem, these methods estimate the 3D hair shapeusing prior knowledge. For example, information from the user’s
sketch is used to determine the flow of the hair [CWW∗ 12,CLS∗ 15,
HMLL15]. Neural networks are also used to estimate the 3D shapeof the hair [CSW∗ 16, ZHX∗ 18, SHM∗ 18, WYY∗ 22, YZM∗ 23].
This type of data-driven approach can predict the entire 3D shapeof the hair. Although the single-view and data-driven approach isbeneficial for easier deployment, it struggles in reconstructing theinvisible area, and tends to fail to reconstruct irregular hairstyles
that are not well represented in the training dataset.
The multi-view camera system can capture accurate hair shapes
without relying on prior knowledge. Pioneering research by Paris etal. introduced a method to reconstruct a hair shape from multi-view
images [PBS04]. Their approach involves analyzing the orientationof hair strands to recover the 3D orientation or 3D lines. Building upon their research, subsequent work has steadily improved
the quality of hair reconstruction [WOQS05, PCK∗ 08, LLP∗ 12,
LZZR13,LLR13,HMLL14]. In addition, some research uses a special capture approach or setup for hair reconstruction such as focal stack photography [JMM09], thermal camera [HZW12], and
RGB-D camera [ZWW∗ 18]. These unique setups can acquire additional physical cues to help reconstruct hair. Moreover, some studies have employed a learning-based approach with multi-view images [ZCW∗ 17, KCF∗ 22].
Among these multi-view hair capture approaches, Nam et al.
proposed a successful hair reconstruction method called LPMVS[NWKS19]. This method expands upon a traditional MVS for hairstrands by considering hair strands as 3D lines rather than 3D
planes. Our method uses LPMVS to generate the initial input ofrough hair shape and subsequently improves its quality through refinement. Some of their subsequent works use LPMVS. For example, Sun et al. combined photometric images as the feature ofmatching strands [SNA∗ 21]. In our approach, we also use photometric images, but we employ them to extract normal vectors andintegrate them into refinement. Recent advanced neural approaches
represent hair features in a latent space, enabling the rendering ofrealistic hair images [RSW∗ 22, WNS∗ 22]. These methods incorporate the output of LPMVS as a geometric guide during optimization, and can benefit from the improved accuracy of prior hairshapes offered by our method. In this sense, our work is complementary to these neural methods and has a potential to contributeto the overall quality improvement of hair reconstruction.

3. Strand Integration
Our strand integration algorithm is inspired by normal integration.
Normal integration, also known as shape from gradients, is a wellknown method in computer vision that reconstructs a 3D surface
from its normal vectors. This method was originally introduced by
Horn and Brooks in 1986 [HB86]. Since then, numerous improved
methods have been proposed over several decades [QDA18]. Despite the advancements, it remains a challenging task. Our work
adapts this idea to hair strands by leveraging the 3D direction as a
gradient.
In this section, we describe the details of our proposed method;© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

3 of 9Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integration

StrandIntegration

LPMVS[Nam et al.]Depth

Input images

Direction

Refined 3D lines

Depth (refined)

Optional (if normal available)

Photometric
StereoPhotometric images

NormalRAW 3D lines of LPMVS

Figure 2: Overall pipeline of our hair reconstruction. The input images are captured under a multi-view camera condition. As a preprocessing, we reconstruct 3D lines (depth and direction) using LPMVS [NWKS19]. These 3D lines become the input to our method, Strand
Integration, which refines the depth map for each view. While our method can work without any extra input, we can also optionally accept anormal map as an additional input. We can obtain the hair geometry as a set of 3D lines by accumulating.

see Fig. 2 for an overview. The input of our method is a 3D line
map which is obtained as the output of LPMVS [NWKS19]. This
3D line map comprises the depth map (3D points) and their 3D directions. The basic concept of our strand integration is to harness
the position and direction information for improving geometricalcoherence. If we move along a 3D line following its direction, we
will likely encounter another 3D line that represents another pieceof the same strand. By following successive 3D lines, we can determine the shape of an entire strand of hair. We formulate thisrelationship similar to normal integration but for 3D lines.
We first introduce the notation of the geometrical parametersof the hair strand (Fig. 3). When a calibrated camera captures
the hair, its strand is projected to a point on the image planeu = [u, v]⊤ ∈ R2 . We define the area where at least one strand
is projected as Ω, and let |Ω| denote its number of pixels. The
prior hair strands obtained by LPMVS are represented as a pairof depth map zprior (u) ∈ R and the 3D direction map dprior (u) =[dx,prior , dy,prior , dz,prior ](u)⊤ ∈ S 2 ⊂ R3 in the camera coordinate
system where u ∈ Ω. We denote the angle of the strand on the image plane as θ(u) = − arctan(dy,prior /dx,prior ) ∈ [−90◦ , 90◦ ).
3.1. Notation of Strand
Resolving Directional Ambiguity. Note that there exists an ambiguity regarding the directionality of a 3D line: the 3D direction
vector dprior and its flipped version −dprior can represent the same
3D line. We eliminate this ambiguity by flipping the direction vector as needed such that the X component of the direction dx,prior isalways positive.

Strand

𝑣

𝐩

𝑢
Ω

𝐮

𝜃

𝑧
𝑦

𝐝

Image plane𝑥

Figure 3: Notation of strand. The 3D line as a strand is projectedto the image plane.

tional derivative of the depth as∇θ z = [cos θ, − sin θ]⊤ · ∇z
where the gradient of the depth is computed by1z(u + 1, v) − z(u, v).
∇z =δ(u, v) z(u, v + 1) − z(u, v)

(1)

(2)

Here, δ represents the length of one pixel seen at depth z(u, v) obtained asδ(u, v) =z(u, v)f(3)

where f is the camera’s focal length.
With Equation (1), we can derive the 3D strand direction as

3.2. Deriving 3D Direction from Depth
In the following, we formulate the derivation of the 3D strand direction d from the to-be-optimized depth z. First, we take the direc© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

d = ⟨[cos θ, − sin θ, ∇θ z]⊤ ⟩(4)where ⟨·⟩ denotes the normalization operation. We are particularly

4 of 9Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integrationinterested in its Z component∇θ z
dz = p
1 + (∇θ z)2(5)

which is used in our loss function introduced in the next subsection.
3.3. Loss Function

Using these two operators, we obtain two versions of the estimatedstrand direction, d+ and d− following the same Equation (4). We
then modify the direction loss and the normal loss as2
2
dz+ − dz,prior + dz− − dz,prior
1
Ld (z) =
(11)
|Ω| ∑
2
Ω
Ln (z) =Our strand integration finds the depth map which minimizes a loss
function consisting of the following terms.

(nprior · d+ )2 + (nprior · d− )2
1
.
∑
|Ω| Ω
2

(12)

Direction Loss. The direction loss Ld evaluates the difference between the prior direction dprior and the direction d computed fromthe unknown depth using Equation (4). Because we are optimizing
the depth, only the Z component needs to be compared:2
1
Ld (z) =dz − dz,prior .
(6)
|Ω| ∑
Ω

3.5. 3D Line Consistency Map

Depth Loss. The depth loss Lz is a regularization term which compares the unknown depth z to the prior depth zprior :Specifically, for each pixel on the reference view, we turn theprior depth zprior into a 3D point pprior in the world coordinate system using the camera information, and then project it to each of
the neighboring views. On the neighboring view i, the projected
location has its prior depth zi,prior which can then be turned intoa world-coordinate 3D point pi,prior . We compute the squared distance between these two points, and take the weighted average ofthis value over the set of neighboring views:

Lz (z) =1c(z − zprior )2 .
|Ω| ∑
Ω

(7)

Here, we introduce a per-pixel weight c; the higher the weight, themore consistent the 3D line with respect to neighboring views. We
defer the details on how to compute this weight to Subsection 3.5.
With this weighting scheme, we can effectively incorporate themultiview constraint into the reconstruction process.

In this subsection, we give details on how to compute the consistency weight map c used in the depth loss (Equation (7)). Our
method is inspired by the consistency checking method used in thefiltering step of LPMVS [NWKS19] (Subsection 4.2): we projectthe 3D line on the reference view to its neighboring views, andcheck for the consistency in terms of both position and direction.

N

r=

nei
wi ||pprior − pi,prior ||2
∑i=1

N

nei
wi
∑i=1

.

(13)

Normal Loss. This loss is optional but has the potential to improvereconstruction quality. If we obtain the normal vector nprior of hairas prior knowledge (e.g., by using photometric stereo or polarization imaging), we can incorporate it as an additional constraint. A
strand’s normal vector should be perpendicular to its 3D direction;i.e., the dot product of these vectors should be small. Accordingly,
we define the normal loss Ln as

Here, the weight wi is derived by considering the consistency ofthe directions of the corresponding 3D lines. Specifically, let dpriorand di,prior denote the 3D direction vectors associated with the 3Dlines on the reference view and the neighbor view i, respectively,
expressed in the world coordinate system. We define the weight as

1
Ln (z) =(nprior · d)2 .
|Ω| ∑Ω

where ∠(·, ·) measures the angle between given two 3D lines. Thisdefinition allows us to ignore outlier or occluded lines in the neighboring views when computing the weighted average of the squared
distances.

(8)

Total Loss. We combine these loss functions into our total loss
function Ltotal defined as
Ltotal (z) = Lz (z) + λd Ld (z) + λn Ln (z)(9)

where λd and λn are the weight parameters controlling the importance of the individual terms. We optimize the depth z such that the
total loss is minimized by using a gradient descent method.
3.4. Robust Discretization
For making our algorithm more robust to noisy 3D lines, we find it
useful to use both the forward and backward differencing schemes
when computing the derivative. We denote the forward differencing operator introduced in Equation (2) by ∇+ z, and likewise introduce the backward differencing operator as1z(u, v) − z(u − 1, v)∇− z =.
(10)
δ(u, v) z(u, v) − z(u, v − 1)

wi = 90◦ − ∠(dprior , di,prior ),(14)

We plug this weighted average of the squared distances into theGaussian kernel to derive the consistency weight:r 
c = exp − 2 ,(15)
2σwhere σ (set to 25mm) is the radius of the Gaussian kernel which
controls the tolerance of depth inconsistency.
Note that we use this consistency map for the depth loss only,
because we found that the direction map estimated by LPMVS ismuch more reliable than the depth map for most pixels.
4. Experiment
We evaluated our method using four synthetic hair geometry data
released by Cem Yuksel [YSK09] (referred to as Straight, Curly,
Wavy and WavyThin) as well as an actual multi-view capture of areal human subject (referred to as Real), as reported below.
© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

5 of 9Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integration

Table 1: Aggregated error across all the views. Our method consistently achieves lower errors than LPMVS.
TopStraight

LPMVS
Ours

Back

Figure 4: Camera poses of our synthetic data. We position thecameras denser on the back side of the head.

4.1. Implementation Details
Since there exists no publicly-available implementation of LPMVS [NWKS19], we created a CPU-based reimplementation of
it in C++ with parallelization via OpenMP. We implemented
our strand integration algorithm using PyTorch, and we used
Adam [KB15] to minimize the loss function in Equation (9) andused the exponential learning rate decay scheme such that the learning rate is scaled down by 0.01 in the last iteration. The initial learning rate was set to 1.0 for all the synthetic cases, while it was set to0.01 for the Real case. We adjusted the total number of iterations
depending on the geometric complexity: 30k for the Straight case,
40k for the Wavy and WavyThin cases, and 50k for the Curly andReal cases. When running on a MacBook Pro with the Apple M1
Max chip and using PyTorch’s GPU support, the typical per-view
running time of our strand integration algorithm for the Straightcase was about 20m.

4.2. Evaluation with Synthetic Data
Data Preparation. We used pbrt-v4 to render the hair geometry
data into images of resolution 2730 × 4096. As shown in Fig. 4, weplaced 60 cameras around the head model with more (less) cameras placed on the back (front) side, simulating an actual multiview capture setup. We used 30 directional light sources evenly distributed in all directions, and rendered images in the one-light-at-atime (OLAT) fashion. We fed this data to the classical photometricstereo technique [Woo80], assuming the Lambertian reflectance, toobtain normal maps needed by our normal loss (Equation (8)). We
generated fully lit images by summing all the OLAT images, andused them as input to LPMVS [NWKS19]. Using the “gbuffer”
functionality of pbrt-v4, we generated ground-truth depth maps aswell as the mask images representing the hair region Ω.
© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

Curly

Wavy

Wavythin

MAE ↓RMSE ↓MAE ↓RMSE ↓MAE ↓RMSE ↓MAE ↓RMSE ↓

34.58
12.79

54.20
18.56

37.60
17.08

59.55
26.09

31.62
11.43

53.06
18.68

32.64
15.11

52.61
23.50

Error Analysis on Depth Maps. Using the ground-truth depth
maps, we analyzed the errors of the depth maps generated by LPMVS [NWKS19], our method without normal loss (λd = 72, λn =0), and our method with normal loss (λd = 72, λn = 36). Note that
we determined these weight values relative to the depth loss whichis assumed to be in the unit of mm2 . We report the mean-absoluteerror (MAE) and root-mean-squared-error (RMSE) of the depth
maps (both in mm) in Fig. 5 for a single view from the back, and in
Table 1 for the aggregated values over all the views. We can see that
our method consistently achieved lower error compared to LPMVS
for all the cases, and the use of normal loss proved effective for all
but the Wavy case where there was no significant difference. Fig. 6
shows how the error varies depending on the direction loss weightλd (the normal loss weight λn was fixed to 0), indicating that ouralgorithm can consistently achieve higher accuracy than LPMVS
without relying on hyperparameter tuning.
Merged Hair Geometry. After obtaining depth maps of all the
views, we merge the point clouds of all the views into one point
cloud. Fig. 7 shows two versions of merged point clouds: those that
use the filtering scheme of LPMVS [NWKS19] (Subsection 4.2)
and those that do not. For the filtering, we set the positional threshold τ p and the directional threshold τd to 7.3mm and 10◦ , respectively, and kept points that satisfy the multi-view consistency criteria in at least 2 out of 6 neighboring views. Note that our method’s
merged point clouds come from the depth maps generated without
using the normal loss. We can clearly see that the unfiltered resultsof LPMVS contain a significant amount of outlier points, while thefiltered results of our method retain significantly more points thanks
to the more accurate depth maps obtained by our strand integration.
An exception to the above assessment is observed with the Curly
hairstyle where the number of remaining points after filtering ishigher for the result of LPMVS than ours (37M vs. 34M). A plausible explanation for this is that the number of remaining points after
filtering is likely not the most useful measure of success, especiallywhen it comes to complex and heavily occluded hairstyles such as
Curly. To show this, we ran the same filtering on the ground truth
point clouds and confirmed that for the Curly hairstyle, 49.6% of
points were removed, whereas only 9.8% of points were removed
for the Straight hairstyle. This significant reduction of points is because a ground truth 3D point seen from one viewpoint may often
be invisible from its neighboring viewpoints due to occlusion. As
such, we deem the reported numbers for the Curly hairstyle to beless meaningful. This insight also suggests a better way of filtering
the point cloud, a subject for future work.
4.3. Evaluation with Real Data
Capture Setup. We photographed a real female person with long
hairs in a multi-view capture setup with 60 cameras arranged in a

6 of 9Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integration

Our refinement Our refinement
LPMVS
Ground Truth[NWKS19] (w/o normal) (w/ normal)

Our refinement Our refinement
LPMVS[NWKS19] (w/o normal) (w/ normal)

Point cloud

Ground Truth

Depth map

Near

Far

Absolute error

40

030.05 / 52.57

9.76 / 15.03

9.67 / 14.92(a) Straight

18.10 / 27.07

17.86 / 26.97(b) Curly

Our refinement Our refinement
LPMVS
Ground Truth[NWKS19] (w/o normal) (w/ normal)

Our refinement Our refinement
LPMVS[NWKS19] (w/o normal) (w/ normal)

Point cloud

Ground Truth

41.47 / 67.87

Depth map

Near

Far

Absolute error

40

034.21 / 60.30(c) Wavy

12.20 / 20.57

12.25 / 20.53

34.96 / 58.54

11.34 / 19.36

11.23 / 19.13(d) Wavythin

Figure 5: Error analysis on synthetic data. Top: Ground truth point cloud and estimated 3D line maps. The color represents the 3D direction(The xyz components of dprior corresponds to RGB, respectively.). Middle: Depth maps. Bottom: Absolute error maps of the depth map. The
numbers represent MAE / RMSE (in mm).
© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integration

7 of 9methods also encounter similar challenges. One possible way to
overcome this issue may be to adapt the one-sided partial derivatives [CSS∗ 22] to our strand integration task.

Figure 6: Impact of weight parameter on the accuracy. We ploterrors resulting from varying λd on the Straight case (Fig 5(a)).
Our strand integration produces more accurate depth maps than
LPMVS over a wide range of values. Note that this result does not
use the normal loss.

way almost identical to the one used in our experiment reported in
the previous section, where the image resolution was 5315 × 8001.
Note that our capture setup was not in the OLAT fashion, so we did
not use the normal loss in our strand integration. We used the facer
toolkit [ZYZ∗ 22] for generating masks of the hair regions. Since
we found that the 3D hair directions estimated by LPMVS tended
to be less reliable compared to the synthetic cases, we set λd = 5
so that the depth maps would not get drifted too much by the noisy
directional prior.
Result. Similar to our experiment reported in the previous section,
Fig. 8 shows both the unfiltered and filtered point clouds resulting from LPMVS [NWKS19] and our method. As expected, ourmethod leaves much more points after filtering compared to LPMVS thanks to the higher accuracy in our optimized depth maps.
5. Conclusion
In this paper, we proposed a refinement method for hair geometry by integrating the gradient of strands with line assumption. We
conducted an evaluation using both synthetically rendered images
and real photographs. The result verified that our method improvesreconstruction quality for different hairstyles. We believe that thisresult has the potential to benefit various downstream tasks such as(semi-)automatic modeling of long hair strands as well as trainingof neural networks that infer hair geometry.
Limitations and Future Work. Our method assumes that hair is
continuous and coherent everywhere, implying that the adjacentpixels belong to the same hair strand. This assumption can, however, break when the hair is strongly curled or scattered (e.g., theCurly case). Nevertheless, our method is less sensitive to discontinuity than the typical normal integration methods thanks to our
depth loss (Eq. 7) that partially mitigates this issue by fixing thedepth values that are deemed reliable. Our method can also struggle
with certain hairstyles (even without curls) where multiple groupsof hair strands with different directions are layered on top of eachother. Such sudden changes in the strand direction will cause our
integration method to generate inaccurate hair shape. This limitation is not unique to our method, as standard normal integration© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

While our input is multi-view, our refinement algorithm is inherently single-view, leveraging the multi-view information only
indirectly through the use of the consistency weight term in thedepth loss (Equation (7)). It may be worthwhile to explore othermore direct ways of introducing the multi-view constraints into therefinement process. In particular, integrating our refinement idea
into the LPMVS algorithm itself could be an interesting avenue for
future work.
To use the resulting oriented point cloud for downstream applications such as rendering and simulation, one would need to convert
it into a set of long 3D strands by using the strand-growing algorithm described in the original LPMVS paper or by utilizing it asa prior for neural rendering [RSW∗ 22, WNS∗ 22]. Either way, ourrefined result will improve the overall quality of these tasks.
References[CLS∗ 15] C HAI M., L UO L., S UNKAVALLI K., C ARR N., H ADAP S.,Z HOU K.: High-quality hair modeling from a single portrait photo. ACM
Transactions on Graphics (TOG) 34, 6 (2015), 1–10. 2[CSS∗ 22] C AO X., S ANTO H., S HI B., O KURA F., M ATSUSHITA Y.:
Bilateral normal integration. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings,
Part I (2022), Springer, pp. 552–567. 7[CSW∗ 16] C HAI M., S HAO T., W U H., W ENG Y., Z HOU K.: Autohair:
Fully automatic hair modeling from a single image. ACM Transactions
on Graphics 35, 4 (2016). 2[CWW∗ 12] C HAI M., WANG L., W ENG Y., Y U Y., G UO B., Z HOU K.:
Single-view hair modeling for portrait manipulation. ACM Transactions
on Graphics (TOG) 31, 4 (2012), 1–8. 2[HB86] H ORN B. K., B ROOKS M. J.: The variational approach to shapefrom shading. Computer Vision, Graphics, and Image Processing 33, 2(1986), 174–208. 2[HMLL14] H U L., M A C., L UO L., L I H.: Robust hair capture using simulated examples. ACM Transactions on Graphics (TOG) 33, 4(2014), 1–10. 2[HMLL15] H U L., M A C., L UO L., L I H.: Single-view hair modelingusing a hairstyle database. ACM Transactions on Graphics (ToG) 34, 4(2015), 1–9. 2[HZW12] H ERRERA T. L., Z INKE A., W EBER A.: Lighting hair fromthe inside: A thermal approach to hair reconstruction. ACM Transactions
on Graphics (TOG) 31, 6 (2012), 1–9. 2[JMM09] JAKOB W., M OON J. T., M ARSCHNER S.: Capturing hair assemblies fiber by fiber. In ACM SIGGRAPH Asia 2009 papers. 2009,
pp. 1–9. 2[KB15] K INGMA D. P., BA J.: Adam: A method for stochastic optimization. In Proceedings of International Conference on Learning Representations (ICLR) (2015). 5[KCF∗ 22] K UANG Z., C HEN Y., F U H., Z HOU K., Z HENG Y.: Deepmvshair: Deep hair modeling from sparse views. In SIGGRAPH Asia
2022 Conference Papers (2022), pp. 1–8. 2[LLP∗ 12] L UO L., L I H., PARIS S., W EISE T., PAULY M.,
RUSINKIEWICZ S.: Multi-view hair capture using orientation fields.
In 2012 IEEE Conference on Computer Vision and Pattern Recognition(2012), IEEE, pp. 1490–1497. 2[LLR13] L UO L., L I H., RUSINKIEWICZ S.: Structure-aware hair capture. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1–12. 2

8 of 9Straight

Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integration

297M

85M

175M

226M

226M

37M

34M

314M

314M

81M

122M

204M
LPMVS

204M
Ours

51M
LPMVS

62M
Ours

Wavythin

Wavy

Curly

297M

Rendered image

Unfiltered

Filtered

Figure 7: Final merged hair shapes. We show both unfiltered and filtered 3D lines. Note that viewpoints of rendered images as a reference
are different from the rendered point clouds. The value expressed in millions (M) below is the total count of 3D lines. A higher value indicatesdenser 3D lines. Before applying the filtering, the raw 3D lines obtained from LPMVS contain a significant number of noisy lines. In contrast,
our refinement can reduce noisy lines while preserving the overall line count.

© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

9 of 9Ryota Maeda & Kenshi Takayama & Takafumi Taketomi / Refinement of Hair Geometry by Strand Integration

Photograph

1056M
LPMVS

1056M
Ours
Unfiltered

214M
LPMVS

300M
Ours
Filtered

Figure 8: Reconstruction result of real hair. The value shown below is the total count of 3D lines in million (M). Also in a real capture
scenario, our refinement method achieves less noise and denser 3D lines.

[LZZR13] L UO L., Z HANG C., Z HANG Z., RUSINKIEWICZ S.: Widebaseline hair capture using strand-based refinement. In Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition(2013), pp. 265–272. 2[NWKS19] NAM G., W U C., K IM M. H., S HEIKH Y.: Strand-accurate
multi-view hair capture. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition (2019), pp. 155–164. 1, 2, 3,
4, 5, 6, 7
[PBS04] PARIS S., B RICENO H. M., S ILLION F. X.: Capture of hairgeometry from multiple images. ACM transactions on graphics (TOG)
23, 3 (2004), 712–719. 2[PCK∗ 08] PARIS S., C HANG W., KOZHUSHNYAN O. I., JAROSZ W.,
M ATUSIK W., Z WICKER M., D URAND F.: Hair photobooth: geometricand photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3(2008), 30. 2[QDA18] Q UÉAU Y., D UROU J.-D., AUJOL J.-F.: Normal integration:a survey. Journal of Mathematical Imaging and Vision 60 (2018), 576–
593. 2[RSW∗ 22] ROSU R. A., S AITO S., WANG Z., W U C., B EHNKE S.,
NAM G.: Neural strands: Learning hair geometry and appearance frommulti-view images. In Computer Vision–ECCV 2022: 17th European
Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, PartXXXIII (2022), Springer, pp. 73–89. 2, 7
[SHM∗ 18]

S AITO S., H U L., M A C., I BAYASHI H., L UO L., L I H.: 3dhair synthesis using volumetric variational autoencoders. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–12. 2[SNA∗ 21] S UN T., NAM G., A LIAGA C., H ERY C., R AMAMOORTHI
R.: Human hair inverse rendering using multi-view photometric data. 2[WNS∗ 22] WANG Z., NAM G., S TUYCK T., L OMBARDI S., Z OLL HÖFER M., H ODGINS J., L ASSNER C.: Hvh: Learning a hybrid neuralvolumetric representation for dynamic hair performance capture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition (2022), pp. 6143–6154. 2, 7
[Woo80] W OODHAM R. J.: Photometric method for determining surface orientation from multiple images. Optical engineering 19, 1 (1980),
139–144. 2, 5
[WOQS05] W EI Y., O FEK E., Q UAN L., S HUM H.-Y.: Modeling hairfrom multiple views. In ACM SIGGRAPH 2005 Papers. 2005, pp. 816–
820. 2© 2023 Eurographics - The European Association
for Computer Graphics and John Wiley & Sons Ltd.

[WYY∗ 22] W U K., Y E Y., YANG L., F U H., Z HOU K., Z HENG
Y.: Neuralhdhair: Automatic high-fidelity hair modeling from a single image using implicit neural representations. In Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition(2022), pp. 1526–1535. 2[YSK09] Y UKSEL C., S CHAEFER S., K EYSER J.: Hair meshes. ACM
Transactions on Graphics (TOG) 28, 5 (2009), 1–7. 4[YZM∗ 23] Y UJIAN Z., Z IRONG J., M ORAN L., H AIBIN H.,
C HONGYANG M., S HUGUANG C., X IAOGUANG H.: Hairstep: Transfer synthetic to real using strand and depth maps for single-view 3d hairmodeling. In Proceedings of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition (2023). 2[ZCW∗ 17] Z HANG M., C HAI M., W U H., YANG H., Z HOU K.: A datadriven approach to four-view image-based hair modeling. ACM Trans.
Graph. 36, 4 (2017), 156–1. 2[ZHX∗ 18] Z HOU Y., H U L., X ING J., C HEN W., K UNG H.-W., T ONG
X., L I H.: Hairnet: Single-view hair reconstruction using convolutional
neural networks. In Proceedings of the European Conference on Computer Vision (ECCV) (2018), pp. 235–251. 2[ZWW∗ 18] Z HANG M., W U P., W U H., W ENG Y., Z HENG Y., Z HOU
K.: Modeling hair from an rgb-d camera. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–10. 2[ZYZ∗ 22] Z HENG Y., YANG H., Z HANG T., BAO J., C HEN D., H UANG
Y., Y UAN L., C HEN D., Z ENG M., W EN F.: General facial representation learning in a visual-linguistic manner. In Proceedings ofthe IEEE/CVF Conference on Computer Vision and Pattern Recognition(2022), pp. 18697–18709. 7