)RYH$5&RPELQLQJDQ2SWLFDOO\6HH7KURXJK1HDU(\H'LVSOD\ZLWK6SDWLDO$XJPHQWHG5HDOLW\3URMHFWLRQV+UYRMH%HQNR(\DO2IHN)HQJ=KHQJ$QGUHZ':LOVRQ
'HSDUWPHQWRI&RPSXWHU6FLHQFH
 0LFURVRIW5HVHDUFK5HGPRQG:$86$
8QLYHUVLW\RI1RUWK&DUROLQD&KDSHO+LOO1&86$
^EHQNRH\DORIHNDZLOVRQ`#PLFURVRIWFRP
]KHQJI#FVXQFHGX

)LJXUH:KHQXVLQJRSWLFDOO\VHHWKURXJKGLVSOD\JODVVHV $ WKHYLHZHULVSUHVHQWHGZLWKDOLPLWHGILHOGRIYLHZLPSDFWLQJWKHLU
VHQVH RI LPPHUVLRQ DQG SUHVHQFH %  )RYH$5 H[WHQGV WKH JODVVHV H[SHULHQFH &  E\ DGGLQJ D YLHZGHSHQGHQW SURMHFWLRQ LQ WKHHQYLURQPHQW ' 1RWH$OOILUVWSHUVRQLPDJHVLQWKLVSDSHUZHUHSKRWRJUDSKHGWKURXJKWKHJODVVHV'XHWRWKHUHODWLYHEULJKWQHVVRIWKHJODVVHVGLVSOD\DQGOLPLWHGG\QDPLFUDQJHRIWKHFDPHUDSURMHFWHGFRQWHQWDSSHDUVVXEMHFWLYHO\GLPPHUWKDQLQUHDOLW\$%675$&7

$&0&ODVVLILFDWLRQ.H\ZRUGV

2SWLFDOO\VHHWKURXJK 267 DXJPHQWHGUHDOLW\JODVVHVFDQ
RYHUOD\ VSDWLDOO\UHJLVWHUHG FRPSXWHUJHQHUDWHG FRQWHQW
RQWR WKH UHDO ZRUOG +RZHYHU FXUUHQW RSWLFDO GHVLJQV DQGZHLJKW FRQVLGHUDWLRQV OLPLW WKHLU GLDJRQDO ILHOG RI YLHZ WR
OHVVWKDQGHJUHHVPDNLQJLWGLIILFXOWWRFUHDWHDVHQVHRILPPHUVLRQRUJLYHWKHYLHZHUDQRYHUYLHZRIWKHDXJPHQWHGUHDOLW\ VSDFH :H FRPELQH 267 JODVVHV ZLWK D SURMHFWLRQEDVHG VSDWLDO DXJPHQWHG UHDOLW\ GLVSOD\ WR DFKLHYH D QRYHOGLVSOD\K\EULGFDOOHGFoveARFDSDEOHRIJUHDWHUWKDQ
GHJUHHV ILHOG RI YLHZ YLHZ GHSHQGHQW JUDSKLFV H[WHQGHGEULJKWQHVVDQGFRORUDVZHOODVLQWHUHVWLQJFRPELQDWLRQVRISXEOLFDQGSHUVRQDOGDWDGLVSOD\:HFRQWULEXWHGHWDLOVRIRXU
SURWRW\SHLPSOHPHQWDWLRQDQGDQDQDO\VLVRIWKHLQWHUDFWLYH
GHVLJQVSDFHWKDWRXUV\VWHPHQDEOHV:HDOVRFRQWULEXWHIRXUSURWRW\SH H[SHULHQFHV VKRZFDVLQJ WKH FDSDELOLWLHV RI
)RYH$5 DV ZHOO DV SUHOLPLQDU\ XVHU IHHGEDFN SURYLGLQJLQVLJKWVIRUHQKDQFLQJIXWXUH)RYH$5H[SHULHQFHV+>,QIRUPDWLRQ,QWHUIDFHVDQG3UHVHQWDWLRQ@
0XOWLPHGLD,QIRUPDWLRQ6\VWHPV²>$UWLILFLDODXJPHQWHGDQGYLUWXDOUHDOLWLHV@$XWKRU.H\ZRUGV$XJPHQWHGUHDOLW\SURMHFWRUFDPHUDV\VWHPKHDGPRXQWHGGLVSOD\VVHHWKURXJKGLVSOD\V
3HUPLVVLRQ WR PDNH GLJLWDO RU KDUG FRSLHV RI DOO RU SDUW RI WKLV ZRUN IRU
SHUVRQDORUFODVVURRPXVHLVJUDQWHGZLWKRXWIHHSURYLGHGWKDWFRSLHVDUH
QRWPDGHRUGLVWULEXWHGIRUSURILWRUFRPPHUFLDODGYDQWDJHDQGWKDWFRSLHVEHDU WKLV QRWLFH DQG WKH IXOO FLWDWLRQ RQ WKH ILUVW SDJH &RS\ULJKWV IRU
FRPSRQHQWV RI WKLV ZRUN RZQHG E\ RWKHUV WKDQ WKH DXWKRU V  PXVW EH
KRQRUHG $EVWUDFWLQJ ZLWK FUHGLW LV SHUPLWWHG 7R FRS\ RWKHUZLVH RU
UHSXEOLVKWRSRVWRQVHUYHUVRUWRUHGLVWULEXWHWROLVWVUHTXLUHVSULRUVSHFLILF
SHUPLVVLRQDQGRUDIHH5HTXHVWSHUPLVVLRQVIURP3HUPLVVLRQV#DFPRUJ
UIST '15,1RYHPEHU±&KDUORWWH1&86$
&RS\ULJKWLVKHOGE\WKHRZQHUDXWKRU V 3XEOLFDWLRQULJKWVOLFHQVHGWR$&0$&0«
'2,KWWSG[GRLRUJ


,1752'8&7,21

$XJPHQWHG UHDOLW\ $5  RIIHUV D FDSWLYDWLQJ YLVLRQ RI WKHIXWXUH LQ ZKLFK FRPSXWHUJHQHUDWHG &*  FRQWHQW LV
SUHVHQWHG GLUHFWO\ RYHU WKH UHDO ZRUOG DQG SUHFLVHO\UHJLVWHUHGLQVXFKDZD\WKDWLWDSSHDUVSDUWRIWKHUHDOZRUOG
$5VFHQDULRVLQSRSXODUPHGLDDQGILOPW\SLFDOO\IHDWXUH&*
FRQWHQWWKDWILOOVDODUJHSRUWLRQRIWKHYLHZHU¶VILHOGRIYLHZ)29  DQG DSSHDUV DOPRVW LQGLVWLQJXLVKDEOH IURP WKH UHDOVFHQH+RZHYHUFXUUHQW$5WHFKQRORJLHVDUHIDUIURPIXOO\UHDOL]LQJWKLVYLVLRQ)RUHUXQQHUFRPPHUFLDO$5VROXWLRQVUHO\RQRSWLFDOO\VHH
WKURXJK 267  QHDUH\H GLVSOD\V LH ³VPDUW JODVVHV´ &XUUHQWRSWLFDOGHVLJQVDQGZHLJKWFRQVLGHUDWLRQVOLPLWWKH
)29 RI 267 JODVVHV WR  GLDJRQDO ZKLFK SDOHV LQ
FRPSDULVRQ WR WKH RYHUDOO ELQRFXODU KXPDQ YLVLRQ )29 RI
FORVHWRKRUL]RQWDO aYHUWLFDO ,QSDUWLFXODUWKLVLV
GXH WR WKH QDWXUH RI WKH RSWLFV XVHG HJ D EHDP VSOLWWHU
HPEHGGHGLQDZDYHJXLGH ZKLFKUHTXLUHDJUHDWO\LQFUHDVHGWKLFNQHVV WR REWDLQ D ZLGHU )29 >@ 2QO\ D IHZ UHFHQWUHVHDUFKSURWRW\SHV>@RIIHUDJOLPSVHRISRWHQWLDOO\ZLGHU )29 DOEHLW DW D FRVWO\ WUDGHRII RI UHGXFHG LPDJHTXDOLW\DQGGLVSOD\UHVROXWLRQ,QWKLVSDSHUZHSURSRVHDQRYHODSSURDFKWRH[WHQGLQJWKH
)29 RI WKH $5 QHDUH\H GLVSOD\ E\ FUHDWLQJ D K\EULGDXJPHQWHG UHDOLW\ GLVSOD\ V\VWHP FDOOHG FoveAR ZKLFKFRPELQHV DQ 267 QHDUH\H GLVSOD\ ZLWK SURMHFWLRQEDVHGVSDWLDO $5 GLVSOD\ )LJXUH   7KLV FRPELQDWLRQ LV
FRPSDWLEOH ZLWK WKH H[LVWLQJ VHW RI 267 JODVVHV ZKLOHHQDEOLQJDZLGHU)29DQGRIIHULQJDQXPEHURIRWKHUXQLTXH
EHQHILWVWRWKHYLHZHU HJSULYDWHVWHUHRVFRSLFYLHZVSHUSL[HOFRQWUROOHGDPELHQWOLJKWLQJVXUIDFHVKDGLQJHIIHFWVQRW
DIIHFWHG E\ KHDG WUDFNLQJ ODJ RU PLVPDWFK HWF  ,Q
SDUWLFXODUZHGHPRQVWUDWHWKHFDSDELOLWLHVRIRXUV\VWHPLQ
IRXUSURWRW\SHH[SHULHQFHVLQFOXGLQJ'PRGHODQLPDWLRQVZLGH DQJOH LPPHUVLYH VLPXODWLRQ D ' OLIHVL]H $5
WHOHSUHVHQFHDQGDJDPHSDGFRQWUROOHG$5VKRRWHUJDPH
:KLOHZHDUHQRWWKHILUVWWRFRPELQHSURMHFWRUVDQGQHDUH\H$5 GLVSOD\V WR WKH EHVW RI RXU NQRZOHGJH SUHYLRXVDSSURDFKHV XVH WKH SURMHFWRU RQO\ IRU VSDWLDOO\ FRQWUROOHGEDFNJURXQGOLJKWLQJ>@RUREMHFWKLJKOLJKWLQJ>@DQGQRW
IRUH[WHQGLQJWKH)29RIWKHYLHZHU,QFRQWUDVWRXUZRUNFRQVLGHUV WKH SURMHFWHG SHULSKHU\ WR EH equally capable ofprovidingspatially-registeredview-dependentARexperiences as the near-eye displays ,W LV E\ FRPELQLQJWKHVHWZRWHFKQRORJLHVWKDWZHDFKLHYHPRUHFRPSHOOLQJ$5
H[SHULHQFHVDQGDJUHDWHUVHQVHRISUHVHQFH>@&RQWULEXWLRQV,QVXPPDU\ZHFRQWULEXWHDQRYHODSSURDFKWRH[WHQGLQJWKH
)29RIWKH267QHDUH\HGLVSOD\E\FRPSOHPHQWLQJLWZLWKSURMHFWLRQEDVHG VSDWLDO $5 GLVSOD\ 6SHFLILFDOO\ ZH
FRQWULEXWHx $ SURRIRIFRQFHSW )RYH$5 V\VWHP ZKLFK FRPELQHVDQH\HJODVVHV267$5GLVSOD\ZLWKDFXVWRPSURMHFWRUGHSWKFDPHUDXQLWIRUDZLGH)29$5H[SHULHQFHx ,PSOHPHQWDWLRQRIH[SHULHQFHVWKDWGHPRQVWUDWHWKHLPSRUWDQWFKDUDFWHULVWLFVRIRXUK\EULGGLVSOD\x $VHWRIFRQVLGHUDWLRQVWKDWIUDPHWKHGHVLJQVSDFHIRU
LPSOHPHQWLQJ)RYH$5H[SHULHQFHV
5(/$7(':25.

2XU ZRUN EXLOGV RQ SUHYLRXV UHVHDUFK LQ QHDUH\H 267GLVSOD\V 6SDWLDO $XJPHQWHG 5HDOLW\ DQG UHVHDUFK RQ
SHULSKHUDOSURMHFWHGGLVSOD\V([WHQGLQJWKH)29RI6HH7KURXJK$5'LVSOD\V,WLVGLIILFXOWWRVSHFLI\DPLQLPXP)29UHTXLUHPHQWIRUDGLVSOD\VLQFHWKHLPSDFWRIDFKDQJHLQ)29GHSHQGVRQWKH
WDVN DW KDQG DQG VHHPV WR KDYH D FRQWLQXRXV LPSDFW RQ
SHUIRUPDQFH +RZHYHU UHVHDUFK KDV VKRZQ WKDW ZLGHQLQJ
WKH)29KDVDSRVLWLYHLPSDFWRQLPPHUVLRQDQGSUHVHQFH
HJVXEMHFWVDUHEHWWHUDWHVWLPDWLQJGLVWDQFHVZLWKZLGHU
)29GLVSOD\V>@ .UHVV DQG 6WDUQHU>@ SURYLGH D PRUH WKRURXJK VXUYH\ RI
WKHVWDWHRIWKHDUWLQQHDUH\HGLVSOD\V
7KHUHDUHVHYHUDOUHFHQWDWWHPSWVWRLQFUHDVHWKH)29RIWKH
267 JODVVHV >  @ 6XFK VROXWLRQV XVXDOO\ WUDGH RIIUHVROXWLRQEULJKWQHVVDQGLPDJHTXDOLW\WRDFKLHYHLQFUHDVHG
)29 &KHQJ HW DO >@ OHYHUDJH WKH IUHHIRUP RSWLFV WR
LPSURYH WKH WUDGLWLRQDO EHDP VSOLWWHU GHVLJQ DQG VKRZ D
SURWRW\SH 267 GLVSOD\ ZLWK D  KRUL]RQWDO )29 7KH\
SURSRVHWLOLQJPXOWLSOHGLVSOD\VWRIXUWKHULQFUHDVHWKH)29
5HODWHGWRRXUDSSURDFKLVWKHXVHRIKHDGPRXQWHG>@RUVKRXOGHUPRXQWHG>@SURMHFWRUVLQVWHDGRIQHDUH\HGLVSOD\V
IRU SURMHFWLQJ DXJPHQWDWLRQV RQ WKH UHDO ZRUOG 6XFKGLVSOD\VDUHOLPLWHGE\WKH)29RIWKHSURMHFWRUXVHGDVZHOODVE\WKHVXLWDELOLW\RIWKHDYDLODEOHSURMHFWLRQVXUIDFHV
7KHUHKDYHDOVREHHQDWWHPSWVWRFUHDWHDZLGH)29QHDU
H\H GLVSOD\V XVLQJ OLJKWILHOG GLVSOD\V LQVWHDG RI EHDPVSOLWWHUV >  @ 3UREDEO\ WKH PRVW SURPLVLQJGHYHORSPHQW LV WKH UHFHQW ZRUN RQ 3LQOLJKW 'LVSOD\V >@ZKLFKGHPRQVWUDWHGDGLDJRQDO)29SURWRW\SHEDVHG
RQDQ/&'EDFNHGZLWKDQDUUD\RISRLQWOLJKWVRXUFHVWKDWDFW DV JULG RI PLQLDWXUH VHHWKURXJK SURMHFWRUV :KLOHFDSDEOH RI SRWHQWLDOO\ DUELWUDULO\ ZLGH ILHOG RI YLHZ WKLVDSSURDFKKDVVLJQLILFDQWOLPLWDWLRQVWKHGLVSOD\PXVWUHQGHUDIXOOOLJKWILHOG DVVXPLQJQRH\HWUDFNLQJ DQGLWVXOWLPDWHUHVROXWLRQLVGLIIUDFWLRQOLPLWHG
6SDWLDO$XJPHQWHG5HDOLW\$VDQDOWHUQDWHWRQHDUH\H$5GLVSOD\V6SDWLDO$XJPHQWHG
5HDOLW\ 6$5  DSSURDFKHV PHUJH WKH SK\VLFDO DQG YLUWXDOZRUOGVWKURXJKYLGHRSURMHFWLRQ>@6KDGHU/DPSV>@ILUVWGHYHORSHGWKHLGHDRIFKDQJLQJWKHDSSHDUDQFHRISK\VLFDO
REMHFWV ZLWK YLGHR SURMHFWLRQ 6HYHUDO SURMHFWV VLQFH KDYH
GHPRQVWUDWHG WKH XVH RI 6$5 WR DXJPHQW WKH HQYLURQPHQWHLWKHU WKURXJKPXOWL SURMHFWRUFDPHUD LQVWDOODWLRQV >
@ RU E\ PRXQWLQJ WKH SURMHFWRU DQG WKH FDPHUD RQWR D
PRYDEOHSODWIRUPWRVWHHUWKHSURMHFWLRQ>@

7DEOH&RPPRQO\XVHGFRPPHUFLDO267JODVVHV LQFOXGLQJSDVWDQGIXWXUHDQQRXQFHG DQGWKHLUUHSRUWHG)29YDOXHV0LUDJH7DEOH>@ZDVWKHILUVWWRGHPRQVWUDWHXVLQJWKHUHDOWLPH JHRPHWU\ FDSWXUH IURP GHSWK FDPHUDV IRU YLHZ
GHSHQGHQWSURMHFWLRQVRQDG\QDPLFDOO\FKDQJLQJUHDOZRUOG
VFHQH5HFHQWO\%HQNRHWDO>@KDYHH[WHQGHGWKLVFRQFHSW
WR KDQGOH WZR VLPXOWDQHRXV YLHZGHSHQGHQW SURMHFWLRQ IRUWZRYLHZHUVLQWKHVFHQHLQDIDFHWRIDFHDUUDQJHPHQW2XU
ZRUN EXLOGV RQ WKHVH FRQFHSWV DQG H[WHQGV WKHP E\
LQFRUSRUDWLQJ WKH QHDUH\H 267 GLVSOD\ IRU DGGLWLRQDOYLVXDOL]DWLRQFDSDELOLWLHV

1HDU(\H*ODVVHV)29 1HDU(\H*ODVVHV

)29

3HULSKHUDO3URMHFWHG'LVSOD\V(SVRQ0RYHULR%7/XPXV'.
0HWD3UR
0LFURYLVLRQ1RPDG
2SWLQYHQW25$













([WHQGLQJWKHGLVSOD\ILHOGRIYLHZFDQFDXVHWKHXVHUWRIHHOPRUHSUHVHQWDQGLPPHUVHG>@EXWLVDOVRPRUHOLNHO\WRFDXVHVLPXODWRUVLFNQHVV>@([WHQGLQJWKH)29RIWKH
H[SHULHQFHXVXDOO\UHVXOWVLQKLJKHUGLVSOD\FRVWDQGORZHUGLVSOD\UHVROXWLRQDQGEULJKWQHVV7REDODQFHWKHVHWUDGHRIIV%DXGLVFK HW DO SURSRVH )RFXV&RQWH[W VFUHHQV >@ WKDWFRPELQHDKLJKUHVROXWLRQFRPSXWHUPRQLWRU IRFXV ZLWKDORZHU UHVROXWLRQ SURMHFWHG SHULSKHU\ FRQWH[W  )HLQHU DQG
6KDPDVKVKRZHGWKDWRQHFDQXVH$5JODVVHVWRH[WHQGWKH

2VWHUKRXW'HVLJQ*URXS5
6RQ\*ODVVWURQ/','%9LUWXDO,2,JODVVHV
9X]L[0
9X]L[6WDU;/'

,Q 7DEOH  ZH VXPPDUL]H WKH PRVW ZLGHO\ XVHG
FRPPHUFLDOO\ DYDLODEOH 267 QHDUH\H GLVSOD\V DQG WKHLUUHSRUWHG)291RQHRIIHUD)29KLJKHUWKDQGLDJRQDOZRUNLQJ DUHD RI WKHLU FRPSXWHU PRQLWRU >@ 2OZDO DQG)HLQHU H[WHQG WKHVH FRQFHSWV E\ FRPELQLQJ WUDFNHG
KDQGKHOGVDQGODUJHSURMHFWLRQVFUHHQV>@:HDUHLQVSLUHGE\,OOXPL5RRP>@ZKLFKFRPELQHVDODUJH
WHOHYLVLRQGLVSOD\ZLWKSURMHFWLRQDURXQGLWDQGVKRZHGKRZ
WKHSHULSKHUDOSURMHFWLRQ FDQEHXVHG WR HQDEOH FRPSOHWHO\QRYHO YLVXDO HIIHFWV IRU JDPHV DQG YLGHR FRQWHQW )RYH$5
EXLOGV RQ WKLV ZRUN E\ HQDEOLQJ FRPSOHWHO\ PRYDEOH DQGYLHZGHSHQGHQWYLHZVLQERWKWKHQHDUH\HGLVSOD\DQGWKH
SURMHFWHG SHULSKHU\ ,Q DGGLWLRQ )RYH$5 FDQ DIIHFW WKHLPDJHLQWKHQHDUH\HGLVSOD\E\SURMHFWLQJFRQWHQWGLUHFWO\EHKLQGLW
%LPEHUHWDO¶V2FFOXVLRQ6KDGRZV>@XVHVDSURMHFWRUDVDUHSODFHPHQWIRUDPELHQWOLJKWVLQRUGHUWRSURJUDPPDWLFDOO\GLP WKH EDFNJURXQG LQ WKH DUHD ZKHUH $5 FRQWHQW LV
GLVSOD\HG LQ WKH 267 GLVSOD\ D VWDWLRQDU\ KDOIVLOYHUHG
PLUURU $5 GLVSOD\  ([WHQGLQJ WKDW ZRUN 0DLPRQH HW DO
>@XVHVWKHSURMHFWRUWRFRPSHQVDWHIRUWKHWUDQVSDUHQF\RI
267 JODVVHV E\ SURMHFWLQJ OLJKW DURXQG WKH YLUWXDO REMHFWV
VHHQ LQ WKH JODVVHV ,Q WKHLU ZRUN WKH DXJPHQWHG FRQWHQWDSSHDUVRQO\LQWKHJODVVHVDQGWKHSURMHFWRUGRHVQRWH[WHQG
RU DGG DQ\ FRORU FRQWHQW WR WKH XVHU¶V +0' YLHZ EXW LV
UDWKHUXVHGDVDVPDUWOLJKWVRXUFH
=KRXHWDO>@FRPELQH267$5JODVVHVZLWKSURMHFWRUVIRU
LQGXVWULDO ZHOGLQJ DSSOLFDWLRQV 7ZR GLVSOD\V SHUIRUP WZRLQGHSHQGHQW IXQFWLRQV JODVVHV GLVSOD\ KHDGVXS '
LQIRUPDWLRQ WH[WDQGLPDJHV DQGWKHSURMHFWRUDQQRWDWHVWKH
ZRUOGE\KLJKOLJKWLQJREMHFWVRILQWHUHVW7KHSURMHFWRUGRHVQRW GLVSOD\ DQ\ YLHZGHSHQGHQW LQIRUPDWLRQ DQG WKLVDSSURDFKLVQRWFDSDEOHRIXQLIRUPO\H[WHQGLQJWKH)29RI
WKHJODVVHVDVZHGHPRQVWUDWHZLWK)RYH$5

FDPHUDV 7KHWUDFNLQJV\VWHPUHSRUWVIUDPHVSHUVHFRQGZLWK D UHSRUWHG WUDFNLQJ ODWHQF\ RI aPV ZKLFK HQVXUHVUHODWLYHO\VPRRWKORZODWHQF\RSHUDWLRQ:HUHQGHUDW+]
UHIUHVKUDWHFRUUHVSRQGLQJWRWKHUHIUHVKUDWHRIRXUJODVVHV)LJXUHVKRZV)RYH$5V\VWHPFRPSRQHQWV:H XVH WKH 8QLW\  JDPH HQJLQH IRU UHQGHULQJ ERWK WKHSURMHFWHG &* FRQWHQW DQG DOO WKH FRQWHQW GLVSOD\HG LQ WKHJODVVHV8QLW\DOORZVXVWRXVHVWDWHRIWKHDUWDQLPDWLRQDQG
UHQGHULQJWRROVWRHDVLO\H[SHULPHQWDQGFUHDWHFRPSHOOLQJ
FRQWHQWIRURXUH[SHULHQFHV)LJXUH  )RYH$5 KDUGZDUH $  /XPXV '. GLVSOD\ ZLWKUHWURUHIOHFWLYHPDUNHUV% SURMHFWRUDQG.LQHFWPRXQWHGRQ
WKHFHLOLQJDQG& 2SWL7UDFN)OH[PRWLRQWUDFNLQJV\VWHP

)RYH$5LVFXUUHQWO\UHVWULFWHGWRDVLQJOHVLGHRIRXUODEEXWPHWKRGVWRH[WHQGWKHSURMHFWLRQPDSSLQJWRWKHHQWLUHURRP
KDYHEHHQSUHYLRXVO\GHPRQVWUDWHG HJ>@,IWKHXVHUVWDQGVIWDZD\IURPRXUSURMHFWLRQZDOOWKHLUYLVLEOH)29
IRUWKHSURMHFWHGFRQWHQWFRYHUVPRUHWKDQ+RZHYHUWKHXVHULVDOVRIUHHWRPRYHDURXQGWKHURRPDQGFDQWKXVDFKLHYHERWKZLGHUDQGQDUURZHU)29$OORXUH[DPSOHVDUH
FXUUHQWO\ GHVLJQHG IRU D VLQJOH XVHU KRZHYHU ZH GLVFXVVPXOWLXVHUSRVVLELOLWLHVLQWKH/LPLWDWLRQVVHFWLRQEHORZ
3URWRW\SH)RYH$5([SHULHQFHV:HLPSOHPHQWHGIRXU)RYH$5H[SHULHQFHV:KLOHERWK0DLPRQHHWDO>@DQG=KRXHWDO>@FRPELQHJODVVHVDQGSURMHFWRUVWKH\DGGUHVVGLIIHUHQWSUREOHPVWKDQ)RYH$5 1RQH HQDEOHV ZLGH )29 YLHZGHSHQGHQW $5
H[SHULHQFHVQRUGRWKH\H[SORUHVFHQDULRVWKDWH[SORLWVXFKRYHUODSSLQJ GLVSOD\V HJ )RYH$5 GHPRQVWUDWHV KRZVXUIDFHERXQG FRQWHQW FDQ EH GHOHJDWHG WR WKH SURMHFWRU
LPSURYLQJLWVVWDELOLW\ QRKHDGWUDFNLQJQRLVH DQGHQDEOLQJPXOWLXVHU YLHZLQGHSHQGHQW UHQGHULQJ

3D Model Animation:,QWKLVH[SHULHQFHWKHXVHUFDQLQVSHFWDYDULHW\RI'PRGHOVSODFHGDQGDQLPDWHGDURXQGWKHURRP)RU H[DPSOH )LJXUH D VKRZV D ' YLUWXDO KHOLFRSWHUKRYHULQJ DURXQG WKH URRP 7KHUH LV D VSRWOLJKW DWWDFKHG WR
WKHKHOLFRSWHUWKDWLOOXPLQDWHVYDULRXVSDUWVRIRXUUHDOURRPZKLFKLVDVXUIDFHVKDGHGSURMHFWHGHIIHFW:HGHVLJQHGWKLVH[SHULHQFH WR KLJKOLJKW WKH GLIILFXOW\ RI NHHSLQJ D PRYLQJ
REMHFWLQWKHQDUURZ)29RIWKHJODVVHV)29($56<67(029(59,(:Wide-Angle Immersive Simulation: 2XU VHFRQG H[SHULHQFHVKRZVDQ$55XEH*ROGEHUJPDFKLQHZKHUHGLIIHUHQWYLUWXDOEDOOVWUDYHODORQJDYDULHW\RIYLUWXDOVKHOYHVDQGREVWDFOHVDVZHOO DV WKH UHDO IXUQLWXUH LQ WKH URRP )LJXUH   7KLVLPPHUVLYHH[SHULHQFHDOVRIHDWXUHVGLIIHUHQWVXUIDFHVKDGHG
WH[WXUHVDSSOLHGWRWKHUHDOURRPWRFKDQJHLWVDSSHDUDQFHDQG
LWLOOXVWUDWHVWKHIXOOYLVXDOL]DWLRQFDSDELOLWLHVRI)RYH$5

)RYH$5FRPELQHVRIDQ267QHDUH\HGLVSOD\ /XPXV'.
[ ZLWKDFHLOLQJPRXQWHGZLGHDQJOHSURMHFWRU
2SWRPD*7'/3[ SDLUHGZLWKD0LFURVRIW
.LQHFWYGHSWKFDPHUD%RWKWKHSURMHFWRUDQGWKHJODVVHVDUHFRQQHFWHGWRWKHVDPH:LQGRZVPDFKLQH 'XDO;HRQ*+] *% 5$0 1YLGLD *7;   UHVSRQVLEOH IRUSURFHVVLQJDQGUHQGHULQJDOOWKH&*FRQWHQW
2XU267JODVVHVDUHHTXLSSHGZLWKUHWURUHIOHFWLYHVSKHUHVWKDW DUH WUDFNHG XVLQJ D FHLOLQJ PRXQWHG PRWLRQ FDSWXUH
V\VWHP 1DWXUDO 3RLQW¶V 2SWL7UDFN )OH[  V\VWHP 

3D Life-Size Telepresence: ,Q WKLV H[SHULHQFH )RYH$5
VKRZVDOLIHVL]HGYLUWXDO'FDSWXUHRIDUHPRWHFROODERUDWRU
DQGHQDEOHVWKHYLHZHUWRFDUU\RXWDIDFHWRIDFHWHOHSUHVHQFH









KWWSZZZOXPXVRSWLFDOFRP
KWWSZZZRSWRPDXVDFRP
KWWSZZZPLFURVRIWFRPHQXVNLQHFWIRUZLQGRZVKWWSZZZRSWLWUDFNFRPSURGXFWVIOH[
KWWSZZZXQLW\GFRPFRQYHUVDWLRQ )LJXUH FG  :LWKRXW )RYH$5 LW LV
LPSRVVLEOH WR NHHS WKH HQWLUH UHPRWH SHUVRQ YLVLEOH LQ WKHJODVVHVDQGPDLQWDLQDFRPIRUWDEOHFRQYHUVDWLRQDOGLVWDQFH
)LJXUH G  )RYH$5¶V SURMHFWLRQ ILOOV LQ DOO WKH UHOHYDQWSHULSKHUDOGHWDLOVZKLFKDOORZVWKHYLHZHUWRVKLIWWKHLUIRFXVZLWKRXWORVLQJWKHRYHUYLHZRIWKHLUSDUWQHURSHUDWLQJ RQ WKH UHDOZRUOG JHRPHWU\ RU RQ WKH UHDOWLPHGHSWKJHRPHWU\FDSWXUHGE\WKH.LQHFWFDPHUD
2QHVLJQLILFDQWGLIIHUHQFHIURPSUHYLRXVSURMHFWLRQPDSSLQJDSSURDFKHVLVWKDWLQWKHILUVWSDVVZHUHQGHUWKHYLHZIURPWKH SHUVSHFWLYH RI WKH XVHU WZLFH RQFH IRU WKH ZLGH )29
peripheryDQGRQFHIRUWKHinsetDUHDZKLFKFRUUHVSRQGVWR
WKH)29RIWKHQHDUH\HJODVVHV,QWKHSURMHFWLRQPDSSLQJSDVVZHFRPELQHERWKWKRVHRIIVFUHHQWH[WXUHVLQWRDILQDOFRPSRVLWHGLPDJH
7KLVUHQGHULQJSLSHOLQHUHTXLUHVXVWRUHQGHUWKHVFHQHILYH
WLPHV IRU HDFK IUDPH WZLFH IRU WKH JODVVHV RQFH IRU HDFK
H\H RQFHIRUWKHSURMHFWHGSHULSKHU\ RIIVFUHHQ RQFHIRU
WKH SURMHFWHG LQVHW RIIVFUHHQ  DQG RQFH IRU WKH SURMHFWLRQPDSSLQJ DQG FRPSRVLWLQJ SURFHVV IRU WKH SURMHFWRU YLHZ
7KLVPXOWLSDVVSURFHVVHQDEOHVXVWRKDYHFRPSOHWHFRQWURORYHUZKDWFRQWHQWZLOOEHSUHVHQWHGLQZKLFKYLHZ)LJXUH  7KUHH )RYH$5 H[SHULHQFHV $  ' KHOLFRSWHUDQLPDWLRQ% $5VKRRWHUJDPH&' 'OLIHVL]HWHOHSUHVHQFH

:H H[SHULPHQWHG ZLWK IRXU GLIIHUHQW FRPELQDWLRQV RI
FRQWHQW SODFHPHQW EHWZHHQ WKH JODVVHV DQG WKH SURMHFWHG
LQVHW )LJXUH   7KH VLPSOHVW FRPELQDWLRQ LV WR KDYHFRPSOHWHO\replicatedFRQWHQWLHWKHVDPHFRQWHQWLQERWKWKH JODVVHV DQG WKH SURMHFWRU 7KH SURMHFWHG LQVHW FDQ DOVRVKRZ DQ occlusion shadow IRU WKH JODVVHV FRQWHQW RU RQO\VKRZWKHsurface shadedFRQWHQWWKDWLVQRWYLHZGHSHQGHQW)LQDOO\WRUHGXFHYLVXDOGLVFRQWLQXLWLHVZHDSSO\DVPRRWKWUDQVLWLRQ EHWZHHQ WKH SHULSKHU\ DQG WKH LQVHW surface
shaded + blended 

AR Shooter Game: 7KLV WKLUGSHUVRQ VKRRWHU JDPH DOORZVWKHSOD\HUWRFRQWURODYLUWXDOFKDUDFWHUYLDD0LFURVRIW;ER[ JDPH FRQWUROOHU )LJXUH E  7KH FKDUDFWHU FDQ UXQDURXQGWKHVXUIDFHVRIWKHURRP LQFOXGLQJWKHZDOOVDQGWKHIXUQLWXUH  ILJKWLQJ WKH DWWDFNLQJ YLUWXDO VRFN SXSSHWV 7KHSOD\HUEHQHILWVIURPKDYLQJDZLGHILHOGRIYLHZWRNHHSDQ
H\H RQ WKH LQFRPLQJ DWWDFNHUV DQG E\ VHHLQJ DOO JDPH
FKDUDFWHUVDV'REMHFWVLQWKHLUVWHUHRJODVVHV'\QDPLF9LHZ'HSHQGHQW3URMHFWLRQV*LYHQWKHSUHFLVHYLHZSRLQWDQGRULHQWDWLRQRIWKHJODVVHVDVZHOO DV WKH JHRPHWU\ RI WKH URRP LW LV VWUDLJKWIRUZDUG WR
UHQGHUJUDSKLFVLQWKHJODVVHVVRWKDWWKH\DSSHDUFRUUHFWIRU
WKDWYLHZHU¶VSHUVSHFWLYH,WLVDOVRUHODWLYHO\VLPSOHWRUHQGHUJUDSKLFV LQ WKH SURMHFWRU RU LQ WKH JODVVHV WKDW FKDQJH WKHVXUIDFH DSSHDUDQFH RI WKH REMHFWV LQ WKH URRP IROORZLQJ D
VXUIDFHVKDGLQJPRGHO>@
8VLQJ D SURMHFWRU WR UHQGHU D YLUWXDO ' REMHFW VR WKDW LW
DSSHDUVFRUUHFWJLYHQDQDUELWUDU\XVHU¶VYLHZSRLQWLVPRUHFRPSOH[ :H LPSOHPHQWHG LW DV D PXOWLSDVV UHQGHULQJ
SURFHVVVLPLODUWR>@,QWKHILUVWSDVVWKHYLUWXDOREMHFWVDQG
WKH UHDO SK\VLFDO JHRPHWU\ DUH UHQGHUHG LQ DQ RIIVFUHHQEXIIHU 7KDW UHQGHULQJ LV WKHQ FRPELQHG ZLWK WKH VXUIDFH
JHRPHWU\IURPWKHSHUVSHFWLYHRIWKHSURMHFWRUIROORZLQJDVWDQGDUG SURMHFWLYH WH[WXULQJ SURFHGXUH ZKHUH RQO\ WKHSK\VLFDO JHRPHWU\ LV UHQGHUHG :H LPSOHPHQWHG WKLVSURMHFWLRQ PDSSLQJ SURFHVV DV D VHW RI FXVWRP VKDGHUV)LJXUH)RXUFRQWHQWFRPELQDWLRQVGHPRQVWUDWHGLQ)RYH$5
&DOLEUDWLRQ

7KHUH DUH IRXU FDOLEUDWLRQ VWHSV QHHGHG WR HQVXUH WKDW DOOFRPSRQHQWV DUH RSHUDWLQJ WRJHWKHU )LUVW ZH FDOLEUDWH WKHSURMHFWRU ZLWK UHVSHFW WR WKH .LQHFW FDPHUD E\ SURMHFWLQJ*UD\ FRGH VHTXHQFHV WR HVWDEOLVK GHQVH FRUUHVSRQGHQFHVEHWZHHQ WKH .LQHFW¶V FRORU FDPHUD DQG WKH SURMHFWRU 7KLVFDOLEUDWLRQ SURFHGXUH LV GHVFULEHG LQ GHWDLO E\ -RQHV HW DO>@DQGLVSXEOLFO\DYDLODEOHDVWKH5RRP$OLYH7RRONLW$V
SDUWRI WKLV FDOLEUDWLRQ ZH DOVR FDSWXUH WKH URRP JHRPHWU\
DQG DSSHDUDQFH ZKLFK LV WKHQ XVHG IRU YLHZGHSHQGHQW
SURMHFWLRQPDSSLQJ$OWHUQDWLYHO\ZHFDQXVHWKHOLYHGHSWKFDPHUD IHHG WR GULYH SURMHFWLRQV RYHU D FKDQJLQJ URRP
JHRPHWU\VLPLODUWR>@
6HFRQG ZH FDOLEUDWH WKH 2SWL7UDFN WUDFNLQJ V\VWHP ZLWKUHVSHFWWRWKH.LQHFW¶VLQIUDUHGFDPHUDE\LPDJLQJWKHVDPHNQRZQFDOLEUDWLRQSDWWHUQFRQVLVWLQJRIDULJKWDQJOHEUDFNHWZLWKWKUHHULJLGO\PRXQWHGUHWURUHIOHFWLYHPDUNHUV7KLUG ZH PHDVXUH WKH RIIVHWV EHWZHHQ WKH UHWURUHIOHFWLYHWUDFNLQJPDUNHUVDQGWKHJODVVHVGLVSOD\VWRILQGWKHSUHFLVH
ORFDWLRQ RI WKH WZR GLVSOD\V 2XU JODVVHV WUDFNHU PRXQW LV
FXVWRP'SULQWHGDQGWLJKWO\ILWWHGWRWKHJODVVHVWRHQVXUHWKDWWKHFDOLEUDWLRQDFFXUDF\LVPDLQWDLQHG)RXUWKZHXVHDSXSLORPHWHUWRPHDVXUHWKHLQWHUSXSLOODU\GLVWDQFHIRUHDFKXVHURIRXUV\VWHPWRHQVXUHWKDWWKH\FDQ
FRUUHFWO\IXVHWKHVWHUHRLPDJHDQGWKDWWKHLUYLHZVFRUUHFWO\DOLJQZLWKERWKWKHSURMHFWLRQVDQGWKHUHDOZRUOG
'(6,*163$&(&216,'(5$7,216

-RQHV HW DO¶V ,OOXPL5RRP >@ VKRZHG D YDULHW\ RI
peripheral illusionsSRVVLEOHIRUDVWDWLFQRQWUDQVSDUHQWLQVHW
79 DQGDSURMHFWHGSHULSKHU\$OOVXFKLOOXVLRQVDUHDOVRSRVVLEOH LQ )RYH$5 KRZHYHU WKH UDQJH RI FRQWHQWFRPELQDWLRQV LQ RXU V\VWHP LV HYHQ JUHDWHU VLQFH WKHH[SHULHQFHGHVLJQHUKDVDQXQOLPLWHGFKRLFHRIZKDWFRQWHQW
WR SODFH LQ WKH JODVVHV YV WKH SURMHFWRU SHULSKHU\ YV WKHSURMHFWHG LQVHW ,Q DGGLWLRQ WKH )RYH$5 GHVLJQHU KDV DQ
DGGLWLRQDO FKRLFH UHJDUGLQJ KRZ WKRVH YLHZV LQWHUDFW ZLWKDQGRYHUOD\HDFKRWKHU7RKHOSIUDPHWKHGHVLJQVSDFHZH
HODERUDWHRQWKUHHZD\VRQHFDQUHDVRQDERXWWKH)RYH$5
FRQWHQWDQGH[SHULHQFHV
3URMHFWLRQ$LGV*ODVVHV,Q DGGLWLRQ WR WKH FRUH LGHD RI H[WHQGLQJ WKH )29 WKHSURMHFWLRQFDQEHWKRXJKWRIDVDQassistivemodalityWRWKH
JODVVHV0DQ\IRUPVRIDVVLVWDQFHDUHSRVVLEOH)RUH[DPSOH
WKH SURMHFWRU FDQ DGG EULJKWQHVV WR WKH VFHQH KLJKOLJKW D
VSHFLILF REMHFW RU DFW DV G\QDPLF OLJKW VRXUFH WR SURYLGH
2FFOXVLRQ6KDGRZV>@IRUWKHJODVVHVFRQWHQW)XUWKHUPRUH
ZKHQSURMHFWLQJHIIHFWVWKDWDUHERXQGWRUHDOZRUOGVXUIDFHV
DQG WKHUHIRUH QRQYLHZ GHSHQGHQW LH VXUIDFHVKDGHG
HIIHFWV RQHFDQDYRLGDQ\WUDFNLQJODJRUMLWWHUWKDWLVRIWHQ
SUHVHQWLQWKHJODVVHVYLHZE\LQFOXGLQJWKHVHUHQGHULQJVLQ
WKHSURMHFWLRQRQO\ DQGQRWLQWKHJODVVHV ,QIDFWVLQFHERWKWKH SURMHFWRU DQG WKH URRP DUH LQ VWDWLF DUUDQJHPHQW VXFKYLUWXDODXJPHQWDWLRQVDSSHDUYHU\VWDEOHDQGSHUVLVWHQW7KLV
FDQ EH DOVR HIIHFWLYH IRU UHQGHULQJ YLUWXDO VKDGRZV RI '
REMHFWV HJ VHH WKH KHOLFRSWHU VKDGRZ LQ WKH ' 0RGHO$QLPDWLRQH[SHULHQFH±)LJXUHD 


KWWSJLWKXEFRP.LQHFW5RRP$OLYH7RRONLW*ODVVHV$LG3URMHFWLRQ$OWHUQDWLYHO\ JODVVHV FDQ EH FRQVLGHUHG DQ DVVLVWLYHPRGDOLW\WRWKHSURMHFWRU*ODVVHVFDQSURYLGHVWHUHRYLHZVRI YLUWXDO REMHFWV PDNLQJ WKHP DSSHDU VSDWLDOO\ WKUHH
GLPHQVLRQDOUDWKHUWKDQDVGHFDOVSURMHFWHGRQWKHZDOO7KH\
FDQDGGPRUHUHVROXWLRQDQGEULJKWQHVVWRWKHDUHDRIIRFXV*ODVVHV FDQ DOVR YLVXDOL]H REMHFWV ZKHQ RXW RI )29 RI
SURMHFWRU RU LQ WKH SURMHFWRU VKDGRZ LH ZKHQ SURMHFWLRQYLVLELOLW\LVFRPSURPLVHG *ODVVHVDQG3URMHFWLRQ3URYLGH'LIIHUHQW&RQWHQW

,QVRPHVFHQDULRVLWLVEHVWWRFRQVLGHUWKHJODVVHVDQGWKH
SURMHFWRU DV EHLQJ DEOH WR SURYLGH GLIIHUHQW EXWFRPSOHPHQWDU\ FRQWHQW 7KH PRVW REYLRXV H[DPSOH LV WR
HQDEOHWKHSULYDWHFRQWHQW HJSOD\LQJFDUGV WREHVKRZQRQO\LQWKHJODVVHVZKLOHWKHSXEOLFFRQWHQWLVYLVLEOHLQWKH
SURMHFWLRQ )LJXUH 6LPLODUGLVWLQFWLRQFRXOGEHPDGHZLWKRWKHU VHPDQWLF UXOHV )RU H[DPSOH RQH FRXOG FKRVH WR
GLVSOD\ODUJHGLVWDQWREMHFWVDVSURMHFWHGDQGQHDUE\REMHFWVLQ WKH JODVVHV RU RQO\ WKH QRQYLHZ GHSHQGHQW VXUIDFHVKDGHGREMHFWVLQWKHSURMHFWLRQ7KHJODVVHVFRXOGDOVREH
WUHDWHG DV D ³PDJLF´ OHQV LQWR D SURMHFWHG VSDFH RIIHULQJDGGLWLRQDOLQIRUPDWLRQ)LJXUH$QH[DPSOH)RYH$5VFHQDULRVKRZLQJSULYDWHFRQWHQWYLHZHU¶V FDUGV  LQ WKH JODVVHV DQG WKH SXEOLF FRQWHQW RWKHUFDUGV LQWKHSURMHFWLRQ7KHJDPHLVSOD\HGRQUHDOIXUQLWXUH

7KHGHVLJQHUFRXOGFKRRVHWRH[SORLWWKHILQGLQJVRI%HQNR
HW DO >@ WKDW WKH YLHZHU LV EHWWHU DEOH WR FRPSUHKHQG WKHVSDWLDO QDWXUH RI D SHUVSHFWLYHO\ SURMHFWHG YLUWXDO REMHFW LI
WKDW REMHFW LV SODFHG FORVH WR WKH SURMHFWLRQ VXUIDFH ,Q
)RYH$5RQHFDQGLVSOD\WKHREMHFWVFORVHWRWKHUHDOVXUIDFHVRQO\ DV SURMHFWHG WR DFKLHYH UHGXFWLRQ RI WUDFNLQJ ODJ DQGQRLVHDQGPRYHWKHPWRWKHJODVVHVRQFHWKH\DUHLQPLGDLUDZD\IURPWKHVXUIDFH*ODVVHVFDQDOVRHQDEOHDQLQWHUHVWLQJPXOWLXVHUH[SHULHQFHZKLFK LV WUDGLWLRQDOO\ GLIILFXOW WR DFKLHYH ZLWK RQO\ YLHZ
GHSHQGHQWSURMHFWLRQV>@)RUH[DPSOHHDFKXVHU¶VJODVVHVFRXOG SURYLGH WKH SHUVRQDOL]HG SHUVSHFWLYH YLHZV IRU WKHYLUWXDO'REMHFWVWKDWUHTXLUHWKHPZKLOHWKHSURMHFWRUFDQ
EHWDVNHGZLWKSURMHFWLQJWKHQRQYLHZGHSHQGHQWVXUURXQG
WRFRQQHFWWKHWZRH[SHULHQFHV)LQDOO\LISUHFLVHSL[HODOLJQPHQWLVDFKLHYHG)RYH$5FRXOG
EHXVHGWRGLVSOD\FRPSOLPHQWDU\FRQWHQWLQERWKWKHJODVVHVDQGSURMHFWLRQLQVHWWRIDFLOLWDWHKLJKG\QDPLFUDQJHYLUWXDOLPDJHV&XUUHQWO\WKHFDOLEUDWLRQDQGWUDFNLQJLQRXUV\VWHP
GRHVQRWDFKLHYHWKHDGHTXDWHOHYHORIDFFXUDF\WRYHULI\WKDWVHFWLRQ:HIRFXVRXUGLVFXVVLRQKHUHRQWKHXVHUIHHGEDFNUHJDUGLQJWKHSRWHQWLDORI)RYH$5

/,0,7$7,216

$OOXVHUVZHUHVXUSULVHGE\WKHQDUURZ)29RIRXUJODVVHVHVSHFLDOO\WKRVHIDPLOLDUZLWKZLGHU)2995JRJJOHV $
IHZFRPPHQWHGWKDW267JODVVHVDORQHZHUHLQDGHTXDWHWR
IXOO\DSSUHFLDWHRXUH[SHULHQFHV:KLOHZHKDYHVSHFLILFDOO\GHVLJQHG RXU H[SHULHQFHV WR VKRZFDVH WKH EHQHILWV RI
)RYH$5 ZH EHOLHYH WKDW WKH\ DUH UHSUHVHQWDWLYH RI PDQ\
GHVLUDEOH$5VFHQDULRV7KUHHXVHUVPHQWLRQHGWKDWPRWLRQVLQWKHLUSHULSKHU\DWWUDFWHGWKHLUDWWHQWLRQDQGDOOFRPPHQWHGWKDW H[WHQGHG )29 ZDV EHQHILFLDO 7KLV ZDV SDUWLFXODUO\REYLRXVZKHQORRNLQJDWWKHKHOLFRSWHU )LJXUHD ZKLFKLVGLIILFXOWWRNHHSLQVLJKWGXHWRFRQVWDQWPRWLRQ2QHXVHU D
95JDPHU QRWHGWKDWEHFDXVHWKHSHULSKHU\ZDVYHU\VWDEOH
DQG YLUWXDOO\ QRLVH IUHH WKH\ GLG QRW H[SHULHQFH VLPXODWRUVLFNQHVVHYHQZLWKVRPHWUDFNLQJQRLVHDQGODJSUHVHQWLQWKH
JODVVHV$QRWKHUXVHUFRPPHQWHGWKDWWKH\GLGQRWKDYHDQ\SUREOHPVIXVLQJVWHUHRLPDJHVLQWKHJODVVHVHYHQZKHQWKH
REMHFWZDVSDUWLDOO\VKRZQLQWKHJODVVHVDQGSDUWLDOO\LQWKH
SURMHFWRU DOEHLW WKH\ QRWHG WKDW WKH FRORU DQG EULJKWQHVVGLIIHUHQFHVZHUHPRUHGLVUXSWLYH

2XUFXUUHQWLPSOHPHQWDWLRQVXIIHUVIURPQRWLFHDEOHWUDFNLQJODJWUDFNLQJQRLVHDQGFRQWHQWPLVDOLJQPHQW7KHVHLVVXHVFRXOGEHDGGUHVVHGZLWKEHWWHUWUDFNLQJDQGFDOLEUDWLRQ7KH
OLWHUDWXUH RIIHUV PDQ\ DGGLWLRQDO FDOLEUDWLRQ WHFKQLTXHV IRU267QHDUH\HGLVSOD\VWKDWLPSURYHWKHFDOLEUDWLRQEHWZHHQ
WKHYLHZHU¶VH\HDQGWKHGLVSOD\VLQWKHJODVVHV HJ>@ 
6XFK WHFKQLTXHV ZKLOH FDSDEOH RI DFKLHYLQJ VXESL[HO
DOLJQPHQWVWHQGWREHWHGLRXVDQGQHHGWREHUHSHDWHGHDFKWLPHWKHXVHUSXWVRQWKHJODVVHVVLQFHZHFDQQRWHQVXUHDULJLG KHDG PRXQW LQ RXU VHWXS ,WRK DQG .OLQNHU >@ KDYH
UHFHQWO\GHPRQVWUDWHGDSURPLVLQJDSSURDFKWRDXWRPDWHWKLVFDOLEUDWLRQE\DGGLQJDEXLOWLQH\HWUDFNHULQWRWKHJODVVHV$QRWKHUOLPLWDWLRQRIRXUV\VWHPLVWKHODFNRIUDGLRPHWULF
FRPSHQVDWLRQWRHTXDOL]HWKHEULJKWQHVVDQGFRORUVDFURVVWKH
JODVVHV DQG WKH SURMHFWLRQ DV ZHOO DV DFURVV D YDULHW\ RI
VXUIDFHVLQWKHHQYLURQPHQW7KHUHDUHQXPHURXVDSSURDFKHVWR UDGLRPHWULF FRPSHQVDWLRQ HJ >@  DGGUHVVLQJ WKLVLVVXHLVDQLPSRUWDQWFRQVLGHUDWLRQIRUIXWXUHZRUN
7KHUHDUHVRPHIXQGDPHQWDOFKDOOHQJHVZLWKVXSHULPSRVLQJ
VWHUHRVFRSLF YLHZV RQ WRS RI PRQRVFRSLF SURMHFWLRQV )RUH[DPSOHVLQFHRXUJODVVHVGRQRWDOORZIRUDQ\IRFXVFRQWUROWKHUH ZLOO EH D IRFXV PLVPDWFK IRU D PDMRULW\ RI YLUWXDO
REMHFWVLQWKHVFHQH6LQFHRXUSURMHFWRUSURMHFWVPRQRVFRSLF
FRQWHQWZHDVVXPHWKH³PLGGOH´H\HSRVLWLRQWRPLQLPL]HGLVSDULW\ LQ HDFK H\H +RZHYHU WKLV DSSURDFK FUHDWHV D
QRWLFHDEOHPLVPDWFKEHWZHHQWKHYLHZVVKRZQLQWKH267
JODVVHV DW HDFK H\H DQG WKH SURMHFWLRQ 7KLV LV SDUWLFXODUO\YLVLEOHIRUYLUWXDOREMHFWVIDUDZD\IURPDSURMHFWLRQVXUIDFH
LH ZLWK ELJ GLVSDULW\  :H KDYH DOVR H[SHULPHQWHG ZLWKVHWWLQJWKHSURMHFWLRQSRLQWWRWKHGRPLQDQWH\HRIWKHXVHUKRZHYHUWKDWPDNHVWKHGLVSDULW\HYHQPRUHREYLRXVLQWKH
QRQGRPLQDQWH\H$QLQWHUHVWLQJVROXWLRQ DQGIXWXUHZRUN ZRXOG EH WR SURMHFW VWHUHR FRQWHQW DQG FRQYHUW RXU 267JODVVHVWRDOVRIXQFWLRQDVVKXWWHUJODVVHV>@)RYH$5LVQRWDPRELOHVROXWLRQZKLFKUHVWULFWVWKHXVHUWR
D SDUWLFXODU VSDFH :KLOH RWKHU VROXWLRQV > @ KDYH
GHPRQVWUDWHG HQYHORSLQJ HQWLUH URRPV LQ SURMHFWLRQH[SHULHQFHV WKRVH VROXWLRQV FRPH ZLWK VLJQLILFDQWLQVWDOODWLRQ DQG PRQHWDU\ FRVWV ,Q WKH IXWXUH LW PLJKW EH
LQWHUHVWLQJ WR FRQVLGHU FRPELQLQJ 267 JODVVHV ZLWK ERG\ZRUQ SURMHFWLRQV >@ ZKLFK FRXOG PDNH )RYH$5 PRELOH$OVR)RYH$5FXUUHQWO\UHTXLUHVDUDWKHUFRPSOLFDWHGVHWXSDQG FDOLEUDWLRQ +RZHYHU LI WKH WUDFNLQJ WHFKQRORJ\ LV
LQWHJUDWHGLQWRWKHJODVVHVWKHPVHOYHV)RYH$5FRXOGHDVLO\EH LPSOHPHQWHG ZLWK DQ DGGLWLRQ RI D VLQJOH LQWHJUDWHG
SURMHFWRUFDPHUDXQLW
35(/,0,1$5<86(5)(('%$&.

:HGHPRQVWUDWHG)RYH$5WRXVHUV IHPDOH\HDUVROG 7KUHHRXWRIILYHZHUHYHU\IDPLOLDUZLWK95JDPLQJ
DQG KHDGZRUQ GLVSOD\V $OO XVHUV FRPPHQWHG RQ WKHWUDFNLQJ SUREOHPV ZKLFK ZH GLVFXVVHG LQ WKH /LPLWDWLRQV(YHU\RQH DOVR QRWLFHG WKH V\VWHP¶V FRPSOH[LW\ DQG ZHUHXQVXUHZKHWKHUWKHEHQHILWVZHUHODUJHHQRXJKWRPRWLYDWH
WKHPWRLQVWDOOLWLQWKHLUKRXVH7ZRVXJJHVWHGRQO\XVLQJWKH YLHZGHSHQGHQW SURMHFWLRQ :H EHOLHYH WKDW ZLWK WKHDYDLODELOLW\ RI WUDFNLQJ LQWHJUDWHG LQ WKH JODVVHV DQG DFRPELQHGSURMHFWRUFDPHUDXQLWPXFKRIWKHFRPSOH[LW\LQ
WKHVHWXSDQGWUDFNLQJFRXOGEHUHGXFHG
7KH ZLGHDQJOH LPPHUVLYH VLPXODWLRQ DQG ' OLIHVL]HWHOHFRQIHUHQFLQJZHUHWKHWZRH[SHULHQFHVWKDWUHVRQDWHGWKH
PRVW ZLWK RXU XVHUV ZKR FRPPHQWHG WKDW ZLWKRXW WKHSHULSKHU\RQHZRXOGKDYHDGLIILFXOWWLPHXQGHUVWDQGLQJWKH
DFWLRQ LQ WKH VFHQH 2QH XVHU FRPPHQWHG WKDW WKH PRVWLQWHUHVWLQJDVSHFWRI)RYH$5ZDVWKHDELOLW\WRVHDPOHVVO\FRPELQHSXEOLFDQGSULYDWHYLHZV:KLOHWKLVLVQRWXQLTXH
WRRXUV\VWHP>@ZHKRSHWRLQYHVWLJDWHWKLVIXUWKHU
&21&/86,216

)RYH$5 LV D SURRIRIFRQFHSW K\EULG $5 GLVSOD\ ZKLFKFRPELQHV D WUDFNHG 267 QHDUH\H GLVSOD\ DQG D VSDWLDOO\UHJLVWHUHG YLHZGHSHQGHQW SURMHFWLRQ :H EHOLHYH WKDW)RYH$5LVDXQLTXHVROXWLRQLQLWVDELOLW\WRFUHDWHDULFKDQG
LPPHUVLYH VHW RI ZLGHDQJOH $5 H[SHULHQFHV :H DOVRGHPRQVWUDWHKRZWKHJODVVHVDQGWKHSURMHFWRUFRPSOHPHQW
HDFK RWKHU DQG RIIHU SULYDWH VWHUHRVFRSLF YLHZV SHUSL[HO
FRQWUROOHGDPELHQWOLJKWLQJDQGVXUIDFHVKDGLQJHIIHFWVQRW
DIIHFWHG E\ WUDFNLQJ ODJ RU QRLVH 7KHVH LPSURYHPHQWV DUHEHQHILFLDOUHJDUGOHVVRIWKH)29
:H EHOLHYH WKDW RXU FXUUHQW LPSOHPHQWDWLRQ VKRZV WKHSRWHQWLDORIXVLQJ)RYH$5LQUHDOZRUOGVFHQDULRVDQGZHDUH
H[FLWHG WR ZRUN ZLWK FRQWHQW GHVLJQHUV WR SURWRW\SH QHZH[SHULHQFHVWKDWWDNHDGYDQWDJHRIRXUXQLTXHFRQILJXUDWLRQ5()(5(1&(6 %DXGLVFK3*RRG1DQG6WHZDUW3)RFXVSOXV&RQWH[W6FUHHQV&RPELQLQJ'LVSOD\7HFKQRORJ\ZLWK9LVXDOL]DWLRQ7HFKQLTXHV,QProc. of ACM UIST 2001.
 %HQNR+:LOVRQ$'DQG=DQQLHU)'\DGLF
SURMHFWHGVSDWLDODXJPHQWHGUHDOLW\,QProc. of ACM
UIST 2014.
 %HQNR+-RWD5DQG:LOVRQ$'0LUDJH7DEOH)UHHKDQG,QWHUDFWLRQRQD3URMHFWHG$XJPHQWHG5HDOLW\7DEOHWRS,QProc. of ACM SIGCHI 2012±
 %LPEHU2DQG5DVNDU56SDWLDODXJPHQWHGUHDOLW\
0HUJLQJUHDODQGYLUWXDOZRUOGV$.3HWHUV/WG
 %LPEHU2DQG)URKOLFK%2FFOXVLRQ6KDGRZV8VLQJ
SURMHFWHGOLJKWWRJHQHUDWHUHDOLVWLFRFFOXVLRQHIIHFWVIRU
YLHZGHSHQGHQWRSWLFDOVHHWKURXJKGLVSOD\V,Q Proc. ofIEEE ISMAR 2002 &KHQJ':DQJ<+XD+DQG6DVLDQ-'HVLJQRIDZLGHDQJOHOLJKWZHLJKWKHDGPRXQWHGGLVSOD\XVLQJIUHHIRUPRSWLFVWLOLQJOpt. Lett.,   ±
-XQ )HLQHU6DQG6KDPDVK$+\EULGXVHULQWHUIDFHV
EUHHGLQJYLUWXDOO\ELJJHULQWHUIDFHVIRUSK\VLFDOO\VPDOOHUFRPSXWHUV,QProc. of ACM UIST 1991
 *HQF<7XFHU\DQ0DQG1DYDE13UDFWLFDO
6ROXWLRQVIRU&DOLEUDWLRQRI2SWLFDO6HH7KURXJK'LVSOD\V,QProc. of IEEE ISMAR 2002. 
 +DUULVRQ&%HQNR+DQG:LOVRQ$'2PQL7RXFKZHDUDEOHPXOWLWRXFKLQWHUDFWLRQHYHU\ZKHUH,QProc. of
ACM UIST 2011±
+RX-1DP<3HQJ:DQG/HH.0(IIHFWVRI
VFUHHQVL]HYLHZLQJDQJOHDQGSOD\HUV¶LPPHUVLRQWHQGHQFLHVRQJDPHH[SHULHQFH&RPSLQ+XPDQ%HKDYLRU  ±
+XD+%URZQ/DQG*DR&6FDSH6XSSRUWLQJVWHUHRVFRSLFFROODERUDWLRQLQDXJPHQWHGDQGSURMHFWLYHHQYLURQPHQWVIEEE Comp. Graph. Appl.
±
,WRK<DQG.OLQNHU*,QWHUDFWLRQIUHHFDOLEUDWLRQIRU
RSWLFDOVHHWKURXJKKHDGPRXQWHGGLVSOD\VEDVHGRQ'
H\HORFDOL]DWLRQ,QProc. of IEEE3D User Interfaces(3DUI), 2014. ±
-RQHV%%HQNR+2IHN(DQG:LOVRQ$',OOXPL5RRP3HULSKHUDO3URMHFWHG,OOXVLRQVIRU
,QWHUDFWLYH([SHULHQFHV,QProc. of ACM SIGCHI 2013.
-RQHV%6RGKL50XUGRFN00HKUD5%HQNR
+:LOVRQ$2IHN(0DF,QWU\UH%5DJKXYDQVKL1DQG6KDSLUD/5RRP$OLYH0DJLFDO([SHULHQFHV(QDEOHGE\6FDODEOH$GDSWLYH3URMHFWRU&DPHUD8QLWV,QProc. of ACM UIST 2014.
-RQHV-$6ZDQ,,-(%RODV03HULSKHUDO
6WLPXODWLRQDQGLWV(IIHFWRQ3HUFHLYHG6SDWLDO6FDOHLQ
9LUWXDO(QYLURQPHQWVTVCG, May 2013.UHVV%DQG6WDUQHU7$UHYLHZRIKHDGPRXQWHGGLVSOD\V +0' WHFKQRORJLHVDQGDSSOLFDWLRQVIRUFRQVXPHUHOHFWURQLFV,QProc. SPIE, YRO

/DQPDQ'DQG/XHENH'1HDUH\HOLJKWILHOGGLVSOD\VACM Trans. Graphics. 32 1RY 

/LYLQJVWRQ0*DEEDUG-/6ZDQ,,-(6LEOH\&0DQG%DUURZ-%DVLFSHUFHSWLRQLQKHDGZRUQDXJPHQWHGUHDOLW\GLVSOD\VHuman Factors in
Augmented Reality Environments.6SULQJHU
0DLPRQH$DQG)XFKV+&RPSXWDWLRQDO$XJPHQWHG
5HDOLW\(\HJODVVHV,QProc. of IEEE ISMAR 2013
0DLPRQH$/DQPDQ'5DWKLQDYHO..HOOHU.
/XHENH'DQG)XFKV+3LQOLJKW'LVSOD\V:LGH)LHOG
RI9LHZ$XJPHQWHG5HDOLW\(\HJODVVHV8VLQJ'HIRFXVHG3RLQW/LJKW6RXUFHV,QProc. of ACM
SIGGRAPH 2014.
0DLPRQH$<DQJ;'LHUN16WDWH$'RX0
DQG)XFKV+*HQHUDO3XUSRVH7HOHSUHVHQFHZLWK+HDG:RUQ2SWLFDO6HH7KURXJK'LVSOD\VDQG
3URMHFWRU%DVHG/LJKWLQJ,QProc. of IEEE VR 2013.
2OZDO$DQG)HLQHU66SDWLDOO\DZDUHKDQGKHOGVIRUKLJKSUHFLVLRQWDQJLEOHLQWHUDFWLRQZLWKODUJHGLVSOD\V,QProc. of TEI 20093S
3LQKDQH]&7KH(YHU\ZKHUH'LVSOD\SURMHFWRU$
GHYLFHWRFUHDWHXELTXLWRXVJUDSKLFDOLQWHUIDFHV,QProc.
ACM Ubicomp 2001.
5DVNDU5:HOFK*&XWWV0/DNH$6WHVLQ/
DQG)XFKV+7KH2IILFHRIWKH)XWXUH$8QLILHG
$SSURDFKWR,PDJH%DVHG0RGHOLQJDQG6SDWLDOO\,PPHUVLYH'LVSOD\V,QProc. of ACM SIGGRAPHµ5DVNDU5:HOFK*/RZ./DQG%DQG\RSDGK\D\'6KDGHU/DPSV$QLPDWLQJUHDOREMHFWVZLWKLPDJHEDVHGLOOXPLQDWLRQ,QProc. of the Eurographics
Workshop on Rendering Techniques.
6ODWHU0/LQDNLV98VRK0DQG.RRSHU5
,PPHUVLRQSUHVHQFHDQGSHUIRUPDQFHLQYLUWXDOHQYLURQPHQWV$QH[SHULPHQWZLWKWULGLPHQVLRQDOFKHVV,QProc. of ACM VRST 1996
7XFHU\DQ0*HQF<DQG1DYDE16LQJOHSRLQWDFWLYHDOLJQPHQWPHWKRG 63$$0 IRURSWLFDOVHH
WKURXJK+0'FDOLEUDWLRQIRUDXJPHQWHGUHDOLW\Presence: Teleoper. Virt. Environ.
:HW]VWHLQ*DQG%LPEHU25DGLRPHWULF
&RPSHQVDWLRQWKURXJK,QYHUVH/LJKW7UDQVSRUW,QProc.
of Pacific Graphics
:LOVRQ$DQG%HQNR+&RPELQLQJPXOWLSOHGHSWKFDPHUDVDQGSURMHFWRUVIRULQWHUDFWLRQVRQDERYHDQG
EHWZHHQVXUIDFHV,QProc. of ACM UIST 2010.
:LOVRQ$'%HQNR+,]DGL6DQG+LOOLJHV2
6WHHUDEOH$XJPHQWHG5HDOLW\ZLWKWKH%HDPDWURQ,Q
Proc. of ACM UIST 2012
=KRX-/HH,7KRPDV%0HQDVVD5)DUUDQW$
DQG6DQVRPH$,Q6LWX6XSSRUWIRU$XWRPRWLYH
0DQXIDFWXULQJ8VLQJ6SDWLDO$XJPHQWHG5HDOLW\Intern. Journal of Virtual Reality (IJVR 2012) 