A STUDY ON MEMORABILITY AND SHOULDER-SURFING ROBUSTNESS OF
GRAPHICAL PASSWORD USING DWT-BASED IMAGE BLENDING
Takao Miyachi*1, Keita Takahashi*2, Madoka Hasegawa*1, Yuichi Tanaka*1, Shigeo Kato*1*1

Graduate School of Engineering, Utsunomiya University,*2 Faculty of Engineering, Utsunomiya University
7-1-2 Yoto, Utsunomiya, Tochigi 321-8585 Japan
E-mail: {miyachi@mclaren., takahashi@mclaren., madoka@, tanaka@, kato@}is.utsunomiya-u.ac.jp

Index Terms— Graphical password, discrete wavelet
transform, authentication, usable securityobservers to memorize. On the contrary, legitimate users areable to find his/her pass-image easily because theknowledge of the original clear image becomes a clue toremember the pass-image. These methods are effective toenhance the security level against observation attacks.
However, a trade-off exists between the security level andthe memorability of pass-images. As a result, the system
becomes less usable.
In the preliminary study [5], [6], we presented a conceptof an image synthesis method for a user authentication
system using graphical passwords. In this method, wecombine low frequency components of a decoy picture withhigh frequency components of a pass-image by using DWT.
For human eyes, it is difficult to recognize the subtle highfrequency components. Especially in the case that theperson is far from the screen, the high frequency
components become less visible. However, former user
study was conducted in a short term and memorability of thegraphical password in longer term was an open question. In
this paper, we evaluate longer term memorability andshoulder-surfing robustness of our graphical password.

1. INTRODUCTION

2. OVERVIEW OF PROPOSED SYSTEMEntering a user name and a textual password is a major
method for the computer login procedure. Unfortunately,
this method is vulnerable to spyware and key-loggers. Inaddition, it is difficult to remember long complex textualpasswords. Studies have shown that users tend to use shortpasswords or passwords that are easy to remember [1].
To address this issue, several authentication methods
have been proposed. In this paper, we focus on graphicalpasswords [2] which use pictures as passwords instead ofusing alphanumeric characters. Pictures are difficult to steal
with key-loggers. In addition, remembering pictures areeasier for human than remembering textual passwords.
Although graphical passwords are generally easy for
legitimate users to memorize, they are also easy forobservers or attackers who stand behind the users tomemorize. To alleviate this risk, graphical passwords, which
use degraded images, have been proposed [3], [4]. These
proposed approaches utilize the property that degradedimages look like noise or ink blots and they are difficult for

Suppose we use our authentication method for PDAs orweb-based services. When a user wants to use a service,s/he may start login procedure at his/her desk and his/herfriends or coworkers may see the computer screen when
they walk behind his/her desk. Our method aims to prevent
such observers from knowing the user’s pass-image. If apass-image is faintly printed on a decoy image, thelegitimate user who is close to the PC screen can see thepass-image but someone who is far from the screen cannotrecognize the pass-image. Our method tries to utilize thisproperty.
Fig. 1 shows an example of a set of images for achallenge using our graphical password method.
Beforehand, users register several images as their passimages. On the screen, a set of images are shown to the user.
One image in the set consists of low frequency components
of a decoy image and high frequency components of a passimage. The other three images are blended images which
consist of low frequency components of decoy images andABSTRACT
Graphical passwords are an authentication method thatuses pictures as passwords instead of using alphanumericcharacters. We propose a graphical password method which
is difficult to steal original pass-image by using
characteristics of human vision system. In our method, we
combine low frequency components of a decoy picture withhigh frequency components of a pass-image. It is easy for
legitimate users to recognize the pass-image in the blended
image. On the other hand, this task is difficult for attackers.
We used discrete wavelet transform (DWT) to blend a
decoy image and a pass-image. User studies are conducted
to evaluate memorability and shoulder-surfing robustness of
this method. We also compared our method with otherexisting methods in terms of the authentication time and thesuccess ratio by the user test. The results show that ourmethod is more usable and secure against shoulder-surfing.

high frequency components of other decoy images. The user
is asked to choose the image in which his/her pass-image isvaguely blended in the image. If s/he does not find any ofhis pass-images in the set, s/he is supposed to choose “Nopass-image” as his/her answer. This challenge is repeatedseveral times and if all the answers are correct, s/he will be
successfully authenticated. The number of challenges andregistered pass-images depend on the requirements forsecurity and usability. If the number is greater, the system
becomes more secure but becomes less usable because theuser has to properly remember more pass-images and takes
more time for the authentication.
Decoy & Decoy

Decoy
image

Pass-image

DWT

Decoy & Pass-image
IDWTNopassimage

Blended image

Fig. 1 Example of images displayed for a challenge.
3. IMAGE BLENDING METHOD FOR GRAPHICALPASSWORD
How to generate the blended image is the key in thisauthentication system. Fig. 2 shows the flow of our image
synthesis method. We used a discrete wavelet transform(DWT) to extract low and high frequency components fromimages because DWT requires less computation than DCT
to split the frequency components of two input images andto blend those frequency components for generating a
blended image.
First, DWT is applied to each color plane in a decoy
image and a pass-image. The LLx band, that is the lowest
frequency band of x-level DWT, contains the average
information of the input image. In contrast, the LH1, HL1,and HH1 bands contain mostly the edges of the images.
Names of each sub-band in 2-level DWT are shown in Fig.
3.
In the second step, the LLx band of the decoy image andthe LH1, HL1, and HH1 bands of the pass-image aremerged. We set zeros for the middle frequency sub-bands,
such as LH2, HL2, and HH2, to add some blur effect for thedecoy image. If the decoy image has many edges, theseedges are mixed with the edges of the pass-images and it
becomes difficult even for the legitimate user to recognize
the pass-image. We obtain a blended image for ourauthentication method after the inverse DWT (IDWT) andmerging of color planes.
An advantage of this method is storage space reduction
for the pass-image. Only the high frequency components,that are LH1, HL1, and HH1 bands, are required to be
stored in the system. For the authentication, the system
randomly chooses a decoy image and mixes it with the passimage to generate a blended image.

Fig. 2 Overview of the image blending procedure.

LL2 HL2
HL1
LH2 HH2
LH1

HH1

Fig. 3 Names of each sub-band in 2-level DWT.

(a) Decoy image( 256  256 pixels, 24bpp)(b) Pass-image( 256  256 pixels, 24bpp)(c) Blended image(d) A part of blended image(Enlarged)

Fig. 4 Example of the blended image.
In this study, we used 5/3-DWT filter which is a reversible
DWT used in JPEG 2000 [7]. Low frequency signals andhigh frequency signals are given by following equations.

 H ( n  1)  H ( n) L ( n )  x ( 2 n)  4(1) x(2n)  x(2n  2) H (n)  x(2n  1)  2

(2)

where x(n) is a pixel value in position n. Samples belonging
to the low and high frequency sub-bands are represented asL(n) and H(n) respectively. Examples of a decoy image, apass-image, and a blended image obtained using this DWT
are shown in Fig. 4.
4. LOW FIDELITY TEST OF PASS-IMAGEMEMORABILITY
We evaluated memorability of pass-images overwritten
on the blended images. Before the user test, 5 pass-images
are given to a user and he memorizes these images for 10minutes. During the user test, a set of 4 blended images, thatare generated using 2-level DWT, is shown to the user at
one challenge. One or no pass-image is included in the set
and the user is asked to specify the location of the passimage. One authentication trial consists of 7 challenges. We
carried out this test 5 times. Therefore, we obtained 35
answers from each participant. We did not add a limitationon the number of decoy image appearances. Therefore, thesame decoy image could appear several times. Participantsof this lo-fi test are 4 male students and all of them are 20s.
Table 1 shows the correct answer ratios for this test. This
result shows that most of the participants could recognize
pass-images correctly.

images, which were imposed by the system, and answered
their location using the mouse pointer.
There were 40 participants and we divided them into fourgroups to evaluate the effect of test intervals. The number ofparticipants and intervals of user tests for each group areshown in Table2. On the day "0", participants practiced theauthentication procedure several times until they becamefamiliar with the usage of the system. Then, they started theuser test of the authentication for each method. The
participants in group A carried out the authentication test
without the practice 1day later, 1 week later, 2 weeks later,
3 weeks later, and 1 month later. The intervals of theauthentication test were longer for other groups.
Table 2 Number of participants and intervals of user testsGroup#Intervals
A
100day, 1day later, 1week later, 2weeks later,
3weeks later, 1month laterB
100day, 1week later, 2weeks later, 3weeks
later, 1month laterC
100day, 2weeks later, 1month laterD
100day, 1month later
Total
40

(a) Original pass-image(b) Blended image(Proposed)(c) Harada’s method 1(Mosaicing, andadding random noize)(d) Harada’s method 2(pass-image: decoy = 6:4)
(Alpha blending , mosaicing,
and adding random noize)

Table 1 Number of correct answers and its ratio
user1
user2user3
user4# of correct answers
35353435
Ratio
100% 100%97% 100%

5. USER TEST AND COMPARISON
5.1. Memorability
We compared our method with the existing graphicalpassword methods proposed by Harada et al. [3] in terms of
the authentication time and the success ratio. Fig. 5 shows
examples of images generated by our method and Harada'smethods. The 2-level DWT was used for the proposedmethod. The mosaicing block sizes for the Harada's
methods were 4 by 4 pixels. The alpha-blending ratio was
pass-image : decoy = 6 : 4 for the Harada's method 2.
Prototypes of the graphical password system wasimplemented as an web application using Apache and PHP.
An authentiocation process consists of seven challenges andfour blended images and an icon of "no pass image" aredisplayed for each challenge. In this test, each participant
played a role of a legitimate user. They memorized 5 pass-Fig. 5 Comparison with Harada’s methods.
Fig. 6 shows the effect of test interval for success ratio.
Fig. 6(a) is the result of the proposed method, Fig. 6(b) is
the result of the Harada's method 1, and Fig. 6(c) is theresult of the Harada's method 2. Fig. 6(a) shows thatparticipants could choose almost correct pass-images
irrelevant to the test intervals in our method. On thecontrary, the success ratio decreases as the interval becomes
longer in the Harada's methods. This is due to thememorability of the pass-images.

success ratio (%)

AB

CD

100%
90%
80%70%60%
0day

1day

1wk

2wks

3wks

4wks

3wks

4wks

3wks

4wks

success ratio (%)(a) Proposed method
100%
90%
80%
70%
60%
0day

1day

1wk2wks

success ratio (%)(b) Harada’s method 1
100%90%80%70%
60%
0day

1day

1wk

2wks

(c) Harada’s method 2

Fig.6 Effect of test interval for success ratio.
5.2. Robustness against Shoulder-surfing
We also compared our method with the graphicalpassword using original clear image and the Harada’s
method 2 in terms of robustness against shoulder-surfing.
There were 6 participants and they played a role of anattacker. First, they complete a short training session on
how each graphical password system is used. Next, theparticipants try to gain pass-images through shouldersurfing while the experimenter carried out the authentication
correctly. The experimenter chose a pass-image out of 5images every 5 seconds. This challenge is repeated 7 times.
The participants stood about 50cm behind the experimenter.
Each participant was permitted taking notes during theshoulder-surfing. After the shoulder-surfing session, theparticipants were asked to carry out the authentication as
attackers using same pass-images. They could use theirnotes if they wanted.
Table 3 shows the false acceptance rate (that is, attack
success ratio). This result shows that proposed method issecure against the shoulder-surfing. Participants indicatedthat they could not recognize high frequency components ofa pass-image because they are far from the screen.
Table 3. Success ratio of shoulder-surfing
w/ image blending
Original image (w/oimage blending)Harada's 2
Proposed
FAR
95%69%7%

6. CONCLUSIONSWe proposed a graphical password using the property
that it is difficult to recognize subtle high frequency
components with human eyes. Our method is difficult forobservers who stand behind a user to know the pass-image,while it is easy for a user who is just in front of thecomputer screen. We also discussed an image blending
method for this graphical password system. We conducted alo-fi tests and a user test. The lo-fi tests revealed thefeasibility of this system and suitable parameters for theimage blending. The results of the user test showed
participants could complete the authentication faster thanexisting methods while the authentication success ratio washigher than them. In addition, the user test of shouldersurfing shows our method prevents attackers from stealing
pass-images.
We will study the details of the relationship between thevisibility of the pass-image and the complexities of theimage content in the future work. Hybrid images [8], which
blends two images of similar structure but slightly different,
are good candidates to compare with our method in terms ofimage content and visibility of pass-image because ourmethod blends two images of completely different structure.
In addition, the further user study with more participants
is required to evaluate robustness of pass-images from theviewpoint of more powerful attackers, i.e. video cameras.
We also need to consider the effect of age on visibility ofhigh frequency components.
ACKNOWLEDGMENTS
This work was supported in part by KAKENHI 22500105.
REFERENCES[1] A. Adams and M. A. Sasse, "Users are not the enemy: whyusers compromise computer security mechanisms and how totake remedial measures," Communications of the ACM, vol.
42, pp. 41-46, 1999.
[2] R. Dhamija and A. Perrig, "Déjà vu: A user study, using
images for authentication," Proc. 9th USENIX Security
Symposium, 2000.
[3] A. Harada, T. Isarida, M. Nishigaki, "A proposal of user
authentication using mosaic images," Proc. of computer
security symposium, pp.385-390, 2004. (in Japanese)[4] E. Hayashi, N. Christin, R. Dhamija, A. Perrig, "Use Your
Illusion: Secure Authentication Usable Anywhere," Proc. ofSOUPS08, pp.35-45, 2008.
[5] M. Hasegawa, Y. Tanaka, S. Kato, "A Study on an Image
Synthesis Method for Graphical Passwords," Proc. of ISPACS,WP2-D-2, Dec. 2009.
[6] M. Hasegawa, T. Miyachi, Y. Tanaka, S. Kato, "A graphicalpassword using discrete wavelet transform and its evaluation,"
IEVC 2010, 1P-5, Mar. 2010.
[7] David Taubman, Michael Marcellin, JPEG2000: Image
Compression Fundamentals, Standards and Practice, Springer,
2001.
[8] A. Oliva, A. Torralba, P.G. Schyns, "Hybrid images", ACM
Trans. on Graphics, Vol.25, No.3, pp.527-530, 2006.