2019年度日本認知科学会第36回大会

P1-3

Proposal of a System Based on Cognitive Architecture to Distract Rumination
While Web Browsing
Thanakit Pitakchokchai, Junya Morita, Yusuke Yamamoto, Hiroyasu Yuhashi, Teppei Koguchi
Graduated School of Integrated Science and Technology, Shizuoka University
pitachokchai.thanakit.18@shizuoka.ac.jp

Abstract[4]. These side-effects of technology are a source of mental

Despite its benefits, technology can negatively affect humansymptoms such as stress, anger, anxiety and depression.

behavior and emotion unconsciously. This research tries toTherefore, there has been an increase of interest in field of

support the attempt of positive computing through thepositive computing: the design and development ofdevelopment of a system based on ACT-R cognitivetechnology to support psychological well-being and humanarchitecture to prevent rumination, repetitive negative

potential [5]; and computational psychiatry: a diagnosis andthinking. The proposed system consists of two sub-systems:

treatment of mental illness with computational method suchdata collection sub-system and distraction sub-system. The

as building computational model of human cognition andformer collects searching data (e.g., website URL, image

reinforcement learning to address psychiatric problems–inURLs) immediately after the individual visits a website. The

other word, mental illness [6]. In this paper, we will follow thedata will be collected through a browser extension so that it

objective and methodology of positive computing andwill be compactible for any computer and stored in local

computational psychiatry to build a cognitive model thatdatabase. The latter includes an ACT-R cognitive model thatprovides a diagnosis and treatment for mental illness to people

utilizes such data as well as physiological data (e.g., eyeunder controlled environment such as website.

movement, gaze fixation, heart rate variability) directly from

2. Related Studies

the individual to predict rumination while searching throughwebsites. The model is activated by a browser extension upon

2.1 Mental Illness

starting it. In addition to predicting rumination, it provides an

We have reviewed several papers regarding to mental

implicit intervention for rumination based on a concept ofillness, and depressive rumination—which is the precedednudge which affects human behavior and decision making.

stage of depression and considered as serious mental

The system displays a product image on the screen as

symptom—is needed to be treated prematurely [7].

advertisement. After rumination is detected, the system

Depressive rumination is defined as the self-focused repetitivechanges the image to mildly intervene not to keep ruminating.

negative thinking about the cause of his/her depression [8], [9]Keywords ― Cognitive Architecture, ACT-R, Mental
Illness, Rumination, Positive Computing, Computational
Psychiatry, Website, Advertisingwhich prolongs dysphoric mood and sadness [10]. Some
researchers have claimed that rumination associates withmind-wandering, a state that one’s thought is out of focus or

1. Introduction

unrelates to main task [11]. At the moment of mind-wandering,rumination distracts one’s attention and concentration fromIn recent decades, more and more innovative technologies

achieving main task and replaces the position with thoughts

have been invented to facilitate human lives. In the meantime,about one’s past memory such as disappointments andtechnology, especially the internet, has unconsciously

mistakes [12], [13]. In addition, many studies reported thechanged human behavior and emotional stability. For example,

symptoms of ruminating individuals: they are more likely tosome people are facing the internet addiction–that is, to have

have attentional biases for negative information [14], [15],difficulties distracting oneself from the internet [1], [2]. Some[16]. People who engage in depressive rumination are facing

are having less empathy and even involving in cyberbullying

a greater risk for depression [7].

– an aggressive, intentional behavior conveyed towards otherpeople like mean comments on social network sites and2.2 Cognitive Modeluploading embarrassing personal videos/pictures of others [3],To address the problem of depressive rumination, we will57

2019年度日本認知科学会第36回大会

P1-3

build a cognitive model that is based on Adaptive Control ofto choose the proper choice. Moreover, the previous study

Thought – Rational (ACT-R) cognitive architecture. The[21] suggests using nudge to prevent behavioral risk leading

ACT-R [17] is capable of simulating human cognitive

to major depressive disorder. One of the suggested methods isoperation according to brain regions such as visual module in

to influence subjects to employ adaptive coping mechanism

the model that simulates human vision is imitated fromover maladaptive one to deal with stressful situation. We also

occipital lobe. In this research, we refer to two ACT-R models.

found that advertisement can used as a nudge to influenceFirst, van Vugt et al. [18] applied their recent computationalcustomer’s behavior and decision making. For instance, anmodel of mind-wandering and proposed a method ofadvertisement showing message “DRINK MORE WATER.”

implementing the model to simulate rumination. The model

followed by “YOU SWEAT IN THE HEAT: YOU LOSE

simulated chunks of different moods (cheerful, content, down,WATER”, is a form of nudge [20] which does encouragesuspicious, and insecure) of 100 normal participants and 100

people without forcing them to drink water. Therefore, we see

ruminating participants. The model is able to produce

great potential to adapt Itabashi’s model that recommends anfrequencies of chucks in each mood that were recalled during

image associated with memory to build the model that showsmind-wandering and predict an effect of rumination against

advertisement based on user memory to draw attention andperformance on attentional task. The simulation result

mildly remind (nudge) user not to keep ruminating.

suggests that ruminating participants tends to fall into mind-

3. Objectives

wandering because they retrieve negative moods more often
than control participants who retrieve chunks equally. InIn this research, we hypothesized that the cognitive model

addition to van Vugt et. al.’s model, Itabashi [19] created abuilt with ACT-R could yield an accurate prediction ofcognitive model based on ACT-R to simulate a reminiscentrumination—not just a simulation—by utilizing physiologicalphoto slideshow. The model tries to select an image thatdata, and the concept of nudge used in form of web advertising

matches participant’s memory by using image data andbanner could draw attention and distract user from ruminating

physiological data. The former is from images in participant’swhile searching through websites.

photo library that were processed by image recognition–theimage data are tagged from the viewpoint of who, what, when,

4. System

where. The latter is from participant directly. Participants wore
a heart rate sensor, myBeat, at the beginning of the experiment.

In this study, we propose an intervention approach for

The model receives heart rate data and uses it as a noisedepressive rumination by applying van Vugt et al.’s ACT-R

parameter.

model [18] to predict rumination and create a system based on
Itabashi’s model that distracts user from ruminating.

2.3 Advertisement

Moreover, we try to improve prediction accuracy of the model

By applying the framework of the above previous studies,by physiological data: heart rate variability (HRV) which isthe current study aims to construct a system to predict andable to signify degree of depression including rumination [22]distract rumination. The system includes a cognitive modeland eye movement and gaze fixation to indicate level ofpredicting rumination through a simulation of memory

attention, etc. [23], [24]. The system will be in a form ofchunks associated with moods. In addition, it includes an eyebrowser extension that can be installed on any website. It will

tracker to collect physiological data from user in real time andshow an advertising banner of products that is images on

use it as parameters to predict rumination more accurately. To

websites that the user has visited before (encoded in user’s

distract rumination, we focus on the concepts of nudge [20]memory). The advertisement is processed by the model towhich is an approach of implicit intervention that affects

show most relevant image associated with user’s memory inindividual’s behavior and decision making. As an example ofterms of recency and frequency. The advertising banner will

nudge, we can consider the slight change of choicefunction as a distractor: drawing user’s attention fromarchitecture where options are presented to lead individuals orrumination while searching through websites. The systemgroup of people to positive cognitive biases encouraging themconsists of two sub-systems: data collection and distraction.

58

2019年度日本認知科学会第36回大会

P1-3

Table 1: An example of a table in database that stores user’s searching data: user ID, timestamp, website URL, and image URL
user_idtimestamp

web_url

img_url

1056724477

2019/7/3 12:12:09

https://www.amazon.co.jp

https://images-na.ssl-images-amazon.com/images/I/
51%2BYxxM-PAL._SL1000_.jpg

1056724477

2019/7/3 12:12:09

https://www.amazon.co.jp

https://images-na.ssl-images-amazon.com/images/I/
61p7Ozqex0L._SL1000_.jpg

1056724477

2019/7/3 12:12:09

https://www.amazon.co.jp

https://images-na.ssl-images-amazon.com/images/I/
618m86YqQ3L._SL1000_.jpg

1056724477

2019/7/3 12:12:09

https://www.amazon.co.jp

https://images-na.ssl-images-amazon.com/images/I/
61A75HOU2xL._SL1500_.jpg

5.1 Data Collection Sub-system
In this sub-system, we will create a Google Chrome
extension that logs all image URLs on any website that uservisits including web URL and timestamp and remotely inserts
into database as can be seen from Figure. 1.

Figure 1: The flow of data collection sub-system

Figure 2: The flow of distraction sub-system

The chrome extension automatically inserts data into

As stated above, we focus on product images that will be

database in four columns: user ID, timestamp, website URL,

used as advertising banner. Therefore, we connect the systemand image URL as shown in Table 1. In the test trail, wewith Google Cloud Vision, a machine-learning-based image

obtained around 2,000 to 5,000 rows per hour.

recognition software. The system retrieves image URLs from

5.2 Distraction Sub-system

database, passes to Google Cloud Vision to categorize images
which a certain user has seen and produce results such as typeFigure 2 displays the flow of distraction sub-system—theof objects, labels, and color. Figure 3 is the result fromACT-R model (the middle part of the figure) in this system

requesting Cloud Vision API to detect an image URL of a fanreceives data via two ways: physiological data (e.g., eyefrom a shopping site. It shows that, for example, the image ismovement, gaze fixation, HRV) from eye tracker and heartclassified as product with 88% detection accuracy and israte monitor (the right side of the figure), and data obtained

categorized as mechanical fan with 86% detection accuracy.

from user (the top left part of the figure).

The ACT-R model processes such data to select and display a
product image on the screen, predict rumination, and distractuser by changing the image to another. The image will be59

2019年度日本認知科学会第36回大会

P1-3

chosen from criteria and parameters such as recency andD’Mello [25]. They used an eye tracker to observe

frequency and is expected to be relevant to user’s memory.

individual’s attention (e.g., gazing data, pupil size) when

Google Chrome extension initiates the model immediately

reading a book. With an application of supervised machine

after a web browser is opened. This way, the system islearning algorithms, the result achieves a 72% accuracy ratecompatible to install on any computer. The ACT-R model

to detect mind-wandering. Therefore, our future model would

transforms product image data and physiological data

include supervised machine learning algorithms in atransmitted from the user in real time into ACT-R data to use

collaboration with ACT-R modeling to predict rumination.

as parameters. The model selects an image from the pool ofimages shown on the websites that user has visited and6. Acknowledgement

displays on the screen as advertising banner. While the user

This research is supported by “JSPS Topic-Setting Program

searches through websites, physiological data is constantly

to Advance Cutting-Edge Humanities and Social Sciencestransmitted to the model as a cycle.

Research”.
{"responses": [{ "labelAnnotations": [{"mid": "/m/01jwgf","description": "Product","score": 0.8828333, "topicality": 0.8828333},{"mid": "/m/02x984l","description": "Mechanical fan","score": 0.8628455, "topicality": 0.8628455},{"mid": "/m/0h8n8m2","description": "Ventilation fan","score": 0.8209952, "topicality": 0.8209952}],"localizedObjectAnnotations": [ {"mid": "/m/019dx1", "name": "Home appliance","score": 0.60034335, "boundingPoly": {"normalizedVertices": [{ "x": 0.18847057, "y": 0.13999999 },{ "x": 0.80023533, "y": 0.13999999 },{ "x": 0.80023533, "y": 0.7790588 },{ "x": 0.18847057, "y": 0.7790588 }]}}]}]}
Figure 3: The result of Google Cloud Vision processing animage URL of a fan in a shopping site

5. Summary
This paper describes the fundament of a system based on
cognitive architecture that predicts rumination and provides
an intervention (distract) through the concept of “nudge” touser while browsing websites.
In the future, we expect a cognitive model that can detectrumination more accurately and a system that distracts
rumination more effectively. We plan to apply the method thatpredicts mind-wandering which was proposed by Bixler &60

2019年度日本認知科学会第36回大会

References

P1-3[15] Bradley, B. P., Mogg, K., & Lee, S. C. (1997) “Attentional biases
for negative information in induced and naturally occurring[1] Fitzpatrick, J. J. (2008) “Internet addiction: recognition andinterventions”, Archives of psychiatric nursing, 22(2), 59-60.
[2] Xiuqin, H., Huimin, Z., Mengchen, L., Jinan, W., Ying, Z., & Ran, T.

dysphoria”, Behaviour research and therapy, 35(10), 911-927.
[16] Donaldson, C., Lam, D., & Mathews, A. (2007) “Rumination andattention(2010) “Mental health, personality, and parental rearing styles ofadolescents with Internet addiction disorder”. Cyberpsychology,Behavior, and Social Networking, 13(4), 401-406.
[3] Slonje, R., & Smith, P. K. (2008) “Cyberbullying: Another main typeof bullying?”, Scandinavian journal of psychology, 49(2), 147-154.

psychiatry: towards a mathematically informed understanding ofmental illness”, J Neurol Neurosurg Psychiatry, 87(1), 53-63.
[7] Kuehner, C., & Weber, I. (1999) “Responses to depression in unipolar

Investigators. (2018) “How does rumination impact cognition? A
first mechanistic model”, Topics in Cognitive Science, 10(1), 175191.
[19] Itabashi K. (2019) “Evaluation of Photo Slideshow Implementinga Cognitive Model and Physiological Indices”, ShizuokaUniversity.
[20] Thaler, R. H., & Sunstein, C. R. (2009) “Nudge: Improving
decisions about health, wealth, and happiness”, Penguin.
[21] Woodend, A., Schölmerich, V., & Denktaş, S. (2015) ““Nudges”
to Prevent Behavioral Risk Factors Associated With Major
Depressive Disorder”, American journal of public health, 105(11),styles theory”, Psychological medicine, 29(6), 1323-1333.
depression and posttraumatic stress symptoms after a natural disaster:2318-2321.
[22] Gorman, J. M., & Sloan, R. P. (2000) “Heart rate variability indepressive and anxiety disorders”, American heart journal, 140(4),the 1989 Loma Prieta Earthquake”, Journal of personality and social
psychology, 61(1), 115.
[9] Treynor, W., Gonzalez, R., & Nolen-Hoeksema, S. (2003)

S77-S83.
[23] Hyrskykari, A. (2006) “Utilizing eye movements: Overcoming
inaccuracy while tracking the focus of attention during“Rumination reconsidered: A psychometric analysis”, Cognitive
therapy and research, 27(3), 247-259.
[10] Nolen-Hoeksema, S., Morrow, J., & Fredrickson, B. L. (1993)

reading”, Computers in human behavior, 22(4), 657-671.
[24] Vazquez, C., Blanco, I., Sanchez, A., & McNally, R. J. (2016)“Attentional bias modification in depression through gaze

“Response styles and the duration of episodes of depressed

contingencies and regulatory control using a new eye-trackingmood”, Journal of abnormal psychology, 102(1), 20.

intervention paradigm: study protocol for a placebo-controlled[11] Killingsworth, M. A., & Gilbert, D. T. (2010) “A wandering mind isan unhappy mind”, Science, 330(6006), 932-932.
[12] Wenzlaff, R. M., & Luxton, D. D. (2003) “The role of thought

and[18] Van Vugt, M. K., van der Velde, M., & ESM‐MERGE

depressed patients: An investigation of Nolen-Hoeksema's response[8] Nolen-Hoeksema, S., & Morrow, J. (1991) “A prospective study ofresearchphysical universe?”, Oxford University Press.

for Wellbeing and Human Potential”, The MIT Press.
[6] Adams, R. A., Huys, Q. J., & Roiser, J. P. (2016) “Computationaldepression”, Behaviour[17] Anderson, J. R. (2009) “How can the human mind occur in the(2012) “Cyberbullying: Bullying in the digital age”, John Wiley &[5] Calvo, R. A. & Peters, D (2014) “Positive Computing: Technologymajortherapy, 45(11), 2664-2678.

[4] Kowalski, R. M., Limber, S. P., Limber, S., & Agatston, P. W.
Sons.

intrial”, BMC psychiatry, 16(1), 439.
[25] Bixler, R., & D’Mello, S. (2014, July) “Toward fully automatedperson-independent detection of mind wandering”, In International

suppression in depressive rumination”, Cognitive Therapy andConference

Research, 27(3), 293-308.

onUserModeling,Personalization (pp. 37-48), Springer, Cham.

[13] Cramer, A. O. J., van Borkulo, C. D., Giltay, E. J., van der Maas, H.
L. J., Kendler, K. S., Scheffer, M.., & Borsboom, D. (2016) “Major
depression as a complex dynamic system”, PLoS ONE, 11(12),e0167490.
[14] Mogg, K., Bradley, B. P., & Williams, R. (1995) “Attentional bias inanxiety and depression: The role of awareness”, British journal ofclinical psychology, 34(1), 17-36.

61

Adaptation,and