## 3 Experiment Design
### Proposed Approach
#### 3.3.1 Preparation  
In the preparation phase, the domain experts gave roles to the participants. The roles were given based on participant's knowledge in the related domain. The roles were given as follows:  
* Domain Experts (2)
* Project Manager (1)
* Team Leader (1)
* Requirements Engineers (3)
* Data Processing Engineers (3)
* Quality Assurance Engineers (2)
* Developers (4)  
Team T1 was comprised of two domain experts, one project, manager, one team leader, three requirements engineers, three data processing engineers, two quality assurance engineers, and four  
Figure 5: Experimental Process. developers. One requirements engineer's role was duplicated and participated as a developer in team T1. During the preparation phase, keywords related to the smartwatch domain were extracted from Facebook and Twitter using their APIs (Application Programming Interface). Related keywords from two Facebook pages (Smartwatch iwatch, Wearable-watch) were extracted. For Twitter, five hashtags (#smartwatch design, #smartwatch app, #smartwatch interface, #smartwatch battery, #smartwatch OS) were used to extract keywords. The hashtag (#) allows users to search for specific keywords on Twitter. Searching through hashtags becomes very effective when Twitter is viewed at a massive scale to get opinions of masses towards a specific topic, such as knowing peoples' opinions of a certain public figure (#[PERSON]) or certain events (#elections) [34]. The two Facebook pages such as Smartwatch iwatch and Wearable-watch were selected because these pages were dedicated to discussing only smartwatch-related topics. Figure 6 shows the statistics of extracted keywords for the smartwatch domain. It shows the frequency of specific keywords on Facebook and Twitter. Based on the frequency of extracted keywords, smartwatch design and smartwatch apps keywords were selected with a frequency of 3230 and 2870 respectively, as shown in Figure 6. After keywords selection, we (in consultation with two domain experts) developed two queries to initiate discussion on SNS to collect users' opinions for the smartwatch domain: one for the application sub-domain and the other for the smartwatch design sub-domain as mentioned below:  
Q1: Does smartwatch provide enough applications for you? Q2: What does your experience say about smartwatch design? The Q1 was expected to be either yes or no. If yes, then members of T1 do not need to go further. In the case of no, \"what is your need\" has been exploited. The Q2 has been formed to address the needs of end-users. By using Q2, end-users were expected to report bugs, the shortcomings of features, etc. The \"smartwatch design\" refers to hardware and software and other services provided in terms of features. In response to Q2, users were expected to share their experiences about their smartwatches in terms of problems (bugs, shortcomings, etc.) they faced or the advantages they got. After developing domain-related queries, well-known SNS were investigated against the criteria mentioned in Section 2 and Facebook and Twitter were selected as candidate SNS to perform the experiment. After selection of SNS, queries were posted on selected SNS and users started the discussion and exchanged their opinions on the posted questions. For example, some users reported that \"My Apple Watch is still malfunctioning after a restart\" and \"I find many problems with this DM98 smart watch. Firstly, the sim is not supported. It shows mobile network not available whereas the same sim card works properly on any mobile handset (in India). Secondly, the weather shows the location of Alaska by default and it can't be changed to my location or any other location. Thirdly, the voice input keyboard is not supported. Dear researchers, if you have solutions for any of the above  
Figure 6: Statistics of smartwatch domain keywords. issues please let us know\", respectively. The users reported bugs in an effective way. Such kinds of information were very useful in order to elicit user requirements.