Noname manuscript No.
(will be inserted by the editor)

How Do Developers Utilize Source Code from Stack
Overow?
Yuhao Wu · Shaowei Wang · Cor-Paul
Bezemer · Katsuro Inoue

Received: date / Accepted: date

Abstract Technical question and answer Q&A platforms, such as Stack Overow, provide a platform for users to ask and answer questions about a wide
variety of programming topics. These platforms accumulate a large amount ofknowledge, including hundreds of thousands lines of source code. Developers
can benet from the source code that is attached to the questions and answers
on Q&A platforms by copying or learning from (parts of ) it. By understanding
how developers utilize the source code from Q&A platforms, we can provide
insights for researchers which help improve next-generation Q&A platforms tohelp developers to reuse the source code fast and easily.
In this paper, we rst conduct an exploratory study on 95 JavaScript lesfrom 89 projects which contain source code that contains an explicit reference
to a Stack Overow post. In 49.4% of the studied cases, developers need tomodify source code to make it work in the required context, with the degreeof modication varying from renaming variables to rewriting the whole algorithm. Developers sometimes choose to implement an algorithm from scratch
based on the descriptions from Stack Overow answers, even if there is an implementation readily available in the post. To further understand the barriers
that prevent code reuse from developers and obtain suggestions for improving
the code reuse from Q&A platforms, we conduct a survey on 453 developers
on Stack Overow. We nd that the top 3 barriers that prevent developersfrom copying code from Stack Overow are: (1) too much code modication
required to t in their projects, (2) incomprehensive code, and (3) low code
quality.

Yuhao Wu · Katsuro Inoue
Graduate School of Information Science and Technology, Osaka University, Japan
E-mail: {wuyuhao,inoue}@ist.osaka-u.ac.jp
Shaowei Wang (B) · Cor-Paul Bezemer
SAIL, Queen's University, Canada
E-mail: {shaowei,bezemer}@cs.queensu.ca

2

Yuhao Wu et al.
We also summarize their suggestions for next-generation Q&A platforms

and discuss implications of our study on future research along the following
dimensions: (1) code quality, (2) information enhancement & management,(3) data organization, (4) license, and (5) human factor.

1 Introduction
Technical question and answer (Q&A) platforms such as Stack Overow havebecome more and more important for software developers to share knowledge.
Developers can post questions on these Q&A platforms, which in turn are answered by other developers. These answers often contain source code snippets.
As of August 2017, Stack Exchange reports that there are approximately
7.6 million users and 14 million questions with 23 million answers on Stack
Overow (Stack Exchange, 2017). Among those answers, 15 million (75%) haveat least one source code snippet attached, which forms a huge code base fordevelopers to reuse source code from.
However, reusing source code is not easy (Gamma et al., 1995). There are
two major challenges when reusing source code: (1) It is dicult for developers
to nd suitable source code based on their particular needs, such as language,
functionality, and performance (Wang et al., 2014a). To address this challenge,
a number of studies have been done to help developers to locate more relevant
source code snippets (Ponzanelli et al., 2014a,b; Wang et al., 2014a). (2) It
may require additional eort to integrate the found source code work in developers' own context. For example, parameters may need to be adjusted, or
additional source code may need to be added (Cottrell et al., 2008). To address
this challenge, prior studies proposed various techniques (Cottrell et al., 2008;
Meng et al., 2011; Yellin and Strom, 1997; Meng et al., 2013; Hua et al., 2015),
such as variable renaming based on the particular context.
Prior studies focus on helping developers locate the source code from online
resources (e.g., Q&A platforms) then integrate that source code into their own
project. However, little is known about how developers utilize these sourcecode from Q&A platforms into their own project in practice. Such knowledge
will help us understand what the potential barriers that prevent developersfrom reusing code from Q&A platforms are. By understanding how developers
utilize code from Q&A platforms, we can get insights to facilitate developers
reusing the source code from Q&A platforms and future research in nextgeneration Q&A platforms.
In this paper, we rst conduct an exploratory study on 95 JavaScript lesfrom 89 projects on GitHub, which contain at least one link to a Stack Overow
post. We manually study each source le and its linked Stack Overow post,
to investigate how developers reuse code from Stack Overow. In particular,
we focus on the following research questions:

RQ1: How often and to what extent do developers need to modify
source code from Stack Overow in order to make it work in theirown projects?

How Do Developers Utilize Source Code from Stack Overow?

3

78.2% of the reused source code is modied from simple refactoring to acomplete reimplementation, which supports the prior studies on automatic
code integration (Feldthaus and Møller, 2013; Alnusair et al., 2016; Wanget al., 2016b). In 24.2% of the cases, developers reimplement code based on
the idea of a Stack Overow answer, which suggests that Q&A platforms
should consider to summarize key points that are discussed in a post to give
developers a quick overview of the question and its answers.

RQ2: From which part of the Stack Overow post does the reused
source code come?
Developers reuse source code from non-accepted answers (26%) for dierent reasons, e.g., the simplicity and performance of the source code. Somedevelopers even adopt answers that are total opposites from what the originalasker wanted but meet their needs. Hence, Q&A platforms should consider toimprove the way of organizing answers, so that developers can nd the mostsuitable answers based on their requirements easily, such as voting on the different aspects (e.g., readability, performance) of answers or adding tags foranswers.
To further understand the barriers that may prevent code reuse and tocollect suggestions for improving the code reuse process on Q&A platforms,
we conduct a survey on 453 developers from Stack Overow. In our survey, we
focus on the following research questions:

RQ3: What are the preferences of developers when it comes toreusing code? Participants reimplement source code slightly more frequently
than reuse source code from Q&A platforms. Code modication according
to their projects, code comprehension, and code quality are the top reasonsthat participants gave regarding why they prefer reimplementing over reusing
source code. Our ndings provide empirical evidence to support current research on code integration and code comprehension, and highlight the need ofproviding code quality indicators on next-generation Q&A platforms.

RQ4: What is the inuence of the code license of Stack Overow? In the exploratory study, we noticed that the number of links to Stack
Overow posts in the source code of open source projects is low, which raises
the question whether developers are aware of the license requirements of Q&A
platforms, since attribution is required by some Q&A platforms such as Stack
Overow. The result shows that 80% of the participants do not have a goodunderstanding of the licenses of the Q&A platforms and most of the participants (57%) think that having more information on license is important.
These ndings suggest that next-generation Q&A platforms should make code
licensing information more visible to developers.

RQ5: How can code reuse be improved in next-generation Q&A
platforms? The suggestions for next-generation Q&A platforms are primarily summarized in ve categories: code quality, information enhancement &management, data organization, license, and human factor. Among code quality, 42.2% of all participants suggest to integrate an online code validator
and 29.7% of the participants suggest to develop a mechanism to deal withoutdated source code.

4

Yuhao Wu et al.
The rest of this paper is organized as follows. Section 2 introduces the

background and motivation of our study. Section 3 describes our data collection
process. Our exploratory study is described in Section 4. Section 5 presents
the survey design and results. Section 6 discusses the implications of our study
on future research for next-generation Q&A platforms. Section 8 describes thethreats to validity. And nally, Section 9 concludes the paper.

2 BackgroundIn this section, we discuss the background of this study.

2.1 Question and Answer on Stack Overow
Nowadays, technical Q&A platforms, with Stack Overow being the mostprominant, have become an important way for researchers and practitioners
to obtain knowledge about and nd solutions to their programming problems(Treude et al., 2011; Abdalkareem et al., 2017). Developers are allowed to post
questions, answer questions, vote questions or answers on the Q&A platforms.
In the questions or answers, developers often attach pieces of source code to
explain their questions or answers. For example, Figure 1 presents an example
of an answer that contains source code in Stack Overow. The asker asked
how to get the HTML of a selected object with jQuery. The answerer posted
an answer that provides a solution in the form of the attached source code.
Stack Overow allows askers to mark at most one answer as the acceptedanswer to indicate whether the answer meets his/her requirement (see Figure 1). Stack Overow allows developers to up vote or down vote a post (e.g.,a question or an answer) to express that whether the post is useful. The total
up and down votes that a post receives is presented as a score beside the post.
For example, the score of the question in Figure 1 is 673. In general, high-voted
answer and question are usually regarded as high-quality one.

2.2 Source Code Reuse from Stack Overow
Source code reuse from Stack Overow can be commonly observed (An et al.,2017). Developers may copy-and-paste source code from Stack Overow posts
into their own projects as long as they adhere to the Creative Commons

1

Attribute-ShareAlike (CC BY-SA 3.0) license , according to an ocial Stack
Overow blog post (Atwood, 2009). One of the requirements of this license isthat attribution is needed from the developers by putting a hyper link to theoriginal Stack Overow post in their source code comments. Figure 2 shows
an example of code reuse. This source code snippet, which is taken from a

2 , is reused from the Stack Overow post shown in Figure 1.

GitHub project1 https://creativecommons.org/licenses/by-sa/3.0/
2 https://goo.gl/X84SFi

How Do Developers Utilize Source Code from Stack Overow?

5Fig. 1: An example of a question and its accepted answer on Stack Overow.

1
2
3
4
5
6
7

// http :// stackoverflow . com / questions /2419749/ get // selected - elements - outer - html
jQuery . fn . outerHTML = function (s ) {return s? this . before (s). remove (): jQuery (" <p >") . append ( this . eq (0) . clone () ). html () ;};Fig. 2: Source code reuse example.

3 Data Collection
The steps of our data collection and analysis processes of the rst part ofour study are shown in Figure 3. First, we collect source les that contain an

explicit reference to a Stack Overow post from searchcode.com (Searchcode,
2016b), a source code search website. Then, we remove irrelevant les suchas false positives and duplicate les. Finally, we analyze the source les andthe corresponding Stack Overow posts to answer our research questions. We
elaborate in more detail on our data collection and analysis processes in the6Yuhao Wu et al.

searchcode.com

Search withkeyword“stackoverflow”

95 JavaScript
files

Manually analyze
the code reusefrom Stack
Overflow

1,059 source
files

Filter out false
positives, small projects
and duplicates

RQ1: How often and to what extent do developers
need to modify source code from Stack Overflow inorder to make it work in their own projects?
RQ2: From which part of the Stack Overflow post
does the reused source code come?

Fig. 3: An overview of our data collection and analysis processes of the exploratory study.

remainder of this section. Note that we explain the second part of our study,i.e., the survey, in Section 5.

3.1 Collecting Source Files from Open Source Project Repositories
As explained in Section 2.2, developers must cite a Stack Overow post when
they reuse code or ideas from that post. Hence, to obtain source les that
contain source code snippets that are reused from Stack Overow posts (either
questions or answers), we search for source les that contain at least one
hyperlink to a Stack Overow post.
To search for such source les, we use searchcode.com (Searchcode, 2016b)

as our search engine. searchcode has indexed over 20 billion lines of sourcecode from 7 million projects. With its API (Searchcode, 2016a), we were able
to collect 1,059 les in total using stackoverow as the search keywords. In
this research, we limit our search results to projects on GitHub, since GitHub
has the largest project population compared to other repositories. We focus
on les that are written in JavaScript since JavaScript is the most popular
language on Stack Overow (Stack Overow, 2016).

3.2 Removing Irrelevant Files
Using stackoverow as the search keyword can result in some source lesthat may contain the stackoverow keyword without containing a link to apost, e.g., in an API name. We manually remove these false positives in thisstep. To mitigate the eects from small projects (Kalliamvakou et al., 2014),
we eliminate les that belong to projects with less than 1,000 commits and 10
contributors. We then remove the les that are duplicates of each other (e.g.,les from forked projects). Finally, we ended up with 95 unique JavaScript les,How Do Developers Utilize Source Code from Stack Overow?

7which belong to 89 projects. Within these les, 115 Stack Overow hyperlinks
were found.

4 Part I: Exploratory StudyIn this section, we present and discuss the results of our exploratory study.
For each research question in our exploratory study we present the motivation,
approach, and results.

RQ1: How often and to what extent do developers need tomodify source code from Stack Overow in order to make it workin their own projects?
4.1

Motivation: Prior research has proposed several ways to utilize the sourcecode on Stack Overow (Rigby and Robillard, 2013; Ponzanelli et al., 2013,2014a,b). For example, Ponzanelli et al. (2013) presented an approach to automatically construct queries from the current context in the IDE (i.e., Eclipse)
and retrieve relevant code and its corresponding discussions from Stack Overow. However, there is no empirical evidence about the process of how developers are reusing the source code, e.g., whether they copy-and-paste the original
source code without any modication or they modify it considerably. Knowing
how developers reuse source code from Q&A platforms will give us insights on
how to make source code reuse easier in next-generation Q&A platforms.

Approach: To understand how developers utilize source code from Stack Overow, we manually analyze the collected JavaScript source code and the linked
Stack Overow posts. We manually extract and categorize the type of code
utilization from Stack Overow posts for each collected source le. We perform
an interactive process similar to Open Coding (Seaman, 1999; Seaman et al.,2008) for identifying the type of the code utilization. This process involves 3
phases and is performed by 3 persons (i.e., P1, P2, P3) who are the rst threeauthors of this study as follows: Phase I: P1 extracts a draft list of types of source code utilization from
Stack Overow based on 20 source les and the linked Stack Overow
post. Then, P1 and P2 use the draft list to categorize the same source lecollaboratively, during which the types are revised and rened. At the endof this phase, we obtain 5 types of source code utilization.

 Phase II: P1 and P2 apply the resulting types of Phase I to independently
categorize all collected source le. They are instructed to take notes regarding the deciency and ambiguity of the types for categorizing certain
source les.

 Phase III: P1, P2, and P3 discuss the coding results obtained in Phase II
to revolve the disagreements until a consensus is reached.

8

Yuhao Wu et al.

Table 1: Types of source code utilization from Stack Overow in JavaScript
les.

ID
C1

Name
Exact Copy

C2

Cosmetic
Modication

C3

Noncosmetic
Modication

C4

Converting
Ideas

C5

Providing
InformationDenition
Developers copy-and-paste the source codefrom Stack Overow without any modication.
Developers copy-and-paste the source codefrom Stack Overow with modications
which do not alter the functionality ofthat source code (e.g., renaming identier
names to make it more readable).
Developers copy-and-paste the source codefrom Stack Overow with modications
which alter the functionality of thesource code (e.g., adding arguments to afunction prototype).
Developers do not copy-and-paste any
source code from Stack Overow. Instead,they write the source code from scratch by
applying the ideas in the answers.
Developers do not reuse any source codefrom Stack Overow. Instead, they treat
the Stack Overow post as an information source related to the issue they are
addressing.

Count
19

Perc.
21.8%

8

9.2%

35

40.2%

21

24.2%

4

4.6%

Table 1 shows nal categorization of the types of source code utilizationfrom Stack Overow in JavaScript les. In our study, one pair could only be
categorized as one type.

Results: 49.4% of the reused source code is modied in one way or
another. Table 1 shows that 21.8% of the studied les reuse source code
without modication (C1). 49.4% (C2 and C3) of the les require modication
when developers reuse source code. Type C4 (24.2%) indicates that it is not
exceptional that developers convert the ideas written in natural language tosource code from scratch. Type C5 (4.6%) indicates that there exist developers
who use Stack Overow as a programming manual.

In 9.2% of the studied les, developers make cosmetic modications when reusing source code, which may improve the readability
or simplicity of the source code. In the Cosmetic Modication category,developers copy-and-paste the source code from a Stack Overow post andmake modications to the source code which may not be necessary to makethe source code work in the target project. In the example shown in Figure 4,
the developers copied two lines of source code in the accepted answer from the

Stack Overow post (see Figure 4b) and renamed the variable name from reto emailRegExp (see Figure 4a).

3 https://goo.gl/9ouSz1
4 https://goo.gl/74oVBu

How Do Developers Utilize Source Code from Stack Overow?
1
2
3
4

9

// http :// stackoverflow . com / questions /46155/ validate // email - address - in - javascript
var emailRegExp = /[ a long regex string ]/;return emailRegExp . test ( value );(a) Source code from the project3 .
1
2

var re = /[ a long regex string ]/;return re . test ( value );(b) Source code from the Stack Overow answer4 .
Fig. 4: C2: An example of code reuse with cosmetic modications.

In 24.2% of the les, developers write the source code fromscratch based on the descriptions of the algorithm. In the example
shown in Figure 5 and Figure 6, developers implement a function to detect
whether a line intersects with the rectangle (see Figure 5), based on an answer
from the Stack Overow post shown in Figure 6.
Another example is shown in Figure 7, where the developers wrote a regular expression that extracts all Youtube video ids in a string (see Figure 7a).
This source code snippet is modied based on the source code from the Stack
Overow post shown in Figure 7b, which is written in PHP. In this example,
developers actually rewrote the regular expression in JavaScript based on the
PHP source code from the Stack Overow post. We categorized this le under the Converting Ideas type since developers cannot reuse the source code
directly from another language, instead, they have to convert the idea andrewrite it from scratch.

Developers use Stack Overow posts in 4.6% of the les as an
information source for later reference. In 4.6% of the les, developers
do not reuse any source code from Stack Overow. Instead, they put a Stack
Overow hyperlink in their source code to provide background information

9 in which the developers

about the issue or solution. For example, there is a lehave tried a solution on Stack Overow which does not work. They then put thelink to the Stack Overow post in the source code with a comment Also trieda method from Stack Overow that caused a security error in all browsers .
In summary, most of the source code reuse (49.4%) requires additionalmodication, which implies that nding the code is only the rst step for code
reuse, more eort is needed to facilitate code reuse from Q&A platforms after
retrieving relevant code from Q&A platforms, such as adjusting the sourcecode to the required context. Prior studies tried to integrate the source code

5 https://goo.gl/4ezMUr
6 https://goo.gl/1Wn9vF
7 https://goo.gl/HK5kyV
8 https://goo.gl/eq1Dnk
9 https://goo.gl/wRoLrn

10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

Yuhao Wu et al.
Rect . prototype . collideLine = function (p1 , p2 ) {var x1 = p1 [0];var y1 = p1 [1];var x2 = p2 [0];var y2 = p2 [1];function linePosition ( point ) {var x = point [0];var y = point [1];return (y2 - y1 )*x + (x1 - x2 )* y + ( x2 *y1 - x1 * y2 );}var relPoses = [[ this . left , this . top ],[ this . left , this . bottom ] ,[ this . right , this . top ],[ this . right , this . bottom ]]. map ( linePosition );var noNegative = true ;var noPositive = true ;var noZero = true ;relPoses . forEach ( function ( relPos ) {if ( relPos > 0) {noPositive = false ;} else if ( relPos < 0) {noNegative = false ;} else if ( relPos === 0) {noZero = false ;}}, this ) ;};if ( ( noNegative || noPositive ) && noZero ) {return false ;}return !(( x1 > this . right && x2 > this . right ) ||( x1 < this . left && x2 < this . left ) ||( y1 < this . top && y2 < this . top ) ||( y1 > this . bottom && y2 > this . bottom ));Fig. 5: C5: An example of converting descriptions into source code: source codefrom the project

5 is implemented based on the description of the algorithm

from Stack Overow (see Figure 6).

to a target context automatically (Feldthaus and Møller, 2013; Alnusair et al.,2016; Wang et al., 2016b). Our ndings also support these existing studies,and it would be interesting to integrate these code integration techniques into
Q&A platforms. In addition, a number of les show that developers write thesource code from scratch based on the description of an algorithm instead of
reusing source code. Such les imply that Q&A platforms should consider toHow Do Developers Utilize Source Code from Stack Overow?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

11

Let the segment endpoints be p1 =( x1 y1 ) and p2 =( x2 y2 ).
Let the rectangle 's corners be ( xBL yBL ) and ( xTR yTR ).
Then all you have to do is
A. Check if all four corners of the rectangle are on thesame side of the line . The implicit equation for a line
through p1 and p2 is :F(x y) = (y2 - y1 )x + ( x1 - x2 )y + ( x2 * y1 - x1 * y2 )If F(x y) = 0, (x y) is ON the line .
If F(x y) > 0, (x y) is " above " the line .
If F(x y) < 0, (x y) is " below " the line .
Substitute all four corners into F (x y ). If they ' re all
negative or all positive , there is no intersection . Ifsome are positive and some negative , go to step B .
B. Project the endpoint onto the x axis , and check if thesegment 's shadow intersects the polygon 's shadow . Repeaton the y axis :If ( x1 > xTR and x2 > xTR ) , no intersection ( line is toright of rectangle ).
If ( x1 < xBL and x2 < xBL ) , no intersection ( line is toleft of rectangle ).
If ( y1 > yTR and y2 > yTR ) , no intersection ( line isabove rectangle ) .
If ( y1 < yBL and y2 < yBL ) , no intersection ( line isbelow rectangle ) .
else , there is an intersection . Do Cohen - Sutherland orwhatever code was mentioned in the other answers toyour question .
You can , of course , do B first , then A.
6

Fig. 6: Description of the algorithm in the Stack Overow answer .

provide a summary of key points that are discussed in a Q&A post to give
developers an overview of the question and its answers.

49.4% of the reused source code requires additional modication, which supports the prior studies on automatic code integration. In 24.2% of the les,developers reimplement code based on an idea, which suggests that Q&A platforms should consider to summarize key points that are discussed in a post to
give developers a quick view of the question and its answers.

12
1
2
3
4
5
6
7
8

Yuhao Wu et al.
YOUTUBE_REGEXP : new RegExp ('(?: https ?://) ?' + // Optional scheme . Either ...
'(?: www \\.) ? '+ // Optional www subdomain'(?: '+ // Group host alternatives' youtu \\. be / '+ // Either youtu .be ,
[...]
') '
// End negative lookahead assertion .
),(a) Source code from the project in JavaScript7 .
1
2
3
4
5
6
7
8
9
10
11
12

// Linkify youtube URLs which are not already links
function linkifyYouTubeURLs ( $text ) {$text = preg_replace ( '~(?#! js YouTubeId Rev :...
# Match non - linked youtube URL in the wild ...
https ?://# Required scheme ...
(?:[0 -9 A -Z -]+\.) ? # Optional subdomain .
(?:# Group host alternatives .
youtu \. be /# Either youtu .be ,[...]$text );
return $text ;}(b) Source code from the Stack Overow answer in PHP8 .
Fig. 7: An example of cross-language source code reuse. Regular expression
strings with the same functionality are highlighted with red.

RQ2: From which part of the Stack Overow post does thereused source code come?
4.2

Motivation: Intuitively, we may expect that accepted or high-voted answers
are the most useful. However, in RQ1 we observed that developers also reuse
source code from answers that were not the highest-voted nor accepted. By
understanding why developers choose the non-accepted or non-highest-voted
answers, we could provide insights to help Q&A platforms organize their answers better so that developers can nd solutions more easily.

Approach: We manually inspect from which part (e.g., accepted answer, nonaccepted answer, or question) of the Stack Overow post the reused sourcecode originates. We also check whether the answer is the highest-voted one.
Two of the authors manually examined each source code le and the linked
post (including the question, all answers, and all comments to the answers)
individually and categorized it. Discrepancies were discussed until a consensus
was reached.

Results: In 26% of the les developers choose the non-accepted an-swers and in 63%, such non-accepted answers are not the highestvoted. The results of the categorization are shown in Table 2. As we can see

How Do Developers Utilize Source Code from Stack Overow?

13

Table 2: Where does the reused source code come from?

SourceAccepted Answer
Non-Accepted Answer
Question
NOT REUSE
Total

Highest-voted
48
11-

Non-highest-voted
2
19
-

Total
50
30
1
34
115

Perc.
43%
26%
1%
30%
100%

Listing 1: Source code in the project that implements a method to generate

10

GUIDs.

1
2
3
4
5
6
7

// http :// stackoverflow . com / questions /105034/ how -to // create -a - guid - uuid -in - javascript
function generateID () {return " avalon "+ Math . random () . toString (36) . substring (2 , 15)+ Math . random () . toString (36) . substring (2 , 15)}from the results, not all source code is reused from accepted answers. In 43% ofthe les, developers choose source code from the accepted answers. However,there are still a considerable number (26%) of les where developers choose thesource code from non-accepted answers. Moreover, among such non-acceptedanswers, 63% are not the highest-voted ones, which indicates that developers
do not always choose source code from the accepted or highest-voted answer.
In the remainder of this section, we discuss dierent situations in which developers reuse source code from non-accepted answers in more detail.

4.2.1 Simpler Source CodeDescription : Developers choose solutions from non-accepted answers because
the source code is simpler.

Example : A developer wants to implement a method to generate GUIDs. The
source code in this example is shown in Listing 1.
This source code snippet is actually from a Stack Overow non-acceptedanswer

12 which has 37 votes, while the accepted answer has 1290 votes. The

source code provided by the accepted answer is shown in Listing 2.
The former source code in Listing 1 is quite straightforward and has higher
readability. According to the description in the answer, this method in Listing 1
is not compliant with the RFC 4122 standard but has a very good performance.
The author of this answer also attached a performance test result of dierent

10 https://goo.gl/Z1pRMS
11 https://goo.gl/xpAcga
12 https://goo.gl/aC4auZ

14Yuhao Wu et al.
11

Listing 2: Source code in the accepted answer on Stack Overow.

1
2
3
4
5
6
7
8
9

function guid () {function s4 () {return Math . floor ((1 + Math . random () )* 0 x10000 ). toString (16). substring (1) ;}return s4 () + s4 () + '-' + s4 () + '-' + s4 ()+ '-' + s4 () + '-' + s4 () + s4 () + s4 () ;}algorithms that are mentioned in other answers of the Stack Overow post,
which shows that this algorithm outperforms others.
Our perception is that the developer who adopted this low-voted answer
prioritize performance and readability more than other metrics such as whether
the generated result is compliant with a standard.

4.2.2 Fixing Bugs

Description : Source code that xes potential bugs of the accepted answer may
also get adopted by developers.

Example : A developer is looking for a method to draw a dashed line aroundselection area in JavaScript

13 .

14 improves the accepted answer by utilizing built-

The non-accepted answer

in transformation functionality of Canvas, and also handles special cases where
the line is vertical, which was not addressed in the accepted answer.

4.2.3 Improving Speed

Description :Answers with higher performance will also attract developers

reusing the source code.

Example : A developer is looking for an algorithm that sorts an array by theLevenshtein Distance in JavaScript. According to the comments under the accepted answer, the implementation in the accepted answer performs better
than the one provided by the original asker. However, a non-accepted answer
provides an improved version of the accepted answer which was described asMost speed was gained by eliminating some array usages , which was reusedby the developers in their project. Thus we believe these developers give performance a higher priority.
Based on our observations, we nd that dierent developers have dierent
requirements for their solutions. Even if answers that are provided in the post

13 https://goo.gl/gzMCgy
14 https://goo.gl/8foVXq

How Do Developers Utilize Source Code from Stack Overow?

15

do not meet the requirements of the asker, other developers may nd them
useful (e.g., solution with higher performance). For developers who are looking
for solutions on Stack Overow, it is better to go through all the answers of a
relevant question instead of focusing on the accepted answers. Q&A platforms
should improve the way of organizing answers, so that developers can ndthe most suitable answers based on their dierent requirements faster. For
example, Q&A platforms may allow users to vote on dierent aspects, such as
the readability or performance of the source code in an answer. The resultsfrom our user survey support the need for this improvement (see Section 6.2).
Developers reuse non-accepted or non-highest-voted answers for dierent reasons, such as the simplicity and performance of the source code. Some evenreuse answers that are total opposites from what the asker wanted. Hence,
Q&A platforms should improve the way of organizing answers, so that developers can nd the most suitable answers based on their requirements easily.

5 Part II: Survey of Code Reuse From Stack Overow
In RQ1, we observed that there are a considerable number of cases in whichdevelopers had to modify the source code before they could reuse it. To further
understand how developers reuse source code from Q&A platforms, we conducted a survey amongst 453 developers. By better understanding the processof code reuse, we can make suggestions to make code reuse more ecient on
next-generation Q&A platforms.

5.1 Research Questions
The goal of our survey is to gain insights on the challenges that developers
face when reusing source code from Q&A platforms. In our survey, we focus
on eliciting information that helps us answer the following research questions:5.1.1 RQ3: Do developers prefer reusing or reimplementing source code?
Intuitively, reusing source code will consume less eort than reimplementing,especially if the source code is well tested. From Section 4, we observed several
cases in which developers preferred reimplementing over reusing. In this partof the survey, we focus on understanding which factors make developers preferreimplementing source code rather than reusing it.

5.1.2 RQ4: What is the inuence of the code license of Stack Overow?
Prior study shows that the amount of code reuse from Q&A platforms (i.e.,
Stack Overow) is low (1.70%) (Abdalkareem et al., 2017). One possible reason is that developers might not be aware of the license requirements of Q&A
platforms, since attribution is required by some Q&A platforms, such as Stack16

Yuhao Wu et al.

Overow. Another possible reason is that some Q&A platforms (e.g., Stack
Overow) have a relatively restrictive license, which might hinder the developers from reusing source code from these Q&A platforms. Hence, in this RQ,
it is interesting to understand whether, in the eyes of developers, license is afactor that they consider as a barrier when reusing code from Q&A platforms.
In addition, previous study has observed potential license violation cases wheredevelopers reused source code from or to Q&A platforms (An et al., 2017).
We are also interested in studying whether having more license information isnecessary in the eyes of the developers.

5.1.3 RQ5: How can code reuse be improved in next generation Q&A
platforms?
The purpose of this part of the survey is to collect suggestions from the developers that can help improve next generation Q&A platforms. The goal ofthis RQ is to dene a roadmap for future research and the implementation of
next-generation Q&A platforms.

5.2 Survey Design
Two of the authors posited the survey questions that cover all the three research questions. The third author checked the questions to eliminate any
ambiguity of the wording of the survey.
The details of the survey are available in the Appendix. The survey is
divided into three parts:
1. Demography (Q1 - Q7): these questions collect the software engineering
background of the participants.
2. Barriers (Q8 - Q17): these questions collect information about the barriers
that the participants are facing in the process of reusing source code from
Q&A platforms. We collected responses from participants who have everreused source code from Q&A platforms (i.e., those who answered yes to
Q7, which is 380 (83.9%) of all the participants).
3. Suggestions (Q18 - Q19): these questions collect suggestions for next generation Q&A platforms. Every participant can answer these two questions
no matter whether they have reused source code from Q&A platforms ornot.

5.3 Data Collection
We use the dataset provided by Vasilescu et al. (2013) to get the candidate
participants. This dataset includes 93,771 email addresses of the intersection ofusers of GitHub and Stack Overow. We took a random sample of 6,000 users
from this dataset and sent them email invitations for our online survey. 1,935
of the emails did not reach the survey candidates because the email addressHow Do Developers Utilize Source Code from Stack Overow?

17

70.0% (317/453)

10+
6−9

17.9% (81/453)

1−512.1% (55/453)
0

200

400

600

Fig. 8: Distribution of the software engineering experience of the participantsin years.

77.7% (352/453)

Industrial
Open Source

73.5% (333/453)

Personal

72.6% (329/453)
20.3% (92/453)

Academic

7.5% (34/453)

Others0200

400Fig. 9: Distribution of the project types of the participants.

does not exist any more. In the end, we received 453 responses which equals a
response rate of 11.1%.
87.9% of the participants are experienced software engineers with morethan 5 years experience, as shown in Figure 8. Industrial, open source, andpersonal projects are the dominant project types that the participants are involved in, followed by academic projects (see Figure 9). Note that a participant
can work on more than one type of project.

5.4 Data Analysis

15 . There are

The responses of the survey are available in the online appendix

12 open-ended questions in the survey where participants can choose to input
their own answers in free text. For each of the question, we use an Open Coding
approach to let the coding schema emerge during the analysis (Glaser, 2017).
We adopt a three phase coding process: Phase I: two of the authors coded the answers of each open-ended question
individually. As a result, each of the two authors had his own set of codes
for the answers. Then, these two authors discussed their draft code schemaand made a revised version of the code schema.

15 http://sel.ist.osaka-u.ac.jp/people/wuyuhao/research-data/so-code-reuse/survey-responses.csv

18

Yuhao Wu et al.
1.1% (4/380)
1.3% (5/380)

Not sure

35.5% (135/380)
52.6% (200/380)
33.2% (126/380)
29.2% (111/380)
25.0% (95/380) Type
15.0% (57/380)
Reimplement
5.3% (20/380)
Reuse
1.8% (7/380)

Less frequent than monthly
Monthly
Weekly
Daily
0

100

200

300

Fig. 10: Comparison of frequency of reusing and reimplementing source code.

 Phase II: the two authors used the revised schema to code the answers.
Then, they discussed and resolved conicts. As a result, a unied coding
schema is developed and applied to all the answers.

 Phase III: the third author veried the developed schema and coding results.

5.5 Results
We answer RQ3 and RQ4 in the remainder of this section and RQ5 in Section 6.

5.5.1 RQ3: Do developers prefer reusing or reimplementing the source code,
why?

Developers reimplement source code slightly more frequently thanreuse source code. Figure 10 shows the comparison of frequency of reimplementing source code and reusing source code from Q&A platforms. The
number of participants who reimplement source code monthly (33.2%) andthose who reuse source code monthly (29.2%) are close, while the dierence
increases to 25.0% vs. 15.0% at a weekly frequency.

The majority of developers (65%) prefer reimplementing thesource code due to the code modication that is required to makethe code from the post work in their project. Table 3 shows the reasonsfor choosing reimplementing over reusing source code. The top reason thatmakes developers reimplement source code is the code modication that is required to make the code from the post work in their own projects. This nding
is consistent with our nding in RQ 1 (i.e., most code needs modication before reusing) and also conrms the need of research in code integration. Codecomprehension ranks as the second most important reason that prevents code
reuse. Prior studies also reported that developers spend 58% of their time on
program comprehension activities (Xin et al., 2017). Hence, more eort is required to improving code comprehension techniques to further facilitate code

How Do Developers Utilize Source Code from Stack Overow?

19

Table 3: Reasons for choosing reimplementing over reusing source code. (Multiselection allowed, hence the sum of the percentages is larger than 100%.)

Category
Context
Comprehension
Quality
Time consuming
Other

Description
The code should be written according to its context.
Do not understand the source code to be reused.
The quality of the source code is too low.
Reusing source code takes more time.
Other reasons.

Perc.
65%
44%
32%
17%
7%

3.7% (14/380)

Others

11.8% (45/380)

No (do not care)

30.5% (116/380)

No (want to know more)

32.9% (125/380)

Yes (a little)

21.1% (80/380)

Yes (fully understand)
0

50

100

150

200

Fig. 11: Participants' awareness of the licenses of Q&A platforms.

reuse from Q&A platforms. 32% of the participants complain about the low
code quality on Stack Overow, which highlights the need for next-generation
Q&A platforms to improve the code quality. 17% of the participants mention
that reusing source code takes more time than reimplementing it, which isagainst the common wisdom. One possible reason is that if the source code is
big or complex, it would take more time to comprehend it or to make it workin a particular context.
Developers reimplement source code slightly more frequently than reuse sourcecode primarily due to the code context, the diculty of code comprehension,and the low quality of source code. Our observations provide empirical evidence
to support current research on code integration and code comprehension, andhighlight the need of improving code quality for next-generation Q&A platforms.

5.5.2 RQ4: What is the inuence of the code license of Stack Overow?

75.3% of the participants do not have a good understanding of thelicense terms of the Q&A platforms, which indicates that there arepotential license violation issues when developers reuse source codefrom Q&A platforms. Figure 11 shows the results of participants' awareness of the licenses of Q&A platforms. An et al. (2017) investigated the code
reuse on Android apps and observed 1,279 potential license violation cases
where developers reused source code from or to Q&A platforms. Our survey
results give a possible explanation for these violations. The Other category

20

Yuhao Wu et al.
17.9% (68/380)

Strongly agree

30.8% (117/380)

Agree39.2% (149/380)

Not sure

8.4% (32/380)

Disagree

3.7% (14/380)

Strongly disagree
0

50

100

150

200

250

Fig. 12: Participants' opinion of license compatibility between Q&A platforms
and their projects.

30.3% (115/380)

Very important

27.1% (103/380)

Important

23.9% (91/380)

Not sure

12.1% (46/380)

Unimportant

6.6% (25/380)

Very unimportant
0

50

100

150

200

Fig. 13: Importance of having more information on license.

in Figure 11 includes cases in which the participants did not give a concreteanswer, e.g., Depends on the platform. Stack Overow is attribution-required,

but the requirements of most other sites are vague or not generally known.

51.3% of the participants disagree on or are not sure whether
the license of Q&A platforms is compatible with their own projects,
which indicates that such participants might be prevented fromreusing code from Q&A platforms (see Figure 12). We actually observed some participants mentioned this when they were asked why they preferreimplementing over reusing source code. For example, one participant mentioned that licensing is sometimes an issue. .

Most of the participants (57.4%) think that having more information about the code license is essential (see Figure 13). This echoeswith the result that most participants do not have a good understanding of
the code license of Q&A platforms, and reveals the needs from the participantsthat Q&A platforms should make their code license clearer.

How Do Developers Utilize Source Code from Stack Overow?

21

Table 4: The categorization of code quality suggestions  64 (37.6%).

Category
Integrated validator

Outdated code

Answer quality

Code review

Description (D)  Example (E)D: Integrated validator that can test the code snippets
on Q&A platforms.
E: An inbuilt REPL environment for as many languages/environments as possible.

D: Answers (including source code) on Q&A platformssuer from out-of-date problems. Participants are seeking for a solution to this problem.
E: make date important in marking outdated code,and deprecate those snippets via the community

D: Classier that helps distinguish high and low quality
answers.
E: Better support for answers that are good, but outof date.

D: Integrated code review tool that helps improve thecode quality.
E: In-browser code review and commenting similar to

Perc.
42.2%

29.7%

17.2%

10.9%

that provided by commercial code review tools.

Total

100.0%

Generally speaking, participants do not have a good understanding of the code
license on Q&A platforms. More than half of them believe that or are notsure of the license compatibility issue exists between their projects and Q&A
platforms. Most of the participants think that they need more informationof the code license on Q&A platforms. Based on the observations, the nextgeneration Q&A platforms should have clearer license and make it more visible
to developers.

6 A Roadmap for Next-Generation Q&A platforms
In this section, we summarize the results for RQ5. In total, we collected 150
responses for Q19 in the survey. 22 of these responses were not actually suggestions for next-generation Q&A platforms (e.g., Not much, quite happy

with Stack Overow. ) and were excluded from the following analysis. Each
response can contain multiple suggestions and we extracted 170 suggestions
from these responses. Using the approach described in Section 5.4, the suggestions were then categorized into ve major categories: code quality, information enhancement & management, data organization, license, and humanfactors. We highlight the ndings and discuss implications on future researchof next-generation Q&A platforms of each category.

6.1 Code Quality

Code quality is the most popular type of suggestion (37.6%) fromthe participants. From Table 4, we observe that 37.6% of the participants

22

Yuhao Wu et al.

provide suggestions on improving the code quality on Q&A platforms. The
two most important suggestions from developers on improving code quality
are adding an integrated validator (42.2%) that can test source code online
and an outdated code detection mechanism (29.7%) that can identify outdated
code.
An integrated validator is a convenient way of testing source code snippets
online to ensure the quality of the source code. Participants describe such toolfor example as follows: The ability to interact with and run the code examples

written in answers and questions .
Some Q&A platforms have started to integrate an online validator into theirwebsites. For example, Stack Overow supports three web-creating languages
for online validation: HTML, CSS, and JavaScript (Stack Overow, 2014).
However, Stack Overow does not support online validator of other languages,
such as Java and C++, which are also very popular on Stack Overow. There
are several challenges to extend the validator to all languages.
One of the biggest challenges is to make an incomplete code snippet run
correctly, since the code snippets on Q&A platforms are usually not complete
as the answerer only needs to implement the core part of a solution and may
leave out necessary context information (e.g., version). Prior studies have proposed several approaches to extend incomplete code snippets (not limited tothose on Q&A platforms) into compilable ones based on program analysis andmachine learning techniques (Wang et al., 2016a; Nguyen et al., 2012; Raychev

et al., 2014). However, none of these approaches can guarantee the correctnessof the extended code. For example, there is a function called foo in a codesnippet. To make this code snippet compilable, Q&A platforms need to infer
where this function comes from and then import the corresponding library.
However, prior approaches cannot automatically infer the exact library only
based on the source code. This problem may be solvable in Q&A platforms
by leveraging the description that comes with the source code. Hence, future
research should investigate whether the description of the source code can beused to improve the correctness of the code extension.
Providing an outdated code detection mechanism is the second top suggestions under the code quality category from the participants. Many of theparticipants mention that the source code on Q&A platforms is often outdated and not suitable for current technologies or situations. For example,
one participant suggests: Have explicit mechanisms for dealing with content

that goes out of date due to platform or language changes. Some participantssuggest a mechanism that claries the API version of the source code: Clear

associates between the code snippets the versions of the API under which it
will work. This is particularly when working with APIs that change frequently,
like iOS and Unity. Some also suggest deprecating the outdated answers:
Make date important in marking outdated code, and deprecate those snippets
via the community. This problem was recognized by various developers on
Stack Overow (Krumia, 2014) and received wide attention from the Stack
Overow communities.

How Do Developers Utilize Source Code from Stack Overow?

23

Table 5: The categorization of Information enhancement & management suggestions  43 (25.3%).

Category
Answer tagging

Code evolution

Resources linking
Answer writing support
OthersDescription (D)  Example (E)D: Better tagging-like information system for answers.
E: Provide/require tagging of the version number(s) of the language [...]

D: Better management of the evolution/revisionsof code snippets.
E: where does the code come from and copied to,
and also the revisions inside the platform.

D: Q&A platforms should suggest for other resources (e.g., books, API documents, libraries etc.)
E: Books suggestions based on questions.
D: Support for writing better questions/answers.
E: [...] it would be nice if it would be easier to ask
a good question [...]

D: Other aspects of information enhancement &management.
E: Built in support within an IDE to make it

Perc.
37.2%

14.0%

11.6%
9.3%
27.9%

faster to get the answer you are interested in.

Total

100.0%

However, as we know, no existing study investigates the outdated sourcecode or solutions on Q&A platforms so far. Hence, there is a need for future research to put more eort on developing mechanisms to deal with the outdated
source code or solutions. There are two primary directions to deal with theoutdated code or solutions. First, future studies should propose approaches to
identify the outdated solution in Q&A platforms using machine learning techniques automatically. Second, future research should develop certain incentive
systems to motivate communities to identify and update such outdated sourcecode or solutions.

6.2 Information Enhancement & Management

Information enhancement & management (25.3%) is the second
most popular suggestion for developers on Q&A platforms. 37.2%of the participants suggest to add tagging-like information for answers (seeTable 5). These participants suggest a mechanism (e.g., adding tagging-like
information) to identify special answers such as those using a specic versionof an API or those which pass a specic test. For example, participants suggest that Provide/require tagging of the version number(s) of the language orother context that the questions and answers are written for. and Answers
which pass the tests could be marked with a special icon and given priority.
Based on the observations, future studies should focus on recommending
the tagging-like information for answers. These recommendations could be24

Yuhao Wu et al.

Table 6: The categorization of data organization suggestions  21 (12.4%).

Category
Code searching/indexing

Description (D)  Example (E)D: Support for easier code search.
E: Source Code indexing for easier retrieval. It

Perc.
47.6%

could also give the possibility to nd example of

Duplicate posts

Comments

usage functions.

D: An automatic way of clustering duplicate questions/answers.
E: Auto-suggest similar questions, particularly
for questions that don't have answers.

D: Support on utilizing the comments of posts.
E: Code in *comments* must be expressed better,

38.1%

14.3%

than on Stack Overow.

Total

100.0%

made automatically using, e.g., machine learning techniques (Wang et al.,2017a, 2014b; Zhou et al., 2017), or aspect-mining techniques (Wong et al.,2008; Wang et al., 2010; Yu et al., 2011; Liu et al., 2015; Zhao and Li, 2009).

6.3 Data Organization

12.4% of the participants suggest Q&A platforms to better organize their data, including a better searching/indexing mechanism
and better duplicate detection. As shown in Table 6, some participants(47.6%) suggest that Q&A platforms should have a better way to index andsearch code. For example, participants mentioned: Source Code indexing foreasier retrieval. It could also give the possibility to nd example of usage functions. and Ability to search questions based on the version of the frameworkor language I'm working with . 38.1% of the participants suggest that Q&A
platforms should have an automatic way to detect duplicate or similar post andbe organized in a better way. Others (14.3%) suggest to have better support
on utilizing the comments in posts.
Prior studies have studied develop code search engines to help developers
to improve their search eciency on source code (Wang et al., 2014a; McMillan

et al., 2011; Lv et al., 2015; Bajracharya et al., 2006; Searchcode, 2016b). Ourndings support these studies, and it would be interesting to integrate thesecode search engines into Q&A platforms.
Prior studies proposed various approaches to help Q&A platforms detect
duplicate questions automatically (Zhang et al., 2015; Ahasanuzzaman et al.,2016; Wang et al., 2017b; Zhang et al., 2017). Our ndings support these studies. Moreover, deep learning has proven its power on capturing the semantic
meaning from natural language in many prior studies (Ganguly et al., 2015;
Lai et al., 2015; Bian et al., 2014; Chen et al., 2016). The common way to identify the duplicate questions is to measure such questions' similarity in terms

How Do Developers Utilize Source Code from Stack Overow?

25

Table 7: The categorization of code license suggestions  23 (13.5%).

Category
Clearer license

Description (D)  Example (E)D: Q&A platforms should make their license terms
clearer.
E: By far the most important requirement is clear li-

Perc.
69.6%

censing. Much of the code provided on such platforms is

Permissive license

not currently usable because the license is unclear.

D: Q&A platforms should use a more permissive license.
E: Let the user choose a more re-user-friendly license

30.4%(e.g. copy without reference).

Total

100.0%of semantic meaning. Hence, future research could consider to employ deeplearning to detect duplicate questions or nd similar questions.

6.4 Code License

13.5% of the participants suggest to improve license-related issues,in particular to make the license more clear (69.6%). Table 7 shows
the suggestions about license. This trend echoes with our previous result that
75.3% of the participants do not have a good understanding of the license termsof the Q&A platforms (see Section 5.5.2). Participants request that the Q&A
platforms should provide a clear explanation of their license terms: By far

the most important requirement is clear licensing. Much of the code provided
on such platforms is not currently usable because the license is unclear. Ifdevelopers neglect the license of the Q&A platforms and reuse source codefrom these platforms, they are under the risk of license violation which may
cause them problems later.
30.4% of the participants suggest that Q&A platforms should use a more
permissive license which has less restrictions on source code reuse. In the example of Stack Overow, CC BY-SA 3.0 was the original license for the sourcecode on this platform. CC BY-SA 3.0 is a copyleft (non-permissive) license
which requires the derivative work be licensed under the same license (CC
BY-SA 3.0). This means that when developers reuse the source code from
Stack Overow into their projects, they have to license these projects under
CC BY-SA 3.0 as well. Otherwise, they are under the risk of license violation.
It is also worth noting that, although Stack Overow has announced thislicense change in the post on Stack Exchange (Stack Exchange, 2015), thecommunity did not reect this change on their homepage (Stack Overow,2017), where it still writes user contributions licensed under cc by-sa 3.0 withattribution required . This mismatch will further deepen developers' misunderstanding of the license terms, hence we suggest that next-generation Q&A
platforms should explicitly describe their license terms of source code reuse in
a consistent manner.

26

Yuhao Wu et al.
Table 8: The categorization of human factor suggestions  19 (11.2%).

Category
Better curator

Gamication related

Description (D)  Example (E)D: Better curators are needed to help improve thequality of the posts.
E: Denitely curators for specic languages to rate
answers in specic areas.

D: Suggestions on improving the gamication systemof the Q&A platforms.
E: Base reputation on number of answers up-voted

Perc.
63.2%

36.8%

by others, not on personal activity.

Total

100.0%

6.5 Human factor

11.2% of the participants suggest to improve Q&A platforms interms of Human factor. Table 8 shows that 63.2% of the participants suggest to have better curators to improve the quality of posts and help withmarking good answers. One participant suggested: Pay some vetted, expe-

rienced developers to check the answers, instead of relying on gamication.
Another suggestion also emphasized on the collaboration from the community:

Arriving at a 'most correct' solution should be a more collaborative eort witha clearly shown path of how it was arrived at by multiple people, not necessarily
just one user who takes all the credit.
36.8% of the participants suggest to improve the gamication system of
Q&A platforms. The usage of gamication on Q&A platforms has been proveneective before (Anderson et al., 2013; Cavusoglu et al., 2015). However, participants revealed some aws in this system. For example, one participant
wrote: Base reputation on number of answers up-voted by others, not on per-sonal activity. (Stack Overow has too many nit-pickers gaining reputation
by down-voting legitimate questions.) This suggestion echoes with anotherparticipant's comments: Gamication sometimes creates wrong incentives.
Our ndings suggest that future studies are necessary on how to improve
the gamication mechanism of Q&A platforms.

7 Related WorkIn this section we discuss prior research that are related to our study.

7.1 Leveraging the Knowledge from Q&A Platforms
Q&A platforms accumulate a large amount of knowledge from the communication among developers, which can be used to assist software development
activities. Thus, numerous studies about how to leverage the knowledge from
Q&A platforms have been done. Treude and Robillard (2016) presented aHow Do Developers Utilize Source Code from Stack Overow?

27

novel machine learning based approach, named SISE, to augment API documentation using the answers on Stack Overow. Wong et al. (2013) leveraged
questions and answers on Stack Overow to automatically generate comments
in system source code. Vassallo et al. (2014) extracted discussions from Stack
Overow and used the extracted data to generate JavaDoc. Gao et al. (2015)
proposed an automated approach to x recurring crash bugs by leveraging information (e.g., questions with similar crash traces) on Q&A platforms. Azadet al. (2017) proposed an approach to extract API call rules from version
history and Stack Overow Posts. Chen et al. (2017) proposed an automatic
approach to build a thesaurus that contains morphological forms of software
engineer terms.
Several studied the attached source code on Stack Overow. Sillito et al.
(2012) performed an empirical study on factors that make a good source code
example on Stack Overow. They found that the source code explaining important elements and presenting a solution step by step makes a good example.
Treude et al. (2011) performed a study on Stack Overow to explore whichquestions are answered well and which ones remain unanswered. They foundthat the code is an important factor for code review questions to get angood answer. Abdalkareem et al. (2017) performed a study on Stack Overow
to explore what developers use the crowd for. They found that development
tools and programming language issues are areas where the crowd is the mosthelpful.
Dierent from above studies which focus on developing a tool to deliver
knowledge from Q&A platforms to developers, our study focuses on understanding how developers utilize the source code from Q&A platforms.

7.2 Source Code Reuse
Rigby and Robillard (2013) proposed a novel approach to extract code elements from various documents such as Stack Overow. They evaluated theirapproach on 188 Stack Overow answer posts containing 993 code elements.
The technique achieved an average 0.92 precision and 0.90 recall. Ponzanelli

et al. (2013) proposed an Eclipse plugin named

Seahawk that helps develop-

ers search and import code snippets from Stack Overow. Then they proposeda Eclipse plugin named Prompter which automatically searches and identies
Stack Overow discussions, evaluates their relevance based on the given thecode context in the IDE, and noties the developer if a user-dened condence threshold is surpassed (Ponzanelli et al., 2014a,b). Armaly and McMillan (2016) presented a novel reuse technique that allows programmers to reuse
functions from a C or C++ program, by recording the state of the dependencies during one program's execution, and replaying them in the context of adierent program.
Dierent from prior work which focus on proposing approaches to retrieve
code for developers to reuse, we study how developers utilize the source code

28

Yuhao Wu et al.

from Q&A platforms. We also performed a user survey to understand what
factors prevent developers to reuse the source code from Q&A platforms.

7.3 Code Licensing
Baltes et al. (2017) studied whether developers attribute (e.g., reference) theoriginal Stack Overow post when they reuse the code snippets from these
posts. They found that 3.22% of all the analyzed repositories and 7.33% of
the popular ones on GitHub contained a reference to Stack Overow. However, for Java, at least one third of the copied snippets were not attributed.
An et al. (2017) studied whether developers respect license restrictions whenreusing source code from Stack Overow in Android apps or vice versa. Witha case study on 399 Android apps, they found 232 code snippets in 62 Android
apps were potentially reused from Stack Overow, while 1,226 Stack Overow
posts contained code in 68 Android apps. 1,279 potential license violations
were observed. Almeida et al. (2017) performed a survey among developers
on the open source license and found that developers struggle when multiple
licenses were involved. The results indicate a need for tool support to help
guide developers in understanding this critical information of license that is
attached to software components.
Dierent from previous studies on code license, we focus on studying whether
the license has impact on source code reuse from Q&A platforms for developers and we nd it is a potentially important factor that prevents code reusefrom Q&A platforms.

8 Threats to Validity
External validity. Threats to external validity relates to the generalizabilityof our ndings. In this research, we use searchcode.com as our search engine
and nd 1,059 source les in total that contain links to Stack Overow posts.
After removing the small projects and duplicate les, there are 95 JavaScriptles left. The number of les may not be large enough to represent all thecases in the real world. However, we think the number is large enough for our
exploratory study to nd out the issues underlying the process of code reusefrom Q&A platforms.
We focused the rst part of our study on JavaScript language, and our
result may not be generalizable to other languages. Future work is necessary
to investigate whether our ndings hold for other languages.

Internal validity. Threats to internal validity relates to the experimenter
bias and errors. In this research, we manually inspect the source code in the
projects and in Stack Overow posts. We also manually categorize each caseof how developers are reusing the source code. Dierent people may havedierent opinions on the categorization. To alleviate this threat, two of ourauthors conduct the manual analysis and any discrepancy is discussed till

How Do Developers Utilize Source Code from Stack Overow?

29

a consensus is achieved. When analysing the results of user survey, we also
manually categorized the open questions in survey. To alleviate the bias due
to human factors, we employ the open coding process.

9 Conclusion
Prior studies focus on developing approaches to help developers reuse andintegrate source code in a required context automatically. However, little tonothing is known about how developers utilize the code from Q&A platforms.
In this paper, we study how developers reuse code from Q&A platforms
and the barriers that prevent developers from reusing source code from Q&A
platforms.
We rst conduct an exploratory study on 95 JavaScript les from 89
projects which contain source code that contains an explicit reference to a
Stack Overow post with the expectation of nding how developers utilize
Q&A platforms. Based on the result of this exploratory study, we conduct asurvey on 453 developers on Stack Overow to further understand the barriers
that may prevent code reuse and to obtain suggestions for the next-generation
Q&A platforms. The most important ndings of our study are:
1. The exploratory study shows that 78.2% of the reused source code from
Stack Overow is modied from simple refactoring to a complete reimplementation.
2. 26% of the developers also reuse source code from non-accepted answers.
3. More developers prefer reimplementing source code over reusing sourcecode because of dierent code context, low code quality and diculty ofcode comprehension.
4. Code quality, information enhancement & management, data organization,code license and human factor are the most popular types of suggestions
for next-generation Q&A platforms from the survey participants.
Future research eorts can be devoted to a next-generation Q&A platform
based on the suggestions summarized in this work.

References
Abdalkareem, R., Shihab, E., and Rilling, J. (2017). What do developers use
the crowd for? a study using Stack Overow. IEEE Software ,34(2), 5360.

Ahasanuzzaman, M., Asaduzzaman, M., Roy, C. K., and Schneider, K. A.
(2016).

Mining duplicate questions in Stack Overow.

In Proceedings ofthe 13th International Conference on Mining Software Repositories (MSR) ,pages 402412.
Almeida, D. A., Murphy, G. C., Wilson, G., and Hoye, M. (2017). Do softwaredevelopers understand open source licenses?

In Proceedings of the 25th

International Conference on Program Comprehension (ICPC) , pages 111.
IEEE.

30

Yuhao Wu et al.

Alnusair, A., Rawashdeh, M., Hossain, M. A., and Alhamid, M. F. (2016). Utilizing semantic techniques for automatic code reuse in software repositories.
In Quality Software Through Reuse and Integration , pages 4262. Springer.
An, L., Mlouki, O., Khomh, F., and Antoniol, G. (2017). Stack Overow: A
code laundering platform? In Proceedings of the 24th IEEE International

Conference on Software Analysis, Evolution, and Reengineering (SANER) ,pages 283293. IEEE.
Anderson, A., Huttenlocher, D., Kleinberg, J., and Leskovec, J. (2013). Steering user behavior with badges.

In Proceedings of the 22nd International

Conference on World Wide Web (WWW) , pages 95106. ACM.
Armaly, A. and McMillan, C. (2016). Pragmatic source code reuse via execution record and replay. Journal of Software: Evolution and Process ,28(8),

642664.

Attribution required  Stack Overow blog. https://stackoverflow.blog/2009/06/25/attribution-required/. (last visited:

Atwood, J. (2009).
Aug 25, 2017).

Azad, S., Rigby, P. C., and Guerrouj, L. (2017). Generating API call rules fromversion history and stack overow posts.

ACM Transactions on Software

Engineering and Methodology (TOSEM) , 25(4), 29.

Bajracharya, S., Ngo, T., Linstead, E., Dou, Y., Rigor, P., Baldi, P., andLopes, C. (2006).

Sourcerer: A search engine for open source code sup-porting structure-based search. In Companion to the 21st ACM SIGPLAN

Symposium on Object-oriented Programming Systems, Languages, and Applications (OOPSLA) , pages 681682. ACM.
Baltes, S., Kiefer, R., and Diehl, S. (2017). Attribution required: Stack Overow code snippets in GitHub projects.

In Proceedings of the 39th Inter-national Conference on Software Engineering Companion , pages 161163.
IEEE.
Bian, J., Gao, B., and Liu, T.-Y. (2014). Knowledge-Powered Deep Learning

for Word Embedding , pages 132148. Springer Berlin Heidelberg.
Cavusoglu, H., Li, Z., and Huang, K.-W. (2015). Can gamication motivate
voluntary contributions?: The case of StackOverow Q&A community. In

Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing , pages 171174. ACM.
Chen, C., Gao, S., and Xing, Z. (2016). Mining analogical libraries in Q&A
discussions - incorporating relational and categorical knowledge into wordembedding. In IEEE 23rd International Conference on Software Analysis,

Evolution, and Reengineering (SANER) , pages 338348. IEEE.
Chen, C., Xing, Z., and Wang, X. (2017). Unsupervised software-specic morphological forms inference from informal discussions. In Proceedings of the39th International Conference on Software Engineering (ICSE) , pages 450
461. IEEE.
Cottrell, R., Walker, R. J., and Denzinger, J. (2008). Semi-automating smallscale source code reuse via structural correspondence. In Proceedings of the

16th ACM SIGSOFT International Symposium on Foundations of Software
Engineering (SIGSOFT) , pages 214225. ACM.

How Do Developers Utilize Source Code from Stack Overow?

31

Feldthaus, A. and Møller, A. (2013). Semi-automatic rename refactoring forjavascript. In Proceedings of the 2013 ACM SIGPLAN International Confer-ence On Object Oriented Programming Systems Languages & Applications ,volume 48, pages 323338. ACM.
Gamma, E., Helm, R., Johnson, R., and Vlissides, J. (1995). Design Patterns:

Elements of Reusable Object-oriented Software . Addison-Wesley Longman
Publishing Co., Inc., Boston, MA, USA.
Ganguly, D., Roy, D., Mitra, M., and Jones, G. J. (2015). Word embedding
based generalized language model for information retrieval. In Proceedingsof the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) , pages 795798.
Gao, Q., Zhang, H., Wang, J., Xiong, Y., Zhang, L., and Mei, H. (2015). Fixing
recurring crash bugs via analyzing Q&A sites. In Proceedings of the 30th

International Conference on Automated Software Engineering (ASE) , pages
307318.
Glaser, B. (2017).

Discovery of grounded theory: Strategies for qualitative

research . Routledge.
Hua, L., Kim, M., and McKinley, K. S. (2015). Does automated refactoring
obviate systematic editing? In IEEE/ACM 37th IEEE International Con-

ference on Software Engineering (ICSE) , volume 1, pages 392402. IEEE.
Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D. M., andDamian, D. (2014).

The promises and perils of mining GitHub.

In Pro-ceedings of the 11th Working Conference on Mining Software Repositories(MSR) , pages 92101. ACM.
Krumia(2014).

Introduceanobsoleteanswer"https://meta.stackoverflow.com/questions/272651/
introduce-an-obsolete-answer-vote. (last visited: Aug 25, 2017).

vote.

Lai, S., Xu, L., Liu, K., and Zhao, J. (2015). Recurrent convolutional neuralnetworks for text classication. In Proceedings of the 29th AAAI Conferenceon Articial Intelligence , pages 22672273. AAAI Press.
Liu, P., Joty, S. R., and Meng, H. M. (2015).

Fine-grained opinion mining

with recurrent neural networks and word embeddings.

In Proceedings ofthe 2015 Conference on Empirical Methods in Natural Language Processing(EMNLP) , pages 14331443. The Association for Computational Linguistics.
Lv, F., Zhang, H., Lou, J.-g., Wang, S., Zhang, D., and Zhao, J. (2015).
CodeHow: Eective code search based on API understanding and extended
boolean model. In Proceedings of the 30th IEEE/ACM International Con-

ference on Automated Software Engineering (ASE) , pages 260270. IEEE.
McMillan, C., Grechanik, M., Poshyvanyk, D., Xie, Q., and Fu, C. (2011).
Portfolio: Finding relevant functions and their usage. In Proceedings of the

33rd International Conference on Software Engineering (ICSE) , pages 111120.
Meng, N., Kim, M., and McKinley, K. S. (2011).

Systematic editing: Gen-erating program transformations from an example.

In Proceedings of the

32nd ACM SIGPLAN Conference on Programming Language Design and32

Yuhao Wu et al.
Implementation (PLDI) , pages 329342.

Meng, N., Kim, M., and McKinley, K. S. (2013). Lase: locating and applying
In Proceedings of the 2013

systematic edits by learning from examples.

International Conference on Software Engineering , pages 502511. IEEE.
Nguyen, A. T., Nguyen, T. T., Nguyen, H. A., Tamrawi, A., Nguyen, H. V.,
Al-Kofahi, J., and Nguyen, T. N. (2012).

Graph-based pattern-oriented,

context-sensitive source code completion. In Proceedings of the 34th Inter-national Conference on Software Engineering (ICSE) , pages 6979.
Ponzanelli, L., Bacchelli, A., and Lanza, M. (2013). Leveraging crowd knowledge for software comprehension and development. In Proceedings of the 17th

European Conference on Software Maintenance and Reengineering (CSMR) ,pages 5766. IEEE.
Ponzanelli, L., Bavota, G., Di Penta, M., Oliveto, R., and Lanza, M. (2014a).
Mining stackoverow to turn the IDE into a self-condent programming
prompter. In Proceedings of the 11th Working Conference on Mining Soft-ware Repositories , pages 102111. ACM.
Ponzanelli, L., Bavota, G., Di Penta, M., Oliveto, R., and Lanza, M. (2014b).
Prompter: A self-condent recommender system. In ICSME , pages 577580.
Raychev, V., Vechev, M., and Yahav, E. (2014). Code completion with statistical language models. In Proceedings of the 35th ACM SIGPLAN Conference

on Programming Language Design and Implementation (PLDI) , pages 419
428.
Rigby, P. C. and Robillard, M. P. (2013). Discovering essential code elements
in informal documentation. In Proceedings of the 2013 International Con-

ference on Software Engineering (ICSE) , pages 832841. IEEE.
Seaman, C. B. (1999). Qualitative methods in empirical studies of software
engineering.

IEEE Transactions on Software Engineering (TSE) , 25(4),557572.
Seaman, C. B., Shull, F., Regardie, M., Elbert, D., Feldmann, R. L., Guo,Y., and Godfrey, S. (2008). Defect categorization: making use of a decadeof widely varying historical data. In Proceedings of the Second ACM-IEEE

international symposium on Empirical software engineering and measurement , pages 149157. ACM.

Searchcode (2016a). searchcode - API. https://searchcode.com/api/. (lastvisited: Aug 25, 2017).
Searchcode (2016b).

searchcode - Homepage.

https://searchcode.com/.

(last visited: Aug 25, 2017).
Sillito, J., Maurer, F., Nasehi, S. M., and Burns, C. (2012). What makes agood code example?: A study of programming Q&A in StackOverow. In

Proceedings of the 2012 IEEE International Conference on Software Maintenance (ICSM) , pages 2534.
Stack Exchange (2015).

The MIT license  clarity on using code on

Stack Overow and Stack Exchange. https://meta.stackexchange.com/

q/271080/337948. (last visited: Aug 25, 2017).

Stack Exchange (2017). All sites - Stack Exchange. https://stackexchange.

com/sites. (last visited: Aug 25, 2017).

How Do Developers Utilize Source Code from Stack Overow?
Stack

Overow(2014).

Feedback

33
requested:

Runnablehttps://meta.stackoverflow.com/questions/269753/
feedback-requested-runnable-code-snippets-in-questions-and-answers.
codesnippetsinquestionsandanswers.

(last visited: Aug 25, 2017).

http://stackoverflow.com/research/developer-survey-2016. (last visited:

Stack Overow (2016). Stack Overow developer survey results 2016.
Aug 25, 2017).
Stack

Overow(2017).

Stack

Overow-

Homepage.

stackoverflow.com/. (last visited: Aug 25, 2017).

Treude, C. and Robillard, M. P. (2016).

https://

Augmenting API documentation

with insights from Stack Overow. In Proceedings of the 38th International

Conference on Software Engineering (ICSE) , pages 392403. ACM.
Treude, C., Barzilay, O., and Storey, M.-A. (2011). How do programmers ask
and answer questions on the web? (NIER track). In Proceedings of the 33rd

International Conference on Software Engineering (ICSE) , pages 804807.
Vasilescu, B., Filkov, V., and Serebrenik, A. (2013).

StackOverow and

GitHub: Associations between software development and crowdsourced
knowledge. In Proceedings of 2013 International Conference on Social Com-puting (SocialCom) , pages 188195. IEEE.
Vassallo, C., Panichella, S., Di Penta, M., and Canfora, G. (2014). CODES:Mining source code descriptions from developers discussions. In Proceedingsof the 22nd International Conference on Program Comprehension (ICPC) ,pages 106109.
Wang, H., Lu, Y., and Zhai, C. (2010).

Latent aspect rating analysis onreview text data: A rating regression approach. In Proceedings of the 16th

ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining (KDD) , pages 783792.
Wang, S., Lo, D., and Jiang, L. (2014a).

Active code search: Incorporatinguser feedback to improve code search relevance. In Proceedings of the 29th

ACM/IEEE International Conference on Automated Software Engineering(ASE) , pages 677682.
Wang, S., Lo, D., Vasilescu, B., and Serebrenik, A. (2014b). EnTagRec: Anenhanced tag recommendation system for software information sites.

In

Proceedings of the 2014 IEEE International Conference on Software Maintenance and Evolution (ICSME) , pages 291300.
Wang, S., Lo, D., and Jiang, L. (2016a). Autoquery: automatic constructionof dependency queries for code search.

23(3), 393425.

Automated Software Engineering ,Wang, S., Lo, D., Vasilescu, B., and Serebrenik, A. (2017a). EnTagRec ++:An enhanced tag recommendation system for software information sites.

Empirical Software Engineering .
Wang, Y., Feng, Y., Martins, R., Kaushik, A., Dillig, I., and Reiss, S. P.
(2016b). Hunter: next-generation code reuse for java. In Proceedings of the24th ACM SIGSOFT International Symposium on Foundations of Software
Engineering , pages 10281032. ACM.

34

Yuhao Wu et al.

Wang, Z., Hamza, W., and Florian, R. (2017b).

Bilateral multi-perspective

matching for natural language sentences. CoRR ,abs/1702.03814.

Wong, E., Yang, J., and Tan, L. (2013). Autocomment: Mining question andanswer sites for automatic comment generation.

In IEEE/ACM 28th In-

ternational Conference on Automated Software Engineering (ASE) , pages
562567. IEEE.
Wong, T.-L., Lam, W., and Wong, T.-S. (2008). An unsupervised framework
for extracting and normalizing product attributes from multiple web sites.
In Proceedings of the 31st Annual International ACM SIGIR Conference on

Research and Development in Information Retrieval (SIGIR) , pages 3542.
Xin, X., Lingfeng, B., David, L., Zhenchang, X., Ahmed, E. H., and Shanping,
L. (2017). Measuring program comprehension: A large-scale eld study withprofessionals. IEEE Transactions on Software Engineering (TSE) ,Yellin, D. M. and Strom, R. E. (1997).

99(26).

Protocol specications and compo-

nent adaptors. ACM Transactions on Programming Languages and Systems(TOPLAS) , 19(2), 292333.

Yu, J., Zha, Z.-J., Wang, M., and Chua, T.-S. (2011). Aspect ranking: Identifying important product aspects from online consumer reviews. In Proceedingsof the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1 , pages 14961505.
Zhang, W. E., Sheng, Q. Z., Lau, J. H., and Abebe, E. (2017).

Detecting

duplicate posts in programming qa communities via latent semantics andassociation rules.

In Proceedings of the 26th International Conference on

World Wide Web (WWW) , pages 12211229.
Zhang, Y., Lo, D., Xia, X., and Sun, J.-L. (2015).
question detection in Stack Overow.

Technology , 30(5), 981997.

Multi-factor duplicate

Journal of Computer Science andZhao, L. and Li, C. (2009). Ontology Based Opinion Mining for Movie Reviews ,pages 204214. Springer Berlin Heidelberg, Berlin, Heidelberg.
Zhou, P., Liu, J., Yang, Z., and Zhou, G. (2017). Scalable tag recommendation for software information sites. In Proceedings of the 24th International

Conference on Software Analysis, Evolution and Reengineering (SANER) ,pages 272282. IEEE.

Appendix
Below are the questions and options in our online survey. Single-selection options are marked
with circle marks (◦) in front; multi-selection options are marked with box marks (2) infront. When participants choose the option Other , they are allowed to input a free text as
an additional answer.
Part I
1. How many years of software engineering experience do you have?
◦1
◦2
◦3
◦4
◦6
◦7
◦8
◦9
2. What type of project(s) are you working on?

◦5
◦ 10+

How Do Developers Utilize Source Code from Stack Overow?
2 Open source

35

2 Personal
2 Industrial
2 Academic
2 Other
3. Which programming language(s) do you use in your projects?
2 Java2 C/C++2 C#
2 Python
2 Other
4. How often do you use Q&A platforms?
◦ Every day◦ Once a week
◦ Once a month◦ Once every few months or less
◦ Never
5. What do you use Q&A platforms for?
2 Learning new techniques/methodologies
2 Refreshing the knowledge of old techniques/methodologies
2 Solving a specic programming issue
2 Finding references that I can refer to in my source code to make future maintenanceeasier2 Answering questions
2 Other
6. Which Q&A platforms do you use to look for solutions to programming-related issues?
2 Stack Overow
2 Quora2 Product-specic support forums
2 Language-specic support forums
2 I do not use Q&A platforms for this purpose
2 Other
7. Have you ever reused source code from a Q&A platform?
◦ Yes
◦ No
Part II
8. How often do you *reuse* source code from Q&A platforms?
◦ Every day◦ Once a week
◦ Once a month◦ Once every few months or less
◦ Never
9. How often do you *reimplement* source code from Q&A platforms?
◦ Every day◦ Once a week
◦ Once a month◦ Once every few months or less
◦ Never
10. If you prefer reimplementing the source code over reusing existing code, why?
2 Code should be written in relation to the context.
2 I don't want to reuse source code that I don't fully comprehend.
2 The quality of the existing source code is too low.
2 Re-implementing the source code takes less time than reusing.
2 Other
11. What do you consider the most important factors when deciding when to reuse codefrom a Q&A platform?
2 Correctness (i.e., bug-free)
2 Performance (i.e., ecient)
2 Readability (i.e., easy to read/understand)
2 Simplicity (i.e., less lines of code)
2 Compatibility (e.g, support more platforms)
2 Whether the answer is accepted by the questioner.
2 Whether the answer has the highest number of upvotes.
2 Other
12. Which aspects cause you diculty when reusing source code from Q&A platforms?
2 Syntax errors need to be xed to make the source code runnable.
2 Bugs (e.g., index out of bounds) need to be xed.
2 Readability needs to be improved.
2 Performance needs to be improved.
2 The code snippet is not in the programming language I need.
2 Code needs to be adapted to my specic use case.
2 The license terms of the Q&A platform are unclear.
2 Other
13. Do you always refer to the Q&A platform post from which you reused source code inyour documentation or code comments? Why (not)?

36

Yuhao Wu et al.

◦ Yes, I add a link to the post/answer to show my respects/appreciation to the original

author.
◦ Yes, because it is required by the license terms of that Q&A platform (e.g., CC-BY-SA
3.0 in the case of Stack Overow).
◦ Yes, to make it easier for future maintenance.
◦ No, I would like to, but always forget.
◦ No, I don't do that. (Please elaborate the reason below if you could)◦ Other
14. Are you aware of the license terms of reused source code from a Q&A platform?
◦ Yes, I fully understand the license terms.
◦ Yes, I know about the existence of such terms, but I am not sure what obligations I have.
◦ No, I did not know about them, but I would like to learn more about them.
◦ No, I did not know about them, neither do I care about them.
◦ Other
15. Which license(s) do the projects into which you reused code from a Q&A platform use?
2 GPL family (any version of LGPL, GPL or AGPL) 2 MIT License
2 Apache License
2 BSD License
2 A proprietary license
2 No license
2 I don't know
2 Other
16. In general, would you say that the license(s) of these project(s) are compatible withthe license of the Q&A platform from which you reused source code?
◦ Strongly disagree◦ Disagree◦ Neutral◦ Strongly agree◦ Agree
17. How important is it to have more detailed information about the license terms andlegal obligations of reusing source code from Q&A platforms?
◦ Very unimportant◦ Unimportant◦ Neutral◦ Very important◦ Important
Part III
18. How useful would it be to let other users tag answers on Q&A platforms with labelsthat describe the source code in an answer as 'performant', 'correct', 'readable', etc.?
◦ Very unuseful◦ Unuseful◦ Neutral◦ Very useful◦ Useful
19. If you could give any suggestions (regardless of whether they are feasible) for anext-generation code Q&A platform, what would you suggest?