# Winning of Hearts and Minds: Integrating Sentiment Analytics into the Analysis of Contradictions

Jennifer Ferreira, Denis Dennehy, Jaganath Babu, Kieran Conboy

► To cite this version:

Jennifer Ferreira, Denis Dennehy, Jaganath Babu, Kieran Conboy. Winning of Hearts and Minds: Integrating Sentiment Analytics into the Analysis of Contradictions. 18th Conference on e-Business, e-Services and e-Society (I3E), Sep 2019, Trondheim, Norway. pp.392-403, 10.1007/978-3-030-29374-1_32. hal-02510159

HAL Id: hal-02510159

https://inria.hal.science/hal-02510159

Submitted on 17 Mar 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Winning of Hearts and Minds: Integrating Sentiment Analytics into the Analysis of Contradictions

Jennifer Ferreira^ Denis Dennehy^ Jaganath Babu^ Kieran Conboy^

^ Lero | Irish Software Research Centre, National University of Ireland Galway, Ireland

**Abstract.** Interactions in open source communities are often informal, and enacted through online discussion forums. While discussion and associated sentiment is critical to sustaining open source communities, they have not been studied to date. To address this gap in knowledge, this study uses sentiment analytics to illuminate the frequency of 2,364 discursive manifestations of contradictions through the theoretical lens of Activi- ty Theory (AT). The study contributes to current discourse on contradictions by demon- strating the importance of dialectical contradictions as a driving force for learning, change, and sustaining open source communities. Implications for research and practice provide opportunities for revising current business methods and practices, which inevi- tably have implications for a sustainable society in the 21st century.

**Keywords:** Activity Theory, Contradictions, Sentiment Analytics, Open Source

## 1 Introduction

OSS<fnref n="1" /> development is a knowledge-intensive activity that involves software develop- ers, who are usually geographically dispersed, using online forums to coordinate their work activities [1-3]. These online forums are communication channels where soft- ware developers express their emotions concerning their degree of satisfaction [4] concerning a specific piece of software code (known as a patch) that is peer reviewed. Peer review is an important quality assurance mechanism in the OSS community but is less well understood when compared to other aspects of OSS development [5].

As the online forums facilitate peer reviews and interactions between members of the open source community, it offers a rich source of insights into community practic- es and social norms [3]. Previous research on online forums focused on discovering knowledge sharing practices [6], information seeking behaviours among developers [7], identifying active contributors [8], and the sentiment of members within the community [4, 9-12]. Research has shown that sentiment affect quality, productivity, creativity, group rapport, and job satisfaction [13]. Understanding the sentiment of software developers is important for project managers as it provides a better under- standing of the social factors that affect the project and the corrective actions required to improve sentiment [4, 5].

<footnote n="1">¹ OSS is a type of computer software in which source code is released under a license in which the copyright holder grants users the rights to study, change, and distribute the software to anyone and for any purpose (Laurent, AMS, 2004).</footnote>
OSS development is also a highly collaborative activity [2], requiring creativity and problem-solving skills, which are influenced by emotion [14]. Further, the sustainability of open source communities requires software developers to maintain healthy relationships with their peers in order to ensure their input and support [15]. It would therefore seem logical that the sentiment of project members plays an important role in the success or failure of a project, however project managers find it difficult to keep track of their people's feelings [1].

As OSS projects are notoriously subject to contradictions (i.e. tensions, conflict, breakdown in communication), we use Activity Theory (AT) to examine contradictions because AT anticipates this [16]. Contradictions are “historically accumulating structural tensions within and between activity systems” and are a fundamental concept in AT [17, p. 137]. The identification of contradictions helps practitioners to focus their efforts on the root causes of problems. This collaborative analysis can lead to the creation of a shared vision for the solution of the contradictions [18]. [19] propose four distinct types of contradictions which they associate with discursive manifestations, namely, (i) double binds, (ii) conflicts, (iii) critical conflicts, and (iv) dilemmas. In this manner, discursive manifestations can be associated with a type of contradiction and with its resolution.

We argue that a greater scrutiny of discursive manifestations is necessary in the study of open source communities for three key reasons.

First, by illuminating discursive manifestations of contradictions rich insights into the social norms and practices of open source communities will be revealed. This is important as organisations in the 21st century play an active role in shaping the structure and direction of open source communities [20].

Second, there is a noticeable absence of research that progress from simply applying sentiment analytics [1, 4, 5] to advancing the accumulative body of knowledge via theoretical development. This lack of cumulative tradition [21, 22] resonates with the issue of ‘fragmented adhocracy’, which has previously overshadowed IS research [23-25]. By grounding the study in AT, we theorise how sentiment analytics can be used to provide a deeper understanding contradictions.

Third, in the context of online forums that are used by open source communities, [26] makes a call for a serious expansion of our understanding of organisations, work, and learning. This study answers this call, by examining sentiment in the context of collaborative work.

Using AT as the theoretical lens is pertinent in this study for three key reasons, namely (i) understanding context in which the words are used is important as it strongly influences accuracy [27, 28] and AT is oriented at understanding the activity in context [29]. AT acknowledges contradictions as a means of understanding and
change [17, 30], a concept that is not explicit in other social theories [31]. Hence, we
make the claim that it is more useful to integrate sentiment analytics with the analysis
of discursive manifestations. In doing so, rich insights into how emotions permeate
work and contradictions, that influence how people work on daily basis is revealed.
Therefore, through the lens of AT the overarching aim of this study is to

"Explore how sentiment analytics can illuminate discursive manifestations of con-
tradictions in the context of open source communities".

The paper is structured as follows. First, a review of literature on contradictions
from the perspective of AT is presented. Next, the method used to extract and clean
data for the purpose of analysis is outlined. Then, key findings and analysis is pre-
sented. Followed by discussion and implications for practice, academia, and society.
The paper ends with conclusions, limitations and future action.

2 Activity Theory

Contemporary thinking on AT, known as third-generation AT emerged from the sem-
inal work of [32] who acknowledges the systemic relations between an individual and
their environment, by highlighting the influential nature and interrelatedness of the
larger social context.

A fundamental concept of AT is the notion of contradictions, which occur within
an activity and/or between multiple interrelated activities and promote dialectical
transformation [17, 33]. While the term ‘contradiction’ may be considered by some as
a weakness, from the perspective of AT, they are a sign of richness and an opportunity
to develop in the activity system [33, 34]. Contradictions are seen as the sources of
learning and can become the driving force for change and development in a system, if
they are addressed [16]. Essentially contradictions are ‘motors of change’ [35]. Con-
tradictions can occur either inside the key constructs (e.g. community) or between
them, or they may occur in networks of activity systems [17, 36]. Contradictions can
be identified through their manifestations, which include, disturbances, errors, prob-
lems, rupture of communication, breakdowns, and clashes [17, 37, 38]. However,
contradictions may not be obvious, openly discussed, or be culturally or politically
challenging to confront [35, 39]. Researchers must therefore rely on indirect methods
to make visible the contradictions and to explain the genesis of their development
[40].

More recently, discursive manifestations of contradictions in organisational change
efforts have been studied [19, 40]. Table 1 lists four distinct types of contradictions
that [19] associate with discursive manifestations and its resolutions.
<table><caption>Table 1. Types of discursive manifestations of contradictions</caption><thead><tr><td>Manifestation</td><td>Features</td><td>Linguistic Cues</td></tr></thead><tbody><tr><td>Double bind</td><td>Facing pressing and equally unacceptable alternatives in an activity system:<br/>Resolution: practical transformation (going beyond words)</td><td>“We”, “us”, “we must”, “we have to” pressing rhetorical questions, expressions of helplessness</td></tr><tr><td>Critical conflict</td><td>Facing contradictory motives in social interaction, feeling violated or guilty<br/>Resolution: finding new personal sense and negotiating a new meaning</td><td>Personal, emotional, moral accounts narrative structure, vivid metaphors “I now realise that . . .”</td></tr><tr><td>Conflict</td><td>Arguing, criticising<br/>Resolution: finding a compromise, submitting to authority or majority</td><td>“No”, “I disagree”, “this is not true”, “this I can’t accept”</td></tr><tr><td>Dilemma</td><td>Expression or exchange of incompatible evaluations<br/>Resolution: denial, reformulation</td><td>“On the one hand [. . .] on the other hand”; “yes, but” “I didn’t mean that”, “I actually meant”</td></tr></tbody></table>

*Double bind* is typically expressed “first by means of rhetorical questions indicating a cul-de-sac, a pressing need to do something and, at the same time, a perceived impossibility of action” [19]. Occurs when a person or group engages in interactions that raise paradoxical and contradictory demands, which make it difficult to step back from their current activities, and consequently create feelings of helplessness. A double bind is typically a situation which cannot be resolved by an individual alone [19]. Resolution requires making practical changes that are transformative and collective actions that go beyond words but is often accompanied with expressions such as “let us do that”, “we will make it” [19, 40].

*Critical conflict* are situations ‘in which people face inner doubts that paralyse them in front of contradictory motives unsolvable by the subject alone’ [19, p. 374]. These critical conflicts are very emotionally and morally charged, which makes it difficult, or even impossible, for them to be resolved solely by the subjects involved (ibid). The discourse is also marked by vivid metaphors [40]. Resolution occurs ‘via a renegotiation of meaning for the subject who was accompanied by the collective in order to allow the former to gain critical distance from their experience and to give it new meaning’ (ibid, p.282).

*Conflict* takes the form of resistance, disagreement, argument and criticism, and occurs “when an individual or a group feels negatively affected by another individual or group, i.e. because of a perceived divergence of interests, or because of another’s incompatible behaviour” [41, p. 1]. [19] observed that people engaged in a conflict tend to argue and to criticise each other. Conflicts are resolved through compromise or submitting to authority or the majority [40].

*Dilemma* is an ‘expression or exchange of incompatible evaluations, either between people or within the discourse of a single person’ and is most often expressed in the form of hesitations, such as “yes, but” [19]. It is typically reproduced rather
than resolved, often with the help of denial or reformulation (i.e. I didn't mean that).

## 3 Methodology

This section outlines the process we used to analyse sentiment and discursive manifestations pertaining to discussions via the DPDK<fnref n="2" /> community platform between 28th Feb and 4th May 2018. As sentiment analysis tools require customisation for the context of software development [42-44] we customised two popular sentiment analysis dictionaries – ‘Opinion Lexicon’ and ‘Comparative Words’. To analyse the sentiment in the message body content, we followed a similar approach to [9] where the message body is split into tokens and using a rule-based algorithm in combination with two dictionaries, assigned a positive, neutral, or negative score. The assigned sentiment scores ranged from ‘Strong negative’ (-20), Weak negative (-10), Neutral (0), Positive (+10), and Strong positive (+20). A token is assigned a score according to the matching word found in the dictionaries and the overall sentiment of a message was computed as the sum of all scores assigned to the tokens contained in that message. The research method consists of three inter-related phases, namely, (i) data extraction, (ii) data pre-processing, and (iii) data analysis.

**Phase 1 Data Extraction:** Comprised of extracting messages from the dpdk-dev mailing list archived at http://mails.dpdk.org/archives/dev/. A total of 13,461 messages were extracted in RAR file format.

**Phase 2 Data Pre-processing:** Executed using Python scripts, messages were converted from RAR file format into CSV file format and messages dated outside the release cycle removed. This resulted in 8,585 messages being included in this study. The message content was cleaned for analysis using regular expressions to ensure that only the message body and natural language remained. All message headers, code, file paths, and non-alphanumeric symbols/characters were removed. This activity was critical to reduce any instances of misclassification [1]. The remaining text was then converted into DataFrame format (tabular data structure in Python) for compatibility purposes with the sentiment analysis algorithm.

**Phase 3 Data Analysis:** As domain-specific terms influence sentiment analysis [52], the research team collaborated with members of the open source community to refine the dictionaries and data in an iterative manner. The natural language dictionary was augmented with domain-specific language of the open source community to include the following terms, ‘NIT’ (e.g. OK but a small problem), ‘NACK’ (e.g. Not accepted by the community), and ‘LGTM’ (e.g. Looks good to me). Also, as noted by [19], their categorisation of manifestations is not exhaustive. Therefore, the linguistic cues unique to the open source community studied are included in the analysis of

<footnote n="2">² The main features of the DPDK review process include, (i) hosting software code in a public repository, (ii) a mailing list where registered members ‘submit’ code, (iii) code is reviewed publicly on the mailing list, and (iv), successfully reviewed code is merged into the main repository for scheduled releases.</footnote>
discursive contradictions, namely, 'NIT' (e.g. Dilemma), and 'NACK' (e.g. Critical conflict). These findings are presented in the next section.

## 4 Findings and Analysis

We investigate sentiment around ‘nack’ and analyse the underlying discursive manifestations of contradictions, these are generally viewed by the community as wasted time and effort (i) of the developer who developed the patch, and (ii) of the community members who review the patch.

**Sentiment analysis:** Fig. 1 illustrates the sentiment score plotted against time, during which activities (e.g. scoping, pre-merge code, bug fix, test, and release) are completed as part of the release cycle. The red bars are the dates that 15 ‘nacks’ occurred during the release cycle - 5 in March, 8 in April, and 2 in May.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Sentiment score plotted against time</figcaption></figure>

Analysis of the sentiment reveals that the overall sentiment is minimally positive (0.210). A number of positive and negative outliers are present at the start and end of release cycle. The underlying reason for these is that initially a patch will have errors/defects but following a series of reviews and revisions, the quality of the patch improves, as does sentiment of the community. As overall sentiment is minimally positive, these findings challenge the assumptions of the community that messages containing ‘nack’ should have strong negative sentiment. This indicates that the ‘nack’ messages can also contain positive sentiment that can have a neutralising effect on the overall sentiment score.

This finding is supported by the distribution of sentiment scores represented in Fig. 2 below. The sentiment score distribution that is normally distributed and the mode is zero. This indicates that the majority of discussions were neutral due to the technical nature of the conversations for each review.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Frequency distribution of sentiment scores</figcaption></figure>

Table 2 provides a summary of the statistics for the 18.05 release cycle. These findings support the previous analysis such as the ‘mean’ progressing from -0.12 in Feb to +0.21 in April.

<table><caption>**Table 2.** Summary statistics of release cycle</caption><thead><tr><th></th><th>Feb</th><th>Mar</th><th>Apr</th><th>May</th><th>Release Cycle</th></tr></thead><tbody><tr><td>Number of messages</td><td>85</td><td>2884</td><td>5224</td><td>392</td><td>8,585</td></tr><tr><td>Mean sentiment score</td><td>-0.12</td><td>0.34</td><td>0.15</td><td>0.11</td><td>0.21</td></tr><tr><td>Standard deviation</td><td>2.98</td><td>2.56</td><td>2.56</td><td>2.38</td><td>2.55</td></tr><tr><td>% of messages with a sentiment score within 1 standard deviation</td><td>89%</td><td>86%</td><td>87%</td><td>83%</td><td>86%</td></tr></tbody></table>

To further investigate the underlying sentiment of ‘nack’ messages, the discursive manifestations of contradictions are analysed.

**Analysis of contradictions:** Table 3 below shows that a ‘nack’ can manifest as different types of contradictions – critical conflict, conflict, and dilemma – indicating that there are subtle differences around instances of ‘nack’ that require further investigation. For example, in the following excerpt from an email message (2nd Mar), “*The proposed patch is a workaround that doesn't address the underlying issue, thus NACK unless proven otherwise :)*” we start to understand why sentiment around ‘nack’ are not strongly negative. Firstly, a smiley emoji at the end of the sentence indicates that the author is not adversarial with this comment. Secondly, the author rejects the patch, but leaves it to the community to prove that this patch is still useful for solving the “underlying issue”, which implies this is a conditional ‘nack’ and the author is willing to retract it. In another excerpt (12th Apr), “*So, as it is, it's a NACK from me, but let's work together on something better:)*” a positive sentiment is displayed by the author who encourages the community to work towards a better solution, despite the rejection of the patch.
<table><caption>**Table 3.** Discursive manifestations of contradictions</caption><thead><tr><th>Manifestation</th><th>Examples in context of this study</th><th>Frequency</th></tr></thead><tbody><tr><td>Double bind</td><td>"We must guarantee"<br>"We must allocate"<br>"We can work around that"<br>"We need to know how"<br>"We must consider a solution"<br>"We must send comments"</td><td>1,380</td></tr><tr><td>Critical conflict</td><td>"I am sorry; I have to NACK because the change is not explained"<br>"I can't agree with this statement"<br>"I'm very unhappy about the..."<br>"No, we must use..."</td><td>933</td></tr><tr><td>Conflict</td><td>"I disagree with this final assessment"<br>"I bet your teacher would disagree with that statement with one single paragraph in your book reports - taste is hard to debate, but you have gone the extreme route with only the bare minimum blank lines and that is not good"<br>"Looks like you assume that the device is always plugged in while the DPDK application starts, this is not true"</td><td>10</td></tr><tr><td>Dilemma</td><td>"I don't like it. It's a NACK from me, but let's work together on something better"<br>"Two nits I think we could add a note"<br>"Self-nack on this patch"<br>"On second thoughts, self-nack"<br>"Does it mean a NACK?"<br>"We would like 2 or 3 more days on this before we can 'ACK' 'NACK' this patch"<br>"Yes, but I have already..."<br>"NACK, I am looking into it"</td><td>41</td></tr><tr><td></td><td>Total</td><td>2,364</td></tr></tbody></table>

## 5 Discussion and Implications

The study revealed that although ‘nack’ is considered by the community to be extremely negative, 7 cases of ‘self-nack’ occurred. Rather than categorise ‘self-nack’ as a critical conflict manifestation, in the context of this study it is categorised as a ‘dilemma’. The reasoning for this is that a person who contemplates a ‘self-nack’ is faced with the dilemma of being ridiculed or rewarded by their peers, depending on when and why the ‘self-nack’ is initiated. The analysis of sentiment and contradictions collectively challenge the assumptions of the open source community, namely, that the community is overly negative due to the online platform that is used to communicate feedback on patch reviews, and that all instances of ‘nack’ are really negative and considered ‘bad’. Further, from the perspective of AT, our analysis highlighted that events that are perceived as “bad” are indeed opportunities for innovation, improved dialogue within the community, and better collaboration between all stake-
holders of the open source ecosystem. Also, rather than view a 'nack' as a waste of
time, resources, and finances, it can be used as an opportunity to create events
(on/offline) that can build cohesion in the open source community and contribute to
the overall health and sustainability of the community.

The findings from this study have important implications for software development research in academia, industry, and the wider society.

*Implications for industry:* First, understanding the pattern of communication is important because it provides an opportunity for management and project teams to stabilise the flow of work and patch reviews during the various activities (i.e. scoping) of a release cycle. Second, sentiment and contradictions provides insight into the emotional states of software developers and holds much promise for better management of people involved in software development projects in general. Third, it is a strategic advantage for organisations involved in open source projects to understand the circumstances of a 'nack' in order for corrective action to be taken.

*Implications for academia:*

First, as all data analysis tools have limitations, researchers need to not only assess the suitability of such tools for their research project, but also need to carefully understand the social context of the research in order to draw meaningful and actionable insights that enable organisational change. A second implication, which is related to the first, is that sentiment analysis, by itself, does not provide rich contextual data to drive organisational change (i.e. at project level). Supplementing this approach with a robust theoretical framework such as AT provides researchers with the opportunity to analyse and conceptualise complex real-world situations where the interrelationship between communities of people (open source community), mediating tools (online forum), and a cultural-historical setting co-evolve (new members join or leave the open community). Third, analysing the natural language used in the mailing list, from the perspective of discursive manifestations provides rich insights into the internal dynamics of online communities, which we know are not well understood [c.f. 6].

*Implications for a sustainable society:* As social sustainability is a key dimension of sustainability [45], the role of big data and analytics can have positive and negative implications for society [46]. Remote working is recognised as a key strategy for a sustainable society as it reduces travel, which in turn reduces carbon emissions. The tools used in this study have a meaningful role to play in enabling sustainable work practices as part of a larger suite of technologies that enable and support distributed work. Combining sentiment analysis with analysis of contradictions are useful indicators of the social well-being of individuals and teams, as well as maintaining the social structure of communities [47]. For example, these indicators can provide companies with opportunities to develop interventions that improve the quality of life and well being of its employees and their families, which in turn would reduce health care costs, as prevention is better than a cure [46].
# 6 Conclusion, Limitations, and Future Action

Obtaining accurate sentiment from mailing lists remains a key challenge [2] but can be mitigated by customising sentiment analysis tools for the context of the study [42]. The research demonstrates that it is feasible to extract and analyse data from mailing lists with high accuracy. We presented sentiment analysis as a mechanism for extracting (i) sentiment expressed in mailing list patch review comments, and (ii) the four types of discursive manifestations and their frequency during the release cycle. While a limitation of the study is that one release cycle is not representative of the DPDK online community, the study does present opportunities for future work in order to gain a deeper understanding of the relationship between discursive manifestations of contradictions and sentiment, as well as the propensity of individual reviewers over time. Future work will indeed focus on multiple release cycles during a full year and/or compare sentiment across multiple projects. This study highlights the importance of not only considering sentiment as quantitative values but to take into consideration the context of the sentiment values and how discourse can directly and indirectly have a positive or negative impact on people within the activity system.

# Acknowledgements

This work was supported with the financial support of the Science Foundation Ireland grant 13/RC/2094 and co-funded under the European Regional Development Fund through the Southern & Eastern Regional Operational Programme to Lero | The Irish Software Research Centre (www.lero.ie).

# References

1. Tourani, P., Y. Jiang, and B. Adams, Monitoring sentiment in open source mailing lists: exploratory study on the apache ecosystem. In Proceedings of 24th Annual International Conference on Computer Science and Software Engineering (pp. 34-44). IBM Corp.. 2014.

2. Mistrík, I., et al., Collaborative software engineering: challenges and prospects. In Collaborative Software Engineering (pp. 389-403). Springer, Berlin, Heidelberg. 2010.

3. Bird, C., et al., Mining email social networks. In Proceedings of the 2006 international workshop on Mining software repositories (pp. 137-143). ACM. 2006.

4. Guzman, E., D. Azócar, and Y. Li, Sentiment analysis of commit comments in GitHub: an empirical study. In Proceedings of the 11th Working Conference on Mining Software Repositories (pp. 352-355). ACM. 2014.

5. Rigby, P.C., D.M. German, and M.A. Storey, Open source software peer review practices: a case study of the apache server. In Proceedings of the 30th international conference on Software engineering (pp. 541-550). ACM. 2008.

6. Sowe, S.K., I. Stamelos, and L. Angelis, Understanding knowledge sharing activities in free/open source software projects: An empirical study. Journal of Systems and Software, 81(3), pp.431-446. 2008.

7. Sharif, K.Y., et al., An empirically-based characterization and quantification of information seeking through mailing lists during open source developers' software evolution. Information and Software Technology, 57, pp.77-94. 2015.
8. Guzzi, A., et al., Communication in open source software development mailing lists. In Proceedings of the 10th Working Conference on Mining Software Repositories (pp. 277-286). IEEE Press. 2013.

9. Rousinopoulos, A., G. Robles, and J. González-Barahona, Sentiment Analysis Of Free / Open Source Developers: Preliminary Findings From a Case Study. Electronic Journal Of Information Systems, 13 (2), P.1. 2014.

10. Pletea, D., B. Vasilescu, and A. Serebrenik, Security and emotion: sentiment analysis of security discussions on GitHub. In Proceedings of the 11th working conference on mining software repositories (pp. 348-351). ACM. 2014.

11. Paul, R., A. Bosu, and K.Z. Sultana, Expressions of Sentiments During Code Reviews: Male vs. Female. arXiv preprint arXiv:1812.05560. 2018.

12. Sinha, V., A. Lazar, and B. Sharif, Analyzing developer sentiment in commit logs. In Proceedings of the 13th International Conference on Mining Software Repositories (pp. 520-523). ACM. 2016.

13. De Choudhury, M. and S. Counts, Understanding affect in the workplace via social media. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (pp. 303-316). ACM. 2013.

14. Carver, J., et al., Gender, sentiment and emotions, and safety-critical systems. IEEE Software, 35(6), pp.16-19. 2018.

15. Ozer, M. and D. Vogel, Contextualized relationship between knowledge sharing and performance in software development. Journal of Management Information Systems, 2015. 32(2): p. 134-161.

16. Hasan, H. and S. Banna, The unit of analysis in IS theory: The case for activity. Information Systems Foundations, 191. 2012.

17. Engeström, Y., Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of education and work, 2001. 14(1): p. 133-156.

18. Engestrom, Y., Activity theory as a framework for analyzing and redesigning work. Ergonomics, 43(7), pp.960-974. 2000.

19. Engeström, Y. and A. Sannino, Discursive manifestations of contradictions in organizational change efforts: A methodological framework. Journal of Organizational Change Management, 24(3), pp.368-387. 2011.

20. Germonprez, M. and M. Levy, Is it Egalitarianism or Enterprise Strategy? Exploring a New Method of Innovation in Open Source. 2015.

21. Metcalfe, M., Theory: Seeking a plain English explanation. JITTA: Journal of Information Technology Theory and Application, 2004. 6(2): p. 13.

22. Weick, K.E., Theory construction as disciplined imagination. Academy of management review, 1989. 14(4): p. 516-531.

23. Fitzgerald, B. and F. Adam, The status of the IS field: historical perspective and practical orientation. 2000.

24. Banville, C. and M. Landry, Can the Field of MIS be Disciplined? Communications of the ACM, 1989. 32(1): p. 48-60.

25. Hirschheim, R., H.K. Klein, and K. Lyytinen, Exploring the intellectual structures of information systems development: a social action theoretic analysis. Accounting, Management and Information Technologies, 1996. 6(1): p. 1-64.

26. Engeström, Y. and H. Kerosuo, From workplace learning to inter-organizational learning and back: the contribution of activity theory. Journal of Workplace Learning, 19(6), pp.336-342. 2007.

27. Aue, A. and M. Gamon, Customizing sentiment classifiers to new domains: A case study. In Proceedings of recent advances in natural language processing (RANLP) (Vol. 1, No. 3.1, pp. 2-1). 2005.

28. Turney, P.D., Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th annual meeting on
association for computational linguistics (pp. 417-424). *Association for Computational Linguistics*. 2002.

29. Cole, M. and Y. Engeström, *A cultural-historical approach to distributed cognition. Distributed cognitions: Psychological and educational considerations*, pp.1-46. 1993.

30. Ilyenkov, E.V., *Activity and knowledge. Philosophy and culture*. 1974.

31. Karanasios, S. and D. Allen, *Mobile technology in mobile work: contradictions and congruencies in activity systems*. *European Journal of Information Systems*, 23(5), pp.529-542. 2014.

32. Engstrom, Y., *Learning by expanding*. Helsinki: Orienta-Konsultit Oy, 1987.

33. Karanasios, S., K. Riisla, and B. Simeonova, *Exploring the use of contradictions in activity theory studies: An interdisciplinary review*. 2017.

34. Karanasios, S., *Toward a unified view of technology and activity: The contribution of activity theory to information systems research*. *Information Technology & People*, 31(1), pp.134-155. 2018.

35. Allen, D.K., et al., *How Should Technology-Mediated Organizational Change Be Explained? A Comparison of the Contributions of Critical Realism and Activity Theory*. *Mis Quarterly*, 2013. 37(3): p. 835-854.

36. White, L., K. Burger, and M. Yearworth, *Understanding behaviour in problem structuring methods interventions with activity theory*. *European Journal of Operational Research*, 2016. 249(3): p. 983-1004.

37. Helle, M., *Disturbances and contradictions as tools for understanding work in the newsroom*. *Scandinavian Journal of Information Systems*, 2000. 12(1): p. 7.

38. Kuutti, K., *Activity theory as a potential framework for human-computer interaction research*. *Context and consciousness: Activity theory and human-computer interaction*, 1995. MIT Press: p. 17-44.

39. Capper, P. and B. Williams, *Enhancing evaluation using systems concepts*. *American Evaluation Association*, 2004.

40. Dionne, P. and S. Bourdon, *Contradictions as the driving force of collective and subjective development group employment programmes*. *Journal of Education and Work*, pp.1-14. 2018.

41. De Dreu, C.K.W. and E. Van De Vliert, *Introduction: Using conflict in organizations*. 1997.

42. Lin, B., et al., *Sentiment analysis for software engineering: How far can we go?*. In *2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE)* (pp. 94-104). IEEE, 2018.

43. Novielli, N., D. Girardi, and F. Lanubile, *A Benchmark Study on Sentiment Analysis for Software Engineering Research*. In *Proceedings of the 15th International Conference on Mining Software Repositories* (pp. 364-375). New York, NY, USA: ACM. https://doi.org/10.1145/3196398.3196403. 2018.

44. Jongeling, R., S. Datta, and A. Serebrenik, *Choosing your weapons: On sentiment analysis tools for software engineering research*. In *2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)* (pp. 531-535). https://doi.org/10.1109/ICSM.2015.7332508. 2015.

45. Moos, M., J. Andrey, and L.C. Johnson, *The sustainability of telework: an ecological-footprinting approach*. *Sustainability: Science, Practice and Policy*, 2(1), pp.3-14. 2006.

46. Pappas, I.O., et al., *Big data and business analytics ecosystems: paving the way towards digital transformation and sustainable societies*. 2018, Springer.

47. Rogers, D.S., et al., *A vision for human well-being: transition to social sustainability*. *Current Opinion in Environmental Sustainability*, 4(1), pp.61-73. 2012.