# Usability Innovations in OSS Development - Examining User Innovations in an OSS Usability Discussion Forum

Netta Iivari

► To cite this version:

Netta Iivari. Usability Innovations in OSS Development - Examining User Innovations in an OSS Usability Discussion Forum. 6th International IFIP WG 2.13 Conference on Open Source Systems, (OSS), May 2010, Notre Dame, United States. pp.119-129, 10.1007/978-3-642-13244-5_10. hal-01056019

HAL Id: hal-01056019

https://inria.hal.science/hal-01056019

Submitted on 14 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Usability Innovations in OSS Development
– Examining User Innovations in an OSS
Usability Discussion Forum

Netta Iivari

University of Oulu, Department of Information Processing Science
P.O.Box 3000, FIN-90014 University of Oulu, Finland
netta.iivari@oulu.fi

**Abstract.** This paper examines the emergence and evolution of user innovations in Open Source Software (OSS) development, with focus on usability innovations. Existing literature on user innovation and usability is reviewed, after which usability innovation is empirically explored in OSS development. The interpretive case study shows that usability innovations emerge and evolve in OSS development. They emerge after a user recognizes a need, after which she invents a fix to meet the need, thereafter needing a developer to realize the fix in the OSS. Afterwards, the user experiments with the solution and may provide feedback, which again may lead to the developer adjusting the OSS accordingly. The process is characterized as a collaborative negotiation process among the users and developers. The results also reveal that the usability innovations may be need, opportunity or creativity based, and connected to improving efficiency, effectiveness or satisfaction. Implications both for theory and practice are discussed.

## 1 Introduction

This paper examines how end user innovations emerge and evolve in Open Source Software (OSS) development, limiting the focus to end users' *usability* innovations. The concept of usability innovation refers to innovations that are trying to contribute to OSS usability, i.e. to 'the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use' [15]. Therefore, the focus is on innovations that try to enable the specified users to achieve their specified goals in the specified context of use more effectively, more efficiently, or generally in a more satisfactory way.

Innovation, on the other hand, refers to "an idea, practice, or object that is perceived as new by an individual or other unit of adoption [15: 12]. Innovation necessitates not only the invention part, but also its implementation and adoption [8]. End user innovation is defined as construction of innovations by end users, instead of manufacturers (or OSS developers) producing the innovations for the users [22]. The term end user implies that the innovation is to be produced by people without skills or interest to develop the solution themselves. Therefore, this kind of user innovation
always necessitates the developers joining in the implementation phase. In OSS development the developers tend to be also users of the software, and their innovation behaviour has already been examined [6], which will not be done in this paper. Researchers have argued that the user population of OSS solutions is constantly growing, including a growing number of users who do not have technical competence or interest to develop the OSS solution [6], [17], [23]. These users' innovation behaviour will be examined in this paper. It has also been argued that usability of the OSS solutions has not traditionally been a major concern among the developers, who have been more interested in improving the functionality of the software than in improving its usability, but since the non-developer users are entering the scene, also improvement of usability is becoming a legitimate concern [1], [17], [20].

OSS development is distributed and the OSS projects rely heavily on Internet tools supporting their work, i.e. on mailing lists, discussion forums, chat and bug reporting and version control systems [6], [19], [23]. In this paper the focus will be on end user innovation, which is observed during communication among OSS users and between OSS users and OSS developers in an OSS usability discussion forum that has been established by an OSS project to increase the software's usability and therefore provides a unique opportunity to study this phenomenon.

The next section reviews user innovation and usability literatures and relates them to the OSS development context. The third section discusses the interpretive case study that has been carried out, particularly focusing on a usability discussion forum of an OSS project, the fourth section outlining the main empirical results. The fifth section discusses their implications and limitations, and identifies paths for future work.

# 2 User and Usability Innovation

## 2.1. User Innovation

OSS development has been praised as an arena fostering user innovations (e.g. [3], [6]). Franke and von Hippel [6] have carried out an empirical study on user innovation in OSS development, arguing that user innovation is needed since the users' needs are highly heterogeneous. However, also they show that there are clear differences in the skill levels of the OSS users, some of them being able to modify the software, while others not, the skilled users being more satisfied with the software than the less skilled users. [6] Therefore, it can be assumed that the possibility to innovate leads to increased user satisfaction, but the results indicate that this may be very challenging for the non-developer users. This paper will examine how, if at all, non-developer user innovation happens in OSS development.

Even though non-developer user innovation has not been examined in OSS development, it has been studied in other contexts. Researchers have analysed it e.g.
in sports communities and in the research and development context (see e.g. [7], [10], [16]). Hyysalo [10] discusses micro-innovations as self-made modifications and adaptations to the existing products. Micro-innovations involve people making “various small alterations” both to the products and to their practices and usage patterns “to make their activity more enjoyable” and products “work better” [10: 250]. Also other researchers [16], [21] have considered user innovations contributing either or both to the development in the user domain and in the technical domain, and the non-developer users have been conceptualised as co-developers in the process that necessitates also the developers joining in [16]. Also in this paper the non-developer user innovations can address either or both the user and the technical domain, the non-developer users needing developer assistance in realizing the innovation in the OSS.

From the viewpoint of the emergence of user innovations, the following phases
have already been identified: user recognizes a need, solves the problem by
invention, builds a prototype and, finally, establishes the prototype's value in use
[22]. Floyd and colleagues [5] have examined user-driven technological innovation,
which, according to them, starts from user's everyday life and experiences, during
which the user encounters problems, to which she rapidly and easily produces fixes
through utilizing technology, after which she reflects on the solution, i.e. on how it
fits the overall systems, and adjusts the system as a result. The cycle starts over when
the user, again, encounters problems in her everyday life. This type of a process can
support the creativity, needs and desires of the users, and it is characterized as rapid,
participatory and collaborative [5]. OSS solutions enable users to explore how
technology could be utilized in their work practices and how their work practices
could be conceptualized anew with technology [5].

## 2.2. Usability Innovation

Usability innovations have not been discussed in depth in the literature. However, there exists a huge amount of literature addressing the development and evaluation of usability. The widely cited usability definition presented already establishes that usability can be developed only after one understands and specifies who the intended users are and what their goals in using the software are and in what kind of context of use the software will be used [14]. For gaining this understanding, different kinds of methods are suggested, e.g. interviews, observation and surveys. Thereafter, one needs to carefully redesign the users’ future tasks that are to be carried out to achieve the goals. Only after this the computer-based solution should be considered. Finally, essential is also to evaluate these design solutions, as early as possible and continually during the development life cycle [14], [11].

Typically these activities are expected to be carried out by usability (or user-
centered design, usability engineering etc.) specialists [11]. Even though not
explicitly mentioning usability innovations, the literature seems to assume that these
specialists produce the innovations, after being in contact with the users. Part of the
literature views this as a structured engineering process while other part maintains that it is a creative, artistic process, but generally in the both literatures it seems to be assumed that there needs to be some understanding of the users and their work practices, some understanding of the capabilities and limitations of technology and HCI background [11], the innovations being generated based on the combination of these understandings. This paper, instead, examines the innovations produced by the end users themselves, when given the chance to do that.

However, it needs to be underlined that usability in this paper is an etic concept introduced by the researcher. Another approach could be to analyze the multitude of meanings attached to usability by the OSS developers and users, but this has been done already, revealing that surprising and rather technical aspects are attached to usability in OSS development [1], [12]. Instead of showing this again, this paper adopts the established, etic usability definition from the literature and limits the focus to the innovations that adhere to this definition.

Existing OSS research has already outlined many issues, which may cause problems for usability innovation in OSS development. Typically, the developers do not have knowledge about the non-developer users, their goals and their contexts of use, and they do not necessarily have any interest in learning about them either. In addition, communicating usability problems to the development has proven to be difficult for the non-developer users. [2], [17], [20], [24] It has been argued that usability bugs are very complex to fix and difficult to explain textually [20]. Furthermore, usability specialists would be very useful in improving OSS usability, but typically there is a lack of usability specialists in OSS projects [2], [17], [24].

## 3. Research design

This research relies on the qualitative research tradition [4]. One case is analyzed: an OSS project developing a media application for end user without necessarily any technical knowledge or programming skills. The project is interested in their users and their feedback regarding the solution. The *usability discussion forum* asks the users of the OSS to take part in improving the program. Altogether, over 1600 messages and nearly 400 topics have emerged in this discussion forum. Nearly 600 message senders have contributed to the discussion forum. The project is a small but active one: there are 9 developers listed for the project, the development status being 5 (production/stable).

The analysis focused on the discussions in the usability discussion forum, which is assumed to be the place where user innovation occurs in this OSS project. The research data consists of the usability discussion forum messages (altogether 1600), all of which were printed out for the analysis purposes. First, the researcher familiarized her with the discussion forum and the message senders. All posts of the usability discussion forum were printed and read thorough. The focus was in identifying messages that could be connected to non-developer users’ usability innovation, i.e. on messages that were suggesting something considered new [8],
[18] by the community as well as dealing with improving usability, i.e. proposing improvements related to how users can achieve their goals more effectively, efficiently or in a more satisfactory way [15]. After the examination, it became evident that there were a lot messages without any replies. However, there were also some topics that had gained popularity in the sense of the number of replies as well as some of the discussions indicating that changes have resulted in the OSS due to these discussions. The focus was further limited to these discussions.

They were analyzed from the viewpoint of non-developer users' usability innovation. The literature presented in section 2 was used as a sensitizing device. In addition, a textual approach was utilized as a theoretical tool. It postulates that during development, the developers always 'configure the users' [9]; i.e. the developers (knowingly or not) delineate the future users and their work practices already while producing the software text [9]. OSS development arena enables the users to gain access and voice in the process through mailing lists, discussion forums, chat and such, which do not necessitate competence in technical development [13]. The users even if they were not capable to directly affect the source code, can participate in the discussions taking place in the OSS projects' websites, during which user innovation might occur. The innovating users "constitute a specific group of users that adopt specific, informed ways of not just reading but also introducing new scripts, by inscribing characteristics of their specific use situation into the product" [21: 186].

## 4. Empirical insights

In this OSS project non-developer user innovations improving effectiveness, efficiency and satisfaction in users' goal achievement were all evident. Especially innovations improving how users achieve their goals more *efficiently* were outlined. Some of them were implemented also fast by the developers, as in the following example. A user asks:

> "Better Keyboard/Shortcut integration: I'm asking for Shortcuts to show/hide the side bar, to change between the different sidebar tabs or a way to focus [an element]. To edit [information] which is not in the first column, a mouse is needed." (User)

A developer advises the user that the user can use tab and no mouse is needed. The user replies, after some experimentation

> "I never thought about using the tab in the edit mode - thanks for this information. Now I found some odd behaviour: the tab key circles through all possible entries including the non-visible one. So in some situations I have to press the tab key three or more times" (User)

Another developer joins the discussion the same day and announces that a new editor has been implemented and asks the user to test the solution.

The discussion reveals that the developer accepts the idea proposed by the user, i.e. he lets the user to establish certain aspects related to the future use practices (efficiency in editing), afterwards realizing those in the OSS text.
Another user also proposes improvement to the efficiency:

"What about a keyboard shortcut for selecting the next tag in the same column?" (User)

A developer suggests to the user to use tab, to which the user replies:

"I have in fact tried tab. Maybe I'm an unusual user; I'd like to jump to the next row, same column. Anyhow, in a recurring action like tag editing possibly it could be more useful with more control over navigation - e.g. the ability to move back and forwards, up and down" (User)

The developer agrees with the user and promises to try to enable it. Therefore, one can again conclude that the developer lets the user to establish certain aspects related to the future use practices (efficiency in editing). Related to realizing those in the OSS text, he promises to do as much as he can, the existing technological solution, however, restricting his possibilities.

Both of these examples illustrate that creating something 'new' involves a collaborative negotiation process among users and developers.

Also *effectiveness* in achieving goals is brought up, e.g. in the following relating to user goal of organizing his files:

"[Another user wrote]: "I would like a way to view [an element] like in [another application]."" Another vote for it. (...) When I have my list sorted [according to a field/field] I cannot just toggle them because the entries are greyed out. This is not convenient. Please permit it and use only the primary entry in this case." (User)

The same day a developer replies that the suggestion was smart and it has already been implemented. A noteworthy observation is also the voting for the suggestion of another user with which the message starts. In this case the developer does not implement it, however. Imitation of other applications is not appreciated in this OSS project, as can be concluded from numerous messages criticizing it in the discussion forum.

The user innovations implemented can be all labelled as small scale, incremental [8] micro-innovations [10] based on the users' needs. In these discussions, the users refer to their needs or problems in the 'configuration of the user' they expect to be met or removed through the solutions they propose. On the other hand, some discussions make it clear that not only 'needed' but also 'cool' and 'eye candy' features are to be included in this OSS, causing not only improvements in effectiveness or efficiency in goal achievement, but clearly also in *satisfaction* in doing so. For example, a user argues for tag guessing and for a tab with favourite and last files that both would be 'cool', and another user argues for using certain kinds of visualizations in the application that he calls 'eye-candy', maintaining that they would make use more fun but also introduce 'cool' usability improvements. Based on the discussions, one can conclude that the users are not only motivated by efficiency or effectiveness improvements, but also by 'cool, 'fun' and 'eye candy', which, nevertheless, also contribute to the users' goal achievement.

However, in the discussions it is not only the 'configuration of the user' in the OSS to be refined according to the users' requests, but the innovation concerns also the users' use practices that are conceptualised anew (see [5]), the change being
informed or inspired by the existing ‘configuration of the user’. In the community there has been a lot of discussion related to enabling accurate ranking. The OSS incorporates ranking but the users argue that it is not done correctly and they wish to be able to have more power related to that. An example message from a user is shown below:

> "I've been thinking a lot about how to do automatic ranking given implicit info on [user's use patterns], and there's really no good way. The problem is that the user's mood changes often. The entire scoring system can be totally broken based on the user's mood. [The system] would have to take the moods in to account to score correctly. You could achieve this with statistics on e.g. whether you were in front of your computer and how long you were there, plus what you did and didn't. Then make a decision how your "mood" changes your taste. There are two solutions for that: 1) Use stats from an entire community instead of making all new users train the system every time. 2) Give the users more guidance in the training process. A simple low tech way would be to provide a slider that tells how much skipping affects the score." (User)

Therefore, the user wishes to be able to achieve his goal related to accurate ranking of media files and proposes few solutions that in this case address both the technical solution and the ‘configuration of the user’ in the OSS. However, a developer informs the user that the idea has been to achieve another goal, altogether: the idea has been to let the OSS to set the scores based on the longer time span usage habits, but the users are arguing for the possibility to give ratings themselves. Nevertheless, the developer informs that due to the users’ requests the developers have implemented the rating functionality as well. However, he also argues that his own habits have changed due to the scoring information that indicates to the users what their long time favourites are. This information may lead the users to reflect on and even change their current habits.

The reciprocal relationship between use practice and technology becomes intensively visible in this discussion: the ‘configuration of the user’ sets the boundaries for the users’ use practices, as well as it is modified to enable more efficient, effective and satisfactory use practices, but the use practices may also be adapted, inspired or conceptualised anew (see [5]) when encountering the existing ‘configuration of the user.

**5. Concluding discussion**

This paper promised to examine the emergence of non-developer user innovations in an OSS usability discussion forum, at a detailed level in the natural setting without researcher intervention. Table 1 summarizes the key empirical findings of the examination.
<table><caption>**Table 1. Characterizing User Innovation in OSS Development**</caption><thead><tr><td>Aspect</td><td>Empirical Finding</td></tr></thead><tbody><tr><td>Encountering usability problems in everyday life</td><td>- Emergence of need-based usability innovations: user recognizes a need<br/>- Emergence of creativity or opportunity-based ('cool') usability innovations: user recognizes ways technology could be utilized in creative or opportunistic ways to improve usability</td></tr><tr><td>Producing usability fixes</td><td>- User invents usability fixes to the 'configuration of the user'<br/>- Developer realizes the fixes in the OSS<br/>- Producing fixes a collaborative process involving negotiation among users and between developers and users</td></tr><tr><td>Reflecting on and adjusting</td><td>- Users provide feedback to the refined 'configuration of the user'<br/>- Developers adjust the 'configuration of the user' accordingly<br/>- Users adjust their use practices accordingly</td></tr><tr><td>Relationship between use practice and technology</td><td>- Users identify usability fixes to the 'configuration of the user' after considering how technology could bring more efficiency, effectiveness or satisfaction to their goal achievement<br/>- Users conceptualize anew (adding efficiency, effectiveness or satisfaction to) their use practices based on the encounter with the current 'configuration of the user'</td></tr></tbody></table>

First of all, the results indicate that in the OSS development context end user usability innovation occurs, i.e. non-developer users innovate usability improvements to the OSS and those may also end up in the OSS. Next this process is characterized in more detail.

The process of user innovation is argued to include the following phases (modified from [5] [22]): a user recognizes a problem during her everyday life and experiences, she solves the problem by invention, she outlines a solution and finally she establishes its value in use, i.e. she reflects on the solution; how it fits the overall system and may also adjust the system as a result. All these phases could be identified in the usability discussion forum, but also clear differences and additions could be identified. Related to the user recognizing a usability problem during her everyday life and experiences, one clear distinction can be made between need-based and creativity or opportunity-based usability innovations (cf. [5]): some of the innovations are to meet the needs presented by the user, but other usability innovations are motivated as 'cool', even though still aiming at bringing efficiency, effectiveness or satisfaction to the users' use practices. Related to professional usability specialists working in the commercial context, one can argue that during user studies and such they are likely to encounter the usability problems and maybe also identify at least partly similar need-based usability innovations, but it might be very difficult for them to introduce the 'cool' factor into their innovations. Their 'cool' or 'eye-candy' may not equal the users' 'cool' or 'eye-candy'.

In the OSS development the users are speaking on behalf of themselves: they are reporting what they consider as needed, cool or eye-candy. The usability professionals, on the other hand, try to represent the users in the commercial software development context [13], i.e. they speak on behalf of these other people
they are trying to learn to know. This 'speaking on behalf of other people' has been reported to exist also in the OSS development context, in which, however, it seems to be carried out by amateur intermediaries, i.e. by users who do so without professional background on the matter [13]. Despite that, the contribution from these users, whether speaking on behalf of themselves or other users, could be of interest also in the commercial software development context, in which there might be professional, hired usability professionals available, but in which the development could still benefit from this type of amateur usability contributions as well. It might be that there are totally different kinds of innovations created by people who encounter the software continuously in their everyday life than by people studying the software use in other people's everyday life or than by people developing the software as part of their everyday life, but not using it. However, one should very carefully consider the motives behind these amateur usability contributions to be able to judge the validity of the claims - it might be that in some cases they are not trying to improve the OSS usability, but instead to accomplish other goals (cf. [13]).

User innovation necessitates the user to solve the problem by invention. In the OSS development context it is evident that people not interested in or capable to code do invent usability fixes to the 'configuration of the user' without touching the source code. Their suggestions are connected to the introducing efficiency, effectiveness and satisfaction to the users' goal achievement. The users rely on their in-depth domain knowledge and reflection on their usage habits and practices in producing the suggestions (cf. [10], [21]), i.e. while they are introducing these new scripts to the OSS. The non-developer users however need the developers for realizing their usability innovations, since the non-developer users are by definition not capable to code. Therefore, usability innovations in this case are necessarily produced during a collaborative process between users and developers, collaboration among the users also emerging related to backing up each other and discussing and further refining the ideas.

After the implementation, the users can reflect on the innovations' value in use and adjust the system accordingly. Of course, this happens during the longer time span, which might also be observable in the OSS forum discussions. In this project it is evident that after the developers have changed the 'configuration of the user', the users eagerly experiment with it and provide feedback. User feedback gathering through OSS discussion forums has already been recommended in the literature [19], [23]. Also in this case the discussion forum has been built up for users to provide feedback and improvement ideas. Interestingly, even though most users are asking fixes to the 'configuration of the user' after considering how technology could be utilized to better support their use practices, in some messages also the question of how their use practices could be conceptualised anew with technology (cf. [5]) emerge: the users may conceptualise their use practices anew after experimenting with the existing 'configuration of the user'. These discussions, however, were clearly in minority in the discussion forum.
This study opens interesting avenues for user innovation researchers. This study provides a lens through which to examine user-developer interaction related to innovation in the context of technology development. The lens emphasizes that technological artefacts always include a ‘configuration of the user’ the users encounter and interpret. The users may wish to provide feedback and suggest fixes to the configuration, but they may also redefine their use practices anew after the encounter. The artefact already implies certain kind of future user with certain kind of future use practices, but the users can also interpret and modify those in new, creative, innovative ways. Other researchers are welcomed to adopt and refine this sensitising device.

Companies are showing increasing interest to enhance user innovation related to their products and services. Use of OSS communities for that has already been recommended (e.g. [3], [6]). The results of this study suggest that also end user innovation occurs in OSS development and online forums provide support for it. However, it might be a great challenge for companies to establish online forums and associated communities and to keep them vital. In traditional OSS development everything operates on a voluntary basis, and it might be a challenge to invite the non-developer users to participate in development and in the associated discussions. The forum examined in this paper has succeeded in inviting also the non-developer users to contribute to the development. However, this study does not provide tools for companies for online community building, but it informs the companies that end user innovation may emerge and evolve in online OSS forums. In addition, this study offers interesting insights for usability research, showing that ‘amateur usability specialists’ are contributing to OSS development. They are at least active and creative usability innovators, probably not replacing the professional ones but surely producing interesting ideas to be considered for the implementation.

This study is based on only one case, which is naturally particular in many ways. In the future more cases and more diversity (small vs. large projects, new-found vs. long-term projects, different kinds of application domains, projects with or without company involvement and so on) should be included in the future analysis in the OSS development context.

## Acknowledgements

This research has been partly funded by the Academy of Finland.

## References

[1] Andreasen, M., Nielsen, H., Schrøder, S. & Stage, J. (2006): Usability in Open Source Software Development: Opinions and Practice. Information Technology and Control 25(3A): 303-312.
[2] Benson, C., Müller-Prove, M., Mzourek, J. (2004): Professional usability in open source projects: GNOME, OpenOffice.org, NetBeans. In Extended Abstracts of CHI. ACM Press, New York. Pp. 1083-1084.

[3] Chesbrough, H. (2003): The Era of Open Innovation. MIT Sloan Management Review 44(3): 35-41.

[4] Denzin, N. & Lincoln, Y. (2000): Introduction: The Discipline and Practice of Qualitative Research. In N. Denzin, Y. Lincoln (Eds.): Handbook of Qualitative Research. 2nd edition. Thousand Oaks: Sage Publications. Pp. 1-28.

[5] Floyd, I., Jones, M. Rathi, D. & Twidale, M. (2007): Wab Mash-ups and Patchwork Prototyping: User-driven technological innovation with Web 2.0 and Open Source Software. In Proc. HICSS 2007. IEEE: Washington.

[6] Franke, N. & von Hippel, E. (2003): Satisfying heterogeneous user needs via innovation toolkits: the case of Apache security software. Research Policy 32: 1199-1215.

[7] Franke, N. & Shah, S. (2003): How communities support innovative activities: an exploration of assistance and sharing among end-users. Research Policy 32: 157-178.

[8] Garcia, R. & Calantone, R. (2002): A critical look at technological innovation typology and innovativeness terminology: A literature review. Journal of Product Innovation Management 19(2): 110-132.

[9] Grint, K. & Woolgar, S. (1997): The Machine at Work. Technology, Work and Organization. Cambridge, Polity Press.

[10] Hyysalo, S. (2009): User innovation and everyday practices: Micro-innovation in sports industry development. R&D Management 39(3): 247-258.

[11] Iivari, N. (2006): Discourses on ‘culture’ and ‘usability work’ in software product development. Acta Universitatis Oluensis, Series A, Scientiae rerum naturalium 457.

[12] Iivari, N. (2008): Usability in open source software development – an interpretive case study. In Proc. ECIS, Galway, Ireland, June 9.-11.2008.

[13] Iivari, N. (2009): “Constructing the Users” in Open Source Software Development – An Interpretive Case Study of User Participation. Information Technology & People 22(2): 132-156.

[14] ISO 13407 (1999). Human-centered design processes for interactive systems. International standard.

[15] ISO 9241-11 (1998). Ergonomic requirements for office work with visual display terminals (VDT)s - Part 11 Guidance on usability. International standard.

[16] Lettl, C., Herstatt, C. & Gemuenden, H. (2006): Users' Contributions to Radical Innovation: Evidence from Four Cases in the Field of Medical Equipment Technology. R&D Management, 36(3): 251-272.

[17] Nichols, D. & Twidale, M. (2006): Usability Processes in Open Source Projects. Software Process Improvement and Practice 11: 149-162.

[18] Rogers, E. (1995): Diffusion of Innovations. 5th Edition. New York, Free Press.
[19] Scacchi, W. (2002): Understanding the requirements for developing open source software systems. IEE Proceedings – Software 149(1): 24-39.

[20] Twidale, M. & Nichols, D. (2005): Exploring Usability Discussions in Open Source Development. In Proc. HICSS. IEEE: Washington.

[21] van Oost, E., Verhaegh, S. & Oudshoorn, N. (2009): From Innovation Community to Community Innovation: User-initiated Innovation in Wireless Leden. Science, Technology & Human Values 34(2): 182-205.

[22] von Hippel, E. (1988): The Sources of Innovation. Oxford University Press: New York.

[23] Ye, Y. & Kishida, K. (2003): Toward an Understanding of the Motivation of Open Source Software Developers. In Proc. ICSE, IEEE: Washington. Pp. 419-429.

[24] Zhao, L. & Deek, F. (2005): Improving Open Source Software Usability. In Proc. AMCIS, August 11-14, Omaha, USA. Pp. 923-928.