Software Development Expertise
Tags: expertise, theory, productivity, research
Published: 2018-10-08 16:31:55

---
title: Software Development Expertise
published: true
description: Towards a Theory of Software Development Expertise
tags: #expertise #theory #productivity #research
---

<p align="center" style="margin-bottom: 30pt;">
    <img alt="Logo Snippets" src="http://empirical-software.engineering/assets/images/developer.png">
</p>

**TL;DR:** *Being an expert in software development requires a certain set of skills, knowledge, and experience. We present a first conceptual theory of software development expertise that is grounded in data from a mixed-methods survey with 335 software developers and in literature on expertise and expert performance. The theory describes central properties of software development expertise and which factors foster or hinder its formation, including how developers’ performance may decline over time. Moreover, our quantitative results show that developers' expertise self-assessments are context-dependent and that experience is not necessarily related to expertise.*

**Source:** This [post](http://empirical-software.engineering/blog/expertise/) on my personal blog. 

## Motivation

An *expert* is, according to Merriam-Webster, someone "having, involving, or displaying special skill or knowledge derived from training or experience". For some areas, such as playing chess, there exist representative tasks and objective criteria for identifying experts. In software development, however, it is more difficult to find objective measures for quantifying expert performance. One reason is that software development itself includes diverse tasks such as implementing new features, analyzing requirements, and fixing bugs. In the past, researchers investigated certain aspects of software development expertise such as the influence of programming experience, desired attributes of software engineers, or the time it takes for developers to become "fluent" in software projects. However, there is currently no theory combining those individual aspects. Such a theory could help structuring existing knowledge about software development expertise in a concise and precise way and hence facilitate its communication. With our [paper](http://empirical-software.engineering/publications#fse18-expertise), we contribute a first theory that describes *central properties* of software development expertise and *important factors* influencing its *formation*. The theory focuses on individual software developers working on different software development tasks, having the long-term goal of becoming experts in those tasks. It is grounded in data from a mixed-methods survey with 335 participants and in literature on expertise and expert performance. Our expertise model is task-specific and currently focuses on programming-related tasks, but includes the notion of transferable knowledge and experience from related fields or tasks. Our *process theory* is a first step towards the long-term goal to build a *variance theory* to be able explain and predict why and when a software developer reaches a certain level of expertise.

## Conceptual Theory

In this blog post, we skip the details of our research design and focus on the resulting conceptual theory, summarzing implications for researchers, developers, and employers.

<p align="center">
    <a href="/assets/images/conceptual-theory-2.png"><img alt="Conceptual Theory" src="http://empirical-software.engineering/assets/images/conceptual-theory-2.png"/></a>
    <br/>
</p>

The above figure visualizes the high-level concepts of our conceptual theory together with their relationships. Some relationships are intentionally labeled with rather generic terms such as "affects" or "generates", because more research is needed to investigate them. The theory describes the *process* of software development expertise formation, that is the path of an individual software development novice towards becoming an expert. This path [consists](https://www.cambridge.org/core/books/the-cambridge-handbook-of-expertise-and-expert-performance/the-influence-of-experience-and-deliberate-practice-on-the-development-of-superior-expert-performance/C56EDDE9E57B259825916E061B025A72) of gradual improvements with many corrections and repetitions, therefore we do not describe discrete steps like, for example, the 5-stage Dreyfus model of skill acquisition (see Section "Experience vs. Expertise"). Instead, we focus on the repetition of individual tasks. In the following, we will describe the core of our theory (gray background in the figure). In particular, we will elaborate on the role of *monitoring*, *feedback*, and *self-reflection*, and connect the corresponding feedback cycle with the concept of *deliberate practice*. All concepts and relationships of our theory are described in more detail in the [paper](http://empirical-software.engineering/publications#fse18-expertise).

### Knowledge and Experience

*Knowledge* can be [defined](https://dl.acm.org/citation.cfm?id=291476) as a "permanent structure of information stored in memory". Some [researchers](https://dl.acm.org/citation.cfm?id=801956) consider a developer's knowledge base as the most important aspect affecting their *performance*. [Studies](https://www.cambridge.org/core/books/the-cambridge-handbook-of-expertise-and-expert-performance/expertise-in-software-design/04E88E182D83E956AB5582CA846AFF73) with software developers suggest that "the knowledge base of experts is highly language dependent", but experts also have "abstract, transferable knowledge and skills". We modeled this aspect in our theory by dividing the central concepts *knowledge* and *experience*, which we derived from a first iteration of our online survey and from related work, into a task-specific and a general part. This is a simplification of our model, because the relevance of knowledge and experience is rather a [continuum](https://www.cambridge.org/core/books/cambridge-handbook-of-expertise-and-expert-performance/modes-of-expertise-in-creative-thinking-evidence-from-case-studies/911934EDB1E0479E536A46E646420DE6) than dichotomous states. However, [Shneiderman and Mayer](https://www.cs.umd.edu/~ben/papers/Shneiderman1979Syntactica.pdf), who developed a behavioral model of software development, used a similar differentiation between general ("semantic") and specific ("syntactical") knowledge. General knowledge and experience does not only refer to technical aspects (e.g., low-level computer architecture) or general concepts (e.g., design patterns), but also to knowledge about and experience with successful strategies.

Many participants  referred to the importance of the *quantity* of software development *experience* (e.g., in terms of years), but some also described its *quality*. Examples for the latter include having built "everything from small projects to enterprise projects" or "[experience] with many codebases." In particular, participants considered *professional experience*, for example having "shipped a significant amount of code to production or to a customer", and working on *shared code* to be important factors. Examples for general knowledge include "paradigms [...], data structures, algorithms, computational complexity, and design patterns"—task- or language-specific expertise includes having an "intimate knowledge of the design and philosophy of the language'' and knowing "the shortcomings of the language [...] or the implementation [...]." 

### Tasks

Since our model of software development expertise is task-specific, we asked our participants to name the three most important tasks that a software development expert should be good at. The three most frequently mentioned tasks were *designing software architecture*, *writing source code*, and *analyzing and understanding requirements*. Many participants not only mentioned the tasks, but also certain quality attributes associated with them, for example

> "architecting the software in a way that allows flexibility in project requirements and future applications of the components"

and "writing clean, correct, and understandable code". Other mentioned tasks include *testing*, *communicating*, *staying up-to-date*, and *debugging*. Our theory currently focuses on tasks directly related to programming, but our participants' responses show that it is important to broaden this view in the future to include, for example, tasks related to requirements engineering (*analyzing and understanding requirements*) or the adaption of new technologies (*staying up-to-date*).

### Motivation

Related work describes certain *individual differences* that affect if and how people come closer to their goal of being an expert in a certain domain.
The *mental abilities* and *personality* are quite stable, but especially the *motivation* can change over time and influence the way people approach software development. Many of the participants in our study were intrinsically motivated, stating that *problem solving* is their main motivation. One participant wrote that solving problems "makes [him] feel clever, and powerful." Another participant compared problem solving to climbing a mountain:

> "I would equate that feeling [of getting a feature to work correctly after hours and hours of effort] to the feeling a mountain climber gets once they reach the summit of Everest."

Many developers enjoy seeing the *result* of their work. They are particularly satisfied to see a solution which they consider to be of high *quality*. One participant wrote that (s)he gets "a feeling of satisfaction from creating an elegant piece of code." Moreover, some participants mentioned refactoring as a rewarding task:  

> "The initial design is fun, but what really is more rewarding is refactoring."

Others stressed the importance of *creating something new* or *helping others*. Interestingly, *money* was only mentioned by few participants as a motivation for their work.

### Performance

At this point, our goal is not to treat *performance* as a dependent variable that we try to explain for individual tasks, we rather consider different *performance monitoring* approaches to be a means for *feedback* and *self-reflection* (see below). For our long-term goal to build a *variance theory* for explaining and predicting the development of expertise, it will be more important to be able to accurately measure developers' performance. However, participants' mentioned different properties that source code of experts should possess: It should be *well-structured* and *readable*, contain "comments when necessary", be "optimized" in terms of "performance" and sustainable in terms of *maintainability*. One participant summarized the code that experts write as follows:

> "Everyone can write [...] code which a machine can read and process but the key lies in writing concise and understandable code which [...] people who have never used that piece of code before [can read]."

To identify factors hindering expertise development, we asked participants whether they ever observed a significant decline of their own programming performance or the performance of co-workers over time. In our study, **41.5% of the participants who answered that question actually observed a performance decline over time**. We asked those participants to describe how the decline manifested itself and to suggest possible reasons. The main reasons for performance decline were related *demotivation*, changes in the *work environment*, *age-related decline*, *changes in attitude*, and *shifting towards other tasks*. Related to *problem solving* being one of the main *motivations* (see above), the most common reason for *demotivation* was *non-challenging work*, often caused by tasks becoming routine over time. One participant described this effect as follows:

> "I perceived an increasing procrastination in me and in my colleagues, by working on the same tasks over a relatively long time (let's say, 6 months or more) without innovation and environment changes."

Other reasons included not seeing a clear *vision or direction* in which the project is or should be going and missing *reward* for high-quality work, which was described above as one of the main factors motivating developers. Regarding the *work environment*, participants named *stress* due to tight deadlines or economic pressure ("the company's economic condition deteriorated''). Moreover, bad *management* or *team structure* were named, for example:

> "[h]aving a supervisor/architect who is very poor at communicating his design goals and ideas, and refuses to accept that this is the case, even when forcibly reminded.''.

*Changes in attitude* may happen due to personal issues (e.g., getting divorced) or due to shifting priorities (e.g., friends and family getting more important). When developers are being promoted to team leader or manager, they "shift towards other tasks", resulting in a declining programming performance. 

Participants also described *age-related performance decline*. We consider this phenomenon, and the resulting consequences for individual developers and the organization, to be an important area for future research. To illustrate the effects that age-related decline may have, we provide two verbatim quotes by experienced developers:

> "For myself, it's mostly the effects of aging on the brain. At age 66, I **can't hold as much information short-term memory**, for example. In general, I am more forgetful. I can compensate for a lot of that by writing simpler functions with clean interfaces. The results are still good, but my productivity is much slower than when I was younger." (software architect, age 66)

> "Programming ability is based on desire to achieve. In the early years, it is a sort of **competition**. As you age, you begin to realize that outdoing your peers isn't all that rewarding. [...] I found that I lost a significant amount of my focus as I became 40, and started **using drugs such as ritalin** to enhance my abilities. This is pretty common among older programmers.'' (software developer, age 60)

### Task Context

The *task* or *work context* can either foster or hinder developers on their way to becoming an expert in certain software development tasks. To investigate its influence on expertise development, we asked what employers should do in order to facilitate a continuous development of their employees' software development skills. We grouped the responses into four main categories: *encourage learning*, *encourage experimentation*, *improve information exchange*, and *grant freedom*. To *encourage learning*, employers may offer in-house or pay for external *training courses*, pay employees to visit *conferences*, provide a good analog and/or digital *library*, and offer *monetary incentives* for self-improvement. The most frequently named means to *encourage experimentation* were motivating employees to pursue *side projects* and building a work environment that is open for *new ideas and technologies*. To *improve information exchange* between development teams, between different departments, or even between different companies, participants proposed to *facilitate meetings* such as agile retrospectives, "Self-improvement Fridays", "lunch-and-learn sessions", or "Technical Thursday" meetings. Such meetings could explicitly target information exchange or skill development. Beside dedicated meetings, the idea of developers *rotating* between teams, projects, departments, or even companies is considered to foster expertise development. To improve the information flow between developers, practices such as *mentoring* or *code reviews* were mentioned. Finally, *granting freedom*, primarily in form of *less time-pressure*, would allow developers to invest in learning new technologies or skills.

### Deliberate Practice

Having more experience with a task does [not automatically lead](http://projects.ict.usc.edu/itw/gel/EricssonDeliberatePracticePR93.PDF) to better performance. Research has shown that once an acceptable level of performance has been attained, additional "common" experience has only a negligible effect, in many domains the performance even [decreases](https://www.cambridge.org/core/books/cambridge-handbook-of-expertise-and-expert-performance/studies-of-expertise-from-psychological-perspectives/3A7FF4C6F3426BE751C71EDF84927741) over time. The length of experience has been found to be only a [weak correlate](https://www.cambridge.org/core/books/cambridge-handbook-of-expertise-and-expert-performance/influence-of-experience-and-deliberate-practice-on-the-development-of-superior-expert-performance/C56EDDE9E57B259825916E061B025A72) of job performance after the first two years—what matters is the *quality* of the experience. According to [Ericsson et al.](http://projects.ict.usc.edu/itw/gel/EricssonDeliberatePracticePR93.PDF), expert performance can be explained with "prolonged efforts to improve performance while negotiating motivational and external constraints". For them, *deliberate practice*, meaning activities and experiences that are targeted at improving the own performance, are needed to become an expert. Traditionally, research on deliberate practice concentrated on acquired knowledge and experience to explain expert performance. However, later studies have shown that deliberate practice is necessary, but [not sufficient](http://journals.sagepub.com/doi/abs/10.1177/0963721411421922), to achieve high levels of expert performance—individual differences play an [important role](http://journals.sagepub.com/doi/abs/10.1177/0963721411422061).

A central aspect of *deliberate practice* is [monitoring](https://www.cambridge.org/core/books/the-cambridge-handbook-of-expertise-and-expert-performance/the-influence-of-experience-and-deliberate-practice-on-the-development-of-superior-expert-performance/C56EDDE9E57B259825916E061B025A72) one's own *performance*, and getting *feedback*, for example from a teacher or coach. Generally, such feedback helps individuals to *monitor* their progress towards [goal achievement](http://psycnet.apa.org/record/1990-97846-000). Moreover, as [Tourish and Hargie](https://www.taylorfrancis.com/books/e/9781134508242/chapters/10.4324%2F9780203634394-18) note:

> "[T]he more channels of accurate and helpful feedback we have access to, the better we are likely to perform."

In areas like chess or physics, studies have [shown](https://www.cambridge.org/core/books/cambridge-handbook-of-expertise-and-expert-performance/two-approaches-to-the-study-of-experts-characteristics/823FCCA5B7E6E0FEDB5A51C042C831C2) that experts have more accurate self-monitoring skills than novices. In our model, the feedback relation is connected to the concept *task context* as we assumed that feedback for a software developer most likely comes from co-workers or supervisors. Moreover, *mentoring* provides feedback and was identified in our study as an important source for *motivation*. To close the cycle, also (self-) *monitoring* and *reflection* influence developers *motivation* and consequently their *behavior*.

### Monitoring and Self-reflection

In our survey, we asked participants whether they regularly monitor their software development activities. **38.7% of the 204 participants who answered that question said that they regularly monitor their software development activity**. The most important monitoring activity was *peer review*, where participants mentioned asking co-workers for feedback, doing code-review, or doing pair-programming. One participant mentioned that he tries to "take note of how often \[he\] win\[s\] technical arguments with \[his\] peers." Participants also mentioned *time tracking* tools like WakaTime or RescueTime, *issue tracking* systems like Jira or GitHub issues, and *project management* tools like *Redmine* and *Scrum story points* as sources for feedback, comparing expected to actual results (e.g., time goals or number of features to implement). Moreover, some developers mentioned writing a [*development diary*](https://news.ycombinator.com/item?id=14382965). Participants reported using simple metrics such as the *commit frequency*, *lines of code added/deleted*, or the number of *issues resolved*. Further, they reported to use *static analysis* tools such as SonarQube, FindBugs, and Checkstyle, or to use GitHub's activity overview. Some participants were doubtful regarding the usefulness of metrics, one participant noted:

> "I do not think that measuring commits [or] LOC [...] automatically is a good idea to rate performance. It will raise competition, yes—but not the one an employer would like. It will just get people to optimize whatever is measured.''

The described phenomenon is also known as [*Goodhart's law*](https://en.wikipedia.org/wiki/Goodhart%27s_law).

### Experience vs. Expertise

Since software developers' expertise is difficult to measure, researchers often rely on proxies for this abstract concept. We investigated the relationship and validity of the two proxies *length of experience* and *self-assessed expertise* to provide guidance for researchers. We found that neither developers' experience measured in years nor the self-assessed programming expertise ratings yielded consistent results across all analyzed settings. We were also interested in the validity of self-assessed expertise, which is, like other self-reports, [context-dependent](https://dornsife.usc.edu/assets/sites/780/docs/01_aje_schw_oys_behavior.pdf). The validity of self-assessed expertise is related to the concept of *self-reflection*, but has also methodological implications for software engineering research in general, because self-assessed programming expertise is often used in studies with software developers to differentiate between novices and experts. To analyze the influence of question context on expertise self-assessments, we asked participants for two self-assessments of their Java expertise: One using a semantic differential scale from novice to expert (without further context) and one based on the [Dreyfus model](https://en.wikipedia.org/wiki/Dreyfus_model_of_skill_acquisition) of skill acquisition (with context for each stage). We found that only the experienced developers tended to adjust their self-assessments to a higher rating after we provided context in form of the Dreyfus model. A possible interpretation of this result could be found in the [*Dunning-Kruger effect*](https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect): [Kruger and Dunning](http://psycnet.apa.org/buy/1999-15054-002) found that participants with a high skill-level underestimate their ability and performance relative to their peers. This may have happened in our sample with experienced developers when they assessed their Java expertise using the semantic differential scale. When we provided context in form of the Dreyfus model, they adjusted their ratings to a more adequate rating, whereas the less experienced developers stuck to their, possibly overestimated, ratings.

## Conclusion

In this blog post, we presented  core concepts of our conceptual theory of software development expertise, which is grounded in data from an online survey with 355 software developers and in existing literature on expertise and expert performance. We described different properties of software development expertise and factors fostering or hindering its development. In particular, we described developers' *motivation* and *performance decline* and how the *task context* and *deliberate practice* can foster expertise development.

**Researchers** can use our methodological findings about (self-assessed) expertise and experience when designing studies involving self-assessments. If researchers have a clear understanding what distinguishes novices and experts in their study setting, they should provide this context when asking for self-assessed expertise and later report it together with their results. In our theory, we did not describe expertise development in discrete steps, but a direction for future work could be to at least develop a standardized description of *novice* and *expert* for certain tasks, which could then be used in semantic differential scales. To design concrete experiments measuring certain aspects of software development expertise, one needs to [operationalize](https://ieeexplore.ieee.org/document/4052585) our conceptual theory. In our [paper](http://empirical-software.engineering/publications#fse18-expertise), we already linked concepts to existing measurement instruments such as *UMS* (motivation), *WAIS-IV* (mental abilities), or *IPIP* (personality) and also mentioned static analysis tools to measure code quality and basic productivity measures. This enables researchers to design experiments, but also to re-evaluate results from previous experiments. There are, for example, no coherent results about the connection of individual differences and programming performance yet.

**Software developers** can use our results to see which properties are distinctive for experts in their field, and which behaviors may lead to becoming a better software developer. For example, the concept of *deliberate practice*, and in particular having *challenging goals*, a *supportive work environment*, and getting *feedback from peers* are important factors. Beside peer-feedback, developers can use different code quality and productivity measures and tools to monitor their *performance* and *reflect* about their progress in becoming better software developers. As outlined above, phenomena like the *Dunning-Kruger effect* may affect developers' ability to *self-reflect*. However, by establishing a combination of metrics, tools, peer-review, and mentoring, developers can try to mitigate such biases. The *task context* is a factor that can heavily influence the ability of developers to improve their software development skills, but is often out of their control. Once developers are in a team lead or management role, they can try to establish some of our participants' suggestions to build a supportive work environment (see below).

**Employers** can use our results to learn what typical reasons for *demotivation* among their employees are, and how they can build a *work environment* supporting the self-improvement of their staff. They should be aware that *problem solving* is one of the main motivations for software developers, leading to the main factor for demotivation, which is *non-challenging work*. To avoid tasks becoming routine over time and thus being perceived as non-challenging, employers should make sure to have a good mix of continuity and change in their software development process.
They should also communicate clear *visions, directions, and goals* and *reward* high-quality work wherever possible. Beside obvious strategies for expertise development such as offering training sessions or paying for conference visits, our results suggest that employers should think carefully about how *information is shared* between their developers and also between the development team and other departments of the company. Facilitating meetings that explicitly target information exchange and learning new skills should be a priority of every company that cares about the development of their employees.

## More Information

A more detailed description of our research design and results can be found in our [research paper](http://empirical-software.engineering/publications#fse18-expertise). If you have any further questions, do not hesitate to contact me.