Usability review & analysis: Microsoft Teams
By: Somesh K S
Tags: Usability, UX, Tools, Microsoft, Microsoft Teams

Review begins…

Target groups

Most of the video conferencing tools are available for general public use. However, Microsoft teams have positioned themselves as solution providers for academic and professional institutions. Due to the pandemic situation, there is a surge in the number of users, and the number of new features addressing remote culture is being rolled out.

General target group

We consider that the general target group is: professional and academic institutions. It’s a similar target to what is focused on by applications like Slack and Zoom.

Particular target group

Our particular target group, the one that has been carrying the tasks out is: College and University students.

Persona snapshot

Most of the students fall under the category of Generation-Z, who are also digital natives. This category of users are socially active and are digitally savvy. They use multiple digital devices in their daily life, whose mental models are equipped with the understanding of Digital User Interface. They belong to the age-group of 18–28 in most of the cases. Especially due to the pandemic, the usage of such internet tools has drastically increased.

User tasks for the analysis

Task 1: Share meeting details to invite a person who’s not a part of the current meeting

Task 2: Add a person named “Austin Brandon” to the call

Task 3: Take some notes during the meeting which would be 50 characters long.

Task 4: Share a particular screen along with the computer audio.

Task 5: Stop sharing your screen after you’ve shared your screen.

Task 6: Send a personal message to one of the participants of the video call.

Effectiveness

It talks about the accuracy and completeness of user goal achievement (ISO 9241–11). In other words, it refers to how good a product is at doing what it is supposed to do.

Effectiveness: Qualitative

Is the product capable of allowing people to learn, carry out their work efficiently, access the information they need?

On one hand, our target users are able to carry out their meetings, share their screens, share files, and a lot of other different things that make it easier to follow up with remote work or online classes. On the other hand, some of the other tasks (such as access to some information, change video conference settings, and so on) are a bit difficult to carry out.

Here are some examples,

It is quite hard for first-time users to know where to find the list of participants of a channel/group.

Group chat UI

2. Fitt’s law violation: Essential action icons are placed so close that the chances for accidental clicks/Errors are higher. And, the chances to leave the meeting while trying to present something by clicking the Share content icon are higher!

Action toolbar of video chat

3. Due to the type of O365 account creation, all the participant’s identity are prioritized with their email IDs instead of the names. When there are a ton of new messages, it’s difficult to connect the messages with the sender’s name.

A message on team chat

4. There’s a steep increase in the cognitive load while finding a participant or fetching one’s name.

Participant list

5. Some of the visual representation of actions menus may not be clear enough,

Action buttons of a team

a. The CTA Meet literally means to meet someone, but the way it actually works is different. This usually causes a mental model mismatch. Since the user might think, by clicking this button would call everyone in a particular channel/team.

When “Meet” is clicked, you would actually start a meeting even without anyone being active on the call. Later you can invite any of the participants to join you on the call.

b. Channel info is another important piece of information for a participant to know. It is placed next to the Meet button, implying that it is related to the video call feature.

If there was a text beside the info icon saying Channel info it would really help users to take the desired action without the need to think.

c. The indication that the particular channel “can be accessed only by the participants of the channel” is not very clear without the tooltip which is used.

6. Most essential actions are “not easily accessible.”

Contact popup

More options with regards to a particular user are available on a click/hover on the profile picture, but there’s no external indication to convey that more options are available by clicking on the profile picture.

7. Some issues with the conversation section,

a. Visually, it is not as welcoming to make a conversation with a button “New conversation” rather than having an open text field with all the icons for attaching a file, emoji, GIF, etc.

In addition, in this way, the number of clicks required in order to upload a file is two, while if the text area is open it would have been just one.

New conversation interaction flow

b. The New conversation button does not allow the users to file drop a file instantly without having it clicked.

User unable to drop a file without clicking the button

c. When the button is clicked, the file drop area is too small to make it more accessible.

Smaller drop area

8. Longer drop-down:

Hick’s law violation: Time is taken to make a decision increases with the number and complexity of choices. The chances to select the wrong option are higher.

Miller’s law violation: An average person can keep only 5–7 items in their working memory. Whereas, in this particular dropdown (the one within an ongoing meeting, when clicking on the three dots icon) we have 12 items to choose from. That’s almost doubling the cognitive load on the user.

Longer drop-down menu for overflow menu

Effectiveness: Quantitative

We measured two parameters,

Task completion rate: The number of participants successfully complete a task by the total number of participants who take part in the activity. We got an average of 66.67% (Higher the better) for all 6 tasks.

Task completion rate

2. Task repetition rate: The number of participants who repeated a task at least once to complete it by a total number of participants takes part in the activity. We got an average of 5.55% (Lower the better)for all 6 tasks.

Task repetition rates

Efficiency

Efficiency is a relationship between results achieved (outputs) and resources used (inputs) (ISO 9241–11). It also refers to the way a product supports users in carrying out their tasks quickly and with as few resources as possible.

Efficiency: Qualitative

Once users have learned how to use a product to carry out their tasks, can they sustain a high level of productivity?

As some of us have already experienced, it seems that it is not that easy to complete tasks even when you have been using the platform for quite a while, but there’s a reason for that: “Teams” has undergone some interface changes lately, and far from making it easier, users that had already used the platform encountered some difficulties to carry out tasks they knew how to do before.

Apart from that, it may be that there’s a difference between people from different age ranges and how much it cost them to maintain their productivity level in a higher position. That may be due to their usage in similar applications, how used are they to using different kinds of technologies, etc.

Efficiency: Quantitative

We measured two parameters,

Number of tasks per time unit (expert user).

We’ve considered a unit time as 60s

The cumulative sum of average time taken to complete 5 tasks is 44s

The average time taken to complete a task is 8.8s

The number of tasks performed per unit time is ~7 tasks

2. Average time taken per task,

Time taken to accomplish each task

Safety

Safety involves protecting the user from dangerous conditions and undesirable situations. The interactive system must enable users to explore the system without the fear of breaking it, and also prevent them bad situations.

Safety: Qualitative

What if the range of errors that are possible using the product and what measures are there to permit users to recover easily from them?

Most of the errors are caused due to design language discrepancy, higher latency (>400ms) while interacting with the software, and violation of some of the UX laws like Fitt’s law, Jacob’s law, and Miller’s law.

Here are some of the errors that were found during the expert analysis and personal observations while using the software,

While sharing the screen, users tend to share the wrong screen. This is due to the screen segregations are overwhelming and not very clear while sharing content. Plus, the spacing between different sections of screens are small which also does not give the sense of grouping to the users.

Share content modal

2. Most of the time, users are unable to share computer audio. This is due to the toggle button for sharing computer audio is just above the ambiguous categories of screens which gets unnoticed.

Share content modal with include computer sound toggle menu

3. Adding / Inviting the wrong participant to the meeting, or unable to add a particular participant. This is because every participant is referenced with their email address but not their name which makes it difficult to know the exact email addresses of other participants while searching for them.

In the case above, the student’s name is “Venkata Prithvi Krishna Palle” but, if the user only knows their name and not their email address, the particular student could not be found in the participant search.

Search participant to add to the call

4. Participants select “Raise hand” by mistake. Because the spacing between the different action icons is less. Plus, the selection área This causes erroneous selection of menus while trying to accomplish a task quickly. And, while trying to lower the hand there’s a huge delay caused during low-bandwidth situations.

Action menus

5. Unable to dismiss the share content modal. When the user tries sharing their screen and decides not to share any, they usually search for a close icon (X) to dismiss the modal. But, the only way to dismiss that is to move the cursor all the way up and click the same icon used to initiate content sharing.

Share content & close share content menus

6. Unable to stop screen sharing. Since the screen sharing toolbar auto-hides as the cursor moves away from the top region of the screen. Most of the users would not recall the hidden toolbar to stop screen sharing.

Share content toolbar

Of course, the users can pin the toolbar to disable the auto-hide behavior. But, the icon is too small to get the user's attention.

Share content box

Here are some of the measures to prevent errors and promote better error recovery,

Errors & improvements

Safety: Quantitative

Level of understanding of errors’ messages (through observation or questionnaire).

As we begin with the level of understanding of errors’ messages, we only a single error message while using the application which was about the notes functionality: we couldn’t get to take notes in our private group call and the error message displayed on the screen was not giving us any information about what was the real problem.

2. Error frequency

Error frequency

3. Undo mechanism

About the undo mechanism, the program is designed in a way in which the situation of making a serious mistake is not frequent, the occasions in which one of these mistakes did happen (like for example, the “inviting someone to the meeting by accident” error) did not make the situation easy to revert.

Undo mechanism

Utility

Utility refers to the extent to which the product provides the right kind of functionality so that users can accomplish their tasks easily.

Utility: Qualitative

Does the product provide an appropriate set of functions that will enable users to carry out all their tasks in the way they want to do them?

This platform provides the users with plenty of functionalities that will for sure help them to carry out their tasks:

Shared screen, shared audio, video conferences, file attachment, chat conversation in groups, private and public groups, etc.

Utility: Qualitative +

Observation: frustration

a. Unable to view the list of participants in a particular channel.

b. There’s quite some latency caused while switching between chats, and files.

c. Unable to terminate the inviting call once it has been placed.

2. Questionnaire about new functionalities

What new functionalities that you think could improve your experience with Microsoft teams (Video conferencing module)?

Break-out rooms: Participants in a meeting are able to split into different smaller groups from the main video call. This feature is usually helpful while brainstorming, and hosting group activities that can be monitored by a global host.

Automatic documentation: A feature that always keeps a record of the automatic closed captions generated during a meeting, and uses that as a virtual notepad for further brainstorming and personal note-making.

Rearrange videos: A feature to enable users to drag & drop videos to rearrange the viewing order of the videos during a meeting. This helps to place the video of a person who’s talking right below the computer’s camera to be able to simulate the sense of actually talking to each other.

Instant window switching: A feature to instantly share a different window while sharing content, without the need to stop sharing the current one. This can drastically reduce the time and efforts spent on switching the shared screens instead of actually focusing on the presentation.

Instant swap: A feature that enables users to swap their primary visual between the video and the screen share. This helps presenters to engage the audience while presenting something so that they do not get bored by just looking at the presentation in a big window.

Learnability

Learnability refers to how easy a system is to learn to use. People’s attention span is lowering every year with increasing information load available through the internet. So, users prefer to learn the system as fast as possible or without too much effort.

Learnability: Qualitative

Is it possible for the user to work out how to use the product by exploring the interface and trying out certain interactions? How hard will it be to learn the whole set of functions in a way?

Most of the functions and features are based on commonly used applications for video conferencing, chat, and more.

The range of possibilities this platform offers is wide enough for the user to need more than one attempt to learn them all. We could indeed say that you learn (at least the most basic usage) by trial and error, with more or less time. However, all of the team members agree that the learning curve for this service is steeper than in other tools that satisfy the same use case. This is due partly to the extra functionality that this software provides if compared to other similar services, which adds extra layers of complexity.

We can find explanatory models, and visual content to provide enough context for users to be able to carry out tasks, and operations they are not familiar with.

While trying to learn the whole set of functionalities of the software there are some external factors that influence, Access and permissions for certain features, the necessity of using almost every feature that exists, and the user’s personal goals.



But, all the functionalities and features can be explored and learned by trying them out and by taking help from the Knowledge base / Technical documentation.

Learnability: Quantitative

How much time does a novel user take to get used to the application?

P1 (Moderate proficiency in ICT) — 20–30mins (having used other similar software) to get to a basic level of command of the application. P2 (Moderate proficiency in ICT) — Around half an hour to get started using the application, know where the basic functionality is during the calls, etc. Not an expert user. P3 (Expert in ICT) — 5–10 mins (Having previous mental models of using Slack & Zoom) to get familiar and be able to perform all the necessary tasks.

Some tasks could be learned very quickly, but some rare functionalities require more time in order to be figured out, like for example starting a private chat with someone in the meeting or sharing the meeting information.

Memorability

Memorability refers to how easy a product is to remember how to use, once learned. It is really important for irregular users who use the product, not on a regular basis.

Memorability: Qualitative

What kinds of interface support have been provided to help users remember how to carry out tasks, especially for products and operations they use infrequently?

Here are some of the interface support that could help users to easily perform infrequent tasks,

Maintaining a uniform design language (Icons, context, content tone, colors, typography, and UI components) has helped users a lot to build or reuse their mental models. Easy navigation system. Spotlight helper to have inbuilt tutorials to efficiently use some of the functionalities. Timely display of quick tips or Did you know tips to improve their familiarity with the software. Clearer error messages and automatic suggestions to overcome them.

Memorability: Quantitative

Number of errors when using the system after a while.

We could not measure this metric because, we did not have enough time to get more trials done, but we could guess that it is going to be easy for us to remember the usage of basic features since we are aware of the previously mentioned intuitiveness of the interface and the uniform design language.

As new features are introduced, it would be a steep learning curve to learn each of the from time to time.

Satisfaction

Freedom from discomfort, and positive attitudes towards the use of the product (ISO 9241–11). When users are satisfied with using the product, they will use the product regularly.

Satisfaction: Qualitative

Do users feel comfortable and have a positive attitude towards the system?

We think that even if users face some troubles while attempting to accomplish the tasks, the overall experience is positive. Because users can achieve the main goal: Follow courses and work meetings when they can’t do so face to face. However, some details related to learnability and safety need to be polished to drastically improve the satisfaction factor for users utilizing this tool.

So, even if users are not comfortable enough, they might have no other choice of tool in a professional or an academic institution. Gradually, the users will cultivate the required skills and mental models to be proficient in using the system. This will eventually make them feel comfortable.

Satisfaction: Quantative

Ease of finding information: How easy the UI is to find the required information?

P1: Ease of finding information

P2: Ease of finding information

P3: Ease of finding information

2. Keeping track of location in site: When searching for a piece of information, how well the user is able to keep track of the current location in the app/website?

Predicting information section: How accurately the user is able to predict which section would contain a required piece of information?

P1: Ease of finding information

P2: Ease of finding information

P3: Ease of finding information

3. Overall Feedback

Usability order: After analysis

After performing usability analysis, and the tasks we have inferred the Usability attributes order as shown in the below,

Effectiveness

Users can perform every task they need in lectures and work meetings. Every possible need that a user could have in a video call situation is addressed in a streamlined manner.

2. Efficiency

Users can figure things out and perform their tasks (normally) in no more than 15–20 seconds.

3. Memorability

Even if not the most intuitive, once you figure out how to perform a task, it is hard to forget, partly due to the simplicity and intuitiveness of the interface.

4. Satisfaction

The fact that numerous academic and professional teams have repeatedly used this platform (when there are countless alternatives) shows us that this platform has achieved its purpose of satisfying the needs of users for this niche.

5. Utility

You can perform all the necessary tasks with but, the problem lies in the time and effort it takes to do them. And, that’s when errors come in.

6. Safety

The users make a lot of mistakes unknowingly trying to navigate the interface, and sometimes they find it hard or impossible to recover from them.

7. Learnability

It’s hard for first-time users since the design language and the context is different when compared to other video conferencing applications. The learning curve is steeper than with other applications that perform the same basic task.

Usability order: Our order

But, we think some reshuffling and reordering can be done to maintain better overall usability.

Important attributes

1. Efficiency

In a digital environment, speed should always be a top priority. Getting as close to a physical meeting experience as possible means that users should not have to struggle and waste time with the digital intricacies of software. Tasks should be performed quickly and effectively.

2. Effectiveness

Getting things done with high accuracy is important in both professional and academic institutions (our target groups). In a remote setting, being able to accomplish the tasks is really satisfying for the users.

3. Safety

In the scenarios which our target audience faces when using this software, making a mistake can in some cases be catastrophic. Therefore, it should be implicit to this kind of service that the chance to make an irreversible mistake is as minimal as possible.

Neutral attributes

1. Learnability

Microsoft Teams has found a niche in professional and academic relations. Therefore, we think Learnability should not be as prioritized since users typically use this service for extended periods of time and frequency, and have time to get used to the interface. Even if the learning curve for this software is steeper than in others, the more advanced functionality will make up for this.

2. Utility

A phenomenon that we have observed is that team dynamics tend to adapt to the medium in which they are carried out. Therefore, we consider that including too many functionalities to the service could make it too difficult to navigate through, and even alienate the more amateur users.

3. Satisfaction

This is not software intended for entertainment. However, this does not mean that we should not pay attention to the sentiment that users will have towards our software. Even if this is the case, we consider that, due to the fact that this program is meant to be used in a serious, academic, or professional way, we should not make this attribute a top priority.

4. Memorability

Seeing as this service is designed to structure an educational or professional communications infrastructure around, we consider that the users will be in contact with this service frequently enough to get acquainted with it without us from a design perspective having to go to excessive efforts to oversimplify the user interface.