Responsible
Technology
Review Template
Gut-checks and guidelines fordeveloping or deploying new tech1

Atlassian| FY2023
Sustainability
Responsible
Technology
Review
Template Report

Introduction Planet
Customers
Introduction
ProjectPeople
Summary
PrincipleCommunity
1 PrincipleGovernance2 PrincipleAppendix
3 Principle 4

Principle 5Introduction“How do I use this thing?”

A note aboutalignment ratings

1. Step through this template as a project team andEach section focuses on a different guiding principle.

don’t expect to work through the whole thing in one

Along with specific questions to consider, you’ll also

go. You may need to split off to gather information

rate how well the project aligns with that principle.

or talk to stakeholders, then come back together.

We’ve found a simple red/yellow/green system

This template is based on Atlassian’s
Responsible Technology Principles
and helps teams build a sharedunderstanding of how their project
affects employees, customers, andcommunity stakeholders. By being
thoughtful in the use and developmentof technology, we can move fast andnot break things.

Key Takeaways

2. Make sure the project’s lead is involved

works well. Here’s what each rating means.

throughout. This person might be called the owner,
approver, directly responsible individual (DRI), or
some other clever name your company has invented.
3. Tip: you don’t have to complete a responsible
technology review before beginning development
work. But you’ll be more objective in your thinking(and end up with less re-work) if you do.
4. Treat this as a living document. Come back andreassess if the project pivots direction, the user
persona changes, you go from beta to GA, etc.

FEELS GOOD
You’re confident this use of technology aligns withthe relevant principle.
NEEDS WORK
You’ve uncovered risks or potential issues. That’s OK!
Reassess your plans and adapt as needed.
NOT ALIGNED
You’ve identified ethical issues and/or problems thatmight emerge later.

5. Tip: use this template to assess the technology
you’re building or buying for internal use, too.

2

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Project summary
Project or product
DriverTeam members

Description

Resources

Date of most recent
technology review
Overall confidence

FEELS GOOD

NEEDS WORKNOT ALIGNED

Further actions

3

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 1: Empower all humans

Technologies should be open, inclusive, fair, and just to reflect the fundamental
rights we all share.

How will customers, employees, or other users experience this tech?
We don’t know our own blind spots. To maximize fairness and inclusivity, we need to think about how all users will be affected.
THEME

DISCUSSION PROMPTS

Who will use this tech? Who else may be affected by it?
y Will the outputs of this tech be used to make decisions?
y If so, who will those decisions be made about?
y Are any of these stakeholders part of a demographic group that is underrepresented or historically marginalized(due to race, gender, age, language, religion, sexual orientation, disability, etc.)?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

Answer:Fairness forusers andstakeholders

How will those users and affected persons experience the tech, and how might this differ between user groups?
y What are the potential benefits for those users and affected persons?
y What are the potential harms they could experience?
y How might the experience differ for users in an underrepresented or historically marginalized demographic group?

What is our plan to improve the risk level in the future?

Answer:4

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 1 (cont.)
How will customers, employees, or other users experience this tech?
We don’t know our own blind spots. To maximize fairness and inclusivity, we need to think about how all users will be affected.
THEME

DISCUSSION PROMPTSIs there a risk that similarly situated users/people won’t have the same experience as one another?
Answer:Fairness forusers andstakeholders(cont.)As a result, could this tech (not limited to, but particularly AI) result in unfair biases for/against certain groups
of users/people?
Answer:5

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 1 (cont.)
How will customers, employees, or other users experience this tech?
We don’t know our own blind spots. To maximize fairness and inclusivity, we need to think about how all users will be affected.
THEME

DISCUSSION PROMPTS

Do we have the right people in the room to properly consider and assess the questions around fairness above?
y Does the team reflect the diversity of the societies we live in and the teams we serve? Think about this broadly, interms of differences in characteristics, backgrounds, experiences and perspectives as well as in the particular crafts
and functions represented on the team.
y Does the work conducted by the team reflect a diversity of perspectives (e.g., in research, data, testing)?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

Answer:Inclusivity

What is our plan to improve the risk level in the future?

If not, have we or could we involve a broader range of people and stakeholders throughout the project?
Answer:6

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 2: Unleash potential (not inequity)

If we use technologies responsibly and intentionally, we can contributeto better outcomes across our communities.

What are the known and potential uses?
It’s our responsibility to control for bad outcomes to the best of our ability and drive toward good outcomes. That starts with ensuring this tech is fit for purpose.
THEME

DISCUSSION PROMPTSALIGNMENT RATING

Is the tech being used a good fit for the intended purpose and use cases, as described in your project summary above?

FEELS GOOD

Answer:

NEEDS WORKACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NOT ALIGNED

Use the righttool for the job

Are there circumstances where the tech doesn’t suit the use case?
y How is the tech necessary for, or otherwise a good fit for, the use case?

What is our plan to improve the risk level in the future?

Answer:7

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 2 (cont.)
What are the known and potential uses?
It’s our responsibility to control for bad outcomes to the best of our ability and drive toward good outcomes. That starts with ensuring this tech is fit for purpose.
THEME

DISCUSSION PROMPTS

Does this tech fit with your company’s mission and values?
y How might it benefit communities or do other good in the world? (Don’t be afraid to think big!)y Does it align with your company’s public commitments on issues like human rights, DEI, fair employment, etc.?
Answer:Use the righttool for the job(cont.)

8

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 2 (cont.)
What are the known and potential uses?
It’s our responsibility to control for bad outcomes to the best of our ability and drive toward good outcomes. That starts with ensuring this tech is fit for purpose.
THEME

DISCUSSION PROMPTS

Could use or misuse result in unintended consequences (including bias, toxicity, and other adverse impacts on individuals)?
y Will this change as the tech is adopted and used over time and feedback loops are created?
y How have we tested, evaluated, verified or validated this (if at all)?
Answer:Risks
and impacts
What is the worst-case scenario of misuse or failure?
y How might a supervillain abuse this tech?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

What is our plan to improve the risk level in the future?

Answer:9

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 2 (cont.)
What are the known and potential uses?
It’s our responsibility to control for bad outcomes to the best of our ability and drive toward good outcomes. That starts with ensuring this tech is fit for purpose.
THEME

DISCUSSION PROMPTS

What is the best-case scenario of use?
Answer:Risks
and impacts(cont.)

What more could we do to drive towards the best-case scenario and control for or mitigate the potential for the worst-casescenario? What are the trade-offs (if any) involved in doing so?
Answer:10

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Principle 3: Build for trust

Responsible Technology Review Template

Key Takeaways

Trust isn’t just about ensuring the security and privacy of technology products, but is also
earned and kept by demonstrating a commitment to reliability and performance.

How does the technology align with our commitments around trust?
New technologies (including AI) come with unknowns about their quality, accuracy, and reliability. Choice, control, and candor go a long way in building trust.
THEME

DISCUSSION PROMPTSWill this tech interact with data in a way that is aligned with the expectations we set with customers and employees?
y Would we lose the trust we’ve built with customers and employees if it interacts with data in a way that is new ordifferent?
y Do we fully understand the commitments we’ve made publicly to customers and internally to employees?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

Answer:Data integrity

What is our plan to improve the risk level in the future?

11

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 3 (cont.)
How does the technology align with our commitments around trust?
New technologies (including AI) come with unknowns about their quality, accuracy, and reliability. Choice, control, and candor go a long way in building trust.
THEME

DISCUSSION PROMPTS

Do we offer users or individuals choice in how they participate or whether they participate at all?
y Describe the applicable controls (e.g., opt-in, opt-out) and choices offered to individuals and where they apply.
Answer:

Choiceand control

What privacy controls and privacy-enhancing technologies (if any) can we apply to provide customers and employees with
better protection?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

What is our plan to improve the risk level in the future?

Answer:12

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 3 (cont.)
How does the technology align with our commitments around trust?
New technologies (including AI) come with unknowns about their quality, accuracy, and reliability. Choice, control, and candor go a long way in building trust.
THEME

DISCUSSION PROMPTS

Are the datasets that we intend to use suitable for our purposes?
y Think about the use cases, stakeholders, geographic areas and data requirements set out above.
Answer:Quality andperformanceHas the tech been tested, validated, and proven for this use case?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

What is our plan to improve the risk level in the future?

Answer:13

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 3 (cont.)
How does the technology align with our commitments around trust?
New technologies (including AI) come with unknowns about their quality, accuracy, and reliability. Choice, control, and candor go a long way in building trust.
THEME

DISCUSSION PROMPTS

How complex are the tasks that the tech must complete?
y If a group of humans had to perform this task, would they easily agree on the “right” way to do it and the “right”
outcome?
Answer:Quality andperformance(cont.)How complex is the environment where this tech will be deployed?
y How might this complexity contribute to unpredictable issues or changes over time(e.g., system drift)?
Answer:14

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 4:Open communication, no bullshit

With new technologies like AI, just telling users that a feature is AI-powered doesn’tmean much unless the user also understands what it means when the feature is in use,
how to best interact with it, and how to respond to (and if needed, question) its outputs.

What will we communicate about this tech and the data that powers it?
Some disclosures are (or will soon be) mandatory, others aren’t. Regardless, many of the issues identified in this review can be solved by being open with users – particularly for AI systems that mimic human interactions (like chatbots)and generative AI.
THEME

DISCUSSION PROMPTS

What do we know (and need to ensure that customers and users know) about the benefits and limits of the tech?
y Are there situations where we know it won’t perform at its best?
y Are there potential uses that we need to explicitly restrict?
y What uses are not supported by us (including where the tech has not been designed or tested for them)?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

Answer:What tocommunicate

What is our plan to improve the risk level in the future?

15

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 4 (cont.)
What will we communicate about this tech and the data that powers it?
Some disclosures are (or will soon be) mandatory, others aren’t. Regardless, many of the issues identified in this review can be solved by being open with users – particularly for AI systems that mimic human interactions (like chatbots)and generative AI.
THEME

DISCUSSION PROMPTSCan we explain to our customers and people (including potential employees) how we thought through the risks of this tech?
Can we provide a satisfying explanation for why we intend to proceed?
Answer:What tocommunicate(cont.)

16

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 4 (cont.)
What will we communicate about this tech and the data that powers it?
Some disclosures are (or will soon be) mandatory, others aren’t. Regardless, many of the issues identified in this review can be solved by being open with users – particularly for AI systems that mimic human interactions (like chatbots)and generative AI.
THEME

DISCUSSION PROMPTS

If we are using AI for this project, how might we best inform users that an AI system is in use?
Answer:

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

Must-havedisclosures

What else might we need to give users a heads-up about?
y What level of disclosure is most appropriate based on the context?

What is our plan to improve the risk level in the future?

Answer:17

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 4 (cont.)
What will we communicate about this tech and the data that powers it?
Some disclosures are (or will soon be) mandatory, others aren’t. Regardless, many of the issues identified in this review can be solved by being open with users – particularly for AI systems that mimic human interactions (like chatbots)and generative AI.
THEME

DISCUSSION PROMPTS

When and how will we tell customers and users this information?
Answer:

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

Communicationplan

Are there any behavioral nudges or notifications we can build into the product itself to help the customer remember thelimitations and risks?

What is our plan to improve the risk level in the future?

Answer:18

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 5:
Accountability is a team sport

Any tech is part of a larger ecosystem. It’s important to take user feedback seriously
and to encourage other technology makers to adopt these practices, too.

How will we hold ourselves and our partners accountable for the impacts of this tech?
Some level of human involvement and oversight is needed in order for responsible tech principles to be effective.
THEME

DISCUSSION PROMPTS

What is the level of human involvement in the tech’s operation?
y Is that level appropriate (and helpful)?
Answer:Humanoversight
Who will make decisions about whether to employ the tech for particular tasks?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

What is our plan to improve the risk level in the future?

Answer:19

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 5 (cont.)
How will we hold ourselves and our partners accountable for the impacts of this tech?
Some level of human involvement and oversight is needed in order for responsible tech principles to be effective.
THEME

DISCUSSION PROMPTS

Who will be responsible for troubleshooting, managing, operating, overseeing, and controlling the tech over its lifecycle?
Answer:Humanoversight(cont.)

20

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 5 (cont.)
How will we hold ourselves and our partners accountable for the impacts of this tech?
Some level of human involvement and oversight is needed in order for responsible tech principles to be effective.
THEME

DISCUSSION PROMPTS

If we’re using a third-party vendor or partner…
y Do their reputation and values align with ours?
y How would using them reflect on our own reputation?
y What are their commitments with respect to AI ethics or responsible technology? Can you confirm whether their actual
capabilities deliver on this?

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

Answer:Third parties

What is our plan to improve the risk level in the future?

21

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 5 (cont.)
How will we hold ourselves and our partners accountable for the impacts of this tech?
Some level of human involvement and oversight is needed in order for responsible tech principles to be effective.
THEME

DISCUSSION PROMPTS

Do we have any leverage to influence vendors, partners or other stakeholders in this project to implement or further
develop any of their ethical practices and processes?
Answer:Third parties(cont.)Can we use our relationships with third parties and stakeholders to learn from them?
Answer:22

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Responsible Technology Review Template

Key Takeaways

Principle 5 (cont.)
How will we hold ourselves and our partners accountable for the impacts of this tech?
Some level of human involvement and oversight is needed in order for responsible tech principles to be effective.
THEME

DISCUSSION PROMPTS

Are there mechanisms for users to report issues with the tech?
Answer:

ALIGNMENT RATINGFEELS GOOD

ACTIONS AND IMPROVEMENTS

What have we already done to improve the risk level?

NEEDS WORKNOT ALIGNED

User feedback

Do we have other mechanisms to obtain user and stakeholder feedback?
y Are they reactive and ad-hoc, or proactive and regular?
y If we have multiple feedback streams coming in, how will we sort through and prioritize the feedback we get?

What is our plan to improve the risk level in the future?

Answer:23

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review Template

Principle 5 (cont.)
How will we hold ourselves and our partners accountable for the impacts of this tech?
Some level of human involvement and oversight is needed in order for responsible tech principles to be effective.
THEME

DISCUSSION PROMPTS

How will we respond to and, if appropriate, implement that feedback?
Can we specify a cadence and process?
Answer:User feedback(cont.)

24

Introduction

Project Summary

Principle 1

Principle 2

Principle 3

Principle 4

Principle 5

Key Takeaways

Responsible Technology Review TemplateKeytakeaways
Use this extra space
to expand on yourtakeaways and to-dos
following the review.

For example:
What did we learn from this review?
What other questions do we need to ask and who else
do we need to engage?
What are the “unknowns” we haven’t been able
to answer?

25

26