AWS CodeArtifact - the long-awaited feature
Tags: aws, cicd, devops
Published: 2021-07-10 10:37:37

---
title: AWS CodeArtifact - the long-awaited feature
published: true
description: AWS CodeArtifact, one of the features of CodePipeline
tags: #aws #cicd #devops
cover_image: https://www.pawelpiwosz.net/assets/article-codeartifact/cover.png
---

## The background

I have to admit, I am not a fan of AWS Codepipeline. Even now, after a few project where I heavily used CodePipeline, I am still not big fan. For me it is rather simple solution. Of course it can do the job, but in my private world, I used to use TravisCI before and now… Azure DevOps for AWS deployments.

And one of great advantages of Azure DevOps is Artifacts, where you can store and publish your packages. Something what we missed in AWS Codepipeline for long time. Until now.

## AWS CodeArtifact

Not long time ago AWS announced this service. Now it is official. And it is a great news, indeed. Until today we had to use workarounds (let’s call it this way). S3 buckets, or build the artifact solution by our own.

This is not longer needed. AWS CodeArtifact is taking a great place in the CodePipeline services. First 2G of stored packages is free (monthly), also first 100K requests are free.

## Let’s get started

![AWS CodeArtifacts splash screen](https://www.pawelpiwosz.net/assets/article-codeartifact/cover.png)

What is important to understand, it is the domain concept. More explanation is in the Steve Roberts’ blog, here [https://aws.amazon.com/blogs/aws/software-package-management-with-aws-codeartifact/](https://aws.amazon.com/blogs/aws/software-package-management-with-aws-codeartifact/). Important to remember is that Domain aggregates all repositories. So, we are consuming the package from the repository, or even multiple repositories, but in fact, this package is stored only once across the Domain. This saves space and possible costs.

## Domain

So, let’s create the Domain! We can do it from AWS GUI, but this time, I will work with CLI. Of course, I assume you know how to configure CLI, if not, there is plenty of articles how to do it. Remember, you need the newest version of awscli, I recommend to use version 2.

![AWSCLI version 2. Notice, in my example it is Powershell version](https://www.pawelpiwosz.net/assets/article-codeartifact/console.png)

Let’s check our domains now:

```json
aws codeartifact list-domains
{
    "domains": []
}
```

You should receive empty JSON.

So, it is time to create the first Domain. The process is very easy

```json
aws codeartifact create-domain --domain mytestdomain 
{
    "domain": {
        "name": "mytestdomain",
        "owner": "012345678",
        "arn": "arn:aws:codeartifact:eu-west-1:0123456789:domain/mytestdomain",
        "status": "Active",
        "createdTime": "2020–06–28T10:46:50.193000+02:00",
        "encryptionKey": "arn:aws:kms:eu-west-1:0123456789:key/aaaa1111-aa11–1234–1234-aa11aa11aa11",
        "repositoryCount": 0,
        "assetSizeBytes": 0
    }
}
```

And… That’s it! Let’s check it again

```json
aws codeartifact list-domains 
{
    "domains": [
    {
        "name": "mytestdomain",
        "owner": "0123456789",
        "status": "Active",
        "encryptionKey": "arn:aws:kms:eu-west-1:0123456789:key/aaaa1111-aa11–1234–1234-aa11aa11aa11"
        }
    ]
}
```

```json
aws codeartifact describe-domain --domain mytestdomain 
{
    "domain": {
    "name": "mytestdomain",
    "owner": "0123456789",
    "arn": "arn:aws:codeartifact:eu-west-1: 0123456789:domain/mytestdomain",
    "status": "Active",
    "createdTime": "2020–06–28T10:46:50.193000+02:00",
    "encryptionKey": "arn:aws:kms:eu-west-1:0123456789:key/aaaa1111-aa11–1234–1234-aa11aa11aa11",
    "repositoryCount": 0,
    "assetSizeBytes": 0
    }
}
```

## Repository

Domain is already created, so we can create the repository.

But first, we need (again) to look on the construction here. We are in the Domain. Now, we can create Repository, and then Upstream Repository. By this approach you can separate your project repositories (by upstreams) and still be able to access all of them with one endpoint.

So, let’s create the repository first.

```json
aws codeartifact create-repository --domain mytestdomain --repository "mymainrepo" --description "Main repository" 
{
    "repository": 
    {
        "name": "mymainrepo",
        "administratorAccount": "01234567890",
        "domainName": "mytestdomain",
        "domainOwner": "01234567890",
        "arn": "arn:aws:codeartifact:eu-west-1: 01234567890:repository/mytestdomain/mymainrepo",
        "description": "Main repository",
        "upstreams": [],
        "externalConnections": []
    }
}
```

Of course, we can look on it, using:

```bash
aws codeartifact list-repositories

aws codeartifact describe-repository --domain mytestdomain --repository mymainrepo
```

In order to login to the repository, you can follow the instruction available in the GUI console

![AWS is giving very straightforward connection instruction](https://www.pawelpiwosz.net/assets/article-codeartifact/artifact1.png)

The connection command is simple (but long)

```bash
aws codeartifact login --tool pip --repository mymainrepo --domain mytestdomain --domain-owner 01234567890

Successfully logged in to codeartifact for pip.
```

For configuring pip itself, we need to run two additional commands. First to get authorization token:

```bash
aws codeartifact get-authorization-token --domain mytestdomain --domain-owner 0123456789 --query authorizationToken --output text
```

second, to use it in our pip config

```bash
pip config set global.index-url [https://aws:HEREGOESTHETOKEN@mytestdomain-0123456789.d.codeartifact.eu-west-1.amazonaws.com/pypi/mymainrepo/simple/](https://aws:HEREGOESTHETOKEN@mytestdomain-0123456789.d.codeartifact.eu-west-1.amazonaws.com/pypi/mymainrepo/simple/)

Writing to C:\Users\pawel\AppData\Roaming\pip\pip.ini
```

As you can see, I’m using Powershell here.

Now we can check, if our pip is updated, by running pip config list. You should be able to see your token there.

## Upstream repository

Let us follow the good practice, and now it is the time to create upstream repositories. We will add npm and pip in one shot,

```json
aws codeartifact update-repository --repository mymainrepo --domain mytestdomain --domain-owner 0123456789 --upstreams repositoryName=pypi-store repositoryName=npm-store
{
    "repository": {
        "name": "mymainrepo",
        "administratorAccount": "0123456789",
        "domainName": "mytestdomain",
        "domainOwner": "0123456789",
        "arn": "arn:aws:codeartifact:eu-west-1:0123456789:repository/mytestdomain/mymainrepo",
        "description": "Main repository",
        "upstreams": [
            {
                "repositoryName": "pypi-store"
            },
            {
                "repositoryName": "npm-store"
            }
        ],
        "externalConnections": []
    }
}
```

Thanks to that, we are able to work with one endpoint for all of ours npm and pip packages. If it works for you — it is up to you.

Be aware of search order here. If you are requesting the package, your request is going through the upstreams in the create-repository order (or update — repository). If you are curious, what is the order, simply run describe-repository, and you will see upstreams in the search order.

## External connections

When we finish our repos, we can create external connections to them. Let’s do it for both of our repos:

For pip:

```bash
aws codeartifact associate-external-connection --domain mytestdomain --domain-owner 0123456789 --repository pypi-store --external-connection "public:pypi"
```

And for npm:

```bash
aws codeartifact associate-external-connection --domain mytestdomain --domain-owner 0123456789 --repository npm-store --external-connection “public:npmjs”
```

Although, it is possible you will receive an error, as this connection already exists.

## Let’s test our new shiny repository

So. We are ready to test our repo. We gonna install some pip package. I’ve created empty virtualenv, so we can install for example awscli. What we expect?

![What is happening when we want to install package?](https://www.pawelpiwosz.net/assets/article-codeartifact/schema.png)

Let me go through this schema. We want to install package X. We configured our pip in the way, it is looking to our AWS CodeArtifact first. If the requested package is not there, it will connect to main public one and download it for us (with all dependencies). And then, the package will be installed from our AWS CodeArtifact repo.

Let’s install our package then:

```bash
pip install awscli
Looking in indexes: [https://aws:****@mytestdomain-0123456789.d.codeartifact.eu-west-1.amazonaws.com/pypi/mymainrepo/simple/](https://aws:****@mytestdomain-043144402336.d.codeartifact.eu-west-1.amazonaws.com/pypi/mymainrepo/simple/)
Collecting awscli
```

You easily can spot, pip is trying to collect the package from our repository. Now, the package is installed, let’s check our repository:

![The requested packages are now available from our repository](https://www.pawelpiwosz.net/assets/article-codeartifact/artifacts2.png)

And voila! We successfully created, configured and used our CodeArtifact service!

Of course, our packages can be published into our repository. AWS CodeBuild and CodeDeploy can use our repository. In the essence, AWS CodePipeline finally has the proper solution for artifacts!

## Summary

I really like the AWS CodeArtifact. It is easy to configure, easy to use. The idea with Domains and Upstreams is pretty nice.

The service is fast, reliable and managed by AWS. What else to expect? I strongly recommend CodeArtifact.

Important is, it can be used externally. not only cross-account inside AWS, but also as a proper external packages repository.

Currently it covers Python, Nodejs and Java repositories (edit: and C#), but I’m sure, we will receive more soon!  
_I wrote this article on Medium about one year ago. Time for migration :)_