A guide to Maven Repositories
Tags: opensource, tutorial, discuss, java
Published: 2021-07-14 18:17:13

We at Appwrite recently published our Android SDK to Maven Central and we learnt a whole lot about the Gradle and Maven ecosystem in the process. 

In this series of articles, we'll aim to teach you how you can build and publish your own Android Libraries using Gradle. Before we dive into building and publishing artifacts with Gradle, we need to get familiar with Maven because after all, Gradle is a build tool for Java based projects ( among several others ).

If you've worked on a Java project, it's highly likely that you would've heard of Maven, as it is the most popular build tool for Java projects. Most of us associate Maven with a `pom.xml` file. However, there's much more to Maven than meets the eye and you can very quickly get lost in a sea of knowledge. This article will aim to walk you through the important concepts in Maven and introduce you to various terminologies in the Maven ecosystem.

## 🗃️ The POM file

The POM file is the cornerstone of any Maven project. Think of it as an entry point to any Maven project. The POM file has all the information required to build the project; a recipe book of sorts. If you're familiar with the `package.json` file in **node** projects, `composer.json` file in **php** projects, `requirements.txt` in **python** projects and `build.gradle` in other **Java/Kotlin** projects, the POM file should seem familiar.

Though we will be using Gradle for building and publishing artifacts, it's a good idea to learn about the structure of the POM file as these concepts can be extended to Gradle as well.

Let's take a sample POM file to understand its structure. You can use the following command to create a simple Maven project if you want to test it out yourself. 

```sh
mvn -B archetype:generate -DgroupId=com.mycompany.app -DartifactId=my-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4
```

Once you have executed this command, you will notice a few things have happened. First, you will notice that a directory named `my-app` has been created for the new project, and this directory contains a file named `pom.xml` that should look like this:

```xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
 
  <groupId>com.mycompany.app</groupId>
  <artifactId>my-app</artifactId>
  <version>1.0-SNAPSHOT</version>
 
  <name>my-app</name>
  <!-- FIXME change it to the project's website -->
  <url>http://www.example.com</url>
 
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.7</maven.compiler.source>
    <maven.compiler.target>1.7</maven.compiler.target>
  </properties>
 
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.11</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
 
  <build>
    <pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
       ... lots of helpful plugins
    </pluginManagement>
  </build>
</project>
```

`pom.xml` contains the **Project Object Model (POM)** for this project. The POM is the most basic unit of work in Maven. This is important to remember because Maven is inherently **project-centric** in that everything revolves around the notion of a project. In short, the **POM** contains every important piece of information about your project and is the go-to place for finding anything related to your project.

This is a very simple POM but still displays the key elements every POM contains, so let's walk through each of them to familiarize you with the POM essentials:

* **project** This is the top-level element in all Maven `pom.xml` files.

* **modelVersion** This element indicates what version of the object model this POM is using. The version of the model itself changes very infrequently but it is mandatory in order to ensure stability of use if and when the Maven developers deem it necessary to change the model.

* **groupId** This element indicates the unique identifier of the organization or group that created the project. The **groupId** is one of the key identifiers of a project and is typically based on the fully qualified domain name of your organization. For example `org.apache.maven.plugins` is the designated groupId for all Maven plugins.

* **artifactId** This element indicates the unique base name of the **primary artifact** being generated by this project. The primary artifact for a project is typically **a JAR file**. Secondary artifacts like source bundles also use the artifactId as part of their final name. A typical artifact produced by Maven would have the form <artifactId>-<version>.<extension> (for example, `myapp-1.0.jar`).

* **version** This element indicates the version of the artifact generated by the project. Maven goes a long way to help you with version management and you will often see a **SNAPSHOT** designator in a version, which indicates that a project is in a state of development. We will discuss the use of snapshots and how they work further on in this guide.

* **name** - This element indicates the display name used for the project. This is often used in Maven's generated documentation.

* **url** This element indicates where the project's site can be found. This is often used in Maven's generated documentation.

* **properties** This element contains value placeholders accessible anywhere within a POM.

* **dependencies** This element's children list the dependencies. The cornerstone of the POM.

* **build** This element handles things like declaring your project's directory structure and managing plugins.

For a complete documentation of all the POM attributes, you can [refer to the official docs](https://maven.apache.org/ref/3.8.1/maven-model/maven.html). 

## 🧩 Artifacts 

An **artifact** is something that is either produced or used by a project. Examples of artifacts produced by Maven for a project include: **JARs ( Java Archives )**, **source** and **binary** distributions, **WARs**, **AARs** ( Android Archives ) etc. Each artifact is identified by a **group id**, an **artifact ID**, **a version**, **an extension** and a **classifier**. 

Take a look at [Appwrite's Android SDK](https://repo1.maven.org/maven2/io/appwrite/sdk-for-android/0.0.1/). You will find the following artifacts ( and more ). 

```
sdk-for-android-0.0.1-javadoc.jar                  2021-06-29 15:13       261            
sdk-for-android-0.0.1-sources.jar                  2021-06-29 15:13     18663      
sdk-for-android-0.0.1.aar                          2021-06-29 15:13     46437      
sdk-for-android-0.0.1.pom                          2021-06-29 15:13      3277
```

You will find a lot of `.md5`, `.sha1`, `.sha256`, `.sha512` and `.asc` files which are all used by Maven Central and other consumers for verifying the signature and authenticity of your artifacts.

You will notice that, these artifacts follow a naming convention as mentioned below.

```
<artifactId>-<version>-<classifier>.<extension>
```

> Note : **classifier** is an optional field and is absent in the case of Primary Artifacts which in the case of our Android project are the aar and pom files.

In our case the following artifacts are the primary artifacts
* **sdk-for-android-0.0.1.aar**            
* **sdk-for-android-0.0.1.pom**   

Our `aar` file can contain **Android resources** and a **Manifest file**, which allows you to bundle in shared resources like layouts and drawables in addition to all the Java classes and methods. Our `pom` file contains information about our library including all the dependencies we use internally. 

And the following are the secondary artifacts ( which are supplementary to the primary artifacts and are not necessarily required to build your project. )

* **sdk-for-android-0.0.1-javadoc.jar**     
* **sdk-for-android-0.0.1-sources.jar** 

When we talk about publishing our project in the next article, we will be referring to all these artifacts.

## 🗺️ Maven Coordinates

We've talked about the pom file and primary and secondary artifacts. In this section we will dive into the structure of a Maven repository and understand how Maven resolves dependencies when you build your project! 

What's a **Maven dependency** you ask ? A **Maven Dependency** is any artifact that your project depends on. A typical dependency declaration in a Maven project looks like this

```xml
<dependencies>
  <dependency>
    <groupId>io.appwrite</groupId>
    <artifactId>sdk-for-android</artifactId>
    <version>0.0.1</version>
  </dependency>
</dependencies>
```

Together, the **groupID**, **artifactID**, and **version** make up the [Maven Coordinates](https://maven.apache.org/pom.html#Maven_Coordinates). Every Maven artifact is identified by these coordinates. 

The next question arises. **Given the Maven coordinates, how does maven locate an artifact?** Maven uses the coordinates values for a given dependency to construct a URL according to the [maven repository layout](https://cwiki.apache.org/confluence/display/MAVENOLD/Repository+Layout+-+Final).

Let's take the example of **Appwrite's Android SDK** with the following coordinates 

```xml
<dependency>
  <groupId>io.appwrite</groupId>
  <artifactId>sdk-for-android</artifactId>
  <version>0.0.1</version>
</dependency>
```

According to the [official specification](https://cwiki.apache.org/confluence/display/MAVENOLD/Repository+Layout+-+Final), the URL for primary artifacts looks like:

```
/$groupId[0]/../${groupId[n]/$artifactId/$version/$artifactId-$version.$extension
```

The **groupId** array in the URL is formed from the `<groupId>` element in the pom file as follows 

> $groupId is a array of strings made by splitting the groupId’s on “.” into directories.

So for the groupId value of `org.example.subdomain`, our `$groupId` array would be `[org, example, subdomain]`, which when translated into directories, becomes `org/example/subdomain`. 

In our case, the groupId `io.appwrite` is translated to the following path `io/appwrite/`.

Our next step is to build the URL for the **primary** and **secondary** artifacts.

### 🔧 **Primary Artifacts**

One of the core features of Maven is its ability to handle [Transitive Dependencies](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies). That is, to find and download the dependencies of your dependencies, and their dependencies and so on until all of dependencies are satisfied.

Just how your own Maven project has a `pom.xml` file listing its main dependencies, those dependencies also have a remote `pom` file serving a similar purpose. Maven uses this file to figure out what other dependencies to download. When a coordinate does not contain a `<classifier>`, it is considered a primary artifact and is expected to have a pom available.

In our case, the primary artifacts are the `.pom` file and the `.aar` file.

* **pom** - If we follow the URL convention above, the pom file for Appwrite's Android SDK can be located at 

```
io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1.pom
```

* **aar** - If we follow the URL convention above, the aar file for Appwrite's Android SDK can be located at 

```
io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1.aar
```

### ⛏️ **Secondary Artifacts**

Secondary artifacts, or “attached artifacts” on the other hand are dependencies that you want Maven to download that are ancillary to your project. Most often they are used to download the `javadocs` and/or `sources` for a particular dependency. However, unlike a primary artifact, a secondary artifact is not expected to have a remote `pom` and thus never has any dependencies.

They can be specified in the `<dependencies>` section just like primary artifacts:

```xml
<dependency>
  <groupId>io.appwrite</groupId>
  <artifactId>sdk-for-android</artifactId>
  <version>0.0.1</version>
  <classifier>sources</classifier>
</dependency>
```
The URL for secondary artifacts is similar to that of the primary artifacts except that they have an optional `$classifier` variable.

```
/$groupId[0]/../$groupId[n]/$artifactId/$version/$artifactId-$version-$classifier.$extension
```

In our case, the secondary artifacts are the `javadocs` and the `sources`.

* **javadoc** - If we follow the URL convention above, the `javadoc` file for Appwrite's Android SDK can be located at 

```
io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1-javadoc.jar
```

* **sources** - If we follow the URL convention above, the sources file for Appwrite's Android SDK can be located at 

```
io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1-sources.jar
```

### 🏁 **Checksums**

To verify the downloaded artifacts Maven computes the `md5`, `sha1`, `sha256` and `sha512` checksum for that artifact and compares it to the values found in the checksum files located at **$ARTIFACT_URL.md5**, or **$ARTIFACT_URL.sha1**, respectively.

> This is strictly meant as a way to quickly verify downloads, and it is NOT meant to be used for authentication or security purposes. This is also NOT a substitute for using HTTPS, as checksums can be trivially intercepted and modified along with the modified artifacts.

* **sha1** - the `sha1` checksum for our aar artifacts for example, would be present at 
  ```
  io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1.aar.sha1
  ```
* **md5** - the `md5` checksum for our aar artifacts for example, would be present at 
  ```
  io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1.aar.md5
  ```
  Similarly for all other artifacts (javadoc.jar.md5, pom.md5, sources.jar.md5 and so on).


### 🔐 **Signatures**

  The **Maven Central** repository makes in mandatory to include a signature for all the uploaded artifacts and checksums. This ensures absolute confidence in the authenticity of the downloaded artifacts. Every **artifact** and **checksum** has a corresponding `.asc` file that contains the signature of the author of the artifact.  

  * **$artifacts.asc** 

    For our `.aar` file for example, the corresponding `.asc` file would be located at 
    ```
    io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1.aar.asc
    ```
  * **$checksum.asc** 

    For our `.md5` ( sha1, sha256 etc.) files, the corresponding `.asc` file would be located at 
    ```
    io/appwrite/sdk-for-android/0.0.1/sdk-for-android-0.0.1.aar.md5.asc
    ```

## 📦 The Maven Repository

A Repository is Maven holds artifacts and dependencies of all kinds. There are exactly two kinds of repositories 

1. Local 
    The **local repository** is a directory on the computer where Maven runs. It caches remote downloads and contains temporary build artifacts that you have not yet released. This is usually located in `$HOME/.m2/repository`

2. Remote
    **Remote repositories** refer to any other type of repository, accessed by a variety of protocols such as file:// and https://. These repositories might be a truly remote repository set up by a third party to provide their artifacts for downloading (for example, **repo.maven.apache.org**). Other **remote** repositories may be internal repositories set up on a file or HTTP server within your company, used to share private artifacts between development teams and for releases.

Local and remote repositories are structured the same way so that scripts can run on either side, or they can be synced for offline use. The layout of the repositories is completely transparent to the Maven user, however.

The last topic that we'll discuss is the concept of a **Release** repository and a **SNAPSHOT** repository. 

### 🏆 **Release repository**

A release repository is where all the published artifacts are stored in Maven. Because caching plays a really important role in maven repositories, it is highly recommended to make the Release repositories immutable. i.e. artifacts once uploaded cannot be modified. Let's say you published `0.0.1` of your library, you must not modify the existing version, instead publish the next version let's say `0.0.2`. This is enforced by default in Maven Central. 

There is also the concept of a **Staging repository** in Maven which serves as a staging area for you to verify all your artifacts before promoting it to the release repository.

### 🧪 **SNAPSHOT repository**

A **SNAPSHOT** variant of a library refers to the **latest** code along a development branch, and provides no guarantee the code is stable or unchanging and these variants are published to the **SNAPSHOT** repository. Conversely, the code in a **Release** variant (any version value *without the suffix SNAPSHOT*) is unchanging.

In other words, a **SNAPSHOT** version is the **development** version before the final **Release** version. The **SNAPSHOT** is **older** than its release.

During the release process, a version of **x.y.z-SNAPSHOT** changes to **x.y.z**. The release process also increments the development version to **x.y.(z+1)-SNAPSHOT**. For example, version **1.0.0-SNAPSHOT** is released as version **1.0.0**, and the new development version is version **1.0.1-SNAPSHOT.**

We already discussed that a **Release** repository is immutable. So the idea behind a **SNAPSHOT repository** is to allow mutability and easy development. You can continuously push your latest changes to **1.0.0-SNAPSHOT** and anyone depending on it will get the latest changes every time they build their project. Then, after a few iterations, and everyone is happy the latest state of **1.0.0-SNAPSHOT**, it can be permanently released as **1.0.0**, and rapid development can continue on **1.0.1-SNAPSHOT**.


Sonatype's **Open Source Software Repository Hosting (OSSRH)** service allows open source projects and individuals to create a Maven Central repository and publish artifacts for free! In the next article, we'll go through all the steps involved in publishing an Android library to Maven Central. So stay tuned 😉

## ✨️ Credits

Hope you enjoyed this article! We love contributions and encourage you to take a look at our [open isuses](https://github.com/appwrite/appwrite/issues) and [ongoing RFCs](https://github.com/appwrite/rfc/pulls).

If you get stuck anywhere, feel free to reach out to us on our [friendly support channels](https://appwrite.io/discord) run by humans 👩‍💻.

Here are some handy links for more information:

- [Appwrite Contribution Guide](https://github.com/appwrite/appwrite/blob/master/CONTRIBUTING.md)
- [Appwrite Github](https://github.com/appwrite)
- [Appwrite Documentation](https://appwrite.io/docs)