A Rounded Solution to Image Handling on the OpenSauced Dashboard
Tags: react, javascript, cloudinary, github
Published: 2023-05-17 21:46:54

*This post is about the journey we took to improve the OpenSauced Dashboard, which provides valuable insights on open-source contributions.* 

Last October, we had an alpha launch of the new OpenSauced Insights dashboard. This launch was to coincide with the biggest open-source hackathon, Hacktoberfest. Our goal was to provide insights and reports on the number of contributions accepted, merged, and even spam. Since then, the dashboard has evolved beyond Hacktoberfest data and now includes GitHub avatars representing active and open GitHub Pull Requests in the last 30 days.

_If you’d like to see your open source contributions [connect your GitHub account to OpenSauced](https://insights.opensauced.pizza?utm=dev)._

## Challenges Faced

At the beginning, our team encountered challenges in sourcing and manipulating GitHub avatars to display a scatter plot e-chart on the dashboard. We struggled to find a suitable solution, especially when it came to handling a large number of images efficiently.

As part of this challenge, GitHub has an aggressive rate limit when you're not authenticated when requesting resources, like avatars from a URL. We needed to fetch the avatar, manipulate them, and then cache them. Because our product was meant to be public, we would attract a lot of usage, and it could cost real money quickly.

## Solution: Leveraging nivo and Cloudinary
To overcome these challenges, the team turned to the **nivo chart library** for visualizations and **Cloudinary** for image manipulation and caching 271k images. We successfully integrated nivo charts into our dashboard, thanks to its rich set of data visualization components built on top of d3 and React.

By leveraging Cloudinary's image manipulation capabilities and caching strategy, we were able to round the avatars and seamlessly integrate them into the dashboard.

![dashboard screenshot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u425gz6xkwdztzg2ucsj.png)

Today this works thanks to [nivo](https://nivo.rocks/) and [Cloudinary](https://cloudinary.com/), but that journey included a lot of trials and testing for the perfect solution.

_Nivo is an open-source library that provides a rich set of data visualization components built on top of the awesome d3 and React libraries._

{% github https://github.com/plouc/nivo %}

_Cloudinary is a hosted solution to manipulate and cache images for reuse._


## Handling Image Processing with Apache E-charts

Before adopting nivo and Cloudinary, we initially used Apache E-charts (specifically a React wrapper called [echarts-for-react](https://github.com/hustcc/echarts-for-react)) to handle image processing and loading. This approach proved extremely slow, and it was quite the process in figuring out a better solution while we had a real-time constraint.

## The Journey to Finding Solutions
We faced a challenge being able to provide images on the page that were sourced directly from GitHub. The avatars needed could be cached and manipulated to match our rounded image design. The total number of displayed contributors during the event was around 150k, and today close to 300k contributors are represented as contributions in the most popular open source repositories on OpenSauced.

![dashboard image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dl5oeh1g6u1xufwxku4y.jpg)

The e-charts for React solution gave us no access to the images when rendered, and provided limited options to edit the chart after it was displayed. We built our product in Figma designs first and were excited at the opportunity to have rounded images. Still, our e-chart library would only allow plain & squared images, and also any manipulation of the image was the challenge. 

[design]

![Rounded image design](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8s9lcqb7qt3uuittb26r.png)

[reality]

![harsh square image design](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8i8oaapjzsdjgvsfa6d3.png)

There comes a time in every developer's career where you are presented a design that may be just out of reach or out of scope. The simplest request around the images broke our charts and required the entire team to brainstorm solutions. 

You can see that brainstorming in our now closed issues. 

{% github https://github.com/open-sauced/insights/issues/373#issuecomment-1249618915 %}

## Solution 1: JavaScript Approach
Our first attempt was manually rounding the images using a quick lib function I threw together.

```js
function applyBorderRadius(imageElement) {
  imageElement.style.borderRadius = '50%';
}

// Get the image element
var image = document.getElementById('myImage');

// Apply the 50% border radius
applyBorderRadius(image);
``` 

It's always good to start with an approach, even if it is wrong. 

This first solution needed to be corrected and didn't work because the images were limiting the e-chart with images on a scattered chart. We could not get direct access to the elements to manipulate after the fact. 

We needed a way to manipulate the images before sending to the chart.

## Solution 2: Imagemagick for Rounding Images
[ImageMagick](https://imagemagick.org/index.php) is a fun open-source platform for displaying, creating, converting, modifying, and editing images. I had some experience working with ImageMagick at a previous employer and quickly found a solution to round the images before sending to the chart. 

But when I found a [stackoverflow answer](https://stackoverflow.com/questions/67342758/displaying-rounded-images-in-github-pages-for-profile-image-using-markdown) doing something similar, I opted to use that instead.

```
https://images.weserv.nl/?url=https://www.github.com/bdougie.png?size=60?v=4&h=300&w=300&fit=cover&mask=circle&maxage=7d
```
[Link to this solution](https://images.weserv.nl/?url=https://www.github.com/bdougie.png?size=60?v=4&h=300&w=300&fit=cover&mask=circle&maxage=7d)

This was working, but we still needed to solve the caching issue.  

I really wanted to try out building a service to use ImageMagick + the new [Supabase storage](https://supabase.com/storage) to do this, but I wasn't willing to maintain that solution and we only had a little time to explore more unique tools. 

(_If you want to build a service like this, find me, and I would love to be a beta tester._)

We needed a way to cache the images before sending to the chart and started looking at tools or services to make this easier.

## Solution 3: Leveraging Cloudinary for Image Manipulation and Caching

To solve the caching issue and optimize image processing, we explored different tools and services. 

Cloudinary offers image manipulation and a caching strategy. They also have a generous free tier--but for full transparency, I'll point out that our initial amount of data for Hacktoberfest was pushed up to the paid tier immediately. 

```
https://res.cloudinary.com/bdougie/image/fetch/f_auto,q_auto/w_400,h_400,c_crop,r_400,g_auto/v1/https://avatars.githubusercontent.com/u/5713670
```
[Link to Cloudinary solution](https://res.cloudinary.com/bdougie/image/fetch/f_auto,q_auto/w_400,h_400,c_crop,r_400,g_auto/v1/https://avatars.githubusercontent.com/u/5713670)

You can see the PR with the solution live as well.

{% github https://github.com/open-sauced/insights/pull/467 %}

The solution was building a wrapper around using the GitHub avatar as the `imageUrl`.

```js
// lib/utils/roundedImages

const roundedImage = (imageUrl: string, cloudName: string | undefined) => {
  return cloudName
    ? `https://res.cloudinary.com/${cloudName}/image/fetch/c_fill,g_face,h_300,w_300,bo_20px_solid_white,r_max/f_auto,e_shadow/${imageUrl}`
    : imageUrl;
};

// components/organisms/Dashboard/dashboard.tsx

import roundedImage from "lib/utils/roundedImages";

scatterChartData = prs.map(({ updated_at, linesCount, author_login }) => {
      const author_image = author_login.includes("[bot]") ? "octocat" : author_login;

      const data = {
        x: calcDaysFromToday(new Date(updated_at)),
        y: linesCount,
        contributor: author_login,
        image: roundedImage(`https://www.github.com/${author_image}.png?size=60`, process.env.NEXT_PUBLIC_CLOUD_NAME)
      };
      return data;
    });
  }

```
_[Link to code]([components/organisms/Dashboard/dashboard.tsx](https://github.com/open-sauced/insights/blob/beta/components/organisms/Dashboard/dashboard.tsx))_

The interaction with Cloudinary is all URL based, allowing us to pass the GitHub user id as an option on the fly. The same id is also used to recall the cached image, meaning we have had caching by default. We also did not need to build any new infrastructure for this. 

With the initial 150k user profiles cached, we immediately needed to pay $90 per month for the pro tier, which we felt is reasonable and predictable for us. 

We are still on the same Cloudinary plan today, mainly due to the inertia and the fact that we still need to be ready to build and maintain something not core to our product. So far, Cloudinary is caching 271k image transforms for us.

This scenario is a classic build vs buy scenario where we could build our own caching and image storage on S3, but the tech debt imposed on a solution like that was not our priority. The $90 was also not going to put us into debt either. 

## Migration to nivo Charts

In December, we made the switch from Apache E-charts to nivo charts due to its modern features and active community. The decision was driven by the outdated state of E-charts and the limited contributions it received. We plan to contribute upstream to nivo to improve its integration and interaction with images.

We have an active conversation in the nivo discussions about this and other feature enhancements. The maintainer has been really responsive, and we look forward to contributing upstream to support the project. 

{% github https://github.com/plouc/nivo/issues/2201 %}

If you have any thoughts or comments about our approach or have alternative solutions to share, we welcome your input. Let's continue learning from each other and enhancing our open-source projects.