It’s Been a Ride 🎢
By: Mathias Kemeter
Tags: GIS, Sap, Database, Software Development, Open Source

Said and Done. Well… uhm… almost. Let’s have a look at how our concise and clear path towards contributing to the QGIS project turned out.

May 2019

On May 21st, 2019 and after being clear about what we would like to achieve, Max from our team forked the QGIS repository and started to technically plan the contribution. We started developing and testing the plugin internally before we placed our first (we didn’t know yet, that it wouldn’t be the last) pull request.

July 2019

That was on July 16th, 2019. We were done with the implementation and just had to wait for the pull request to get merged. We thought. Then we got some feedback from the community:

“[…] we might need a bit more context on what this is and why it needs to be a core provider?” “HANA database, that’s new to me :)” “[…] a [HANA] provider would not be of use to a vast majority of its user” “Historically when we’ve done this is becomes a huge maintainance burden on the project” “I’d like to point out process of QGIS Enhancement Proposal […] before going deeply in implementation like in Pull requests”

There were some valid questions, some advice and a couple of comments that hurt our developer egos (Come on! If there is no HANA support, how could it possibly have a large HANA user base?). There were also kind comments and, of course, there was the pointer to the QGIS Enhancement Proposal (QEP) process, which we have — regrettably — not considered before.

After having a couple of discussions with the community and clarifying some of the questions, including license concerns, we sorted ourselves and made the official enhancement proposal according to the process.

That was July 24th, 2019. Sticking to the process helped (Hey, we are SAP. We should’ve known before). The discussion got considerably more objective. Well, almost.

A: „Adding a new provider isn’t just a free gift to the project. Rather it’s more akin to giving someone an unexpected puppy for Christmas. Sure, it’s all fun at first, but then there’s always someone who inevitably has to feed it, clean up after it, pay the vet bills,…“ B: „It seems unfair to me that QGIS has core providers for commercial databases, but does not allow to contribute support for our database in the same way it has been done for other commercial databases. Even worse, it feels like we are being punished for a competitor neglecting the maintenance of its core provider, while the competitor was actually rewarded by the community taking over the maintenance of that provider.“ A: „Unfair? […] Maybe in the same way that a first child might get different standards for parenting and have different rules from a second child, simply because the parents have learnt more about parenting due to their previous mistakes experiences!“

Photo by Karsten Winegeart on Unsplash

When looking at the history of communication the above dialogue really nails down the core suspicions between both parties — the QGIS community and our SAP team.

The QGIS community has learnt from past experiences and was concerned about the additional maintenance burden, while we felt misunderstood and unfairly treated.

August 2019

The discussion went on for a couple of weeks. On August 15th, 2019 our original pull request got closed due to inactivity. In the beginning, it felt that discussing with an open community (rather than with a single representative) is like fighting windmills. Once you agreed on one point, another person stepped out of the dark and brought a totally new aspect, so that you could never be sure, what the state of the discussion actually is.

September 2019

Finally, on September 21st, 2019 we concluded on the major pillars of our contribution (e.g. testing platform, CI integration and maintenance).

For us this was a major break-through! We finally had a reliable agreement we could build upon. It was clear, that we had some additional work in our backlog — but it all seemed manageable and clear.

December 2019

Last day before Christmas break on December 20th, 2019, I had the wishful thinking, that we could issue our second pull request and could possibly get into the next version of QGIS. I even had a celebration blocker in my calendar — but of course, this remained a wishful thinking. We have learnt that we would require more community expertise to efficiently flesh things out.

We missed the Bossche bollen 😢

January 2020

There was a QGIS Contributor Meeting (also known as Hackfest) planned to take place in Den Bosch in March 2020. That would be the best opportunity to get in touch with the community and discuss technical topics! (…as a 2021-reader you will already anticipate that something ‘unprecedented’ came in the way). We gathered a travel group of 3 team members and — since you do not go to a house party without having a present — SAP also sponsored the event.

In the meantime, we adapted our 1-year-old codebase to some major API changes, which affected our contribution.

March 2020

On March 10th, 2020 the Contributor Meeting finally got cancelled due to COVID-19.

As it was clear, that in near future there was no opportunity to liaise with the community, we finally issued our second pull request on March 11th, 2020. Almost certain that almost everything has been done and we are almost good to go. Due to my devotion for the number Pi, I remember that a realistic feature release was QGIS 3.14 at that time. (I have learnt that I seem to be an optimist rather than a realist)

April 2020

We received a couple of review comments, which we considered until April 22nd, 2020. Afterwards, Stefan started our internal peer review process — as it is part of our engineering standards.

July 2020

We finished the full peer review process of the meanwhile quite large codebase. On July 3rd, 2020 I took another stab in declaring production-readiness. However, there was still this one nasty issue: Due to memory consumption, we did not manage to run our HANA Express docker image on the Travis CI of the QGIS project next to the images of the other databases. This ultimately led to the fact, that we could not run our test suite on the CI.

September 2020

Meanwhile Max gained expert knowledge of Travis, which is why today he is unwillingly considered to be the CI expert in our team. Finally, we gave up to run the database on the CI.

On September 17th, 2020 we could run our tests using an SAP HANA Cloud instance on the QGIS Travis CI. Thus, we took advantage of the fact that SAP HANA Cloud has been released in the meantime.

December 2020

We have been undergoing further community reviews until December, which we have thankfully considered for our contribution. On December 18th, 2020 — last day before Christmas break — I had a nostalgic look back and thought of the foolish feeling last year, that we could get our contribution merged in 2019. This time, I felt confident to see the merge happening in 2020 still.

January 2021

Of course, based on prior experience my confidence was clouded. On January 6th, 2021 Matthias — one of the main contributors of the QGIS project — announced that he plans to merge our contribution by the end of the week if there are no further concerns.

After not sleeping for 4 full days (actually, I slept well… but this would not add to the dramaturgy), our almost 2-year-old pull request finally got merged into the QGIS master branch on January 10th, 2021.

Still, our tests ran for almost 10 minutes as the CI infrastructure was in the US and our HANA Cloud instance in Europe. So, we spun up an instance on the US east coast and came out with an acceptable latency.