Building Data Science capability at the UKHO: our October 2020 Research Week
By: Cate Seale
Tags: Data Science, Capability Building, Image Segmentation, Machine Learning, Reinforcement Learning

Notes from the IEEE International Geoscience And Remote Sensing Symposium 2020: Cate Seale

For my research week I attended IGARSS 2020 which had a packed technical program of remote sensing, satellites and machine learning. Whilst I was sad not to be learning about these topics from a beach in Hawaii (original location Waikoloa), I was excited to see the latest research.

From the start of the first UK lockdown in March, I have attended numerous webinars, workshops and working groups that were held remotely (I’m sure many others have had a similar experience). This was the first time I’d attempted a week-long conference with a full day’s agenda, and as expected, some things were good, and some not-so-good…

On the one hand:

I went to a conference (for $10!) that would have been prohibitively expensive otherwise.

It’s super easy to type a question in the chat and have it answered, in comparison to sticking your hand up in a full auditorium and/or standing up to speak into a microphone.

Attending conferences in person with a physical body limits you to attending one track at a time and inevitably there were 10+ sessions running concurrently. Here, the poster presentations were prerecorded videos, with the authors available to ask questions after broadcast, and slides and video available for download. Very useful if you get FOMO.

I avoided the environmental impact of travel, had no jet-lag or hotel costs — multiple wins here!

I was able duck out of a session without doing a walk of shame. 😊

On the other hand:

It’s so easy to get distracted by work and home life when you aren’t physically removed from it all, so conferencing ends up adding to your mental load.

Getting the information is only achieved when the presenter is understandable, but not all microphones are equal! Presenters ranged from crisp and clear to seemingly recorded-underwater-next-to-pneumatic-drill. Not really anyone’s fault, but unfortunately degraded the experience.

I personally didn’t get along with the use of text-to-speech software and found it hard to concentrate (my brain obviously switches off when it hears computer-voice), but appreciate this may be preferable for speakers that were confident in English.

There was no networking, and I really missed the opportunity to talk to people.

Overall, it was a positive experience and my respect and admiration go out to the organisers for the smooth coordination of hundreds of speakers and thousands of attendees. The technology needed to run an event like this remotely seems intimidating at best, but (barring the odd terrible microphone) everything went well, and I appreciated being able to continue to attend conferences and training courses while COVID rages outside my door.

Investigation of tools and techniques: Mike Hudgell

Moving window technique

We have written a geomorphology function that classifies pixels of a bathymetric surface into 10 canonical forms, namely: flat, summit, ridge, shoulder, spur, slope, hollow, foot-slope, valley, depression. The function is smart in that you can have different focal ranges and thresholds to suit your use-case, and is proving useful in our current work with LIDAR imagery and seagrass detection. However, the function is slow as it needs to calculate the slope at eight compass points for every pixel. Therefore, the investigation was to see if we could improve the performance of the function by using a DASK moving window technique.

The “moving window” technique essentially splits the raster into many thousands of windows (with overlaps so you don’t get any unwanted edge artefacts) and processes the windows in parallel. We achieved the following results:

Image credit: UK Hydrographic Office © Crown copyright 2020 UK Hydrographic Office

So, this was a successful experiment showing an 82% improvement in performance was possible and one that will be worth factoring into the development of our functions going forwards. However, it is worth observing:

we saw a 35% performance improvement by running the baseline code on a higher spec machine, without adding complexity to the code.

we saw a 44% and 72% performance improvement (business and experimental laptops respectively) with DASK, indicating that if we go to the trouble of parallelising our functions, we will gain most benefit on higher spec machines.

RAPIDS

We have started to use a new library called RAPIDS that enables us to run many different data science algorithms on GPU rather than CPU which has the potential not just to save us time but also make some algorithms viable — as sometimes the algorithm we really want to use can be just too slow.

Two experiments were carried out, timing k-means clustering and loading a data frame:

Image credit: UK Hydrographic Office © Crown copyright 2020 UK Hydrographic Office

So, these experiments were successful in showing improvements in performance and an early indication that we are likely to want to exploit GPUs more than we do currently.

Additional techniques

Finally, I spent time reading around various techniques that we think are going to be useful, namely: Morphological Building Index; Morphological Shadow index; Bathymetric Position Index; Topographic Position Index.

Here is an example synthetic surface we have created from an aerial image using the Morphological Building Index algorithm:

Image credit: UK Hydrographic Office © Crown copyright 2020 UK Hydrographic Office

Conclusions

Thanks to all of our contributors this month! Developing new techniques to solve complex problems is an important part of being a data scientist, as off-the-shelf algorithms and techniques only get you so far when it comes to solving new data problems.

Knowing where and how to spend time developing new approaches can be difficult, especially when deadlines are looming. But, by protecting time each quarter for our data scientists to explore new ideas and techniques without the pressure of immediate project deliverables, we’ve found that our understanding of complex problems has increased, our delivered solutions have improved, and the overall skill and knowledge level of our team has increased.

We’d recommend the approach is taken up by other data science teams — no matter your domain.