fabiochiu/medium-articles huggingface dataset:
How Oath Pizza Embraced Digital and Turned Delivery Profitable
By: Aaron Newton
Tags: Coronavirus, Digital Marketing, Loyalty, Restaurant

How Oath Pizza Embraced Digital and Turned Delivery Profitable

by Aaron Newton, Chief Product Officer, Thanx

Ed. Note: This article was written before the current coronavirus crisis; restaurant sales in the United States have dropped by an average of 80% in the last two weeks. Given the current landscape, we originally decided to indefinitely delay publication of this case study; we are sensitive to the fact that every restaurant is making gut-wrenching decisions in response to the pandemic.

But after talking to several restaurants over the last few days, it became clear that digital ordering is now even more critical to their success than ever before and they were seeking a playbook for how to do it well. Nationwide, Thanx-partnered restaurants who offer digital ordering are suffering “only” 20-40% declines vs. 80%-100% declines for on-premise-only businesses.

Oath Pizza —profiled here — is no exception. While this review was written before coronavirus hit the US, Oath’s focus and investment on digital and customer loyalty over the past year has meant that, thus far, their business has weathered the storm better than most. This is the story of their journey which sheds some light on the path they took to be digital and data-ready when they needed it most. We offer it hoping that indeed it serves as somewhat of a playbook for others looking to make the same leap.
Time is the element for restaurants
Tags: Startup, Coupon, Restaurant

At BidShore, we believe that our business is truly changing the world by best utilizing one very important element: TIME.

With only about 2% of printed coupons from restaurants actually getting redeemed, there is no doubt about it that the way of printed coupons leaves the world with so much additional waste in both materials, and time from our friends- the restaurant owners.

As for our amazing BidShore customers, precious moments of their life can slip away in a hunt for a coupon, a deal, or finding the local specials in order to maximize their visit at the best price.

As local small businesses grow, they require to always be filling their days with hungry customers — it’s a race against their biggest nemesis- time. The more time passes that they are not serving customers, food is going bad.

Let time be on your side to fill your stomach, your wallet, while supporting our local business owners and eliminating landfill waste when you maximize your time with an experience like BidShore.

Together, we can continue to make such a positive impact on the world around us!
Use Somm.ai to convert your wine list into a retail store
By: David Kong
Tags: Coronavirus, Wine, Restaurant, Somm

See our platform at https://shop.somm.ai

100% of proceeds go to small businesses.

New emergency measures enacted because of COVID allow most restaurants in the US to sell wine. Rules differ state-by-state but the most common rule is that restaurants are allowed to sell wine with take-out and delivery orders that accompany food.

For restaurants who are struggling through COVID this is another revenue stream that can help. Somm.ai has powerful technology in the wine space to be able to help you make that transition in only a few clicks. We help restaurants through the entire process.

We launched with 15 restaurants in NYC (with 7 Michelin stars!) and have facilitated $45,000 in sales. Nationwide we have 30 restaurants.

Here’s what you need to know.

In which states can you do this?

Most states allow this in some degree, including Alabama, Arizona, Arkansas, California, Colorado, Connecticut, DC, Florida, Georgia, Illinois, Iowa, Kansas, Kentucky, Louisiana, Maine, Maryland, Michigan, Mississippi, Missouri, New Hampshire, New Jersey, North Carolina, Ohio, Pennsylvania, Rhode Island, South Carolina, Tennessee, Texas, Vermont, Virginia, Washington, West Virginia and Wisconsin. Some of these states require a separate license — we are happy to help send you the documentation so you can decide for yourself if this is the right option for you.

How do I set up a store?

This is the easiest part. Fill out this form, and we’ll create the store for you. There’s even a chance a store is already set up for you.

How do I set the pricing?

Somm.ai has technology that lets you price your wines intelligently. Using three sliders, you can choose how competitive you would like your wines to be, and set minimums to make sure your more cherished wines sell at good prices.

Intelligent Pricing with Somm.ai

How do I market the wines?

Leverage the following you already have at your restaurants. Send out a mailing blast. Post on social media. Get other restaurants in your community to participate so you can create a network effect in your community.

Can I add a takeout food menu?

Yes!

What happens when an order comes in?

For the most part, an order should be viewed as a “request”. The customer, restaurant, and the Somm.ai team work to fulfill the order. Typically, the restaurant will handle its own payment processing and fulfillment but Somm.ai can help.

Can you make a custom branded page?

Yes, absolutely. Let us know what you need.

Questions?

Contact us at hello@somm.ai
Location Analytics for delivery & mobility companies with ground operations
By: Aditi Sinha
Tags: Analytics, Delivery, Geospatial, Geolocation, Mobility

1: Supply-Demand Analysis: Analyzing the gaps between demand & supply

For an on-demand company (aka “Uber for X”), that is primarily a three-sided marketplace, matching demand and supply is at the core of the business and also extremely challenging. Moreover, the real-world introduces far more variation that leads to starker results than in the model.

When supply-demand gaps exist, we are either losing orders or our riders are idle — both of which are contributing to losing money. Normally, every micro-mobility company has a matching algorithm. But, how does that algorithm perform at different times of the day, days of the week, locations? Do they have a pattern? How do you analyze that?

Supply-Demand Analysis. Kepler.gl

Supply-demand analysis helps you decode where, when and why the gap between supply and demand occurs. For every unit of demand which is not met, you have lost an order. For every unit of supply which is idle, you are losing money on your vehicles.

Analyzing the gaps behind supply and demand helps you optimize for metrics such as utilization rate and the number of unserviced orders.

Decisions that can be taken using this analysis

Depending on the time of the day, day of week and location, you can decide:

Rider Incentives: The incentive that needs to be given to the rider to make them move to a particular area of higher demand

The incentive that needs to be given to the rider to make them move to a particular area of higher demand Surges and Discounts: The surges and discounts that we can give to the user to incentivize them to use your service in areas of high supply?

Insights required for taking these decisions

You can analyze the following metrics from the demand and supply side across different locations:

Number of Orders Lost vs Order Frequency: How many orders you are receiving and what percentage of that you are losing?

How many orders you are receiving and what percentage of that you are losing? Idle Supply vs Total Available Supply: How much total supply you have and what percentage of that is idle?

How much total supply you have and what percentage of that is idle? Distance of Lost Orders to Available Supply: What is the distance of orders that you are losing to the nearest idle supply?

Read more about these metrics here:
Is Alteryx the right choice for your analysis needs?
By: Andrew Hershy
Tags: Data Analysis, Geospatial, Tutorial, Alteryx, Data Science

History/Context

Alteryx is both the name of the company that makes the software and the name of the software… it actually has an interesting backstory. The company was founded in 1997 and used to be called SRC LLC. For its first 15 years, the company focused primarily on geographic business intelligence solutions. It has a history of working on various interesting data-related geographic projects, including partnering with the U.S. census in 2012 to create a tool which helped customers strategically mine/analyze demographic data for their respective use-cases. This was a groundbreaking project, as it was the first time the general public could dig into the demographic makeup of the U.S with such granularity.

In 2010, the company re-branded to “Alteryx”, taking the name of their leading data analysis software. From this point, the company started defining itself as an all-around data science and analysis company, not exclusively a geographic business intelligence solution. They designed their software to be a drag and drop solution for any associate to perform advanced analytics without as much of a coding / hard-science background.

However, due to Alteryx’s decades-long background in geospatial data analysis, I believe this is still what separates its software from others.

On a side note, Alteryx had its IPO on March 24, 2017 at $14 per share. The stock price reached a high of $158 on February 14th, 2020….
Carto vs Kepler vs Locale: Which product to use for geospatial analytics?
By: Aditi Sinha
Tags: Real Time Analytics, Keplergl, Carto, Comparison, Geospatial

Comparision

This image shows the comparison between the three tools in a nutshell on different features or parameters. This section dives deeper into all of the parameters.

1- Analysis & Visualizations

Kepler.gl

Kepler is one of the best geospatial visualization tools for exploratory data analysis (EDA). The Uber team has done thorough and detailed research on the kind of visualization features relevant to mobility companies.

One of its biggest drawbacks is that any kind of processing that needs to be done on the data needs to be done outside. Kepler.gl is, unfortunately, a desktop tool that takes data only in the form of CSV, JSON, and geoJSON files.

These characteristics make it very painful when you are dealing with large amounts of data and need to keep downloading CSVs to plot onto the tool

Carto

Carto’s visualization and analysis capabilities are much more horizontal casting a wide net on the use cases they tackle as a company which includes different kinds of filters, widgets. They also have several built-in analyses (pre-built functions to help you manipulate data) inside the product.

The downside over here is that all these analyses can only be done on one data set at a time. Hence, the pre-processing out here also needs to happen outside the system, especially in cases of more complicated data sets.

Locale

At Locale, we have tried to include the best of both worlds. We have taken the inspiration from Kepler for visualizations and added analyses onto the product — not as functions but instead as use cases. We also ingest all your data across different databases, formats, and systems, get it onto a single place and model it so that it becomes super easy to be able to carry out these analyse

2- Target Personas

Kepler.gl

Kepler.gl is targeting mostly business users, who essentially don’t know how to write code or SQL because of its sheer simplicity. However, we have seen a vast number of data scientists across Google and Airbnb adopting it for visualizations.

Carto

Carto is positioned as a location intelligence platform with an aptitude for data analysis and visualization that does not require previous GIS or development experience. Their UX tailored to the needs of modern web developers and data scientists. Their visualizations can be used for business users to make decisions.

Locale

Our target users at Locale are also business users as well — supply and operations teams, product teams, strategy teams and marketing teams inside companies. We have found that they are the ones who are in maximum pain and most of the time end up using excel sheets. They still depend on an analyst for BI reports or developers for live dashboards.
SpaceNet 6 Challenge Launch
By: Jake Shermeyer
Tags: Artificial Intelligence, Geospatial, Open Source, Data Science, Deep Learning

Preface: SpaceNet LLC is a nonprofit organization dedicated to accelerating open source, artificial intelligence applied research for geospatial applications, specifically foundational mapping (i.e. building footprint & road network detection). SpaceNet is run in collaboration with CosmiQ Works, Maxar Technologies, Intel AI, Amazon Web Services (AWS), Capella Space, Topcoder, and IEEE GRSS.

The SpaceNet 6 challenge is officially live, and offering $50,000 in cash prizes to our top participants! In the SpaceNet 6 challenge, participants are asked to automatically extract building footprints with computer vision and artificial intelligence (AI) algorithms using a combination of Capella Space synthetic aperture radar (SAR) and electro-optical satellite imagery from Maxar. The challenge will run from March 16th until May 1st. Top participants will be invited to share their work at CVPR EarthVision in Seattle, Washington on June 14th, 2020.

Register and compete in the challenge on TopCoder:

https://www.topcoder.com/spacenet

Learn more about SpaceNet:

https://spacenet.ai/

Win AWS Credits:

The first 30 participants to achieve an F1 Score of 0.2 on the public leaderboard (recall that the the SpaceNet 6 baseline (link: https://medium.com/the-downlinq/the-spacenet-6-baseline-3b8ae8068351) achieved a score of 0.21) will be eligible to receive an AWS credit worth $500 in compute time.

Download the dataset from our AWS S3 Bucket for free:

aws s3 cp s3://spacenet-dataset/spacenet/SN6_buildings/tarballs/SN6_buildings_AOI_11_Rotterdam_train.tar.gz .

Read about or try out the baseline:

We recently released a new baseline algorithm, which could be used as an example or starting point for SpaceNet 6 participants for extracting building footprints from SAR imagery.

git clone https://github.com/CosmiQ/CosmiQ_SN6_Baseline/

Important Dates:

March 16th — Challenge Launch

April 17th — Challenge Midpoint

May 1st — Challenge Ends

June 4th — Winners Announced

June 14th — CVPR EarthVision

What’s Next?

As always, stay tuned to The DownLinQ and check out https://spacenet.ai for more details on the dataset and challenge.
Predicting Traffic Accident Hotspots with Spatial Data Science
By: Miguel Álvarez García
Tags: Geospatial, Spatial Analysis, Traffic, Hotspot Analysis, Regression Kriging

Predicting Traffic Accident Hotspots with Spatial Data Science

Image by Author

Road traffic injuries are among the ten leading causes of death worldwide and they are the main cause of death among young adults aged 15–29 years [1]. In addition to the tragic impact in human lives, road traffic injuries have a significant effect on the world’s economy. One study estimates that road traffic injuries will cost the world economy US$1.8 trillion in the period 2015–30 [2]. The European Union estimates the impact represents 2% of their GDP [3]. These costs include property damage costs, insurance administration costs, hospital costs, and productivity losses, among others [2][4][5].

Because of these enormous impacts, governments and the private sector are making big efforts to reduce these numbers. Today, as a result of these efforts, we have GPS-based systems that provide real-time information on traffic and weather conditions, with governments providing information such as accident hotspots (a.k.a., accident blackspots) and other useful statistics.

Image by Michael Jin

However, this information is available either post accident or it is static. For example, one location can have a relatively low concentration of accidents, but all occurring under a same set of rare circumstances. In this case, it would be useful to identify this location as a hotspot only when those circumstances take place, i.e., as a dynamic hotspot.

Knowing the conditions under which accidents happen and where they happen is very powerful information that can be used to take action to avoid them. For example, logistics companies can use this information to avoid specific routes, insurance companies can share this information with their clients, and cities can assign their traffic police to the dynamic hotspots.

In this blog post, we present a detailed analysis and a powerful predictive model that can help identify the factors affecting accident concentration and how these results can be further used to define dynamic hotspots. The analysis focuses on the city of Barcelona (Spain) using traffic accident data from 2019 and can be replicated in other cities and regions of the world. Barcelona was selected as it is a mid-large city and has available a rich Open Data catalog.

Data Sources

We identified different data sources that can influence accidents and worked with open and premium data. All premium datasets were obtained through CARTO’s Data Observatory.

The following map (Map 1) shows all the datasets we used for this model. The layer selector on the legends can be used to activate/deactivate one layer and analyze them. Note that data is only shown for an area of the city.
A New Planetary Perspective
By: Nikita Marwaha
Tags: Company, Technology, Environment, Satellite, Geospatial

We’re UP42 and we care about bringing knowledge about our planet into the hands of those that need it. That’s why we’ve spent the past year working on a way to make planetary data, or geospatial data easy to find, analyze, and use.

Why?

By going up, we can look down. Our planet’s palette of changing colors, textures, and shapes — tells a story of land, wind, water, ice, and air — as they can only be viewed from above. Earth holds the answers to many of its problems.

Where can we find those answers? Geospatial data. There are many types of geospatial sources that are rich with knowledge from different perspectives. From constellations of satellites circling the planet in space, to drones soaring overhead, and IoT connected devices dotted across the globe — rich sources of geospatial data help us to understand how Earth was, is, and will be.

However, accessing and using geospatial data has (until now) been complicated, expensive, and time-consuming. Algorithms were a closely guarded secret — making analyzing the data once you find it — another hurdle to jump over. With UP42, we’re changing things.

UP42 opens up access to what previously was out of reach. We’re sharing the knowledge and tools to study the world, with the world.

What is UP42?

UP42 is a platform to access and analyze portions of the planet at scale. We bring together multiple geospatial data sources, alongside processing algorithms — empowering you to discover the planetary insights you need.

Through UP42, you can find the kind of data that you need, select what you’d like to study and pinpoint your chosen corner of the world. Three simple steps, all in one place. We’re a marketplace too — bringing together partners that have the data and algorithms that you’re looking for. We believe that by bringing people together, we can bring information together, and create change together.

From observing urban growth and land use to monitoring deforestation and ice thickness — we are committed to helping people access planetary insights that are tailored to their needs. Whether you’re a large business owner interested in analyzing volumes of satellite data at scale, or a researcher looking at smaller portions of the planet for a specific cause, we can help.

What’s next?

It’s been almost three months since our beta launch and with just one month until we go public, we’re launching this blog to share more about the wonders of geospatial insights.

We’ll be shedding light on how making geospatial data easy to use sparks innovations in industries far and wide. We’ll be introducing the people behind the data and algorithms, sharing opinion pieces from our team, and discussing why geospatial insights are good for businesses, people, and the planet.

In a time when the planet is undergoing rapid change, it’s crucial to not only see this change but to understand it too. We’re here to make that easy.

UP42. A new planetary perspective.
SpaceNet 6: Dataset Release
By: Jake Shermeyer
Tags: Synthetic Aperture Radar, Mapping, AI, Machine Learning, Geospatial

Preface: SpaceNet LLC is a nonprofit organization dedicated to accelerating open source, artificial intelligence applied research for geospatial applications, specifically foundational mapping (i.e. building footprint & road network detection). SpaceNet is run in collaboration with CosmiQ Works, Maxar Technologies, Intel AI, Amazon Web Services (AWS), Capella Space, Topcoder, and IEEE GRSS.

The SpaceNet partners are proud to announce the release of the SpaceNet 6: Multi-Sensor All Weather Mapping (MSAW) dataset. This openly-licensed dataset features a unique combination of half-meter Synthetic Aperture Radar (SAR) imagery from Capella Space and half-meter electro-optical (EO) imagery from Maxar. We believe this dataset will help to further research and expand remote sensing analytics beyond the EO spectrum and into new modalities. You can download the data now, all you need is an AWS account and the AWS CLI installed and configured. Once you’ve done that, simply run the command below to download the training dataset to your working directory!

aws s3 cp s3://spacenet-dataset/spacenet/SN6_buildings/tarballs/SN6_buildings_AOI_11_Rotterdam_train.tar.gz .

SpaceNet 6 focuses on foundational mapping and building footprint extraction using a combination of these datatypes over the newest and 11th SpaceNet city: Rotterdam, the Netherlands. In this challenge, the training dataset contains both SAR and EO imagery, however the testing and scoring datasets contain only SAR data. Consequently, the EO data can be used for pre-processing the SAR data in some fashion, such as colorization, domain adaptation, or image translation, but cannot be used to directly map buildings. We structure the dataset in such a way to mimic real-world scenarios where historical EO data may be available, but concurrent EO collection with SAR is often not possible due to inconsistent orbits of the sensors, or cloud cover that will render the EO data unusable.

A gif showing a small portion (~5%) of the aerial SAR collects and flight lines over Rotterdam as they happen over time on August 23, 2019. SAR intensity data is visualized with polarizations HH, VV, and HV. Imagery courtesy of SpaceNet partners Capella Space and Maxar Technologies.

The SAR data featured in SpaceNet MSAW is provided by Capella Space via an aerial mounted sensor and closely mimics the spaceborne sensors that will be present on Capella’s future constellation of satellites. The total aerial collect features 204 individual image strips captured from both north and south facing look-angles over a three day span. Each of the image strips features four polarizations (HH, HV, VH, and VV) of data and are preprocessed to display the intensity of backscatter in decibel units at half-meter spatial resolution.
Our Story- Data for Governance Hackathon
By: Aminah Mardiyyah Rufai
Tags: Geospatial Intelligence, Geospatial, Data Science, Data Visualization

Data4Governance Project Documentation

TEAM Reispar Analytics Academy

Duration of Hackathon: February 17th-27th, 2020

Team: Reispar Analytics Academy Team

Target Sector: Health

Use Case: Lagos State

Problem Statement: In a densely populated city such as Lagos State, with 20 local governments, there is a major concern for accidents, security and fire outbreaks. There is a need to improve the efficiency of Emergency control services and the response time to help minimize the loss of lives and properties due to these situations. Hence the incentive for this project.

Aim of the Project: To Improve Emergency Response Time in Lagos State, Nigeria.

Objectives:

To identify existing fire emergency units

2. To identify towns and local government areas that were more sensitive to fire outbreaks

3. To identify LASEMA response units within these local government areas.

4. To map out the proximity of LASEMA response units to these susceptible local government areas.

5. To identify possible areas for sighting these fire service stations

1 Team, 5 Tools, 5 Unique Skills

17th February 2020, the hackathon commenced at CCHUB, Sabo, Lagos State, Nigeria, with 20 selected teams. The teams were introduced to, provided links and access to databases on various sectors such as Health, Transportation, Education, Agriculture, Urban planning and Citizen Accountability. Each team was required to brainstorm and choose a sector to work with.

Our team chose to work with and come up with a solution in the “Health sector”. With a combination of skilled individuals across Data mining, Data wrangling, Data Engineering, Data visualization, Design, and Geospatial analysis, we embarked on the 8-Days “Data4Governace visualization” hackathon powered by CCHub in partnership with the World Bank to accelerate the utilization of data for good governance and improved Public service delivery.
A Practical Guide to Working with Geospatial data using QGIS: Part 1
By: Aditya Dutt
Tags: Geography, Qgis, Computer Science, Geospatial, Analysis

Introduction

Recently, I got to work on a research project involving analyzing GeoSpatial data. To analyze such data, visualization is an extremely important step. You can see several patterns easily on a map. If the data contains 100 features, it would be nice if we could plot a map of those features and see if the data intuitively makes sense. QGIS is a nice application that allows analysis, editing, and visualization of geographical data. In Part 1 of this tutorial, we will learn how to use QGIS to display maps and analyze geospatial data. We will also learn to perform some basic manipulations on data. In Part 2, I will talk about some complex operations and how to use geopandas to manipulate geospatial data.

Introduction to Geographical data and QGIS

Geographical data contains a geometry component to give data a position on the map. Geometries can be of different types like Point/ Multi-Point, Line/Multi-Line, and Polygon/Multi-Polygon.

QGIS is a free geographical information system application that supports viewing, editing, and analysis of geospatial data.

You can download QGIS from here.

There are different formats for shapefiles like .shp, .kml, .kmz, .geojson, etc. You can check all formats here: https://gisgeography.com/gis-formats/

For this tutorial, I will use a .shp format shapefile from the SEDAC website.

SEDAC is the Socioeconomic Data and Applications Center. It is one of the Distributed Active Archive Centers (DAACs) in the Earth Observing System Data and Information System (EOSDIS) of the U.S. National Aeronautics and Space Administration (NASA). SEDAC focuses on human interactions in the environment.

I will use a village-level shapefile of the Indian State of Uttar Pradesh. It contains socio-economic features at the village level. You can download it from here.

Load/ Display Data on QGIS

Firstly, open the QGIS application. Click on “New Empty Project”. You can save the project by selecting: Project -> SaveAs option.

A. Load Shapefile Data

To load a shapefile follow these steps:

Layer -> Add Vector layer. Select the .shp file or select the directory in which the shapefile is. Now, Click on Add.

Now, you can view the shapefile.
Visualizing India’s Seismic activity
By: Parul Pandey
Tags: Data Analysis, Earthquake, Geospatial, Python, Data Visualization

Last week, a mild quake occurred around the area where I live. It happened early in the morning and was barely felt (thank God for that). In fact we only came to know about it from the daily news.

This led me to look for the pattern and history of Indian earthquakes. While researching for resources online, I stumbled upon an excellent article from Giuseppe Macrì titled Animating 40 years of California Earthquakes, where he analyzed forty years of California’s earthquake data. In fact the reason why Giuseppe decided to visualize the earthquake dataset because a small earthquake had happened in California back then. Too much for a coincidence, I guess 😃. He writes:

One day, while reading the news, I noticed a small earthquake had happened in California. I decided that earthquake data would make for an interesting visualization.

While reading the article, I was impressed by the tool — Kepler.gl , the tool that was used for the analysis. So I decided to replicate the results for an Indian earthquake dataset, and the results are as shown below:

Analyzing India’s Earthquake data since 2019 | Image by Author

Pretty Cool! Right. So in this article, I’ll help you understand what Kepler.gl is and how it makes visualizing and analyzing geospatial data a seamless task. I hope after reading this article, you’ll be able to use it with your datasets.
OS Hackathon: What is the open-innovation challenge?
By: Os Developer
Tags: Hackathons, Software Development, Geospatial, Ordnance Survey, Innovation

In a recent series of blog posts, we’ve explored how the OS Map & Hack challenges aim to solve problems within sustainability and transportation on a national, organisational, and individual level.

The fourth and final challenge is one of ‘open-innovation.’ As the name suggests, the challenge allows limitless scope and creativity. It’s intended for those who may be new to the developer headspace, and/or have an amazing new idea in mind and a desire to build on that concept, and maybe, even help others explore ideas of their own.

The issue of sustainability is one that will affect us all; indeed, sudden and severe changes in the planet’s weather and climate is already having an effect. We reach a moment in time where differences need to be put aside in order to problem-solve something much bigger than ourselves for the greater good.

Can this ever be achieved? Does it seem too good to be true?

It will take a combined effort of all; and fortunately, the changes required to achieve these goals may already be in motion. The technology is there, we just need to work together to put the most effective solutions in place.

Technological innovation is in an exponential growth phase, across different areas and almost every industry. Technologies such as Artificial Intelligence, next-generation connectivity, autonomous tech and robotics — once the science-fiction dreams that were years away, and now already part of our everyday lives. And they continue to enhance it. It could be with these technologies, forging a disruptive path of innovation, that we find some of the solutions to reaching our sustainability targets.

For example, robotics is key within the manufacturing sector for producing infrastructure and vehicles at an increased pace. Applied with machine learning, it can also identify the quickest and most sustainable methods to the development process.

The deployment of the 5G infrastructure changes the way we communicate and approach data across the world — this could also provide and share real-time data related to the way we manage our energy storage and consumption. It could also be used for gathering real-time information for charging networks and EV routing.

What these technologies need are the new ideas and methods to put them into action, and make the most out of them. Iterative engineering methods and rapid prototyping are examples of developers and data scientists changing the game. They build on existing concepts and architectures that, coupled with the tools and communities to support, can move at a rate never seen before.

Geospatial data is a key contributor to solving sustainability issues, and it becomes an essential piece of the puzzle for analysis and modelling, but to also scale the production and deployment. Particularly in diverse landscapes such as Great Britain.

If you are interested in getting into this space, or collaborating with other great minds, OS Map & Hack provides a great opportunity to do so. For more information and registration, visit our Hackathon page.
Govt. Opens ISRO Facilities To Private Sector
By: The Geospatial
Tags: Space, India, GIS, Geospatial, Isro

With the coming up of Lockdown 4.0, the government has been announcing Economic Packages for industries, businesses, and individuals across the country. Union Finance Minister Nirmala Sitharaman announced the final tranche of the relief package in a press conference in New Delhi.

While the package is all set to bring relief to various sectors, a landmark revelation has come forward. In the press conference, the Finance Minister announced that private players in the space technology and satellite field will now be given access to ISRO’s assets for creating a level playing field.

ISRO has been India’s crown jewel, but the government understands that private companies and startups in the advanced tech space such as GIS, IoT, Big Data, Satellite-based Technology, and Space Technology need to be boosted for further growth. The government aims to provide this support through access to ISRO’s facilities and data, improved policy, and a conducive environment.

It was also said that future projects that involve outer space travel, exploration of other planets, and space travel would not be open to the private sector. The government also plans to make GIS and remote-sensing data more accessible by reducing the policy regulations and putting safeguards in place. This initiative has been taken to give tech entrepreneurs more benefits to enable innovation and economic growth.
Working with Geospatial data — Part:4, Cloud Optimized GeoTiffs
By: Manish Sahu
Tags: Gdal, API, GIS, Geospatial, Python

Introduction

What is CoG? — A cloud optimized GeoTIFF is a regular GeoTIFF file, aimed at being hosted on a HTTP file server. Its internal structure is friendly for HTTP get requests.

It uses famous Byte serving technique. It is the technology that lets you stream a video or music file online and skip forward or backwards through the content. Instead of having to download the full video file, you can tell the server that you want to start at a particular point.

The COG format works the same way by allowing users and processes to access just the portion of a raster file that they need.

COG can do the same thing to Geospatial data what youtube did for video streaming

Why CoG?

Efficient Data Access — Suppose we have 10GB GeoTIFF stored in cloud server. Now we want to make NDVI map of small portion of dataset.

Before CoG — we would have to download whole 10GB dataset then clip your area of interest, then do the processing.

After CoG —Download only desired part of dataset and do the processing. It makes process super efficient.

2. Reduced Duplication of Data — Accessing COG’s with cloud workflows enables diverse software to all access a single file online instead of needing to copy and cache the data.

3. Image Processing of the fly — With CoGs, we can create an micro-service in Flask, Django etc to do image processing on the fly. CoG also removes the need of gdal2tiles like services to view GeoTIFFs on the browers. Several web based platforms like Planet.com, Digital Globe etc are using CoG to do processing on the fly.

For Example — Farmshots is using Planet’s CoG dataset to calculate NDVI on the fly. They didn’t donwload any images from planet’s to their servers rather did all the processing on the fly fetching data from Planet’s server.

Farmshots applying an NDVI to Cloud Optimized GeoTIFF’s pulled in dynamically from Planet

4. Legacy Compatibility — Traditional GIS software is able to treat Cloud Optimized GeoTIFF’s just like normal GeoTIFF’s, so data providers need only produce one format

Try it yourself — Try putting any Cloud Optimized GeoTIFF to a storage service like S3 etc. Just copy and paste the link in the urls given below. You could visualize your dataset on the fly.

Liked it? Do you want to have customization in the processing? Do you want to know how Farmshots like company leverages CoG to do custom processing?

I have create two Github repository

CoG converter — This will convert your standard GeoTIFF file to Cloud optimized GeoTIFF. Repo link Tiling — This repo will help you create an API via which you can do custom processing like visualization of Elevation models, calculate NDVI or even run Deep Neural Networks on the fly. Repo Link [Updated]

For detailed tutorial on cloud optimized GeoTIFF, refer this link.
An open building outlines dataset for New Zealand
By: Toitū Te Whenua Linz
Tags: Building Outlines, New Zealand, Geospatial, Topography, Open Data

We’ve added several new attributes to each building outline.

These attributes provide context and make it easier to work with a dataset of this size — for example they enable filtering to a specific administrative boundary or identifying building outlines that have recently been modified.

Capture attribution

The method and source from which building outlines have been captured provides important context — particularly the relevant dates for the source.

capture_method : Generally Feature Extraction or Trace Orthophotography. Feature extraction consists of image processing techniques to determine building outline geometries from imagery. Trace orthophotography is where a building outline has been manually digitised from imagery.

capture_source_group : For all building outlines in the current dataset, the relevant capture source group is NZ Aerial Imagery.

capture_source_name : The name of the capture source is the name of an aerial imagery layer also freely available on the LINZ Data Service, for example Auckland 0.075m Urban Aerial Photos (2017).

capture_source_from / capture_source_to : This is the date range for the capture of the related aerial imagery. We can therefore be confident that the physical building existed at some point within this range.

Deduplication of geometries

The pilot layer contained building outlines from overlapping capture sources — so in some areas the same building was captured twice or even three times.

NZ Building Outlines only contains one representation of a building, making this layer immediately suitable for visualisations of New Zealand’s buildings.

NZ Building Outlines (All Sources) contains multiple representations of a building — one for each of the capture sources that LINZ has obtained. This allows building outlines to be filtered to those that match a specific imagery source.

It also allows comparative analysis over time.

Below are the same building outlines, symbolising new, matched and removed buildings between the 2015 and 2017 captures. You can view them against the two different imagery sources by changing the layers in the top right of the map window.
Geospatial Machine Learning: Structuring Unstructured, Structured Data
By: Shay Strong
Tags: Geospatial Analytics, Machine Learning, Big Data, Geospatial, Deep Learning

My first, out-of-body-moment reaction to structured vs. unstructured data occurred in the fall of 2016. I was invited to participate in a panel at the IBM Watson developers conference. I met with a group of 6 people, including the panel moderator, to discuss elements of scaling machine learning workflows and hash through our list of panel topics. One of the topics was to discuss the idea of structured vs. unstructured data and how our respective companies use these data types. I think I have been plagued by this topic ever since.

Honestly, structured vs. unstructured buzzwords start to sound a lot like our trendy friend ‘BigData’: weighty but meaningless terms. Internally, I started a fragmented feedback loop in my brain: ‘Do I have structured or unstructured data as a geospatial-based company?’ My brain has been digesting this for the last 3 years.

What I feel like when I think too much about unstructured vs. structured data.

Inherently, geospatial data has a structure. There is an underlaying coordinate system for (all?!) geospatial records: a way to map the data back to a latitude and longitude point on the Earth. Address/parcel (vector) data is structured in polygons or points with geo-awareness with known shapefile or geojson schemas. Imagery (raster) data are pixels captured by a digital camera system. Each pixel may be geo-registered to the ground. The pixels within the raster are quantized elements in spectral (e.g. red, green, blue, infrared) space, limited to a certain electronic well-depth, and constrained in signal responsivity by wavelength via optical filters, bandpasses, and/or the inherent properties of the CCD or CMOS semi-conductor detectors.

(Top) The black & white world of structured and unstructured data. (Bottom) My geospatial world of starting with structured data, asking unstructured questions, and using machine learning to structure the answers over time and content.

This is all to say that the geospatial imagery and vector information itself is structured in capture but not in content. When you get to the level of extracting information from an image, you may know where that pixel maps to on the ground, but you don’t know what it contains beyond being a digital, electronic signal. It is the machine learning, and particularly deep learning techniques, that enable us to scale the content extraction in a meaningful way. We start with structured (captured) data and ask unstructured questions of that data. We then structure the information content that can fulfill those questions at scale.

Oh, and we structure this content not just once! We structure it as a function of time, regional location, complexity (are we looking to just find something? Or categorize it?), and resolution. We are structuring unstructured, structured content. Damn I wish there was a cooler way to say that.
Making Cool Maps with Geopy and Folium
By: Alberta Odamea Anim-Ayeko
Tags: Geopandas, Geospatial, Python, Visualization, Folium

The code

The modules used for the task are imported. A list is created, made up of the 11 cities where the company has branches. For the get_coordinates function, it takes the list of city names as the argument. Nominatim is the geocoder used, stored in the geolocator variable. The iteration is then done, for every city in our list and the geocoder converts them to coordinate values. At the end of every iteration, the coordinates are saved into the dictionary that was initialized at the start as values and the city names, the keys. The get_coordinates function is called on city_list and saved to the city_coords_dict.

Plotting with Geopandas

Since we’ll mainly be working in geopandas, which has a plot function, we can use that to visualize our cities before going on to plot in folium. One reason for using it is because it’s great for vector data and another is because we’re working with real world points(cities) in 3D and not just points in a 2D plane, so we’ll need that CRS attribute. Okay, let’s dive in!

We create the list of shapely points for all city points and store in a cities_geom. d, the dict is created with Cities as the key, city names, the values and for the geometry key, the shapely points of the cities are stored as values. The cities_gdf is created by passing the dict, d and crs to the GeoDataFrame(a pandas dataframe with a geometry column) method of geopandas and then plotted. But it doesn’t mean much, does it? It just a number of blue points plotted on 2D axes. Now, if these points are plotted on the African map, the visualization will be improved and you can see these points in relation to each other and also see countries which have no points(or representation, if you like).

Geopandas has a number of datasets that you can explore. The one we’ll be using is the naturalearth_lowres, which gives us access to the world map, but our cities are all African so we’ll extract all rows in the geodataframe which have continent value equal to Africa, and store it as africa_gdf. We then employ the plot function again and pass parameters color and edgecolor, just to customize our plot. plt.rcParams lets you control the size of the plot. To label the African countries(which are represented by polygons), we find, for every country, points which will definitely be found in their respective polygon(using representative point().x, representative point().y) and then place the labels there, using annotate. The final thing we do here is to assert that crs of the cities_gdf is equal to that of the africa_gdf because if it isn’t, our points would not fall at their right places.
The US Tree Map
By: Taylor Mcdowell
Tags: AI, Remote Sensing, Artificial Intelligence, Geospatial, Trees

The US Tree Map

Who doesn’t love trees? They are benign beasts that bring beauty to our lives. Some grow to dizzying heights, eke out a living in the most inhospitable of places, or live to be thousands of years old. Others simply shade us from harsh weather, light up in brilliant colors each autumn, and provide habitat to wildlife. When grown in large numbers, they not only support the largest timber industries globally, they can also be effective carbon-sequesters, climate-regulators, water-filtrators, and ground-stabilizers. Undoubtedly, our nation’s forests are an American heritage.

Trees say a lot about us as a society. They are managed, cultivated, and cut down as per our demand for wood resources. They have been known to influence real estate values, or reveal social and racial inequities that plague our cities. Urban neighborhoods are getting greener as a result of an increasing value in green spaces, and to many, planting more trees is a solution to the worsening climate situation. The point is: trees deeply matter to us.

Why map them [again]?

It’s been done before. Like all natural resources, mapping the distribution of trees is a tool to better manage our forests and study changes over time. Even dated maps such as the one below have their value in understanding long-term changes to forest cover.

Woodland Density Map in 1873 by W.H. Brewer for the “Statistical Atlas of the United States” — courtesy of the US Library of Congress

In the era of remote sensing and machine-learning, mapping trees has become quicker, more accurate, and can be done with greater frequency. Take the National Land Cover Database (NLCD) for instance, which relies on 30-meter Landsat imagery to produce a tree canopy density map across the nation. With seven epochs of this dataset dating back to 2001, the NLCD is the longest-running land cover database available here in the U.S.

While the NLCD might suffice to observe spatial-temporal trends over larger tracts of forest, it lacks the finesse required for high-resolution, large-scale analyses. What if we want to observe the individual trees themselves rather than proxies like “density” or “probability,” yet maintain the sheer spatial and temporal scale that the NLCD succeeds in? Cue the US Tree Map.
Remote Sensing and its future?
By: Rishabh Jain
Tags: Remote Sensing, Sar, Geospatial, Space Technology, Satellite Imagery

Remote Sensing

If you guys are reading this then you would have come across this term while reading about climate change or something to do with satellite image processing in various domains. Many of you would have given a pass to this term and wouldn’t have bothered to understand the broad meaning of it. But as remotely sensed images are becoming more common and are being used in many analytical data stack there is a need to understand this term.

According to NOAA : Remote sensing is the science of obtaining information about objects or areas from a distance, typically from aircraft or satellites.

Remote sensing in itself has been a hot topic for research for a long time and still is, it was present before the inception of Deep Learning. Then the Data was coarser in resolution (Spatial, Temporal & Radiometric) and was limited to only few use cases.

At present, remote sensing is being perceived in a very different way than what it use to be before. This change is due to availability of varied open source satellite data and high resolution satellite imagery surfacing in open datasets. Moreover new research integrating deep learning with these datasets have opened up a plethora of use cases in tech as well as in business world.

It is also considered to be the heart of satellite image data processing. Science of remote sensing is actually about how information is captured, stored and how we can generate insights from it.