SAP BW vs. Snowflake: End-User Experience
By: Serge Gershkovich
Tags: User Experience, Sap, Tableau, Technology, Snowflake

A year after the platform migration, do the business users share BI’s excitement?

Photo by Scott Graham on Unsplash

A year after my company migrated from on-premise SAP BW to Snowflake Data Cloud, I wrote an article comparing the two Data Warehouse (DW) platforms and highlighting the key factors which made the project a success. Given such a niche topic, I never expected the amount of feedback and healthy discussion that the article continues to generate.

However, one crucial detail was missing from that conversation: the end-user experience.

In a DW migration, winning over the business users is just as important as getting the model right. Although our migration was a massive success from a cost and performance perspective, I couldn’t yet speak for the users. At that point, we had already migrated the main reports, but the users were still getting their footing and becoming acquainted with the new platform’s various features.

Now that another year has passed, I would like to extend the analysis beyond the Business Intelligence (BI) team’s perspective and focus on our analysts and business users. What did the transformation mean for them, and are they happier and more productive as a result?

Conclusion

Traditionally, this comes at the end. But anyone who has been following Snowflake already knows that their platform empowers companies to transform their BI in ways they never imagined. Our experience was no different.

The user’s transition to the new platform was rapid, and the feedback was, without exaggeration, 100% positive.

Independent surveys have shown similar findings in other Snowflake customers:

Snowflake customer feedback

So for me to take such a typical outcome and attempt to use it as a conclusion would have been anticlimactic indeed. Especially when the question on everyone’s mind is:

How does Snowflake manage to achieve such results?

That’s what I’d like to explore.

Data Cloud > Cloud Data

Snowflake is not just another cloud-based DW. It’s a platform built to “mobilize data” using scalable architecture, engineered for the cloud. It uses four key pillars to achieve this goal:

Single source of truth (SSoT), many workloads

Unlimited performance and scale

Secure and governed access

Zero-maintenance as a service

To this, I would also add a fifth element which I think is overlooked when it comes to business-user adoption:

Ease of access (Through a web-based portal, using standard SQL)

With these principles in mind, my team saw an opportunity to do more than replicate existing reports in the cloud. We realized that Snowflake had the potential to change the relationship our users had to their data.

Not only that, it meant that the BI team would no longer have to be the bottleneck to getting that data to the users exactly when they need it. Instead, the BI team could become the facilitator of a company-wide analytics strategy that empowered all our business users through a self-service model.

Sounds great, but let’s not lose sight of what we’re up against: a mature, powerful, and robust on-premise SAP HANA database with hundreds of existing reports and user-configured views.

Senior management had not mandated or even authorized the migration. As the users were under no obligation to switch platforms, we knew that we’d have to win their support if we wanted our plan to succeed. That left us with only one viable strategy:

Let data quality and user experience drive the conversion.

Let’s look at how Snowflake made this possible over a pre-existing SAP BW implementation.

Same Reports, Only Better

After having migrated the model, our first objective was to provide the core set of reports in a format that the users were already familiar with. In BW, this meant that the bulk of the analysis was done through tabular(and interactive) WEBI reports, while the graphical (but static) reports were distributed through Business Objects. Content creation in both of these tools came with a steep learning curve and licensing constraints, making it impractical for business users.

No longer confined to the SAP ecosystem, we chose to do our core reporting in Tableau — a Gartner Magic Quadrant leader in BI analytics, who also offers a SaaS cloud-based option that plays well with Snowflake. Tableau combines tabular and graphical reporting in an intuitive interface, capable of connecting to many different data sources.

Tableau not only gave us the possibility to migrate existing reports with all the interactive and visual features that our users had come to expect, but it allowed the users to create their own dashboards, either from scratch or using pre-defined templates.

With reporting taken care of, what else could the BI team do to empower our various business teams?

Business Users in a Data Warehouse?

Beyond the mere learning curve of BW reporting, there was a constant risk of runaway queries bringing down the performance of the entire system. Sure, there is always the QA environment, but its data is perpetually out of sync with PROD and offers little value.

Enter Snowflake with virtual databases that can read or write to one another without copying the physical data. Couple that with the ability to adjust compute resources independently from the storage, and you give users real freedom.

Unlike with our BW system, there are absolutely zero risks in giving users read access to the DW models or even raw data. Since their queries run in a separate and adequately sized reporting warehouse, they never interfere with the one used by BI.

Hold on, how are business users even able to access a data warehouse?

Remember, Snowflake uses native ANSI-standard SQL and can be accessed through a web interface with no additional software to install. Unlike BW, which takes years to master, a day’s worth of prep can allow someone to start writing basic SQL queries and extracting insights from the underlying data.

The recent release of row-based security puts Snowflake’s security features on par with those of SAP. However, Snowflake can do something that SAP users have never even dreamed of: give individual teams their own database.

What would the business users want with their own database? Quite a lot, it turns out! First, let’s review all the new abilities that our users gained by migrating from BW to Snowflake, and in the next section, we will look at what they were able to achieve as a result.

Snowflake gave our users:

Ability to access an SSoT across the entire business

Ability to access data beyond the reporting layer

Ability to extend existing reports and create new ones (Tableau)

Ability to manage their own warehouses for more complex tasks

Ability to choose their budget and size their warehouse to their needs

Inability to bring down the system or even penalize the performance of other users and processes

Maslow’s Hierarchy of Self-Service

Ok, maybe Maslow had individuals in mind when he introduced the concept, but Snowflake helped us see how it applies to analytics. By continuously empowering our business users to get more out of existing data, we realized that self-service isn’t a singular concept — it has layers.

The layers of organizational self-service

Each department has distinct and creative ways of using data. Instead of providing a one-size-fits-all solution, the BI team should be looking for ways to give these teams as much autonomy as they were willing to take on.

Exploration

For some of our users, the core reports completely fulfilled their everyday needs. However, the new capabilities that the Data Cloud gave us, meant that we could do more for the users and departments who wanted to go beyond the basics — to ascend the self-service pyramid.

Insight

One of the most requested features was direct access to master data and the ability to track ETL logs. Teams who previously never went beyond front-end reporting saw this as the gateway into the SQL world. Suddenly, every master data table became a data source for business consumption. Users could even query metadata (e.g., last refresh date, count, and distinct values). Requests like this used to sit for weeks on our BW backlog — needing dashboards to be built in order to expose simple tables.

The next rung in the self-service pyramid is business users’ ability to load and maintain department-specific data (e.g., groupings, hierarchies, classifications) and join it to existing reporting models. Previously, BI would have needed to build the pipeline, the data model, and then accommodate every change which arose over time. Now that users could operate their own warehouses with shared and secure access to productive data, they are free to perform ad-hoc analyses without depending on BI.

Actualization

Finally, the self-actualization of analytic needs: ad-hoc modeling and data science. Why should technology be a barrier for teams with the technical knowledge and desire to run these analyses?

Whether it’s doing department-specific modeling or running predictive analytics on business metrics, Snowflake’s ease of access makes it possible for teams to move at the pace of their own ideas — without depending on BI.

A department’s ability to manage its own warehouse and take on the associated costs gives our most technical users complete autonomy. Whether the data resides within the warehouse of a specific team or the central DW managed by BI, it can instantly be made available to the entire organization thanks to the Data Cloud.

We Could Have Done Better

Despite having achieved our goal of getting the buy-in of our business users, there were indeed things that we wished we knew in advance.

Before we understood the self-service hierarchy and how the business would eventually come to use Snowflake, it wasn't easy to foresee the importance of governance and automation— both at the database and reporting levels.

Database Governance

Once our business users began to derive value from having database access, knowledge of the database landscape could no longer reside entirely within the BI team. Strict database naming conventions become essential. Unless the users can navigate the DW, they do not have true self-service. Since this wasn’t even a possibility with BW, it took us a while to realize its importance (and find time for the necessary clean up).

Reporting Governance

Beyond the staging layer, as data models get more complex and business logic is introduced, a data dictionary becomes essential. An SSoT is only as good as the user’s ability to make sense of it.

Since Snowflakes’ Data Cloud makes sharing data within an organization trivially easy, we could integrate more sources into our reporting model and do so with a much faster release cycle. Unfortunately, this meant that our reporting layer was evolving faster than users' ability to keep up with our memos—a good problem to have, but a problem nonetheless.

Eventually, it became clear that maintaining an up-to-date data dictionary was crucial to the success of our self-service strategy. Although we’ve begun to address this through a manual effort, it’s becoming clear that we will soon have to rely on an automated solution.

We’ve begun looking at tools such as SqlDBM, which comes with an integrated data dictionary that syncs with data model changes. Because SqlDBM also supports visual modeling, enforces naming conventions, and helps visualize relationships between data entities, it would provide tremendous value to BI, as well as business users at every stage along the self-service hierarchy.

Continuous Integration and Deployment

Finally, even the best-made model can suffer from poor data quality. SAP systems do a great job of ensuring (else failing on) data integrity checks. With an open ELT approach, additional safeguards must be built-in before the data can be trusted in the reporting layer.

Once we understood this, we incorporated duplicate value and threshold checks into our data model. But if we want to scale, we’ll need to implement a CI/CD solution like DBT, which will allow us to automate data integrity checks and ensure a more stable and reliable delivery cycle.

Conclusion. (For Real This Time)

In the first year of being introduced to Snowflake, the BI team managed to migrate the entire data model and completely transform our data warehousing strategy.

In the second year, the business users caught on to Snowflake’s potential and wholly abandoned the on-premise BW/HANA system in favor of what the Data Cloud could do for their data needs.

As we enter our third year on Snowflake, we continue to push the frontiers of what the platform can deliver. Using the Data Cloud, BI and business-matter experts from across the business are collaborating on machine learning initiatives and automated apps.

Data scientists, programmers, and business analysts can work across organizational lines along with BI on a single shared platform and reinforce each other’s strengths. Despite boasting of performance and efficiency, this collaboration would have never been possible with the traditional limitations of an on-premise system like BW.

Admittedly, this is still not much of a conclusion — it’s a promise of bigger things to come on our Snowflake journey.