How to Export Raw Data from Google Analytics
By: Александр Пономаренко
Tags: Pipeline, Google Analytics 360, Google Analytics, Owox Bi, Raw Data

Benefits of sending raw data to Google Analytics

For businesses that opt for using the OWOX BI Pipeline for raw-data exports, all hits get transferred to GA (Google Analytics) and are concurrently delivered to GBQ (Google Big Query), from the site, including every click and action. It enables the availability of relevant data at almost the same time in GBQ. OWOX BI’s unique sessionization algorithm helps to compute data from every session. Users can learn about the detailed process at the Help Center.

This makes it possible to get raw-data from GA, irrespective of the quantity of traffic your site gets. There is a free trial period of two weeks after which OWOX BI is priced at $115 on a monthly basis. The collection of unprocessed and raw hits with the export of Google Analytics Data to GBQ carries several benefits.

The paid version of GA is limited to 200 CDs (customized dimensions) with the standard version set at 20 dimensions, at present. However, the use of GTM (Google Tag Manager) makes it feasible to gather as many versions as required in GBQ.

Unlimited reports (with all types of combination) of metrics and dimensions are possible for an unlimited time.

There are no hit limits for user behaviour sent in real-time to GBQ.

It is also legally valid to collect the personal data of users such as phone-numbers and email-ids.

Making use of GA’s APIs

Data retrieval through APIs is feasible through GA. For specific reports not within the GA interface, core reporting API offers accessibility to metrics and dimensions.

Additionally, to get data for every hit and enable integration of data there is a workaround. It is possible in GA through CDs along with API functionality.

For the capture, analysis and to evaluate information not offered by GA there is the option to use CDs. Apart from using them to combine data from GA and others, it can be utilised to get improved reports of the significance of business. E.g. A database’s user id can be saved that can be used to integrate online and offline activities.

CDs samples:

User ID: To check all the hits of a specific user in different sessions a CD can be used to gather values of the user.

Client ID: Session wise dimensions can be used to gather single values of a device from the _ga cookie.

Timestamp: The precise timestamp of when the hit occurred yy-mm-dd-hh-mm-ss from along with the time zone is possible with a CD.

Session ID: Individual session-scoped dimensions can be used to gather random values of hits from a similar session.

The list of samples is from Simo Ahava’s article on data collection using GTM in conjunction with CDs.

API not a perfect resolution

One of the major concerns for users is sampling and whether GA will help to resolve the problem. This is subject to the online traffic at a site with those sites having less or limited traffic having the option to elect a brief reporting period, thus precluding the need for sampling. In cases where the traffic is heavy, it necessitates executing queries in their hundreds for data not sampled.

When the data is sent from GA, it includes processing, metrics, dimensions and time taken to process data. However, for small businesses API offers a provisional solution. Some of the limitations to GA APIs include metrics, dimensions and data quantity on a daily basis. Besides, there is a requirement for space for the data that is transferred.

BQ transfer for GA 360

While GA 360 is fairly expensive it definitely does offer value for money. Apart from the latest features, GA 360 enables transfer of unprocessed data per session from GA to GBQ. This is accomplished through native integration.

The two ways to go about it include:

Data transfer continuously: On a daily basis, a single file along with GA data from the preceding day and 3 more files are transferred per day. Additional information from Google’s other services is also accessible.

Transfer data continuously: Using this method the user gains access to fresh data that is transferred to GBQ around every quarter of an hour. There is a price of $0.05 per gigabyte of processed data charged by GBQ. However, a point to note is that GA linked services like ADX, AdSense and Double Click (publishers) are sent daily for the preceding day.

For users grappling with sampling, the good news is upon connecting GA to GBQ, GA360 transfers 13 months or 10 billion hits of data automatically to BQ. The icing on the cake is GA360 subscribers get $500 monthly credit to compensate the import, storage and data processing in GBQ.

Create your personal connector

There is the option to copy the hits sent to GA and process copied information somewhere other than GA. There is the option to use cloud servers or as an alternative to consider the use of a personal server to store cloned hits data. While hit based data does not provide source or campaign data, location or cost info it will help in other ways. You can get unprocessed hits from your site that can be utilised where there is no need for session-based data. E.g. analysing site performance and sending transactional emails.

Collection of data stored

Irrespective of the scope of your business whether a start-up or a major corporation certain aspects need evaluation before opting for data storage. A brief outlook at some of the available options includes:

Scope of processing data: While unprocessed data collection is a must, there is a need to be able to effectively process the data to get the info needed, or else it will be worthless. Subject to the requirements of the enterprise there must be flexibility to scale. When the venture grows your storage and processing facility should be able to adapt as per business needs.

Top-notch security protocol: Every bit of vital data must be secured from hackers and cybercriminals using state-of-the-art security protocols.

Cost-effective: Thanks fully with a plethora of options available online and excellent services offered by GBQ it is an effective solution to store your data. It is specifically designed for data analytics and offers cloud-based storage.

Reasons to opt for GBQ

When it comes to storage and processing of data, GBQ in unmatched with data limits up to petabytes! It uses SQL and offers the best in its class features like an amazing speed of processing, absolute data security and scalability etc. In short GBQ offers all required to process and analyse gigantic data quantities.

Although GBQ charges a fee, the cost is limited to data processed and stored only. For the first 10 GB of storage and 1 TB of data processed there is no charge. Post that there is a fee of $5 per TB (processed) and $0.02 per GB of storage. At present new GBQ subscribers get an attractive offer of $300 worth credit for 1 year.

Limitations of GBQ

For newcomers to GBQ, it will take time to get the hang of how GBQ works. Remember GBQ offers nested/ repeated fields. On the other hand, GA’s data is hierarchical (hits/sessions/users) so it takes time to learn how to extract information from nested/repeated fields. It will take time and practice to master using GBQ. Also note, when we use GBQ with certain metrics from GA e.g. complete number of users and events, these are not automatically calculated.

Conclusion: Initially moving unprocessed data from GA seems easier than it actually is. However, it still is invaluable and worth exploring to use.