A Paradigm for Business Intelligence Evolution

This is a series of ideas that will outline how we need to build an e-commerce Analytics solution for a digital marketing agency or consultant to join the data from all channels and client’s data together in one central repository. This data should include online transactional data as well social interactions.

This repository should also be able to import offline customer information too. Additionally, this data will then be combined to create analyses across platforms and, ultimately to be used as a source of predictive analysis.

We will end this document with some timelines around the tools to use, the timelines around their deployment and kind of internal team to execute this plan.

Process Flow in Summary

1. Setting up data collection within current data sources.
2. Merging all data sources into one platform and automate such a collection.
3. Analyzing patterns in these datasets to builds reports and dashboards based on KPIs
4. Based on past behavior of customers, create prescriptive and predictive Analytics around key metrics and goals.

Data Collection

It is imperative to get all the measurements set up correctly in every marketing tool that is used. We tend to focus mostly on Google Analytics and believe that data alone. Sometimes, when tagging is incorrect on email platforms, Facebook or Bing, we cannot even measure conversions in the same way and that leads to reporting that is erroneous and leads to bad decisions. In this step, it is important to measure each source of traffic to a web domain and be able to capture that either within that domain (for example: clicks from Facebook reporting) or make it work so that the data is captured correctly in another tool like Google Analytics or Adobe Analytics. We need to be able to account for 90% of traffic correctly before we move ahead.

As this data collections are all in separate silos that usually do not support each other well, we will get some numbers that will be off between each tool. That is a cost of doing business at this stage and we should only focus the primary data source when making decisions. For example, if we want to determine whether Facebook CPC is effective, we use the conversion metrics from Facebook reporting rather than try and match that data with Google Analytics.

This becomes rather onerous and requires us to look at several disparate places to get a complete picture of our business, so we then are forced, for the sake of efficiency to move to the next stage.

Merging Data Sources

At some point the enterprise slows down because there is too much data, but no unit can understand the other’s language. If there is an Analytics team, an SEO and SEM team, the confusion arising from a lack of a “single source of truth” can be quite debilitating in terms of decision making. At this point we start to go down two different pathways. We can either build our own data warehouse or buy a tool that will do this for us.

Building a data warehouse comes with the advantage of designing your own data collection and downstream KPI’s that the can be customized as desired. There is a high initial cost but then the flexibility of operation tends to result in increased ROI. Dealing with internal resources also means a quick deployment of any changes and newer traffic sources can be ingested sooner. The data warehouse can be used with any of the data visualization tools like Tableau, Power BI, Qlik or Google Data Studio.

Buying a data merging tool like, Alteryx, Looker, ReportGarden, Optymyzer or BIME has it’s advantages in that the initial cost is lower and they provide built-in connectors to all major data sources. You can also customize them by importing new data sources using API’s or just csv files. Over time, they also end up creating new features and introducing concepts of machine learning and programmatically produced insights that can be of great use to clients also. One doesn’t have the flexibility of an in-house team though.

For an agency, perhaps the latter option is one to start with and the former to aim for in the long-term if the revenues from the Analytics stream are high enough.

Analysis

Once the processes are in place and perhaps even at the point of data collection, we need to have a core set of people who are going to make sense of all the analysis and provide recommendations to the clients based on all the dashboards and reports that can be built. It has been my experience that lots of dashboards are made and reports are regularly generated but the consumption on the client side is poor because the analytics practice does not translate the numbers into business speak. Please read this great article by Avinash Kaushik that further elaborates the point.

I cannot stress enough the fact that the quality of good analysis and a set of concrete recommendations that clearly explains what the client is going to get by following them goes a long way in the adoption of data driven decision making. For example, a recommendation that says “Moving the add to cart button 300 pixels up will increase revenues by $30k a month” is much better than saying “We find that testing suggests moving the button 300 pixels up will increase conversion rates”

A good web analyst will have experience working with product development, a smattering of coding and long exposure to different kinds visualization techniques. An investment in a good resource(s) of this kind will yield more value over time than the best tools around.

Predictive Analytics

Once a website is running and customers are using it, a one size model in terms of pricing, deals or discounts does remain a good strategy for long. Very soon clients run into a wall where they keep acquiring new customers or current customers without enough understanding of any of them. At this point, you are looking for tools that segment the population in meaningful chunk and target them based on their needs at a particular time. A pet food company can predict when a customer will run out of the current cat food that they have purchased and send a targeted message (email or ad or website chat) at the right time.

There are several open source languages like R or statistical packages like SAS and Tableau that can help an internal analytics team do this work for a client. These require the in-house team to have a strong data science background to create a product offering for clients. This method will give you the best ROI because the tools will be made in-house and can be customized based on a client.

The other method is to buy a tool like Domo, Qlik, clearbit, Zaius and use it’s features to conduct user based analyses that can segment and target customers in different ways.

Timelines for Capability Development

NOW: We are doing Data collection and collation right now in our efforts to serve customers. Pulling data for separate business functions from different tools is the ground situation now.

Six Months: We can build up a capability using an external tool to combine data sources and provide comprehensive and cross-platform reporting.

18 Months: It will take a lot of time and learning to build up an analytics resource team in-house that has experience in all the different domains. If you start hiring now, such a team will be able to start adding value within six months and be giving very good ROI in a year and a half.

2 Years: This is the time it will take for a good team and a set of tools to be able to make deep inroads and to build predictive analytics and modeling capabilities. It will also take time for our client base to become mature enough to realize that this is the ultimate need for any online business.

Conclusion

I hope this plan gives you a sense of what I think the plan to success in the e-commerce Analytics domain. There are many ways besides this to get data together and put analytics to work but in the e-commerce world this should work the best.