This post is a useful way to think about how to start on a data journey if you’re a young startup that’s just pushed data to the back burner (say until you had ‘enough’ traction) or even if you’re part of a more mature company that’s used to making decisions more on instinct and experience, but now want to complement it by building data capabilities.
Data analytics doesn’t promise magic. That said, the commitment to becoming data informed helps you make measurably better decisions, backed not just by gut but also fact. As this post will show though, the journey to becoming a data-driven or data informed business is definitely a marathon, not a sprint.
There are three fundamental objectives that your data can help you with:
- Build better relationships with your customers
- Grow your topline
- Maximize your bottom-line, i.e. ensure you spend as efficiently as possible in accomplishing business objectives
These top-level objectives give rise to lower-level objectives that vary according to the nature of your business. Once you’ve identified the lower-level objectives, data can shape the maximizing activities or initiatives your company needs to work on.
In order to get on board bus to becoming a data informed business, you have to prime the usual suspects – People, Process and Technology. Let’s begin.
Eventually, you want to have a data team with three distinct functions:
- Analytics (think business analysts) – answering ad hoc questions, building reports
- Data Science – answering more strategic questions
- Data Engineering – building data pipelines, supporting the data technology stack
How these teams are structured within your organization depends on your specific business, but here’s a sample org chart
First hire: You want to gradually build a team that grows with the rest of the company on this data journey. My recommendation is to start with the Analytics team, by hiring an all-star – someone that can work as an individual contributor in the early stages, so you start seeing returns on your investment immediately. The secret sauce here is in hiring an individual who also demonstrates great leadership skills because they will be key in evangelizing the culture of data throughout the company, so hire someone with this skill ratio
Leadership – Interpersonal relationship building, in the context of business objectives; BA skills – Basic Quant skills, Excel whiz, Fluency with SQL and BI tools
This hire is a crucial one, because in the infancy of this data-driven journey, she’s going to be the workhorse that helps answer basic questions from the quagmire of data available. She’s also going to help evangelize the culture of asking questions of the data throughout the organization, and will be closely tied to decision makers throughout your organization.
Here’s a timeline for the hiring of the rest of the team.
Notice the hiring of the Data Science lead around 8 months out. My view is by that time, your organization will be asking more interesting, challenging questions which will require more advanced skills than those of the BAs.
Note also the hiring of the Data Engineers and Architect around the 18 month timeframe. I’ve assumed here that the company grows in scale and complexity, and with it come operational issues around data hardware as well as the business need for rapid experiments that require a concerted effort from engineers with a broad expertise set in managing and marshalling data. These are all good problems to have, but they should be planned for.
<REMINISCENCE> As a child, I found myself enrolled in a foreign language class. Come the exams, I had studied nothing. Each letter of the alphabet looked to me like the same squiggle, with minor variations. I’m told the questions on the tests were trivial, for someone that could read the language. The question would go something like “Why did the boy go to the beach and pick up a shell?” I had memorized what the interrogatives in the language looked like, so in my answer, I’d just drop the ‘why’ and copy the rest of the squiggles – “the boy go to the beach and pick up a shell…” – as though the answer were on the tip of my tongue, but I’d gotten done in by the lack of time. And yes, I’d get partial credit for this </REMINISCENCE>
Pictured: Me at the foreign language exam. Hadn’t the foggiest. Source
You know the context of your business best, and you will come up with the best discipline/rigour/process that your data needs for your business to derive the most benefits from it.
Make sure your process consultant doesn’t just drop your interrogatives and regurgitate your questions to you.
That said, here are some guidelines to start you off:
- A lot of data architecture focuses on the storage of data, with far less attention to the down-the-road need to access and retrieve it quickly for analysis
- Data collection is an iterative process, both in terms of quantity and quality
- Quantity – Think of the questions you want to be able to ask of your data 6 months down the line, and then examine the types of data you currently collect to see if you’re missing essential parameters. Tweak your data collection process
- Quality – The data you’re collecting is bound to be messy. There will be incomplete fields, subjective entries, redundancy. Budget for data cleaning
- The types of questions you will want to ask will change over time. Keep your people agile, and your infrastructure flexible
- Kick off a culture of of asking questions of the data (simple questions) every day and seeing where the current system gets stuck or slows down in answering them
- Plan for automation/lubrication of those friction points. It is usually one of the triad of People, Process or Technology that will be the blocker. Before you take action on automation, see if the plan is a bandaid or if it is strategic. That should inform how much you invest in the action
- Modeling for persistent questions – build processes to ensure that questions that come up repeatedly can get answered faster and cheaper every time they are asked
By leaning into data, you’re inserting a fundamental cornerstone of how your company will take decisions and do business over the long term, into its core structure. In this regard, one doesn’t want to bite off more than one can chew. Effectively, even though AI and ML seem to be the buzzwords in every iconic company’s data outlook, you need to start with the basics and work up, over time, to understanding which, if any, trending technologies will add value to its business.
Think about your data technology in three layers:
- Infrastructure – Basically Storage, Network, Security.
Many organizations find that Amazon’s cloud computing services are a great way to get started. Their services are low cost to implement, they have usage-based pricing models, which is handy when you don’t want to invest too much in technology you may not find useful later. They’ve invested heavily enough in network and security that you cannot separately build a stronger versions of without spending through your nose.
A combination of AWS’ Elastic Compute Cloud (EC2), Simple Storage Service (S3), and Relational Database Service (RDS) should suffice for a minimum 24 month long data journey.
- Business Application(s) – These are the applications core to the nature of your business, that help you transact with your end customer, and feed the data back to the main database of your company.
For example, if you’re selling a catalog of items to consumers in an e-commerce format, you might start with shopping cart applications like Shopify or Magento. When your business gets large enough, or your data gets complex enough, you might find that you need more advanced or customized features, and that’s a good problem to have. At that point, vet the state-of-the-art again to see how it measures up against the option of building in-house.
- Tools – the software and tech stack that help you manage the limbic system of your business.
I’d further classify these tools by staggering their implementation to keep pace with the data maturity of your business. The specific roadmap of tools is part of the data strategy the consultant you hire will provide, but you might broadly consider the staggering by
- 6 months out: Inventory management software; CRM; Business intelligence tools
- 15 months out: Data pipeline; Data modeling tools; Visualization & App analytics
- 24 months out: Large-scale compute; Augmented data discovery
Undoubtedly, you see the limitations of this post as a roadmap or a data strategy for your company. Both of those are completely context sensitive, but my hope is that this assembly of the People-Process-Technology triad gives you a sense of scope and what you need to be thinking about as you start the transformation in your own company. I’m always happy to discuss this in detail.