What is the most optimal way to structure your data organization for maximum impact to your business and your customers?
I've spent a number of years organizing and growing data teams and have some results to share…
Today's business world calls for adaptive data teams that can produce results in advance of people asking for them. As a data team we need to be thinking ahead of what requests we'll get and start preemptively collecting and structuring data for analysis. This way when questions do arise we are well prepared to provide guidance to business decision makers.
First and foremost you need executive support. Without this you'll have little chance of success since the resources and guidance needed are substantial for any business.
Let's take a look at the process for how data is transformed from raw bits into meaningful insights.
Data are generated in a few places typically for most organizations. From there, the data is landed into a back office staging environment, typically some form of a database, then is transformed via data modeling techniques, into usable data sets for analysis.
After the data is collected, organized, cleansed, and structured in a human friendly format, your analysts and data scientists can start using it to solve business problems.
This leads us to the three teams I believe are necessary to give your data team the most opportunity for impacting the business in a positive way. Here is the breakdown of those three teams and their responsibilities:
Data Engineering
- Building the platform for which all data is collected, organized, and analyzed
- Ingesting all data into this platform
- Cleanse data ingested into data platform
- Prep data for analysis based on guidance from data modelers and data analysts/scientists
- Maintain and support these environments to ensure reliability and performance
Data Analytics
- Work with Data Engineering to ensure data is properly structured and complete for analysis
- Support business decision makers by providing easy to use data tools and guidance on how to use them
- Coordinate with Data Science team deeper studies that should be performed and any preliminary findings
- Develop and provide analytics platform for business users to find and interact with data
Data Science
- Work with Data Engineering and Data Analytics teams to ensure sufficient data is being collected and is organized in a proper way for use
- Work with business users to support strategic decision making that will have a great impact to the business or customers
- Work with Data Analytics to uncover deeper questions to explore using statistical and machine learning methods
- Build intelligence systems that can enhance the business' processes and products
- Provide guidance and mentorship to business and product teams on proper interpretation of data and how to test hypothesis
With these three teams all working together you'll be setup for success. In following posts I plan to dive deeper into what the key ingredients in each one of these areas are so you can understand what it really will take to become a data-driven organiztaion.