Best practices, Building data pipelines

Speed, data science, cloud and more speed: How to modernize analytics

Reporting and analytics are critical for effective decision-making in any company and many companies today are reevaluating their programs, with an eye to a more unified approach that can include data science, and modern cloud architecture.

In good economic times, you want to make sure that you’re investing in the right areas and making the right moves to ensure continued growth. As a result, there’s always some re-evaluation of analytic capabilities going on as companies’ needs change and new tools with new capabilities become available in the market.

But when times get tough, as they have for many companies during the pandemic, reporting and analytics take on even greater importance. Companies have been forced to do a lot more analysis. Some of the questions being asked are existential: Should we double down on our core competency, or change direction? What is the best way to survive? The number and consequence of the decisions, and the urgency with which they have to be made has given a new urgency to modernizing the analytics stack. 

To that end, I’m pleased to have had the opportunity to collaborate with David Loshin and TDWI Research on a new webinar, “Modernizing Your Approach to Reporting and Analytics.” In it, we cover topics such as how to select a modern environment; why you shouldn’t simply “lift and shift your existing systems;” and enabling data democratization and data science storytelling. It’s a really nice guide for what to consider as you are looking at your data analytics strategy.

For me, three high level things stand out as ones that companies really need to get right as they modernize: speed, increased data science capabilities, and strategic use of the cloud. All three are closely related.

Speed is absolutely essential, because markets are changing so rapidly. Companies are disappearing every day, and new companies are popping up and getting to scale in an amazingly short amount of time. Due to the availability of technology on the cloud, they can get up and running with little in the way of investment in infrastructure and IT. 

To stay relevant, established companies need to really be nimble. Part of that is being able to access data very quickly so they can make decisions very quickly. If decisions are taking weeks or months, you’re going to fall further and further behind.

But it’s not just getting at your data, but getting at the detailed data. This is what’s required for data science, which is now becoming a very real contributor to better decision making, and therefore a bigger part of people’s strategies. 

The way of old has been to take your detailed data, aggregate it, summarize it, put it into a data warehouse, and then do analysis and reporting at the aggregated level. However, data science is not done at the aggregated level. You need that historical detail, and the more the better, because that is what gives a model good predictability.

You want your data scientists building more and better models, but they have been forced to spend way too much time curating their data to get it into a trusted format that they can then put through their models. And data scientists are not cheap. Enabling data scientists is in essence, a speed challenge–it just takes too long to produce models if you can’t get fast access to detailed data. The old way does not work for data science –or best-in-class analysis.

With new tools like Incorta, data scientists can get directly at large volumes of data in its detailed native form, without the need to extract, transform, and load. And, that same data that is available for data scientists is also available for reporting and analysis via insights and dashboards. 

The cloud is now a trusted platform for providing these capabilities. Besides all the IT savings and efficiencies that we’re familiar with, the elasticity of the cloud makes it the best fit for the dynamic nature of analytic workloads

What’s holding a lot of companies back is that they have really big on-premise data warehouses, and it’s difficult and expensive to move them to the cloud. With Direct Data Mapping, you don’t need to move your data warehouse to the cloud. You can keep it where it is and access it directly with Incorta.  In fact, with Incorta, you can access any data source directly, without the need for a data warehouse in the middle. This all translates to greater speed.

These are the top considerations to think about at a high level as you’re reevaluating your approach to reporting and analytics. Speed, access to detailed data, and the efficiency of the cloud. The greatest of these, the one that underpins the others, is speed. When you can get to the detailed data directly at its source with speed, and leverage the power of the cloud without the delays of moving your data warehouse, that’s what will allow you to do better and more accurate analytics and data science, now and in the future.

For more detailed information on how to modernize your data analytics, watch the webinar on demand.