Everywhere you look today, enterprises are revamping ERP systems and moving their reporting and analytics to the cloud in pursuit of greater speed and scalability. There’s just one problem: most are still using the same old data transformation processes they have been for years.
In a recent DBTA webinar, Santosh Balasubramanian, Group Product Manager at Microsoft Azure Synapse Analytics, and Incorta SVP of Products, Rich Rodgers, gave a technical overview of Incorta Intelligent Ingest for Synapse.
In a nutshell: Intelligent Ingest provides a new way to get Oracle data into Azure Synapse for reporting and analytics — without the burden of data transformation.
To help you learn more about Intelligent Ingest, we sat down with Rich for a conversation about how it works and what this new capability means for Synapse customers working with Oracle data, as well as customers working with source data from other systems.
Q. Let’s start with the basics. What is Intelligent Ingest for Synapse?
Rich: It’s a modern and more efficient way of getting data from your multiple source systems into Synapse. We created a low code, logical flattening of data that can be loaded it into Synapse for analytics and reporting.
The traditional methods are slow, cumbersome, and iterative. Many customers with analytical data warehouses on the cloud have difficulty getting their data into them using traditional methods. Intelligent Ingest offers a modern, cloud-friendly way to do it.
Q. Can you explain how it works in more detail?
Rich: When we say Intelligent Ingest, we’re talking about everything from accessing the data to making it available to analytical systems.
Incorta gets the data from data sources in native form. It can do this very quickly and at very high volumes. It brings this data into a staging area where our proprietary Direct Data Mapping technology intelligently maps the relationships between every piece of data.
Once the relationships are mapped, we compress the data and store it in a much smaller footprint, as Parquet files.
What’s also very valuable is that Intelligent Ingest preserves the granularity of the raw data sets. By contrast, when you have to extract, transform, and load data from source systems to a data warehouse, you inevitably have to aggregate and reshape the data, and valuable detail is lost in the process. As a result, you can no longer drill down into the transaction-level details of your data, for example. That matters because being able to drill down and ask more questions of data is essential for informed and confident decision-making.
Q. Talk a little bit about Incorta’s partnership with Microsoft.
Rich: We got together with Microsoft to make it very easy for people who invest in Azure as their cloud of choice and Synapse as their analytics warehouse of choice to bring data into their Azure cloud data environment.
We worked closely with Microsoft to make the experience of getting data into Synapse and available to end users for reporting and analysis simple and smooth. The result is Intelligent Ingest for Synapse.
Any Synapse customer today can log in to their Azure Synapse Analytics Workspace portal, and see Incorta as a Synapse Technology Partner. They simply click a button and confirm their email to install Intelligent Ingest. Once they use Incorta to get the data in Synapse, it is also automatically available for analysis and reporting through visualization tools like Power BI.
Q. Talk about how Incorta’s data applications come into play here.
Rich: data applications are essentially analytical applications. They define the entire schema of a source system, such as Oracle EBS or NetSuite or JD Edwards — all the tables, relationships between the tables, the joins. The schemas underlying source systems rarely change, so once we understand the entire structure of a transactional system, it makes it very easy to pull in the data with Incorta data applications.
But, once you’ve pulled in the data, there are still thousands of tables that the customer would have to make sense of and query. Data applications include analytics, insights, and dashboards we’ve built on top of that data, so the customer doesn’t have to do any of that — it’s already done for them which provides a huge head start to any implementation.
Intelligent Ingest is not limited to systems that we have data applications for, but when you combine the speed and granularity of data of Intelligent Ingest with the efficiency of data applications, you can do meaningful analysis on your data much faster than with traditional methods.
Q. A lot of companies are working with hybrid data — data from on-premises systems, warehouses, SaaS systems, and warehouses. How can they use Intelligent Ingest?
Rich: It doesn’t really matter where your data is stored. Most customers have a patchwork quilt of systems and tools. They can still take advantage of Intelligent Ingest. We have the ability to connect to it whether it’s in the cloud or on premises. We have over 250 connectors to various source systems and we understand how to work with them.
Q. Anyone working with Oracle reporting and analytics has massive amounts of data. Can companies with less data take advantage of Intelligent Ingest?
Rich: Yes! Because we have a cloud offering, users only pay for what they use with Incorta. While we are proud that we can handle data for some of the world’s largest and most well-known companies, we also have the ability to deploy very small environments for smaller companies or departmental use cases.
Q. The old way of ingesting data has been with us for a long time. How is this new way now possible?
While at Oracle, the founders of Incorta witnessed their customers going through this ETL process over and over again and they realized there was a better way.
That was the big idea. Making it possible was a confluence of technological advances, especially the rise of open standards for connectivity and data formatting, as well as inexpensive, elastic, and infinite computing power available on demand in the cloud.
Get your data for analysis faster with Incorta Intelligent Ingest. Sign up for a free trial today!