One of the most common questions I get from clients is “why is getting data out of SAP so hard? Isn’t it just another source system?”. After a while pondering over this question I thought I would list out the reasons based on our numerous projects getting data from SAP into cloud systems such as Snowflake. Once I’d started I couldn’t stop. Here is my top 5 outlined below.
1. The data is complex
SAP systems are the nerve centres of global enterprises. Many business critical processes are managed and controlled with SAP systems. Consequently, SAP systems contain the most treasured information large organisations have – namely their financial and operational data. As a result, these systems are both complex in terms of the number of processes that they have but also in the volume of the data. A typical SAP system contains 100,000s tables and there are many complex relationships between the tables.
2. SAP systems and SAP data are heavily governed
As a system with the most crucial and sensitive data it’s only right that there is a lot of governance and processes in place to protect the data and the system itself. That means added time and complexity when trying to get the data out of SAP. This will often involve various SAP teams and stakeholders will need to be involved to ensure the correct access is given to enable you to get your data out of SAP, and operational processes are not jeapordized by the process of ‘getting the data out’.
3. SAP at its core is old technology
SAP is 50 years old this year. When SAP started, memory was precious, to the extent that table names and field names were abbreviated to four and six characters respectively. SAP was originally developed in Germany, so the abbreviations are German abbreviations. Over time, SAP have created several metadata layers which can sometimes help to get more descriptive names in a data model but when you look at a system today, at its core you still find the incomprehensible abbreviations. This is why you need to be a SAP functional expert to understand how to both get to and make sense of the data that you need.
4. Lots of different options and frameworks for ‘getting the data’
SAP systems have a wide variety of SAP specific object types and is not immediately obvious to those not from an SAP background – extractors, ABAP reports, BADIs, iDocs, ADSOs, CompositeProviders etc.
Lastly, to be able to use things such as the ODP framework to be able to use SAP extractors then you will need to set up various things on the SAP system itself. This will often require the help of SAP basis teams to ensure that you are able to use delta extraction enabling incremental loading which is a must given the data volumes of some key sources of data such as GL line items. For more information on the SAP extraction options then please read this excellent blog from the SAP guru Jan van Ansem here.
5. Complex licensing model, which means options which are technically available may not be used within the license agreement.
Often customers think that the easiest way to extract the data will just be by connecting to the HANA database and replicating the data that they need. Whilst this is a relatively simple process there are licensing constraints that constrain most organisations from doing so. Those that have a HANA Runtime license (which is the majority of clients) are not able to extract from the database layer and can only extract from the application layer. SAP has been known to sue for some extremely large sums of money when their licensing constraints are broken by their clients, as Anheuser Busch found out to their peril.
Hopefully the above gives you a bit of an idea of why loading from SAP to cloud data platforms is not just the same as other source systems and why it’s imperative to have people that are both experts in SAP systems and cloud data platforms and architectures. Luckily at Snap we are a team of SAP data experts with a focus on modern data cloud technologies such as Snowflake and Matillion and have a range of accelerators to simplify the process of extracting your data from SAP systems. Please do reach out to us if you’re interested in maximising the value of your SAP data in the cloud.
Photo credit: Mitchell Luo on Unsplash
