SAP ERP on HANA

SAP Enterprise Resource Planning (ERP) applications like SAP ECC and SAP S/4HANA manage customer finance, operations, supply chain, and HR processes. With the SAP ERP on HANA connector, Fivetran offers the possibility to replicate data from SAP ECC and S/4HANA systems running on a HANA platform.

SAP HANA is an in-memory, column-oriented, relational database management system. Fivetran replicates data from your SAP HANA source database and loads it into your destination.

Features

Known limitations

• We support column-store and row-store transparent tables, as well as views from the SAP data dictionaries. We recommend avoiding replication of base tables that point to proxy objects (such as compatibility views), as they may return incomplete or no data. Instead, we recommend syncing the compatibility view directly. Parameterized and projection views are not supported. For more information, see Excluded objects.
• For row-store transparent tables and views, we do not support change data capture (CDC). For more information, see Replication of views and row-store tables.
• No support for SAP BW Advanced DataStore Object (ADSO) tables.
• There are two limitations when selecting tables to sync:
  • Each search term must include at least three alphanumeric characters. The following special characters are allowed: *, ?, _, -, ., and /.
  • Each search query returns a maximum of 1,000 tables, sorted alphabetically.
• No support for the following DDL (Data Definition Language) statements: CREATE TABLE and most ALTER TABLE operations. If such operations occur, the respective table must be manually re-synced. However, ADD COLUMN and DROP COLUMN operations are supported.
• No support for updating a PrimaryKey value. If a primary key is changed in the source, we treat it as a new INSERT in the destination.
• We require an SAP ABAP server version 7.5 or higher with Unicode support. For more information, see Prerequisites in SAP ERP on HANA Setup Guide.
• Authorization via API is only supported when using SSH tunneling.
• Limited support for special* or lowercase characters in column and table names.
• No support for special* or lowercase characters in schema names. For example, lowercase_schema , lowercase_SCHEMA, or lOwErCaSeScHeMa are all unsupported names.
• No support for adding new tables through schema updates due to SAP API limitations.

* Special characters include (, &, @, *, $, |, %, ~, -, and ). Underscores are not considered special characters.

Setup guide

Follow our step-by-step SAP ERP on HANA Setup Guide for specific instructions on how to set up your SAP ERP on HANA using the Fivetran connector.

Sync overview

The SAP ERP on HANA connector interfaces with the application layer of an SAP system. Within this application layer, you need to install the Fivetran Netweaver API, which then pulls a full dump of all selected data from your SAP system. We then use our proprietary tracking mechanism to retrieve modified data at regular intervals.

Selecting tables to sync

The SAP ERP on HANA connector does not sync all tables by default. To include specific tables or views in your sync, you must manually search for and select them using the Schema tab of your connection's dashboard. This behavior is intentional - SAP environments often include hundreds of thousands of tables, so only selected tables are included in the sync.

Searching for tables

Use the search field to find specific tables or views by entering search terms in one of the following ways:

• Enter exact table or view names, for example: T000, T001, T002 (spaces are optional and ignored internally).
• Use wildcards, for example: T00*, *BAK.
• Combine exact names and wildcards, for example: VBAK, T00*, DD0*.

The following are the search requirements:

• Each search term must contain at least three alphanumeric characters.
• You can use the following special characters in your search terms: *, ?, _, -, ., or /.

Only the following SAP object types are shown in the search results:

• Column-store transparent tables
• Row-store transparent tables
• Views

For more information about how views and row-store transparent tables are handled during replication, see Replication of views and row-store tables.

Limiting table availability

You can control which tables are available for selection in the Schema tab by defining the Source tables list in the Fivetran's SAP configuration screen accessible through the SAP transaction /N/FIVETRAN/CONFIG. For more information, see Configure Fivetran in SAP.

• If the list is empty (default), all supported tables and views are available for search and selection.
• If the list includes specific tables (for example, T000 and BSEG), only those tables will be visible in the Schema tab.

To stop syncing a previously added table that may already contain data in your destination, remove it from the Source tables list. We will disable the table in the Schema tab during the next sync and prevent further replication.

Example scenarios:

Here are some examples to illustrate how table filtering works:

• Scenario 1 - No Filtering: If no tables are added to the Source tables list in the Fivetran's SAP configuration screen, all supported tables and views will be available for replication.

• Scenario 2 - Adding new tables: The Schema tab currently contains tables T000 and T001. If a user adds tables T000, T001, T002, and BSEG to the Source tables list, the existing tables (T000 and T001) can still be replicated. However, only the newly added tables (T002 and BSEG) will be available for searching and adding for sync in the Schema tab.

• Scenario 3 - Replacing existing tables: The Schema tab currently contains tables T000 and T001. If a user adds only table BSEG to the Source tables list, the existing tables (T000 and T001) will be automatically removed from the connection's Schema tab. Only table BSEG will be available for searching and adding for sync in the Schema tab.

• Scenario 4 - Removing and adding tables: If the Schema tab currently contains tables T000 and T001, and these tables are no longer permitted because they are not specified in the Source tables list, Fivetran will gray them out in the Schema tab and will not sync them. If you re-add tables T000 and T001, Fivetran will sync them and gray them out during the next sync cycle.

Filtering rows based on column values

You can apply row-level filters to replicate only the data you need from a specific table. For more information on creating, editing, or deleting filters, including supported operators and data types, see our Row Filtering documentation.

Syncing empty tables and columns

Fivetran can sync empty tables for your SAP ERP on HANA connection.

We can also sync empty columns. However, we require at least one row of data that includes this new column to be added. This is because we detect changes to a table's column cardinality when we see a row with a new or removed column during an update.

For more information, see our Features documentation.

Schema information

Fivetran replicates the exact schema and tables from your HANA source database to your destination according to our standard database update strategies. This ensures that the data in your destination is in a familiar format to work with.

When you connect to Fivetran and specify a source database, you also select a schema prefix. We map the SAP schema in your source database to your destination and prepend the destination schema name with the prefix you selected.

Fivetran-generated columns

Fivetran adds the following columns to every table in your destination, depending on which sync mode you use:

• Either sync mode: _fivetran_synced (UTC TIMESTAMP) which indicates the time when Fivetran last successfully synced the row.
• Soft delete mode: _fivetran_deleted (BOOLEAN) marks rows that were deleted in the source database.
• History mode: _fivetran_active (BOOLEAN), _fivetran_start (UTC TIMESTAMP), and _fivetran_end (UTC TIMESTAMP) . These are used to mark which rows are currently active. They keep a record of the previous record and their lifetime.

We add these columns to give you insight into the state of your data and the progress of your data syncs. For more information about these columns, see our System Columns and Tables documentation.

The _fivetran_deleted (soft delete mode) and _fivetran_active (history mode) columns are not available for views and row-store transparent tables. For more information, see Replication of views and row-store tables.

Additionally, we may generate columns during the unpacking of certain packed tables, such as STXL and MDTC. See Unpack data section.

Type transformation and mapping

As we extract your data, we match SAP data types to the types that Fivetran supports. Our system attempts to infer the types of any columns with data types that we do not recognize.

The following table illustrates how we transform your SAP data types (based on the SAP Data Dictionaries) into Fivetran supported types:

*NOTE: The String or LocalDate type conversion depends on the Enable SAP date to LocalDate conversion toggle’s state in the connection setup form.

If we are missing an important data type that you need, reach out to support.

In some cases, when loading data into your destination, we may need to convert Fivetran data types into data types that are supported by the destination. For more information, see the individual data destination pages.

Data mapping workflows

Depending on the values, the data mapping for the connection can be in one of the following situations:

• The value can be parsed to a Fivetran data type - Fivetran takes the value and sends it to the destination in the appropriate data type.
• The value cannot be parsed to a Fivetran data type, and the column is not a primary key - Fivetran coerces the value a null and displays a warning. The sync continues.
• The value cannot be parsed to a Fivetran data type, and the column is a primary key - The sync for the respective table ends with an error.

Schema changes

New tables are not automatically added. Use the Schema tab to add tables as needed.

Initial sync

The initial sync can be performed once the connection to the SAP ERP on HANA database is established. The NetWeaver Fivetran API manages all data transport during this process.

Updating data

Fivetran performs incremental updates of any new or modified data from your source database. New changes are fetched and then synchronized into the destination. The update process varies depending on your sync mode:

Soft delete mode

• An INSERT in the source table generates a new row in the destination with _fivetran_deleted = FALSE.
• A DELETE in the source table updates the corresponding row in the destination with _fivetran_deleted = TRUE.
• An UPDATE in the source table updates the corresponding row in the destination with _fivetran_deleted = FALSE.
• An UPDATE of the primary key in the source table generates is treated like a new row being added. We therefore add a new row in the destination with _fivetran_deleted = FALSE. The old row remains in the destination with _fivetran_deleted = FALSE until the corresponding table is re-synced.

History mode

• An INSERT in the source table generates a new row in the destination with _fivetran_active = TRUE, _fivetran_start = <CURRENT_TIMESTAMP>, and _fivetran_end = 9999-12-31 23:59:59.999+00.
• A DELETE in the source table marks the corresponding row in the destination inactive with _fivetran_ACTIVE = FALSE and _fivetran_end = <CURRENT_TIMESTAMP>.
• An UPDATE in the source table first marks the corresponding row in the destination inactive with _fivetran_ACTIVE = FALSE and _fivetran_end = <CURRENT_TIMESTAMP>. We then add a new row in the destination with the same data and _fivetran_active = TRUE, _fivetran_start = <CURRENT_TIMESTAMP>, and _fivetran_end = 9999-12-31 23:59:59.999+00.
• An UPDATE of the primary key in the source table is treated like a new row being added. We therefore add a new row in the destination with _fivetran_active = TRUE, _fivetran_start = <CURRENT_TIMESTAMP>, and _fivetran_end = 9999-12-31 23:59:59.999+00. The old active row remains in the destination with _fivetran_active = TRUE, _fivetran_start = <CURRENT_TIMESTAMP>, and _fivetran_end = 9999-12-31 23:59:59.999+00 until the corresponding table is re-synced.

Deleted columns

We do not delete columns from your destination.

Capturing deletes

Capturing deletes requires careful consideration as they can go unnoticed if they occur in portions of data already imported during previous cycles.

To detect deletes, we use triggers. The Fivetran NetWeaver API creates two objects for each imported table:

• A shadow (DELETES) table that follows the naming convention /FIVETRAN/DELETES_ plus the table name (for example, /FIVETRAN/DELETES_T000 for the T000 table).
• A trigger on the table that detects deletes and populates the associated DELETES table.

During each import cycle, we check the DELETES table and update the destination accordingly.

Re-sync partition and re-sync table scenarios

The DROP and TRUNCATE operations have certain side effects that can impact the reliability of your connection. Fivetran can detect and automate specific processes without disrupting the data replication:

• In the case of a TRUNCATE (partition or table) operation: Fivetran automatically detects truncations for each partition and initiates a re-sync during the next sync cycle.
• In the case of a DROP partition operation (typically done via ALTER TABLE): Automatic detection from Fivetran side is currently not supported. This capability is still under development. Therefore, you must manually re-sync your table in case one partition is dropped.
• In the case of a DROP TABLE operation: No further action is taken for that table, and it is treated as if there are no updates. The data remains intact on the destination.

Create partitions

Creating a new partitions for any table is automatically detected by Fivetran. This will be synced normally during the next sync cycle.

Fivetran also detects modified/restructured partitions, triggering a re-sync for those partitions.

Unpack data

We support unpacking for two tables, STXL and MDTC.

STXL unpacking

We unpack long text STXL table into a readable and transparent format. We have modified the table structure to account for this. The following is the replicated STXL version:

We replaced the SRTF2, CLUSTR and CLUSTD columns by the new /FIVETRAN/STXL_TEXT column. The aggregation of these three columns contain the necessary information to decode the human-readable text for the new column.

MDTC unpacking

We unpack table MDTC table into a readable and transparent format. We have modified the table structure to account for this. The following is the replicated MDTC version:

We have replaced the CLUSTR and CLUSTD columns by adding 59 new columns:

• We expanded the primary key with two columns, DTNUM (PK) and DTPOS (PK).
• We added 57 non-primary key columns containing unpacked data.

Unpacking MDTC is a many-to-many type of unpacking - one or more records from the source could be unpacked into one or more records in your destination, in human-readable format.

Handling deleted data during SAP archiving process

You can mark records archived by a SAP archiving process as non-deleted in your destination by checking the Start tracking archive deletes box under SAP Archiving job users in the Fivetran's SAP configuration screen to enable this feature.

When the box is switched on, an input list labeled SAP Archiving job users appears under SAP Archiving job users. Here, you specify users related to the SAP archiving process users (or any user for whom you would like to enable this feature).

When you have enabled marking archived records as non-deleted for the SAP source, Fivetran does the following:

• A new column is added to all tables synced from the relevant source in your destination, _fivetran_sap_archived (BOOLEAN). The column value for the previously synced data becomes NULL.
• The _fivetran_sap_archived column value is set to FALSE for the records synced after the feature was enabled.
• The _fivetran_sap_archived column value is set to TRUE for the records archived on the SAP side if the user committing the delete is included in the input list of users. The specified username and the username of the SAP user related to the SAP archiving process must be an exact match. In all other cases, (for example, if the box was checked, but the input user list remains empty), the _fivetran_sap_archived column value remains FALSE.
• If this column is TRUE, then the corresponding record will not be marked as deleted. For example, depending on the sync mode, such record continues to have _fivetran_deleted=FALSE in case of soft deletes or _fivetran_active=TRUE in case of history mode.

See our Configure Fivetran in SAP documentation for more information about configuring SAP archiving deletes.

Monthly Active Rows calculation

The records archived by SAP archiving processes are seen in Fivetran as either updates (if this feature is switched on) or deletes (if the feature is switched off) and count towards paid MAR. The standard rules for Monthly Active Rows (MAR) still apply for these operations.

Replication of views and row-store transparent tables

Fivetran allows you to replicate data from:

• Row-store transparent tables
• Column-store transparent tables
• Views

Due to structural limitations in views and row-store transparent tables, we cannot incrementally sync them. Instead, we fully re-sync these objects at each sync cycle. Here’s how we handle records during the re-sync of views or row-store transparent tables:

• If a record doesn't exist in the destination, we insert it into the destination.
• If a record exists in the destination, we update the existing record.
• If a record exists in the destination but is not present in the latest sync cycle, we permanently delete it from the destination.

Handling deletes

Since views and row-store transparent tables are fully re-synced at each cycle, Fivetran does not use the delete-capturing mechanism. As a result:

• We do not create triggers or shadow DELETES tables
• We do not add the Fivetran-generated columns for tracking deletes:
  • _fivetran_deleted (soft delete mode)
  • _fivetran_active, _fivetran_start, and _fivetran_end (history mode)

Primary keys and unique identifiers

Views don't have primary keys. To ensure uniqueness, we introduce the _fivetran_id (LONG) column, which auto-increments for each record.

Monthly Active Rows calculation

The standard rules for Monthly Active Rows (MAR) still apply to views and row-store transparent tables.

MANDT filtering

When logging into an SAP system, the login is client-specific, and data is automatically filtered by client (MANDT). Our connector has a broader data access and can replicate data for all clients, regardless of the logon client (client-independent data selection). You can enable MANDT filtering in the Fivetran's SAP configuration screen to limit the replicated data to only data relevant to the connected client.

For more information about enabling the MANDT filtering, see our Configure Fivetran in SAP documentation.

Excluded objects

Fivetran does not sync the following objects:

• Database-level system tables
• Tables and views not present in the SAP data dictionaries
• Temporary tables
• SAP functions
• Projection views
• Parameterized views
• Tables with underlying proxy objects (for example, compatibility views). However, it is possible to replicate compatibility views themselves, as they are treated as views. For more information, see Replication of views and row-store tables. Example:
  • In some SAP systems, table MBEW is based on compatibility view MBVMBEW. Do not replicate table MBEW, as it may be empty, contain partial information, or, in rare cases, include unexpected data. Instead, replicate the view MBVMBEW.