SAP PI is commonly used as the central middleware tool in large SAP landscapes to exchange (SAP IDOC) messages between the various SAP backend systems.
PI will be responsible for transforming (mapping) these messages from format A to B but it has no default extensive functionality to change the actual content of these messages depending on their source and target system. This kind of functionality is typically expected to be handled by the backend system itself (via master data or SAP IDOC configuration) or could be handled by a separate 'master data management' (MDM) system. The first option can quickly become a maintenance nightmare and the second solution requires a complete project implementation affecting all SAP backend systems.
The ideal solution would be that SAP PI would be capable of performing these data conversions itself while transforming the message from format A into B ... with a central, highly customizable tool.
Standard there is some limited functionality during mappings to handle value translations and one can perform real-time value mapping lookups on remote systems ... that's all fine for some exceptional translation but that functionality is not sufficient if you need to map a large number of values for a large number of messages coming and going towards a large number of systems ...
Data Mapping Engine
A custom data mapping engine was built from the ground up on the ABAP stack to be able to handle a large number of value mappings within a message using configurable behavior.
New interfaces can easily be added by defining new mapping rules or benefit from existing mapping entities or rules.
The standard mapping types will provide already some serious mapping power with support for attributes and conditions checks.
If the standard behavior is insufficient then one can add 'custom mapping programs' via own ABAP OO classes which need to implement a specific API and are then 'plugged' into the mapping system via configuration.
All executed value mappings are logged into the system as well for potential review.
Error behavior can be configured to allow the whole PI message to fail in the PI integration engine or to continue processing.
Alerting can be defined to inform users about missing translation data values.
A web application is available to display the configuration and the usage of the engine.
'Source' and 'Target' systems are defined based upon existing SAP PI logical destinations and can be configured for data replication behaviour.
A 'mapping entity' represents an interface independent functional data entity which needs to be translated.
A 'mapping interface' is defined based upon existing PI interfaces and linked with mapping entities via configurable XPATH expressions.
A 'mapping rule' defines for a 'mapping entity', 'source system', 'target system' and 'mapping interface' how the engine will transform the data value for that mapping entity.
A mapping rule offers various configurable options:
- data can be mapped, not mapped at all or replaced by a fixed value
- various default mapping types can be used - 'easy mapping', 'attribute mapping, 'custom mapping', etc ...
- the behavior when no mapping data is found
- extensive configurable conditions definition to select one of many potential target values based upon 1 or more conditions (using 'AND' / 'OR' relationships)
- mapping rule priority
The mapping rules can be configured for 'all interfaces', 'interface groups' (eg all SAP IDOC messages) or for specific interfaces. The data engine will determine all the rules for a given interface during the PI mapping step and execute those rules based upon their priority.
The actual data is stored in database tables on the SAP PI system.
This solution was chosen as most of the master data does not change often and can thus be replicated on an hourly or daily basis or is maintained on the central sharepoint system where real-time triggers call web services on SAP PI to update changed data.
Due to the high amount of value translations, real-time data extraction in the back ends would mean an intensive high load on the network and involved backend systems and was therefore never considered (but could theoretically be possible as well).
The data mapping engine has been used now for several years and is responsible for mapping +10000 idocs a day between a dozen backend systems with an average translation of 50 values per message.
The SAP PI system performs well and no special performance impact was found by an SAP technical audit.