Transformation rules

AD
DM

Administrators can create rules to transform field values in the APIs and in export files for downstream systems. You can also use transformation rules to override OpenData values before records are pushed to Veeva CRM.

Example use cases

  • Remove specific address types from Veeva CRM, for example, mailing addresses.

  • Display only the Ownership Hierarchy in Veeva CRM (remove the noise of relationships for reps in CRM)

  • Set fields (for example, Specialty or Medical Degree) with no value to a custom value.

  • Limit the postal_code__v field to only five digits for US addresses

  • Hide OpenData addresses in Veeva CRM if they have an ordinal greater than 10.

  • Change all corporate names to uppercase characters

  • Override OpenData's retired HCP status based on a target flag.

Note: Transformation rules can be applied to locally managed records and records managed by OpenData and third party data providers.

Network expression (NEX) rules are used to transform the data.

Benefits

  • Retain original data in Network but transform the data output for your downstream system.

  • Each system can have its view of the data. Rules are assigned to specific systems, so you can customize data for different systems.

  • Flexible integrations with Veeva CRM and other downstream systems. Using transformation rules reduces the need to create customizations for other systems.

  • Network API support. Data can also be transformed using transformation queries in target subscriptions, but transformation queries are not supported for API requests.

  • Network widget support. Data that is downloaded from Search widgets or Profile/DCR widgets can be transformed for downstream applications.

    For details, see Apply transformation rules to Network widgets.

Process overview

  1. Define a rule or set of rules for a specific source system. Rules are created using Network Expressions for a specific object and field.

  2. Apply the rules to target subscriptions or the API.

  3. Run the job to transform the data and then export it to the downstream system.

Veeva CRM integration

Use transformation rules to customize the data that is pushed to CRM.

By applying transformation rules on target subscriptions and to the integration user, the rules can then be applied when data is pushed to Veeva CRM.

This includes the following processes in CRM:

  • Network Account Search

  • DCR Inbound Process

  • Network Bridge subscription

Filtering child account and address records pushed to Veeva CRM

Leveraging transformation rules, you can identify the addresses and parent affiliations that should be set to Inactive in Veeva CRM. Then, use CRM functionality to prevent these Inactive records from displaying CRM.

Example

To ensure that Mail Onlytype addresses are not pushed to CRM, create a NEX rule that changes the status of all mailing addresses to Inactive. Then, the CRM settings will delete those inactive addresses so they do not display on accounts in CRM.

To do this, use the following CRM settings and values:

  • FILTER_INACTIVE_NETWORK_RECORDS_VOD - Set to 1

  • NETWORK_ADDRESS_DELETION_PROCESS_VOD - Set to 2

For detailed information about the settings , see Handling Inactive Network Records in the Veeva CRM Online Help.

CRM considerations

  • Picklist values

    If a transformation rule is applied to filter specific values from CRM, for example, remove mailing addresses or display the ownership hierarchy, then it is recommended to remove those values from being selected by users in CRM.

    Examples

    • If you are filtering out mailing addresses, ensure that Mail Only addresses cannot be selected by users in the picklists.

    • If you are filtering to only display the Ownership hierarchy (Ownership,Affiliation), ensure that Claims relationship types cannot be selected.

  • Network Bridge updates

    After transformation rules are applied to a the target subscription, run a Network Bridge job to ensure CRM is in sync.

    Options

    • Update all records - Run a full Bridge job. In the Network Bridge configuration, set the Revision Data Value setting to 0 to update all records.

    • Update affected records - If you know the records that need to be updated in CRM, add the Veeva IDs (HCP/HCO) to the Export by VID option in the target subscription. When the next Bridge job runs, it updates those records.

      For example, if you know which HCPs records contain Mail Only addresses, add those VIDs to the Export by VID job so your next Bridge job removes those addresses.

Create a rule

To add a transformation rule:

  1. In the Admin console, click System Interfaces > Transformation Rules. Click New Rule.

  2. Type a Name and Description.

    Names cannot contain spaces.

  3. Select a System. All rules are associated to a specific source system.

    The system cannot be changed after the rule is saved.

    Tip: If you make a mistake, you can clone the rule and change the system.

    Veeva CRM or Vault CRM considerations

    If this rule will be used to transform data before it is exported to Veeva or Vault CRM, ensure that the following settings are applied:

    • System - Define the system for CRM

    • Target subscription - Choose the subscription used for CRM.

    • Search and Retrieve API - Choose your CRM integration user.

    Widget considerations

    If this rule will be used to transform data that is downloaded from Network widgets, ensure that the rules are applied to the following:

    • System - Define the system for the widget.

    • Search and Retrieve API - Choose the system.

  4. In the NEX Rules section, add the rules.

    For each NEX rule, define the following:

    • Object - All objects (except custom keys) are supported.

    • Field - The field. The list is filtered for the selected object.

    • Code Description - Define a meaningful description. This description will also display in associated target subscriptions.

    • NEX Rule - Type the Network Expression for transforming the data. The NEX function must return a value for the transformed data.

      Expand the field if you need additional space. The expression validates as you write the rule.

      Click Test to test the outcome of the rule using a valid Veeva ID. For details about testing NEX rules, see NEX tester.

      See a list of examples in the Example rules section below. For help with creating network expressions, see NEX functions and NEX operators.

    To create another rule for this system, click Add Rule.

    Tip: Use the Handle icon to reorder the rules if the sequence of the rules matters. For example, you can chain together several rules to transform data for a specific field.

    After a NEX rule is created, it can be enabled, disabled, or deleted. If a rule is not enabled, it will not be applied to the target subscription/API.

  5. Apply to Target Subscriptions - Associate the rules to target subscriptions. Choose one of the following options:

    • All Target Subscriptions that match the system - Apply these rules to any target subscription that uses the source system.

    • Specific Target Subscriptions - Choose the specific target subscriptions that these rules apply to. All of the enabled target subscriptions that use this source system display in the list.

  6. Apply to Search and Retrieve API - Apply the transformation rule on the API for a specific user or a system.

    • Apply Rules to Search and Retrieve API calls that use the System - Choose to apply the rules to the system defined for this rule.

      Use this option if you are applying the rules to the Search widget or Profile/DCR widget. This transform the data for any user that downloads records from the widget.

    • Apply to a Specific User - Use to apply the rule to an integration user.

      Active users that have API access display in the list.

  7. Save the rule.

    The NEX rule functions are validated. If there are issues, an error displays.

All rules in your Network instance display on the Transformation Rules page (System Interfaces).

Target subscriptions

On the target subscription configuration, the Transformation Rules section displays any rules that have been applied to the subscription.

Note: When the job runs, transformation rules are applied before transformation queries.

The following details display:

  • Object, Field, Description

  • Transformation Rule - Click the link to open the rule configuration.

  • View Rule - Click to see the NEX rule. This is the current rule (from the Transformation Rules configuration page).

Dataflow View

After you save a subscription configuration, the transformation rules display in the Data Flow View.

Target subscription job details

After the job runs, you can view the rules on the Job Details page.

The following details display in the Transformation Rules section:

  • Object, Field, Description

  • NEX Rule - The code as it existed when the job ran so you can understand the customizations that were made to the exported data if the NEX rule has since changed.

  • Transformation Rule - The current name of the rule on the Transformation Rule configuration page. Click the name to open the configuration page. If the rule has since been deleted, the original rule name displays with a (Deleted) comment beside the name.

Job error log

If there is an issue with the NEX rule, errors will be logged. The job will fail in some cases; for example, if the NEX rule references a lookup table that does not exist in your Network instance.

Clone rules

After a transformation rule is saved, it can be cloned. This is helpful if you want to apply the same or similar rules to another source system.

When a rule is cloned, only the description and the NEX rules are copied to the new rule. You can create a name, select the source system and apply the rules to target subscriptions or the API.

Network API

Transformation rules can be applied to Retrieve and Search API requests for an integration user or a system.

Search API - Supported requests

  • Search API

  • Search API + supplemental

Retrieve API - Supported requests

  • Retrieve Entity

  • Retrieve Child Entity

  • Batch Retrieve Entity

  • Batch Retrieve Child Entity

  • Retrieve HCO

  • Retrieve HCP

  • Retrieve Change Request (IncludeEntity = True)

  • Batch Retrieve Change Request (IncludeEntity = True)

Example request

During this Retrieve HCP request, the transformation rule sets the Mail Only address record status to Inactive.

Support for fields with null values

In the Network API, if a field is empty, the record is not returned in the JSON. For transformation rules, the field is returned in the response entity JSON because the transformation rule adds a value.

Example

The specialty_1__v field value is NULL, so it is not returned in the Retrieve HCP response. A transformation rule runs that changes the NULL value to "UNSPECIFIED__c". Now, the Specialty field will be returned in the JSON: "specialty_1__v":"UNSPECIFIED__c".

Hashtag considerations

Hashtags reflect the untransformed values. They do not change for transformed values.

If you search using hashtags, the results returned are based on the original value of the data.

Example transformation rules

Object Description Field NEX Rule
Address Deactivate mailing addresses address_status__v if(addres_type__v =='M', 'I',address_status__v)
Limit US zip codes to 5 digits postal_code__v if(country__v == 'US', left(postal_code__v,5),postal_code__v)
ParentHCO Deactivate relationship types that are not Ownership or Affiliation parenthco_status__v if(not(relationship_type__v in ['2','7356','12']), 'I', parent_hco_status__v)
HCP Override retired HCP status hcp_status__v if(hcp_status__v == 'R' && status_override__c == 'Y', 'A', hcp_status__v)
When Specialty is null, then set it to a custom reference type (UNSPECIFIED__c)

 specialty_1__v if(ISNULL(specialty_1__v), 'UNSPECIFIED__c', specialty_1__v)
When Medical Degree is null, then set it to a custom reference type (UNSPECIFIED__c) medical_degree_1__v if(ISNULL(medical_degree_1__v), 'UNSPECIFIED__c', medical_degree_1__v)
Set record type

Note that this uses a lookup table.		 record_type__v LOOKUP('type_mapping__t', 'record_type', hcp_type: hcp_type__v)

Note: For rules that change an object status to Inactive (for example, addresses or relationships), if it is pushed to Veeva CRM, some CRM settings will then remove those records. For more information, see the Veeva CRM integrations above.

Deactivating fields

If a rule uses a field that has been deactivated, the rule is automatically disabled and cannot be enabled.

Before a field is deactivated, a dialog notifies you that transformation rules that use the field will be deactivated.