Bulk merge

You can bulk merge existing HCP, HCO, and custom objects in your Network instance using a source subscription by providing the surviving Network custom keys (or the Network Entity ID) in a .csv file. Remember that for HCPs and HCOs, Veeva CRM only permits merging 5,000 records at a time; most other systems have similar restrictions. Merging existing records in bulk is helpful if you have loaded records in your Network instance and they have under matched during the job, leaving you with a number of duplicates. To resolve this, you can update your match rules to prevent future data loads from under matching, and run a bulk merge job to merge the existing duplicate records.

Specify surviving records

In a .csv file, provide the Network custom key of the surviving record in the merge_survivor_key column; for example:

vid__v	corporate_name__v	merge_survivor_vidkey
462313821042639872	University Health Center	ERP:ACCOUNT:00OF0000006w4WY

Custom Key	corporate_name__v	merge_survivor_vidkey
ERP:ACCOUNT:00OF0000009u0D	University Health Center	ERP:ACCOUNT:00OF0000006w4WY

You can also provide the Network Entity ID of the surviving record in the merge_survivor_vid column, for example:

vid__v	corporate_name__v	merge_survivor_vid
462313821042639872	University Health Center	462313820975531008

 

Supported record states

Records can be merged using bulk merge depending on their record state. In general, VALID and INVALID records are the only records that can be merged into other records, depending on the record state of the target record.

Bulk merge jobs did support merging UNDER_REVIEW and DELETED records into other records, but this is no longer allowed. This change (added in version 19R2.0) ensures that bulk merge job rules are consistent with the suspect match rules that are applied in the Network UI.

Enable bulk merge

Create a new source subscription specifically for running a bulk merge job. The source subscription does not need to be configured for data loading, because the bulk merge job will just load the specified .csv file with the Network entity IDs and use the advanced properties to merge the records.

To enable bulk merge for a the source subscription, click the Advanced button in the source subscription and enable the following properties in the Module Properties:

• "job.merge.allowSourceMerge": "true"

• "job.merge.mergeInstruction": "InclusiveMerge"

  When this property is enabled using InclusiveMerge, all items (keys, sub-objects, attributes) are considered for merging. The survivorship rules for the Network instance are applied to determine which values end up in each field on the winning record. Survivorship rules are defined by administrators using source rankings by system (System Interfaces > Source Rankings) or for individual fields (Data Model > Data Domains).

  In comparison, ExclusiveMerge means that the values on the winning record always win. If the winning record does not have a value in a field, the value from the losing record will be kept. When data stewards merge two records manually using the Find Suspect Match feature, the merge_request__v subscription (internal Network subscription) is used. The default merge instruction in the subscription is ExclusiveMerge.

Bulk merge behavior

By default, using bulk merge completes the following actions:

• Updates record states
• Moves custom keys

• Moves locally managed sub-objects and attributes.

  If locally managed records are merged into Veeva managed records, the properties "job.merge.allowCustomerOwnedChildren": "true" and "job.merge.allowCustomerOwnedChildren.CHILD": "true" can ensure that locally managed sub-objects are retained on the Veeva record. For more information, see Merge instructions for sub-objects.

Merge from custom key value

In most cases the source file will only include the custom key value. You can use a Network expressionA user-defined rule applied to Network data loads which can be used to apply transformations to the data during the load. to construct a full custom key from the value. For example if your incoming file is:

loser_id	corporate_name__v	survivor_id
00OF0000004d9XQ	University Health Center	00OF0000006w4WY

Using a transformation Network expression you can define the merge_survivor_vidkey:

[
  "merge_survivor_vidkey = CONCAT('ERP:ACCOUNT:', survivor_id)"
]

Considerations

A single source subscription can process additions, updates, merges, and invalidations.

You should ensure that no circular merges exist. For example, a merge of A > B > C > D will work properly, but a merge of A > B > C > A results in an error. The latter merge operations will not be processed, but all other updates and merges will be unaffected.