Source subscriptions

AD
DM

Use source subscriptions to load and merge external data into your Network instance. Source subscriptions refer to an external source of data or system you subscribe to. You can schedule subscription activity, determine what data is imported, specify whether records from a source can be unmerged, and specify whether certain data sources should be considered proprietary.

When you create a source subscription, you define properties in the subscription that manage the incoming data to create new records or to merge incoming data with existing records.

Veeva standard objects and custom objects are supported.

Source subscription process

During a source subscription job, Network picks up the raw source data, transforms it into a format consistent with the Network data model, and then runs match/merge evaluations. Preparation for data loadingClosed A record that has not yet been validated by a data steward. consists of a number of steps to ensure that data loading occurs cleanly and as expected.

The process of preparing for and configuring the data load should consist of the following activities:

  • Define a system - Add and configure a system in Network that corresponds to a data source for data loads and change requests, and which includes its own set of reference aliases. Configure parameters for proprietary or restricted data (if applicable) and define survivorshipClosed The process by which the final record is determined as duplicate records are merged, resulting in winning and losing values. trust values.
  • Define reference aliases - Create mappings between external reference codes and Network reference codes. Update mappings through a .csv file and link them to a system.

  • Create a source subscription - Define the data load schedule, incoming FTP folder, external keys, model map, data loading rules, and blockers and classifiers. Enable match testing and match log export for test purposes.

    There are two options for creating a source subscription:

    • Wizard Mode - Simplify the subscription configuration process by using a sample file to define the advanced properties, field normalization, and model map. When these settings have been configured, the classic subscription opens so you can continue defining match rules, Network expression rules, and so on.
    • Classic Mode - Create the entire subscription using the traditional configuration. Manually define all of the settings, field normalization and model map.
  • Test model map - The model map includes target entity specification, data inclusion statements, and entity cross references. Using match testing and match log export, upload a subset of data to the FTP folder, start a job from the subscription, and review the match log for errors (such as missing aliases or records).
  • Review match rules - Matching is based on features, blockers, and confidence, which use defined subsets of fields and blocks to determine confidence levels and match data. Optimize match rules by running jobs on small subsets of data and reviewing output for correctness.
  • Advanced settings - Using the Advanced button, you can set additional properties or change the default values. For example, you can set a property to determine the records that are updated according to record state (any, valid, or non-valid).
  • Load data - Disable match testing and export and run the subscription job. Addresses, licenses, and affiliations are de-duplicated and changes are committed.

Considerations

The following apply for data loaded into a Network instance:

  • Records loaded through a data load do not require mandatory fields.
  • Data added by API add requests or through the Network UI must include values for mandatory fields.
  • Mandatory fields on sub-objects are only required if the sub-object is included for a record. For example, if HCP data is added without address data, values are not required for mandatory fields under the address object for the record.
  • When you unmerge records, if a record was added by data load without an address, the resulting unmerged record will not include an address.

  • Parent HCOs that are included in subscription feeds will be dropped or rejected if the related entity is not Valid or Under_Review. This applies to candidate records and opted-out records also. If any Parent HCO records were not created, a job warning displays.

  • Locally managed sub-objects are added to Veeva OpenData records during merges if the job.merge.allowCustomerOwnedChildren property value is true. For more information about this property, see Advanced subscription settings.

Note: Mandatory fields vary by country.

Address ordinals

During a data load, address ordinals are set according to the following criteria, in order:

  1. Veeva-owned addresses before non-Veeva owned addresses
  2. Active addresses before inactive addresses
  3. According to type: "Professional" before "Professional and Preferred Mail" before "Mail only" before "Address"
  4. According to the system's survivorship rankClosed A ranked list of systems in order of decreasing "trust" level. (set in Source Rankings)
  5. According to preferences defined by the incoming system
  6. Incoming addresses with identical criteria (that is, the same type, ordinal, rankings) are given ordinals in the order in which they are loaded.

If a new address is submitted without an ordinal value, a default value of 99 is assigned. Once the change request has been processed, all ordinal values for that record, including the new one, are re-evaluated and updated if required. For example, if there are three addresses with 99 as an ordinal ranking, Network re-ranks the addresses so that they have unique values. If addresses are sorted outside of Network, the source rankings are not changed.

Ordinals for customer-owned addresses on Veeva OpenData records are numbered immediately after the highest ordinal value for Veeva OpenData addresses.

Veeva OpenData addresses are sorted within each OpenData instance using the same rules that are applied to customer-owned addresses. Consequently, Veeva OpenData customers benefit from unique and sequential address ordinals for Veeva OpenData addresses. This applies to all Veeva OpenData addresses except for US Veeva OpenData addresses, which are sorted and ranked outside of Network.

Note: Criteria that do not apply in a given situation are skipped.