Viewing data lineage

AD
DM
DS
ST

Data lineage contains source values for a record across contributing systems (or change requests). Field values that were loaded in a source file, or added through a change request, display in the data lineage. The values may or may not be applied to the record, depending on your source rankings.

To see changes that occur on a record, review the revision history.

View data lineage details

On record profiles, the Data Sources preview box lists the source systems that contributed to the record. The box displays up to ten sources and provides a count for the remaining sources.

Click More Details to navigate to the Data Lineage page to review the sources in detail.

The data lineage view contains the following columns:

  • Field - A list of the fields for the object.

  • Current Source - Displays the name of the source that provided the winning value for each field.

  • Network Record - The field values currently on the record in your Network instance.

  • Sources - A column for each contributing system or change request. Green check marks indicate the source values that are in sync with the Network record.

About sources

Only sources that add a custom key to the record display in the Sources section.

A source can be one of the following:

  • System name - This is the system associated with the job that loaded the data.

  • change_request- The record was added or updated through a data change request in the Network UI.

    This is a default system that is automatically applied when a record is added or changed through the Network UI.

  • Veeva OpenData - Updates were made through an OpenData subscription.

Considerations for system name

The name that is added to the Sources column is the system that first adds the contributing custom key to the record. As records are updated, system names are added to Sources column only if they add a custom key to the record. For example, if a record is key matched during a job and a field is updated (new source wins survivorship), the Current Source column is updated with the new source name, but the new source will not be added to Sources column because the custom key exists on the record.

Note: Custom keys that are added to a record by a Network widget or the associate key API call are not contributing keys. These sources will display in the Current Source column beside the custom key, but they are not added to the Sources section.

For more details about sources, review the following example.

A source subscription is configured to use the system called candidate.

In the model map, the source for custom keys is defined as ERP.

When the job runs, a record is added and an ERP custom key is added to the record.

Result: On the Data Lineage page, a Sources column is created for the candidate system. The Current Source column lists candidate as the contributing source for the fields.

Another job runs using a system called ERP. It key matches to this record using the existing ERP custom key. The job updates the Corporate Name field.

Result: In this case, the Current Source column is updated to indicate ERPas the contributing source for the Corporate Name field value. A Sources column is not added for ERP because the ERP custom key exists on the record.

Another job runs using a system called AACT. It adds a custom key to the record and tries to update the Corporate Name field.

The AACT system is ranked lower than ERP but is ranked higher than candidate.

Result: The Corporate Name field is not updated because the ERP system wins survivorship. A Sources column is created for AACT because it added a custom key to the record.

Current Source column

This column displays the name of the source that provided the winning value for each field.

Other source values

The Current Source column can display values other than a source name if the field value was updated by Network.

  • Updated by System - System fields that are updated by Network. This value also displays for data that was calculated by Network using rules before version 22R1.1; that data is not backfilled for older values. Examples of system fields: Created Date, Record State, and Veeva ID.

  • Calculated Value - Displays on fields where the value is calculated by Network rules (for example, fields updated by NEX rules, default values, or primary values when a source file does not contain a value).

    Hover to view a tooltip that identifies the rule and job that calculated the value. Administrators and data managers can click the job ID to navigate to it for more details.

    Note: If the field value is calculated because of an update from a source, the source name displays in the Current Source field.

     

    Network rules that calculate values

    Any of the following rules can calculate a value for a record.

    • Address Cluster Management

    • Address Rank

    • Alternate ID

    • Customer Data Opt Out

    • Convert OpenData Opt Out into Customer Owned

    • Default Value

    • Network Address Inheritance

    • Formatted Address

    • Formatted Name

    • HCO Record Status

    • Entity Invalidation

    • Invalidate OpenData Mail Only Address

    • License Address Link

    • Master Data Opt Out

    • Network Calculated Primary Address

    • Network Calculated Primary Affiliation

    • Unique Checkbox Primary Sub-Object

    • Cluster 1 Ambiguous Codes

    • Cluster 1 Code

    • Cluster 1 Error Message

    • Cluster 1 Label

    • Cluster 1 Level

    • Cluster 1 Provider

    • Cluster 1 Status

    • Cluster 1 Version

    • Cluster 2 Ambiguous Codes

    • Cluster 2 Code

    • Cluster 2 Error Message

    • Cluster 2 Label

    • Cluster 2 Level

    • Cluster 2 Provider

    • Cluster 2 Status

    • Cluster 2 Version

    • Address Line 1

    • Address Line 2

    • Address Line 3

    • Address Status

    • Address Type

    • Address Verification Code

    • Address Verification Status

    • Administrative Area

    • Parsed Administrative Area

    • Building

    • Building Identifier

    • Building Leading Type

    • Building Name

    • Building Trailing Type

    • Core Based Statistical Area

    • Congressional District

    • Country

    • Date this record was created

    • Set of custom keys for this record

    • Delivery Address

    • Delivery Address Line 1

    • Delivery Address Line 2

    • Delivery Address Line 3

    • Delivery Point Bar Code

    • Department Info

    • Dependent Locality

    • Dependent Thoroughfare

    • Dependent Thoroughfare Leading Type

    • Dependent Thoroughfare Name

    • Dependent Thoroughfare Post Direction

    • Dependent Thoroughfare Pre Direction

    • Dependent Thoroughfare Trailing Type

    • Double Dependent Locality

    • Delivery Point Confirmation

    • Delivery Point Validation Footnotes

    • DVP Footnotes from Loqate CASS

    • Footnotes

    • Footnotes from Loqate CASS

    • Full Address

    • Full Address Furigana

    • Geo Accuracy Code

    • Geo Subdivision

    • Geo Subdivision 2

    • Geo Subdivision Label

    • Geo Subdivision Label 2

    • ISO 3166 3-character country code

    • ISO 3166 3-digit numeric country code

    • Latitude of address

    • Locality/City

    • Longitude of address

    • NUTS1

    • NUTS2

    • NUTS3

    • Administrative Subdivision of a country

    • Organization

    • Organization Name

    • Organization Type

    • Parent Address Sync

    • Parent Address VID

    • Post Box

    • Post Box Number

    • Post Box Type

    • Postal Code

    • Primary Postal Code

    • Secondary Postal Code

    • Premise

    • Premise Number

    • Premise Type

    • Private Consultation

    • Provincial Website Presence

    • Delta ID

    • Merged ID

    • Record Owner Name

    • Record Owner Type

    • Address Delivery Type

    • Record Type from Loqate CASS

    • Residential Delivery

    • Return Code

    • Sub Administrative Area

    • Sub Building

    • Sub Building Floor

    • Sub Building Name

    • Sub Building Number

    • Sub Building Type

    • Super Administrative Area

    • Thoroughfare

    • Thoroughfare Leading Type

    • Thoroughfare Name

    • Thoroughfare Post Direction

    • Thoroughfare Pre Direction

    • Thoroughfare Trailing Type

    • Urban Rural Typology

View specific sources

View the Data Lineage page by source system using the Select drop-down list at the top of the page. Select the checkboxes to view specific source systems or specific custom keys. You can also begin typing in the text field to limit the list of sources as you type.

The Data Lineage page refreshes to display only the sources that you have selected.

For information about unmerging sources from the record, see Unmerge records.

Legend

Icons display to help you identify information about objects and field values. Click Legend to view a description of each icon.

Same data as Network record - The field value from the source is the same as the Network record.

Current source for Network record - The field value is the current source for the Network record. The background is highlighted in green.

Different from Network record - The field value is different than the current value on the Network record.

Inactive sub-object - The status of the sub-object is inactive.

Deleted, Invalid or Merged_Into sub-object - The record state of the sub-object.

Note: If the Current Source column is not enabled for your Network instance, the Legend does not display.

Show or Hide options

Click the Show/Hide button to toggle column and object options. By default, all options are enabled. Your preferences are retained the next time you open the Data Lineage page.

  • Show Current Source Column - Display or hide the column. If you hide the column, the current source remains highlighted in green in its respective source column.

  • Show Inactive Sub-Objects - Display or hide inactive sub-objects. When the objects display, they can be identified by the Inactive icon.

  • Show Invalid, Merged_Into, and Deleted Sub-Objects - Display or hide sub-objects with these record states. When the objects display, they can be identified by the Invalid icon.

Jump to section

You can quickly jump to a specific section of the Data Lineage page by selecting that section in the drop-down list at the top left of the page. You can also begin typing to have your selection auto-complete as you type. You can expand or collapse sub-object detail by clicking the Arrow icon to the right of the object heading.

Sync from Veeva OpenData

Use the Sync with OpenData button to sync to the latest version of the record from Veeva OpenData.

Sub-object details

Addresses, licenses, and parent HCOs now contain a summary so you can easily view relevant information about the sub-object. Additionally, these sub-objects are now sorted so the active objects display before the inactive objects. Inactive objects display so you have a history of specific addresses. You can remove inactive sub-objects using the Show/Hide button.

Addresses

Address summaries contain the following details:

  • Formatted address

  • Fields that are defined as Is Summary Field on profile layouts.

    For example, in the profile layout, the Is Summary Field option is selected for Address Line 1. This means that the field will display in the summary header on the record profile, and it will also display in the address summary on the Data Lineage page.

  • Status or State (hidden if the address is Active and Valid)

  • Primary address flag (if applicable)

Sort order

Addresses are sorted in the following order:

  • Rank

  • Status (Active before Inactive)

  • Record state (ordered by Valid, Merged_Into, Merge_Inactivated, Merge_Added, Invalid, and then Deleted)

Parent HCO

Parent HCO summaries contain the following details:

  • HCO corporate name (click the link to view the business card)

  • Fields that are defined as Is Summary Field on profile layouts.

  • Status or State (hidden if the Parent HCO is Active and Valid)

  • Primary affiliation flag (if applicable)

Sort order

Parent HCOs are sorted in the following order:

  • Status (Active before Inactive)

  • Record state (ordered by Valid, Merged_Into, Merge_Inactivated, Merge_Added, Invalid, and then Deleted)

Licenses

License summaries contain the following details:

  • Fields that are defined as Is Summary Field on profile layouts.

Sort order

Licenses are sorted in the following order:

  • Status (Active before Inactive)

  • Record state (ordered by Valid, Merged_Into, Merge_Inactivated, Merge_Added, Invalid, and then Deleted)

Report on data lineage

Source data files can be created when target subscriptions run. Use transformation queries to save the exported source data as a custom table so you can report on data lineage.

For details, see Post-process source view data.