Sources of data
Within your Network instance, you can have data from multiple sources. This provides you with the flexibility to purchase data for specific countries or entity types, or to maintain your own data.
Veeva OpenData
Veeva OpenData content is purchased from and maintained by Veeva. In the Network UI, this data is identified by orange icons and fields.
You can subscribe to Veeva OpenData for any country where Veeva OpenData is provided. Veeva OpenData is loaded into your Network instances using subscriptions and ad hoc downloads. For more information, see Veeva OpenData subscriptions.
Add requests can be routed to Veeva OpenData if you have a subscription for the primary country of the new record. Network users can send change requests for the data provided to them and Veeva data stewards validate and update that data accordingly.
Third party data
Third party data is purchased from an external source. In the Network UI, this data is identified by blue icons and fields.
Third party vendors typically provide customers with stewardship services, so changes to data are routed to third party data stewards for validation and processing. Third party master data is never used to improve Veeva OpenData.
Data from third party sources is loaded into your Network instance using source subscriptions. When you create a system, you can define the DCR routing criteria by country, entity type, and fields.
Local data
Local data is created within your Network instance or is purchased from a source and is managed by local data stewards. In the Network UI, this data is identified by gray icons and fields (green icons and fields in China).
Locally managed data is loaded into your Network instance using source subscriptions and add requests.
Types of locally managed data
For data privacy, locally managed data that is loaded from source systems can be identified as proprietary and/or restricted data. These flags can be selected when administrators create the system in Network.
Additionally, for contact organizations, data can be defined as externally mastered using the is_externally_mastered__v field. The externally mastered field is typically used with proprietary data sources.
| Type | Defined By | Description |
|---|---|---|
| Restricted | System setting (System Interfaces > Systems) | Data loaded into Network using the system is flagged as restricted. |
| Proprietary | System setting (System Interfaces > Systems) | Data loaded into Network using the system is flagged as proprietary. |
| Externally mastered | Data model field | Enable the field in your Network instance and include the field in your source file. |
Restricted data
Information about the source and its contributions to a particular record (data lineage information) cannot be viewed by Network users unless they have explicit permission through their data visibility profile. Restricted systems can be proprietary or non-proprietary.
Proprietary data
The proprietary flag applies to data change requests only. Defining local data as proprietary ensures that add and change requests that are submitted through the Network API will not be sent to Veeva OpenData.
Proprietary settings are tracked by system rather than field.
The is_proprietary__v data model field is not being used in Network currently but displays in the Record Information section of a record profile if enabled. Regardless of whether this field is enabled in a record profile, its value is ignored; the record's proprietary status is strictly determined by the system that provided it.
Externally mastered data
The is_externally_mastered__v field can be used to identify contract organization records; records that contain legal and other data and that should not be updated by other data sources or even viewed by most users. This field is typically used with proprietary sources for legal reasons.
Defining externally mastered records
The is_externally_mastered__v field must be enabled in the data model and added to the source file for your source subscription. To flag records as externally mastered, set the value of the field to Y (Yes/True) in the source file. Run your source subscription (for example, the subscription for your proprietary system) to load the data into Network.
If records are loaded before the field is enabled, the data will not be secured from updates and merges. Similarly, if the field is enabled and included in the source file but then the field is disabled after the data is loaded, those records will not be considered externally mastered.
Externally mastered record behavior
The is_externally_mastered__v field is used for logic in match rules and data visibility profile permissions.
When records are flagged as externally mastered in data loads, the following behavior occurs:
- Network retains all sub-object data on data load (for example, Network does not deduplicate or cleanse addresses).
- Records cannot be updated or merged with Veeva OpenData.
- Records can be filtered from target subscriptions so they are not sent to downstream systems.
- Records are restricted from users through data visibility profiles. The Can Search Contract Organizations permission must be enabled for users to see these records.
- Addresses are not cleansed by Network's third party cleansing service.
- Records should be updated only by the original source system using data loads. Any other changes will be overwritten by source updates.
Data management for record types
Records can contain data from different sources
. On the Profile page, users can view all types of records and can identify ownership of the data from the colors of the fields on the record: orange (Veeva OpenData), blue (third party source), gray (locally managed), green (locally managed for China). When users add or update data, the change requests are routed to the appropriate source for verification and processing.
The source of the record can determine the specific behavior for some Network features.
| Locally Managed | Third Party | Veeva OpenData | |||
|---|---|---|---|---|---|
| Regular | Proprietary | Restricted | |||
| Record data (viewed on Profile page) | Records contain locally managed data only. | Records can contain third party and locally managed data. | Records can contain locally managed and Veeva OpenData data. | ||
| Data change requests (DCRs) | DCRs are routed to local data stewards. | DCRs are routed to third party or local data stewards based on field ownership. | DCRs are routed to Veeva OpenData or local data stewards based on field ownership. | ||
| Matching and merging records |
Locally managed records can be matched and merged with third party and Veeva OpenData records. Source survivorship rules determine the fields that win in a merge. If you do not want records to be matched with master records, you can add restrictions using the match rules. Note: If proprietary records are flagged as externally mastered, those records will not be matched or merged. |
Locally managed records can be matched to third party records. Third party records always win. Third party data cannot be matched with other master sources. Suspect match is not permitted with local or Veeva OpenData records. |
Locally managed records can be matched to OpenData records. Veeva OpenData always wins. Suspect match is not supported in customer instances for Veeva OpenData records. |
||
| Match against OpenData | Yes |
Yes Note: Proprietary records that are flagged as externally mastered will not be matched. |
Yes | No. This option does not display in source subscriptions for third party systems. | Not applicable |
| Send to OpenData | Yes | No - if one source contributing to the record is flagged as proprietary, this option is not available on the record. | No | No - third party records are never sent to Veeva OpenData. | Not applicable |
| Data Deduplication | Yes |
Yes Note: Proprietary records that are flagged as externally mastered will not be deduplicated. |
Yes | No | No |
| Data lineage | Yes | Yes | No | Yes | Yes |
| Address cleansing |
Yes Note: Address cleansing is not applied to locally managed records that are flagged as externally mastered. |
No | Yes | ||
Incoming data from multiple sources
Incoming data is automatically cleansed and deduplicated by Network. If you subscribe to Veeva OpenData, certain incoming data can also be cleansed and matched against it, depending on the incoming data source and system settings. Similarly, if you subscribe to third-party data sources , you can match the third-party data with the data in your Network instance to get cross-references between identifying keys of the third-party data and data from other systems.
Example
The HCP record "Frederick Robert Jones" in your Network instance could match against these incoming records from the following sources:
• "Fred Jones" - Veeva CRM
• "Frederic R Jones" - ERP data
• "Frederick Jones" - expense system
Matching and deduplicating against these sources enables you to get a consolidated view of all interactions with that customer physician. You can then compare call activity (from CRM) and spending (from expenses logged in the expense system) with ERP data (from the ERP data provider).
Note: Data is never matched or cross-referenced between Veeva OpenData or multiple third-party systems. Data from proprietary systems is also never shared with Veeva OpenData or third-party systems.