Downloading records from Veeva OpenData

AD
DM

Veeva OpenData records can be downloaded to your Network instance through the following actions:

When records are downloaded, affiliations and sub-objects (addresses, licenses, Parent HCOs) might also be included in the download. To understand what is being downloaded with each action, see Records downloaded to your Network instance.

The following sections explain the different actions for downloading or updating records and when you would use each action.

Scheduled jobs (working set and filters)

Veeva OpenData subscriptions are used for scheduling regular data downloads. Subscriptions can also be manually run.

When a subscription runs, it uses the following settings to identify the records that are updated or downloaded to your Network instance:

  • Working set

  • HCP and HCO filters

After the initial implementation, a delta export is downloaded. This means that each time your subscription runs, only records that have been added or changed in the Veeva OpenData instance since the last time your subscription ran are downloaded.

Working set

The working set is a .csv file that is typically used during your initial implementation. The .csv file lists all of the Network entity IDs (VIDs) that you want to download to your Network instance. For example, to subscribe to HCPs and HCOs that specialize in cardiac medicine, you would include those VIDs in your file.

Only the records that are listed in the working set are downloaded; parent affiliations are not downloaded unless those VIDs are included in the file.

The list is added to an FTP directory. When you run the Veeva OpenData country subscription, those records are downloaded to your Network instance. Each time the subscription runs, the records from the working set are updated only if the record has been changed in the OpenData instance.

Note: To subscribe to additional records after the initial implementation, use the HCP/HCO filters in the country subscription or ad hoc download jobs. If you add VIDs to the working set, those records will only be downloaded when they are changed by OpenData. To force the new records in your working set to be downloaded, use the Update all records setting in the country subscription.

HCP and HCO filters

In addition to the records in the working set, you can also subscribe to HCP and HCO records by Specialty and Type.

Note: If you add an HCO filter, all HCOs that meet that criteria are downloaded; not only the HCOs that are related to the HCPs in your working set and Network instance.

Downloading newly subscribed records

If you update the HCO or HCP filters after your initial implementation, there are two options for downloading new records:

  • Download all records - All records that match the filter criteria will be downloaded the next time the subscription runs.

    If you choose this option, after the subscription is refreshed to include all records, the next time the job runs it reverts back to being a delta export.

  • Download the delta - Records are downloaded over time as they are added or changed by Veeva OpenData (delta export).

The option to download all records is available when you save your updated subscription. By default, all records will be downloaded, but you can clear the selection because this could result in a very large number of records being downloaded.

For more information about creating and updating country subscriptions, see Updating Veeva OpenData subscriptions.

Records that are downloaded during scheduled jobs

The following records are downloaded or updated when a subscription runs:

  • Records listed in the working set

    • If a losing record of a merge is included in the working set, the merge winner is also downloaded so your Network instance contains complete data. Parent HCOs of the merge winner are not added unless you have subscribed to additional HCOs and they match your HCO filters.
  • Records that match the HCP and HCO filters (specialty and type). The HCOs that are downloaded do not need to be related to the HCPs in the working set file or your Network instance.
  • Records that you've manually downloaded using ad hoc jobs. When you download an individual record, the record is not added to the working set or the filters, but their updates are included in the delta export every time the OpenData subscription runs.

Records that are not downloaded during scheduled jobs

The updates exported from OpenData in the regular delta files do not include the following records:

  • Parent HCO records - Parent affiliations of the entities listed in the working set or that meet the filters defined in the subscription are not downloaded during the job.

    Use the Additional OpenData Parent HCO Records option to download these related records. See Downloading OpenData parent HCOs.

  • Changes to Veeva OpenData custom fields - Any changes to these fields are for Veeva OpenData internal use only; they do not contain useful information for customer instances.

This ensures that the data takes less time to load and process in your Network instance.

Data change requests

All users can submit add or change requests for Veeva OpenData records. Any update to an existing OpenData record creates a change request that is routed to Veeva OpenData.

Records that are downloaded

When DCRs are processed by Veeva data stewards, the following records are downloaded:

  • DCR record - The record that you submitted the request for is added or updated in your Network instance.

  • Affiliated records - Up to five levels of HCO affiliations might be downloaded to your Network instance.

    You can control the affiliated HCOs that are downloaded by using basic filters (types or specialties) or NEX rules. You can also specify the number of parent HCOs that are downloaded with change requests. For more information, see Filtering Parent HCO records.

    Note: Related HCOs will be downloaded only if they are in the same country. For example, if an HCP is related to an HCO that is in another country, the related HCO will not be downloaded.

Ad hoc jobs

Ad hoc jobs are one-time job actions that download or update Veeva OpenData records in your Network instance.

Types of ad hoc jobs:

  • Download from OpenData - Download a record that is not in your Network instance.

  • Ad Hoc Download - Download or update one or more records using their Network entity IDs (VIDs).
  • Sync with OpenData - Download or update one or more records by selecting the records.

Note: The Ad Hoc Download and Sync with OpenData jobs perform the same actions (updating and downloading one or more records).

Records downloaded in an ad hoc job

When ad hoc jobs run, the following records are downloaded to your Network instance:

  • Specified record - The record that you selected or defined.

  • Affiliated records - Up to five levels of HCO affiliations might be downloaded to your Network instance.

    You can control the affiliated HCOs that are downloaded by using basic filters (types or specialties) or NEX rules. You can also specify the number of parent HCOs that are downloaded with change requests. For more information, see Filtering Parent HCO records.

    Country considerations
    Related HCOs will be downloaded only if they are in the same country. For example, if an HCP is related to an HCO that is in another country, the related HCO will not be downloaded.
    If you have HCPs that practice in more than one country, use the Multi-country HCP feature to ensure that the HCPs affiliations from all countries are downloaded.

Updates for downloaded records

After you download a record using an ad hoc job, the record is not added to the working set or the filters but it is updated when the OpenData subscription runs (if changes have occurred on the record).

Download from OpenData

Download individual records from Veeva OpenData using the Download from OpenData button.

Records can be downloaded from different areas in your Network instance.

Search results

When an administrator has enabled the Search OpenData setting (Settings > General Settings) in the Network instance, the search results can display records in the Veeva OpenData database depending on the permissions in your data visibility profile. If the record is not in your Network instance, a Download icon displays beside the record.

Click the Download icon to immediately download the record.

Search map

You can also download the record using the Download icon on the business card that opens from the map pin.

Record profile

You can review an OpenData record that has not yet been downloaded to your Network instance. Select the record from the search results to open the profile page.

To add the record to your instance, click the Download from OpenData icon beside the record name or the button at the top of the page.

Ad Hoc Download

In the Veeva OpenData subscription for each country, the Ad Hoc Download button can be used to immediately download or update specific records using Network entity IDs (VIDs). The VIDs must be for the same country as the subscription. For example, VIDs for OpenData in France cannot be downloaded using Ad Hoc Download in a US Veeva OpenData subscription.

To immediately download or update specific records:

  1. Open your Veeva OpenData country subscription.
  2. At the top of the page, click Ad Hoc Download.

  3. Add the VIDs to the Ad Hoc Download dialog and click Next.

    A confirmation dialog displays the count of records that will be downloaded or synced (updated).

  4. Click Sync to begin the download.

Sync with OpenData

Use the Sync with OpenData job to download records to your Network instance and to update OpenData records that have already been downloaded.

Sync with OpenData is available from the search results page or the data lineage page. This is the same job action as the Ad Hoc Download button on the Veeva OpenData page; in this case, you select the records instead of defining the VIDs.

Search results

When Search OpenData is enabled in your Network instance (Settings > General Settings), records in the Veeva OpenData database display in your search results if you have permission to search for them through your data visibility profile.

  1. In the search results, select the records to download from OpenData and to update in your Network instance.
  2. Click Sync with OpenData at the top of the page.

    A confirmation dialog displays the number of records that will be downloaded or synced (updated).

  3. Click Sync to immediately start the job.

Data lineage

Use the Sync with OpenData button on the data lineage page to get the latest data for a record from Veeva OpenData.

  1. To access the Data Lineage page, on the record profile, find the Data Sources preview box and click More Details.
  2. In the record header for the OpenData source, click the Sync with OpenData button.
  3. In the confirmation dialog, click Ok.

    A message displays at the top of page to advise you that the record is being updated from the Veeva OpenData database.

  4. Refresh the page. The Database Last Updated date for the record is updated to reflect today's date.