Updating Veeva OpenData subscriptions

AD
DM

As your data requirements change, you can update your subscriptions; for example, add or remove HCP and HCO specialty filters, filter parent HCOs, enable Veeva OpenData fields, and so on.

Refreshing subscriptions

After the initial implementation, a delta export is downloaded each time a subscription runs. This means that records are downloaded only if they have been added or changed by OpenData since the last time the subscription ran.

To download additional records the next time your subscription runs (instead of waiting for the records to trickle down into your Network instance as OpenData makes changes to those records), edit your country subscription and use these options:

  • Add HCP or HCO filters - If you add filters to subscribe to additional records, when you save the subscription there is an option to download all records that meet the new filter criteria the next time the job runs. The option is selected by default, but it can be cleared if you do not want to download a potentially large number of records. After the records are downloaded, the next time the subscription job runs, it goes back to the usual delta export behavior.

    Note that if you update your OpenData subscription filters and restrict the records that you receive from Veeva OpenData, any records that used to meet the filters are not removed from your Network instance.

  • Add records to your working set - To force the new records in your working set to be downloaded, use the Update all records setting in the country subscription.

  • Subscribe to record components - When you enable a field-level subscription (for example, Email Addresses) or an OpenData field, the new data is included in the OpenData delta export only if the record has been changed by OpenData. You can request updates for specific records from Veeva OpenData. During the targeted job, all available updates for the record are pushed to your Network instance. For more information, see Request targeted record updates.

    Note: Using the Update all records setting will not necessarily download the new data; if the version number of the record in your Network instance is the same as the version number in the most recent full export from OpenData, no updates will occur even if your record doesn't have the latest data.

Update subscriptions

To update a subscription for a particular country:

  1. In the Admin console, select System Interfaces > Veeva OpenData Subscriptions.

    On the Veeva OpenData Subscriptions page, click a country name. The country name displays as a hyperlink only if the subscription is enabled.

    The country subscription page opens.

  2. Job Schedules & Triggers - Define the schedule for the job and any subsequent actions that will start when this job finishes.

    • OpenData exports to process - Identifies the files that are pending for subscribed and enabled countries.

      These are the files in the delta export; the records that have been added or changed in the Veeva OpenData instance for this country since the last time your subscription ran.

    • Updates from OpenData - The time of day that updates are available using your timezone. If a schedule is not set for an OpenData instance, administrators will see the following message: OpenData provides updates for records for this country at their discretion.

      See OpenData countries to view the export schedule for all OpenData country subscriptions.

    • Job Schedule - Define the schedule for the subscription (if not set to Manual). You can set a schedule for the subscription similarly to other subscriptions in Network.

      Tip: Group countries that have a similar schedule so you only need to manage one schedule for all the countries. For details, see OpenData country groups.

    • Job Triggers - Trigger other actions to start after a job runs.

      Available triggers:

      • Send email - Specify users that should be notified for successful and unsuccessful job outcomes.

      • Start a job - Start a subsequent job when this job successfully completes.

      For more information, see Subscription job triggers.

  3. Working Set FTP - The FTP path to the working setClosed A list of Veeva IDs (VIDs) in a file that determines the initial data set to download from a Veeva OpenData instance to a customer instance. for the subscription. The FTP path cannot be edited.

    The working set is a file that lists all of the Network HCO or HCP IDs (VIDs) that you want to download to your Network instance. This is typically used in your initial implementation. Only the records that are listed in the working set are downloaded; HCOs related to the HCPs in the Working Set will only be downloaded if the VID of the HCO is specifically listed.

    Note: The working set should not be updated on an ad-hoc basis; use the HCP/HCO filters or ad hoc download jobs to subscribe to additional records.

  4. Health Care Professional - To subscribe to additional HCP records to download to your Network instance, select the Subscribe to additional Veeva OpenData HCP records option.
    1. Click + Add Condition.
    2. In the Field list, choose Types or Specialties. The field values are reference types.
    3. In the Condition list, choose Contain or Does not contain.
    4. In the Value list, select all of the values for the filter. Start typing to find a specific value. The values in the list map to the reference codes assigned to the selected reference type.

    In this example, only prescribers who specialize in thoracic surgery will be downloaded from Veeva OpenData.

    Tip: You can also use the filters to exclude records (by specialty and type) that you do not want downloaded to your Network instance from Veeva OpenData.

    If multiple master support is enabled in your Network instance, any HCP types that are owned by a third party data provider are automatically excluded from the Value list to ensure that they are not selected. To review any types owned by third party providers, click the See excluded HCP types below link.

  5. Health Care Organization - Subscribe to additional HCO records. You can also use the filters to exclude records (by specialty and type) that you do not want downloaded to your Network instance.

    To subscribe to additional records:

    1. Click + Add Condition.
    2. In the Field list, choose Types or Specialties.
    3. In the Condition list, choose Contain or Does not contain.
    4. In the Value list, select all of the values for the filter. Start typing to find a specific value.

    In this example, HCOs that specialize in Dermatology will be downloaded from Veeva OpenData when the country subscription runs..

    If multiple master support is enabled in your Network instance, any HCO types that are owned by a third party data provider are automatically excluded from the Value list to ensure that they are not selected.

  6. Parent HCO - Define the following settings to manage the Parent HCOs that are downloaded as affiliations when you download HCPs and HCOs.

    • Level of parents to download - By default, five levels of Parent HCOs are downloaded. This setting applies to add requests, change requests, Ad Hoc Download, and Sync with OpenData actions. It does not apply to the working set or filters; parent affiliations are not downloaded during scheduled jobs.

    • Download repointed Parent HCOs - Download Parent HCOs that have been changed by Veeva OpenData but do not meet your filter criteria. For information, see Updating repointed Parent HCOs.

    • Filter Parent HCOs during Ad Hoc Download and Data Change Requests - Select to filter unwanted Parent HCOs from being added to your Network instance when data change requests are processed or when users are syncing or downloading Veeva OpenData records.

      When this setting is selected, two options display:

      • Use Basic Filters to Filter Records - Use the Specialty or HCO Type fields to filter the parent HCO records that are downloaded.

      • Use NEX Rules to Filter Records - Use Network Expression rules to specify the parent HCOs to include or ignore during OpenData downloads. Lookup tables can be used in the NEX rule.

        Write the NEX rule in the provided box. The expression validates as you write the rule.

        • Format and Validate - Click to organize the expression in a readable format.

        • Copy All to Clipboard - Use if you need to copy the expression and save to your clipboard.

        • Test - Use a valid Veeva ID (main object) to test the outcome of the expression.

        • Expand - For long rules, use the handle () to expand the box so you can see the entire rule.

          For more details about formatting and testing NEX rules, see NEX tester.

      For more information, see Filtering Parent HCO records.

  7. Additional OpenData Parent HCO Records - Download Parent HCOs that are related to the OpenData HCOs and HCPs that are already downloaded to your Network instance.

    When Veeva OpenData country subscriptions run, parent HCOs are not downloaded; parent HCOs are only downloaded during ad hoc download jobs. Use this option to download those missing Parent HCOs so you have complete relationships in your instance.

    Parent HCOs will be downloaded in a separate job after the country subscription runs. For more information, see Downloading OpenData parent HCOs.

  8. Updates to OpenData records -Select Update all records if you want to update all of your records to the latest version the next time the subscription runs.

    Important: After the Update all records checkbox has been selected, it cannot be cleared until after the next job runs.

    Each time your subscription runs, only the records that have been changed in OpenData since the last time the subscription ran are included. This means that records in your working set will trickle into your Network instance as they are changed by OpenData, so some of them might not be downloaded to your Network instance. When you select Update all records, all of the records in your working set are forced to be downloaded and any records that you have already downloaded will be updated to their latest version.

    • If the version number for the record in your Network instance and the full export is the same, no updates occur even if the record in your Network instance doesn't have the latest data. For example, if you enable the email subscription for a country, emails will only be exported as OpenData makes changes to records.

    • To receive the updates when you subscribe to new data subscriptions, you can request updates for specific records from Veeva OpenData. During the targeted job, all available updates for the record are pushed to your Network instance. To learn more about this feature, see Request updates to OpenData records.

    Note: The next time the job runs, either manually or on a schedule, the updates will be completed and this setting will be reset automatically; Network will clear the checkbox. There is no need to return to the OpenData subscription to manually clear the Update all records setting before the next job runs.

  9. Field Level Subscriptions - Select any additional subscriptions that Veeva OpenData offers in that country.

    The following subscriptions are available:

    • Include Email Addresses - Receive email address updates from Veeva OpenData. For existing customers where email addresses are available in Veeva OpenData, this checkbox is selected by default. For new customers who have just signed up for the Veeva OpenData email subscription, this checkbox is not enabled by default. For more information, see Veeva OpenData email addresses.

    • Include HIN data - Receive Health Industry Number (HIN) IDs. Available for the United States. For more information, see HIN data.

    • Include NCPDP Data - Receive National Council for Prescription Drug Programs (NCPDP) IDs for HCOs. For more information, see NCPDP data.
    • Geo Subdivision Data - Subscribe to Geo subdivision data for addresses. Available for France, Hungary, and Monaco.
    • Geo Subdivision 2 Data - Subscribe to Geo subdivision data for addresses. Available for Hungary.
    • Include CIP ID Data - Subscribe to Club Inter Pharmaceutique (CIP) data. Available for Andorra, France, and Monaco.
    • Include AMA data - Ignore. US OpenData subscriptions no longer include AMA data.
    • Pharmacy Operating License - Subscribe to data confirming a pharmacy's permit to operate as a public pharmacy.

    To subscribe to these additional offerings, contact a Veeva OpenData representative.

    Important: Before selecting any of these options, ensure that any locally managed data that currently uses these fields is backed up using target subscriptions. Locally managed data will be overwritten when Veeva OpenData begins to manage the field.

  10. Handling of OpenData Opt-Outs - Automatically convert records that have been opted-out by HCPs into locally managed records.

    The option is disabled by default. Select the option if you want to start managing future opted-out records so you can retain the record history and continue to engage with the HCP.

    For more information, see Converting opt-outs from OpenData into local records.

  11. OpenData Instance Details - Information about the OpenData instance that your Network instance is connected to for this country subscription.

    • Domain Name - The OpenData instance name.

    • Network ID - The OpenData instance ID.

  12. Add requests routing to Veeva OpenData - Route add requests for specific HCP or HCO types to your local data stewards.

    This is helpful if you use any Veeva-owned HCP or HCO types for local records so those add requests can be processed locally. In the HCP or HCO section, click Add Exceptions to Add Request Routing to define the type.

    By default, all HCP and HCO types are routed to Veeva OpenData unless it is a custom type or a type managed by a third party data provider.

    Note: This section displays only if this feature is enabled in your Network instance. For more information, see Routing add requests.

  13. Veeva OpenData Fields - This section displays if a new field has been added in that country.

    1. In the Available Fields column, expand the entity or sub-object to view the new fields.

    2. Select the field and click the Right Arrow icon to move the field into the Selected Fields column.

      3. To quickly receive data in that field for all records, in the Updates to OpenData records section, choose Update all records. The next time the subscription runs, the field will be updated for all records that you have downloaded in your Network instance.

    For more information, see Adding Veeva OpenData fields.

    Note: All Veeva OpenData fields are mastered by default in new Network instances, so all fields will display in the Selected Fields pane. When new fields are introduced, you can add them to the Selected Fields pane.

  14. Excluded Record Types - This section displays if you have multiple master support enabled in your Network instance. Any HCO/HCP types that are owned by third party data providers display in this section and are excluded from the lists in the HCO and HCP subscription section. This section is for reference only.

    When you filter HCO and HCP subscriptions for Veeva OpenData, Network also cross-references the HCO/HCP types that are defined for third party data sources before downloading any records.

    Example

    Your Veeva OpenData subscription for France is filtered to download HCP records that specialize in cardiology. In your Network instance, you have already confirmed a third party data source to master records for prescribers (HCP type) for France. When a subscription runs, Network cross-references the filters. Because prescribers are owned by the third party data source, Network will not download any prescribers who specialize in cardiology in France from Veeva OpenData.

  15. Save your changes.

  16. A pop-up displays to confirm your changes. If you have changed HCP and HCO filters, the option to download additional records is selected by default. Clear the option if you do not want the next job to include all of the records that meet your new criteria.

  17. Click Save Changes.

    Tip: Refresh your web browser page after making changes to the OpenData subscription.

    If you changed HCP or HCO filters, a message displays at the top of the subscription to notify users that the filters have been changed and additional records may be downloaded the next time the subscription job runs. After the job runs, the message is removed.

Country subscriptions for all records

If you purchase all OpenData records for a particular country and you have specifically requested to have all records downloaded to your instance, the country subscription page indicates these details.

The page also does not include the section for the Working Set and the Health Care Professional and Health Care Organization sections because there is no need to maintain a working set or HCO/HCP filters.

You can continue to manage the parent HCOs that are downloaded, additional field level subscriptions (for example, Email and Geo Subdivision), and Veeva OpenData fields.

Job details

After an OpenData country subscription runs, a Job History section displays at the bottom of the page. Click the link in the ID column to open the Job Details page.

The Job Details page provides information about the objects that were added, updated merged, and so on.

Note: Job Details are also available for Ad Hoc Download jobs. On the Details page, the Subscription label identifies these jobs as dfb_imports.

Job errors

If issues occurred, a Job Error Log section displays on the page. Administrators can export the job error log to further investigate the issues.

The log downloads as a Microsoft® Excel® file to your local computer. The name format of the downloaded file is: <country_code>-OpenData-Subscription-<Job_ID>-Error-Log.xlsx.

For example, an error log for a US OpenData subscription would be US-OpenData-Subscription-Job-46279-Error-Log.xlsx.

Error log details

The log contains a maximum of 500,000 rows. The errors do not display in any specific order.

The following columns are included in the log file:

  • Job ID - The ID of the OpenData subscription job.

  • Timestamp - The data and time the job ran.

  • Entity ID - The Veeva ID (VID) of the object or sub-object.

    In some cases, an error occurs and the record cannot be loaded/processed so no VID can be found and cannot be displayed here. In these cases, the column will show the best 'identifier' information it can. For example, it might display Address:7 to indicate an error was encountered for the seventh address record on a new, incoming record.

    In other cases, the error is due to something about the job and the fact it couldn't run or it was canceled. In those cases, the column will list the job ID in the first column (@job:46279).

  • Level - The issue level: WARN, INFO, or ERROR.

  • Stage - The processing stage that the error occurred.

  • Rule - The Network rule that triggered the error. For example, [entity.model.rules:ADDRESS]:IF( address_type__v == 'M', REJECT()).

  • Code - The Network error code.

  • Message - An explanation of the error.

  • Parameters - VID and object information, when applicable.