Updating records with the Data Updater

AD
DM

Administrators and data managers can use this feature to quickly make simple updates to records using .csv files and Microsoft Excel® (.xlsx) files. For example, you might want to update primary addresses or deactivate records. Data update jobs are ad hoc jobs; they cannot be scheduled.

Veeva standard objects and custom objects are supported.

Data updater benefits

  • Users can update records without needing the Network training that the robust source subscriptions feature requires.
  • Parent objects are not required. For example, to update an address, provide the VID for the address only; you do not need to upload the HCP that is associated with the address.
  • Microsoft Excel and .csv files are supported. Microsoft Excel files must be .xlsx files only.
  • Files that you upload can be stored locally. Users do not need access to your FTP server.
  • Data is scanned to ensure it is valid before records are updated.
  • Job validation rules ensure that critical values are not changed on a large number of records.

Data supported for updates

Before you create a file to update the records in your Network instance, review the records and fields supported by the Data Updater.

  • Objects - Veeva standard objects and custom objects can be updated.
  • Record state - Only records that have a VALID record state can be updated.

  • Fields - The following fields are not supported for update:

    • system fields (created_date__v, record_state__v, and so on)
    • alternate keys
    • read-only fields (For example, primary_country__v. The primary country of an existing record cannot be changed).
    • custom keys
    • entity_vid__v
  • Field ownership - Custom fields can be updated on OpenData or third party master records. Updates to OpenData or third party managed fields will not be processed on OpenData records or third party records.

Job validation rules

When data is updated, job validation rules can prevent critical value changes from occurring on a large number of records. The rules are applied to the job if the uploaded files contain the fields that are defined in the rules. If changes try to occur on a large number of records, the job can skip the updates or fail depending on each rule configuration.

Administrators can set all enabled job validation rules to run on data updater jobs (Settings > General Settings). When the setting is selected, applicable rules will run on all jobs.

Create a file

To update the records in your Network instance, create a .csv or Microsoft® Excel® (.xlsx) file that includes the VID of the record and the fields that you want to update. Each file should contain updates for one object type only; for example, create one file for HCPs and one file for licenses.

File requirements

Files are validated when they are loaded. If errors are found, the upload can fail.

The following requirements must be met:

  • File data - The file must include at least two columns:

    1. vid__v - The VID of the object.

    2. A field to update - One or more fields that you want to update on the records. This could be a Veeva field (__v) or a custom field (__c).

  • Format - The file format must be either .csv or .xlsx.

    If you are uploading a Microsoft Excel (.xlsx) file, ensure that the cells in the VID field column are defined as a text; otherwise, the VIDs might be trimmed in the file.

  • Size - A maximum of 200,000 records can be included in the file.
  • Delimiter - A comma must be used to separate items in a column.

Example file - Addresses

The primary address field is being updated for the addresses specified by their VIDs. Note that the VIDs for the owning objects (HCPs or HCOs) is not required.

Support for null values

The Data Updater supports updating fields to have a null value. In your file, add null__v to the column. After the job runs, the field value will be cleared on the specified record.

Upload the file

Add a file to update the records in your Network instance.

  1. On the Network menu bar, click Data Updater.

    This tab displays for administrators and data managers only.

  2. Click Update Records.

  3. On the Upload File tab, the following details are required:

    • Select Object - Expand the list and choose the object type that you are updating.

      Custom objects that are enabled in your Network instance display if you have access to the object through your data visibility profile. Custom sub-objects and custom relationship objects display if you have access to the main object.

    • Verify and clean addresses - This option displays if you choose the Address object. The option is disabled by default.

      Address cleansing verifies addresses and parses the data into Network fields. Because cleansing can change the address data in your uploaded file, you might not want cleansing to occur; for example, if you are correcting postal codes on existing addresses. However, it is helpful to use Network's address cleansing to correct entire addresses. You can enable address cleansing depending on the address data in your file.

      File requirements

      • Address cleansing is disabled - Only the vid__v column is required in the file that you upload. When the job runs, the updates in your file are applied to existing address records.
      • Address cleansing is enabled - Several column headers (fields) are required in your file: vid__v, address_line_1__v, locality__v, administrative_area__v, country__v, postal_code__v. These fields ensure that address cleansing can occur. If the fields are not in your file as column headers, the upload will fail.

        • If address cleansing is not enabled for a country, but the Verify and Clean Addresses option is set, the data updater job simply skips the address verification and cleansing for any records for that country.

    • Source System - Select the system that applies to these updates. Third party, proprietary, and restricted systems do not display in the list.

      The system is used for source rankings. Ensure that this system is the highest source in your source ranking so you can override the existing field value.

      Source aliases are also supported; they are extracted from the system that you select.

    • Third Party Data - Indicate if the file that you are uploading contains third-party data. If you choose Yes, confirm that you either have a TPA in place, or that your agreement with the third-party data provider does not require a TPA for Veeva to receive the data.

      The responses to these questions are logged in the System Audit Log. For details, see Confirming third-party agreements.

    • File Upload - Drag your file into the box or click Upload File to choose a file. Click See Example to display a sample of the expected file.

    For example, we can upload this file to update HCP records.

    When the file is added, it is scanned to validate the data. If there are any issues with the file, a message displays.

    Possible issues:

    • Missing VID field - the vid__v column is missing from your file. (Error)
    • No Field to Update - There is no other field included in your file. This can also occur if you have selected the incorrect Object for the file. (Error)
    • Invalid VID Fields - A VID is not correct. (Error)
    • Record State issues - The record state is not supported for update. Only VALID record states are supported for these jobs. (Warning)
    • Data Type issues - The value does not match the type of reference field; for example, there is text in an integer field. (Warning)

    A count for each issue is provided. Download the error or warning file to identify the records with issues. The file downloads as a Microsoft Excel file. The file can contain a maximum of 250,000 errors. If the limit is reached, errors will stop being written to the file.

    Fix the issues and upload the file again. If the file contains warnings only, you can download the file and fix the warnings, or ignore the warnings and upload the file.

    The upload will fail if at least half of the records have issues; otherwise, the file will be uploaded.

Review the file summary

When the file is uploaded, you can view a summary of the data before you commit the changes to Network.

  1. Review the details on the File Summary page.

    A count for the following details display at the top of the summary:

    • Records read - The total number of rows included in the file for the specified object.
    • Fields Recognized - The number of valid fields included in your file.
    • Fields Unrecognized - A count of fields that are not known. Fields are not recognized because the column header is not a field in the Network data model or the field is not associated with the specified object.
    • Fields Ignored - A count of fields that were ignored. Fields are ignored because they are not supported for update; for example, read-only fields, alternate keys, system fields, custom key fields (other than status), and the entity_vid__v fields (ownership of sub-objects cannot be changed).

    The details for these counts display in the table below. For reference, the first three columns for each of the fields display.

    If any address cleansing was enabled for address object updates, a message displays in the Status column to advise you that address cleansing will occur.

  2. To commit your changes, click Update Records. In the Confirm Record Updates dialog, you can add details to the Job Notes box. This can be helpful to provide context about the updates and will display on the Job Details page after the Data Updater job runs.
  3. Click Yes, Update Records.

Update records

When you commit your updates, the Data Updater page opens and a message displays to notify you that the job is started. The Status column displays the progress of the job.

On the Data Updater page, data managers can see the jobs that they have run; administrators can see all of the jobs run by any user in the Network instance. Filter the list of jobs by object type or source system.

Click the link in the Job ID column to open the Job Details page.

Job details

The Job Details page provides a summary of the data updater job.

Job Results

This section provides an overview of the records that were updated.

The following counts can display:

  • Records updated - The number of records that were updated for the selected object.
  • Other affected objects - A count of records that were updated that were not the selected objects.

  • Records Skipped - A count of records that were not updated. Click Download Skipped Records to view the issues.

    Example skipped records file

    The file might not contain VIDs. Typically, this means that the records are updated already, or updates were not made due to survivorship.

    vid__v Category Type Details
      No Update ERROR No update made to 4 PARENTHCO record(s).

Job Overview

This section displays the job information, including any job notes that you provided. If the job updated address objects, this section also indicates if address cleansing was selected for address object jobs.

File Summary

Review the file details, selected object, and fields that were included in the file.

Primary fields update summary

Updates that occur on primary fields during subscription jobs. Many customers use primary fields for business purposes like incentive compensation or territory alignment. When changes occur to primary fields, it can impact those processes, so this ensures that you are aware of the updates and you can make necessary changes.

Primary field updates display for the Unique Checkbox primary type field only. The summary and counts do not reflect changes to the Network Calculated primary type field.

The summary does not display if your Network instance does not have any primary fields defined.

Sub-objects and relationship objects

For each object, the enabled primary fields display and they are categorized so you can see how many primaries were added, changed, or removed during the job.

The following actions are tracked:

  • New Primary - A primary field was assigned to an entity.

  • Primary Changed - The primary was assigned to an entity but it was moved to another entity during this job.

  • Primary Removed - The primary was assigned to an entity but during this job it was removed and was not assigned to another entity.

Job Error Log

This section displays if the job included any errors.

Data lineage

Updates that are made to records using a data updater job display on the Data Lineage page, similar to a source subscription.

Reporting

Data updater job statistics are available in Network reports.