Anonymize HCP records

AD
DM

Network administrators and data managers can create data maintenance jobs to anonymize deleted HCP records. This feature helps customers comply with the European General Data Protection Regulation (GDPR) by supporting requests from HCPs to remove their data and to remove records that exceed data storage period limitations.

Highlights

When deleted HCP records are anonymized, the following behavior occurs:

  • All of the personal data is masked or blanked out and access to the record is further restricted in the Network instance.

  • If records are exported to downstream systems, only record information and data that is not personally identifiable (created date, last modified date, VID, record state, and so on) can be viewed.

  • Users can no longer access the record in their Network instance.

For additional details, see the Anonymize job impact section below.

Enable the feature

Administrators must enable the feature in their Network instance.

  1. In the Admin console, click Settings > General Settings.

  2. In the Data Maintenance section, click the Anonymize records setting.

Selecting the feature automatically enables a new HCP field in the Network data model, is_anonymized__v. This field value is False on all records until a record has been anonymized using this data maintenance job.

Applicable records

Only deleted records (record state = Deleted) can be anonymized.

There are different processes for deleting records depending on record ownership:

  • Locally managed records - Deleted using the Delete Locally Managed Records data maintenance job.
  • Third party managed records - Deleted from Network by unsubscribing to third party records.
  • Veeva OpenData records - Deleted using the Unsubscribe from OpenData records data maintenance subscription. This subscription can be temporarily added to your Network instance upon request.

Anonymize job impact

After the records have been anonymized, they are no longer accessible in the Network instance. They can be exported to downstream systems using target subscriptions, but not using the Network API.

Review this list to understand the behavior to expect for anonymized records for the following features:

  • Record Status - The HCP and related sub-object records are set to Inactive.

    These fields ( hcp_status__v, address_status__v, license_status__v, and parent_hco_status__v) are retained on the record.

  • Search - Anonymized records cannot be found using Search.

  • Recent Items - If you have viewed an HCP record recently and then it is anonymized, when you try to access the HCP again by clicking Recent Items, the record will not display in the list; anonymized records are removed from Recent Items.

  • Tasks - Tasks (add and change requests and suspect matches) related to anonymized records cannot be viewed; tasks will not be available in My Requests, the inbox, task audit history, or by entering the task ID in the Network URL.

    Note: Pending DCRs for deleted records are closed in Network and the task status should be pushed to downstream systems (for example, Veeva CRM). There could be some cases where DCRs are not being closed in downstream systems, so to avoid disruptions within the CRM Data Subscription, DCRs for anonymized records are accessible through the Network API.

  • Profiles - Anonymized records cannot be viewed on the profile page. If you enter the Veeva ID of an anonymized record into the Network URL, the "locked" page with the "No entity found" message displays instead of the record. All related pages (for example, Revision History) are also unavailable for the record. Locally managed anonymized records cannot be reversed.

    For deleted Veeva OpenData records, if the Search OpenData option is enabled in your Network instance, the profile page of the record from OpenData displays and users can download the record again. If the record is downloaded again it will be considered a new record, so all previous custom values and custom sub-object fields will be empty.

  • Opted-out records - If an opted-out HCP record is included in an anonymization job, the anonymization behavior overrides the opted-out behavior in target subscriptions and the API. The data model fields (data_privacy_opt_out__c and data_privacy_opt_out__v) are set back to No Value.

    If you re-subscribe to a Veeva OpenData record, and the original record was opted out, the opt out behavior depends on which field was set to True:

    • data_privacy_opt_out__c - the opt out behavior is not restored.
    • data_privacy_opt_out__v - the opt out behavior is restored.
  • Data model fields - The is_anonymized__v field is set to True. Fields on the HCP record, sub-objects, and associated merge losers will be blanked out, masked, or retained.
    • Masked with "Anonymized Record" - All name fields (for example, formatted name, first name, last name, and alternate names) and address data (the Address Line 1 field).
    • Blanked out - All custom (__c) fields (except for alternate keys and checkbox fields) and standard (__v) fields that contain personally identifiable information (phones, email, and so on).
    • Retained - Record information and data that is not personally identifiable (for example, VID, created date, record state, record status, last modified date), custom alternate keys, and any checkbox fields.
  • Data Lineage - When a record is anonymized, the data lineage is unassociated from the record.
  • Merged records - Records that have been merged into the deleted HCP (merge losers) are also anonymized during the job. After the job runs, unmerge is no longer possible.
  • Reports - Anonymized records and related information (for example, DCRs, revision history, and so on) are not returned in reports. After the anonymize job runs, all data associated with the anonymized records are purged from the reporting database during the next database update.
  • Custom sub-objects - If an HCP is anonymized, any associated custom sub-objects will also be anonymized. Any data in custom sub-objects will also be masked, blanked-out, or retained.
  • Source subscriptions - If data are loaded that match existing anonymized records in your Network instance (based on the VID, values, or keys), it does not merge with the anonymized records.

  • Target subscriptions - If anonymized records are exported to downstream systems, only the fields that have been retained on the record can be viewed.

  • Network API - After records are anonymized, data cannot be retrieved using the API.

Impact on Veeva CRM

For details, see the Network blog article called GDPR Compliance with Veeva Network - Part 3: Managing HCPs that opt out from your organization.

Data model

To support this new feature, a new field, is_anonymized__v is added to the Network data model. The field is disabled by default, but will be enabled if the feature is turned on in your Network instance. The field value cannot be edited by users in the Network UI, it can only be updated through the data maintenance job.

Anonymize records

To anonymize records, administrators and data managers must create a .csv file containing the applicable Veeva IDs (VIDs). Locally managed and Veeva OpenData managed HCP records with a record state of Deleted can be anonymized.

Prerequisite

Ensure that deleted records have been exported and updated in downstream systems.

To create the data maintenance job:

  1. In the Admin console, click System Interfaces > Data Maintenance Subscriptions.
  2. Click Add Subscription.

  3. In the Add Subscription dialog, select Anonymize Deleted HCP Records.

    The New Anonymization Job page displays.

  4. In the Details section, type a Name and Description for the job.

    5. New subscriptions are Enabled by default. For more information, see Inactive subscriptions.

  5. In the Select Master Records section, define the file path and name of the .csv file.

    The .csv file should contain only a single column that contains the list of VIDs for the records that you want to anonymize.

    Note: A limit of 60,000 HCPs can be included in the .csv file. The job will fail if the limit is exceeded.

  6. In the Job Trigger Configuration section, define any subsequent actions that will start when this job finishes. For example, trigger a target subscription to export the anonymized records to a downstream system.

    Job Triggers - Trigger other actions to start after a job runs.

    Available triggers:

    • Send email - Specify users that should be notified for successful and unsuccessful job outcomes.
    • Start a job - Start a subsequent job when this job successfully completes.

    For more information, see Subscription job triggers.

  7. Save your changes.

Run the anonymization job

When the anonymization job runs, personally identifiable information on the record is masked or blanked and the field is_anonymized__v is set to True. This only occurs for the records in the .csv file that have a deleted record state. Any associated merge losers (the record state is merged_into) are also anonymized, even if they are not included in the .csv file.

To start the data maintenance job:

  1. In the data maintenance job that you just configured, click Start Job.

  2. When the job completes, a Job History section displays at the bottom of the page with some summary information about the job.

  3. Click the job ID to open the Job Details page.

  4. On the Job Details page, review the number and type of records that were anonymized during the job. The Job Error Log section identifies any records that could not be anonymized during the job.

    The Job Trigger Summary section identifies subsequent jobs or emails were triggered by this job.

    Only HCP records that have been deleted can be anonymized. Any records in the file that have other records states are ignored.

    Errors will occur for the following situations:

    • The record is an HCO.
    • The record state is anything other than deleted.
    • The sub-objects on the record are not in deleted state. This occurs if the sub-object was not deleted using the Delete Locally Managed Records job; they were deleted in a non-standard way.
    • The record has already been anonymized.

Next steps

After the HCP records have been anonymized, you can export the records to your downstream systems using target subscriptions.

In the target subscription, all personal data is masked with the Anonymized Record string or removed completely (blanked out). Only record information and data that is not personally identifiable are retained. If downstream systems accept the update, the anonymization can flow into the downstream system.

CRM considerations

Network records that have a DELETED record state are not updated in Veeva CRM or Vault CRM by default, so anonymized records do not flow to CRM.

Network Bridge

If you are using the Network Bridge, it is possible to update records in CRM even if the Network record state is DELETED.

To update the records in CRM so they are also anonymized, the following configurations must be set:

Network Bridge configuration

CRM configuration

When these options are set, records will be anonymized during the Network Bridge process.