Disclaimer
In case you browsed directly to this page, let me state once again that I am a product manager and not a legal adviser. You should not rely on any information from this blog as legal advice. This blog is only based on my personal experience of talking to our legal team and various customers over the last four years, and it just represents my personal understanding of the GDPR. For any guidelines on how to comply with GDPR, you should always rely on legal advice from the legal counsel of your organization.
Options for managing HCPs that opt out from your organization
When an HCP uses their “right to be forgotten” under the GDPR against your organization, i.e. when an HCP opts out from your organization, then Veeva Network offers you two different options to manage this request.
You will probably now ask yourself why there are two options, and the answer is simple:
The first option already existed for quite some time before GDPR came around in order to support data privacy requirements in other regions of the world. However, when we launched our GDPR compliance program in the beginning of 2017 and started talking to customers about their view on GDPR requirements, the feedback that we have gotten was that this existing option was not sufficient for most customers. Instead, customers were demanding a more rigorous and extensive solution to address the “right to be forgotten”, which is why we came up with additional out-of-the-box functionality to make GDPR compliance easier for our customers.
And while the first option is used by several customers outside of Europe, in my experience the vast majority of customers (if not all) in Europe have chosen this second option to handle the use case when HCPs opt out from their organization. So, if you are a customer in Europe implementing a process to comply with the “right to be forgotten”, my recommendation is to focus your attention on the second option. But of course, I will also deep-dive into the first option to give you the full picture and to let you know what the differences are between both options.
One further note on HCOs before we proceed with detailing the options: The data privacy rights defined under the GDPR apply only to natural persons, not to organizations. Therefore everything in the following refers to HCP records and is not applicable to HCOs.
Option 1 - Data Privacy Opt-Out Flag
The first option consists of a field flag (data_privacy_opt_out__c) that you can set to “yes/true” in order to identify an HCP as opted out. Setting this flag to “yes/true” then drives certain behaviour of how that HCP record is treated in Network and CRM.
One important thing to note here is that this option applies in exactly the same way to OpenData-managed (“orange”) records, third-party-managed (“blue”) records as well as customer-managed (“grey”) records. This means, unlike the second option, there are no differences between orange, blue or grey records.You can set the data privacy flag on all records regardless of the record owner and the behavior will be the same.
In the following, I want to focus on the main impact that this flag has on the behaviour of the HCP record in Veeva Network and Veeva CRM if the data privacy flag is set to “true/yes”. Please note that the overview below is not entirely complete since there are more aspects to consider (for example, the behaviour in reporting, etc.). I would only like to focus here on the most important aspects in Network as well as CRM. For more details on this feature please refer to Data privacy opt out in the online help.
Impact on Network
If the data privacy flag is set to “yes/true”, the HCP record in Veeva Network behaves as follows:
- Record State - The record state remains VALID. The record is not soft-deleted from Network (“soft-deletion” would mean that the record state is set to INVALID or DELETED)
- HCP Status - The HCP status remains ACTIVE.
- Record Access - Whether or not the opted out record is accessible depends on the user’s Data Visibility Profile. If the setting “HCP Opt Out Visibility” is “True”, then the user can still access the record like any other record. Otherwise, the record is inaccessible to users; you cannot search for the HCP or view the profile page.
- Anonymization - The record is not anonymized at rest in the Network database. Users who have access to opted out HCPs can still see all personal information on the record. The record is only anonymized at transfer, i.e. the record is anonymized in the target subscription that exports the record to Veeva CRM. Anonymization means that all fields that contain personally identifiable information are either masked or blanked. System-managed fields like the Network Entity ID, the record state, and the last modified date retain their values. Please note that this anonymization at transfer is the default behaviour, but it is configurable. This means by setting the option “Unmask Customer Opt-out records” on the target subscription to “yes/true”, you can disable the anonymization during export.
- Reversibility - Opting out the HCP record is reversible. You can just set the data privacy flag back to “no/false” in case the HCP changes their mind and opts back into your organization.
Impact on CRM
If the data privacy flag is set to “yes/true” the HCP record in Veeva CRM behaves as follows:
- Account Status & Customer Master Status - The account status remains ACTIVE and the customer master status remains VALID.
- Territory Alignment - The account is not removed from territories.
- Anonymization - The HCP record is anonymized at rest in CRM (if the target subscription in Network was configured to mask the record). Anonymization means, here as well, that all fields which contain personally identifiable information (like name, email, address etc.) are either masked or blanked, while system-managed fields retain their values.
There are two important things to note here:
- The anonymization in CRM only applies to the account objects in CRM that are mastered by Network, but it does not apply to any transactional objects in CRM. I am highlighting this because the HCP name itself might be stamped on some transactional records in CRM and this name will not be masked by Network, since Network never touches any transactional data in CRM.
- The anonymization does not apply to custom fields. This means if you have for example any custom fields to store emails or phone numbers, you need to make sure to mask or blank them in CRM as well.
- Network Account Search - The HCP record is not searchable if it is flagged as an opt-out in Network.
- Data Change Requests - Any data change requests on opted-out HCPs will be auto-rejected. This means if a sales rep will try to revert the anonymization in CRM and try to add personal data back into the HCP record, then the system will auto-reject that request.
Option 2 - Unsubscribe / Soft-Delete and Anonymize
The second option consists mainly of a bunch of data maintenance jobs that you run for the respective HCP records. However, one important thing to note is that for option 2 the process slightly differs depending on who the record owner is. So, depending on whether you are dealing with an “orange”, “grey” or “blue” record, the steps to take will be different.
OpenData Records
The process for OpenData-managed HCP records is:
-
Step 1: Unsubscribe the record via the maintenance job called “Unsubscribe from OpenData Records”
Please note that this type of data maintenance job is not enabled by default in your instance. You need to create a support ticket to have it enabled, and in that ticket you need to specify which OpenData IDs you want to unsubscribe. After you have completed unsubscribing the respective records, our support team will disable the job again. This is an SOP that was implemented on request of the OpenData teams.
-
Step 2: Anonymize the record via the maintenance job called “Anonymize Deleted HCP Records”
Running the anonymization job is not mandatory, but it is highly recommended because the unsubscribe job will only soft-delete the record and make sure that the record is no longer updated by OpenData. But while soft-deleted records are not accessible via the search and profile page, the personal data of unsubscribed records is still in the database and still accessible through reporting.
The anonymization job will mask or blank any personally identifiable information at rest in the database and also remove any data of this record from reporting.
One thing to note here is that the anonymization job is only available for soft-deleted records and will not apply to valid records. This means you have to unsubscribe the record first before anonymizing it.
Customer-Managed Records
The first step for customer-managed records obviously needs to be different because you cannot unsubscribe from a customer-managed record. Therefore the process is:
-
Step 1: Soft-delete the record via the maintenance job called “Delete Locally Managed HCP/HCO”
Similar to the unsubscribe job for OpenData records, this job soft-deletes the record(the record state is set to DELETED).
-
Step 2: Anonymize the record via the maintenance job called “Anonymize Deleted HCP Records”
This is exactly the same process as for OpenData-managed records.
Third-Party-Managed Records
The process for third-party-managed records requires again an unsubscribe, however unlike for OpenData records there is no maintenance job for unsubscribing third-party-managed records. Instead there is a NEX function to unsubscribe “blue” records that you need to call within your third-party master subscription.
Please refer to Unsubscribing from third-party records for details about how to use this NEX function.
-
Step 1: Unsubscribe the record via the NEX function in the third-party master subscription
Similar to the unsubscribe job for OpenData records, the NEX function soft-deletes the record.This means the record state is set to DELETED.
-
Step 2: Anonymize the record via the maintenance job called “Anonymize Deleted HCP Records”
This is exactly the same process as for OpenData-managed records.
Impact on Network
If the HCP was unsubscribed / soft-deleted and anonymized, then the HCP record in Veeva Network behaves as follows:
- Record State - The record state is set to DELETED, i.e. the record is soft-deleted.
- HCP Status - The HCP status is set to INACTIVE.
- Record Access - The record is not accessible via the application (i.e. search, profile page, reporting, etc.). The only way to access a deleted and anonymized record is via a target subscription. This is required to update downstream systems accordingly.
- Anonymization - The record is anonymized at rest in the Network database. All fields that store personally identifiable information are either masked or blanked. This includes custom fields as well, unless their field type is “Alternate Key” or “Checkbox”.
- Reversibility - The questions of whether the opt out is reversible or not depends again on the record ownership:
- If an “orange” record was unsubscribed, there is an option to re-subscribe the record from OpenData in case the HCP changes their mind and opts back into your organization. However, you probably want to control who in your organization should have the permission to re-subscribe an HCP and who should not. I have dedicated another blog post to just this topic, which I highly recommend to read as well. For information on how to do this, see Filter unsubscribed HCPs from Search against OpenData.
- If a “grey” record was soft-deleted, then this record stays soft-deleted permanently, i.e. there is no way to bring back this same record with the same Network Entity ID. Of course, you can recreate the same “grey” HCP as a new record and with a new Network Entity ID from the original source, but the original record that was deleted remains soft-deleted permanently.
- If a “blue” record was unsubscribed and deleted, then this record also stays soft-deleted permanently. Of course, you can re-subscribe the record from the third-party in the sense that they send you the record again in the third-party master subscription, but that will also create a new record with a new Network Entity ID. The record that was originally unsubscribed remains soft-deleted.
Impact on CRM
If the HCP was unsubscribed / soft-deleted and anonymized, then the HCP record in Veeva CRM behaves as follows:
- Account Status & Customer Master Status - The account status is set to INACTIVE and the customer master status is set to REJECTED
- Territory Alignment - The account is removed from territories and the integration user becomes the account owner (this is the default behaviour in CRM for soft-deleted records).
- Anonymization - The HCP record is anonymized at rest in CRM if the “Enhanced Inactive Record Sync” is enabled. See Anonymize HCP records for more details.
- Network Account Search - The questions of whether the opted out HCP is searchable via Network Account Search again depends on the the record owner:
- “Orange” records are searchable, if “Search against Master” is enabled and if the integration user has the permission to search and download unsubscribed HCPs. A best practice is to deny sales reps the permission to search and download HCPs that were unsubscribed for data privacy reasons. For information on how to do this, see Filter unsubscribed HCPs from Search against OpenData.
- “Grey” and “blue” records are not searchable since they were soft-deleted in Network.
- Data Change Requests - Unsubscribed or soft-deleted records are not accessible to sales reps in CRM, so there is no issue with change requests being submitted on those records. However, when sales reps cannot find the HCP via search, then they will most likely create an add request as their next action. To learn more about how add requests are handled for unsubscribed orange records, see Add requests for unsubscribed HCPs.
Summary
Okay, I admit this was a lot of information to take in. Therefore, let’s briefly summarize the most important points:
For Option 2 the process slightly differs depending on the record ownership:
Here is an overview that summarize the impact of option 1 and option 2 on the record behaviour in Network:
And this is an overview that summarizes the impact of option 1 and option 2 on the record behaviour in CRM:
Note on Hard-Deleting Records
When discussing the “right to be forgotten” and the available options in Veeva Network with customers, usually the question about hard-deleting the data also comes up. And it is not a surprise that the opinions differ widely here. On the one side of the spectrum, there is the faction that says something like “our data privacy officer told us that we must hard-delete the record, because the regulation says so”. And on the other end of the spectrum, I have also heard customers saying the complete opposite: “Our data privacy officer said we must not hard-delete any records, because if we hard-delete everything we cannot provide any evidence to the regulatory authorities anymore”.
I have to admit that this question is a little tricky indeed, because on the one hand the GDPR requires you to “erase” the data as it is phrased in Article 17 of the GDPR (“Right to erasure”). But on the other hand, if you truly erase every information from your harddisks, then this creates several other issues, like for example the lack of evidence.
If you have read this blog post carefully, you will have noticed that when I write about “deleting” records in Network then that always means “soft-deleting” the record. Actually, there is no option in Network that supports the hard-deletion of HCP records and this is by design. We have deliberately decided against an option to hard-delete HCP records for the following reasons:
- As an enterprise MDM system, Veeva Network is tightly integrated with downstream systems that consume the HCP data. While it may be safe to hard-delete all HCP data from Veeva CRM or a data warehouse, it would cause issues if we would just hard-delete the data in the MDM system. Because if we would hard-delete the record in Network, then there would be no way to notify downstream systems about the deletion and anonymization of the record.
- Secondly, a hard-deletion would also mean that you lose the complete audit trail. If everything that you ever stored about an HCP is permanently deleted, then how would you be able to provide evidence to the HCP or a regulatory authority that you have processed the “right to be forgotten” request correctly?
For these reasons instead of hard-deleting the data we have chosen a different approach, which is the complete and irreversible anonymization of the personally identifiable information, while keeping the record itself as a soft-deleted record. And I think this is the right approach, because on the one hand, as far as I know, the GDPR does not mention the term “hard-deletion” anywhere, but it mentions data anonymization as an adequate technical measure to ensure proper data privacy. And on the other hand, many conversations with customers, as well as with our legal team, have confirmed that our approach is suited well to meet the requirements of the GDPR.