Veeva OpenData AMA subscription
Beginning January 1, 2022, US Veeva OpenData subscriptions will no longer include American Medical Association (AMA) data. Currently, this data is stored in a list of data model fields on US records. Those fields have been grouped together to form an AMA data subscription which controls the behavior of the fields depending on your agreement with the AMA.
November 19, 2021 - AMA subscriptions are available in Network instances that have US OpenData subscriptions enabled. The subscription was included in the 21R2.1.3 release. It is enabled by default.
If you will receive AMA data from a different data source, you can begin testing the subscription. Disable the subscription so the Veeva AMA fields become locally managed, and create a source subscription to populate those fields.
December 15, 2021 - If you will continue to receive AMA data from a different data source, disable the AMA subscription by this date so the existing data is retained in the AMA fields. If Veeva is your Database Licensee (DBL), OpenData will disable the subscription for you if it is still active when they begin the process to remove the data.
December 16-31, 2021 - After December 15, Veeva OpenData will begin removing the contents of the AMA fields. The data will be completely removed by December 31, 2021. If the AMA subscription was disabled prior to this action, the data will be retained.
January 5, 2022 - If you canceled your agreement to receive AMA data from a Database Licensee (DBL), send confirmation letter of the data removal to the DBL. Contact your Veeva OpenData CSM for more details.
When the 21R2.1.3 release is implemented on November 19, 2021, the AMA subscription is initially enabled by default if you have a US OpenData subscription enabled in your Network instance.
After the release, the AMA subscription should remain enabled or be disabled depending on your agreement with the AMA.
Your AMA agreement status
Review the table to understand what actions are required, depending on your AMA agreement status.
You will no longer receive AMA data.
You will receive AMA data from another data licensee.
|Network Action Required||
The Include AMA Data subscription should remain enabled.
Disable the Include AMA data subscription by December 15, 2021.
|AMA subscription behavior||
The fields will continue to be OpenData fields.
The existing data will be emptied from the fields by December 31, 2021.
The OpenData managed fields will become locally managed fields.
The existing data in the fields will be retained if the subscription is disabled by December 15, 2021.
|Customer action required||Provide confirmation letter to Database Licensee (DBL) by January 5, 2022.||No action required.|
|Impact to downstream systems||
No field mapping updates are required.
Data in the fields is emptied by OpenData and those updates are sent to downstream systems, as usual.
No field mappings are required.
Updates that you make to the data are sent to downstream systems, as usual.
Enabled subscription behavior
The Include AMA data subscription should remain enabled if you canceled your agreement with the AMA because you will no longer receive AMA data from any data source.
The following behavior occurs to the Veeva managed AMA fields and data:
Veeva OpenData continues to manage the AMA fields.
In mid-December, OpenData will remove the existing AMA data from the fields, per their agreement with the AMA. You will no longer have access to the data in the fields.
You can choose to disable the AMA fields in your Network instance. When the fields are disabled, they no longer display on record profiles and cannot be included in data change requests.
Disabled subscription behavior
If you will receive AMA data from another data provider, you should disable the Include AMA data subscription by December 15, 2021. If Veeva is your DBL, OpenData will disable the subscription for you if it is still active when they begin the process to remove the data.
The following behavior occurs to the Veeva managed AMA fields and data when the subscription is disabled:
The AMA fields become locally managed. You now own and manage the fields.
If the subscription is disabled by December 15, 2021, the existing AMA data remains in the fields for you to own and manage.
If the subscription is not disabled before that date, Veeva OpenData will begin removing the existing data. You will continue to own and manage the fields, but you will need to load all AMA data again from another data provider.
The AMA field names will not change; for example, the ME ID field remains
me_id__v. No field mappings need to be adjusted. No new custom fields need to be added.
If data change requests are submitted on AMA fields, the changes will be sent to your local data stewards.
Tip: You might add or update your internal process documents regarding how DCRs on those fields are managed.
AMA data considerations for OpenData records
The AMA fields remain on Veeva OpenData records, but they are now locally managed. You will need to load AMA data from another data provider if you want data in those fields on OpenData records.
If the existing OpenData AMA data is retained in the fields, keep it up-to-date using data from your new AMA data provider.
Ensure that you load new data into those fields on OpenData records.
Disable the AMA subscription
To disable the subscription:
In the Field Level Subscriptions section on your US Veeva OpenData subscription, clear the checkbox beside Include AMA Data.
Fields included in the subscription
The following HCP fields are exclusive to AMA data and will no longer be managed by OpenData if you disable the AMA subscription.
If you have canceled your AMA agreement, the fields can also be disabled.
|HCP Field Name||Localized UI Label||Field Type|
|Alternate First Name||String|
|2||alternate_last_name__v||Alternate Last Name||String|
|3||alternate_middle_name__v||Alternate Middle Name||String|
|4||ama_do_not_contact__v||AMA Do Not Contact?||Reference|
|10||grad_trg_end_date__v||Grad Training End Date||Date|
|11||grad_trg_start_date__v||Grad Training Start Date||Date|
|13||mpa__v||Major Professional Activity||Reference|
|14||pdrp_optout__v||PDRP Opt Out?||Reference|
|15||pdrp_optout_date__v||PDRP Opt Out Date||Reference|
|16||place_of_employment__v||Place of Employment||Reference|
|17||type_of_practice__v||Type of Practice||Reference|
|18||years_in_progress__v||Years in Progress||Number|
Loading AMA data from other data sources
If you are using a new data provider to receive AMA data, you can load the data into Network through a source subscription.
Options for loading AMA data:
Existing subscription - Augment an existing subscription to use fields that already exist on the files from that source.
Source subscriptions that already add data from that source to OpenData records into custom fields should not require updates to match rules but this should be reviewed to ensure that updates and new data is added.
Clone a subscription - Make a copy of an existing subscription that uses this source. This enables you to leverage some of the subscription configuration and just add new files and fields.
New subscription - Create a new subscription to load this data from files provided by your new AMA data provider.
Note: If you receive raw AMA fields from your new data licensee (DBL), data transformation work is required to create files that can be loaded into Network. Then, use a new subscription to load the data into the locally managed fields. See the “Transforming raw AMA data files” section below.
Configure a source subscription
Create the subscription configuration so you can load your new data files.
- In the Admin console, click System Interfaces > Source Subscriptions.
Click Add Subscription > Use Wizard Mode.
When you use the wizard, you add a sample file for your data. The file can have no more than 50 rows.
On Step 1 - Define Subscription, provide the following details:
Subscription name - Type a meaningful name for this subscription.
Third party data - Confirm that the file does contain third party data and that you have an agreement. You must have an agreement with the AMA to load the data to Network.
Source system - Choose the system that you have created for the data source.
If you haven't yet created a system, go to System Interfaces > Systems to add the data source.
Countries - Add United States as the country. AMA data applies only to the United States.
Description - Type a meaningful description for this subscription.
Upload Sample Files - Add the AMA data file to the subscription. The file will be validated. If there are any warnings or errors, fix them and upload the file again.
Click Save & Continue.
- On Step 2 - Define File Format, define the following settings:
Filename prefix on the FTP server - The prefix of the file that you will regularly use for this subscription. For example, the file name might change to indicate a date but the beginning of the file name will always be the same.
The field is automatically populated with the prefix of the file that you loaded. Edit the prefix if this is not correct for future files.
File Format - Indicate how the fields in the file are separated. Choose Fixed Length or Delimited. Delimited is selected by default.
If you choose Fixed Length, type the allowed character lengths for each column into the Fixed Length Widths field.
- If you choose Delimited, Network tries to identify how the file is formatted. Ensure that the following settings are correct:
- Delimiter - Identify how the data in the file is separated.
- File Text Qualifier - The characters that identify separate fields.
Note: If you uploaded an .xlsx file, these fields do not display. Network automatically identifies the format.
Headers - Confirm that your file includes headers. Your file should use the Network field names as column headers.
A preview of the file displays in the format that you identified.
On Step 3 - Define objects, expand the list and choose Health Care Professional (HCP).
There are no other objects in the file, so there is no need to define criteria for loading records.
Click Save & Continue.
On Step 4 - Select Keys, identify the primary key and configure the custom key for the HCP object.
Primary key - Choose the field that will be the primary key for the HCP. For example, ama_source_id.
Custom Key - Custom keys are based on the system name, object, and external ID; for example, ama_system:HCP:0540417133.
Source - The source system for this subscription. The system that you selected for this subscription is the default value. You can choose any system that is defined in your Network instance or choose Plain Text to type a source; for example, AMA.
Item - A value distinguishes this key from other keys. Objects are typically used for the Item value. You can also click Plain Text and type a unique value.
Value - Typically, the custom key value is an external ID. Expand the Columns List to choose any of the columns in this file. Alternately, select Expression to use a Network Expression to create the value. The syntax is not validated here but is validated when the subscription runs.
Click Save & Continue.
On Step 5 - Map Fields, map the fields from your source file to Network fields.
In some cases, fields in your file are automatically mapped:
- If the field names in your file match Network fields exactly, the Network fields are automatically populated.
- Network identifies the fields that you have selected as the primary key. A Primary Key icon displays beside the field so you know that it does not need to be mapped.
- Click Generate Subscription. The subscription opens in Classic Mode so you can review the model map and field normalization, define match rules, and make any other changes.
Consider the following for the Match Configuration section:
Key Matching - If you are using data from an existing source and they’ve provided keys that match keys that you have already loaded into Network for HCPs by that source, this incoming data will match using those keys automatically. The default match rules are not required.
Match on NPI - US default match rules already match on the NPI number.
As long as your incoming data has an NPI value, you can use that for matching purposes for new records.
Match on ME ID - You can also match on ME ID but that will only maintain records that already have AMA data loaded, including ME ID.
Since ME ID is one of the data elements that you need to maintain yourself, you need to find another element (custom key, NPI) to match with. Once a match is found, a new or updated ME ID can then be associated to records.
- Add the source files to the FTP server so the subscription can retrieve them when it runs.
If you receive raw data files, they must be transformed before they can be loaded into Network. After you develop the transformation, it can be used each time you receive updated files.
Supported data files
There are many raw AMA data files; this information focuses on content in the weekly PPD delta file only. AMA provides two versions of this weekly file: a before file and an after file.
This information only covers updating or adding data to OpenData or locally managed records for the 18 fields that will be transformed from the PPD file.
Note: You can use all of the 18 HCP fields or you can choose to use fewer fields.
Not covered in this document
This information does not cover the following files from AMA:
Student file - Adding and updating student records.
Dead file - Updates for HCPs who are deceased.
Quarterly file - A file that AMA provides each quarter that contains all updates.
Transforming the PPD file data
The raw data in the PPD file must be transformed before it can be loaded into Network. This is a process that customers will develop internally. Once the transform has been created, it can be used weekly each time you receive the PPD delta file.
When you build the transformation, it should include the following elements:
File headers - The AMA PPD file does not contain headers. The following headers are based on the file layout provided by the AMA.
Use the following headers so the columns can be mapped to columns with Network data model names.
Field mapping - Map the AMA field names to the 18 Network field names.
Only these 18 fields are required for updating the AMA data in Network. You can use all 18 fields or use a fewer number of fields.
When this field mapping is done as part of the transformation, no additional field mapping is required (for example, no mapping is required in the source subscription).
AMA Raw File Field Name Network Field Name 1 MOST_RECENT_FORMER_FIRST_NAME alternate_first_name__v 2 MOST_RECENT_FORMER_LAST_NAME alternate_last_name__v 3 MOST_RECENT_FORMER_MIDDLE_NAME alternate_middle_name__v 4 CONTACT_INDICATOR ama_do_not_contact__v 5 BIRTH_COUNTRY birth_country__v 6 This is a calculated field based on INSTITUTION_ID and internal logic education_level__v 7 SPECIALTY_TRAINING_TYPE fellow__v 8 GRAD_MEDICAL_SCHOOL_STATE+GRAD_MEDICAL_SCHOOL_ID grad_school__v 9 CONFIRMATION_FLAG grad_training__v 10 TO_DATE grad_trg_end_date__v 11 FROM_DATE grad_trg_start_date__v 12 ME_NUMBER me_id__v 13 MAJ_PROF_ACTIVITY_PRIM mpa__v 14 RX_RESTRICTION_IND pdrp_optout__v 15 RX_RESTRICTION_START_DATE pdrp_optout_date__v 16 PRIM_PE place_of_employment__v 17 PRIM_TOP type_of_practice__v 18 YEAR_IN_PROGRAM years_in_progress__v
File - Create a file called HCP.csv. This file will contain only the 18 fields for the transformed data. It is the only file to use in the source subscription to load this data.
Changes are required for some of the fields during the transform.
ME_NUMBER (me_id__v) - The ME_NUMBER provided by the AMA is 11 characters. The first 10 characters are the true ME_NUMBER and the 11th character is a check number.
UPDATE_TYPE - This column is used for the AMA Weekly Delta PPD file to indicate versions of the record. There are two versions of the delta file sent weekly; a before version and an after version.
When UPDATE_TYPE = 1, then the record is the previous version of the data.
When UPDATE_TYPE <> 1, then the record is the new version of the data and should be loaded.
CONTACT_INDICATOR (ama_do_not_contact__v) - This field can contain the following values: Y, N.
The values have opposite meanings between the AMA field and the Network field because of the name of the fields.
AMA PPD Field
AMA DO NOT CONTACT?
Y (Contact is allowed) N (Contact is allowed) N (Contact is not allowed) Y (Contact is not allowed)
- RX_RESTRICTION_IND (pdrp_opt_out__v) - This field can contain the following values: Y, N.
The values have the same meaning between the AMA field and the Network field: Y = Y and N =N.
CONFIRMATION_FLAG (grad_training__v) - This field can contain the following values: Y, N.
AMA PPD Field
(blank value) N Y Y
PRIM_TOP (type_of_practice__v) - This is a lookup field.
The AMA values map to the following Network reference codes:
AMA Value Network Code English Description 012 20 Resident 020 4 Direct Patient Care 030 40 Administration 040 1 Medical Teaching 050 21 Medical Research 062 50 Non-Patient Care 070 90 Inactive 071 98 Retired 072 97 Semi-Retired 073 96 Disabled 074 95 Temporarily Not In Practice 075 90 Inactive 100 99 Not Classified
- MAJ_PROF_ACTIVITY_PRIM (mpa__v) - This is a lookup field.
The AMA values map to the following Network reference codes:
AMA Value Network Code English Description ADM 40 Administration HPP 30 Full-Time Hospital Staff HPR 20 Resident INA 90 Inactive LOC 5 Locum Tenens MTC 1 Medical Teaching NCL 99 Not Classified OFF 4 Office Based Practice OTH 80 Other RES 21 Research SRT 89 Semi-Retired
PRIM_PE (place_of_employment__v) - This is a lookup field.
The AMA values map to the following Network reference codes:
AMA Value Network Code English Description 011 2 Self-Employed Solo Practice 013 6 Group Practice 014 6 Group Practice 021 4 Other-Patient Care 022 5 Locum Tenens 030 6 Group Practice 035 7 HMO 040 1 Medical School 050 30 Non-Government Hospital 060 99 Not Classified 063 31 City/Country/State Government Hospital 064 32 City/Country/State Other Than Hospital 080 99 Not Classified 081 33 Federal Government-Hospital Army 082 34 Federal Government-Hospital Navy 083 35 Federal Government-Hospital Air Force 084 36 Federal Government-Hospital USPHS 085 37 Veteran Affairs 086 38 Other Federal Agency 101 50 Other-Non-Patient Care 110 99 Not Classified
Data loading considerations
There are other considerations after transforming the AMA data and producing an HCP file. Additional work might be required to load the data into Network.
Matching data to existing records
The HCP.csv file will contain only the 18 fields. Network's default match rules will match on the ME ID field. If each row in the newly created HCP file has a value in that field, each incoming row should match using the ME ID. This will only match and update records with existing data.
Loading data to new records
Existing records can be updated by matching data. Customers must determine how to load this data into new records.
Source subscription requirements
When you create the source subscription, you'll need to specify the following:
HCP custom keys
These need to be specified when you load any data into Network.
If you have questions about transforming the data, contact your Veeva OpenData Customer Service Manager (CSM).