Data storage options in Network
Network provides a standard data model for customer master data, but you can also add and store other data domains, such as product master, payer master, or employee master, as well as supplementary data.
It's important to understand the best method to store any data that you add to Network. Consider what kind of data it is and how it will be used. It could be data that is independent of Network's standard data model (for example; employee data, or clinic trial data) that requires Network's master data management features; for example, search and data stewardship. Or, or it could be data that will be used to supplement or validate Network data that only requires the ability to report against it.
Review the following options to understand the best feature to use for storing different kinds of data:
-
Custom objects - Entities that are essential to your business that are not part of the standard Network data model.
Use for data that will be loaded into Network and require Network's MDM features like search, data lineage, revision history, and change requests.
Example data: Product data, clinical trial data, employee data, and payer and plan data.
Any data that is not considered "master data" (for example, transactional data) or any data that does not require Network's MDM features (search, DCRS, data lineage, revision history, and so on, must not be stored in a custom object.
See Custom objects for more information.
-
Custom tables - A lightweight storage option for data that you can query in Network reports.
Use for data that should be stored in Network but does not require the overhead of Network's MDM features. For example, load data into custom tables that you can join to master data and then run queries against the data.
Custom tables are available to all advanced reporting users and are not restricted by file sizes, so they can be used as an alternative to lookup tables if you do not need to reference the data in the NEX lookup functions.
Example data: Sales or transactional data, staging data, temporary data, and auxiliary data.
See Custom tables for more information.
-
Lookup tables - Lookup tables that are created by uploading .csv files. They can be referenced through NEX rules and SQL queries can be run against the tables.
Lookup tables are available for Administrators and Data Stewards. Files over 1GB or 5 million rows are not supported.
Use this data to supplemental additional information or calculate field values.
Example data: Address data to validate changes to State/Province, City, and Postal Code combinations, or specialty exclusion matrix.
Note: Lookup tables are available for Administrators and Data Managers. Files over 1GB or 5 million rows are not supported.
See Lookup tables for more information.
Network features
Use the following matrix to compare the Network features available for custom objects, custom tables, and lookup tables.
| Custom Object | Custom Table | Lookup Table | |
|---|---|---|---|
| Loading data (source subscriptions) | Yes | Yes (through transformation queries) | Yes (through transformation queries) |
| Exporting data (target subscriptions) | Yes | Yes (through transformation queries) | Yes (through transformation queries) |
| Data Access (Network API) | Yes | Yes (through Bulk Export API) | Yes (through Bulk Export API) |
| Search | Yes | No | No |
| DCRs | Yes | No | No |
| Profile page | Yes | No | No |
| Data Lineage | Yes | No | No |
| Revision history | Yes | No | No |
| Used in NEX Lookup functions | No | No | Yes |
Checklist
To make the right decision for choosing between these different data storage options, answering the following questions can help:
-
Does the data have to be searchable in the Network user interface?
-
Does the data require stewardship and data change requests?
-
Does the data require data lineage?
-
Does the data require revision history?
If the answer to all of these questions is "No", you should not choose a custom object; choose either a custom table or a lookup table.
Custom table or lookup table?
To make the decision between a custom table and a lookup table, you should consider whether you need to use the data in a NEX lookup function or not. If data access through a NEX lookup function is not required, you should use a custom table.