Conditional matching

Filters can be applied to match rules to determine the records that you want to be considered during the matching process. Use the filters to include or exclude records based on the field values on the records.

For example, if you only want to match to active HCOs, the match process will not look at inactive HCOs during the job. The filters ensure that the match process only considers the records that comply with the filters.

Available filter methods

Filters can be applied to individual match rules or to the entire match configuration.

Applied to individual match rules

  • Filters - Determine the match pair by including or excluding values for the field for that specific match rule.

    • These conditions are created on rules on the Match Rules tab. They override any filters created on the Match Filters and Ranked Grouped Filters tabs.

Applied to match configuration

  • Match filters - General filters that are applied to all individual match rules for the defined object during the match process. The filters can include or exclude specific records from being considered for match pairs.

    If filters are defined on individual match rules for the same object, these rules are ignored. Match filters are applied before any defined ranked group filters.

  • Ranked group filters - A collection of filters to match incoming records to data in your instance. The filter groups are ranked so they are applied in a specific order.

    During the match process, ranked group filters run after the match filters.

These conditions can be applied to match rules for the same object in all match configurations:

  • Default Match Configuration

  • Ad Hoc Match Configuration

  • Add Request Match Configuration

  • Match Rule Collections

  • Source subscriptions

  • Data Deduplication data maintenance subscriptions

Determining the type of filters to use

If you want to only match to certain records, create filters on individual match rules or create Match Filters. If you want to work through different kinds of records in priority order, use Ranked Filter Groups.

Review the following diagram to help understand which type of filters to use.

Supported objects

All conditional filter methods can be created for Veeva standard object fields (HCP, HCO, license, address, and parent HCO) and on custom object fields.

Filters on individual match rules

You can use filters in match rules to include or exclude values from specific fields. For example, a license match rule can exclude DEA type licenses, or an HCO match rule can include a specific list of HCO types.

  • Include filter - Only records with that value in the specified field will be compared to each other using that rule.

  • Exclude filter - Records with that value in the specified field will be excluded from the rule.

The matching process uses exact match for defined filters, not similar match; for example, if the filter specifies John for a first name, the filter portion uses only exact matches for John.

Supported comparison methods

Filters are supported for match rules that rule the Direct Fields comparison method only. If the match rule uses any other comparison method, the filters are ignored.

Conditional matching can be used in the basic match UI or in advanced mode.

Basic match UI

In the example below, a filter is applied to the "licenses match" rule so licenses that have the status Inactive, Status Unknown, or Unknown are excluded from the match rule.

Advanced mode

In Advanced mode, the INCLUDE and EXCLUDE filter must be in all upper case. If the <type> line is omitted, the default is INCLUDE.

Limiting criteria

The following examples illustrate how to create a match rule so it only applies to records that meet the criteria.

This example matches corporate names but only for records where values in the HCO type field equal 4:1 or 4:15. 

<feature>
   <name>Corporate Names are the same for selected HCO types</name>
   <collate>
      <direct>
         <field>corporate_name__v</field>
         <filter>
            <field>hco_type__v</field>
            <value>4:1</value>
            <value>4:15</value>
            <type>INCLUDE</type>
         </filter>
         <nullMatching>IGNORE</nullMatching>
         <nGramComparison>
            <gramLength>2</gramLength>
            <threshold>0.7</threshold>
         </nGramComparison>
      </direct>
   </collate>
</feature>

The following example matches licenses where the type value is DEA:

<feature>
   <name>DEA licenses match</name>
   <collate>
      <set>
         <field>licenses__v</field>
         <filter>
            <field>type_value__v</field>
            <value>DEA</value>
            <type>INCLUDE</type>
         </filter>
         <setIntersectionComparison>
            <collate>
               <direct>
                  <field>license_number__v</field>
                  <equalComparison>
                     <compareAs>INTEGERS</compareAs>
                  </equalComparison>
               </direct>
            </collate>
         </setIntersectionComparison>
      </set>
   </collate>
</feature>

Excluding criteria

The following examples illustrate how to create a match rule to exclude records that meet the criteria. This example matches on the corporate name but only for records where the values in the HCO type field are not equal to 4:1 or 4:15. 

   <feature>
      <name>Corporate Names are the same for HCO types not equal 
            to 4:1 or 4:15</name>
      <collate>
         <direct>
            <field>corporate_name__v</field>
            <filter>
               <field>hco_type__v</field>
               <value>4:1</value>
               <value>4:15</value>
               <type>EXCLUDE</type>
            </filter>
            <nullMatching>IGNORE</nullMatching>
            <nGramComparison>
               <gramLength>2</gramLength>
               <threshold>0.7</threshold>
            </nGramComparison>
         </direct>
      </collate>
   </feature>

This example excludes Puerto Rican addresses within this address matching rule: 

<feature>
  <name>address matches but excludes Puerto Rico</name>
  <collate>
    <set>
      <field>addresses__v</field>
      <filter>
        <field>country__v</field>
        <value>PR</value>
        <type>EXCLUDE</type>
      </filter>
      <setIntersectionComparison>
        <collate>
          <direct>
            <field>premise__v</field>
            <field>thoroughfare__v</field>
            <field>locality__v</field>
            <jaroWinklerComparison>
              <threshold>0.80</threshold>
              <usingWinklerExtention>false</usingWinklerExtention>
              <usingLargeStringTolerance>false</usingLargeStringTolerance>
            </jaroWinklerComparison>
          </direct>
        </collate>
      </setIntersectionComparison>
    </set>
  </collate>
</feature>

Using multiple filters

More than one filter can be used in conditional matching. If multiple filters are used, they are all considered AND conditions, not OR conditions.

In the example below, two filters are applied. Records are considered if the HCO type field (hco_type__v) equals any of the specified values (for example, 4:15, 11:2) AND the specialty field (specialty_1__v ) does not equal IM or ACU.

<field>corporate_name__v</field> 
<filter>
<field>hco_type__v</field>
<value>4:15</value>
<value>11:2</value>
<value>11:3</value>
<value>13:1</value>
<value>13:3</value>
<type>INCLUDE</type>
</filter>
<filter>
<field>specialty_1__v</field>
<value>ACU</value>
<value>IM</value>
<type>EXCLUDE</type>
</filter>

Applying filtered match rules to incoming records

When filters are applied to individual match rules or all match rules, the rules can consider incoming records even if they are missing the field value of the filter.

Often, the incoming data is not as robust as the data in your instance, so records are skipped if they don't contain the filtered field and value.

Set record options

The options to allow incoming records with empty values to be considered for match display on the Match Rules tab.

It applies to any filters on individual match rules on the Match Rules tab and to filters defined on the Match Filters tab.

  • Apply match rules whether incoming records have a value in a filtered field or not.

  • Apply match rules only if incoming records have a value in a filtered field.

    This is the default for match rules because it is existing behavior.

To allow incoming records with empty or missing field values to be considered for match rules, select the first option on the Match Rules tab.

If the incoming data is robust and has values in filtered fields, the second option can be selected.

This is supported when the filter function is Include. Exclude functions require the records to have the specified field and value.