How to use validation rules for indexing attributes

In order for a message to be successfully indexed, its indexing attributes need to be valid according to the indexing attributes defined for the index layer. The Data Archiving Library validates the indexing attributes of each message from a User-Defined Function to make sure the index attributes are valid for the target index layer.

If the indexing attributes of a message fail validation, the configured error handling strategy is invoked. For more information, see Configure error handling.

Validation rules

For every message, a User-Defined Function returns indexing attributes (a key-value map). Depending on the type of User-Defined Function, an indexing attribute can have a single value or a collection of values. The same validation rules apply to every value in the collection of values.

Using the index definitions of the index layer, each indexing attribute value is validated as follows:

1. If the index definition type is timewindow, the indexing attribute value must be long parsable (epoch time in milliseconds).
   1. If the value is non-parsable long, then the message is considered invalid.
   2. If the value is null, then the message is considered invalid.
   3. For all other values, the message is considered valid. The value is truncated according to the duration configuration.
2. If the index definition type is heretile, the indexing attribute value must be long parsable (a HERE Tile ID).
   1. If the value is non-parsable long, then the message is considered invalid.
   2. If the value does not match with the zoomLevel in the index definition, then the message is considered invalid.
   3. If the value is null, then the message is considered valid.
   4. For all other values, the message is considered invalid.
3. For all other index definition types (int, bool, string):
   1. If the value is invalid (for example, failed the instanceOf check), then the message is considered invalid.
   2. If the value is null, then the message is considered valid.
   3. For all other values, the message is considered valid.

Note

If the indexing attributes (key-value map) do not contain any index definition, then the pipeline will fail without considering the error handling strategy.