How to continously read data using Flink

Flink can read data continuously from non-stream layers. This means that during the runtime of the Flink execution environment the regular stream layer processing is ongoing and if there is a change in the data of a non-stream layer, the data is read and injected into the processing. This gives the functionality to not just read such layers once at the startup of the Flink execution environment but during runtime as well.

The behavior slightly differs depending on layer type like this:

• Versioned layer:

  For versioned layer this function is enabled by default. It uses layer notifications. Therefore it does not need to poll but gets notified if a new version is published. To disable this function just add the property olp.connector-refresh-interval with value -1 to your environment or application configuration.

  Note: For versioned layer this feature works only for catalogs with notificationsEnabled = true in catalog configuration.

• Volatile, Index and Interactive Map layers:

  For these layers you can define the refresh (polling) interval in milliseconds by adding the property olp.connector-refresh-interval to your environment or application configuration. By default, this interval is 60000L meaning 60 seconds.