Docker standalone

The simplest way to create a standalone Docker deployment for self-managed servers is by running Flink with Docker Compose as shown in How to run HERE Anonymizer Preprocessor locally in Docker.

As HERE Anonymizer Preprocessor is an Apache Flink application, the basic running pattern is running a single JobManager with one or more TaskManagers. The JobManager is the main point of configuration, where all the configuration parameters should be set.

In most cases, the TaskManager should be configured through JobManager address. However, depending on the authentication methods used when connecting to source and sink URIs, TaskManager nodes might expect to have credentials files, certificate files, and other files properly mounted (corresponding to paths of SINK_URI and SOURCE_URI of JobManager configuration).

This is a sample configuration for a JobManager and a TaskManager in a standalone Docker deployment:

indexer-jobmanager:
  image: here-anonymizer-preprocessor:latest
  # ...
  command: index
  environment:
    - ANONYMIZATION_CONFIG_FILE=/opt/anonymization.conf
    - SOURCE_URI=rabbit://user:pass@host:5672/input?keyFile=/cert/client.pem&crtFile=/cert/client.crt&caFile=/cert/ca.pem
    - SOURCE_FORMAT=HERE_PROBE
    - SINK_URI=rabbit://user:pass@host:5672/input?keyFile=/cert/client.pem&crtFile=/cert/client.crt&caFile=/cert/ca.pem
    - SINK_FORMAT=HERE_PROBE
    - |
      FLINK_PROPERTIES=
      jobmanager.rpc.address: jobmanager
      parallelism.default: 1

indexer-taskmanager:
  image: here-anonymizer-preprocessor:latest
  # ...
  volumes:
    - /home/me/here-anon/certs-for-rabbit:/cert
  environment:
    - JOB_MANAGER_RPC_ADDRESS=indexer-jobmanager
    - TASK_MANAGER_NUMBER_OF_TASK_SLOTS=1
    - |
      FLINK_PROPERTIES=
      parallelism.default: 1

preprocessor-jobmanager:
  image: here-anonymizer-preprocessor:latest
  # ...
  command: preprocess
  environment:
    - ANONYMIZATION_CONFIG_FILE=/opt/anonymization.conf
    - SOURCE_URI=rabbit://user:pass@host:5672/input?keyFile=/cert/client.pem&crtFile=/cert/client.crt&caFile=/cert/ca.pem
    - SOURCE_FORMAT=HERE_PROBE
    - SINK_URI=rabbit://user:pass@host:5672/input?keyFile=/cert/client.pem&crtFile=/cert/client.crt&caFile=/cert/ca.pem
    - SINK_FORMAT=HERE_PROBE
    - |
      FLINK_PROPERTIES=
      jobmanager.rpc.address: jobmanager
      parallelism.default: 1

preprocessor-taskmanager:
  image: here-anonymizer:latest
  # ...
  volumes:
    - /home/me/here-anon/certs-for-rabbit:/cert
  environment:
    - JOB_MANAGER_RPC_ADDRESS=preprocessor-jobmanager
    - TASK_MANAGER_NUMBER_OF_TASK_SLOTS=1
    - |
      FLINK_PROPERTIES=
      parallelism.default: 1