Monitoring and metrics in HERE Anonymizer Preprocessor
Apache Flink metrics
You can access an Apache Flink dashboard at the IP address mapped to jobmanager-ui. For example, you can access the dashboard of a local instance at http://localhost:8081/.
This browser-accessible dashboard offers a large quantity of built-in Flink and Java VM metrics.
See Monitoring Apache Flink Applications for a detailed explanation of existing counters and what the values of these counters can tell you about the health of your Flink application.
Indexer metrics
HERE Anonymizer Preprocessor registers its counters using Flink's metrics system.
The following counters are available at the Flink dashboard when the Indexer is running:
| Processing step | Metric name | Data object | Description |
|---|---|---|---|
| Indexing | HERE_indexer_input_files_total | File | Number of files received and successfully processed. |
| Indexing | HERE_indexer_input_files_dropped_corrupted | File | Number of files dropped due to failure in decoding to configured data format. |
Preprocessor metrics
HERE Anonymizer Preprocessor registers its counters using Flink's metrics system.
The following counters are available at the Flink dashboard when the Preprocessor is running:
| Processing step | Metric name | Data object | Description |
|---|---|---|---|
| Decoding | HERE_decoding_message_dropped_corrupted | Message | Number of received messages that could not be decoded |
| Decoding | HERE_decoding_point_info_all | Point | Number of points received and successfully decoded |
| Decoding | HERE_decoding_chunk_info_all | Chunk | Number of chunks received and successfully decoded |
| Decoding | HERE_decoding_message_info_all | Message | Number of messages received and successfully decoded |
| Decoding | HERE_preprocessor_input_points_total | Point | Number of files received and successfully processed. |
| Decoding | HERE_preprocessor_input_traces_total | Trace | Number of Traces received and successfully processed. |
| Output | HERE_output_point_info_all | Point | Number of points output by the pipeline |
| Output | HERE_output_chunk_info_all | Chunk | Number of chunks output by the pipeline |
| Output | HERE_output_message_info_all | Message | Number of messages output by the pipeline |
| Output | HERE_preprocessor_output_point_total | Point | Number of points output by the pipeline |
| Output | HERE_preprocessor_output_traces_total | Trace | Number of traces output by the pipeline |
| Output | HERE_preprocessor_output_files_total | File | Number of files output by the pipeline |
AWS metrics
AWS EKS provides a metrics API which offers a set of metrics in the Prometheus format. This means that you can install a Prometheus instance in your Kubernetes cluster and further process the metrics data or watch it using the Prometheus UI.
Additionally, AWS EKS offers a dashboard which shows Kubernetes-level data.
To learn more, see Metrics for Amazon EKS and Kubernetes.
Azure metrics
In Azure, you can use the Azure Monitor to monitor the health and performance of your AKS cluster. The tool offers a variety of charts and counters as well as a Prometheus API.
To learn more, see Monitor AKS with Azure Monitor.
Updated 16 days ago