Current Usage Metrics is a default dashboard available to you in Grafana. The following metrics are available for you to monitor your current usage of the HERE platform. The metric in the single-stat panel to the right of the dashboard shows the current value within the last one hour, while the graph panel to the left shows the time series presentation of this value.
| METRIC | DETAILS | DESCRIPTION |
|---|
| Versioned | layer_used_bytes with layer_type="versioned" and type="data"
storage_used_bytes with storage_type="stream_durable_volume" | Storage of versioned controlled, Geo-Indexed data |
| Volatile | layer_allocated_bytes with layer_type="volatile" and type="data"
storage_allocated_bytes | Storage in volatile memory for more performance with small writes and reads |
| Stream | <code>in_throughput_kbps|out_throughput_kbps</code> with layer_type="stream" | Capacity allocated to queue Stream data |
| Stream TTL | ttl_kb_allocated with layer_type="stream" | Capacity allocated to store long lived streaming jobs. This covers the EBS volumes associated with the Stream Nodes |
| MetaData | catalog_used_metadata_bytes with type="metadata" | Storage of meta-data about the data you store |
| METRIC | DETAILS | DESCRIPTION |
|---|
| Log Search IO | splunk_index_usage_bytes_previous_hour with operation="splunk_index" | Storage of indexed log data for debugging search |
| Pipeline IO | This metric is computed using:
internet_ingress_bytes_total
internet_egress_bytes_total | Network I/O traffic generated by Pipelines |
| Data Service IO | This metric is computed using:
flink_taskmanager_job_task_operator_rlm_dsio_requestBytes_<organization-id>*
flink_taskmanager_job_task_operator_rlm_dsio_responseBytes_<organization-id>* | Data transfer (I/O) from within and outside of the platform for accessing and storing data. |
* organization-id in the following format, if you organization id is olp-here this value will be olp_here.
| METRIC | DETAILS | DESCRIPTION |
|---|
| Compute Core | pipeline_cpu_core_seconds | CPU Core Hours consumed by Batch or Stream Pipelines |
| Compute RAM | pipeline_ram_gb_seconds | RAM Hours consumed by Batch or Stream Pipelines |
| METRIC NAME | LABELS |
|---|
layer_used_bytes | billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
layer_type to filter out for a particular layer type, possible values index, stream, versioned, volatile, objectstore.
project to filter out for a particular project ID.
type with value set to data. |
storage_used_bytes | billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
storage_type to filter out for a particular storage type, possible value storage_durable_volume.
project to filter out for a particular project ID. |
layer_allocated_bytes | billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
layer_type to filter out for a particular layer type, in this metric volatile is the only value.
project to filter out for a particular project ID. |
ttl_kb_allocated | billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
layer_type to filter out for a particular layer type, in this metric stream is the only value.
project to filter out for a particular project ID. |
catalog_used_metadata_bytes | billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
project to filter out for a particular project ID.
type with value set to metadata. |
splunk_index_usage_bytes_previous_hour | billingTag to filter out for a particular billing tag.
clientAppId to filter out for a particular client app ID.
operation with value set to splunk_index.
project to filter out for a particular project ID. |
internet_ingress_bytes_total
internet_egress_bytes_total | app to filter out for the framework app used, values can be flink-1-x and spark-1-x.
billingTag to filter out for a particular billing tag.
jobId to filter out for a particular job ID.
pipelineIdto filter out for a particular pipeline ID.
pipelineVersionId to filter out for a particular pipeline version ID. |
pipeline_cpu_core_seconds
pipeline_ram_gb_seconds | billingTag to filter out for a particular billing tag.
clientAppId to filter out for a particular client app ID.
pipelineDeploymentIdto filter out for a particular pipeline deployment ID.
pipelineFrameworkto filter out for a particular pipeline framework. Currently, the possible values are: FLINK_1_7_1, FLINK_1_X, SPARK_2_1_1, SPARK_2_4_0 .
pipelineIdto filter out for a particular pipeline ID.
pipelineTemplateId to filter out for a particular pipeline template ID.
pipelineVersionId to filter out for a particular pipeline version ID.
project to filter out for a particular project ID. |
