Guides
Guides
  1. HERE Anonymizer Preprocessor User Guide
  2. Deployment environments
  3. Kubernetes

Azure AKS

Prerequisites

  1. Install tools:

  2. Create an Azure Container Registry

Deploy HERE Anonymizer Preprocessor playbook

You can complete the deployment scenario by running the supplied scripts from the path where you unpacked the Preprocessor.

Run the commands and export environment variables indicated in the code block.

# The azurecr.io registry name (without ".azurecr.io")
export AZURE_TENANT={YOUR_TENANT_ID}
export AZURE_REGISTRY_NAME={YOUR_REGISTRY_NAME}

# Azure Service Principal ID (not its name but ID) and password.
# To create Service Principal:
# `az ad sp create-for-rbac -n "some-bot" --role Contributor --scopes /subscriptions/$AZURE_SUBSCRIPTION_ID`
# Check `az account list` for getting Azure Subscription ID
export AZURE_CLIENT_ID={CLIENT_ID}
export AZURE_CLIENT_PASSWORD={CLIENT_PASSWORD}

# Azure storage account details for using it as data source and sink
export AZURE_STORAGE_ACCOUNT={STORAGE_ACCOUNT}
export AZURE_BLOB_CONTAINER={BLOB_CONTAINER}

# Path where the HERE Anonymizer Preprocessor is unzipped
export PREPROC_DIST_DIR={PATH_TO_UNPACKED_ANONYMIZER}

# Login in to azure
./deployments/kubernetes/azure/azure-login.sh

# Build and push container images, create AKS cluster, deploy
# HERE Anonymizer Preprocessor images and Apache Ignite server for demo purposes
./deployments/kubernetes/azure/azure-deploy-aks.sh

# There the HERE Anonymizer Preprocessor is deployed
# and processing data present in the source container.

# Shutdown the cluster and all its resources
./deployments/kubernetes/azure/azure-shutdown-aks.sh

Additional variables

List of optional variables for configuring azure-deploy-aks.sh and azure-shutdown-aks.sh with their default values:

export APP_NAME="hereanonaks"
export APP_VERSION="latest"
export AZURE_RESOURCE_GROUP="hereanonaks-latest"
export AZURE_CLUSTER_NAME="hereanonaks-latest"
export AZURE_VM_SIZE="standard_l8s_v2"
export AZURE_LOCATION="germanywestcentral"

Deploy manually

Push container images to Azure Container Registry (ACR)

  1. Log in to Azure using the CLI. Note that the user must have the Contributor role.

    az login

  2. Create a resource group if one doesn't exist.

    az group create --name "$AZURE_RESOURCE_GROUP" --location "$AZURE_LOCATION"

  3. Log in to private container registry.

    REGISTRY_PASS=$(az acr credential show -n "$AZURE_REGISTRY_NAME" --query "passwords[0].value" -o tsv)
docker login -u "${AZURE_REGISTRY_NAME}" -p "${REGISTRY_PASS}" "${AZURE_REGISTRY_NAME}".azurecr.io

  4. Build and push the HERE Anonymizer Preprocessor image.

    APP_IMAGE=${AZURE_REGISTRY_NAME}.azurecr.io/${APP_NAME}-flink:${APP_VERSION}
docker build --tag "$APP_IMAGE" $PREPROC_DIST_DIR
docker push "$APP_IMAGE"

  5. Push the Apache Ignite image.

    IGNITE_IMAGE=${AZURE_REGISTRY_NAME}.azurecr.io/ignite
docker pull apacheignite/ignite:3.0.0
docker tag apacheignite/ignite:3.0.0 "$IGNITE_IMAGE"
docker push "$IGNITE_IMAGE"

Create Kubernetes cluster

  1. Create a Kubernetes cluster with one Node. The type (size) of Nodes is defined by AZURE_VM_SIZE.

    az aks create \
  -g "$AZURE_RESOURCE_GROUP" \
  -n "$AZURE_CLUSTER_NAME" \
  --enable-managed-identity \
  --node-count 1 \
  --node-vm-size "$AZURE_VM_SIZE" \
  --enable-addons monitoring \
  --enable-msi-auth-for-monitoring  \
  --generate-ssh-keys

  2. Log in to cluster and wait until all Pods are ready.

    az aks get-credentials --resource-group "$AZURE_RESOURCE_GROUP" --name "$AZURE_CLUSTER_NAME" --overwrite-existing
kubectl get nodes
kubectl wait --for=condition=Ready pods --all --all-namespaces --timeout=300s
kubectl get pods,deployments -A

Deploy HERE Anonymizer Preprocessor

  1. Prepare input data in the Azure Blob Storage:

    export SOURCE_PATH=$AZURE_BLOB_CONTAINER/$APP_NAME/$APP_VERSION/input
export SINK_PATH=$AZURE_BLOB_CONTAINER/$APP_NAME/$APP_VERSION/output
az storage blob upload-batch \
  --account-name "${AZURE_STORAGE_ACCOUNT}" \
  --destination "${SOURCE_PATH}" \
  --source "${PREPROC_DIST_DIR}"/deployments/common/here-probe-example-data/

  2. Edit configuration files and environment variables. See Configuration for details.

    export USERNAME=$AZURE_STORAGE_ACCOUNT
export PASSWORD=$AZURE_STORAGE_KEY
export INDEX_CONNECTOR_SCHEME="az+blob+index"
export PREPROC_CONNECTOR_SCHEME="az+blob+preprocess"
export SINK_CONNECTOR_SCHEME="az+blob+batch+files"
envsubst '${USERNAME} ${PASSWORD} ${SOURCE_PATH} ${SINK_PATH}' <"${PREPROC_DIST_DIR}/deployments/kubernetes/env-configmap.yml" | kubectl apply -f -
kubectl create -f "${PREPROC_DIST_DIR}/deployments/kubernetes/conf-files-configmap.yml"

  3. Configure container registry credentials as a Kubernetes secret. Note that the command exports all local Docker credentials.

    kubectl create secret generic regcred \
  --from-file=.dockerconfigjson="$HOME/.docker/config.json" \
  --type=kubernetes.io/dockerconfigjson

  4. As the private container registry is used, container image substitution is required before each deployment.

    # Deploy ignite service
kubectl create -f "${PREPROC_DIST_DIR}/deployments/kubernetes/ignite/ignite-service-account-role.yml"
kubectl create -f "${PREPROC_DIST_DIR}/deployments/kubernetes/ignite/ignite-service.yml"
IMAGE=$IGNITE_IMAGE envsubst <"${PREPROC_DIST_DIR}/deployments/kubernetes/ignite/ignite-deployment-template.yml" | kubectl apply -f -

# Wait for ignite to be running
kubectl wait --for=condition=available --timeout=300s deployment/ignite-service

# Deploy flink cluster for Indexer
IMAGE=$APP_IMAGE envsubst <"${PREPROC_DIST_DIR}/deployments/kubernetes/indexer/jobmanager-batch-job.template.yml" | kubectl apply -f -
kubectl create -f "${PREPROC_DIST_DIR}/deployments/kubernetes/indexer/jobmanager-service.yml"
IMAGE=$APP_IMAGE envsubst <"${PREPROC_DIST_DIR}/deployments/kubernetes/indexer/taskmanager-deployment.template.yml" | kubectl apply -f -

# Wait for the indexer to finish
kubectl wait --for=condition=complete job/indexer-jobmanager --timeout=300s

# Deploy flink cluster for Preprocessor
IMAGE=$APP_IMAGE envsubst <"${PREPROC_DIST_DIR}/deployments/kubernetes/preprocessor/jobmanager-batch-job.template.yml" | kubectl apply -f -
kubectl create -f "${PREPROC_DIST_DIR}/deployments/kubernetes/preprocessor/jobmanager-service.yml"
IMAGE=$APP_IMAGE envsubst <"${PREPROC_DIST_DIR}/deployments/kubernetes/preprocessor/taskmanager-deployment.template.yml" | kubectl apply -f -

  5. Review all deployed resources:

    kubectl wait --for=condition=Ready pods --all --all-namespaces --timeout=300s
kubectl get nodes,pods,deployments,jobs,configmaps,secrets,services

Run smoke test

Run a smoke test on the deployed cluster to check the preprocessed data.

  1. Run this command to check the indexer report:

    az storage blob download --account-name "${AZURE_STORAGE_ACCOUNT}" --container-name "${AZURE_BLOB_CONTAINER}" --name "$APP_NAME/$APP_VERSION/input/INDEXER_REPORT.json" --no-progress | jq "."

  2. In a new terminal, run this command to access the Flink UI:

    kubectl port-forward jobs/preprocessor-jobmanager 8081:8081
    📘

    Note

    The Flink UI is only accessible for the period when preprocessor job is running.

  3. Next, go to http://localhost:8081 and open Running Jobs -> HERE Anonymizer preprocessor, then Source: Preprocess ... task's Accumulators tab.

  4. Check if the essential metrics like HERE_decoding_point_info_all and HERE_output_point_info_all have a value greater than zero. See all the anonymization metrics explained here.

  5. After the preprocessor job completes, run this command to access the preprocessor report:

    az storage blob download --account-name "${AZURE_STORAGE_ACCOUNT}" --container-name "${AZURE_BLOB_CONTAINER}" --name "$APP_NAME/$APP_VERSION/output/PREPROCESSOR_REPORT.json" --no-progress | jq "."

  6. Additionally, the preprocessed data can be browsed at the location referred by ${SINK_PATH}.

Cleanup

  1. Remove the Azure Resource Group to delete all associated resources.

    az group delete --name "$AZURE_RESOURCE_GROUP" -y

  2. Delete the created container image.

    az acr repository delete --name ${AZURE_REGISTRY_NAME} --image "$IMAGE" -y

Updated 16 days ago