GuidesPipeline API ReferenceChangelog
Guides
  1. Command Line Interface
  2. OLP CLI Command reference

pipeline template

pipeline template

The OLP CLI supports the following:

  • list pipeline templates
  • create pipeline templates
  • show pipeline templates
  • delete pipeline templates

pipeline template list

Lists all pipeline templates to which you have access.

olp pipeline template list [filter] [command options]

Optional parameters:

  • [filter] Freeform text for filtering the pipeline template list: the filter
    checks if the name or the description field contains the filter string.
  • --credentials <path to credentials file> The name of a credentials file to use
    with the command. Credentials files are downloaded separately from the platform portal.
  • --profile <profile name> The name of the credentials profile to use from the
    olpcli.ini file.
  • --json Displays the command result in JSON format.
  • --quiet Displays pipeline template IDs, each on a new line.
  • --scope <project HRN> Specifies the project HRN to use as the scope in the
    request. The value specified with --scope overrides any value for
    here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see
Credentials setup.

Example:

The command below lists all the pipeline templates to which you have access
and whose name or description field contains the word spark.

olp pipeline template list "spark" --json

Output:

{
    "pipelineTemplates": [
        {
            "created": "2018-03-01T15:16:53.796Z",
            "groupId": "GROUP-9479863e-a13b-4d35-9eb1-5a054669046e",
            "defaultClusterConfiguration": {
                "workerResourceProfileId": "HS1B",
                "supervisorResourceProfileId": "HS1B",
                "workerAdditionalOverheadMemory": 0,
                "gcProfile": "parallelgc",
                "supervisorUnits": 1,
                "workerUnits": 1,
                "workers": 1,
                "supervisorAdditionalOverheadMemory": 0
            },
            "name": "sparktestcompiler",
            "packageId": "68d723f6-2ae7-40e4-8c24-61512d511852",
            "entryPointClassName": "com.example.Main",
            "description": "",
            "id": "5c0660a3-0fb4-4f35-bcd0-be6ce25075f6",
            "state": "created",
            "defaultRuntimeConfiguration": "",
            "updated": "2018-03-01T15:16:53.796Z",
            "runtimeEnvironment": "batch-5.2"
        }
    ]
}

pipeline template create

Creates a pipeline template.

olp pipeline template create <name> <runtime environment> <JAR file> \
 <class name> <group ID> --input-catalog-ids catalogExample1 [command options]
olp pipeline template create <name> <runtime environment> <JAR file> ^
<class name> <group ID> --input-catalog-ids catalogExample1 [command options]

Required parameters:

  • <name> The name for a pipeline template. The length of this parameter must
    be between 3 and 64 characters.

  • <runtime environment> A distributed processing framework used to execute the
    pipeline template with the possible runtime environment ID. To list all the currently enabled runtime
    environments, enter the olp pipeline environment list
    command.

  • <JAR file> A fat JAR file, including the path on a local file system,
    to upload.

  • <class name> The name of the pipeline template main class.

  • Either of the following:

    • <group ID> The ID of the group whose members are allowed to access the
      pipeline.
    • --scope <project HRN> The HRN of the project whose members are allowed to
      access the pipeline.

  • --input-catalog-ids <fixed catalog identifiers...> Fixed identifiers
    of the input catalogs expected by the pipeline (separated by space).
    This list should match the catalog identifiers used in the
    pipeline-config.conf file.

    Note

    --input-catalog-ids may also accept the path to the pipeline-config.conf
    configuration file that contains fixed identifiers of catalogs.
    This file is used when a pipeline version is created.

    For more information on pipeline-config.conf, see
    Pipeline Configuration.

Optional parameters:

  • --description <template description> A description of the pipeline template.
    The length of this parameter must be between 0 and 512 characters.

  • --supervisor-units <size in units> The size of a supervisor node (1 to 15
    units). If not specified, the default value is 1.

  • --supervisor-units-profile <resource profile ID> The ID of the resource profile requested for the supervisor units.
    If not specified, the Default profile is used. To list all the currently available resource
    profiles, enter the olp pipeline profile list
    command.

  • --supervisor-additional-overhead-memory <overhead memory in fraction> The factor of proportionality of additional
    non-heap memory per supervisor unit. The value can be greater than or equal to 0.00 with default maximum value of 0.05
    (but this could be raised on a realm basis).

  • --workers <number of workers> The default number of workers. If not specified, the default value is 1.

  • --worker-units <size in units> The default size of a worker node (1 to 15
    units). If not specified, the default value is 1.

  • --worker-units-profile <resource profile ID> ID of the resource profile requested for the worker units.
    If not specified, the Default profile is used. To list all the currently available resource
    profiles, enter the olp pipeline profile list
    command.

  • --worker-additional-overhead-memory <overhead memory in fraction> The factor of proportionality of additional
    non-heap memory per worker unit. The value can be greater than or equal to 0.00 with default maximum value of 0.05
    (but this could be raised on a realm basis).

  • --garbage-collector-profile <profile ID> ID of the garbage collector profile that is used to configure the
    garbage collection strategy for the pipeline execution.
    To list all the currently available garbage collector profiles, enter the
    olp pipeline environment list command.

    Note::

    If garbage collector profile is not specified, the Parallel GC profile is used for batch pipeline templates
    and the G1 GC profile is used for stream pipeline templates.

  • --default-runtime-config <key1=value1 key2=value2 ...> A map of default
    configuration values for the pipeline application. The maximum key name size
    is 256, and the maximum value size is 1024. Runtime configuration can be
    overridden while creating a pipeline version.
    If a pipeline version has its own runtime configuration values, they are added
    to any defaults available from the parent pipeline template.

    Note

    The value of the --default-runtime-config parameter should be quoted with double quotes, like
    "key1=value1 key2=value2". In case some value contains spaces, it should be quoted with
    single quotes like "key1='some value1' key2=value2".
    Take into account that a parameter value with the \n splitter is also acceptable.
    Example: "key1='some value1'\nkey2=value2\n"

  • --credentials <path to credentials file> The name of a credentials file to use
    with the command. Credentials files are downloaded separately from the platform portal.

  • --profile <profile name> The name of the credentials profile to use from the
    olpcli.ini file.

  • --json Displays the created pipeline template contents in JSON format.

  • --quiet Displays the pipeline template ID.

For more information on using credentials and profiles, see
Credentials setup.

For more information on pipeline configuration, see
Pipeline Configuration.

For more information on additional non-heap memory, see
Additional Non-Heap Memory.

Example:

The command below creates a pipeline template with the specified name, runtime environment,
package, main class, project that the pipeline belongs to, and input
catalog IDs that are expected in the pipeline version configuration.

olp pipeline template create "example-pipeline-template" batch-5.2 \
example-pipeline.jar com.example.Main \
--input-catalog-ids optimized-map sensor-data \
--scope hrn:here:authorization::org:project/project-example
olp pipeline template create "example-pipeline-template" batch-5.2 ^
example-pipeline.jar com.example.Main ^
--input-catalog-ids optimized-map sensor-data ^
--scope hrn:here:authorization::org:project/project-example

Output:

Pipeline template has been created
ID: 5c0660a3-0fb4-4f35-bcd0-be6ce25075f6

pipeline template show

Shows all the information about the specified pipeline template.

olp pipeline template show <pipeline template ID> [command option]

Required parameters:

  • <pipeline template ID> The ID of the pipeline template to show.

Optional parameters:

  • --credentials <path to credentials file> The name of a credentials file to use
    with the command. Credentials files are downloaded separately from the platform portal.
  • --profile <profile name> The name of the credentials profile to use from the
    olpcli.ini file.
  • --json Displays the template information in JSON format.
  • --quiet Displays the pipeline template ID.
  • --scope <project HRN> Specifies the project HRN to use as the scope in the
    request. The value specified with --scope overrides any value for
    here.token.scope provided in the credentials file used for the command.

Example:

olp pipeline template show 5c0660a3-0fb4-4f35-bcd0-be6ce25075f6 --json

Output:

The example below gets all the information about the specified pipeline
template.

{
    "created": "2018-03-01T15:16:53.796Z",
    "groupId": "GROUP-9479863e-a13b-4d35-9eb1-5a054669046e",
    "hrn": "hrn:here:pipeline-template::org:5c0660a3-0fb4-4f35-bcd0-be6ce25075f6",
    "defaultClusterConfiguration": {
        "workerResourceProfileId": "HS1B",
        "supervisorResourceProfileId": "HS1B",
        "workerAdditionalOverheadMemory": 0,
        "gcProfile": "parallelgc",
        "supervisorUnits": 1,
        "workerUnits": 1,
        "workers": 1,
        "supervisorAdditionalOverheadMemory": 0
    },
    "name": "sparktestcompiler",
    "packageId": "68d723f6-2ae7-40e4-8c24-61512d511852",
    "entryPointClassName": "com.example.Main",
    "description": "",
    "id": "5c0660a3-0fb4-4f35-bcd0-be6ce25075f6",
    "state": "created",
    "defaultRuntimeConfiguration": "",
    "updated": "2018-03-01T15:16:53.796Z",
    "runtimeEnvironment": "batch-5.2"
}

pipeline template delete

Deletes the specified pipeline template. The pipeline template that
you want to delete cannot be in use. This means that there are no pipeline
versions defined with that pipeline template.

olp pipeline template delete <pipeline template ID> [command option]

Warning

Pipeline template is removed completely

This command removes the pipeline template completely,
and there is no way to restore it.

Required parameters:

  • <pipeline template ID> The ID of the pipeline template that should be
    deleted.

Optional parameters:

  • --credentials <path to credentials file> The name of a credentials file to use
    with the command. Credentials files are downloaded separately from the platform portal.
  • --profile <profile name> The name of the credentials profile to use from the
    olpcli.ini file.
  • --quiet Displays empty output with no additional information.
  • --scope <project HRN> Specifies the project HRN to use as the scope in the
    request. The value specified with --scope overrides any value for
    here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see
Credentials setup.

Example:

The command below deletes the specified pipeline template.

olp pipeline template delete 5c0660a3-0fb4-4f35-bcd0-be6ce25075f6

Output:

Pipeline template 5c0660a3-0fb4-4f35-bcd0-be6ce25075f6 has been deleted

Updated 6 days ago

Did this page help you?