catalog layer stream

The OLP CLI supports the following:

• get partitions from a stream layer
• put partitions to a stream layer
• ingest partitions to a stream layer

catalog layer stream get

Subscribes to the specified stream layer and downloads or prints every
partition that is uploaded to the layer during the subscription period.

olp catalog layer stream get <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --output <path to dir> The output directory to store data received
  from the stream. Files are created with the names of partitions received from
  the stream. If the specified directory doesn't exist, it is automatically
  created. If no directory is specified, partitions from the stream are printed
  to the standard output separated by a delimiter.

  Note

  The command can produce empty output when trying to write the content of the partition
  that contains non-ASCII symbols to stdout. This is a Windows-specific problem, related to
  the peculiarity of rendering UTF-8 symbols on Windows console. As a workaround, please redirect
  command output to a file or specify the output directory with the --output parameter.

  Note

  When a partition with a specific name is downloaded multiple times,
  new files are created using the following naming pattern:

  • partitionName
  • partitionName_1
  • partitionName_2

• --delimiter <character> A delimiter used for printing stream data
  to the standard output; empty by default.

• --limit <max number of paritions> The maximum number of partitions to be
  downloaded.

• --timeout <duration in seconds> The subscription timeout duration in seconds
  (infinite by default).

• --consumer-group <consumer ID> The name of a consumer, or a
  group if the consumer is distributed.

• --decode <true|false> Determines whether the
  partition content is decoded from protobuf and converted to JSON on-the-fly.
  The default value is false, which means the content is not decoded.
  You can specify the path to a schema on a local file system
  with --decode <true|false:[path/to/schema.zip]>;
  otherwise, the default schema linked to the layer is used.
  An exception is thrown when there is no schema linked to
  the layer or no path to a local schema ZIP provided.
  Schemas can be downloaded from the HERE platform portal or with the
  olp schema get command.
  Using --decode true with --output parameter produces files where
  the following name pattern is used: <partition-name>-decoded.json.

• --credentials <path to credentials file> The name of a credentials file to use
  with the command. Credentials files are downloaded separately from the platform portal.

• --profile <profile name> The name of the credentials profile to use from the
  olpcli.ini file.

• --quiet Displays the absolute path to the output directory, if available.
  Otherwise, the partitions' contents are printed from the stream to the standard
  output.

• --scope <project HRN> Specifies the project HRN to use as the scope in the
  request. The value specified with --scope overrides any value for
  here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see
Credentials setup.

Example:

The command below prints all partitions from stream-layer1 in the
catalog1 using the default empty delimiter.

olp catalog layer stream get hrn:here:data::org:catalog1 stream-layer1

Output:

Partition content example

catalog layer stream put

Publishes the content from a local machine into a catalog's stream layer.

olp catalog layer stream put <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --input <path to dir>/<glob-pattern> The input directory, single file, or filename glob pattern (non-recursive)
  from where to read partitions. Relative paths are supported. Partitions are populated by walking the file
  tree rooted at the given input directory. The file tree is traversed
  depth-first and in lexicographical order. If not specified, the partitions are
  taken from the standard input and split by the delimiter.
• --delimiter <character> A delimiter used for reading partitions
  from the standard input.
• --limit <max number of partitions> The maximum number of partitions to be
  published.
• --encode <true|false:[path/to/schema.zip]> Determines whether the
  partition content is encoded from JSON and converted to protobuf on-the-fly.
  The default value is false, which means the content is not encoded.
  Supported layer content types: protobuf.
  You can specify the path to a schema on a local file system;
  otherwise, the default schema linked to the layer is used.
  An exception is thrown when there is no schema linked to
  the layer or no path to a local schema ZIP provided.
  Schemas can be downloaded from the HERE platform portal or with the
  olp schema get command.

Note

You can encode only the JSON file that you got from the olp catalog layer stream get command.

• --credentials <path to credentials file> The name of a credentials file to use
  with the command. Credentials files are downloaded separately from the platform portal.
• --profile <profile name> The name of the credentials profile to use from the
  olpcli.ini file.
• --quiet Displays empty output with no additional information.
• --scope <project HRN> Specifies the project HRN to use as the scope in the
  request. The value specified with --scope overrides any value for
  here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see
Credentials setup.

Example:

The command below publishes all partitions from /path/to/directory.

olp catalog layer stream put hrn:here:data::org:catalog1 stream-layer1 --input /path/to/directory

Output:

Publishing of partitions to stream-layer1 layer completed

catalog layer stream ingest

Publishes the content from a local machine into a catalog's stream layer.

olp catalog layer stream ingest <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --input <path to dir>/<glob-pattern> The input directory, single file, or filename glob pattern (non-recursive)
  from where to read partitions. Relative paths are supported. Partitions are populated by walking the file
  tree rooted at the given input directory. The file tree is traversed
  depth-first and in lexicographical order. If not specified, the partitions are
  taken from the standard input and split by the delimiter.
• --delimiter <character> The delimiter used for reading partitions
  from the standard input.
• --limit <max number of partitions> The maximum number of partitions to be
  published.
• --billing-tag <tag string> A tag for cost allocation used to group billing
  records.
• --x-here-trace-id <trace ID> The trace ID used to track this ingestion
  session. If skipped, a random UUID is generated instead.
• --credentials <path to credentials file> The name of a credentials file to use
  with the command. Credentials files are downloaded separately from the platform portal.
• --profile <profile name> The name of the credentials profile to use from the
  olpcli.ini file.
• --quiet Displays empty output with no additional information.
• --scope <project HRN> Specifies the project HRN to use as the scope in the
  request. The value specified with --scope overrides any value for
  here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see
Credentials setup.

Example:

The command below publishes all partitions from /path/to/directory.

olp catalog layer stream ingest hrn:here:data::org:catalog1 stream-layer1 --input /path/to/directory

Output:

Publishing of partitions to stream-layer1 layer completed