local catalog layer stream

The OLP CLI supports the following:

• get partitions from a local stream layer
• put partitions to a local stream layer

local catalog layer stream get

Subscribes to the specified local stream layer and downloads or prints every
partition that is uploaded to the layer during the subscription period.

olp local catalog layer stream get <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --output <path to dir> The output directory to store data received
  from the stream. Files are created with the names of partitions received from
  the stream. If the specified directory doesn't exist, it is automatically
  created. If no directory is specified, partitions from the stream are printed
  to the standard output separated by a delimiter.

  Note

  When a partition with a specific name is downloaded multiple times,
  new files are created using the following naming pattern:

  • partitionName
  • partitionName_1
  • partitionName_2

• --delimiter <character> A delimiter used for printing stream data
  to the standard output; empty by default.

• --limit <max number of partitions> The maximum number of partitions to be
  downloaded.

• --timeout <duration in seconds> The subscription timeout duration in seconds
  (infinite by default).

• --consumer-group <consumer ID> The name of a consumer, or a
  group if the consumer is distributed.

• --decode <true|false> Determines whether the
  partition content is decoded from protobuf and converted to JSON on-the-fly.
  The default value is false, which means the content is not decoded.
  You can specify the path to a schema on a local file system
  with --decode <true|false:[path/to/schema.zip]>;
  otherwise, the default schema linked to the layer is used.
  An exception is thrown when there is no schema linked to
  the layer or no path to a local schema ZIP provided.
  Schemas can be downloaded from the HERE platform portal or with the
  olp schema get command.
  Using --decode true with --output parameter produces files where
  the following name pattern is used: <partition-name>-decoded.json.

• --quiet Displays the absolute path to the output directory, if available.
  Otherwise, the partitions' contents are printed from the stream to the standard
  output.

• --credentials <path to credentials file> The name of a credentials file to use
  with the command. Credentials files are downloaded separately from the platform portal.

• --profile <profile name> The name of the credentials profile to use from the
  olpcli.ini file.

For more information on using credentials and profiles, see
Credentials setup.

Example:

The command below prints all partitions from stream-layer1 in the
catalog1 using the default empty delimiter.

olp local catalog layer stream get hrn:local:data:::catalog1 stream-layer1

Output:

Local partition content example

local catalog layer stream put

Publishes the content from the filesystem into a local catalog's stream layer.

olp local catalog layer stream put <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --input <path to dir> The input directory from where to read partitions.
  Relative paths are supported. Partitions are populated by walking the file
  tree rooted at the given input directory. The file tree is traversed
  depth-first and in lexicographical order. If not specified, the partitions are
  taken from the standard input and split by the delimiter.
• --delimiter <character> A delimiter used for reading partitions
  from the standard input.
• --limit <max number of partitions> The maximum number of partitions to be
  published.
• --encode <true|false:[path/to/schema.zip]> Determines whether the
  partition content is encoded from JSON and converted to protobuf on-the-fly.
  The default value is false, which means the content is not encoded.
  Supported layer content types: protobuf.
  You can specify the path to a schema on a local file system;
  otherwise, the default schema linked to the layer is used.
  An exception is thrown when there is no schema linked to
  the layer or no path to a local schema ZIP provided.
  Schemas can be downloaded from the HERE platform portal or with the
  olp schema get command.

Note

You can encode only the JSON file that you got from the olp local catalog layer stream get command.

• --credentials <path to credentials file> The name of a credentials file to use
  with the command. Credentials files are downloaded separately from the platform portal.
• --profile <profile name> The name of the credentials profile to use from the
  olpcli.ini file.
• --quiet Displays empty output with no additional information.

For more information on using credentials and profiles, see
Credentials setup.

Example:

The command below publishes all partitions from /path/to/directory.

olp local catalog layer stream put hrn:local:data:::catalog1 stream-layer1 --input /path/to/directory

Output:

Publishing of partitions to stream-layer1 layer completed