Local data workflows

The OLP CLI supports management of catalogs locally for testing or development purposes. It provides tools for managing local data, such as:

List all catalogs to which you have access.
Create a new catalog and layer.
Upload data to a catalog.
Copy a platform-hosted catalog into a local catalog.

For more details, see local catalog commands.

Note

Local catalogs may only be used for development and testing purposes. Running production use cases with local catalogs is not permitted.

List all local catalogs

To list the local catalogs you have in your local machine, enter the olp local catalog list command.

olp local catalog list

The command lists the HRNs of the local catalogs you have, each in a new line.

Create a new local catalog and layer

Follow the steps below to create a new local catalog and layer.

Enter the olp local catalog create command.

    olp local catalog create first-catalog-example-id first-catalog-example \
    --summary "This catalog contains a sample layer containing data in CSV format" \
    --description "A longer description of the catalog contents"

    olp local catalog create first-catalog-example-id first-catalog-example ^
    --summary "This catalog contains a sample layer containing data in CSV format" ^
    --description "A longer description of the catalog contents"

To add a layer to your local catalog, enter the olp local catalog layer add command.

    olp local catalog layer add hrn:local:data:::first-catalog-example-id example-layer-id \
    "Example layer" --versioned --summary "This layer contains simple data in CSV format" \
    --description "A longer description of the layer contents" --content-type=text/csv

    olp local catalog layer add hrn:local:data:::first-catalog-example-id example-layer-id ^
    "Example layer" --versioned --summary "This layer contains simple data in CSV format" ^
    --description "A longer description of the layer contents" --content-type=text/csv

To verify the local catalog configuration, enter the olp local catalog show command.

olp local catalog show hrn:local:data:::first-catalog-example-id --json

The command displays the following results.

{
        "summary": "This catalog contains sample layers containing simple text",
        "owner": {},
        "billingTags": [],
        "hrn": "hrn:local:data:::first-catalog-example-id",
        "created": "2018-03-26T12:48:10.315Z",
        "name": "first-catalog-example",
        "layers": [{
            "summary": "This layer contains simple data in CSV format",
            "volume": {
                 "volumeType": "durable"
            },
            "layerType": "versioned",
            "billingTags": [],
            "name": "Example layer",
            "description": "A longer description of the layer contents",
            "partitioningScheme": "generic",
            "partitioning": {
                 "scheme": "generic"
            },
            "id": "example-layer-id",
            "contentType": "text/csv",
            "tags": []
        }],
        "description": "",
        "notifications": {
            "enabled": false
         },
        "id": "first-catalog-example-id",
        "version": -1,
        "tags": []
    }

Upload data to a local catalog

Create a file with the details you want to upload, such as partition-0.csv in the following example.

   City,Country
   Berlin,Germany
   Seattle,USA
   Chicago,USA
   Boston,USA

Enter the olp local catalog layer partition put command.

   olp local catalog layer partition put hrn:local:data:::first-catalog-example-id \
   example-layer-id --partitions testPart0:<path_to_file>/partition-0.csv

   olp local catalog layer partition put hrn:local:data:::first-catalog-example-id ^
   example-layer-id --partitions testPart0:<path_to_file>\partition-0.csv

The command displays the following result:

   Uploading the files into partitions.

    100% [=======================================] 1/1 (0:00:01 / 0:00:00)
   The data was successfully added to the partitions: [testPart0]

To verify the partition, enter the olp local catalog layer partition list command.

olp local catalog layer partition list hrn:local:data:::first-catalog-example-id example-layer-id

The console displays the following text:

   partition           size                checksum
   testPart0           23                  665d2817c51cbccd4312eb22699163c05b9d5a0a

   Total size: 23 B
   Use olp local catalog layer partition get <catalog HRN> <layer ID> to download partitions

To retrieve the partition-0.csv file from the olpcli-example-catalog-id, enter the olp local catalog layer partition get command.

   olp local catalog layer partition get \
   hrn:local:data:::first-catalog-example-id example-layer-id --partitions testPart0

   olp local catalog layer partition get ^
   hrn:local:data:::first-catalog-example-id example-layer-id --partitions testPart0

The console displays the text stored in the partition-0.csv file:

   City,Country
   Berlin,Germany
   Seattle,USA
   Chicago,USA
   Boston,USA

Copy a platform-hosted catalog into a local catalog

Copy a bounding box around Berlin from the topology-geometry and cartography layers of the HERE Map Content catalog into a local catalog:

   olp local catalog copy create hrn:here:data::olp-here:rib-2 \
   --include topology-geometry cartography \
   --filter [heretile]=bounding-box:52.67551,52.33826,13.76116,13.08835 \
   --id here-map-content

   olp local catalog copy create hrn:here:data::olp-here:rib-2 ^
   --include topology-geometry cartography ^
   --filter [heretile]=bounding-box:52.67551,52.33826,13.76116,13.08835 ^
   --id here-map-content

The console displays the following text:

   state:                              skipped
   history:                            skipped
   address-locations:                  skipped
   building-footprints:                skipped
   3d-buildings:                       skipped
   cartography:                        100% [=====] 44/44 (0:00:01 / 0:00:00)
   administrative-places:              skipped
   road-traffic-pattern-attributes:    skipped
   traffic-patterns:                   skipped
   street-names:                       skipped
   lane-attributes:                    skipped
   address-attributes:                 skipped
   adas-attributes:                    skipped
   road-attributes:                    skipped
   topology-geometry:                  100% [=====] 45/45 (0:00:01 / 0:00:00)
   administrative-locations:           skipped
   navigation-attributes:              skipped
   advanced-navigation-attributes:     skipped
   truck-attributes:                   skipped
   cartography-metadata:               skipped
   building-metadata:                  skipped
   here-places:                        skipped
   places-metadata:                    skipped
   distance-markers:                   skipped
   indexed-locations:                  skipped
   sign-text:                          skipped
   environmental-zones:                skipped
   here-places-essential-map:          skipped
   landmarks-3d:                       skipped
   landmarks-2d:                       skipped
   vehicle-regulations:                skipped
   flag-images:                        skipped
   postal-code-points:                 skipped
   postal-area-boundaries:             skipped
   toll-cost:                          skipped
   electric-vehicle-charging-stations: skipped
   here-truck-service-locations:       skipped
   here-fueling-stations:              skipped
   generalized-junctions-signs:        skipped
   annotations:                        skipped
   bicycle-attributes:                 skipped
   warning-locations:                  skipped
   complex-road-attributes:            skipped
   recreational-vehicle-attributes:    skipped
   administrative-place-profiles:      skipped
   Completed sync up to version 5
   Local catalog hrn:local:data:::here-map-content created

Synchronize an existing local copy with the platform hosted source catalog:

   olp local catalog copy update hrn:local:data:::here-map-content

The console displays the following text:

   Catalog configuration is up to date
   here-map-content: syncing version 6
   state:                              skipped
   history:                            skipped
   address-locations:                  skipped
   building-footprints:                skipped
   3d-buildings:                       skipped
   cartography:                        100% [=====] 3/3 (0:00:00 / 0:00:00)
   administrative-places:              skipped
   road-traffic-pattern-attributes:    skipped
   traffic-patterns:                   skipped
   street-names:                       skipped
   lane-attributes:                    skipped
   address-attributes:                 skipped
   adas-attributes:                    skipped
   road-attributes:                    skipped
   topology-geometry:                  100% [=====] 9/9 (0:00:00 / 0:00:00)
   administrative-locations:           skipped
   navigation-attributes:              skipped
   advanced-navigation-attributes:     skipped
   truck-attributes:                   skipped
   cartography-metadata:               skipped
   building-metadata:                  skipped
   here-places:                        skipped
   places-metadata:                    skipped
   distance-markers:                   skipped
   indexed-locations:                  skipped
   sign-text:                          skipped
   environmental-zones:                skipped
   here-places-essential-map:          skipped
   landmarks-3d:                       skipped
   landmarks-2d:                       skipped
   vehicle-regulations:                skipped
   flag-images:                        skipped
   postal-code-points:                 skipped
   postal-area-boundaries:             skipped
   toll-cost:                          skipped
   electric-vehicle-charging-stations: skipped
   here-truck-service-locations:       skipped
   here-fueling-stations:              skipped
   generalized-junctions-signs:        skipped
   annotations:                        skipped
   bicycle-attributes:                 skipped
   warning-locations:                  skipped
   complex-road-attributes:            skipped
   recreational-vehicle-attributes:    skipped
   administrative-place-profiles:      skipped
   Completed sync up to version 6

To get a full list of the available commands, enter olp local --help.