nx_metadata_sdk 1.0 Metadata SDK

Object Streamer Plugin

// Copyright 2018-present Network Optix, Inc. Licensed under MPL 2.0: www.mozilla.org/MPL/2.0/

## Introduction

This Plugin is intended to be used for debugging/investigating/developing purposes and can generate any needed sequence of Objects that are taken from a specific user-generated file.

The Taxonomy of the generated Objects (their types and Attributes) must be defined in the Device Agent manifest - a .json file specified in the Device Agent settings. An example of such manifest is provided in `manifest.json`.

The Object stream to be "played back" by the Plugin is taken from the .json file specified in the Device Agent settings. An example of such file is provided in `stream.json`.

To generate a stream and manifest files, the provided `generate.py` script can be used.

## Stream file format

A stream file is essentially a script which defines when and what metadata will be generated by the plugin. It is a JSON file containing an array of per-frame object entries. There are several types of entries like bounding boxes and best shots. The following fields are allowed for each entry:

- `frameNumber` [required], [int] Frame number the entry will be instantiated on.
- `typeId` [required], [string] Object Type of the entry.
- `trackId` [required], [guid/placeholder] Can be either a fixed UUID, or `"$"` or `"$$"` followed by a number, which makes a new random UUID for each such number:
  - `$<number>` Id of the Object Track is changed every time the Device Agent is recreated, chosen randomly.
  - `$$<number>` Id of the Object Track is changed every stream cycle or every time the Device Agent is recreated, chosen randomly.
- `attributes` [optional], [map(string, string)] A string or string map of Attributes. Here the key is the Attribute name, and the value is the Attribute value.
- `boundingBox` [required], [object] Object containing coordinates of the Object bounding box. Contains the following fields:
  - `x` Top-left corner horizontal coordinate, 0..1.
  - `y` Top-left corner vertical coordinate, 0..1.
  - `width` Width of the bounding box, 0..1.
  - `height` Height of the bounding box, 0..1.
- `imageSource` [optional], [string], [applied only for best shot entries] A string containing either a path to a local image or an image URL that will be used as a Best Shot.
- `entryType` [optional], [string] Either `"bestShot"` or `"regular"`. The default is `"regular"`.

Example:

{
"frameNumber": 0,
"typeId": "nx.base.Vehicle",
"trackId": "$$10",
"attributes": {
"Color": "Red",
"Brand": "Lada",
"Model": "2106",
"LicensePlate.Number": "A001AA",
"LicensePlate.State/Province": "77",
"LicensePlate.Country": "Russia"
},
"boundingBox": {
"x": 0.5822482733134051,
"y": 0.3331168814679488,
"width": 0.17938270955154084,
"height": 0.15537941471920017
}
},
{
"frameNumber": 0,
"typeId": "nx.base.Vehicle",
"trackId": "$$10",
"attributes": {
"Best Shot Type": "URL",
"Color": "Dark Red",
"Brand": "Mercedes",
"Model": "EQE",
"LicensePlate.Number": "KEN560",
"LicensePlate.State/Province": "05",
"LicensePlate.Country": "Kazakhstan"
},
"boundingBox": {
"x": 0.5822482733134051,
"y": 0.3331168814679488,
"width": 0.17938270955154084,
"height": 0.15537941471920017
},
"imageSource": "https://example.com/photos/200/300",
"entryType": "bestShot"
}

## Stream generation script

The script `generate.py` can generate `stream.json` and `manifest.json` based on `config.json`. The example of `config.json` file is provided.

### Command line options

- `--config, -c` [required] Path to the configuration file containing parameters of the stream to be generated.
- `--manifest-file, -m` [optional] Path to the output Device Agent Manifest file, default is `manifest.json`.
- `--stream-file, -s` [optional] Path to the output stream file, default is `stream.json`.
- `--compressed-stream` [optional] Removes all formatting from the output stream file.

Example:

generate.py --config config.json --manifest-file my_manifest.json --stream-file my_stream.json
--compressed-stream

### Configuration file format

The configuration file is a simple JSON file with a JSON object at the top level. The following settings are available:

- `objectCount` - [int] Count of objects that simultaneously exist on each frame. Default is `5`.
- `streamDurationInFrames` - [int] Duration in frames of a single stream cycle. Default is `300`.
- `typeLibrary` - [object] The Type Library section of the generated Device Agent manifest. Object Types auto-generated from the `objectTemplates` section can be added to the `typeLibrary` of the output manifest as well. See `manifests.md` in the SDK for more information about Type Library and Device Agent Manifest.
- `objectTemplates` - [array] Settings of the Object Track generator, as an array of JSON objects each representing a template for the particular Object type. The set of fields of each template depends on the generator type which is defined by the `movementPolicy` field of the template. The default and the only currently available value of `movementPolicy` is `random`. The structure of a template JSON object is given below.

#### `random` Object template settings

All fields are optional.

- `objectTypeId` - [string] Object Type id of the generated Object Track. If not defined, the id is auto-generated. If the id is not represented in the `typeLibrary` section, the name of the type is auto-generated from the id, and the corresponding Object Type is added to the output Device Agent manifest.
- `trackIdPolicy` - [string] The policy of Object Track id behavior. Three options are available:
  - `fixed` - Object track id never changes (neither between Device Agent instantiations nor between stream cycles).
  - `changeOncePerStreamCycle` - Object Track id is changed every stream cycle.
  - `changeOnDeviceAgentCreation` - Object Track id is changed every time the Device Agent is instantiated.
- `minWidth` - [float(0.0, 1.0]] Minimum width of the Object bounding box. Default is `0.05`.
- `maxWidth` - [float(0.0, 1.0]] Maximum width of the Object bounding box. Default is `0.3`.
- `minHeight` - [float(0.0, 1.0]] Minimum height of the Object bounding box. Default is `0.05`.
- `maxHeight` - [float(0.0, 1.0]] Maximum height of the Object bounding box. Default is `0.3`.
- `attributes` - [object] Key-value pairs, the key is an Object attribute name, the value is an Object Attribute value. All values must be represented as strings. This set of Attributes is assigned to each Object generated from the template.
- `bestShots` - [array] A list of Best Shot settings for the particular Object template. Each Best Shot item can contain the following fields:
  - `frameNumber` - [int, required] The frame number the Best Shot has to be generated on.
  - `imageSource` - [string] Can contain an HTTP or HTTPS URL or a path to an image. If empty, the current bounding box of the Object is used as its Best Shot.
  - `attributes` - [object] Attributes of the Best Shot. The same as Attributes for an Object template.

#### Example

{
"objectCount": 5,
"streamDurationInFrames": 100,
"typeLibrary":
{
"objectTypes":
[
{
"id": "stub.objectStreamer.custom.type.1",
"name": "Stub OS: Custom Type 1"
}
]
},
"objectTemplates":
[
{
"objectTypeId": "nx.base.Vehicle",
"movementPolicy": "random",
"trackIdPolicy": "changeOncePerStreamCycle",
"minWidth": 0.1,
"minHeight": 0.1,
"maxWidth": 0.3,
"maxHeight": 0.3,
"attributes":
{
"Color": "Red",
"Brand": "Ford",
"Model": "F150",
"LicensePlate.Number": "LHD 8448",
"LicensePlate.State/Province": "Texas",
"LicensePlate.Country": "USA"
},
"bestShots": [
{
"imageSource": "https://picsum.photos/200/300",
"frameNumber": 10,
"attributes": {
"some attribute": "attribute value"
}
}
]
},
{
"objectTypeId": "stub.objectStreamer.custom.type.1",
"movementPolicy": "random",
"attributes":
{
"string attribute 1": "first string attribute",
"string attribute 2": "second string attribute",
"number attribute 1": "0.1",
"number attribute 2": "300"
}
}
]
}