Tutorial

This tutorial goes through the main features of ZOTI-YAML, and is supposed to complement the Syntax Reference. ZOTI-YAML extends the YAML 1.1 language with support for modules and library-like component imports.

First, follow the README instructions in the ZOTI-YAML project folder on how to build lhe package. Optionally (if not in a Pipenv shell) you might want to make the package visible by adding it to PYTHONPATH, e.g.

export PYTHONPATH=$PYTHONPATH:/path/to/zoti-yaml/src

To showcase a workflow based on ZOTI-YAML modules, we set up a toy example in an arbitrary folder where the ZOTI-YAML CLI tool is accessible. In this example we run the CLI tool as executable Python module:

export ZOML=python3 -m zoml

Let a “top” module (i.e., a module which imports and uses nodes from other modules) called main.zoml be defined in this folder as follows:

main.zoml

module: main
import:
  - {module: mod1}
  - {module: sub.mod, as: mod2}

---

!default
- root:
    - mark: DEFAULT_MARKING
- root:
  - name: n1
    nodes:
      - name: n1_n1
        nodes:
          - name: n1_n1_n1
            data:
              !attach
              ref: !ref {path: "../../../nodes[n1_n2]/extra"}
      - name: n1_n2
        extra: "I am referenced by n1_n1_n1!"
        data:
          !attach
          ref: !ref {module: mod1, path: "/root/node[n_who]/data"}
  - !attach
    ref: !ref {module: mod2, path:  "/root/nodes[n1]"}
    name: n2
    zoti-args:
      i-need-this: "This field is used only to pass caller argument and will be destroyed"
    content-extra: "I will be ignored!" 
    

This module imports two other modules: mod1, used with its qualified name throughout the document (see line 24), and sub.mod called with an alias mod2 throughout this document (see line 26).

To keep this example simple we do not extend pathvar (see how configuration works in the Syntax Reference), hence mod1 needs to be defined in the same root as the top module:

mod1.zoml

module: mod1

---

root:
  node:
    - name: n_who
      data: Hello World!

whereas sub.mod is defined at the coresponding path:

sub/mod.zoml

module: sub.mod

---

root:
  nodes:
    - name: n1
      zoti-args:
        i-need-this: "This will be replaced by the caller"
      content1: !include {file: test.txt}
      content2: !include {file: test.txt, begin: 3, end: 6}
      content3: !include {file: test.txt, name: block1}
      content4: !include {file: test.txt, begin: ":(!)", end: "(!):"}
      # Recommended way to pass caller arguments
      passed-arg: !attach {ref: !ref {path: "../zoti-args/i-need-this"}}
      # OBS: direct access to caller members is possible but NOT RECOMMENDED
      floating-ref: !ref {path: "../../root[n1]"} 
      

As seen above, module sub.mod references a text file in the same relative path called test.txt and includes raw text from it under its contentX entries, using different directives. This file contains the following text:

sub/test.txt

BEGIN block1
I am the 'block1'.
Fear me!
END block1

:(!)
I am a monkey! Ook!
(!):

Finally, let us define a configuration file in the root of the project so that we do not need to manually pass the respective CLI arguments:

zoticonf.toml

[zoti.yaml.toy]
tool = "toy-example-0.1.0"  # for bookkeeping in node metadata
keys = ["root", "nodes"]    # adds metadata to children of these nodes
ext = [".zoml"]             # checks only files with extension .zoml
pathvar = ["."]             # (not required) does not look for
			    # libraries outside the current path

To process the source files above, we run the command the command:

$ZOML --verbose --spec=toy --out=toy.yaml main

The argument --verbose can be used to print out extensive debug information to stderr and can be used to trace every step in the processing of the source file. It can be ignored for silent output.

Since we declared main to be the top module then the output file toy.yaml will contain the following info:

toy.yaml

---
import:
- {module: mod1}
- {as: mod2, module: sub.mod}
module: main
path: tests/scenario1/main.zoml
tool-log:
- ['2023-04-22 01:35:38.854440', '']
---
root:
- _info:
    _pos:
    - [11, 4, 135, 486, tests/scenario1/main.zoml, '']
  mark: DEFAULT_MARKING
  name: n1
  nodes:
  - _info:
      _pos:
      - [13, 8, 163, 325, tests/scenario1/main.zoml, '']
    name: n1_n1
    nodes:
    - _info:
        _pos:
        - [15, 12, 202, 325, tests/scenario1/main.zoml, '']
      data: I am referenced by n1_n1_n1!
      name: n1_n1_n1
  - _info:
      _pos:
      - [19, 8, 327, 486, tests/scenario1/main.zoml, '']
    data: Hello World!
    extra: I am referenced by n1_n1_n1!
    name: n1_n2
- _info:
    _pos:
    - [6, 6, 43, 624, tests/scenario1/sub/mod.zoml, '']
    - [24, 4, 488, 721, tests/scenario1/main.zoml, ':!attach']
    _prev_attrs: {name: n1}
  content1: |
    BEGIN block1
    I am the 'block1'.
    Fear me!
    END block1

    :(!)
    I am a monkey! Ook!
    (!):
  content2: |
    Fear me!
    END block1

    :(!)
  content3: |
    I am the 'block1'.
    Fear me!
  content4: |
    I am a monkey! Ook!
  floating-ref: {module: main, path: '/root[n1]'}
  mark: DEFAULT_MARKING
  name: n2
  passed-arg: This field is used only to pass caller argument and will be destroyed

The new preamble has additional information such as the original file path and a log. Also, notice that every child of root and nodes contains an _info entry with positional metadata.

The effects of !default (in main.zoml line 8) can be seen in the fact that each 1st level child node (corresponding to the first argument specification) contains a mark: DEFAULT_MARKING entry, as a result of merging the default values under !policy:union (see Syntax Reference).

The !ref command has been resolved each time pointing to a valid node in the module trees, enabling the !attach command to act accordingly. !attach is used in three places, from last to first:

• in main.zoml line 25: attaches the contents of the n1 node from sub.mod under the node n2 in main. The behavior of the !include command in sub/mod.zoml lines 10-13 can be seen in the result. Notice that floating-ref is resolved according to its new location (module resolved to main instead of sub.mod). This node also shows an example of passing an argument from the caller to the calle using a dedicated field zoti-args which is destroyed after resolving the document. Also notice that its metadata contains its entire history and previous attributes.

• in main.zoml line 23: attaches the data entry of node n_who from mod1 to the data entry of n1_n2 from main. This is seen in toy.yaml at line 30. Since the attached node is of type string, it does not contain any metadata.

• in main.zoml line 19: attaches the extra entry of the sibling node n1_n2 from main (implicit) to the data entry of node n1_n1_n1 in the same module. This is seen in toy.yaml at line 25.

This tutorial showcases a typical scenario for ZOTI-YAML where document trees are built from information spread across multiple files. In turn, this should enable any tool downstream to focus on parsing the right information and construct its core objects irrespective to where information originated, while also bookkeeping metadata for debugging purposes. As further shown in the Genny Producer-Consumer Example, these capabilities can be used to (minimally) emulate a component framework where parameters and interfaces can be specified after loading core components from designated libraries.