Skip to main content
A parsing schema is a small JSON document that tells Atomscale how to read a CSV tool log: which column holds the timestamp, which columns to keep as channels, what to call them, and in which units. You load or write it in the Parsing schema section of the Tool Log Streamer configuration. EpiTrend logs don’t need one. You don’t need to write the JSON by hand. Ask the Gateway assistant to draft and preview the schema using a sample of your log. Use this reference if you want to review or edit the schema yourself.

Example

For a log that starts like this:
sample.csv
this schema keeps three channels and ignores the Step column. It is the built-in example you get from Create new parsing schema.
schema.json
To record the growth step instead of ignoring it, move Step from drop_columns to recipe_columns; see Recipe Columns.

Fields

Columns in the file that the schema doesn’t mention are ignored. Unknown fields in the schema are rejected.

Time Axis

Every row needs a timestamp. Choose the mode that matches your log: Time zones. If timestamps include a UTC offset, include %z in the format. If they don’t, Atomscale reads them in the gateway computer’s time zone, and handles the repeated hour when daylight saving time ends. Don’t use %Z (a zone name); it is rejected. See Timestamps and Clocks. Timestamps must not go backward from one row to the next. Rows with equal timestamps are allowed.

Columns

Each entry in columns describes one channel: Columns can be listed in any order; they are matched by header name. Values must be numbers. Text values, such as OPEN or CLOSED, are stored as missing, so record shutter states as 0 and 1 where your software allows.

Recipe Columns

If your log records which recipe and step are running, recipe_columns lets Atomscale split the growth into process steps named after them.
Cells that are empty, -, 0, n/a, na, nan, none, or null mean no recipe is running. A recipe column can’t also be the time axis, a channel, or a dropped column.

More Examples

Separate Date and Time Columns

A semicolon-separated log with a comma decimal separator, in the gateway computer’s local time:

Unix Timestamps

Fields left out of these examples take their defaults.

Validation Rules

Atomscale checks a schema in two stages. When you apply the configuration, the schema’s structure is checked:
  • The file is valid JSON, UTF-8 text, and at most 256 KiB.
  • time_axis.columns has the right number of headers for its mode, and datetime_format is readable and doesn’t use %Z.
  • columns has at least one entry, and each source_name and property_name is unique and not blank.
  • No header appears in more than one of time_axis.columns, columns, drop_columns, and recipe_columns.
  • The delimiter is a single ASCII character other than a quote or newline.
When a file is parsed, the schema is checked against the file:
  • Every header named in the schema must exist in the file’s header row exactly. A mismatch fails the file immediately with Schema columns absent from header.
  • The file’s header names must be unique and not blank.
  • Rows with the wrong number of fields, or with a missing, unreadable, or backward timestamp, are skipped. Non-numeric channel values are stored as missing.
  • After 50 consecutive rows that can’t be used, parsing stops and the data item shows Stream Error.
Write the schema against a real log file from your system, then test it with a short run before relying on it for a growth campaign.