Skip to main content
This page sets up the Tool Log Streamer integration. Before you start, check that your logs meet the requirements.

Before You Start

  • A gateway with read access to the folder where the tool’s control software writes its logs.
  • For CSV logs, a sample log file from your system to draft and check the parsing schema.

Set up with the assistant

Open the Gateway assistant and choose Stream tool logs in a new chat, or write your own request. Give it the log folder and format. For CSV logs, use a sample to draft and check the schema before applying it:
  1. Give the assistant the full path to a representative CSV log on the gateway. For example:
    Set up Tool Log Streamer for CSV files in D:\MBE\Logs. Use D:\MBE\Logs\sample.csv to draft a parsing schema. Keep substrate temperature in °C and chamber pressure in Torr. Show me the preview before applying the schema.
    If the assistant can’t read the file, paste the start of a sample you can share. Include any preamble, the header, and several data rows.
  2. Check the parsed timestamps, channel names, units, and any rejected rows in the preview. Ask the assistant to correct any wrong assumptions and run the preview again. Previewing doesn’t change the integration.
  3. Ask the assistant to apply the schema, then approve the change. Once the configuration is saved, start the integration when you’re ready. When the tool writes a new log, plot a channel to check that data arrives.
EpiTrend logs don’t need a parsing schema. Give the assistant the EpiTrendData folder instead.

Set Up the Integration

To configure Tool Log Streamer yourself, follow the steps for your log format.
1

Add the integration

Open Integrations, click Add Integration, and click Add on Tool Log Streamer. Add one copy per log folder.
2

Choose the format and folder

The configuration page opens after you add the integration. If you added it during First-Run Setup, click it on Integrations and click Configure. Leave Format on CSV, and set Watch Path to the folder where logs are written, with Browse or an absolute path.
3

Adjust the folder scan if needed

The defaults watch .csv files up to two subfolder levels deep, and end a file’s capture after it hasn’t grown for 600 seconds. If your logs differ, turn on Show advanced settings and adjust Traversal Depth, Filename Regex (for files that don’t end in .csv), or Inactivity Timeout (Seconds). The timeout must be longer than the longest normal pause between writes during a run.
4

Add a parsing schema

Ask the Gateway assistant to draft and check a schema, or add one yourself. Under Parsing schema, click Browse to load a schema file, or Create new parsing schema to start from the built-in example. Edit it so the column names match your log’s header exactly. See the Parsing Schema Reference.
5

Apply and start

Click Apply Configuration. Atomscale checks the schema and saves a copy with the integration. Then click Start Integration.
6

Verify with a run

When the tool software starts a new log, a Tool Logs - … data item starts uploading as Stream Active. Its chart starts empty: right-click the chart, or click its gear, and click + next to a channel to plot it. The chart then fills in as rows are written.
By default, CSV files already in the folder when the integration first starts are not uploaded. To backfill them, turn on Show advanced settings and Upload Existing Files before the first start. Existing files that grow later are picked up either way.

Working with the Parsing Schema

The schema control shows the active schema with Preview Schema and Edit Schema views. In the editor, Format JSON tidies the text, and How to write a parsing schema summarizes the rules.
  • Loading a file takes a snapshot. Apply Configuration validates and saves that snapshot. Editing the source file afterward doesn’t change the saved schema; load it again and apply.
  • Each data item keeps the schema that was active when its file started. A schema change applies to new files only.
  • The schema is saved with this integration on this gateway. Import configuration doesn’t include it, so keep your schema file somewhere safe.
To test a CSV schema without the tool, turn on Show advanced settings, set Source to Synthetic (Testing), apply your schema, and start the integration. It writes a test CSV with one row per second from the schema’s columns. Synthetic data needs date-time timestamps (not Unix seconds), and doesn’t write recipe columns. Switch back to Files when you’re done.

Settings Reference

Only Format, Watch Path, and the parsing schema show by default. Fields marked advanced appear with Show advanced settings turned on.

Example Filename Patterns

Fixing a File That Failed

Parsing problems don’t stop the raw upload, but a file that failed parsing can’t be fixed in place:
  1. Stop writing to the problem file.
  2. Stop the integration, correct the parsing schema, and apply it.
  3. Start the integration again and start a new log file.
Only new files use the corrected schema. The original file’s raw bytes stay stored in its data item.

Troubleshooting

Check the integration’s Activity Logs on the gateway, and the data item’s status in the web app.