Skip to main content
The File Watcher integration watches a folder on the gateway computer and uploads each new or changed file to Atomscale once the file stops changing. Use it for instruments that save finished files, such as images, spectra, or exported measurements, to a folder. It replaces the standalone File Watcher app. For the lifecycle shared by all integrations (configure, start, statuses), see Working with Integrations.

Before You Start

  • Read access, for the Windows user running the gateway, to the folder your instrument saves files to. A network share works if it is connected for that user.
  • Instruments that write each file and then leave it alone. A file that keeps growing is not uploaded until it stops changing.

Set Up the Integration

To have the Gateway assistant set up File Watcher, give it the folder path and tell it whether to upload files already there.
1

Add the integration

Open Integrations, click Add Integration, and click Add on File Watcher. Add one copy per folder.
2

Choose the folder

Set Watch Path with Browse or a full path, such as C:\InstrumentData or \\server\share\data. Set Traversal Depth to how many levels of subfolders to include, and Filename Regex if you only want some files.
3

Choose the options

To upload files already in the folder, turn on Upload Existing Files. To put each subfolder’s files on one physical sample, turn on Group Uploaded Files. To combine matching files into one data item, set File Coalescing. See Settings.
4

Apply and start

Click Apply Configuration, then Start Integration.
5

Confirm uploads

Save a test file into the folder. After the File Stability Wait (5 seconds by default), the Activity Logs show Upload started for … and then Upload completed for …. The file then appears on the Data page (Data Catalog in Data Ingestion mode). See Finding Your Data.

Moving from the Standalone App

If you used the standalone File Watcher app, set up this integration on the same computer instead:
  1. Note the old app’s Watch Path, Traversal Depth, Group Uploaded Files, and File Coalescing settings. The integration has fields with the same names.
  2. Quit and uninstall the standalone app, so the two don’t upload the same files.
  3. Add the integration, enter those settings, and leave Upload Existing Files off so files already uploaded aren’t sent again.
Files the old app hadn’t uploaded yet won’t upload either; upload them with Add Data > Upload. The integration uploads through the desktop app’s sign-in, so it needs no API key.

How Files Are Picked Up

When a file is ready: the integration waits until a file’s size and modification time have stayed the same for File Stability Wait (Seconds). Each new write restarts the wait, so a file that is appended to continuously uploads only after its writer stops. When the integration starts, files last modified longer ago than the wait are processed right away. Files already in the folder: with Upload Existing Files off, the first start records the files already in the folder and doesn’t upload them. Only files created or changed afterward upload. Turn it on before the first start to upload existing files too. Pointing Watch Path at a different folder starts a new first start for that folder. Files added while stopped: after the first start, the integration remembers each file it has seen. When it starts again, it uploads files that were added or changed while it was stopped. Changed files: a file whose size, modification time, or identity on disk changes uploads again as a new upload. Replacing a file with a new copy counts as a change. Copies and renames: the integration tracks files by path, not by content. A copy saved under another name, a renamed file, or a file moved out of the folder and back in uploads again. Deleted files: deleting a file from the folder doesn’t delete it from Atomscale. Ignored files: .DS_Store files, and files deeper than Traversal Depth.
The upload queue stores a reference to each file, not a copy. Leave files in place and unchanged until the activity log shows Upload completed for …. If a file is moved or deleted before then, the queue drops it. If a file is rewritten before then, the queue may drop it.
Because the queue holds references, a large backlog of files uses little disk space for the queue itself. See Disk Buffering.

Settings

For example, with Group Uploaded Files on and File Coalescing set to *.jpg, a folder GBL_UNLIT\ containing img_001.jpg, img_002.jpg, and metrology.txt becomes one physical sample with two data items: the images together, and the text file on its own.

Troubleshooting

Check the integration’s Activity Logs, and the Last dropped delivery field in its Advanced view. For upload backlogs and expired or dropped deliveries, see When Uploads Fall Behind and Troubleshooting.