Before You Start
- Read access, for the Windows user running the gateway, to the folder your instrument saves files to. A network share works if it is connected for that user.
- Instruments that write each file and then leave it alone. A file that keeps growing is not uploaded until it stops changing.
Set Up the Integration
To have the Gateway assistant set up File Watcher, give it the folder path and tell it whether to upload files already there.1
Add the integration
Open Integrations, click Add Integration, and click
Add on File Watcher. Add one copy per folder.
2
Choose the folder
Set Watch Path with Browse or a full path, such as
C:\InstrumentData or \\server\share\data. Set Traversal Depth to how many levels of
subfolders to include, and Filename Regex if you only want some files.3
Choose the options
To upload files already in the folder, turn on Upload Existing Files. To put each subfolder’s
files on one physical sample, turn on Group Uploaded Files. To combine matching files into
one data item, set File Coalescing. See Settings.
4
Apply and start
Click Apply Configuration, then
Start Integration.
5
Confirm uploads
Save a test file into the folder. After the File Stability Wait (5 seconds by default), the
Activity Logs show Upload started for … and then Upload completed for …. The file then
appears on the Data page (Data Catalog in Data Ingestion mode). See Finding Your Data.
Moving from the Standalone App
If you used the standalone File Watcher app, set up this integration on the same computer instead:- Note the old app’s Watch Path, Traversal Depth, Group Uploaded Files, and File Coalescing settings. The integration has fields with the same names.
- Quit and uninstall the standalone app, so the two don’t upload the same files.
- Add the integration, enter those settings, and leave Upload Existing Files off so files already uploaded aren’t sent again.
How Files Are Picked Up
When a file is ready: the integration waits until a file’s size and modification time have stayed the same for File Stability Wait (Seconds). Each new write restarts the wait, so a file that is appended to continuously uploads only after its writer stops. When the integration starts, files last modified longer ago than the wait are processed right away. Files already in the folder: with Upload Existing Files off, the first start records the files already in the folder and doesn’t upload them. Only files created or changed afterward upload. Turn it on before the first start to upload existing files too. Pointing Watch Path at a different folder starts a new first start for that folder. Files added while stopped: after the first start, the integration remembers each file it has seen. When it starts again, it uploads files that were added or changed while it was stopped. Changed files: a file whose size, modification time, or identity on disk changes uploads again as a new upload. Replacing a file with a new copy counts as a change. Copies and renames: the integration tracks files by path, not by content. A copy saved under another name, a renamed file, or a file moved out of the folder and back in uploads again. Deleted files: deleting a file from the folder doesn’t delete it from Atomscale. Ignored files:.DS_Store files, and files deeper than Traversal Depth.
Because the queue holds references, a large backlog of files uses little disk space for the queue itself. See Disk Buffering.
Settings
For example, with Group Uploaded Files on and File Coalescing set to
*.jpg, a folder GBL_UNLIT\ containing img_001.jpg, img_002.jpg, and metrology.txt becomes one physical sample with two data items: the images together, and the text file on its own.
Troubleshooting
Check the integration’s Activity Logs, and the Last dropped delivery field in its Advanced view.
For upload backlogs and expired or dropped deliveries, see When Uploads Fall Behind and Troubleshooting.