> ## Documentation Index
> Fetch the complete documentation index at: https://docs.atomscale.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# File Watcher

> Upload files from a folder on the gateway computer to Atomscale as they appear

The **File Watcher** integration watches a folder on the gateway computer and uploads each new or changed file to Atomscale once the file stops changing. Use it for instruments that save finished files, such as images, spectra, or exported measurements, to a folder. It replaces the standalone File Watcher app.

For the lifecycle shared by all integrations (configure, start, statuses), see [Working with Integrations](https://docs.atomscale.ai/gateway/integrations.md).

## Before You Start

* Read access, for the Windows user running the gateway, to the folder your instrument saves files to. A network share works if it is connected for that user.
* Instruments that write each file and then leave it alone. A file that keeps growing is not uploaded until it stops changing.

## Set Up the Integration

To have the [Gateway assistant](https://docs.atomscale.ai/gateway/assistant.md#set-up-an-integration) set up File Watcher, give it the folder path and tell it whether to upload files already there.

**1. Add the integration**

Open **Integrations**, click Add Integration, and click
Add on **File Watcher**. Add one copy per folder.

**2. Choose the folder**

Set **Watch Path** with Browse or a full path, such as
`C:\InstrumentData` or `\\server\share\data`. Set **Traversal Depth** to how many levels of
subfolders to include, and **Filename Regex** if you only want some files.

**3. Choose the options**

To upload files already in the folder, turn on Upload Existing Files. To put each subfolder's
files on one physical sample, turn on Group Uploaded Files. To combine matching files into
one data item, set **File Coalescing**. See [Settings](#settings).

**4. Apply and start**

Click Apply Configuration, then
Start Integration.

**5. Confirm uploads**

Save a test file into the folder. After the **File Stability Wait** (5 seconds by default), the
**Activity Logs** show Upload started for … and then Upload completed for …. The file then
appears on the **Data** page (**Data Catalog** in Data Ingestion mode). See [Finding Your Data](https://docs.atomscale.ai/gateway/data.md#where-to-look).

## Moving from the Standalone App

If you used the standalone File Watcher app, set up this integration on the same computer instead:

1. Note the old app's **Watch Path**, **Traversal Depth**, **Group Uploaded Files**, and **File Coalescing** settings. The integration has fields with the same names.
2. Quit and uninstall the standalone app, so the two don't upload the same files.
3. Add the integration, enter those settings, and leave **Upload Existing Files** off so files already uploaded aren't sent again.

Files the old app hadn't uploaded yet won't upload either; upload them with Add Data > Upload. The integration uploads through the desktop app's sign-in, so it needs no API key.

## How Files Are Picked Up

**When a file is ready**: the integration waits until a file's size and modification time have stayed the same for **File Stability Wait (Seconds)**. Each new write restarts the wait, so a file that is appended to continuously uploads only after its writer stops. When the integration starts, files last modified longer ago than the wait are processed right away.

**Files already in the folder**: with **Upload Existing Files** off, the first start records the files already in the folder and doesn't upload them. Only files created or changed afterward upload. Turn it on before the first start to upload existing files too. Pointing **Watch Path** at a different folder starts a new first start for that folder.

**Files added while stopped**: after the first start, the integration remembers each file it has seen. When it starts again, it uploads files that were added or changed while it was stopped.

**Changed files**: a file whose size, modification time, or identity on disk changes uploads again as a new upload. Replacing a file with a new copy counts as a change.

**Copies and renames**: the integration tracks files by path, not by content. A copy saved under another name, a renamed file, or a file moved out of the folder and back in uploads again.

**Deleted files**: deleting a file from the folder doesn't delete it from Atomscale.

**Ignored files**: `.DS_Store` files, and files deeper than **Traversal Depth**.

> [!WARNING]
>
> The upload queue stores a reference to each file, not a copy. Leave files in place and unchanged
> until the activity log shows Upload completed for …. If a file is moved or deleted before
> then, the queue drops it. If a file is rewritten before then, the queue may drop it.

Because the queue holds references, a large backlog of files uses little disk space for the queue itself. See [Disk Buffering](https://docs.atomscale.ai/gateway/uploads.md#disk-buffering).

## Settings

| Field | Default | Description |
| - | - | - |
| **Watch Path** | None | The full path of the folder to watch. Required. Must be an absolute path to a folder that exists on this computer, including a network path such as `\\server\share`. |
| **Traversal Depth** | 2 | Levels of subfolders to watch, from 0 to 64. 0 watches only the folder itself. |
| **Filename Regex** | Blank (all files) | A regular expression matched against each file name, including its extension. For example, `(?i)\.csv$` watches CSV files. |
| **Group Uploaded Files** | Off | Treats each subfolder directly inside the watched folder as one physical sample, named after the subfolder. Files directly in the watched folder aren't grouped. |
| **File Coalescing** | None | File name patterns, such as `img*.jpg`. Matching files in the same folder become one data item instead of separate ones. When several patterns match, the last one wins; a leading `!` excludes. Patterns don't match hidden files unless they start with `.`. |
| **Upload Existing Files** | Off | Uploads files already in the folder when the integration first starts. |
| **Use Polling** (advanced) | Off | Checks the folder on a timer instead of relying on Windows to report changes. Slower, but more reliable on network and shared drives. |
| **File Stability Wait (Seconds)** (advanced) | 5 | Seconds a file must stop changing before it uploads, from 1 to 3,600. Raise it when a tool writes files in slow bursts. Files take at least this long to appear in Atomscale. |
| **Upload Backlog Size Limit**, **Free Disk Space To Keep** (advanced) | Auto, 1 GB | See [Disk Buffering](https://docs.atomscale.ai/gateway/uploads.md#disk-buffering). |

For example, with **Group Uploaded Files** on and **File Coalescing** set to `*.jpg`, a folder `GBL_UNLIT\` containing `img_001.jpg`, `img_002.jpg`, and `metrology.txt` becomes one physical sample with two data items: the images together, and the text file on its own.

## Troubleshooting

Check the integration's **Activity Logs**, and the **Last dropped delivery** field in its **Advanced** view.

| Message or symptom | Cause and fix |
| - | - |
| Watch path is required. Enter the folder this adapter should watch. | **Watch Path** is empty. |
| Watch path "…" was not found. Check the spelling, and that the folder exists on this computer. | The folder doesn't exist, or the Windows user can't reach it. For a network share, check it is connected for that user. |
| Watch path "…" is not a folder. Enter the folder that contains the files to ingest. | **Watch Path** points to a file. Enter its folder. |
| Filename Regex is invalid: … | Correct the regular expression. |
| Coalesce pattern "…" uses the extended-glob group … | Groups such as `!(...)` aren't supported. Use plain wildcards such as `*.csv`, or a leading `!` to exclude. |
| Coalesce patterns cannot contain a blank entry. … | Remove the empty **File Coalescing** entry. |
| File is unavailable before upload: … | The file was moved or deleted before it uploaded. Copy it back as a new file, and leave files in place until they upload. |
| File changed before upload: … (expected … bytes, found …) | The file was rewritten after it was queued. Raise **File Stability Wait (Seconds)** if the tool writes in bursts. |
| New files on a network share aren't noticed | Turn on Use Polling. |
| Files that were already in the folder didn't upload | **Upload Existing Files** was off at the first start. Copy them into the folder again, or move them out and back in. |
| The same file uploaded twice | It was renamed, copied, or modified, or the integration stopped unexpectedly just after queueing it. See [How Files Are Picked Up](#how-files-are-picked-up). |

For upload backlogs and expired or dropped deliveries, see [When Uploads Fall Behind](https://docs.atomscale.ai/gateway/uploads.md#when-uploads-fall-behind) and [Troubleshooting](https://docs.atomscale.ai/gateway/reference/troubleshooting.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.