> ## Documentation Index
> Fetch the complete documentation index at: https://docs.atomscale.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Uploads and Buffering

> How a gateway queues, prioritizes, and uploads data, and what to do when uploads fall behind

Each integration writes what it collects to its own queue on disk, and the gateway uploads from those queues as bandwidth allows. This keeps data safe through network outages, sign-outs, and app restarts.

```text theme={null}
Integration ─► upload queue ─► gateway ─► Atomscale
               (disk)          uploads
               │               │
               │ bounded by    │ timeseries first,
               │ backlog limit │ then files, within
               │ and free-space│ the bandwidth limit
               │ floor         │
```

## Upload Limits

Upload bandwidth and parallelism apply to the whole computer. Change them in the desktop app under **Settings > Data Ingestion**, or remotely from the gateway's **Settings** in the web app (see [Remote Management](https://docs.atomscale.ai/gateway/remote-management.md#settings)). Changes apply immediately.

| Setting | Options | Default |
| - | - | - |
| **Upload bandwidth** | 250 Mbps, 500 Mbps, 1000 Mbps, or Unlimited | Unlimited |
| **Concurrent file uploads** | 2, 4, 8, or 16 uploads | 8 uploads |

The bandwidth limit applies to file uploads, which include RHEED frames and tool log files. Timeseries uploads go first: while one is in progress, new file uploads wait, so live charts stay responsive. When several integrations have files waiting, the next upload slot goes to the integration with the fewest uploads in progress.

Files over 100 MiB upload in 50 MiB parts, so an interrupted transfer resumes from the last completed part.

## Disk Buffering

Two advanced settings, present on every integration, bound its queue. Both are in bytes.

| Setting | Default | Description |
| - | - | - |
| **Upload Backlog Size Limit** | Blank (Auto) | The most data the queue may hold. Auto lets it grow until it reaches the free-space floor. |
| **Free Disk Space To Keep** | `1000000000` (1 GB) | The free space the queue always leaves on the drive. The minimum is 1 GB. |

When the queue nears a limit, the **Upload Queue** shows a Backpressure badge. When it reaches one, the integration stops accepting new data until uploads drain or disk space frees. Queued data is never discarded to make room.

> [!WARNING]
>
> A full queue means new instrument data is not being collected. For RHEED, frames that arrive while
> the queue is full are dropped. Size the disk and the limits for the longest outage you need to
> survive.

Uploads retry automatically after network errors, waiting from 1 second up to 30 seconds between attempts. Data that can't be uploaded within **30 days** is discarded. See [Keep the Gateway Running](https://docs.atomscale.ai/gateway/keep-running.md) for what happens on sign-out and restart.

## When Uploads Fall Behind

On the integration's page, **Health** shows **Last successful upload**, **Upload Queue** shows what is waiting, and the **Advanced** view shows **Last upload error** and **Retry attempt**.

| Symptom | What to do |
| - | - |
| **Last upload error** shows network errors or timeouts | The gateway can't reach Atomscale or Amazon S3. Check the hostnames in [Network Requirements](https://docs.atomscale.ai/gateway/reference/network.md), including the S3 buckets, which some firewalls block separately. |
| The queue drains slowly, with no errors | The bandwidth limit may be too low for the data rate. Raise **Upload bandwidth**. |
| Uploads paused | Sign in with an account from the gateway's owning organization. See [Sign In](https://docs.atomscale.ai/gateway/first-run.md#3-sign-in). |
| Backpressure, or The connector's upload queue is at capacity. | Fix the upload problem first so the queue drains. If it drains but is too small for your data rate or outage length, free disk space or raise **Upload Backlog Size Limit**. |
| Delivery expired after 30 days without acknowledgement | Data waited more than 30 days and was discarded. Find out why the gateway was offline or signed out for that long. |
| **Last dropped delivery** shows a reason | Atomscale permanently rejected that data, for example as malformed, and removed it from the queue. If it repeats, send the reason and the [logs](https://docs.atomscale.ai/gateway/reference/files-and-logs.md#collecting-logs-for-support) to Atomscale support. |
| Hub unreachable | The gateway uploads through a Connection Sharing Hub that is down. See [Connection Sharing](https://docs.atomscale.ai/gateway/connection-sharing.md#troubleshooting). |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.