Integration Sync Failures

Every integration sync runs as a sync job — a tracked unit of work with its own state machine. The system supports three job types:

Job state machine:

PENDING → RUNNING → COMPLETED
                  ↘ PENDING (retry)
                  ↘ FAILED (max retries exceeded)
         CANCELLED (user-initiated)

Each job exposes real-time progress in the dashboard: total items, processed items, failed items, and the number of memories created, updated, or deleted by the run.

Each tenant has a cap on how many sync jobs may run at the same time. This prevents one tenant from monopolizing platform resources and keeps scheduling fair for everyone.

When the limit is hit, attempts to start additional jobs are rejected with a clear “too many concurrent syncs” error.

How to fix:

• Wait for a running job to complete before starting a new one.
• Cancel a stuck or unneeded job from the dashboard.
• Stagger your sync schedules across connections so they do not overlap.

Idempotency protection: Each sync job creation accepts an optional idempotency key. If a job with the same key already exists, the create request is rejected so a scheduler firing twice (for example, due to clock skew) cannot create duplicate jobs.

When a sync job fails, the platform retries it a small number of times with increasing delays. The delays are designed to ride out transient external API outages that typically resolve within a short window.

Between retries: The job goes back into a pending state with a scheduled retry time, the retry count is incremented, and the latest failure reason is surfaced on the job.

After all retries are exhausted: The job is marked as failed and the connection’s health status is updated so the dashboard surfaces the connection as unhealthy.

Incremental syncs use a cursor to track progress and avoid re-fetching already-synced data. The cursor falls back gracefully when one is not available:

1. Primary: the cursor from the last completed sync.
2. Fallback: the completion timestamp of the last completed sync, used as a “changes since” marker.
3. No history: if there has never been a completed sync, the incremental run effectively becomes a full sync.

Common cursor issues:

• “Incremental sync isn’t catching new changes.” If the cursor is correct but the external API is not returning changes, the issue is on the external source’s side (pagination, cursor format change, etc.).
• “Incremental sync re-fetches everything.” This happens when no usable cursor or completion timestamp is available, or when the provider does not support time-based change detection.

Fix for cursor drift: run a full sync to reset the baseline, then let incremental syncs resume from the fresh cursor.

Sync jobs process objects in batches. When an individual object fails, it does not abort the entire job — the failure is recorded for that object and the job continues with the rest.

What you can see for each per-object failure:

• The object’s identifier in the external system.
• The object type (for example, “document” or “page”).
• A human-readable error description.
• A snapshot of the object payload to help reproduce the issue.

Interpreting job results: After a job completes, check the failed-items counter. If it is non-zero, some objects failed individually. The job’s overall status can still be completed even with individual object failures — the job-level status reflects whether the sync process itself completed, not whether every object succeeded.

Before a sync job can start, two preconditions must be met:

1. The connection must be active. If the connection is in any other state (disconnected, error, or revoked), the sync job is rejected with a “connection not active” error.

2. A valid access token must be available. The platform refreshes tokens before they expire. If a refresh fails (for example, the upstream provider revoked the refresh token), the next sync job surfaces a “no access token available” error.

Troubleshooting:

• “Connection not active” — re-authorize the integration from the dashboard. The OAuth flow re-establishes the connection.
• “No access token available” — your refresh token has been revoked or has expired. Re-authorize the connection from the dashboard.

The job status endpoint returns real-time progress, timing, and error information for an in-flight sync. A typical response includes:

• The job’s current state and overall percent complete.
• Counts for total, processed, and failed items.
• Counts for memories created, updated, and deleted by the run.
• Start time, completion time (if finished), and total duration.
• The latest error message, if any.

Sync scheduling: After a sync completes, the connection’s next scheduled sync is updated based on the configured frequency: