Scaling Self-Hosted n8n: Infrastructure and Database Tuning

When moving from proof-of-concept automations to handling millions of monthly executions, scaling self-hosted n8n becomes a critical infrastructure challenge. Out-of-the-box, n8n runs in a single-instance configuration that writes every execution step to its database. While this is excellent for debugging, under heavy loads it quickly leads to database CPU exhaustion, locked tables, and degraded execution times.

Successfully scaling self-hosted n8n requires transitioning from default settings to a decoupled, queue-based architecture, alongside strict database retention and resource allocation strategies. At botifyit, we help businesses transition from fragile single-instance setups to highly available, resilient automation platforms through our custom automation services.

Why Default n8n Deployments Fail at Scale

In a default installation, n8n uses an SQLite database (or a small Postgres instance) and runs in "own" mode, where the main application process handles the UI, API, scheduler, and execution engine.

This setup introduces several bottlenecks:

1. CPU/Memory Starvation: A memory-intensive JavaScript node can crash the entire application, killing other active executions.
2. Database Write Amplification: n8n saves the input, output, and binary metadata of every single node in an execution path. Under sustained volume, this results in massive write operations that degrade database performance.
3. Lack of Concurrency: In own mode, the execution queue is constrained by the single node process's event loop, throttling throughput.

To build a platform capable of supporting enterprise workloads—such as our e-commerce operations scaling work which handles high-volume fulfillment and cart recovery—you must decouple execution from orchestration.

Moving to Queue Mode with Redis

To scale horizontally, you must run n8n in Queue Mode. This splits the application into three distinct components:

• n8n Main (Webhooks and UI): Receives incoming HTTP requests, handles cron triggers, and hosts the visual editor. It writes execution jobs to a Redis instance instead of running them directly.
• Redis: Acts as the message broker, distributing execution jobs to available workers.
• n8n Workers: Headless instances dedicated solely to pulling jobs from Redis, executing the workflow logic, and writing results back to the database.

# docker-compose.queue-mode.yml
version: '3.8'
services:
  redis:
    image: redis:7-alpine
    restart: always
    volumes:
      - redis_data:/data

  n8n-main:
    image: docker.n8n.io/n8nio/n8n:latest
    command: start
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
    ports:
      - 5678:5678

  n8n-worker:
    image: docker.n8n.io/n8nio/n8n:latest
    command: worker
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres

Using queue mode ensures that if a worker crashes due to a resource-heavy execution, the main n8n instance remains online to accept incoming webhooks, preventing data loss. For workflows handling critical external data, we also recommend pairing queue mode with dead-letter queues to isolate and retry failed runs.

Database Tuning for High Write Volume

Even with queue workers, Postgres can easily become the primary bottleneck. Every execution state transition triggers writes to the execution_entity table. If left unconfigured, index maintenance and table bloat will eventually degrade query performance.

1. Tune Postgres Connection Pools

By default, n8n workers open multiple connections to Postgres. In a multi-worker setup, you can quickly exceed the max_connections limit of your database. Utilize a connection pooler like PgBouncer in front of your database, or fine-tune the following environment variables on your workers and main instances:

• DB_POSTGRESDB_POOL_MAX_SIZE: Set this to a lower value (e.g., 5 to 10 per worker) to prevent connection starvation.
• Optimize write operations directly by applying strategies detailed in our guide on optimizing Postgres queues.

2. Configure Aggressive Execution Pruning

To prevent your Postgres database from growing exponentially, you must enforce strict data retention. n8n allows you to configure pruning via environment variables.

We recommend disabling successful execution logging for high-frequency workflows and keeping failed execution logs for a limited period:

# Disable saving successful executions globally (or configure per-workflow)
EXECUTIONS_DATA_PRUNE=true
EXECUTIONS_DATA_MAX_AGE=168 # Keep data for 7 days (in hours)
EXECUTIONS_DATA_PRUNE_TIMEOUT=3600 # Run pruning job every hour

If you have workflows running every minute, even 7 days of logs can accumulate millions of rows. For these high-frequency pipelines, configure the individual workflow settings to not save execution data unless the run fails.

Optimizing Worker Resources and Concurrency

Each n8n worker runs in a single-threaded Node.js process. By default, a worker processes one execution at a time. You can increase throughput by tuning the concurrency limit per worker:

# Increase concurrency on high-performance worker nodes
N8N_WORKERS_CONCURRENCY=10

However, setting concurrency too high can lead to memory exhaustion (OOM crashes) if your workflows parse large JSON payloads or handle binary files.

• Memory Limit Allocation: Always run your workers in containers with explicit memory limits (e.g., 2GB to 4GB).
• Process Isolation: Set N8N_ENFORCE_SETTINGS_FILE_PER_USER=true to secure multi-tenant environments, and use isolated Docker networks to segregate your database, Redis, and worker containers.

Next Steps for Production Readiness

Scaling self-hosted n8n requires a shift from a "run-everything-anywhere" mindset to a structured, decoupled platform. Implement Redis queue mode, configure aggressive execution pruning, and tune your Postgres pool size before production traffic exposes structural bottlenecks.

If you are scaling complex, business-critical integrations and need expert engineering support to optimize your database, configure resilient queues, or build scalable pipelines, explore our case studies to see how we build production-grade automation systems for our clients.