Connection Pool Configuration

A job queue system opens many concurrent database connections – one per worker goroutine, plus connections for heartbeats, scheduling, and the stale lock reaper. To keep that from overwhelming the database, NewGormStorage installs a bounded default pool automatically (see below), so you get a safe pool without writing any configuration. Pool tuning is an optional optimization on top of that default – you reach for it to match a specific workload (higher concurrency, latency-sensitive paths, or a tight cloud connection cap), not to avoid an unbounded pool.

The simple-durable-jobs library exposes Go’s database/sql pool settings through a clean option-based API so you can tune connection usage to match your workload.

Default Pool (Applied Automatically)

NewGormStorage(db) installs a bounded connection pool unless the pool has already been sized. You do not have to configure anything to be safe – the unbounded-by-default behavior of database/sql is overridden for you.

PostgreSQL and MySQL receive DefaultPoolConfig() values (25/10/5 min/1 min) – the real guard against exhausting a Postgres max_connections limit. SQLite receives a small, single-writer-friendly pool (4 open, 2 idle) with no connection expiry: a zero lifetime/idle-time keeps the sole connection of an in-memory (:memory:) database from being dropped (which would silently discard the database) and keeps WAL connections warm.

The default is applied only when the pool is still unset. The constructor checks the underlying *sql.DB and skips the default when:

• The caller already sized the pool before construction (any non-zero MaxOpenConns, e.g. a db.DB().SetMaxOpenConns(...) or ConfigurePool call made before NewGormStorage).
• Storage is created via NewGormStorageWithPool, which has already sized the pool itself – including an explicit MaxOpenConns(0) meaning unlimited. The auto-default is suppressed in this case so your unlimited choice is honored rather than silently re-bounded.

A nil DB, or a DB whose *sql.DB cannot be obtained (a mock or dialector-only connection), is skipped silently so construction stays panic-free.

Creating Storage with Pool Configuration

Using a Preset

The fastest way to get started is to pick a preset that matches your deployment profile:

storage, err := jobs.NewGormStorageWithPool(db, jobs.WithPoolConfig(jobs.HighConcurrencyPoolConfig()))
if err != nil {
    log.Fatal(err)
}

Customizing Individual Settings

You can mix and match individual pool options:

storage, err := jobs.NewGormStorageWithPool(db,
    jobs.MaxOpenConns(50),
    jobs.MaxIdleConns(20),
    jobs.ConnMaxLifetime(10 * time.Minute),
    jobs.ConnMaxIdleTime(2 * time.Minute),
)
if err != nil {
    log.Fatal(err)
}

Configuring an Existing Database

If you already have a *gorm.DB that you opened elsewhere, you can apply pool settings to it directly and then create storage normally:

err := jobs.ConfigurePool(db, jobs.MaxOpenConns(100))
if err != nil {
    log.Fatal(err)
}
storage := jobs.NewGormStorage(db)

Available Presets

Each preset returns a PoolConfig struct with values tuned for a specific deployment scenario:

DefaultPoolConfig

Suitable for most workloads. Allows up to 25 concurrent database connections with 10 kept warm in the idle pool. Connections are recycled every 5 minutes to prevent stale connections from accumulating.

HighConcurrencyPoolConfig

Designed for deployments running 50 or more worker goroutines. Opens up to 100 connections and keeps 25 idle. Longer lifetimes reduce the overhead of re-establishing connections under sustained load.

LowLatencyPoolConfig

Optimized for workloads where connection acquisition time matters. Keeps 40 of 50 connections warm in the idle pool, so most dequeue and completion operations hit a pre-established connection. Longer idle times prevent unnecessary teardown.

ResourceConstrainedPoolConfig

Use this when your database has strict connection limits (for example, a small managed database instance or a serverless database with a low connection cap). Limits the pool to 10 open and 5 idle connections with aggressive recycling.

Individual Options

Each option sets a single field on the underlying PoolConfig:

Choosing the Right Configuration

Here are guidelines to help you pick the right settings:

Match MaxOpenConns to Worker Concurrency

Every worker goroutine may hold a database connection while processing a job. Add connections for heartbeats, the stale lock reaper, the scheduler, and the dequeue poll loop. A safe starting point is:

MaxOpenConns >= total worker concurrency + 10

For example, if you run three queues with concurrency 20, 10, and 5 (total 35), set MaxOpenConns to at least 45.

Size the Idle Pool at 25-50% of MaxOpenConns

Idle connections avoid the latency of establishing a new TCP connection and TLS handshake. For typical workloads, keeping 25-50% of connections warm provides a good balance between responsiveness and resource usage.

Set ConnMaxLifetime to 5-15 Minutes

Recycling connections periodically prevents issues with stale TCP sessions, load-balancer idle timeouts, and DNS changes in cloud environments. Five minutes is a good default; increase to 10-15 minutes under sustained high throughput to reduce churn.

Respect Cloud Database Limits

Many managed database services impose connection limits (for example, a small PostgreSQL instance may allow only 100 connections shared across all applications). Use ResourceConstrainedPoolConfig or manually set MaxOpenConns to stay within your quota. Remember that each application replica has its own pool, so divide the limit across replicas.