Skip to main content
Version: Next

Linux Host Configuration for High-Performance MQTT

When running Mosquitto in production (especially within Docker or Kubernetes—standard Linux kernel) settings are often too conservative. High-concurrency workloads (10k+ connections) can cause the host system to drop packets or refuse connections, even if the Mosquitto broker itself is healthy.

This guide details how to tune the Linux host to support high-throughput MQTT traffic.

1. Network Connection Tracking (Netfilter)​

Crucial for Docker deployments.

Docker uses the Linux kernel's netfilter framework (specifically NAT) to route packets between the host and containers. The kernel tracks every active connection in a table. If this table fills up, the kernel will silently drop new packets (nf_conntrack: table full, dropping packet).

Step A: Ensure Kernel Module Persistence​

The Problem: sysctl settings for connection tracking will fail at boot if the nf_conntrack kernel module is not yet loaded.

The Solution: We must explicitly force the module to load before the network configuration is applied.

  1. Create a configuration file to load the module at boot:

    echo "nf_conntrack" | sudo tee /etc/modules-load.d/mqtt-tuning.conf

  2. (Optional) If you are using IPv6, you may also want to load:

    echo "nf_conntrack_ipv6" | sudo tee -a /etc/modules-load.d/mqtt-tuning.conf

Step B: Configure Sysctl Parameters​

We will increase the maximum number of tracked connections and reduce the time closed connections remain in the "tracking" state.

Create or edit the file /etc/sysctl.d/99-mqtt-tuning.conf:

# /etc/sysctl.d/99-mqtt-tuning.conf

# -------------------------------------------------------------
# 1. Connection Tracking Limits
# -------------------------------------------------------------
# Increase the max number of connections the kernel can track.
# Default is often 65,536. We recommend 1 Million for IoT workloads.
# Note: We set both names to cover different kernel versions.
net.netfilter.nf_conntrack_max = 1000000
net.nf_conntrack_max = 1000000

# -------------------------------------------------------------
# 2. Timeout Optimizations
# -------------------------------------------------------------
# Reduce the time a connection stays in TIME_WAIT state.
# This helps recycle connection slots faster during high churn.
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30

# Increase the TCP backlog (pending connections) to handle bursts.
net.core.somaxconn = 4096
net.ipv4.tcp_max_syn_backlog = 4096

# -------------------------------------------------------------
# 3. Port Range
# -------------------------------------------------------------
# Widen the local port range to allow more outgoing connections
# (Useful if the broker also acts as a bridge or client).
net.ipv4.ip_local_port_range = 1024 65535

2. File Descriptor Limits (Open Files)​

Every MQTT connection consumes a file descriptor (FH) on the Linux host. The default limit is usually 1,024, which is far too low.

Step A: System-wide Limit​

Ensure the kernel can handle the total number of open files. Add this to /etc/sysctl.d/99-mqtt-tuning.conf (if not already present):

# Allow 2 million open files system-wide
fs.file-max = 2097152

Step B: User/Session Limits​

Configure limits for user sessions (required for debugging or running Mosquitto directly). Edit /etc/security/limits.d/mqtt-tuning.conf:

# <domain>      <type>  <item>         <value>
* soft    nofile         100000
* hard    nofile         100000
root            soft    nofile         100000
root            hard    nofile         100000

Note for Docker Users: Docker containers do not inherit limits from /etc/security/limits.conf. You must set ulimits in your docker-compose.yml or docker run command explicitly (e.g., --ulimit nofile=100000:100000).

3. Applying and Verifying Changes​

Apply Changes​

You do not need to reboot. Run the following to apply changes immediately:

# 1. Manually load the module (for the current session)
sudo modprobe nf_conntrack

# 2. Apply the sysctl parameters
sudo sysctl --system

Verification​

Always verify that the settings have taken effect.

  1. Check Module Status:

    lsmod | grep nf_conntrack
    # Output should show 'nf_conntrack' and 'nf_defrag_ipv4'

  2. Check Connection Limit:

    sysctl net.netfilter.nf_conntrack_max
    # Expected: net.netfilter.nf_conntrack_max = 1000000

  3. Monitor Current Usage: Use this command to see how many connections are currently being tracked:

    cat /proc/sys/net/netfilter/nf_conntrack_count