Spring Kafka Parameters

Spring Kafka Producer Parameters

acks

Specifies the acknowledgment level required before the producer considers a request complete. Common values include 0 (no acknowledgment), 1 (only the leader acknowledges), and all (all replicas acknowledge).

spring.kafka.producer.acks=all

Recommended value: all (to ensure high reliability)

retries

Specifies the number of retry attempts when a send fails. This helps enhance the reliability of message delivery.

spring.kafka.producer.retries=3

Recommended value: 3 or more, depending on application fault tolerance needs.

batch.size

Specifies the size (in bytes) of the batch sent by the producer. Increasing this value can boost throughput but may add latency.

spring.kafka.producer.batch-size=16384

Recommended value: 16384 (16KB), adjustable based on actual requirements.

linger.ms

Sets the time the producer waits before sending a batch of messages. Increasing this time can gather more messages to improve throughput.

spring.kafka.producer.linger-ms=1

Recommended value: 1 (to reduce latency), can be adjusted higher to increase throughput.

Spring Kafka Consumer Parameters

auto.offset.reset

Specifies what the consumer should do when it cannot find the initial offset or if the current offset does not exist on the server. Options include earliest (reads from the earliest available data) and latest (reads from the latest data).

spring.kafka.consumer.auto-offset-reset=earliest

Recommended value: earliest (to ensure no messages are missed)

enable.auto.commit

Controls whether offsets are automatically committed. If set to true, the consumer periodically commits offsets. Set to false if you prefer to manage offset commits manually.

spring.kafka.consumer.enable-auto-commit=false

Recommended value: false (for precise manual control of commits)

auto.commit.interval.ms

Controls the interval for automatically committing offsets. This parameter specifies the interval (in milliseconds) at which offsets are automatically committed when enable.auto.commit is set to true.

If you set the auto-commit interval to 5000 milliseconds (5 seconds), you can configure it as follows:

spring.kafka.consumer.enable-auto-commit=true
spring.kafka.consumer.auto-commit-interval=5000

The recommended value depends on your application scenario:

• Low-latency applications: For low latency with tolerance for more frequent reprocessing, set a shorter interval, such as 1000 milliseconds (1 second).
• High-throughput applications: To minimize commit overhead, set a longer interval, such as 5000 milliseconds (5 seconds) or longer.

max.poll.records

Specifies the maximum number of records returned in a single call to poll(). This helps control the amount of data processed per poll to avoid delays.

spring.kafka.consumer.max-poll-records=500

Recommended value: 500 (adjust based on throughput needs)

max.poll.interval.ms

Controls the maximum time between consecutive poll calls. This parameter ensures the consumer calls poll() within the specified interval. If the interval is exceeded without calling poll(), Kafka considers the consumer inactive and reassigns the partition to another group member.

spring.kafka.consumer.max-poll-interval=300000

The recommended value depends on your application scenario:

• Low-latency applications: For quick detection of consumer inactivity, set a shorter interval, such as 30000 milliseconds (30 seconds).
• High-throughput applications: For complex processing logic requiring more time, set a longer interval, such as 300000 milliseconds (5 minutes) or longer.

session.timeout.ms

Specifies the session timeout between the consumer and Kafka server. If no heartbeat is received within this time, Kafka considers the consumer inactive and rebalances the partitions.

spring.kafka.consumer.session.timeout.ms=10000

Recommended value: 10000 milliseconds (10 seconds), adjustable based on actual needs.

request.timeout.ms

Controls the request timeout for consumer requests to the Kafka server. This parameter specifies the maximum time to wait for a response. If no response is received within this time, the request is considered failed and error handling (like retries) is triggered.

spring.kafka.consumer.request.timeout.ms=30000

The recommended value depends on your application scenario:

• Low-latency applications: For quick timeout detection, set a shorter interval, such as 10000 milliseconds (10 seconds).
• High-reliability applications: For better network conditions and response tolerance, set a longer interval, such as 60000 milliseconds (60 seconds).