Options
All
  • Public
  • Public/Protected
  • All
Menu

Class KafkaConsumer<K, V>

Vert.x Kafka consumer.

You receive Kafka records by providing a {@link KafkaConsumer#handler}. As messages arrive the handler will be called with the records.

The {@link KafkaConsumer#pause} and {@link KafkaConsumer#resume} provides global control over reading the records from the consumer.

The {@link KafkaConsumer#pause} and {@link KafkaConsumer#resume} provides finer grained control over reading records for specific Topic/Partition, these are Kafka's specific operations.

Type parameters

  • K

  • V

Hierarchy

  • KafkaConsumer

Implements

  • any

Index

Methods

assign

  • Manually assign a partition to this consumer.

    Due to internal buffering of messages, when reassigning the old partition may remain in effect (as observed by the record handler)} until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new partition.

    Parameters

    Returns PromiseLike<void>

  • Manually assign a partition to this consumer.

    Due to internal buffering of messages, when reassigning the old partition may remain in effect (as observed by the record handler)} until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new partition.

    Parameters

    • topicPartition: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

  • Manually assign a list of partition to this consumer.

    Due to internal buffering of messages, when reassigning the old set of partitions may remain in effect (as observed by the record handler)} until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new set of partitions.

    Parameters

    Returns PromiseLike<void>

  • Manually assign a list of partition to this consumer.

    Due to internal buffering of messages, when reassigning the old set of partitions may remain in effect (as observed by the record handler)} until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new set of partitions.

    Parameters

    • topicPartitions: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

assignment

  • Get the set of partitions currently assigned to this consumer.

    Returns PromiseLike<TopicPartition>

  • Get the set of partitions currently assigned to this consumer.

    Parameters

    Returns KafkaConsumer<K, V>

batchHandler

  • Set the handler to be used when batches of messages are fetched from the Kafka server. Batch handlers need to take care not to block the event loop when dealing with large batches. It is better to process records individually using the [#handler(Handler) record handler] KafkaConsumer.

    Parameters

    Returns KafkaConsumer<K, V>

beginningOffsets

  • beginningOffsets(topicPartition: TopicPartition): PromiseLike<number>
  • beginningOffsets(topicPartition: TopicPartition, handler: ((res: AsyncResult<number>) => void) | Handler<AsyncResult<number>>): void
  • Get the first offset for the given partitions.

    Parameters

    Returns PromiseLike<number>

  • Get the first offset for the given partitions.

    Parameters

    • topicPartition: TopicPartition
    • handler: ((res: AsyncResult<number>) => void) | Handler<AsyncResult<number>>

    Returns void

close

  • close(): PromiseLike<void>
  • close(completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): void
  • Close the consumer

    Returns PromiseLike<void>

  • Close the consumer

    Parameters

    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns void

commit

  • commit(): PromiseLike<void>
  • commit(completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): void
  • Commit current offsets for all the subscribed list of topics and partition.

    Returns PromiseLike<void>

  • Commit current offsets for all the subscribed list of topics and partition.

    Parameters

    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns void

committed

  • Get the last committed offset for the given partition (whether the commit happened by this process or another).

    Parameters

    Returns PromiseLike<OffsetAndMetadata>

  • Get the last committed offset for the given partition (whether the commit happened by this process or another).

    Parameters

    Returns void

demand

  • demand(): number
  • Returns the current demand.

      If the stream is in flowing mode will return MAX_VALUE.
    • If the stream is in fetch mode, will return the current number of elements still to be delivered or 0 if paused.

    Returns number

endHandler

  • endHandler(endHandler: ((res: void) => void) | Handler<void> | null | undefined): KafkaConsumer<K, V>
  • Parameters

    • endHandler: ((res: void) => void) | Handler<void> | null | undefined

    Returns KafkaConsumer<K, V>

endOffsets

  • endOffsets(topicPartition: TopicPartition): PromiseLike<number>
  • endOffsets(topicPartition: TopicPartition, handler: ((res: AsyncResult<number>) => void) | Handler<AsyncResult<number>>): void
  • Get the last offset for the given partition. The last offset of a partition is the offset of the upcoming message, i.e. the offset of the last available message + 1.

    Parameters

    Returns PromiseLike<number>

  • Get the last offset for the given partition. The last offset of a partition is the offset of the upcoming message, i.e. the offset of the last available message + 1.

    Parameters

    • topicPartition: TopicPartition
    • handler: ((res: AsyncResult<number>) => void) | Handler<AsyncResult<number>>

    Returns void

exceptionHandler

  • exceptionHandler(handler: ((res: Throwable) => void) | Handler<Throwable> | null | undefined): KafkaConsumer<K, V>
  • Parameters

    • handler: ((res: Throwable) => void) | Handler<Throwable> | null | undefined

    Returns KafkaConsumer<K, V>

fetch

  • Parameters

    • amount: number

    Returns KafkaConsumer<K, V>

handler

offsetsForTimes

  • Look up the offset for the given partition by timestamp. Note: the result might be null in case for the given timestamp no offset can be found -- e.g., when the timestamp refers to the future

    Parameters

    Returns PromiseLike<OffsetAndTimestamp>

  • Look up the offset for the given partition by timestamp. Note: the result might be null in case for the given timestamp no offset can be found -- e.g., when the timestamp refers to the future

    Parameters

    Returns void

partitionsAssignedHandler

  • Set the handler called when topic partitions are assigned to the consumer

    Parameters

    Returns KafkaConsumer<K, V>

partitionsFor

  • Get metadata about the partitions for a given topic.

    Parameters

    • topic: string

    Returns PromiseLike<PartitionInfo>

  • Get metadata about the partitions for a given topic.

    Parameters

    Returns KafkaConsumer<K, V>

partitionsRevokedHandler

  • Set the handler called when topic partitions are revoked to the consumer

    Parameters

    Returns KafkaConsumer<K, V>

pause

  • Returns KafkaConsumer<K, V>

  • Suspend fetching from the requested partition.

    Due to internal buffering of messages, the will continue to observe messages from the given topicPartition until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will not see messages from the given topicPartition.

    Parameters

    Returns PromiseLike<void>

  • Suspend fetching from the requested partition.

    Due to internal buffering of messages, the will continue to observe messages from the given topicPartition until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will not see messages from the given topicPartition.

    Parameters

    • topicPartition: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

  • Suspend fetching from the requested partitions.

    Due to internal buffering of messages, the will continue to observe messages from the given topicPartitions until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will not see messages from the given topicPartitions.

    Parameters

    Returns PromiseLike<void>

  • Suspend fetching from the requested partitions.

    Due to internal buffering of messages, the will continue to observe messages from the given topicPartitions until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will not see messages from the given topicPartitions.

    Parameters

    • topicPartitions: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

paused

  • Get the set of partitions that were previously paused by a call to pause(Set).

    Returns PromiseLike<TopicPartition>

  • Get the set of partitions that were previously paused by a call to pause(Set).

    Parameters

    Returns void

pipe

  • Returns Pipe<KafkaConsumerRecord<K, V>>

pipeTo

  • pipeTo(dst: WriteStream<KafkaConsumerRecord<K, V>>): PromiseLike<void>
  • pipeTo(dst: WriteStream<KafkaConsumerRecord<K, V>>, handler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): void
  • Parameters

    Returns PromiseLike<void>

  • Parameters

    • dst: WriteStream<KafkaConsumerRecord<K, V>>
    • handler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns void

poll

  • Executes a poll for getting messages from Kafka.

    Parameters

    • timeout: any

    Returns PromiseLike<KafkaConsumerRecords<K, V>>

  • Executes a poll for getting messages from Kafka.

    Parameters

    Returns void

pollTimeout

  • Sets the poll timeout for the underlying native Kafka Consumer. Defaults to 1000ms. Setting timeout to a lower value results in a more 'responsive' client, because it will block for a shorter period if no data is available in the assigned partition and therefore allows subsequent actions to be executed with a shorter delay. At the same time, the client will poll more frequently and thus will potentially create a higher load on the Kafka Broker.

    Parameters

    • timeout: any

    Returns KafkaConsumer<K, V>

position

  • position(partition: TopicPartition): PromiseLike<number>
  • position(partition: TopicPartition, handler: ((res: AsyncResult<number>) => void) | Handler<AsyncResult<number>>): void
  • Get the offset of the next record that will be fetched (if a record with that offset exists).

    Parameters

    Returns PromiseLike<number>

  • Get the offset of the next record that will be fetched (if a record with that offset exists).

    Parameters

    • partition: TopicPartition
    • handler: ((res: AsyncResult<number>) => void) | Handler<AsyncResult<number>>

    Returns void

resume

  • Returns KafkaConsumer<K, V>

  • Resume specified partition which have been paused with pause.

    Parameters

    Returns PromiseLike<void>

  • Resume specified partition which have been paused with pause.

    Parameters

    • topicPartition: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

  • Resume specified partitions which have been paused with pause.

    Parameters

    Returns PromiseLike<void>

  • Resume specified partitions which have been paused with pause.

    Parameters

    • topicPartitions: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

seek

  • seek(topicPartition: TopicPartition, offset: number): PromiseLike<void>
  • seek(topicPartition: TopicPartition, offset: number, completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • Overrides the fetch offsets that the consumer will use on the next poll.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    Returns PromiseLike<void>

  • Overrides the fetch offsets that the consumer will use on the next poll.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    • topicPartition: TopicPartition
    • offset: number
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

seekToBeginning

  • seekToBeginning(topicPartition: TopicPartition): PromiseLike<void>
  • seekToBeginning(topicPartition: TopicPartition, completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • seekToBeginning(topicPartitions: TopicPartition): PromiseLike<void>
  • seekToBeginning(topicPartitions: TopicPartition, completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • Seek to the first offset for each of the given partition.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    Returns PromiseLike<void>

  • Seek to the first offset for each of the given partition.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    • topicPartition: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

  • Seek to the first offset for each of the given partitions.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    Returns PromiseLike<void>

  • Seek to the first offset for each of the given partitions.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    • topicPartitions: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

seekToEnd

  • seekToEnd(topicPartition: TopicPartition): PromiseLike<void>
  • seekToEnd(topicPartition: TopicPartition, completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • seekToEnd(topicPartitions: TopicPartition): PromiseLike<void>
  • seekToEnd(topicPartitions: TopicPartition, completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • Seek to the last offset for each of the given partition.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    Returns PromiseLike<void>

  • Seek to the last offset for each of the given partition.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    • topicPartition: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

  • Seek to the last offset for each of the given partitions.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    Returns PromiseLike<void>

  • Seek to the last offset for each of the given partitions.

    Due to internal buffering of messages, the will continue to observe messages fetched with respect to the old offset until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new offset.

    Parameters

    • topicPartitions: TopicPartition
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

subscribe

  • subscribe(topic: string): PromiseLike<void>
  • subscribe(topic: string, completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • subscribe(topics: string): PromiseLike<void>
  • subscribe(topics: string, completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • Subscribe to the given topic to get dynamically assigned partitions.

    Due to internal buffering of messages, when changing the subscribed topic the old topic may remain in effect (as observed by the record handler}) until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new topic.

    Parameters

    • topic: string

    Returns PromiseLike<void>

  • Subscribe to the given topic to get dynamically assigned partitions.

    Due to internal buffering of messages, when changing the subscribed topic the old topic may remain in effect (as observed by the record handler}) until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new topic.

    Parameters

    • topic: string
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

  • Subscribe to the given list of topics to get dynamically assigned partitions.

    Due to internal buffering of messages, when changing the subscribed topics the old set of topics may remain in effect (as observed by the record handler}) until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new set of topics.

    Parameters

    • topics: string

    Returns PromiseLike<void>

  • Subscribe to the given list of topics to get dynamically assigned partitions.

    Due to internal buffering of messages, when changing the subscribed topics the old set of topics may remain in effect (as observed by the record handler}) until some time after the given completionHandler is called. In contrast, the once the given completionHandler is called the {@link KafkaConsumer#batchHandler} will only see messages consistent with the new set of topics.

    Parameters

    • topics: string
    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

subscription

  • subscription(): PromiseLike<string>
  • subscription(handler: ((res: AsyncResult<string>) => void) | Handler<AsyncResult<string>>): KafkaConsumer<K, V>
  • Get the current subscription.

    Returns PromiseLike<string>

  • Get the current subscription.

    Parameters

    • handler: ((res: AsyncResult<string>) => void) | Handler<AsyncResult<string>>

    Returns KafkaConsumer<K, V>

unsubscribe

  • unsubscribe(): PromiseLike<void>
  • unsubscribe(completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>): KafkaConsumer<K, V>
  • Unsubscribe from topics currently subscribed with subscribe.

    Returns PromiseLike<void>

  • Unsubscribe from topics currently subscribed with subscribe.

    Parameters

    • completionHandler: ((res: AsyncResult<void>) => void) | Handler<AsyncResult<void>>

    Returns KafkaConsumer<K, V>

Static create

  • Create a new KafkaConsumer instance

    Type parameters

    • K

    • V

    Parameters

    • vertx: Vertx
    • config: {}
      • [key: string]: string

    Returns KafkaConsumer<K, V>

  • Create a new KafkaConsumer instance

    Type parameters

    • K

    • V

    Parameters

    • vertx: Vertx
    • config: {}
      • [key: string]: string
    • keyType: any
    • valueType: any

    Returns KafkaConsumer<K, V>

  • Create a new KafkaConsumer instance

    Type parameters

    • K

    • V

    Parameters

    Returns KafkaConsumer<K, V>

  • Create a new KafkaConsumer instance

    Type parameters

    • K

    • V

    Parameters

    Returns KafkaConsumer<K, V>

Generated using TypeDoc