Executar o Faust com kafka trava com ConsumerStoppedError

Nov 10 2020

Recentemente instalei o Faust e executei um programa básico para enviar e receber mensagens pelo Kafka. Usei o código de amostra mencionado em ( Exemplo do Faust de publicação em um tópico kafka ) Ao executar o programa inicialmente, ele se conecta ao Kafka (que também está sendo executado no meu máquina). Mas então, ao tentar consumir, Kafka se desconecta e o aplicativo trava com a exceção abaixo

[2020-11-11 07:08:26,623] [76392] [ERROR] [^---Fetcher]: Crashed reason=ConsumerStoppedError() 
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/services.py", line 779, in _execute_task
    await task
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 176, in _fetcher
    await self._drainer
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 1039, in _drain_messages
    async for tp, message in ait:
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 640, in getmany
    records, active_partitions = await self._wait_next_records(timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 676, in _wait_next_records
    records = await self._getmany(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 1269, in _getmany
    return await self._thread.getmany(active_partitions, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/drivers/aiokafka.py", line 805, in getmany
    return await self.call_thread(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/threads.py", line 436, in call_thread
    result = await promise
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/threads.py", line 383, in _process_enqueued
    result = await maybe_async(method(*args, **kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/utils/futures.py", line 134, in maybe_async
    return await res
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/drivers/aiokafka.py", line 822, in _fetch_records
    raise ConsumerStoppedError()

Ao depurar o motivo da desconexão do consumidor, vejo que em fetcher.py do consumidor alokafka a conexão está sendo fechada devido à exceção abaixo

    Unable to display children:Error resolving variables Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 205, in resolve
    def resolve(self, dict, key):
KeyError: 'Exception'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 1227, in do_it
    def do_it(self, dbg):
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_vars.py", line 262, in resolve_compound_variable_fields
    def resolve_compound_variable_fields(thread_id, frame_id, scope, attrs):
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_vars.py", line 169, in getVariable
    def getVariable(thread_id, frame_id, scope, attrs):
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 205, in resolve
    def resolve(self, dict, key):
AttributeError: 'dict' object has no attribute 'Exception'

As versões do software são as fornecidas abaixo

  • Mac OS: 10.15.4
  • Kafka: 2_12.2.1.1
  • Aiokafka: 1.1.6
  • Python: 3.9.0
  • Faust: 1.10.4

Por favor ajude aqui.

Respostas

MarceloNunesAlves Jan 21 2021 at 18:13

Acho que seu problema é o mesmo que eu tinha com o python 3.9 e mudei para 3.8 agora funciona.