Hummingbot as an API

Author: James 0xda6

Currently Hummingbot functions as an application. The logic, data and view are not well separated. It would be interesting to refactor the core logic of Hummingbot and expose it as an API. The current Hummingbot view would still be available in the same form, but under the hood it would be an application talking to an API.

This offers numerous advantages:

  • Make it easier for users to create their own frontends.
  • The view (frontend) and the logic/data (backend) can be in different places.
  • Multiple frontends can interact with a single backend.
  • This forces the code to be separated and should make it easier to read. We can document the API and this would be a good starting point for new developers to the project.

I would like to gauge interest in this project from the community users and developers. I have found two related issues in the hummingbot github repo: Run as normal command-line daemon and Externalize core hummingbot events to Kafka. It would be a large project.

Author: klpanagi 0xfba

Agree 100%! I will support this proposal. Currently hummingbot is a monolithic application, which limits scalability, extendability and reusability. By having Hummingbot Bots to provide their own API, it will be easy to build applications on top of that, such as community-driven GUIs and TUIs. It will be a huge upgrade for extending current capabilities and allow for easier integration of community-driven addons/plugins/etc.

I am interested to work as a developer on this!

Author: weltam 0x94F

i think it would be nice if the API also asynchronous (not REST API) using ZMQ or Kafka or even something lightweight message passing. so it can insert the command into the existing tick/ event loop and change the behaviour without need to restart the whole instance.

Author: klpanagi 0xfba

Agree. We should use message queuing middlewares, such as Kafka, RabbitMQ, MQTT brokers, Redis, etc. I would go with MQTT for easier integration with web components (e.g. front end GUIs).

I have developed a framework in Python for my PhD, for high-level message-driven communication over message brokers. It abstracts low level middleware-specific information and allow for the creation of PubSub and ReqRest (and many more) endpoints in distributed systems. We can use that to implement the aync API. We can even build hybrid sync/async APIs using commlib.

P.S. I am currently working on a Kafka transport layer for commlib.

Author: weltam 0x94F

i think this thread has given the clue for us to get started.

self.app = HummingbotCLI(
input_handler=self._handle_command,
bindings=load_key_bindings(self),
completer=load_completer(self),
command_tabs=command_tabs
)
basically we can start with HummingbotCLI, we can treat handle_command as message handler for incoming message/command from queue.

Author: weltam 0x94F

for intercepting the event to publish from core we can intercept here as start
hummingbot/core/event/event_reporter.pyx
cdef c_call(self, object event_object):
try:
if dataclasses.is_dataclass(event_object):
event_dict = dataclasses.asdict(event_object)
else:
event_dict = event_object._asdict()

    event_dict.update({"event_name": event_object.__class__.__name__,
                       "event_source": self.event_source})
    self.logger().event_log(event_dict)
except Exception:
    self.logger().error("Error logging events.", exc_info=True)