[HIP] Bot Orchestration

Improvements
1

After discussions with the Hummingbot community in the past few weekly developer calls, there appears to be a consensus supporting a improvement bounty to enable orchestration, enabling Hummingbot instances to be controlled by a third party interfaces, respond to external events, etc would be highly beneficial to everyone.

See https://www.notion.so/hummingbot-foundation/Bot-Orchestration-fcac18bd90d74b0ebca9b260617522f0 for a requirements doc for this project and past discussions.

Community members klpanagi and TheHolyRoger have existing work that can be hardened and used for this:

  • TheHolyRoger has submitted a remote commands execution module PR along with an external TradingView web server
  • klpagani has forked Hummingbot and built a broker-based communication interface

Therefore, the Foundation proposes to fund a collaborative effort between the two of them to work together to deliver the Bot Orchestration project in phases.

Since this is a substantial amount of work with a lot of value to the community, we believe that 2,000,000 HBOT, out of the 15,000,000 Dev Grant Budget for Epoch 2, should be allocated to this project, with 1M paid upon completion of each phase (i.e. when the Foundation takes ownership of a working repo containing the changes).

Phase 1

Build an interface to execute remote commands to and from Hummingbot instances.

Commands covered:
- Start
- Stop
- Import
- Config strategy
- Balance
- Change balance limits

All this commands can be called using an unified web application that also receives the following information from the bots:
- Heartbeat - Status
- PNL - History

Get feedback on community on user interface and make corresponding changes
- Information that they want to see from the bots
- Extra commands to implement

The configuration of the broker in the Client should be in the conf_client.yml

Total HBOT allocation for completion: 1,000,000 HBOT

Phase 2

Create an interface compatible with Phase I that let the strategy listen to multiple topics to trigger internal events or store data inside the class.

With this solution the user can plug in:
- Signals from third parties, including TradingView
- Data to be processed in the strategy - Like having a machine learning model that sends predictions

Total HBOT allocation for completion: 1,000,000 HBOT

Future Phases

Afterwards, the Foundation foresees allocate future dev grants for additional Phases of work, which are listed in the requirements doc.

2 Likes
2

An update on the scope, the goals, the phases and the individual tasks and deliverables.

Goal

The goal of this proposal is to enable remote control and monitoring of Bot instances towards a plugin-based approach of building HF AMM bots and the development of bot collaboration schemas in the future.

Implementation

The implementation is divided into two (2) Phases. In Phase 1, a thin communication and messaging layer will be injected into the hummingbot codebase to enable remote control and monitoring via a broker-based communication middleware, essentially supporting the MQTT protocol. Next, in Phase 2, an event and data layer will be integrated into the hummingbot codebase to support receiving and handling remote events via the message broker (MQTT), such as the case of TradingView signals.

Phase 1

A broker-based communication middleware will be integrated into the hummingbot codebase in order to allow for:

  • Remote execution of Bot Commands
    • Start
    • Stop
    • Import
    • Config
    • Balance Limit
    • History
  • Remote monitoring of Bots
    • State monitoring via heartbeats send to the broker
    • Remote Logging. The Logger class will be refactored to also send logs (optional / enabled by default) to the broker
    • Remote Notifications - The Notifier class will be extended to send notifications to the broker

An MQTT broker will be used for the implementation of the communication middleware, as a single point of reference for communication between bots and third-party software and services.

The configuration of the broker endpoint (Host, Port, Credentials etc) will be in the ‘conf_client.yml’ file so that these parameters can be seen as normal global bot parameters.

Finally, deployment scripts for the EMQX message broker will be delivered and integrated into the hummingbot codebase.

Phase 2

In this Phase, an event and data layer will be integrated into the hummingbot codebase to support receiving and handling remote events via the message broker (MQTT), such as the case of TradingView signals.

More specifically, an MQTT event listener will be developed and integrated into the hummingbot codebase, which will provide configuration for setting the URIs of the events to listen on. Upon receiving an event, a handling callback provided by the user/developer will be executed by the MQTTEventListener, so that users operate/develop their strategy based on the input event.

In the context of Phase 2, the RogerThat TradingView implementation will be delivered as an individual module and will bridge TradingView signals to bots (TradingView → MQTT broker → Bots) for developers to be able to use them within their strategies. The aforementioned MQTTEventLlistener will provide means of consuming TradingView signals within strategies, in the form of external events and callback functions.

3

The HIP for this project has been created: Snapshot