General Proposal Information
Title: Hazelcast Discovery SPI
HEP Shortname: Hazelcast Discovery SPI
Author: Christoph Engelbert (@noctarius2k), Paulo Pires (@el_ppires), Ray Tsang (@saturnism)
Signed-Of: Christoph Engelbert
Lead: Paulo Pires
Release: Hazelcast 3.6
The Hazelcast Discovery SPI will provide an extension SPI to attach external cloud discovery mechanisms. Discovery is meant to find other Hazelcast instances based on certain filters and provide their corresponding IP addresses.
- Define a list of necessary information
- Create a set of interfaces to be implemented by external discovery providers
- Create a specification document
- Create documentation of the SPI
- Create reference implementations (JClouds and Kubernetes)
- The currently available Multicast, TCP/IP, AWS discovery mechanisms will not be re-implemented as part of this HEP.
With growing user-base of Hazelcast and more cloud environments coming up, Hazelcast needs to offer a way to support external discovery providers.
The current discovery mechanisms (Multicast, TCP/IP, AWS) must be possible to be re-implemented on top of the new SPI. Therefore all necessary information for the internals of Hazelcast need to be made available by SPI and provider implementations.
At the current state Hazelcast provides a set of well-known discovery mechanisms, such as Multicast, TCP/IP and Amazon EC2 (AWS). With the already began and still rising cloud area, more and more cloud providers coming up. In addition there are also new deployment strategies such as Docker and last but not least company driven private clouds based on VMware or other hypervisors.
The current approach on Hazelcast is to provide a specific configuration block for every discovery mechanism. This approach is not scalable with the speed of new cloud environments arise. Eventually there will be a huge set of configurations and it is hard to maintain the sourcecode of the configuration parser.
A better approach is to provide a single public Discovery SPI to implement shipped and third-party providers against. These providers can be used as drop-in replacements and might also be combined (needs to be discussed - is there any interest need to combine multiple providers?) such as TCP/IP and a cloud discovery provider.
Expected SPI package: com.hazelcast.spi.discovery
- internal implementation classes
The new implementation needs to maintain backwards compatibility to currently used configurations. If additional reference implementations are done those need to have the typical unittests as well as an idea how to automatically test those. Otherwise they have to be kept as unsupported third-party providers.
Backward compatibility could break with careless changes. Care needs to be taken and possibly an additional set of unittests must be implemented first to prevent this situation from happening.