Page tree
Skip to end of metadata
Go to start of metadata

General Proposal Information

HEP: 2
Title: Hazelcast Discovery SPI
HEP Shortname: Hazelcast Discovery SPI
Author: Christoph Engelbert (@noctarius2k), Paulo Pires (@el_ppires), Ray Tsang (@saturnism)
Sponsor: 
Signed-Of: Christoph Engelbert
Lead: Paulo Pires
Created: 2015/02/27 
Status: Completed
Type: Feature
Component: Integration
Discussion: https://gitter.im/hazelcast/hazelcast
Specification: https://docs.google.com/document/d/1YoNjsE_Iz96ZKzFPUTJqMu4hhU96LaqL0vdY9FXF1ts
Project: graduated, merged into Hazelcast core

Process Information

Start: 2015/Q1 
Depends:
Effort: M
Duration: M
Release: Hazelcast 3.6

Summary

The Hazelcast Discovery SPI will provide an extension SPI to attach external cloud discovery mechanisms. Discovery is meant to find other Hazelcast instances based on certain filters and provide their corresponding IP addresses.

Goals

  • Define a list of necessary information
  • Create a set of interfaces to be implemented by external discovery providers
  • Create a specification document
  • Create documentation of the SPI
  • Create reference implementations (JClouds and Kubernetes)

Non-Goals

  • The currently available Multicast, TCP/IP, AWS discovery mechanisms will not be re-implemented as part of this HEP.

Motivation

With growing user-base of Hazelcast and more cloud environments coming up, Hazelcast needs to offer a way to support external discovery providers.

Success Metrics

The current discovery mechanisms (Multicast, TCP/IP, AWS) must be possible to be re-implemented on top of the new SPI. Therefore all necessary information for the internals of Hazelcast need to be made available by SPI and provider implementations.

Description

At the current state Hazelcast provides a set of well-known discovery mechanisms, such as Multicast, TCP/IP and Amazon EC2 (AWS). With the already began and still rising cloud area, more and more cloud providers coming up. In addition there are also new deployment strategies such as Docker and last but not least company driven private clouds based on VMware or other hypervisors.

The current approach on Hazelcast is to provide a specific configuration block for every discovery mechanism. This approach is not scalable with the speed of new cloud environments arise. Eventually there will be a huge set of configurations and it is hard to maintain the sourcecode of the configuration parser.

A better approach is to provide a single public Discovery SPI to implement shipped and third-party providers against. These providers can be used as drop-in replacements and might also be combined (needs to be discussed - is there any interest need to combine multiple providers?) such as TCP/IP and a cloud discovery provider.

 

Expected SPI package: com.hazelcast.spi.discovery

Expected Changes:

  • com.hazelcast.config.NetworkConfig
  • com.hazelcast.config.JoinConfig
  • internal implementation classes

Testing

The new implementation needs to maintain backwards compatibility to currently used configurations. If additional reference implementations are done those need to have the typical unittests as well as an idea how to automatically test those. Otherwise they have to be kept as unsupported third-party providers.

Risks

Backward compatibility could break with careless changes. Care needs to be taken and possibly an additional set of unittests must be implemented first to prevent this situation from happening.

Dependencies

 

  • No labels

3 Comments

  1. We need to come up with:

     

    1 more flexible Config mechanism. The current mechanism is very inflexible; people should be able to extend the XML and the Config structure to include their own config.

    2 we need to expose more of our internals. We provide custom hooks for additional services, but we don't provide the ability to change/modify our existing services. Using IOC instead will make this a lot simpler.

    1. Yeah I like that but obviously IOC is not really scope of this HEP for now (smile) I guess this HEP should be fine without IOC for now but we should do it pretty quickly if we want to go that way. This one has to come along with the ServiceLoader API.

  2. If we make our XML parser 'generic' so it will just follow your config structure and it will bind to whatever it finds. Then at least the XML parsing doesn't need to modified when a user adds its own XML. So an automatic XML to POJO binding approach.

Write a comment…