HEP 5 - Quartz JobStore Integration

General Proposal Information

HEP: 5
Title: Quartz JobStore Integration
HEP Shortname: quartz-jobstore
Author: Wolfgang Haag (wolfgang_dot_haag_at_canon_dot_de)
Sponsor: Canon Europe N.V.
Signed-Of: Christoph Engelbert
Lead: Wolfgang Haag, Canon Europe N.V.
Created: 2015/06/04 
Status: Draft
Type: Feature
Component: Integration
Discussion: https://gitter.im/hazelcast-incubator/quartz-jobstore
Specification: 
Project: https://github.com/hazelcast-incubator/quartz-jobstore

Process Information

Start: 2015/Q2 
Depends:
Effort: S
Duration: S
Release:

Summary

This proposal contains a Cluster-Job-Store based on Hazelcast objects to be used with Quartz  > V2.2. The job store is a derivation of the JDBC-Job-Store, delivered with the Quartz package.

Goals

At the moment, there is no extension to use a Quartz job-scheduler with Hazelcast. Many companies are using Quartz as their preferred scheduler, this depends on history, so with that solution it is possible to use Quartz as clustered scheduler. Therefore the integration module must be able to store tasks from Quartz inside Hazelcast.

Configuration must support Spring and non-Spring configurations as well as API based configuration.

 

Non-Goals

At the moment only the Quartz cron syntax is tested. The function of extended calendar triggers is not guaranteed, it might be implemented as part of a later HEP.

Motivation

Quartz is a proven scheduler technology in the Java ecosystem. To use Quartz in a clustered environment, the JDBC-Job-Store. There are also integration into other datagrid products, so it seems a natural fit to integrate it with Hazelcast. A search on the Internet brings up a lot of requests to integrate Quartz as an extension to use it with Hazelcast.

Success Metrics

The integration implementation must be able to store tasks from Quartz inside Hazelcast. That includes serialization and deserialization on other nodes as well as storing any kind of required metadata. Tasks are not written to any permanent storage.

Description

The use of the Hazelcast Quartz JobStore is very easy. All people which are familiar with Quartz configuration can use this extension by configure it, with the default Quartz properties.

Sample of configuration:

Properties quartzProperties = new Properties();
quartzProperties.put(StdSchedulerFactory.PROP_SCHED_INSTANCE_ID, "AUTO");
quartzProperties.put(StdSchedulerFactory.PROP_SCHED_INSTANCE_NAME, "CRONJOB_SCHEDULER");
quartzProperties.put(StdSchedulerFactory.PROP_SCHED_RMI_EXPORT, "false");
quartzProperties.put(StdSchedulerFactory.PROP_SCHED_RMI_PROXY, "false");
quartzProperties.put(StdSchedulerFactory.PROP_THREAD_POOL_CLASS, "org.quartz.simpl.SimpleThreadPool");
quartzProperties.put("org.quartz.threadPool.threadPriority", "5");
quartzProperties.put("org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread", "true");
quartzProperties.put("org.quartz.jobStore.misfireThreshold", "2000");
quartzProperties.put(StdSchedulerFactory.PROP_JOB_STORE_CLASS, "com.oce.cosmos.scheduler.cluster.ClusterJobStore");
quartzProperties.put("org.quartz.jobStore.isClustered", "true");
quartzProperties.put("org.quartz.plugin.shutdownhook.class", "org.quartz.plugins.management.ShutdownHookPlugin");
quartzProperties.put("org.quartz.plugin.shutdownhook.cleanShutdown", "true");
quartzProperties.put("org.quartz.threadPool.threadCount", "5");

 

Spring Hazelcast configuration:

<hz:hazelcast id="instance">
  <hz:config>
    <!-- ... other configuration -->
    <!-- This settings are used from QUARTZ -->
    <hz:map  name="quartz.cron.triggers" eviction-policy="LRU" max-size="100" eviction-percentage="25"  />
    <hz:map  name="quartz.cron.job.details" eviction-policy="LRU" max-size="100" eviction-percentage="25" />
    <hz:map  name="quartz.cron.fiered.triggers" eviction-policy="LRU" max-size="1000" eviction-percentage="25" />
    <hz:map  name="quartz.paused.trigger.groups" eviction-policy="LRU" max-size="50" eviction-percentage="25" />
    <hz:map  name="quartz.calendars" eviction-policy="LRU" max-size="50" eviction-percentage="25" />
    <hz:map name="quartz.simple.triggers"  eviction-policy="LRU" max-size="100" eviction-percentage="25"/>
    <hz:map  name="quartz.simple.prop.triggers" eviction-policy="LRU" max-size="50" eviction-percentage="25"/>
    <hz:map  name="quartz.simple.operable.triggers" eviction-policy="LRU" max-size="100" eviction-percentage="25"/>
    <hz:map  name="quartz.scheduler.state" eviction-policy="LRU" max-size="50" eviction-percentage="25" />
    <hz:topic id="quartz.scheduler.topic" name="quartz.scheduler.topic" instance-ref="instance"/>
    <hz:lock id="quartz.lock.state" name="quartz.lock.state" instance-ref="instance" />
    <hz:lock id="quartz.lock.trigger" name="quartz.lock.trigger" instance-ref="instance" />
  </hz:config>
</hz:hazelcast>

Testing

A simple JUnit test is included. To test this extension some more JUnit tests must be implemented.

The best way to test this proposal is to install a clustered Hazelcast environment and create Quartz jobs with some CronTriggers. Shutdown a Hazelcast instance and start it later again. The Quartz jobs should be pended between the instances.

Please Note: This solution doesn't persisted the running jobs. If all instances fall down. The jobs will be initialized again white start up.

Please Note: To shutdown a instance Quartz must be brought down before Hazelcast shuts down. This can be done via Hazelcast LifecycleListener.

 

Risks

This proposal is a extension for Quartz job scheduling and does not impact other Hazelcast parts. It is compatible with Quartz 2.2 and Hazelcast 3.4.

Dependencies