lucene-discovery

Workflow for remote lucene server subscription

1. API server starts
2. tells remote lucene server about its pubsub service
3. lucene derives access key and location id in a repeatable fashion
4. subscribes to pubsub
5. pubsub persists the subscription by key and location, including cookie used to send events to lucene
6. events are sents to lucene and persisted if lucene is non-responsive

API server restarts

• events queued and undelivered are sent as the persistent subscription is reloaded
• API server tells lucene to subscribe, which results in a no-op, since the subscription already exists

Lucene restarts

• All API server pubsubs will queue messages as lucene becomes non-responsive
  • As lucene comes back online queued messages are sent to lucene
  • Lucene doesn't have to resubscribe

Required changes

• pubsub accepts location key and access key for a subscription (done)
• pubsub persists subscription (done)
• pubsub persistently queues messages until delivered and periodically retries if delivery fails (done)
• pubsub expires subscription after a certain interval of continuous delivery failure (done)
• DreamServices need to be able to specify their own internal and private keys so that cookies stay the same after service restart (done)
• lucene generates subscription location and access key from subscription request in a repeatable manner (done)
• It must be possible to configure a DreamHost with a server IP (needs to be tested)

Caveats

• Still only one lucene server for all Cloud deployments
• lucene must be manually configured with a static internal access key
• API server and lucene must continue to share a single master apikey