Module: Cybertrain::Stream::Poller

Defined in:
cybertrain/stream/poller.rb

Overview

The one thread per process that turns rows of cybertrain_messages into deliveries: every Config#stream_poll_ms ms it reads SELECT MAX(id) (one B-tree descent on the primary key) and, when that passed the last id it delivered, the new rows, which it hands to hub. Every minute it also deletes the rows older than Config#stream_retention seconds; the one place that prunes. Started by Application#serve before it binds the port (so no subscription can come before the start position) and stopped with the server; the endpoint answers 503 while it is not running. It starts at the current MAX(id): a message is delivered live only to the subscribers present when it is polled; a reconnecting client asks for what it missed (Stream.replay).

Class Method Summary collapse

Class Method Details

.run_once(hub) ⇒ Integer

One tick: delivers the rows above last_id to hub, in id order. The thread calls it; tests call it instead of waiting. last_id is “the highest row already delivered”: it moves to a row’s id inside Hub#deliver_polled, under the hub’s lock and after the row went to the subscribers, so a subscription taken while a row is in flight gets a boundary below it and receives it.

Parameters:

Returns:

  • (Integer) —

    how many rows it delivered



106
107
108
109
110
111
112
113
114
115
116
117
118
# File 'cybertrain/stream/poller.rb', line 106

def self.run_once(hub)
  last = @last_id
  return 0 if Stream.max_id <= last

  rows = Stream.messages_after(last, 1000)
  i = 0
  while i < rows.size
    msg = rows[i]
    hub.deliver_polled(msg.message_id, msg.message_stream, msg.message_payload)
    i += 1
  end
  rows.size
end

.running? ⇒ Boolean

Returns:

  • (Boolean)


64
65
66
# File 'cybertrain/stream/poller.rb', line 64

def self.running?
  @running
end

.start!(config = Cybertrain.config) ⇒ Boolean

Starts the thread unless it runs; true when it started it.

Parameters:

  • config (Config) (defaults to: Cybertrain.config)

Returns:

  • (Boolean)


29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# File 'cybertrain/stream/poller.rb', line 29

def self.start!(config = Cybertrain.config)
  return false if @running

  # Read before anything is marked running: a database that fails
  # here leaves the poller stopped, not "running" with no thread.
  # (Stream.max_id creates the table on a fresh database.)
  head = Stream.max_id
  started = false
  generation = 0
  LOCK.synchronize do
    unless @running
      @running = true
      @generation += 1
      generation = @generation
      started = true
    end
  end
  return false unless started

  @last_id = head
  @last_prune = Clock.monotonic.to_i
  Poller.spawn_loop(config.stream_poll_ms, config.stream_retention, generation)
  true
end

.stop! ⇒ nil

Asks the thread to end; it does after its current tick.

Returns:

  • (nil)


57
58
59
60
# File 'cybertrain/stream/poller.rb', line 57

def self.stop!
  LOCK.synchronize { @running = false }
  nil
end