Module: Cybertrain::Stream

Defined in:
cybertrain/stream.rb,
cybertrain/stream/hub.rb,
cybertrain/stream/poller.rb,
cybertrain/stream/endpoint.rb,
cybertrain/stream/websocket_server.rb

Overview

Cybertrain::Stream – the pub/sub behind the realtime endpoint (docs/design.md D37). A broadcast is a row in the application’s own SQLite database, in the cybertrain_messages table the framework creates itself (CREATE TABLE IF NOT EXISTS) and leaves out of db/schema.rb; every web process polls the table (Poller, every Config#stream_poll_ms ms) and hands new rows to the browsers subscribed to their stream through Endpoint (Server-Sent Events or a WebSocket). So a broadcast from a job, a scheduled task, a feed callback or another process reaches every page, in one order.

# in an action, a job or a Main.command: Cybertrain::Stream.broadcast_append_to(“alerts”, target: “alerts”, html: Cybertrain::Html.safe(“<li>Hi</li>”))

<%# in a template: subscribes this page (Turbo picks the messages up) %> <%= turbo_stream_from “alerts” %>

A page may read a stream only when it was rendered with a signed token naming it (Stream.sign, what turbo_stream_from writes): the endpoint checks the signature, never the session (as Turbo’s signed stream names). A per-user stream is Stream.name_for(current_user), rendered only on that user’s pages.

Defined Under Namespace

Modules: Poller Classes: Endpoint, Hub, Message, Subscriber

Class Method Summary collapse

Class Method Details

.broadcast(name, payload) ⇒ Integer

Adds a message to name: one row, delivered to every subscriber of the stream (in this process and every other one polling the same database) within about Config#stream_poll_ms. Inside a DB.transaction the message is sent only if the transaction commits.

Parameters:

  • name (String) —

    the stream (name_for for a record)

  • payload (String) —

    what the subscribers receive: Turbo Stream HTML, JSON or any text

Returns:

Raises:

  • (ArgumentError) —

    for an invalid stream name



72
73
74
75
76
77
78
79
80
81
82
83
84
# File 'cybertrain/stream.rb', line 72

def self.broadcast(name, payload)
  stream = name.to_s
  raise ArgumentError, "Stream.broadcast: invalid stream name #{stream.inspect}" unless Stream.valid_name?(stream)

  ensure_installed
  binds = Cast.binds(stream, payload.to_s, Clock.now.to_i)
  id = 0
  DB.with do |c|
    c.execute("INSERT INTO cybertrain_messages (stream, payload, created_at) VALUES (?, ?, ?)", binds)
    id = c.last_insert_id
  end
  id
end

.broadcast_after_to(stream, target:, html: "", partial: "", locals: {}, component: nil) ⇒ Object

As broadcast_replace_to, with action="after".



458
459
460
461
# File 'cybertrain/stream.rb', line 458

def self.broadcast_after_to(stream, target:, html: "", partial: "", locals: {}, component: nil)
  content = Stream.render_content(html, partial, locals, component)
  Stream.broadcast(Stream.name_for(stream), Turbo.after(target, content).to_s)
end

.broadcast_append_to(stream, target:, html: "", partial: "", locals: {}, component: nil) ⇒ Object

As broadcast_replace_to, with action="append".



437
438
439
440
# File 'cybertrain/stream.rb', line 437

def self.broadcast_append_to(stream, target:, html: "", partial: "", locals: {}, component: nil)
  content = Stream.render_content(html, partial, locals, component)
  Stream.broadcast(Stream.name_for(stream), Turbo.append(target, content).to_s)
end

.broadcast_before_to(stream, target:, html: "", partial: "", locals: {}, component: nil) ⇒ Object

As broadcast_replace_to, with action="before".



451
452
453
454
# File 'cybertrain/stream.rb', line 451

def self.broadcast_before_to(stream, target:, html: "", partial: "", locals: {}, component: nil)
  content = Stream.render_content(html, partial, locals, component)
  Stream.broadcast(Stream.name_for(stream), Turbo.before(target, content).to_s)
end

.broadcast_prepend_to(stream, target:, html: "", partial: "", locals: {}, component: nil) ⇒ Object

As broadcast_replace_to, with action="prepend".



444
445
446
447
# File 'cybertrain/stream.rb', line 444

def self.broadcast_prepend_to(stream, target:, html: "", partial: "", locals: {}, component: nil)
  content = Stream.render_content(html, partial, locals, component)
  Stream.broadcast(Stream.name_for(stream), Turbo.prepend(target, content).to_s)
end

.broadcast_refresh_to(stream) ⇒ Integer

<turbo-stream action="refresh"> to stream: the pages reload themselves (Turbo 8’s morphing refresh).

Returns:



474
475
476
# File 'cybertrain/stream.rb', line 474

def self.broadcast_refresh_to(stream)
  Stream.broadcast(Stream.name_for(stream), Turbo.refresh.to_s)
end

.broadcast_remove_to(stream, target:) ⇒ Integer

<turbo-stream action="remove" target="..."> to stream.

Returns:



466
467
468
# File 'cybertrain/stream.rb', line 466

def self.broadcast_remove_to(stream, target:)
  Stream.broadcast(Stream.name_for(stream), Turbo.remove(target).to_s)
end

.broadcast_replace_to(stream, target:, html: "", partial: "", locals: {}, component: nil) ⇒ Integer

Renders the content (html: as given, escaped unless a SafeString; or partial: with locals: through Views.render_partial; or component: through Views.render_component), wraps it in a <turbo-stream action="replace"> for target (Turbo.replace) and broadcasts it to stream (a name or a record, name_for).

Examples:

Cybertrain::Stream.broadcast_replace_to(article, target: article, partial: "articles/article", locals: { article: article })

Returns:



423
424
425
426
# File 'cybertrain/stream.rb', line 423

def self.broadcast_replace_to(stream, target:, html: "", partial: "", locals: {}, component: nil)
  content = Stream.render_content(html, partial, locals, component)
  Stream.broadcast(Stream.name_for(stream), Turbo.replace(target, content).to_s)
end

.broadcast_update_to(stream, target:, html: "", partial: "", locals: {}, component: nil) ⇒ Object

As broadcast_replace_to, with action="update".



430
431
432
433
# File 'cybertrain/stream.rb', line 430

def self.broadcast_update_to(stream, target:, html: "", partial: "", locals: {}, component: nil)
  content = Stream.render_content(html, partial, locals, component)
  Stream.broadcast(Stream.name_for(stream), Turbo.update(target, content).to_s)
end

.install! ⇒ nil

Creates the table and its index if they are missing. broadcast and the poller call it once per connected database.

Returns:

  • (nil)


258
259
260
261
262
263
264
265
# File 'cybertrain/stream.rb', line 258

def self.install!
  DB.with { |c| c.exec_script(TABLE_SQL) }
  # As Job.install!: a CREATE inside a transaction that rolls back is
  # gone again, so only one outside a transaction is remembered.
  pool = DB.pool
  @installed_generation = DB.generation if !pool.nil? && pool.held_index < 0
  nil
end

.name_for(record_or_string) ⇒ String

The stream name of a record, "articles:12" (its table and id); a String is returned as it is.

Parameters:

  • record_or_string (Model, String)

Returns:

  • (String)


91
92
93
94
95
96
97
98
99
100
101
102
# File 'cybertrain/stream.rb', line 91

def self.name_for(record_or_string)
  case record_or_string
  when Cybertrain::Model
    raise ArgumentError, "Stream.name_for: the record is not saved (it has no id)" if record_or_string.id.to_i == 0

    table = record_or_string.class.table_name
    id = record_or_string.id.to_s
    "#{table}:#{id}"
  else
    record_or_string.to_s
  end
end

.prune(before) ⇒ Integer

Deletes the messages created before before (epoch seconds); the poller runs it every minute with Config#stream_retention.

Parameters:

  • before (Integer) —

    epoch seconds

Returns:

  • (Integer) —

    how many rows were deleted



278
279
280
281
282
283
284
285
286
# File 'cybertrain/stream.rb', line 278

def self.prune(before)
  ensure_installed
  count = 0
  DB.with do |c|
    c.execute("DELETE FROM cybertrain_messages WHERE created_at < ?", [before])
    count = c.changes
  end
  count
end

.replay(names, after_id, limit = REPLAY_CHUNK) ⇒ Array<Message>

The messages of names still in the table with an id above after_id, oldest first, at most limit (one chunk): what a reconnecting client missed (Last-Event-ID, ?after=). Filtered in SQL on the (stream, id) index. The endpoint pages through the backlog with it, from the last id it sent, until a chunk comes back short or passes the subscriber’s cursor; retention is what bounds it.

Parameters:

  • names (Array<String>)
  • after_id (Integer)
  • limit (Integer) (defaults to: REPLAY_CHUNK)

Returns:



330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
# File 'cybertrain/stream.rb', line 330

def self.replay(names, after_id, limit = REPLAY_CHUNK)
  out = Array.new(0) { Message.new(0, "", "") }
  return out if names.empty?

  ensure_installed
  marks = +""
  binds = Cast.binds
  i = 0
  while i < names.size
    marks << ", " if i > 0
    marks << "?"
    binds << Cast.to_sql(names[i])
    i += 1
  end
  binds << Cast.to_sql(after_id)
  binds << Cast.to_sql(limit)
  sql = +"SELECT id, stream, payload FROM cybertrain_messages WHERE stream IN ("
  sql << marks
  sql << ") AND id > ? ORDER BY id LIMIT ?"
  rows = DB.with { |c| c.execute(sql, binds) }
  i = 0
  while i < rows.size
    row = rows[i]
    out << Message.new(Cast.int(row["id"]), Cast.str(row["stream"]), Cast.str(row["payload"]))
    i += 1
  end
  out
end

.sign(names) ⇒ String

A token naming names: base64url of the names joined by “,”, then “–”, then the hex HMAC-SHA256 of that text keyed with a key derived from Config#secret_key_base for streams alone (signing_key, as Rails’ “turbo/signed_stream_verifier_key”), so a token never verifies as a session cookie, whose layout is the same. Tokens do not expire; changing the secret invalidates all of them.

Parameters:

  • names (Array<String>)

Returns:

  • (String)

Raises:

  • (ArgumentError) —

    for no names, more than MAX_NAMES, or an invalid name



115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
# File 'cybertrain/stream.rb', line 115

def self.sign(names)
  raise ArgumentError, "Stream.sign: no stream names" if names.empty?
  raise ArgumentError, "Stream.sign: more than #{MAX_NAMES} stream names" if names.size > MAX_NAMES

  i = 0
  while i < names.size
    name = names[i].to_s
    raise ArgumentError, "Stream.sign: invalid stream name #{name.inspect}" unless Stream.valid_name?(name)

    i += 1
  end
  joined = names.join(",")
  encoded = Base64.urlsafe_encode64(joined, padding: false)
  signature = Crypto.hmac_hex(Stream.signing_key, encoded)
  out = +""
  out << encoded
  out << "--"
  out << signature
  out
end

.verify(token) ⇒ Array<String>?

The names a token carries, or nil when its signature is wrong or missing, or a name in it is invalid.

Parameters:

  • token (String)

Returns:

  • (Array<String>, nil)


141
142
143
144
145
146
# File 'cybertrain/stream.rb', line 141

def self.verify(token)
  names = Stream.verified_names(token)
  return nil if names.empty?

  names
end