Module: Cybertrain::WebSocket

Defined in:
cybertrain/websocket.rb,
cybertrain/websocket/frame.rb,
cybertrain/websocket/client.rb,
cybertrain/websocket/feed_table.rb,
cybertrain/websocket/status_writer.rb

Defined Under Namespace

Modules: FeedTable, Frame Classes: BlockedAddressError, Client, ClosedError, Connection, Error, FeedTableWriter, HandshakeError, Message, ProtocolError, RefusedError, Status, StatusWriter, Subscription, TLSError, TimeoutError, TooLargeError

Class Method Summary collapse

Class Method Details

.dead_feeds ⇒ Array<String>

The names of the feeds start! started whose thread has ended while the feeds run, for the supervisor to restart.

Returns:

  • (Array<String>)


526
527
528
529
530
531
532
533
534
535
536
537
# File 'cybertrain/websocket.rb', line 526

def self.dead_feeds
  out = Array.new(0) { "" }
  return out unless @running

  i = 0
  while i < SUBSCRIPTIONS.size
    sub = SUBSCRIPTIONS[i]
    out << sub.sub_name if sub.sub_started_at > 0 && !sub_alive?(sub)
    i += 1
  end
  out
end

.feed_names ⇒ Array<String>

The names of the registered subscriptions, in registration order.

Returns:

  • (Array<String>)


642
643
644
645
646
# File 'cybertrain/websocket.rb', line 642

def self.feed_names
  out = Array.new(0) { "" }
  SUBSCRIPTIONS.each { |sub| out << sub.sub_name }
  out
end

.feed_status(name) ⇒ Status

The state of the feed named name. Not part of /up: an application’s own health check can use it.

Parameters:

  • name (String)

Returns:

Raises:

  • (ArgumentError) —

    when no feed has that name



497
498
499
500
501
502
503
# File 'cybertrain/websocket.rb', line 497

def self.feed_status(name)
  i = subscription_index(name.to_s)
  raise ArgumentError, "WebSocket.feed_status: no subscription named #{name.to_s.inspect}" if i < 0

  sub = SUBSCRIPTIONS[i]
  Status.new(sub.sub_connected?, sub.sub_last_message_at, sub.sub_reconnects, sub.sub_restarts, sub_alive?(sub))
end

.publish_status(now) ⇒ Integer

Writes one status row per feed start! started, with updated_at = now, through the registered StatusWriters (in an application, the cybertrain_feeds row of FeedTable, when a database is connected); nothing when the feeds do not run. The supervisor calls it every Config#feed_status_interval seconds.

Parameters:

Returns:

  • (Integer) —

    how many rows were written

Raises:

  • (Cybertrain::DB::Error) —

    when the database is not reachable



568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
# File 'cybertrain/websocket.rb', line 568

def self.publish_status(now)
  return 0 unless @running

  process = status_process
  written = 0
  i = 0
  while i < SUBSCRIPTIONS.size
    sub = SUBSCRIPTIONS[i]
    started_at = sub.sub_started_at
    if started_at > 0
      connected = sub_alive?(sub) && sub.sub_connected?
      last_activity_at = sub.sub_last_activity_at
      restarts = sub.sub_restarts
      j = 0
      while j < STATUS_WRITERS.size
        wrote = STATUS_WRITERS[j].status_write(sub.sub_name, process, connected, started_at, last_activity_at,
                                               restarts, now)
        written += 1 if wrote
        j += 1
      end
    end
    i += 1
  end
  written
end

.random_source=(source) ⇒ Proc

The source of the retry delay’s jitter: a Proc answering a Float in 0.0..1.0. Tests set -> { 0.5 }.

Parameters:

  • source (Proc)

Returns:

  • (Proc)


774
775
776
# File 'cybertrain/websocket.rb', line 774

def self.random_source=(source)
  @random_source = source
end

.restart_feed(name) ⇒ Boolean

Starts the feed named name again on a new thread, when the feeds run, it is one of those start! started and its thread has ended; counts it in Cybertrain::WebSocket::Status#restarts. The supervisor calls it.

Parameters:

  • name (String)

Returns:

  • (Boolean) —

    true when it started a thread



545
546
547
548
549
550
551
552
553
554
555
556
557
# File 'cybertrain/websocket.rb', line 545

def self.restart_feed(name)
  i = subscription_index(name.to_s)
  return false if i < 0 || !@running

  sub = SUBSCRIPTIONS[i]
  return false if sub.sub_started_at == 0 || sub_alive?(sub)

  sub.sub_disconnected!
  count = sub.sub_restarted!
  Cybertrain.logger.info("WebSocket restarted: #{sub.sub_name} (restart #{count})")
  sub.sub_thread!(spawn_feed(sub, @generation))
  true
end

.retry_delay(attempt, jitter) ⇒ Float

Seconds to wait before connecting again after the attemptth failure in a row: min(60, 2 ** (attempt - 1)), times 0.8 to 1.2.

Parameters:

  • attempt (Integer) —

    1 for the first failure

  • jitter (Float) —

    0.0..1.0 (0.5 is the plain doubling)

Returns:

  • (Float)


754
755
756
757
758
759
760
761
762
763
764
765
766
767
# File 'cybertrain/websocket.rb', line 754

def self.retry_delay(attempt, jitter)
  n = attempt.to_i
  base = 1
  if n >= 7
    base = 60
  elsif n > 1
    base = 2**(n - 1)
  end
  j = jitter.to_f
  j = 0.0 if j < 0.0
  j = 1.0 if j > 1.0
  # Tenths, so (1, 1.0) is exactly 1.2 (0.8 + 0.4 is not).
  (base.to_f * (8.0 + 4.0 * j)) / 10.0
end

.running? ⇒ Boolean

Returns true between start! and stop!.

Returns:



744
745
746
# File 'cybertrain/websocket.rb', line 744

def self.running?
  @running
end

.start!(config = Cybertrain.config, only: Background::NO_NAMES) ⇒ Boolean

Starts one thread per subscription (only those named in only, when it is not empty), unless Config#worker is off, nothing is subscribed or the feeds already run. Application#serve calls it (through the Background service) with every subscription; ./<app> feeds NAME... with the named ones.

Parameters:

  • config (Config) (defaults to: Cybertrain.config)
  • only (Array<String>) (defaults to: Background::NO_NAMES) —

    subscription names; empty for all

Returns:

  • (Boolean) —

    true when it started the threads

Raises:

  • (ArgumentError) —

    when a subscription has no on_message, or a name in only is not subscribed



669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
# File 'cybertrain/websocket.rb', line 669

def self.start!(config = Cybertrain.config, only: Background::NO_NAMES)
  return false if @running
  unless config.worker
    Cybertrain.logger.info("Background worker off (CYBERTRAIN_WORKER=0): WebSocket subscriptions run elsewhere")
    return false
  end
  return false if SUBSCRIPTIONS.empty?

  problem = subscriptions_problem
  raise ArgumentError, "WebSocket.start!: #{problem}" unless problem.empty?
  unknown = unknown_feed(only)
  raise ArgumentError, "WebSocket.start!: #{unknown}" unless unknown.empty?

  # spikes/NOTES.md rule 22: before the first Thread.new.
  ENV["SPINEL_WORKERS"] = "1" unless ENV["SPINEL_WORKERS"]
  @running = true
  @generation += 1
  generation = @generation
  started_at = Clock.now.to_i
  i = 0
  while i < SUBSCRIPTIONS.size
    sub = SUBSCRIPTIONS[i]
    if only.empty? || only.include?(sub.sub_name)
      Cybertrain.logger.info("WebSocket started: #{sub.sub_name} (#{sub.sub_origin})")
      sub.sub_started!(started_at)
      sub.sub_thread!(spawn_feed(sub, generation))
    else
      sub.sub_started!(0)
    end
    i += 1
  end
  true
end

.status_process ⇒ String

“:", the `process` this process writes in its status rows.

Returns:

  • (String)


598
599
600
601
# File 'cybertrain/websocket.rb', line 598

def self.status_process
  @process_label = "#{Socket.gethostname}:#{Process.pid}" if @process_label.empty?
  @process_label
end

.stop! ⇒ nil

Asks every feed to end and waits for them, at most 2 s in all: a connected feed sends close 1000 and waits at most 1 s for the server’s answer. Every feed reads as disconnected once it returns (a thread still closing finishes on its own). Does nothing when the feeds do not run.

Returns:

  • (nil)


710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
# File 'cybertrain/websocket.rb', line 710

def self.stop!
  return nil unless @running

  @running = false
  ends_at = Net::Timer.now + STOP_JOIN_SECONDS
  i = 0
  while i < SUBSCRIPTIONS.size
    thread = SUBSCRIPTIONS[i].sub_thread
    left = ends_at - Net::Timer.now
    join_feed(thread, left) if !thread.nil? && left > 0.0
    i += 1
  end
  i = 0
  while i < SUBSCRIPTIONS.size
    SUBSCRIPTIONS[i].sub_disconnected!
    i += 1
  end
  nil
end

.subscribe(name, url, headers: Client::NO_HEADERS, timeout: 5, max_message: 1_048_576, ping_interval: 15, stale_after: 45, allow_private: Cybertrain.config.http_allow_private) ⇒ Subscription

Registers a feed to keep connected while the server runs. Register before the server starts; give it Cybertrain::WebSocket::Subscription#on_message (and usually Cybertrain::WebSocket::Subscription#on_open).

Examples:

Cybertrain::WebSocket.subscribe("trades", "wss://feed.example.com/v1")
  .on_open { |conn| conn.send_text("{\"op\":\"subscribe\"}") }
  .on_message { |conn, text| Trade.record(text) }

Parameters:

  • name (String) —

    for the log and feed_status; unique

  • url (String) —

    ws:// or wss://

  • headers (Hash{String => String}) (defaults to: Client::NO_HEADERS) —

    extra handshake headers

  • timeout (Numeric) (defaults to: 5) —

    seconds per wait while connecting and per write

  • max_message (Integer) (defaults to: 1_048_576) —

    the largest message accepted, in bytes

  • ping_interval (Integer) (defaults to: 15) —

    seconds of quiet before a ping, at least 1

  • stale_after (Integer) (defaults to: 45) —

    seconds of quiet before reconnecting, more than ping_interval

  • allow_private (Boolean) (defaults to: Cybertrain.config.http_allow_private) —

    allow non-public destinations

Returns:

Raises:

  • (ArgumentError) —

    an empty, reserved (“status”) or taken name, a URL that is not ws:// or wss://, a header the client refuses, a bad interval



455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
# File 'cybertrain/websocket.rb', line 455

def self.subscribe(name, url, headers: Client::NO_HEADERS, timeout: 5, max_message: 1_048_576,
                   ping_interval: 15, stale_after: 45,
                   allow_private: Cybertrain.config.http_allow_private)
  label = name.to_s
  raise ArgumentError, "WebSocket.subscribe: the name is empty" if label.empty?
  # `./<app> feeds status` is the status command, so no feed can be
  # started by that name.
  raise ArgumentError, "WebSocket.subscribe: the name \"status\" is reserved (./<app> feeds status)" if label == "status"
  raise ArgumentError, "WebSocket.subscribe #{label.inspect}: already registered" if subscription_index(label) >= 0

  target = HttpClient.parse_url_for(url, "ws://", "wss://")
  ping = ping_interval.to_i
  stale = stale_after.to_i
  raise ArgumentError, "WebSocket.subscribe #{label.inspect}: ping_interval must be at least 1" if ping < 1
  raise ArgumentError, "WebSocket.subscribe #{label.inspect}: stale_after must be more than ping_interval" if stale <= ping
  raise ArgumentError, "WebSocket.subscribe #{label.inspect}: timeout must be positive" unless timeout.to_f > 0.0
  raise ArgumentError, "WebSocket.subscribe #{label.inspect}: max_message must be at least 1" if max_message.to_i < 1

  # The headers are checked now, not at every connect.
  Client.handshake_head(target, "", headers)
  sub = Subscription.new(label, url.to_s, target, headers, timeout.to_f, max_message.to_i, ping, stale, allow_private == true)
  SUBSCRIPTIONS << sub
  sub
end

.subscriptions_problem ⇒ String

Why start! would refuse to start: the first subscription without on_message, named. Application#serve asks it (through Background.boot_problem) before binding the port.

Returns:

  • (String) —

    ”” when every subscription has on_message



628
629
630
631
632
633
634
635
636
637
# File 'cybertrain/websocket.rb', line 628

def self.subscriptions_problem
  i = 0
  while i < SUBSCRIPTIONS.size
    sub = SUBSCRIPTIONS[i]
    return "WebSocket subscription #{sub.sub_name.inspect} has no on_message" if sub.sub_message_block.nil?

    i += 1
  end
  ""
end

.thread_alive?(name) ⇒ Boolean

Whether the thread of the feed named name is running.

Parameters:

  • name (String)

Returns:

  • (Boolean)

Raises:

  • (ArgumentError) —

    when no feed has that name



510
511
512
513
514
515
# File 'cybertrain/websocket.rb', line 510

def self.thread_alive?(name)
  i = subscription_index(name.to_s)
  raise ArgumentError, "WebSocket.thread_alive?: no subscription named #{name.to_s.inspect}" if i < 0

  sub_alive?(SUBSCRIPTIONS[i])
end

.unknown_feed(names) ⇒ String

unknown feed "NAME" (registered: a, b) for the first of names that is not a subscription’s name, “” when all are (./<app> feeds NAME... checks its arguments with it, through Background).

Parameters:

  • names (Array<String>)

Returns:

  • (String)


654
655
656
# File 'cybertrain/websocket.rb', line 654

def self.unknown_feed(names)
  Background.unknown_name_in(names, feed_names)
end

.withdraw_status ⇒ Integer

Deletes the status rows this process wrote, through the registered StatusWriters: a feeds process does it after a clean stop (Application#run_feeds_threads), so the health check stops counting feeds nobody runs. A crashed process deletes nothing, and its rows go stale. Does nothing while the feeds run.

Returns:

  • (Integer) —

    how many rows were deleted



610
611
612
613
614
615
616
617
618
619
620
621
# File 'cybertrain/websocket.rb', line 610

def self.withdraw_status
  return 0 if @running

  process = status_process
  removed = 0
  i = 0
  while i < STATUS_WRITERS.size
    removed += STATUS_WRITERS[i].status_withdraw(process)
    i += 1
  end
  removed
end