Class: Cybertrain::HttpClient
- Inherits:
-
Object
- Object
- Cybertrain::HttpClient
- Defined in:
- cybertrain/http_client.rb
Overview
Sends HTTP requests (GET, POST, PUT, PATCH, DELETE, HEAD) to other servers, with bounded waits.
Defined Under Namespace
Classes: BadResponseError, BlockedAddressError, Error, RefusedError, Response, TLSError, TimeoutError, TooLargeError, TooManyRedirectsError
Instance Attribute Summary collapse
-
#allow_private ⇒ Boolean
readonly
Whether non-public destinations are allowed.
-
#deadline ⇒ Float
readonly
Seconds the whole request may take.
-
#max_body ⇒ Integer
readonly
The largest body accepted, in bytes.
-
#redirects ⇒ Integer
readonly
How many redirects a request follows (0: none).
-
#timeout ⇒ Float
readonly
Seconds any one wait may take.
Class Method Summary collapse
-
.gzip=(decoder) ⇒ GzipDecoder
Replaces the gzip support (cybertrain/http/gzip_ffi installs its decoder when it is required).
-
.gzip_decoder ⇒ GzipDecoder
The gzip support requests use: GzipDecoder’s base (no gzip) until
require "cybertrain/http/gzip_ffi"installs the zlib one. -
.resolve_location(current, location) ⇒ String
Resolves a redirect’s Location against the URL that answered it (RFC 3986 section 5.2): an absolute URL as it is,
//host/pathwith the current scheme,/pathon the current origin,?queryon the current path, a relative path against the current path’s directory, dot segments removed.
Instance Method Summary collapse
-
#get(url, headers: NO_HEADERS) {|chunk| ... } ⇒ Response
GET
url; with a block, the body is streamed to it (see #request). -
#get_to_file(url, path, headers: NO_HEADERS) ⇒ Response
GETs
urlinto the filepath: a 2xx body is written topath + ".part"as it arrives (never held in memory,max_bodydoes not apply) and renamed ontopathwhen the response is complete. -
#initialize(timeout: 5, deadline: 15, max_body: 1_048_576, allow_private: Cybertrain.config.http_allow_private, redirects: 0) ⇒ HttpClient
constructor
A new instance of HttpClient.
-
#post(url, body, content_type: "application/json", headers: NO_HEADERS) {|chunk| ... } ⇒ Response
POST
bodytourl; with a block, the response body is streamed to it (see #request). -
#request(verb, url, body: "", content_type: "", headers: NO_HEADERS) {|chunk| ... } ⇒ Response
Sends
verbtourl.
Constructor Details
#initialize(timeout: 5, deadline: 15, max_body: 1_048_576, allow_private: Cybertrain.config.http_allow_private, redirects: 0) ⇒ HttpClient
Returns a new instance of HttpClient.
396 397 398 399 400 401 402 403 404 405 406 407 |
# File 'cybertrain/http_client.rb', line 396 def initialize(timeout: 5, deadline: 15, max_body: 1_048_576, allow_private: Cybertrain.config.http_allow_private, redirects: 0) @timeout = timeout.to_f @deadline = deadline.to_f @max_body = max_body.to_i @allow_private = allow_private == true @redirects = redirects.to_i raise ArgumentError, "timeout must be positive" unless @timeout > 0.0 raise ArgumentError, "deadline must be positive" unless @deadline > 0.0 raise ArgumentError, "max_body must be at least 0" if @max_body < 0 raise ArgumentError, "redirects must be at least 0" if @redirects < 0 end |
Instance Attribute Details
#allow_private ⇒ Boolean (readonly)
Returns whether non-public destinations are allowed.
378 379 380 |
# File 'cybertrain/http_client.rb', line 378 def allow_private @allow_private end |
#deadline ⇒ Float (readonly)
Returns seconds the whole request may take.
372 373 374 |
# File 'cybertrain/http_client.rb', line 372 def deadline @deadline end |
#max_body ⇒ Integer (readonly)
Returns the largest body accepted, in bytes.
375 376 377 |
# File 'cybertrain/http_client.rb', line 375 def max_body @max_body end |
#redirects ⇒ Integer (readonly)
Returns how many redirects a request follows (0: none).
381 382 383 |
# File 'cybertrain/http_client.rb', line 381 def redirects @redirects end |
#timeout ⇒ Float (readonly)
Returns seconds any one wait may take.
369 370 371 |
# File 'cybertrain/http_client.rb', line 369 def timeout @timeout end |
Class Method Details
.gzip=(decoder) ⇒ GzipDecoder
Replaces the gzip support (cybertrain/http/gzip_ffi installs its decoder when it is required). The reader is gzip_decoder.
363 364 365 |
# File 'cybertrain/http_client.rb', line 363 def self.gzip=(decoder) GzipDecoder.installed = decoder end |
.gzip_decoder ⇒ GzipDecoder
The gzip support requests use: GzipDecoder’s base (no gzip) until
require "cybertrain/http/gzip_ffi" installs the zlib one. With it,
every request says Accept-Encoding: gzip (unless the caller sent an
Accept-Encoding of its own) and a Content-Encoding: gzip (or
x-gzip) body is inflated: a kept body under max_body (the bytes on
the wire and the inflated bytes alike), a streamed one piece by piece,
at most 64 KiB at a time. Without it, a gzip response is a
BadResponseError naming the file to require; a Content-Encoding other
than gzip or identity is one either way. The Response’s headers are
the ones received (Content-Encoding included).
354 355 356 |
# File 'cybertrain/http_client.rb', line 354 def self.gzip_decoder GzipDecoder.installed end |
.resolve_location(current, location) ⇒ String
Resolves a redirect’s Location against the URL that answered it
(RFC 3986 section 5.2): an absolute URL as it is, //host/path with
the current scheme, /path on the current origin, ?query on the
current path, a relative path against the current path’s directory,
dot segments removed. The fragment is dropped.
725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 |
# File 'cybertrain/http_client.rb', line 725 def self.resolve_location(current, location) base = current.to_s # NOTES rule 62 loc = HttpClient.without_fragment(location.to_s) return loc if HttpClient.scheme_prefix?(loc) base = HttpClient.without_fragment(base) n = base.bytesize mark = base.index("://") raise ArgumentError, "the current URL has no scheme" if mark.nil? if loc.start_with?("//") out = String.new out << base.byteslice(0, mark) << ":" << loc return out end stop = mark + 3 while stop < n b = base.getbyte(stop) break if b == 47 || b == 63 # / ? stop += 1 end origin = base.byteslice(0, stop) rest = base.byteslice(stop, n - stop) query_at = rest.index("?") path = query_at.nil? ? rest : rest.byteslice(0, query_at) path = "/" if path.empty? out = String.new out << origin if loc.empty? out << path out << rest.byteslice(query_at, rest.bytesize - query_at) unless query_at.nil? return out end if loc.start_with?("?") out << path << loc return out end loc_query_at = loc.index("?") loc_path = loc_query_at.nil? ? loc : loc.byteslice(0, loc_query_at) merged = String.new if loc.start_with?("/") merged << loc_path else slash = path.rindex("/") merged << path.byteslice(0, slash + 1) unless slash.nil? merged << loc_path end out << HttpClient.remove_dot_segments(merged) out << loc.byteslice(loc_query_at, loc.bytesize - loc_query_at) unless loc_query_at.nil? out end |
Instance Method Details
#get(url, headers: NO_HEADERS) {|chunk| ... } ⇒ Response
GET url; with a block, the body is streamed to it (see #request).
417 418 419 |
# File 'cybertrain/http_client.rb', line 417 def get(url, headers: NO_HEADERS, &block) request("GET", url, headers: headers, &block) end |
#get_to_file(url, path, headers: NO_HEADERS) ⇒ Response
GETs url into the file path: a 2xx body is written to
path + ".part" as it arrives (never held in memory, max_body does
not apply) and renamed onto path when the response is complete. The
body of any other final status (a 404 page, a 503) never reaches the
disk: it is read and counted, the .part is removed and path is
left as it was; check Cybertrain::HttpClient::Response#success?. On any error (or an
interrupt) the .part file is removed and path is left as it was.
497 498 499 500 501 502 503 504 |
# File 'cybertrain/http_client.rb', line 497 def get_to_file(url, path, headers: NO_HEADERS) final = path.to_s part = "#{final}.part" file = File.open(part, "wb") sink = FileChunkSink.new(file) res = fetch_into(url.to_s, final, part, file, sink, headers) Response.new(res.status, res.headers, "", sink.sink_bytes) end |
#post(url, body, content_type: "application/json", headers: NO_HEADERS) {|chunk| ... } ⇒ Response
POST body to url; with a block, the response body is streamed to
it (see #request).
432 433 434 |
# File 'cybertrain/http_client.rb', line 432 def post(url, body, content_type: "application/json", headers: NO_HEADERS, &block) request("POST", url, body: body, content_type: content_type, headers: headers, &block) end |
#request(verb, url, body: "", content_type: "", headers: NO_HEADERS) {|chunk| ... } ⇒ Response
Sends verb to url. POST, PUT and PATCH always carry body (and
its Content-Length, and content_type unless empty); DELETE carries
one when it is not empty; GET and HEAD carry none. Content-Type
accompanies a body only: a content_type: given to a request that
sends none (GET, HEAD, a DELETE with an empty body) is not sent. A
HEAD response has no body.
With a block, the response body is not kept: each piece is yielded as
it arrives (a chunked frame, up to 1 MiB at a time, or what each read
delivered) and the Response’s body is "". max_body does not apply
then; timeout and deadline do, so give a long stream a long
deadline:. The body of a redirect that is followed is never yielded.
464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 |
# File 'cybertrain/http_client.rb', line 464 def request(verb, url, body: "", content_type: "", headers: NO_HEADERS, &block) name = verb.to_s unless VERBS.include?(name) = "unknown HTTP verb #{name.inspect}: GET, POST, PUT, PATCH, DELETE or HEAD" raise ArgumentError, end data = body.to_s if !data.empty? && (name == "GET" || name == "HEAD") = "a #{name} request carries no body" raise ArgumentError, end if block.nil? perform(name, url.to_s, data, content_type.to_s, headers, ChunkSink.new) else perform(name, url.to_s, data, content_type.to_s, headers, BlockChunkSink.new(block)) end end |