Module: Cybertrain::Multipart

Defined in:
cybertrain/http/multipart.rb

Overview

A binary-safe multipart/form-data parser (RFC 7578 over RFC 2046), in the HTTP layer so Request can use it before routing: CsrfProtection and MethodOverride read their field from a multipart body too.

Every offset is a byte offset (NOTES rule 27), and the whole parse runs on ONE binary copy of the body (body.b), so CRuby’s byteindex never sees a UTF-8 String with a stray byte (rule 52). File data is never compared with ==, interpolated, matched with a regexp or checked with valid_encoding? (rule 60): it is cut out with byteslice after the size gates, which look at offsets only. Scanning is linear: every search resumes where the last one stopped, and a hit that is only a look-alike (“–BOUNDARYX” inside file data) moves the cursor on by one byte.

The memory used is about three times the body per request in flight (the raw body, the binary copy, the slices), and the body is bounded by max_body_bytes.

Error messages are fixed strings and numeric limits: never request bytes.

Class Method Summary collapse

Class Method Details

.boundary_from(content_type_header) ⇒ String

The boundary parameter of a Content-Type header value (the parameter name is case-insensitive; the value is a token or a quoted-string).

Examples:

Multipart.boundary_from('multipart/form-data; boundary="xyz"') # => "xyz"

Parameters:

  • content_type_header (String) —

    the whole header value

Returns:

  • (String) —

    ”” when there is none, when it is given twice, or when it is not 1 to MAX_BOUNDARY_BYTES bytes of ASCII without CR, LF or NUL



183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
# File 'cybertrain/http/multipart.rb', line 183

def self.boundary_from(content_type_header)
  found = Multipart.params_named(content_type_header, "boundary", false)
  return "" unless found.length == 1

  b = found[0]
  n = b.bytesize
  return "" if n < 1 || n > MAX_BOUNDARY_BYTES

  i = 0
  while i < n
    byte = b.getbyte(i)
    return "" if byte == 0 || byte == 10 || byte == 13 || byte >= 128
    i += 1
  end
  b
end

.fill(params, form) ⇒ nil

Puts a parsed form into a Params tree: text fields through set_path (a nested name such as post[title] builds the tree, the same as an urlencoded body), files through set_upload_path. Called by the Router. The names are used as they are: they are not percent-decoded (a multipart name is raw text; Query.add_pair would decode “%5B”).

Parameters:

Returns:

  • (nil)

Raises:

  • (QueryTooMany) —

    more fields than Query::MAX_PAIRS

  • (QueryTooDeep) —

    a name nested deeper than Query::MAX_DEPTH

  • (MultipartMalformed) —

    a file sent as name[] (several files per field are not supported)



298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
# File 'cybertrain/http/multipart.rb', line 298

def self.fill(params, form)
  total = form.text_count + form.file_count
  raise QueryTooMany, "too many parameters (limit #{Query::MAX_PAIRS})" if total > Query::MAX_PAIRS

  i = 0
  while i < form.text_count
    path = Query.split_key(form.text_name_at(i))
    params.set_path(path, form.text_value_at(i))
    i += 1
  end
  i = 0
  while i < form.file_count
    path = Query.split_key(form.file_name_at(i))
    raise MultipartMalformed, "multiple files per field are not supported" if path[path.length - 1] == ""

    file = form.file_at(i)
    params.set_upload_path(path, file)
    i += 1
  end
  nil
end

.parse(body, content_type_header) ⇒ MultipartForm

Splits a multipart/form-data body into its text fields and files.

Parameters:

  • body (String) —

    the raw request body

  • content_type_header (String) —

    the Content-Type header value (it carries the boundary)

Returns:

Raises:



212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
# File 'cybertrain/http/multipart.rb', line 212

def self.parse(body, content_type_header)
  boundary = Multipart.boundary_from(content_type_header)
  raise MultipartMalformed, "missing or invalid multipart boundary" if boundary.empty?

  data = body.b
  delim = +""
  delim << "--" << boundary
  sep = +""
  sep << "\r\n" << delim
  seplen = sep.bytesize
  form = MultipartForm.new
  pos = Multipart.first_delimiter_end(data, delim, sep)
  parts = 0
  text_total = 0
  done = false
  while !done
    if Multipart.close_at?(data, pos)
      done = true
    else
      head_at = Multipart.padding_end(data, pos)
      raise MultipartMalformed, "malformed multipart delimiter" if head_at < 0

      parts += 1
      raise MultipartTooManyParts, "too many multipart parts (limit #{@max_parts})" if parts > @max_parts

      hend = data.byteindex("\r\n\r\n", head_at)
      raise MultipartMalformed, "multipart part has no end of headers" if hend.nil?
      raise MultipartHeaderTooLarge, "multipart part headers too large (limit #{MAX_HEADER_BYTES} bytes)" if hend - head_at > MAX_HEADER_BYTES

      block = data.byteslice(head_at, hend - head_at)
      heads = Multipart.read_headers(block)
      disposition = heads[0]
      part_type = heads[1]
      Multipart.require_form_data(disposition)
      name = Multipart.part_name(disposition)
      filenames = Multipart.params_named(disposition, "filename", true)
      raise MultipartMalformed, "duplicate filename in multipart part" if filenames.length > 1

      is_file = filenames.length == 1
      raw_filename = is_file ? filenames[0] : ""
      raise MultipartMalformed, "multipart filename too long (limit #{MAX_RAW_FILENAME_BYTES} bytes)" if raw_filename.bytesize > MAX_RAW_FILENAME_BYTES

      start = hend + 4
      nxt = Multipart.find_delimiter(data, sep, start)
      raise MultipartMalformed, "unterminated multipart part" if nxt < 0

      # The size gates look at offsets only: nothing is sliced yet.
      len = nxt - start
      if is_file
        raise MultipartPartTooLarge, "uploaded file too large (limit #{@max_upload_bytes} bytes)" if len > @max_upload_bytes

        # A file input left empty sends filename="" and no bytes: dropped.
        unless raw_filename.empty? && len == 0
          chunk = data.byteslice(start, len)
          fname = UploadedFile.sanitize_filename(raw_filename)
          dtype = UploadedFile.sanitize_type(part_type)
          up = UploadedFile.new(fname, dtype, chunk)
          form.add_file(name, up)
        end
      else
        text_total += len
        raise MultipartPartTooLarge, "multipart text fields too large (limit #{MAX_TEXT_BYTES} bytes)" if text_total > MAX_TEXT_BYTES

        chunk = data.byteslice(start, len)
        value = Query.utf8!(chunk)
        form.add_text(name, value)
      end
      pos = nxt + seplen
    end
  end
  form
end