Gun 2.0

The right of Erlang nodes to keep and
bear arms shall not be infringed. #2A

Loïc Hoguin, Nine Nines

What is Gun?

Erlang Web client

  • HTTP/1.1
  • HTTP/2
  • Websocket
  • SSE (server-sent events / eventsource)
  • CONNECT and Socks proxies

Asynchronous

  • Functions to perform actions
  • Most functions return immediately
  • Messages sent on input
  • Works very well with gen_statem:

Close to specs

If the specification allows it, Gun must allow it

Allow Gun to work through the
most exotic network topologies

Features

Connection management

  • One process per connection
  • Supervised under Gun by default, configurable
  • No connection pooling in Gun itself

Always connected

  • Gun is built as an always-connected client
  • Retries connecting after losing the connection
  • retry and retry_timeout by default
  • retry_fun for more complex needs

One function per method

  • gun:get, gun:head
  • gun:patch, gun:post, gun:put
  • gun:delete
  • gun:options
  • gun:headers (and gun:data), gun:request

Responses as messages

  • gun_up, gun_down
  • gun_upgrade
  • gun_error
  • gun_push
  • gun_inform, gun_response
  • gun_data, gun_trailers
  • gun_ws

Semi-synchronous mode


{ok, Pid} = gun:open("host.name", 443),
{ok, http2} = gun:await(Pid),
Ref = gun:get(Pid, "/"),
{response, nofin, 200, RespHeaders} = gun:await(Pid, Ref),
{ok, RespBody} = gun:await_body(Pid, Ref).
	

Flow control


Ref = gun:get(Pid, "/", [], #{flow => 1}),
{response, nofin, 200, _} = gun:await(Pid, Ref)
{data, nofin, _} = gun:await(Pid, Ref),
{error, timeout} = gun:await(Pid, Ref),
gun:update_flow(Pid, Ref, 2),
{data, nofin, _} = gun:await(Pid, Ref),
{data, nofin, _} = gun:await(Pid, Ref),
{error, timeout} = gun:await(Pid, Ref).
	

Configurable per stream or per protocol:


{ok, Pid} = gun:open("host.name", 443, #{
	http_opts => #{flow => 10}
}).
	

Websocket


{ok, Pid} = gun:open("host.name", 443),
{ok, _} = gun:await_up(Pid),
Ref = gun:ws_upgrade(Pid, "/ws"),
{upgrade, [<<"websocket">>], _} = gun:await(Pid, Ref),
gun:ws_send(Pid, Ref, [
	{text, <<"Hello!">>},
	{binary, <<1,2,3,4>>}
]),
{ws, Frame} = gun:await(Pid, Ref).
	

Websocket over HTTP/2 currently missing.

SSE


CommonOpts = #{content_handlers => [gun_sse_h, gun_data_h]},
{ok, Pid} = gun:open("host.name", 443, #{
	http_opts => CommonOpts,
	http2_opts => CommonOpts 
}).
	

Ref = gun:get(Pid, "/events", #{
	<<"accept">> => <<"text/event-stream">>}),
]),
{response, nofin, 200, RespHeaders} = gun:await(Pid, Ref),
{sse, Event} = gun:await(Pid, Ref),
#{
	last_event_id := LastEventID,
	event_type := <<"message">>,
	data := Data
} = Event.
	

CONNECT support


{ok, Pid} = gun:open("proxy1.name", 443),
Ref1 = gun:connect(Pid, #{
	host => "proxy2.name", port => 443
}),
{response, fin, 200, _} = gun:await(Pid, Ref1),
Ref2 = gun:connect(Pid, #{
	host => "origin.name", port => 443
}),
{response, fin, 200, _} = gun:await(Pid, Ref2),
Ref3 = gun:get(Pid, "/"),
{response, nofin, 200, RespHeaders} = gun:await(Pid, Ref3),
{ok, Body} = gun:await_body(Pid, Ref3).
		

#{
	transport := tls, protocol := http,
	origin_scheme := <<"http">>,
	origin_host := "origin.name", origin_port := 443,
	intermediaries := [
	#{	type := connect,
		host := "proxy1.name", port := 443,
		transport := tls, protocol := http},
	#{	type := connect,
		host := "proxy2.name", port := 443,
		transport := tls, protocol := http}]
} = gun:info(Pid).
		

CONNECT over HTTP/2 currently missing.

Socks support

  • Currently being worked on
  • Similar to CONNECT
  • Open connection to server using socks protocol
  • Options determine where to connect next
  • Exotic scenarios: Socks -> CONNECT -> Socks -> ...
  • Support Socks4a, Socks5
  • Socks6 to be supported as well once available

Events

  • init, terminate
  • domain_lookup_start, domain_lookup_end
  • connect_start, connect_end
  • tls_handshake_start, tls_handshake_end
  • disconnect
  • request_start, request_headers, request_end
  • push_promise_start, push_promise_end
  • response_start, response_inform, response_headers, response_trailers, response_end
  • ws_upgrade
  • ws_recv_frame_start, ws_recv_frame_header, ws_recv_frame_end
  • ws_send_frame_start, ws_send_frame_end
  • protocol_changed
  • transport_changed
  • origin_changed
  • cancel

Wishlist

Some features might make it into 2.0.
Others in 2.x releases.

Raw socket mode

  • Using HTTP/1.1 Upgrade to non-Web protocols
  • Using CONNECT to access non-HTTP servers
    • Including over HTTP/2

Cookie jars

  • Automatic storing and retrieving of cookies
  • Jars exclusive to one connection
  • Jars shared between connections
  • Options for a per-request jar
  • Filters to allow/reject specific cookies
  • Configurable storage (memory, disk, DB)

Client-side cache

  • RFC 7234 (or corresponding httptre RFC)
  • Cache responses per connection/shared
  • Skip requests for which we have a cached response
  • Option to skip the cache when doing a request
  • Configurable storage (memory, disk, DB)

Internals

gen_statem

  • not_connected
  • domain_lookup
  • connecting
  • tls_handshake
  • connected
  • closing
  • One callback module per protocol
  • Most callbacks return commands
    • For example to switch the protocol

HTTP/2 machine

  • HTTP/2 code shared with Cowboy
  • Improvements benefit both projects
  • Same options available for both projects

TLS over TLS

  • gun_tls_proxy and gun_tls_proxy_cb
  • Uses ssl's callback module option
  • Routes the output to Gun, not a socket
  • Makes data go through multiple layers of encryption

Upcoming

  • Gun 2.0.0-pre.1 to be released in September 2019
    • Contains many improvements for elixir-grpc
  • Cowboy 2.7 with many improvements for elixir-grpc
  • Ranch 2.0:
    • num_acceptors
    • num_conns_sups
    • num_listen_sockets
  • Serious work started on REST framework

Thanks

Sponsors:

  • SameRoom.io / 8x8 (long term sponsorship)
  • Pleroma (Socks5 and more)
  • Kobil (elixir-grpc related improvements)

ninenines.eu