Efficient Web Applications with Erlang and Cowboy

The Web is made for Erlang

Loïc Hoguin - @lhoguin

Erlang Cowboy and Nine Nines Founder

The Web yesterday

The Web today

What makes it possible?

• XmlHTTPRequest
• EventSource
• Websockets
• SPDY

What does the Web needs exactly?

• Efficiency
• Concurrency
• Availability

Efficiency?

• Low latency
• Low bandwidth footprint
• Low memory footprint

Concurrency?

• High number of concurrency connections
• True concurrency: each request has the same weight
• No loss of latency due to an expensive connection
• No loss of latency due to many connections

Availability?

According to Joe Armstrong, you need:

• Isolation
• Concurrency
• Failure detection
• Fault identification
• Live code upgrade
• Stable storage

Trine

• Efficiency requires concurrency
• Concurrency requires availability
• No point measuring efficiency if the server is unavailable

Erlang, Erlang, Erlang

• Erlang did this long before it became important for the web
• Erlang did this long before multicore CPUs
• Erlang gives you all this for free

Erlang models the real world

• People don't share memory
• People evolve concurrently
• People communicate by sending messages
• People die, for a reason we can identify
• The world doesn't crash when someone dies

Error handling and recovery

• Erlang features pattern matching
• Error handling is straightforward: "let it crash!"

  
      ok = file:write_file(Name, Contents)
  

• Application design depends on how you want them to fail
• Your system becomes failure-safe, even from your own bugs

Erlang is easy to read and write

• Code is written sequentially
• Asynchronous code is hidden behind a module API
• The language is very small and easy to understand
• Modules are written in a top-down manner

Erlang systems grow naturally

• It's almost the same code to go from 1 to 2 nodes
• It's the same code to go from 2 to 1000 nodes
• It's the same code to handle local or remote faults or netsplits

What does a HTTP server look like?

• Accept connection
• Receive and parse request line
• Receive and parse header lines
• Do something
• Send a reply
• Maybe process another request

This is what Cowboy is doing

Then is Cowboy sequential?

• Accepting the connection in a separate process
• Many processes accept connections concurrently
• Requests are processed concurrently

Is it really effi-- YES!

• Prove it!
• Sure.

And then I realized...

• Testing efficiency is hard!
• Many tools exist to benchmark web servers
• Problem: they don't scale as much as Cowboy

Trying out Siege

% siege -b -c 100 -t 30s http://192.168.1.101:8080 
** SIEGE 2.70
** Preparing 100 concurrent users for battle.
Transactions:                 518565 hits
Availability:                 100.00 %
Elapsed time:                  29.76 secs
Data transferred:               5.93 MB
Response time:                  0.01 secs
Transaction rate:           17424.90 trans/sec
Throughput:                     0.20 MB/sec
Concurrency:                   98.86
Successful transactions:      518565
Failed transactions:               0
Longest transaction:            0.06
Shortest transaction:           0.00

Bigger Siege

% siege -b -c 1000 -t 30s http://192.168.1.101:8080
** SIEGE 2.70
** Preparing 1000 concurrent users for battle.
Transactions:                 106280 hits
Availability:                 100.00 %
Elapsed time:                  52.41 secs
Data transferred:               1.22 MB
Response time:                  0.37 secs
Transaction rate:            2027.86 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                  747.29
Successful transactions:      106281
Failed transactions:               0
Longest transaction:            6.26
Shortest transaction:           0.00

The thing about benchmarks

• They're never a "fact" to anyone other than yourself
• Perform your own benchmarks before taking decisions
• Always check where the bottleneck is

The Horse project

Sweet Poney benchmark

• Loops of "Connection: close" requests
• Start with 500 concurrent clients
• Add 500 clients every 3 seconds
• 5K clients running after just 27s

Hardware

• Same hardware type for client and server
• Motherboard: ASUSTeK Computer INC. M5A78L-M LX
• AMD Athlon(tm) II X4 640 Processor (3000 Mhz)
• 4 GB of 1333 Mhz memory

Hello world results

Raspberry Pi results

Raspberry Siege

% siege -b -c 100 -t 30s http://192.168.1.14:8080
** SIEGE 2.70
** Preparing 100 concurrent users for battle.
Transactions:                   7986 hits
Availability:                 100.00 %
Elapsed time:                  29.58 secs
Data transferred:               0.09 MB
Response time:                  0.37 secs
Transaction rate:             269.98 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   99.27
Successful transactions:        7986
Failed transactions:               0
Longest transaction:            0.52
Shortest transaction:           0.20

1KB static file

100KB static file

• Benchmark client breaks down
• When it doesn't it's serving about 1000 files/s
• Couldn't get a proper graph
• But I don't recommend serving files from Erlang

Integrate Cowboy in your applications

• Experimental FCGI handler
• rake for Rails
• Direct some queries to Erlang and others to your existing system

Hello PHP FCGI

% siege -b -c 100 -t 30s http://192.168.1.101:8080/hello.php
** SIEGE 2.70
** Preparing 100 concurrent users for battle.
Transactions:                  65535 hits
Availability:                 100.00 %
Elapsed time:                  29.94 secs
Data transferred:               0.75 MB
Response time:                  0.03 secs
Transaction rate:            2188.88 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   56.34
Successful transactions:       65535
Failed transactions:               0
Longest transaction:            0.43
Shortest transaction:           0.00

Experimental FCGI code is experimental

wsdemo benchmark

Real world application stats

• Dating website chat, peak time
• 70k sessions online
• 35k active websocket connections
• 

Is it webscale?

• Tested with 1 million active websocket connections on Amazon EC2
• Takes about 20GB of memory
• That's about 20KB per connection

Is there a catch?

• Yes
• We haven't started optimizing Cowboy

How is this possible?

Erlang the OS

• Erlang the VM runs a release
• Release contains applications
• Applications contain processes
• Processes are supervised
• Erlang the VM is also supervised

Real world Erlang system design

• Each connection is its own process
• Each session is its own process
• Session data is kept inside the process
• Live events are kept inside the session process
• Cache is often done internally in your application
• Results in few database and FS calls

Erlang system design 101

• Think about how you want your system to fail
• Protect your core services into inner layers
• Just let outer layers fail
• Results in small focused services applications
• They only do one thing and do it well

Architecture of an Erlang web app

• Thin HTTP communication layer
• Makes calls to services in your system
• Services are live processes, not a model library
• Services can be on this or another node
• Connection processes are mostly waiting for replies
• They do not prevent other connections from running!

Scale to the infinite and beyond

• Remove bottlenecks (core processes...)
• Distribute responsability (master-less)
• Let it crash (but handle faults where it matters)
• Reduce disk or network access
• Measure

Erlang gives you the tools to measure

• Default installation contains:
  • 4+ profilers
  • 2+ debugging interfaces
  • 3+ trace interfaces
  • Direct peeking into running processes
  • Tons of statistics
  • SNMP
  • A lot more...

What you don't need anymore

• MVC and friends
• Cache servers (memcache, redis...)
• Messaging servers (Rabbitmq, 0mq...)
• Overtime

Getting started

• https://github.com/extend/cowboy
• http://learnyousomeerlang.com
• #erlang and #erlounge on irc.freenode.net
• erlang-questions mailing list
• Contact me directly! @lhoguin or [email protected]

Questions?