[99s-extend] Cowboy load test

[Loïc Hoguin]

This sounds like you have too many timers, or something else is using 
the CPU. 40k connections is literally nothing. Also make sure the 
clients are on a separate VM/machine.

Either way 99% CPU for 10k sounds high, I've had that with 0 CPU (though 
on real hardware).

Someone a couple years back got above 1 million after fixing 'too many 
timers issues' but I don't think it was on a medium instance.

On 09/15/2013 07:01 PM, Joe Freeman wrote:
> Hi,
>
> I've started work on a project using Clojure, but I was wondering
> whether (and secretly hoping that) Erlang would be a better fit, so I've
> been load testing a few web server frameworks. I'm particularly
> interested in how the server can handle a large number of concurrent
> WebSocket connections, and the test I've been running is similar to Eric
> Moritz's [1].
>
> I've setup a simple Cowboy 'echo' server running on an EC2 instance
> (m1.medium, as in Eric's test) which could comfortably handle 10k
> concurrent WebSocket requests (as in Eric's results), while echoing
> about 200 messages/second. The CPU usage of the VM at this point is
> about 99%, but the server continues to handle up to 40k concurrent
> connections with a consistent average response time (<30ms). Pushing the
> test beyond this number results in a spike in response times and lots of
> connection timeouts.
>
> 40k connections seems pretty good, but when comparing this to the same
> test against a couple of Clojure/JVM-based frameworks (specifically
> Aleph/Netty and http-kit) I find I can get higher numbers of concurrent
> connections with slightly better average response times (100k
> connections, <10ms response time) using much less CPU (~20%). In fact,
> memory seems to be the limiting factor.
>
> So I have two questions:
>
> 1) Should I be concerned about the CPU usage in the Erlang/Cowboy test?
> I have limited experience with Erlang so far, but 100% CPU feels like a
> bad thing.
>
> 2) Is there any way to increase the performance of the cowboy server?
> Are there any Erlang VM parameters I can change? The fact that the
> Clojure/JVM tests (on the same machine) have managed to get to 100k
> connections suggests that the limitation isn't being imposed by the
> operating system (I've applied changes various changes to sysctl and
> ulimit).
>
> (Perhaps an echo server isn't the best way to compare HTTP servers, but
> it feels like a good starting point.)
>
> Thanks for any help.
>
> [1] https://github.com/ericmoritz/wsdemo/blob/results-v1/results.md -
> the GitHub repo actually contains code for an Aleph server, but results
> from this aren't included in the summary here.
>
>
>
> _______________________________________________
> Extend mailing list
> Extend at lists.ninenines.eu
> http://lists.ninenines.eu:81/listinfo/extend
>


-- 
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu