[99s-extend] Cowboy HTTPS connection memory usage

[rambocoder]

In my preliminary testing, I used Jmeter this morning since it's an
easy GUI load testing app and this is what I am seeing:

With R15B03-01 [smp:4:4] [async-threads:4] [hipe] [kernel-poll:true], when
I establish 1K concurrent connections via HTTPS, each connection takes up
about 68K of memory.

Unfortunately, after about 1050-1200 connections, on my test server the
Erlang scheduler jumps to 100% CPU utilization on all 4 schedulers, while
up to that point the scheduler's load was oscillating up and down. Using
the Observer, there is only 1 ssl_connection_sup  in the ssl application,
having to deal with 1000+ gen_fsm workers, so that might be the bottleneck.
Since the ulimit on my server is 50000 I don't think I am hitting any type
of file handler's limit.

Loïc and the group, am I missing some setting that is causing the scheduler
to go to 100% CPU and the run que in observer to be 99?

Sincerely,

rambocoder


On Fri, Dec 21, 2012 at 6:45 AM, Loïc Hoguin <essen at ninenines.eu> wrote:

> On 12/21/2012 04:34 AM, rambocoder wrote:
>
>> Does anybody know either from benchmarks or real world data what is the
>> average memory footprint of each concurrent HTTPS connection to cowboy?
>>
>
> I don't have anything, sorry. I'm guessing it consumes a lot more than TCP
> though.
>
>
>  SSL app in Erlang reuses SSL session-ids so I am not sure if the Apache
>> Bench I test with reuses the session id or it does not.
>>
>
> I wouldn't know, but I wouldn't trust Apache Bench doing the right thing.
> Any other benchmark tool usually works better in my experience.
>
>
>  BTW, what makes an erlang api "documented" vs "undocumented". For
>> example ssl:session_info/1 function here (
>> https://github.com/erlang/otp/**blob/maint/lib/ssl/src/ssl.**erl#L411<https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl.erl#L411>) has
>> a spec and a short doc, but session_info is not described
>> http://www.erlang.org/doc/man/**ssl.html<http://www.erlang.org/doc/man/ssl.html>.ssl:session_info/1 is a useful
>> function to be able to track if the load generator is reusing the SSL
>> session_id or it is generating new one, because that would make all the
>> difference during measurement due to Erlang caching SSL sessions by
>> default.
>>
>
> The documentation is separate (they're not using edoc). It's perhaps not
> deemed useful enough for documenting it. I wouldn't worry about using it
> for measurements though.
>
> Try asking Ingela on the ML about it, perhaps they just forgot to document
> it.
>
> --
> Loďc Hoguin
> Erlang Cowboy
> Nine Nines
> http://ninenines.eu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ninenines.eu/archives/extend/attachments/20121221/8bfb2f11/attachment.html>