Binding customization and throttles-Load testing a PollingDuplexHttp WCF service-Part 2

Part 1 of this entry already covered a load testing app for a service, and in this part I’m looking at some of the binding problems I found when I started hammering the service with it.

I’d already run into the MaxSessionsPerAddress throttle in my original prototype, but the MaxEstablishingSessions throttle didn’t come into play until I started load testing with more than 10 clients at once.  The tester hit this throttle because it created a large number of proxies at the same time. If this throttle is exceeded, an HTTP error is returned by the service (Service too busy), however it’s worth noting that this does NOT fault the proxy. The tests in the library all have a retry mechanism whereby they will retry the operation if an http error is received (optionally waiting between attempts).  In most cases, I put a 1 second retry delay between attempts at the service.

The default value for this throttle is at 10 sessions, but what if you want to customize it?  Well, it turns out that it’s not exposed on PollingDuplexHttpBinding, so it means creating a custom binding.  As it turns out, this can be easier than you’d think:

PollingDuplexHttpBinding b= new PollingDuplexHttpBinding(PollingDuplexHttpSecurityMode.TransportWithMessageCredential, System.ServiceModel.Channels.PollingDuplexMode.MultipleMessagesPerPoll);
b.Security.Transport.ClientCredentialType = HttpClientCredentialType.Digest;
b.Namespace = "";
b.SendTimeout = new TimeSpan(0, 0, 10);  //timeout for sending messages to the client channel. 
b.InactivityTimeout = new TimeSpan(0, 5, 0);  //minimum message frequency-if no traffic in this interval, channel is faulted. 
b.ServerPollTimeout = new TimeSpan(0, 10, 0);
b.ReceiveTimeout = new TimeSpan(0, 10, 0);
b.UseTextEncoding = false;
b.MaxReceivedMessageSize = int.MaxValue;  

//customize the binding...

CustomBinding bc = new CustomBinding(b);
foreach (BindingElement e in bc.Elements)
    if (e is PollingDuplexBindingElement)
        ((PollingDuplexBindingElement)e).MaxPendingMessagesPerSession= 1000;

You create your PollingDuplexHttpBinding (b) with all the security, encoding, and polling options that you normally would, but then just create a custom binding based on the original , add your new throttles, and use that binding instead of the standard one.  Best of all, there are no code changes on the client, everything else seems to work.

Now, the question is, should you change that value?  Here’s some tests I ran using the default value for MaxPendingSessions.  The number of retries is the number of times a service call failed because of a “service busy” response, and each had a 1s delay followed by another attempt.  The number of threads is the number of simultaneous clients started against the service:

# Threads Average time Max Time Retries
10 2 2.05 0
50 3 4.76 21 1x
100 6 8.73 90 1x 18 2x

As you can see, large numbers of clients resulted in large numbers of retries.  Once I changed the throttle to 1000 though, here’s what happened:

# Threads Avg Max Retries
10 1 1.72 0
50 3 3.74 0
100 14 23.75 0

All of these tests were doing the same thing, so as the number of threads scales up, the performance of the service as a whole starts to actually decrease.  True, a slightly higher value (maybe 20 or 30) may work out to be optimal, but it appears that letting the calls fail might be the better option.

Note that this throttle only really affects new sessions.  Another test I tried had threads create a proxy, wait, and then invoke a second method.  In that case, once the first method has been successfully called, the client didn’t get any more http failures from the service.  In cases where there’s a network interruption between your clients and service though, you could see this throttle come into play, so the important take away here is to build a suitable retry logic into the client when accessing a service like this.

This entry was posted in Silverlight, WCF. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s