SignalR, Azure and Scaleout
Recently Resgrid started utilizing SignalR to allow real-time updates to clients. If your unfamiliar with SignalR is a server/client Web Socket framework giving an easy API for Microsoft backend server solutions (like MVC, ASP.Net, etc) to utilize Web Sockets. Resgrid has a fairly simple infrastructure: Dual Azure Web App instances for web and api, backend SQL database and workers.
Because of our multi web server configuration we needed to use a Scaleout system, to allow for clients connected to one web or api instances to receive/send to all instances. If your running multiple web servers and your users trigger SignalR events on one instances, without scaleout only users connected to that same instance will get the event.
SignalR currently supports three methods for scaling out, Redis, Azure Service Bus and SQL Server. Unfortunately with the current version of SignalR (2.2.0) Redis and Azure Service Bus options are in some stages or broken.
This is an unfortunate situation, SignalR’s last release was December 11th 2014, that’s well over a year ago for a pretty critical technology on the Microsoft stack. The last commits on the repository’s master branch (at this time are over 4 months old). If this Github was a physical place, I’d expect to see boarded up windows and tumble weeds. There is light on the maintenance front, SignalR-Server (a different repo) appears to be pretty active.
Why is this an issue? Well because the SignalR repo is over a year old it’s out of sync with the Azure Service Bus assemblies. As of this writing the latest Service Bus Nuget packages greater then 3.0 are incompatible with SignalR Azure Service Bus Scaleout. The error your likely to receive will be something like:
Could not load type ‘Microsoft.ServiceBus.Messaging.MessageClientEntity’ from assembly ‘Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’
If you can stay in lock-step with SignalR’s Azure Service Bus version (2.1.2) then this is a viable option for scaleout, but for most projects this won’t work. Especially if your utilizing features like Azure Web Jobs, which require a version of Service Bus >3 to use some of it’s features.
Another option is to use a Redis server for scaleout. But it seems that Redis is currently in just as bad (or maybe worse) state. Using Redis for scaleout you may not be able to keep an open connection between SignalR and Redis for any period of time, or at all.
Errors you may receive may be:
SignalR.ScaleoutMessageBus Error: 0 : Stream(0) – Send failed: System.AggregateException: One or more errors occurred. —> System.ObjectDisposedException: Cannot access a disposed object.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
at System.Runtime.CompilerServices.TaskAwaiter article.ThrowForNonSuccess(System.Threading.Tasks.Task task) at offset 38
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task task) at offset 40
at Exceptionless.Api.Hubs.MessageBusConnection.<OnConnected>d__2.MoveNext() at offset 322 in C:\projects\exceptionless\Source\Api\Hubs\MessageBusConnection.cs:line 23:col 17
The connection has been closed (Error); no new messages can be delivered; the last command was sent xxxxms ago
In testing on our Azure Web Apps instances talking to the Azure Redis server we found that Redis scaleout is completely unusable. At this point the only scaleout option that seems to work is SQL Server. Although we don’t use this option (we have a hacked version using the latest Service Bus version). Our reasoning is we didn’t want to put any additional load on our SQL server, which might not a concern for your needs.
It’s a sad state of affairs for SignalR. Although not totally abandoned there doesn’t seem to much, if any, resources put behind it. Microsoft has pushed this technology hard in the .Net space and, at a minimum, open and honest communication would be nice.
My guess is that like Entity Framework 6.x they will only release bug fix updates and minor versions, while they spin up a SignalR Core style version, matching the pattern of EF6 and EF Core.
If you’re a First Responder or know one check out Resgrid which is a SaaS product utilizing Microsoft Azure, providing logistics, management and communication tools to first responder organizations like volunteer fire departments, career fire departments, EMS, search and rescue, CERT, public safety, disaster relief organizations.
