Today I’ve upgraded a SignalR library from 1.x to 2.0 in an MVC 5 (server) and WPF (client) projects I am developing. The upgrade itself went smooth but afterwards the client started yielding odd errors and refusing to connect anymore. It was yielding an exception saying something like Transport Connection time out (I can’t remember the exact message). Since it was a rather simple upgrade this error made no sense. Google didn’t help much either.
So I went debugging.
Investigation
First step is to create a simple repro sample – a solution with a basic server and a console client (see attached zip).
Second step is to turn on SignalR tracing on client:
hubConnection.TraceLevel = TraceLevels.All; hubConnection.TraceWriter = Console.Out;
Armed with these two steps I was able to get an error that made a bit more sense although it was even more bizzare.
18:43:53.9073548 - null - ChangeState(Disconnected, Connecting)
18:43:55.4913444 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - WS Connecting to: ws://localhost:53705/signalr/connect?transport=webSockets&connectionToken=0dQ3scw2aJ7RbXuUsTfa4wlY%2FJlMzW%2FVL1sVb%2FyswewEO8n4qAth7Erpt0ga0laLV%2B8BCD833lZZy1MblOe0HuQs0O1mYi%2FfiiLSHc5%2F%2B1wk6tUsbFxMKVQFwRO9BvSW&connectionData=[{"Name":"formsHub"}] 18:43:55.5800651 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - WS: OnMessage({"C":"d-5AB67EA2-B,0|C,0|D,1|E,0","S":1,"M":[]})
18:43:55.5940751 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - OnError(System.MissingMethodException: Method not found: 'System.Collections.Generic.IEnumerator`1 Newtonsoft.Json.Linq.JArray.GetEnumerator()'. at Microsoft.AspNet.SignalR.Client.Transports.TransportHelper.ProcessResponse(IConnection connection, String response, Boolean& shouldReconnect, Boolean& disconnected, Action onInitialized) at Microsoft.AspNet.SignalR.Client.Transports.WebSocketTransport.OnMessage(String message) at Microsoft.AspNet.SignalR.WebSockets.WebSocketHandler.d__e.MoveNext())
18:43:55.6110864 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - WS: OnClose()
It says something like – I can’t find a method in the Newtonsoft.Json assembly. That’s totally weird as this kind of error should be handled at compile time.
I went looking into Newtonsoft.Json assembly referenced by SignalR 2.0. It correctly references version 5.0.8. The assembly in package folder has the method it was mentioned as missing. Mystery. One thing that caught my eye while .net reflecting the assembly was its version. While the file version is 5.0.8. the assembly version is, oddly, 4.5.0.0. How come?
Fourth step was looking into modules loaded by my application. They are listed through Visual Studio-Debug/Windows/Modules. Amazingly, Visual Studio was listing Newtonsoft.Json v4.05.8.15203 from … GAC. Not from my package. And by looking at its date it was clear that it was an older version (4.8.2012). Heck, I didn’t even know it was in GAC. I didn’t put it, some other app had to.
From previous Googling I remembered an explanation somewhere that the missing GetEnumerator method was introduced in version 5.0.5. So this was probably it – .net was loading an older version from GAC - that was really missing that method. At this point I knew what was wrong.
Just in case, to make sure, I checked whether the assembly really was in GAC:
C:\Windows\system32>gacutil /l newtonsoft.json
Microsoft (R) .NET Global Assembly Cache Utility. Version 4.0.30319.18020
Copyright (c) Microsoft Corporation. All rights reserved.
The Global Assembly Cache contains the following assemblies:
newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchi
tecture=MSIL
Number of items = 1
It was.
Remedy
The proper way would be to have a properly versioned Newtonsoft.Json assembly (those are strong key signed as well). That’s why there is versioning and .net wouldn’t make this confusion. I really wonder why James uses the same assembly version for assemblies that should really have different signatures. Hopefully he will fix this problem in next versions. But until then there is …
… the next best way is to force .net to not load it from GAC. If the assembly is removed from GAC, .net won’t find the wrong version anymore. See, .net always tries GAC first and only if it doesn’t find a suitable assembly it will go looking elsewhere, like into the folder where your application is. Perhaps I could configure it to load the proper assembly through .config file but this solution is better – it’ll solve the problem unless an older newtonsoft.json is reinstalled again in GAC. Hm.
Removing assembly from GAC should be a rather simple process. Just fire
GACUTIL /U Newtonsoft.Json
and it should be removed. But not this time. Instead I got this warning:
C:\Windows\system32>gacutil /u newtonsoft.json
Microsoft (R) .NET Global Assembly Cache Utility. Version 4.0.30319.18020
Copyright (c) Microsoft Corporation. All rights reserved.
Assembly: newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, proces
sorArchitecture=MSIL
Unable to uninstall: assembly is required by one or more applications
Pending references:
SCHEME: <WINDOWS_INSTALLER> ID: <MSI> DESCRIPTION : <Windows Installer>
Number of assemblies uninstalled = 0
Number of failures = 0
Basically it is saying that some installed application did install it into GAC and now it is preventing me from removing it from GAC. Uf, nasty. To make it worse, it doesn’t say which application. This article lists the way to “unreference it” to be able to remove it from GAC. Basically you have to remove its entry from the registry and then execute gacutil removal command again.
After registry modification it succeeded.
C:\Windows\system32>gacutil /u newtonsoft.json
Microsoft (R) .NET Global Assembly Cache Utility. Version 4.0.30319.18020
Copyright (c) Microsoft Corporation. All rights reserved.
Assembly: newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, proces
sorArchitecture=MSIL
Uninstalled: newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, pro
cessorArchitecture=MSIL
Number of assemblies uninstalled = 1
Number of failures = 0
The aftermath
Removing that old Newtonsoft.Json from registry did the trick. SignalR works like charm again. However, I had to manually remove a component of an unidentified application. And this removal might cause problems with that application in the future.
I really hope that Newtonsoft.Json gets proper versioning to avoid situations like this. This should be a lesson to every library developer out there – do use proper version numbering!.