Problem
After migrating from Newtonsoft.Json to System.Text.Json (branch stj-migration, commit 9782a98ea), the direct CLI path works:
dotnet vstest.console.dll test.dll → Passed: 1, Failed: 1, Skipped: 1
But the wrapper/DesignMode path hangs:
- Translation layer wrapper → socket → vstest.console → socket → testhost
- 4 vstest.console + 4 testhost processes sit at near-zero CPU indefinitely
- No error messages, no exceptions — both sides just wait for a message that never comes
- Smoke tests and integration tests all hang in this path
What was fixed so far
TestRunStatistics.Stats private setter → custom converter (was causing NullReferenceException)
IDictionary<string, object> values deserialized as JsonElement → ObjectConverter unwraps to primitives (was causing InvalidCastException)
Root cause hypothesis
The DesignMode communication loop (DesignModeClient.ProcessRequests) receives messages via SocketCommunicationManager.ReceiveMessage() and processes them. If a message fails to deserialize silently (swallowed exception), the loop may continue waiting for the next message while the sender waits for a response — causing a deadlock.
The message loop should abort if it cannot process data, rather than silently continue.
Investigation needed
- Add
--diag to wrapper-based test execution to capture the full protocol exchange
- Compare the messages sent/received between wrapper and vstest.console vs what the direct CLI path sends
- Check if any try/catch in the message processing loop swallows serialization exceptions
- Consider adding a message-level error response when deserialization fails, so the other side knows and can abort
Branch
stj-migration on nohwnd/vstest — commit 9782a98ea
Problem
After migrating from Newtonsoft.Json to System.Text.Json (branch
stj-migration, commit9782a98ea), the direct CLI path works:But the wrapper/DesignMode path hangs:
What was fixed so far
TestRunStatistics.Statsprivate setter → custom converter (was causing NullReferenceException)IDictionary<string, object>values deserialized asJsonElement→ObjectConverterunwraps to primitives (was causing InvalidCastException)Root cause hypothesis
The DesignMode communication loop (
DesignModeClient.ProcessRequests) receives messages viaSocketCommunicationManager.ReceiveMessage()and processes them. If a message fails to deserialize silently (swallowed exception), the loop may continue waiting for the next message while the sender waits for a response — causing a deadlock.The message loop should abort if it cannot process data, rather than silently continue.
Investigation needed
--diagto wrapper-based test execution to capture the full protocol exchangeBranch
stj-migrationonnohwnd/vstest— commit9782a98ea