VMWare Horizon Client Remote Desktop Issue

Same issue here and I have contacted T-Mobile they said that they won't do anything to fix it. My IT department has spent many hours trying to fix this and it is t-mobiles issue.  Update T-Mobile made a firmware update that caused this problem and has now has accepted a ticket to address this issue. 

Yes, I have been experiencing issues connecting to Horizon for the last few weeks. Usually I am completely unable to connect, it just spins at authenticating endlessly, then throws a tunnel error. If I am able to connect I experience disconnects randomly.

I do not have this happen when connecting via my secondary CenturyLink internet connection.

Additionally, there are two of us here who work for different companies, we both use Horizon, and both of us are experiencing the issue I described above.

Graber wrote:

Update T-Mobile made a firmware update that caused this problem and has now has accepted a ticket to address this issue. 

Any chance you could provide the ticket number? I'd be happy to add my name to the list of people who need this fixed. After an hour and a half on the phone today with their tech support I don't feel like they really understood the issue. Having a ticket to reference could be useful.

dbrush wrote:

Graber wrote:

Update T-Mobile made a firmware update that caused this problem and has now has accepted a ticket to address this issue. 

 

Any chance you could provide the ticket number? I'd be happy to add my name to the list of people who need this fixed. After an hour and a half on the phone today with their tech support I don't feel like they really understood the issue. Having a ticket to reference could be useful.

I did not revive a ticket number but I was told I would receive a call back on Tuesday of next week with any solutions the find. Will update then. There tech support is very hit or miss the first time I was on the phone with them I got no where.

Graber wrote:

dbrush wrote:

Graber wrote:

Update T-Mobile made a firmware update that caused this problem and has now has accepted a ticket to address this issue. 

 

Any chance you could provide the ticket number? I'd be happy to add my name to the list of people who need this fixed. After an hour and a half on the phone today with their tech support I don't feel like they really understood the issue. Having a ticket to reference could be useful.

I did not revive a ticket number but I was told I would receive a call back on Tuesday of next week with any solutions the find. Will update then. There tech support is very hit or miss the first time I was on the phone with them I got no where.

Alright, thank you. I’m looking forward to hearing what they have to say.

The VMware client probably is using PCoIP via an HTML5 session and it uses UDP ports with AES encryption it seems so my guess is there is port blocking via the 426XLAT solution T-Mobile uses. The session might go out but they might be blocking the return. That might be a problem. If you have a VPN that you use for your personal protection you might try using the VPN tunnel to get through the 426XLAT environment as a work around. If you have a VPN solution that provides no blocking for the protocol(s) or ports involved it should then work. I know it is not optimal but the 426XLAT environment T-Mobile has does present some port forwarding limitations.

Thanks for the insight iTinkeralot, but neither of us connect over PCoIP, we instead use Blast. Could the same thing be happening on a protocol other than PCoIP?

I’m not a networking professional, and I don’t know what a 426XLAT solution is (and neither does Google by the looks of it) but if what you are suggesting is the case, would the Horizon connections work intermittently as we are seeing now, or would they have ever worked at all? it is the inconsistency in our ability to recreate the issue that is bugging me about this whole thing.

I went and found the logs for the Horizon client at C:\Users\UserName\AppData\Local\VMware\VDM\logs and this is what it shows at the time of failure. From what I can see it is identical to a successful connection attempt right up to the 404 error. Perhaps this makes more sense to you:

2022-12-02T12:10:21.266-06:00 INFO (01) [libcdk] : TaskCombiner: ParseResult for CdkGetTunnelConnectionTask(PEND).
2022-12-02T12:10:21.266-06:00 INFO (01) [libcdk] : CdkTunnelClient_SetFingerprint: Tunnel Server expected fingerprint is [redacted]
2022-12-02T12:10:21.266-06:00 INFO (01) [libcdk] : CdkProxy_GetProxyForAutoSettings: Call WinHttpGetProxyForUrl for https://server.domain.com:443/ice/tunnel?22E368E6_EC77_4E2D_8717_EC59583B2DCF.
2022-12-02T12:10:21.268-06:00 INFO (01) [libcdk] : CdkProxy_GetProxyForAutoSettings: Failed to get WinHTTP proxy info with error 0x00002f94.
2022-12-02T12:10:21.268-06:00 INFO (01) [libcdk] : CdkTunnelClient_Connect: Connecting to tunnel server 'server.domain.com:443' over HTTPS.
2022-12-02T12:10:21.268-06:00 INFO (01) [libcdk] : TaskCombiner: CdkGetTunnelConnectionTask(DONE) removed, group task num:2, total task num:2.
2022-12-02T12:10:21.268-06:00 INFO (01) [libcdk] : TaskCombiner: SetResult for CdkGetTunnelConnectionTask(DONE).
2022-12-02T12:10:21.323-06:00 INFO (01) [libcdk] : CdkTunnelClient_SocketConnectCb: The tunnel connection is running in non-FIPS mode.
2022-12-02T12:10:21.523-06:00 EROR (01) [libcdk] : CdkTunnelClient_HandleHttpError: Received Http error: 
HTTP/1.1 404 Not Found
Date: Fri, 02 Dec 2022 18:09:54 GMT
Content-Length: 1934
Content-Type: text/html
Strict-Transport-Security: max-age=31536000
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Security-Policy: default-src 'self';font-src 'self' data:;script-src 'self' 'unsafe-inline' 'unsafe-eval' data:;style-src 'self' 'unsafe-inline';img-src 'self' blob: data:
X-Frame-Options: SAMEORIGIN
Connection: close
2022-12-02T12:10:21.581-06:00 EROR (01) [libcdk] : CdkTunnelClient_HandleHttpError: Received Http error: 
HTTP/1.1 404 Not Found
Date: Fri, 02 Dec 2022 18:09:54 GMT
Content-Length: 1934
Content-Type: text/html
Strict-Transport-Security: max-age=31536000
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Security-Policy: default-src 'self';font-src 'self' data:;script-src 'self' 'unsafe-inline' 'unsafe-eval' data:;style-src 'self' 'unsafe-inline';img-src 'self' blob: data:
X-Frame-Options: SAMEORIGIN
Connection: close

I found a reference to error 0x00002f94 in a VMware link. Old bug related to a proxy fix but should be fixed in the 5.4 client back in 2020. We can see it does use port 443 for a secure HTTP connection. It connects to the tunnel server and then has an HTTP error. Maybe searching on VMNware community to see if there are more matches? I am not sure what browser is best/recommended for use with the client. In effect they should be pretty much the same with respect to behavior due to the standards. It has been a bit since I went fishing in the VMware community but there might be something there. 

The 464XLAT is the IPv4/IPv6 construct that T-Mobile appears to use. Some refer to it as CGNAT which is referencing for carrier grade NAT, network address translation. The behavior may have something to do with a change of how T-Mobile is doing the traffic handling over IPv6. You can see clearly in your client both IPv4 and IPv6 are active. i.e. the dual stack. In packet captures I have seen they tend to use IPv6 for types of multicast delivery. The whole IPv4 services over IPv6 is a bit complicated and does have limitations so port forwarding as with a more traditional IPv4 routing process is not possible. 

The way the VMware client sets up the "tunnel" communication to the server seems like it should work. It might be good to check with IT and see if they have done upgrades to the VMware environment in the office. Maybe the client needs to be updated. However, you stated that in the office it works so that sort of seems to rule that out. In another thread in the VMware community there is a reference to a change in the F5 configuration that may have been involved at one point. The tunnel seems to be established and then closed. Maybe something again related to the F5s and load balancing the traffic. I would try a different browser or two and see if the same behavior persists. Check the VMware community as well. I don't have an account in there now. I know this could be a rabbit hole so I might better grab a root and stop the fall. 😎

One thing you might look at is the external IP. Use the site whatismyipaddress.com before setting up and then if the tunnel forms and holds but then fails check the IP address again. Maybe it is related to the change in the IP address from an external perspective impacting the routing and the session? I don't know just speculating. You state when in the office it works but over the T-Mobile links it fails. Well the external IPv4 addresses do change from time to time so it might be something to look at. Again VMware would probably be the best resource for pulling this one apart.

Graber wrote:

dbrush wrote:

Graber wrote:

Update T-Mobile made a firmware update that caused this problem and has now has accepted a ticket to address this issue. 

 

Any chance you could provide the ticket number? I'd be happy to add my name to the list of people who need this fixed. After an hour and a half on the phone today with their tech support I don't feel like they really understood the issue. Having a ticket to reference could be useful.

I did not revive a ticket number but I was told I would receive a call back on Tuesday of next week with any solutions the find. Will update then. There tech support is very hit or miss the first time I was on the phone with them I got no where.

Graber did you hear anything back from T-Mobile on this? Curious as to what they had to say about it.