Fix for Disconnects and Some Crashes

[Riley_D]

Hey everyone,

I wanted to pass along a new option we’ve added to the game. Right now it’s only available through the Steam client (for all headsets) but not the Oculus Store version, but we’re working on getting it added there as well.

Essentially it allows you to connect to the server using TCP Websockets instead of UDP plain sockets. For some people this has fixed disconnect issues and also as a result fixed crashes. It’s most likely to help if you are seeing frequent (e.g. every hour or two) disconnects or crashes. Note that at this time I still think the default way of connecting to the server is best if you’re not having issues.

Anyway, to try it out, just select the “Websocket (Alternate)” connection option from the login screen on the left-hand side from where you choose your Character.

If all goes well you should connect as normal and you shouldn’t see any difference in gameplay, but hopefully it might fix those disconnect or crash issues.

Thanks!

[Sttoh]

Going to give this a try and report back in a few hours. I crashed 3 times and felt more random loads than normal this morning in about 4 hours of gameplay, thank you for the quick response.

[Czech_E]

Wait, it sounds like you’re saying that you’re using UDP but not attempting to reconnect to the server while in-game should the client expect packets that are no longer being received or if the connection has terminated unexpectedly. Is that right? The perk of UDP being stateless is that you are given the option to re-request data that was expected but never received, at your own discretion: you’re code should be able to ignore failed packet transfers depending on what they were and re-request those that it requires and expects. TCP will help mitigate such an issue since you will “know” (more-or-less) if a connection was terminated, but you should be handling the case in your UDP-based consumer loop that you’re no longer receiving packets when you expect them, whether it’s from a timeout being hit, the server falling over, or something else that’s unexpected. Sorry, I just want to understand why you’re having problems utilizing UDP - it’s probably best not to use TCP if it can be avoided because of the overhead involved (e.g. the handshake, trailing window, all that jazz…) and the lag it can introduce under certain cirsumstances. Take Minecraft as an example, that game could have been much more performant (network-wise) if it was using UDP from the outset.

I just don’t want you to rely on a TCP websocket connection when a raw UDP socket connection is much better suited for the job. You can quickly dig yourself into a slew of network-based issues for a game whereby you expect to have hundreds of people in a single zone. I just don’t think that TCP is going to scale well if you plan to have tens/hundreds of thousands of concurrent connections.

[Riley_D]

I’m not really sure where you’re getting any of that out of the simple comment that we are offering an alternate connection option.

We are using a fully-featured 3rd party UDP library with multiple channels, options for sending packets such that they arrive in-order reliably, or unreliably, or out of order (depending on the type of data we are sending such as frequent position updates, voice chat packets, or “must-arrive” information). It handles reconnects and all of that as well. UDP is better to use for this game due to the frequent sending of data that becomes quickly stale, such as the position of someone’s hands, for example. That’s why it’s the default.

But some people don’t seem to do well with UDP, whether it’s an issue with their network, or a firewall between them and us, or just random bugs in the system, I have no idea. So we are offering a secondary option over TCP Websockets to help with that. Yes, it has the downside that if there is packet loss you will end up having a lag spike as all the packets catch up, but since most network connections don’t have large packet loss these days I think the trade-off for them of not getting disconnected randomly is probably worth it.

In addition to that, using Websockets allows us to take advantage of Cloudflare routing of the traffic, which may take a better path for them than using a standard “connect directly to us” approach.

Honestly, this is way outside the scope of this thread. If you have some sort of commentary about our networking stack you want to share, feel free to email it to me.

[Czech_E]

Alright, it sounds like you have things covered and I jumped the gun; I don’t have the source code and have yet to attempt any analysis with wireshark or the like, so my insight is fairly limited in this aspect. Sorry to bother you guys with technical mumbo jumbo…

[Sttoh]

As a note to all this, I’m actually doing much better since the change. No crashes and very minimal “white room” load screens or very short load screens. I can get out of my house much faster now too.

[Brian_B2]

So far it’s working much better for me since switching over.

[Riley_D]

Great glad to hear it!

[Daeresha]

Same here, so far this is working great
Yesterday, played most of the evening, about 5 hrs, 4 crashes with error
Today, same time played, no Orbus related crashes! (One disco caused by home Inet, doesn’t count )

[Daeresha]

Quick update, been playing almost daily since this fix a week ago, happy to say I rarely have Orbus crash now. Thanks so much @Riley_D

I did find my computers inet connection would turn off since this fix, but no disconnect on other devices in my home, turned off Win10 option to put the network card to sleep and seems ok so far. Dunno if the fix and sleep mode are related, just wanted to mention it in case it’s happening to anyone else.

[Riley_D]

If anyone out there who was having these crashes that the new option fixed gets a chance, I could use some help testing a potential fix.

Just right-click on your “OrbusVR” in your Steam game listing, hit “Properties…” and then choose the “Betas” tab. Select the “udpfix2” option from the dropdown. Let it update. Also press the “Set Launch Options…” button and delete the “-connectws” string (so the box is blank) to turn off the fix.

Then run with that for a bit and see if it helps with the crashes you were having before at all. It might not but I’m just testing a theory.

If it doesn’t help, just go back to the “Betas” tab and choose “Opt out of all betas”, and then follow the directions in the first post to re-enable the “-connectws” option to go back to how it was before and let me know it didn’t work for you.

Thanks!

[Damander]

I will try this tomorrow .

[Daeresha]

Likely not tomorrow as I don’t want to experiment during a fellowship event(people’s lives on the line here ), but will definitely try this and let you know how it goes after Tuesday

[quiksilver]

I can try today after work, but my crashing only really happened for one night, so I’m not sure it will be a conclusive test; though I do get more lag than i used to now.

[Riley_D]

Okay so actually I rolled this code into the main branch today, so you don’t have to do any of the “Betas” tab thing, just remove the “-connectws” option and see if it crashes less. Thanks!