WebSockets: A Deeper Dive

LiveSwitch has just released its new feature Media-over-WebSockets. This feature helps to continue LiveSwitch’s mission of making it easier to power human connections. With this new feature, video in a single session can be streamed using either the WebRTC or WebSockets or a hybrid-based model.

For most LiveSwitch customers, no change is required. Our traditional WebRTC-based communications using SFU and MCU connections still exist, and we have not changed the flow of media for these to ensure the same great quality that our customers have come to expect from LiveSwitch.

For those customers or potential customers interested in this feature, it is important to understand how to use WebSockets, why you would want to use them, and if using a pure Websocket or hybrid model is the best approach.

Why use WebSockets?

In the feature announcement article, we touched on why a customer may be interested in WebSockets. The primary reason for choosing to use Websockets is for network security. With WebRTC, media traditionally flows using UDP. This is different from most internet traffic which is transmitted using TCP. 

In order to support WebRTC traffic, customers need to ensure that their network firewalls allow UDP traffic over a wide range of ports which creates more vulnerabilities in their network. For companies that have protected data or ultra-secure network standards – government facilities come to mind – their goal is to limit all traffic to TCP over a very select number of ports.  This can be accomplished using WebRTC however some companies still require blocking WebRTC traffic over TCP.

This concept of limiting potential access points can be traced back to a time-tested strategy that even the Spartans used in the battle of Thermopylae. If you limit your enemy's attack point to one or two paths, you are able to focus all your defensive effort on that limited attack surface area.

By changing how media streams from UDP over multiple ports to TCP over a single port, we allow our customers to mount a strong defense and ensure their networks stay secure.

How is Media-over-WebSockets supported?

LiveSwitch is able to do this thanks to its new WebSocket-based Media Servers. These servers sit side-by-side with our traditional media servers and are assigned by a gateway when an application opts in to stream media over WebSockets. As connections are made to a channel, the LiveSwitch platform tracks which media servers are being connected to. These new WebSocket based media servers are immediately connected to our traditional media servers when they are spun up.  This connection between the servers allows them to share the media being ingested by each servertype. This media sharing inside our own network allows us to ingest media over WebSockets and either transmit it back out via WebSockets or via webRTC using the appropriate servers.

Thanks to this information sharing, we can enable a hybrid connection model for our customers.  This allows customers to take advantage of the years of work that have gone into making our SFU connections over WebRTC reliable and adaptable while being able to take advantage of the security benefits of WebSockets.

How do I enable this in my application?

If you are interested in adding this to your application, you just need to update the code that creates the upstream and downstream connections.

You can see an example of enable this here:

var upstreamConnection = channel.createSfuUpstreamConnection(audioStream, videoStream, true);

var downstreamConnection = channel.createSfuDownstreamConnection(audioStream, videoStream, true);

Please note, this feature is currently only supported for Javascript and .NET based applications.

There are some additional steps required to enable this for .NET applications. Based on how the native libraries have implemented WebSockets, there is not a need to have the audio and video data packetized when sending and depacketized when receiving.

Based on this difference, we need to adjust our Local and Remote Media classes to change how we create our AudioPipe and VideoPipe.

For Local Media, you should have the following code:

protected virtual AudioPipe CreateOpusPacketizer(AudioConfig config)
{

        if (PacketizerDisabled)
    {

                return new IdentityAudioPipe(new   AudioFormat(AudioFormat.OpusName, config) { IsPacketized =   false });

    }

        else
    {

                return new Opus.Packetizer(config);

    }

}



protected virtual VideoPipe CreateVp8Packetizer()
{

        if (PacketizerDisabled)
    {

                return new IdentityVideoPipe(new   VideoFormat(VideoFormat.Vp8Name) { IsPacketized = false });

    }

        else
    { 
                return new Vp8.Packetizer();

    }

}

For Remote Media it should be:

protected virtual VideoPipe CreateVp8Depacketizer()
{

        if (DepacketizerDisabled)
    {

                return new IdentityVideoPipe(new VideoFormat(VideoFormat.Vp8Name));

    }

        else
    {

                return new Vp8.Depacketizer();

    }

}



protected virtual AudioPipe CreateOpusDepacketizer(AudioConfig config)
{

        if (DepacketizerDisabled)
    {

                return new IdentityAudioPipe(new AudioFormat(AudioFormat.OpusName, config));

    }

        else
    {

                return new Opus.Depacketizer(config);

    }

}

The last changes you will need to make for .NET is to the audio and video track for both local and remote media objects. You can see examples of both below:

audioTrack.Next(new IdentityAudioPipe(Opus.Format));
videoTrack.Next(new IdentityVideoPipe(Vp8.Format));

Everything else is handled by the LiveSwitch platform.  If you have any questions or would like to see how to handle more advanced use cases, you can find additional information in our documentation.

If you are just getting started with LiveSwitch, you can sign up for a free trial and test Media-over-WebSockets using our latest open source sample application.