Delayed audio throughput on SIP softphone lines

Observation

In the case of telephone calls, the audio feed-through is significantly delayed after a connection is established.
Despite successful connection establishment, the bidirectional voice can only be heard a few seconds later: The first 2–3 seconds of the call are "missing", otherwise the connection is available in undisturbed quality.

Where does this behavior occur?

Phone calls on a SIP softphone line in the ProCall Enterprise client for Microsoft Windows or in the ProCall mobile app for iOS and Android may be affected by this issue.

Which calls are affected?

Incoming calls on the ProCall Enterprise client for Microsoft Windows are more frequently affected by this problem, while outgoing calls are less affected.

In the ProCall Mobile app, this behavior can occur for both incoming and outgoing connections.

Possible reasons

The media connection (encrypted RTP audio stream G.711 aLaw-, G.711 µLaw- or Opus-encoded) between ProCall Enterprise client for Microsoft Windows and estos UC Media Server or between ProCall Mobile app for iOS/Android and estos UC Media Server is negotiated using the Interactive Connectivity Establishment (ICE) procedure according to RFC 8445. In this process, client and server exchange candidates (host, srflx, relay, on IPv4 and IPv6, UDP, and TCP).
Candidates represent the possible connection endpoints for possible audio or video connection, which are exchanged between the communication participants and checked as possible suitable connection paths.

Number of candidates

Depending on the operating environment, there can be a different number of candidates.

For example, if an estos UC Media Server has only one network interface and only the IPv4 protocol is bound to it, but not also the IPv6 protocol, the UC Media Server will only communicate an estimated 5 candidates to the client as possible connection endpoints (IPv4 UDP host, IPv4 UDP srflx, IPv4 TCP host active and passive, IPv4 UDP relay).

If, for example, the ProCall Mobile app is in operation in a typical home office environment, it will probably inform the UC Media Server of more than 20 possible connection endpoints: host address in the WLAN, host address in the mobile data network, srflx address at the router of the home network, srflx address in the mobile data network, relay address in the TURN server. These addresses, each taken into account under IPv4 and IPv6, UDP and TCP, result in a calculated 20 candidates without taking details into account.

During the connection negotiation via ICE, the mutually communicated candidates are suitably paired to check the endpoint connections from both sides. 
Without considering further details, 10 times 20 connection paths can be formed with 10 UC Media Server Candidates and 20 ProCall Mobile App Candidates, which must be checked within the shortest possible time, but a sufficiently fast connection path determination cannot be guaranteed. It could take several seconds until a suitable connection path is found by both the ProCall Mobile App and the UC Media Server and agreed upon as a transmission channel. During these seconds of connection negotiation, the audio transmission is then lost and the user perceives this as a delayed audio connection.

Compared to the previous example, in which IPv4 and IPv6 candidates are evaluated and there could be a rough estimate of 200 connection paths to be tested, the deactivation of IPv6 candidates results in a rough estimate of only 50 connection paths to be tested, which corresponds to a reduction of connection tests by 75%.

Procedure

Reduction of the number of connecting paths to be tested

Reducing the number of connection paths to be tested can therefore be a suitable means of speeding up audio throughput.

If required, the client candidates fed to the estos UC Media Server via the estos UC Web Server can be filtered. In this way, the UC Media Server receives fewer client candidates, forms fewer candidate pairs for the connection check, and thus nominates the successfully tested connection more quickly.

Warranty disclaimer

It is explicitly pointed out that the reduction of ICE Candidates by means of the filtering described below and the associated reduction of possible connection paths can lead to a situation where no connection at all can be established between estos UC media server and ProCall Enterprise clients or ProCall mobile apps.

In particular, the use of ProCall mobile apps in the mobile data network may depend on the use of IPv6 connections. Likewise, a WLAN hotspot or internet access in hotels, for example, can completely block the exclusive use of UDP, making connections to the ProCall Enterprise server infrastructure impossible.

Therefore, estos cannot provide any functional warranty for the configuration and filtering measures described below. If these measures are considered, estos strongly recommends a complete functionality test in all required operating scenarios in interaction with the individual customer infrastructure.

Filter technology for ICE Candidates

Depending on the operating environment, it makes sense to exclude the IPv6 or TCP candidates from the ICE connection check. For example, if IPv4 communication is ensured in all connection scenarios (office, mobile data network, home office, VPN, ...), the IPv6 connection check can be omitted. TCP candidates can also be dispensed with if the audio stream can be transported via regular and preferred UDP without any problems.

In estos ProCall Enterprise, the client communicates the candidates to the estos UC Media Server via the estos UC Web Server. In the estos ProCall Enterprise Server, the connection check is controlled in such a way that only the desired type of client candidates are sent to the estos UC Media Server from the UC Web Server. Candidates reported by the client that should not be checked (for example, IPv6 and TCP candidates) are filtered out by the estos UC Web Server and the UC Media Server will therefore not receive them. If the estos UC Web Server blocks the forwarding of IPv6 client candidates, the UC Media Server will not receive any IPv6 candidates and will therefore not create any candidate pairs for IPv6, even though it may have IPv6 candidates itself.

This filtering technique for ICE Candidates is thus independent of the network infrastructure, so it is not necessary to disable network adapters on the client or server or to remove the IPv6 protocol from the adapter.

Filter parameters

The desired filter parameters are entered in the kurentoservers section of the configuration file eucwebconfig.json (default C:\Program Files\estos\UCServer\config\eucwebconfig.json) of the estos UC Web Server:

The default value for all three parameters is 0 (even if there is no parameter entry), this filtering is thus inactive.

Filtering of IPv6 and TCP candidates shows up in the eucwebconfig.json as follows:

Example screenshot: eucwebconfig.json

Please note the exact syntax regarding comma and bracket placement, as well as upper and lower case letters.

In the example shown here, the UC Media Server and the UCServer or UC Web Server are operated on the same host (WebSocket address 127.0.0.1).
For remote UC Media Servers, please note to keep the corresponding different URL.

The section here for the sake of completeness again as code block:

   "kurentoservers" : [
      {
         "url" : "ws://127.0.0.1:8888/kurento",
         "candkillipv6": 1,
         "candkilltcp": 1,
         "candonlyturn": 0
      }
   ],

Changes in the estos UC Web Server configuration file eucwebconfig.json only become active after restarting the estos UC Web Server service. A change of the filter parameters, therefore, requires a restart of this service to activate the desired filters.

Restarting the estos UC Web Server service will disconnect ongoing calls via SIP softphone lines.
This process should be performed outside of regular productive operations due to this brief interruption in operation.