C++ SDK overview
Quobis C++ SDK allows developers to integrate the voice and video capabilities of the Quobis Communication Platform into their C++ applications. Although the SDK was originally developed for usage in an embedded Linux environment based on an ARM SoC with resource constrains, it can be used in any ordinary ARM or x86_64 Linux system.
As a hardware reference, Quobis C++ SDK is able to place an audio and video call in a system running in single core ARMv7 with 46MB of RAM, and to do that using less than the 25% of the CPU and 16MB of RAM. Quobis C++ SDK also disables some of the features of the WebRTC default audio processing pipeline as echo cancellation, noise suppressions among any others, as they can be more costly than valuable specially for embedded devices in IOT and smart home applications.
Pre-encoded audio and video
Quobis C++ SDK was developed targeting audio and video systems with low resources, something in the range of Cortex A7 with a few hundred megabytes of RAM. Usually, systems like this handle video and audio relaying in some application specific hardware designed for the task, offloading that way, the CPU with some tasks that could put it to its limit when not completely overpassing what it can handle.
Although Quobis C++ SDK can do video and audio encoding/decoding using all the WebRTC supported codecs, it was designed for a system with a hardware H.264 video encoder with custom audio processing pipelines. The SDK is able to inject pre-encoded H.264 video streams in a WebRTC peer connection. When doing that, the application driving the SDK is the one in charge of providing the video information at the required rates.
In terms of audio, Opus is supported, but PCMU or PCMA are preferred as they just provide signal compassion, which is far less resource consuming than audio encoding. By default, audio processing technologies like, AGC, echo cancellation, noise management, filtering, etc are disabled, prioritizing resource consumption over audio quality.
Does it look like the Quobis Javascript SDK?
Some of the patterns that it uses are quite unique to the C++ SDK but it shares so many others with the Javascript SDK. Having this kind of similarity between the Javascript and the C++ SDKs is quite arguable. On one hand, at the moment of writing, the Javascript SDK has a lot more features than the C++ one, as it fully supports all the functionality that the Quobis Communication Platform can offer, as well as having support for different WebRTC providers. On the other hand, the C++ SDK has to deal with things like memory management and concurrency which are definitely not a problem in a browser. Some concepts present in the Javascript SDK are present on the C++ SDK, like user management or contacts, but just these ones that are strictly necessary to make a call.
How is it organized?
Sippo SDK is organized into multiple libraries, which are: libqlog, libqutils, libqss, libwac and libsippo. Depending on the library it could require some other ones as dependencies. A client application must link against all of them for correct functioning. The C++ SDK building and linking system it’s quite complex, make sure you have a complete understanding of it before making any changes. The library names are quite descriptive, they are in the format libxxx, with xxx being the part of the system that the library deals with, i.e:
libqlog: handles system logging
libqutils: has several utility functions
libqss: handles QSS communication
libwac: handles communication with the core services
libsippo: combines the functionality of every other library under a common entry point.
About ‘rootfs’
The rootfs is the filesystem mounted at root /. A rootfs together with your kernel is what composes your linux distribution of choice. Building embedded linux rootfs can be quite challenging, imagine it as building a linux distribution for your own usage and use case. Multiple solutions exist for making this process easier like Buildroot or Yocto. When building the SDK for a custom rootfs you must be sure that it is compatible with the libraries available in the rootfs.
This project contains a custom rootfs, built using Buildroot, which can be used for testing ARM builds. More information about this rootfs can be found in the docs and rootfs folders of the SDK.
Threading
Threads have been created and destroyed on demand when requiring asynchronous operations in the SDK. This approach to thread management dramatically increases the concurrency inside the client, increasing the synchronization required to manage these threads as well.
About libwebrtc
libwebrtc is an open source library that gives Sippo C++ it’s WebRTC powers. It is also the library that powers Google’s Chrome WebRTC capabilities. As Google’s Chrome is one of the most popular browsers having, at the moment of writing, more than half of the browser market share. By using libwebrtc we make sure that our WebRTC implementation is compatible with Chrome, and that indeed makes it compatible with almost every other WebRTC implementation.
Testing
The C++ SDK has a test stack that can be helpful by making a faster development using techniques like TDD while improving and assuring the quality of the system. These types of tests do exist in the SDK: e2e tests (to test the system all together); integration tests (to test a part of the system which usually is slow or requiring to test, like the websocket connection); and unit tests (which test parts of the system in isolation and because of that are the fastest). Depending on the library, it can have from none automated tests to tests of all these types.
One test that is quite interesting is the libsippo end-to-end test named CallTest, as it can serve as an example of how a call can be established using the C++ SDK. The application can be built without tests by passing the -DWITH_TEST=OFF option to CMake.
Known limitations
In the current version, the Quobis C++ SDK is able to place audio and video calls. Other functionalities of the Quobis Communication Platform, like messaging, file transfer, meetings, etc… are not implemented.
Please note that the SDK is not able to handle any incoming video stream, any received video stream in the peer connection will be just ignored.
Issues with audio or video quality
You might review the audio and video pipelines. Maybe you are poorly processing the audio or the video, or just not getting the frames at time. If you are not feeding audio or video directly to the SDK take a look at what video and audio processing tools are you providing when creating the PeerConnectionFactory.
More information