NVIDIA: RTX Broadcast Engine, an AI-based solution to suppress streamers chroma

[[Nike]]

Streaming is a growing trend in the entertainment industry. More and more people are betting on creating live content, whether playing video games or informing and / or entertaining. To help the creators, the RTX Broadcast Engine technology has been introduced by NVIDIA. This technology has been specially developed for the GeForce RTX Series that have Tensor Cores.

RTX Broadcast Engine is a solution with which we can modify the background without the need for a chroma. Nor is additional software needed to eliminate the fund and put, for example, the game on which our transmission is based. This is achieved thanks to the dedicated RTX hardware for AI. NVIDIA manages to create a digital chroma through AI that is complemented by Augmented Reality and special filters.

NVIDIA announces RTX Broadcast Engine technology

The new function that will allow us is without having a chroma, to be able to customize the background. Simply explained, it makes a scan of our body and so we can modify the background to taste. A series of filters will be added that can be changed during transmission to make it more dynamic.

Three functions are added:

RTX Greenscreen: It allows us to change the background captured by the webcam in real time. This shows only the face and body of the person making a live broadcast. The Artificial Intelligence generated by this technique is able to recognize which parts of the scene are personal and which are the background. This allows us to have the advantages of a chroma without having it.

RTX AR: Allows face detection, mapping facial features such as eyes and mouth and even modeling the surface of the face. So we can add augmented reality effects in real time with any webcam. Any developer can use it for fun to create augmented reality effects, either by superimposing 3D content on the face or controlling 3D characters with facial movements.

RTX Style Filters: These filters make use of an AI called “style transfer”. They allow the transformation of the image captured by the webcam into another image. With just one key we can change the style of the video based on pictures or designs.