This is the first article in a series that take a closer look at the underlying technologies that power the world of live streaming and content creation It may seem like live streaming has been done the exact same way for the past many years. However, under the hood, new hardware capabilities, encoding technologies, and technical innovations have been constantly shifting the underlying landscape. When I joined XSplit back in 2012, live streaming was just starting to gain traction. Since then, I've had a front-row seat to the incredible technological advancements that have transformed and are continuing to transform the industry. One of the most significant shifts we've seen is the move from CPU-based encoding to hardware encoding on the GPU. This change wasn't just a minor upgrade; it was a revolution as it made streaming accessible to the masses by no longer requiring a prohibitively expensive streaming rig with a beefy CPU to produce a high-quality HD live stream. Big players like Intel, Nvidia, and AMD all rolled out their own dedicated encoders, recognizing the profound impact live content creation was beginning to have on our digital world. A new piece of chip architecture is now changing the landscape yet again, with the advent of NPUs (Neural Processing Units). These chips are game changers for AI-driven tasks, which can now run more smoothly without bogging down your CPU or GPU. Intel is a driving force for this shift, as they are integrating NPUs into all new laptop CPUs and most desktop CPUs to bring their AI PC vision to life. For content creators, this will make it even more accessible in the future to take advantage of AI-based enhancements to the creator experience, such as AI-driven background removal, audio enhancements, gaming highlights recognition, subtitles, automated reactions and much more, without negatively impacting the gaming or the streaming quality. At XSplit, we've leveraged the power of NPUs in exciting ways. We teamed up with Intel to enhance our VCam software, which uses our own patented AI technology for webcam background removal. When running on a machine with an NPU, e.g. any machine with the new Intel Core Ultra processor, VCam now offers a more powerful model that delivers noticeably better results—particularly with tricky details like chair edges, headsets or when someone else suddenly moves in the background.<div style='display: flex; flex-flow:row; gap: 10px;'><img alt='VCam Camera Settings' style='width:45%;' src='/blog/live-streaming-technology-past-present-and-future/static/image4.png'/><img alt='VCam Performance Settings' style='width:45%;' src='/blog/live-streaming-technology-past-present-and-future/static/image1.png'/></div> The performance boost is clear: when measured against a perfect removal of all background elements, our previous best AI model has an average inaccuracy rate of 2.5 percent, but with the new NPU-optimized model, that drops to just 1.5 percent - a 40% reduction of inaccuracies. While the inaccuracy rate may sound negligible in either case, this improvement makes a real difference in live streaming, where clarity and visual quality can make or break viewer engagement. Examples:<div style='margin:0 auto;width:80%;display:flex; flex-flow:column;justify-content:center;align-items:center;gap:30px;'><div style='display:flex;flex-flow:column;gap:10px;width:100%;justify-content:center;align-items:center;'><div style='display: flex; flex-flow:row; justify-content:space-evenly; gap: 10px;width:80%; margin: 0 auto;'>New Highest (NPU-optimized) modelPrevious Highest Model</div><img alt='Sample image 1' src='/blog/live-streaming-technology-past-present-and-future/static/image3.png' style='width:100%;'/></div><div style='display:flex;flex-flow:column;gap:10px;width:100%;justify-content:center;align-items:center;'><img alt='Sample image 2' src='/blog/live-streaming-technology-past-present-and-future/static/image2.png' style='width:100%;'/>Higher level of precision when segmenting edge elements like chair and headphones</div><div style='display:flex;flex-flow:column;gap:10px;width:100%;justify-content:center;align-items:center;'><img alt='Sample image 3' src='/blog/live-streaming-technology-past-present-and-future/static/image5.png'/style='width:100%;'>Significantly reduced risk of model confusion when a person moves in the background</div></div> Here's how it plays out in real life: We ran a series of live tests on an ASUS Zenbook with an Intel Core Ultra 7 processor with NPU and an Intel Arc graphics card. We benchmarked the impact on FPS when playing popular games like Cyberpunk 2077, Sims 4 and Fortnite while encoding the gameplay in 720p in XSplit Broadcaster, at the same time as running the highest-tier background removal models on both the CPU and NPU. The results were impressive. The games ran smoothly, with only a minimal impact on frame rates when the NPU took on the heavy lifting for background processing. In these scenarios, when running the previous best background removal model on the NPU, we saw an average of ~13% higher FPS in the games compared to running the model on the CPU. Conversely, when running the new higher NPU-optimized performance model on the CPU, it would bring the game to a constant stutter - while running the new model on the NPU would give a smooth experience with 7.5% higher average FPS than running the previous best model on the CPU.<table style='width:80%;margin:10px auto;'><thead style='background-color:#f1f1f1;font-weight:600;text-align:center;'><tr><td>Background removal model</td><td>Processor</td><td>Average FPS</td></tr></thead><tbody><tr><td>Previous Best</td><td>CPU</td><td>95.2</td></tr><tr><td>Previous Best</td><td>NPU</td><td>107.5</td></tr><tr><td>New Best (NPU-optimized)</td><td>CPU</td><td>N/A (constant stutter)</td></tr><tr><td>New Best (NPU-optimized)</td><td>NPU</td><td>102.3</td></tr></tbody></table> We also saw significant power savings and battery life improvements to be gained when running any of the background removal AI models on the NPU. VCam uses Intel's <a href='https://docs.openvino.ai/2024/home.html'>OpenVINO</a> API for inference on CPU and NPU. By default, inference in VCam uses FP16 precision on NPU and GPU, and Int8 quantization on CPU. In a future update, VCam may switch to use Int8 also on NPU for even lower power usage. VCam optimizes inference cycles when the camera has little or no movement, but by using our in-house model testing tool we were able to measure worst case power consumption used by the background removal model by forcing inference on every frame and testing different models with different precisions.<table style='width:80%;margin:10px auto;'><thead style='background-color:#f1f1f1;font-weight:600;text-align:center;'><tr><td>Previous Best Model</td><td>*Package Power (in Watts)</td><td>New Best Model (NPU-optimized)</td><td>*Package Power (in Watts)</td></tr></thead><tbody><tr><td>CPU (Int8)</td><td>22.90</td><td>CPU (Int8)</td><td>28.19</td></tr><tr><td>GPU (FP16)</td><td>16.23</td><td>GPU (FP16)</td><td>20.72</td></tr><tr><td>NPU (FP16)</td><td>13.05</td><td>NPU (FP16)</td><td>14.21</td></tr><tr><td>NPU (Int8)</td><td>11.24</td><td>NPU (Int8)</td><td>12.23</td></tr></tbody></table>*Tests were run using an in-house model testing tool (smdnnTestBench), without running any apps in the background. Power consumption was measured using HWiNFO In conclusion, using the NPU for background inference reduced the battery consumption by up to 60% compared to inferencing on the CPU/GPU, and running the new higher model on the NPU only increases power consumption by ~8% compared to running the previous best model on the NPU. Overall, this means both longer battery life and lower energy bills—a win-win for streamers and the planet. Looking forward, the potential of NPUs in content creation is enormous. We're just scratching the surface of what's possible in terms of real-time video analytics, automated moderation, and personalized interactions. Imagine a live stream that adapts on the fly to viewer reactions and comments, creating a truly interactive experience. In this series, we'll continue exploring the technologies shaping the past, present and future of live streaming. Stay tuned for more insights into how these innovations are redefining not just how we create content, but how we connect with the world.<iframe loading='lazy' width='1280' height='720' src='https://www.youtube.com/embed/WsL5F_Jm0V0?si=iAn8MKnQygt_Z-vO&amp;controls=0' title='YouTube video player' frameborder='0' allow='accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share' referrerpolicy='strict-origin-when-cross-origin' allowfullscreen/>

This is the first article in a series that take a closer look at the underlying technologies that power the world of live streaming and content creation

Live streaming technology - Past, Present and Future

NPU processors and AI technology

<img loading="lazy" class="alignnone" alt="XSplit sponsors OBS" width="1000" height="500" src="/blog/why-xsplit-is-sponsoring-obs/static/obs-xsplit-thumbnail.png">
Recently, several companies have taken the open source OBS code, rebranded it, and built entire new venture-funded businesses, largely based on the technology created by Jim and the rest of the OBS team over the past 7 years as a true passion project. This makes it more important than ever to remember and appreciate the hard work that they’ve done and are still doing, to develop and maintain OBS, and how it has been a huge contributor, alongside XSplit, to empower a whole new generation of content creators, to turn their passions and entrepreneurial spirit into entire new careers that we couldn’t even have imagined a decade ago.
Back in 2010, when live streaming was still in its infancy, we launched XSplit Broadcaster in Open Beta. Those were the days before Twitch spun out of Justin.tv, and own3d.tv appeared as the first content platform dedicated to making live streaming of gameplay something more than a rare oddity. We were proud and humbled to be there from the very beginning – to be a part of forging a whole new industry out of dreams, passion and new technology.
Some time later, OBS came into being in 2012, born out of a shared passion for technology and the same dream of making game streaming more accessible for everyone. While we set out to make XSplit a business to continually and sustainably create and explore new innovation and technology, OBS was made by the community, for the community, as a free, non-profit and open source project driven by the brilliant Hugh “Jim” Bailey. The introduction of OBS to the market kept us on our toes, spurred innovation, and truly helped to even further accelerate the growth of the game streaming industry. Great competition drives innovation, making us all better and making the pie bigger.
Live streaming has truly exploded over the last few years, and the powerhouse of Twitch has been joined by major companies like Youtube, Facebook and Microsoft, to create platforms where millions of people gather every day to watch new and exciting live stream content. Yet to this day, most of the content creators broadcasting to these platforms still use XSplit or OBS technology in one form or another to create the content itself.
We at XSplit want to support the OBS project and the OBS contributors to be able to sustainably continue the hard work they do for the streamers and the industry, which is why we’ve committed to support the OBS Project as a Gold Sponsor through Open Collective.
Today, XSplit is no longer just a streaming application. XSplit VCam was our first big step into our endeavours to create more and broader content creation apps for the market, with more to come later this year. Our goal is to make everything we do inclusive and useful for XSplit and OBS users alike.
Let’s continue to keep each other on our toes, spur thoughts and innovation, grow the industry, create new technology and work together where we can, to help all the streamers, vloggers, producers, businesses and content creators out there in all shapes and forms, to create amazing content!
-Andreas Hoye, COO SplitmediaLabs

OBS and XSplit are the main contributors in livestreaming industry. XSplit supports the OBS project to continue to spur innovation in live streaming.

Andreas Hoye

Live streaming technology - Past, Present and Future

Why XSplit is sponsoring OBS