MENLO PARK — Facebook has unveiled significant advancements in its Smart Camera technology with the launch of three new Portal video-calling devices. The Smart Camera, which is designed to intelligently frame the action during video calls, has been upgraded with cutting-edge AI and computer vision technologies to enhance user experience.

The core technology behind the Smart Camera is built on Facebook AI’s innovative Mask R-CNN framework, which excels in object instance segmentation and keypoint detection. With the introduction of the new Portal models, Facebook has further refined these capabilities by leveraging Detectron2, a second-generation object detection platform also developed by Facebook AI. Detectron2, created in PyTorch, allows for faster and more precise model training, leading to quicker iterations and improvements in the Smart Camera’s performance.

In addition to software upgrades, Facebook has integrated custom hardware solutions that optimize object detection and introduce features like foveated processing. This approach prioritizes processing power on specific regions of the camera sensor, enhancing the overall video-calling experience on devices powered by lower-energy IoT chipsets.

These advancements reflect Facebook’s commitment to continuously improving the technology behind its Portal devices, offering users a more seamless and engaging video-calling experience.

Click here to learn more.