Qualcomm Announces The Vision Intelligence Platform, A Pair of Chipsets Designed For IoT Devices

Chipmaker Qualcomm is undoubtedly best known for its Snapdragon series, but there's more to the company's portfolio than smartphone-optimized system-on-chips (SoCs). Take its Internet of Things lineup, for example: in January, it announced the Home Hub Platform, designed with smart home speakers, screens, and all-in-one devices in mind. And on Wednesday, it took the wraps off the Qualcomm Vision Intelligence Platform, a pair of new 10nm FinFET SoCs engineered from the ground up for camera-equipped IoT home and industrial gadgets.

"It's ideally suited for IoT segments that require a significant amount of edge computing and AI," Sashu Madhavapeddy, Vice President of Product Management at Qualcomm, told XDA Developers in a phone interview. "We've been very active in these segments for a few years now, [and] what we've been able to do is learn from those engagements, understand what the market really needs, and create a custom family of chipsets that directly target those markets and are able to deliver much greater performance."

Two SoCs headline Qualcomm's lineup: the QCS605 and QCS603. Both consist of a powerful new image signal processor (ISP), the Spectra 270; a Snapdragon processor; Qualcomm's Hexagon 685 Vector Processor; and an Adreno 615 graphics processing unit (GPU).

Chips in the Vision Intelligence Platform are engineered for smart security cameras, 360-degree cameras, sports cameras, robotics, and smart displays, Mr. Madhavapeddy said. Specifically, they're optimized for specialized applications such as on-device video stitching, autonomous navigation, obstacle avoidance, image tracking, and face and object recognition.

"We are going back to the building blocks [of] IoT, and what we find is that the critical elements are AI engines, camera ISP, and heterogeneous compute platform," Mr. Madhavapeddy said. "We're seeing very strong demand for increased edge computing capability, and it's coming for multiple reasons. For example, [if] you want to make millisecond decisions in order to avoid obstacles [in an autonomous vehicle], you've got to do it at the edge."

Mr. Madhavapeddy described the Spectra 270 as a "close cousin" of the Spectra 280 ISP in the Snapdragon 845. It boasts support for dual 16MP sensors and a dual 14-bit processor, plus advanced post- and real-time processing capabilities including electronic image stabilization (EIS), de-warping and de-noising algorithms that smooth out unwanted artifacts, chromatic aberration correction, and motion-compensated temporal filters. It's capable of recording 4K video at 60 frames per second (fps), or 5.7K at 30 fps, or multiple concurrent video streams (up to 16 maximum) at lower resolutions.

"We have to do a lot more image processing in the [Vision Intelligence Platform] that goes well beyond the image processing what is required for smartphones, because the use cases are different," Mr. Madhavapeddy said. "[If] it's powering an outdoor security camera, the low-light video performance is much greater than what a smartphone demands [because] security cameras have to operate in almost pitch darkness — as low as 1 lux of light. [And] think about a sports camera — [with] a sports camera that sits on top of your helmet, the magnitude of the shake is no comparison to what you see on a smartphone."

The QCS605 also packs an eight-core processor with Kryo 360 cores and the Adreno 615 GPU, the rumored components of Qualcomm's yet-to-be-confirmed Snapdragon 670.  The integrated display processor can drive screens up to WQHD (2560×1440) resolution with hardware-accelerated composition, touch, and 3D overlays, and natively supports graphics APIs including OpenGL, OpenCL, and Vulkan.

What really makes the Vision Intelligence Platform's stand out from the competition, though, is a robust support framework for machine learning-enabled applications. The QCS605 and QCS603 benefit from the Qualcomm Artificial Intelligence (AI) Engine, a heterogeneous compute platform that comprises the Qualcomm Snapdragon Neural Processing Engine (NPE) software framework and the Hexagon 685 Vector Processor.

Diagram of the Snapdragon 845's Hexagon DSP.

AI providers such as SenseTime and Pilot.ai supply compatible facial and object recognition models for Qualcomm's AI Engine. The NPE, a framework of optimization and analysis tools, is compatible with Google's TensorFlow, Facebook's Caffe and Caffe2, the Open Neural Network Exchange (ONNX), Android Neural Networks API, and the Qualcomm Hexagon Neural Network library. And as we discussed in our in-depth look at the Hexagon 685, the DSP substantially improves on its predecessor, the Hexagon 682, in the areas of data streaming, HVX register controlling, and image streaming. It's up to three times faster than the DSP in the Snapdragon 835 and uses "less than 10 percent" overhead on the system-on-chip's processor, according to Qualcomm.

"We expect the huge amount of AI performance we put in this platform to be taken advantage of by our customers," Mr. Madhavapeddy said. "It's all coming together. [These IoT devices] may look different, and the primary purpose of the product is different, but they have the same components."

Rounding out the Vision Intelligence Platform's accouterments is 2×2 802.11ac Wi-Fi (with MU-MIMO and dual-band simultaneous transmission), Bluetooth 5.1, hardware-based security features such as secure boot and root of trust, and Qualcomm's 3D Audio Suite for digital audio workstation (DAW) software. Both the QCS605 and QCS603 also support Qualcomm Aqstic, a melange of sound and voice tech with noise and echo cancellation, on-device audio analytics, natural language processing, audio speech recognition, ambient noise isolation, and "barge-in" features.

The QCS605 and QCS603 are available today in multiple SKUs, and Qualcomm teamed up with Altek Corporation to develop a QCS605-based 360-degree camera reference design. (The company expects a QCS603-based industrial security camera reference design to be available in the second half of 2018.)

Already, KEDACOM and Recoh Theta have Vision Intelligence Platform-based devices in the works. "There's a lot of innovation, and we are ready to be amazed with what our customers do with this platform," Madhavapeddy said.

from xda-developers https://ift.tt/2GSjml1
via IFTTT