Qualcomm to launch next-generation AI inference-optimized solutions

Qualcomm Technologies (QCOM) announced the launch of its next-generation AI inference-optimized solutions for data centers: the Qualcomm AI200 and AI250 chip-based accelerator cards, and racks. “Building off the company’s NPU technology leadership, these solutions offer rack-scale performance and superior memory capacity for fast generative AI inference at high performance per dollar per watt-marking a major leap forward in enabling scalable, efficient, and flexible generative AI across industries. Qualcomm AI200 introduces a purpose-built rack-level AI inference solution designed to deliver low total cost of ownership and optimized performance for large language & multimodal model inference and other AI workloads. It supports 768 GB of LPDDR per card for higher memory capacity and lower cost, enabling exceptional scale and flexibility for AI inference. The Qualcomm AI250 solution will debut with an innovative memory architecture based on near-memory computing, providing a generational leap in efficiency and performance for AI inference workloads by delivering greater than 10x higher effective memory bandwidth and much lower power consumption. This enables disaggregated AI inferencing for efficient utilization of hardware while meeting customer performance and cost requirements. Both rack solutions feature direct liquid cooling for thermal efficiency, PCIe for scale up, Ethernet for scale out, confidential computing for secure AI workloads, and a rack-level power consumption of 160 kW,” the company stated.