SANTA CLARA — At its annual AWS re:Invent conference, Amazon Web Services (AWS) announced an expanded collaboration with NVIDIA to enhance the deployment of generative AI applications. NVIDIA NIM microservices, optimized for high-performance inference, are now available through AWS Marketplace, Amazon Bedrock Marketplace, and Amazon SageMaker JumpStart. This integration aims to deliver faster AI inference, reduced latency, and cost-effective scalability for enterprises leveraging increasingly complex AI models.

NVIDIA NIM, part of the NVIDIA AI Enterprise software platform, provides developers with secure, enterprise-grade microservices for deploying AI model inference across cloud, data center, and workstation environments. Built on robust inference engines like NVIDIA Triton Inference Server and NVIDIA TensorRT, these prebuilt containers support a wide range of AI models, from open-source community models to proprietary NVIDIA AI Foundation models.

Key Models and Applications

AWS services such as Amazon EC2, Amazon EKS, and Amazon SageMaker can now host over 100 NVIDIA NIM microservices, including:

• NVIDIA Nemotron-4: Available on multiple AWS platforms, this cutting-edge model generates synthetic data to enhance custom LLMs.
• Llama 3.1 8B-Instruct and 70B-Instruct: Multilingual large language models optimized for text generation and dialogue.
• Mixtral 8x7B Instruct v0.1: A sparse mixture of experts model designed for versatile text generation and task completion.

Industry Adoption and Use Cases

Organizations across industries are leveraging NIM microservices to streamline AI application deployment while maintaining security and reducing costs. SoftServe, a digital services provider, has created six generative AI solutions on AWS, powered by NVIDIA NIM and AWS services. These include applications in drug discovery, industrial assistance, and speech recognition, all based on NVIDIA AI Blueprints for rapid deployment.

Getting Started

Developers can explore NVIDIA NIM microservices on the NVIDIA API catalog, which offers over 100 optimized models. Deployment is available through AWS Marketplace, Amazon Bedrock Marketplace, and Amazon SageMaker JumpStart. Developers can also access a developer license or a 90-day NVIDIA AI Enterprise trial to evaluate NIM microservices for their specific needs.

This collaboration between AWS and NVIDIA highlights their commitment to advancing generative AI by providing scalable, secure, and high-performance solutions for enterprises.