Getting Started with LLMs on Your NVIDIA DGX Spark
Welcome! Below you'll find a curated collection of tools, platforms, and resources to help you start building and running large language models on your NVIDIA DGX Spark.
NVIDIA DGX Spark Playbooks
The official NVIDIA DGX Spark documentation and playbooks. Step-by-step guides for setting up, optimizing, and deploying AI workloads on your DGX Spark hardware.
Hugging Face
The go-to hub for open-source AI models, datasets, and demos. Browse thousands of pre-trained LLMs, fine-tune models, and share your own — all in one place.
vLLM
A high-throughput and memory-efficient inference engine for LLMs. vLLM enables fast serving of large models with PagedAttention, making it ideal for production deployments on DGX Spark.
Ollama
Run large language models locally with a simple CLI and API. Ollama makes it easy to pull, run, and manage models like Llama, Mistral, and Gemma directly on your machine.
LangChain
A powerful framework for building LLM-powered applications. LangChain provides tools for chaining prompts, connecting to data sources, and building agents and pipelines.
LlamaIndex
A data framework for LLM applications that simplifies ingesting, indexing, and querying your own data. Perfect for building RAG (retrieval-augmented generation) systems on your DGX Spark.
NVIDIA NIM
NVIDIA Inference Microservices (NIM) let you deploy optimized AI models as containerized microservices. Get production-ready LLM endpoints with enterprise-grade performance on your DGX hardware.
NVIDIA NGC Catalog
A curated catalog of GPU-optimized AI software, containers, models, and SDKs from NVIDIA. Find ready-to-run LLM containers specifically optimized for DGX systems.
PyTorch
The leading open-source deep learning framework. PyTorch is the foundation for most modern LLM research and training, with excellent CUDA support for NVIDIA GPUs.
OpenAI API (Compatible Endpoints)
The OpenAI API specification has become an industry standard. Many local LLM tools (like vLLM and Ollama) expose OpenAI-compatible endpoints, letting you reuse existing integrations.
Weights & Biases (W&B)
The industry-standard platform for tracking ML experiments, visualizing training runs, and collaborating on model development. Essential for fine-tuning and evaluating LLMs on your DGX Spark.
NVIDIA Container Toolkit
docs.nvidia.com/container-toolkit
Enables GPU support inside Docker containers. A must-have for running LLM workloads in isolated, reproducible environments on your DGX Spark.