How to Install and Run DeepSeek R1 Locally With vLLM V1

You can run DeepSeek R1 locally using vLLM to serve the model through an OpenAI-compatible API. For smaller distilled models such as DeepSeek-R1-Distill-Qwen-14B, a multi-GPU server with sufficient VRAM can provide a practical self-hosted AI environment. Choosing reliable GPU server hosting with adequate GPU memory, storage, and network capacity is essential for stable inference. Businesses evaluating AI hosting can also compare USA hosting options and infrastructure from a suitable hosting provider or USA hosting provider.

Why Use vLLM?

vLLM is an inference and serving engine designed for efficient large language model deployment. Its key features include:

• Efficient GPU memory management with PagedAttention.
• High-throughput inference for concurrent requests.
• Support for compatible Hugging Face models.
• OpenAI-compatible API endpoints for application integration.

Current vLLM releases use the V1 engine by default where supported, so you generally do not need to set VLLM_USE_V1=1 manually. Check the official vLLM documentation for version-specific options. <Cite refs={["turn994142search0","turn994142search3"]}/>

Prerequisites

Before installing vLLM, prepare the following:

• Ubuntu Linux or another supported Linux distribution.
• A compatible Python version and NVIDIA driver.
• An NVIDIA GPU supported by your selected vLLM release.
• Sufficient GPU VRAM, system RAM, and disk space for the model and its cache.

For this example, we use a server equipped with two RTX 4090 GPUs to run the 14B distilled model. This setup is not sufficient for the full 671B DeepSeek-R1 model. Hardware requirements vary by model, precision, context length, and workload.

Explore GPU server recommendations for local AI inference and GPU server hosting.

How to Install vLLM and Run DeepSeek R1

Step 1: Create a Python Virtual Environment

python3 -m venv deepseek_env
source deepseek_env/bin/activate
python -m pip install --upgrade pip

Step 2: Install vLLM

Install vLLM in the activated environment:

pip install vllm

Verify the installation:

vllm --version

Use the installation instructions for your specific CUDA and Python environment if additional dependencies are required.

Step 3: Download and Run DeepSeek R1

Start the server using the distilled Qwen 14B model:

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-14B \
  --max-model-len 4096 \
  --tensor-parallel-size 2 \
  --host 127.0.0.1 \
  --port 8000

The model will be downloaded from Hugging Face if it is not already available locally. The first startup may take several minutes while the model loads and GPU memory is initialized.

The --max-model-len parameter sets the maximum context length, while --tensor-parallel-size 2 distributes model computation across two GPUs. Adjust these values to match your model and hardware.

You can also use the alternative command:

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-14B \
  --max-model-len 4096 \
  --tensor-parallel-size 2

For a remote production deployment, configure authentication and network access appropriately. Avoid exposing an unauthenticated inference API directly to the public internet.

How to Query the vLLM API

1. Check the API

With the server running, open the API documentation at:

http://127.0.0.1:8000/docs

You can list the loaded model using:

curl http://127.0.0.1:8000/v1/models

2. Send a Chat Completion Request

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-14B",
    "messages": [
      {"role": "user", "content": "Explain how GPU memory affects LLM inference."}
    ],
    "max_tokens": 512
  }'

3. Access DeepSeek R1 Using Python

Install the OpenAI Python client:

pip install openai

Then send a request to your local vLLM server:

from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://127.0.0.1:8000/v1",
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-Distill-Qwen-14B",
    messages=[
        {"role": "user", "content": "Explain the benefits of self-hosted AI."}
    ],
    max_tokens=512,
)

print(response.choices[0].message.content)

If you configure API-key authentication on the server, replace the placeholder key with the configured value.

Conclusion

Running DeepSeek R1 locally with vLLM provides an efficient way to deploy reasoning models and expose them through an API. Choose a model that fits your available GPU memory, configure tensor parallelism appropriately, and secure the API before enabling remote access.

For production AI workloads, reliable GPU server hosting can help provide the compute resources needed for model inference. When comparing hosting, USA hosting, or a USA hosting provider, consider GPU availability, VRAM, storage, network connectivity, and scalability. Database Mart provides GPU server solutions, while DeepSeek R1 hosting options may be useful for evaluating self-hosted AI infrastructure.