Pre-Installed Qwen

Overview

Qwen is a state-of-the-art, open-source AI model ecosystem developed by Alibaba Cloud's Tongyi Lab. It includes a comprehensive range of large language and multimodal models—from lightweight 0.6B models to flagship 397B MoE architectures, supporting over 100 languages and dialects. Qwen delivers leading performance in reasoning, coding, instruction-following, tool use, and agentic workflows.

Select a pre-installed Qwen Marketplace image and get a fully configured server that's ready to use immediately—no manual installation or complex setup required. Once deployed, simply access the Qwen interface through your browser or API endpoint to start building AI applications, generate content, analyze documents, and automate workflows right away.

Supported Operating Systems

The Qwen Marketplace image is available for Ubuntu Server 24 LTS 64-bit.

Supported Products / Plans

1. Qwen3-VL-4B

The Qwen3-VL-4B Marketplace image is currently available across 2 GPU VPS plans and 2 GPU-dedicated server plans:

GPU VPS Plan:

• Professional GPU VPS - RTX A4000
• Advanced GPU VPS - RTX 5090

GPU-Dedicated Server Plans:

• Advanced Dedicated GPU Server - RTX A5000
• Enterprise Dedicated GPU Server - RTX 4090

View compatible plans →

2. Qwen3-VL-8B

The Qwen3-VL-8B Marketplace image is currently available across 2 GPU VPS plans and 2 GPU-dedicated server plans:

GPU VPS Plan:

• Professional GPU VPS - RTX A4000
• Advanced GPU VPS - RTX 5090

GPU-Dedicated Server Plans:

• Advanced Dedicated GPU Server - RTX A5000
• Enterprise Dedicated GPU Server - RTX 4090

View compatible plans →

3. Qwen3-32B

The GPT-OSS-20B Marketplace image is currently available across 4 GPU-dedicated server plans:

• Advanced Dedicated GPU Server - RTX A5000
• Enterprise Dedicated GPU Server - RTX 5090
• Enterprise Dedicated GPU Server - RTX 4090
• Enterprise Dedicated GPU Server - A100

View compatible plans →

4. Qwen3-VL-32B

The GPT-OSS-20B Marketplace image is currently available across 1 GPU VPS plan and 3 GPU-dedicated server plans:

GPU VPS Plan:

• Advanced GPU VPS - RTX 5090

GPU-Dedicated Server Plans:

• Enterprise Dedicated GPU Server - RTX A6000
• Enterprise Dedicated GPU Server - A40
• Enterprise Dedicated GPU Server - A100

View compatible plans →

Qwen Version

Database Mart offers the latest version of Qwen, pre-installed and ready to use.

What's Installed

When users deploy a Qwen pre-installed image through Database Mart, the following components are automatically installed on the server:

Core Operating Environment

• Operating System: Ubuntu Server 24 LTS
• GPU Drivers & Acceleration: NVIDIA drivers, CUDA 12.x, and cuDNN pre-installed for GPU acceleration.
• Python Environment: Python with a dedicated virtual environment (Conda or venv) for dependency management

Qwen Core Components & Inference Tools

• Deep Learning Frameworks: PyTorch 2.0+, Transformers library, accelerate, tiktoken, einops, and other essential libraries
• Model Weights & Configurations: Pre-trained model weights and configuration files (config.json, tokenizer.json) downloaded automatically based on the selected model
• Inference Engines: High-performance frameworks like vLLM, SGLang, or TensorRT-LLM. For vision-language models, qwen-vl-utils and other specialized tools are also installed
• Quantization Support: Model quantization (GPTQ, AWQ) support is included to reduce VRAM usage

Installation Process

During server provisioning, Database Mart automatically performs the following steps:

1. Deploys the selected Linux operating system (Ubuntu Server 24 LTS).

2. Installs the selected Qwen model version with all pre-trained weights and configuration files.

3. Installs and configures all required components, including:

   • Python 3.10+ with a dedicated virtual environment (Conda or venv)
   • PyTorch 2.0+ and the Transformers library
   • vLLM for high-performance inference
   • CUDA and cuDNN for GPU acceleration
   • FlashAttention for optimized attention computation
   • Model weights downloaded from Hugging Face or ModelScope
   • Quantization tools (GPTQ, AWQ) for reduced memory footprint

Everything is fully automated—your Qwen server is ready to use as soon as deployment is complete.

Getting Started After Deployment

After your server has been provisioned, follow these steps to begin using Qwen.

Step 1: Access the Server

Once deployment is complete, connect to your server via SSH using the credentials provided by Database Mart:

ssh administrator@your-server-ip

The Qwen model files are pre-installed in the designated directory (typically /opt/qwen or a similar location).

Step 2: Verify the Installation

Check that all components are properly installed and the model is ready:

# Verify CUDA and GPU availability
nvidia-smi

# Check Python environment
python3 --version

# Navigate to the Qwen directory
cd /opt/qwen

# List model files
ls -la

The model weights should be present in the model directory.

Step 3: Start the Inference Service

The server comes with vLLM pre-configured for high-performance inference. Start the service using:

# For Qwen3-8B (or similar size)
vllm serve Qwen/Qwen3-8B --host 0.0.0.0 --port 8000

# For Qwen3-32B
vllm serve Qwen/Qwen3-32B --host 0.0.0.0 --port 8000

Important flags explained :
--host 0.0.0.0: Binds the server to all network interfaces
--port 8000: Specifies the API port
--tensor-parallel-size <N>: Enables distributed inference across N GPUs (use if your server has multiple GPUs)
--enable-reasoning: Enables the model's reasoning capabilities

Note: If the model does not point to a valid local directory, vLLM will download model files from Hugging Face Hub by default. If you need to use ModelScope as the download source, set export VLLM_USE_MODELSCOPE=true before running the command.

The API server will start and be accessible at http://your-server-ip:8000/v1.

Step 4: Test the API with a Simple Request

Once the service is running, you can test it using the OpenAI-compatible API endpoint :

Using curl:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-8B",
    "messages": [{"role": "user", "content": "Hello, who are you?"}]
  }'

Using Python with the OpenAI client :

from openai import OpenAI

# Connect to your local Qwen server
client = OpenAI(
    api_key="EMPTY",  # Not required for local deployment
    base_url="http://localhost:8000/v1"
)

# Send a test request
messages = [{"role": "user", "content": "Explain quantum computing in simple terms"}]

response = client.chat.completions.create(
    model="Qwen/Qwen3-8B",  # or your specific model name
    messages=messages
)

print(response.choices[0].message.content)

The API follows the standard OpenAI Chat Completions format, making it easy to integrate with existing applications that already use OpenAI's API.

Step 5: Use System Prompts and Multi-Turn Conversations

For more advanced interactions, you can include a system prompt and build multi-turn conversations:

messages = [
    {"role": "system", "content": "You are a helpful assistant that responds like a Michelin-starred chef."},
    {"role": "user", "content": "Can you name two techniques to cook lamb?"},
    {"role": "assistant", "content": "Bonjour! Let me tell you about sous vide and hot smoking."},
    {"role": "user", "content": "Tell me more about the first method."}
]

response = client.chat.completions.create(
    model="Qwen/Qwen3-32B",
    messages=messages
)

Step 6: Enable Streaming for Long Responses

For real-time output (especially useful for longer responses), enable streaming:

response = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[{"role": "user", "content": "Tell me a long story about AI"}],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Notes

• Vague prompts produce vague results. The more context and constraints you provide, the better Qwen can deliver. Giving Qwen a specific persona or expertise level dramatically improves output quality.
• vLLM is the recommended inference engine for Qwen models. It uses PagedAttention to manage KV cache efficiently, supports continuous batching, and can deliver 3-5x higher throughput than vanilla Transformers.
• Use INT8 or GPTQ 4-bit quantization to reduce memory usage and improve inference speed. INT8 quantization can reduce memory usage by 50% while maintaining <2% accuracy loss.
• Qwen3.7 and later support parallel tool calling — the model can trigger multiple tools in a single response. This significantly reduces latency for multi-step tasks.
• When tool count exceeds 20, the model's selection accuracy decreases. Consider grouping tools by scenario and injecting only the relevant subset.

Frequently Asked Questions

Q: What Qwen models are available from Database Mart?

Database Mart offers several pre-installed Qwen variants to fit different hardware and use cases:

The 32B models represent the top end of the Qwen3 family and deliver "95% of flagship accuracy" on benchmarks while fitting on a single 24GB GPU when quantized.

Q: What's the benefit of using a pre-installed image vs. manual deployment?

The pre-installed image handles all the heavy lifting for you:

• OS installation and configuration
• Python environment setup with all dependencies
• Model weight download (up to 65GB+ for 32B models)
• vLLM inference engine configuration
• CUDA and cuDNN setup for GPU acceleration
• REST API endpoint pre-configured

You get a "ready-to-run" environment immediately after deployment, skipping hours of manual setup and troubleshooting.

Q: What inference engines are supported?

The pre-installed image comes with vLLM pre-configured and ready to use. Qwen also supports:

• vLLM: Recommended for production serving with high throughput (3-5x faster than vanilla Transformers)
• Ollama: For local deployment on consumer hardware
• LM Studio: User-friendly desktop app
• SGLang: Alternative high-performance inference engine
• Transformers (Hugging Face): For flexible, code-level control

Q: What are the hardware requirements?

Based on Qwen's official guidance:

Note: Quantization (GPTQ, AWQ) reduces memory usage by approximately 70% without severe accuracy loss.