Pre-Installed Gemma

Overview

Gemma is a family of open-weight AI models developed by Google DeepMind, built from the same research and technology powering the Gemini models. It includes a comprehensive range of models—from lightweight 2B and 4B effective-parameter models optimized for edge and mobile devices, to powerful 12B and 31B dense models, to a highly efficient 26B Mixture-of-Experts (MoE) architecture with 4B active parameters. Gemma delivers leading performance in natural language understanding, reasoning, coding, multimodal comprehension (text, image, and audio on smaller models), and agentic workflows, with context windows up to 256K tokens and native support for over 140 languages.

Select a pre-installed Gemma Marketplace image and get a fully configured server that's ready to use immediately—no manual installation or complex setup required.

Supported Operating Systems

The Gemma Marketplace image is available for Ubuntu Server 24 LTS 64-bit.

Supported Products / Plans

The Gemma3-27B Marketplace image is currently available across 3 GPU VPS plans and 3 GPU-dedicated server plans:

GPU VPS Plan:

• Advanced GPU VPS - RTX Pro 4000
• Advanced GPU VPS - RTX Pro 5000
• Advanced GPU VPS - RTX 5090

GPU-Dedicated Server Plans:

• Advanced Dedicated GPU Server - RTX A5000
• Enterprise Dedicated GPU Server - RTX 4090
• Enterprise Dedicated GPU Server - A100

View compatible plans →

Gemma Version

Database Mart offers the latest version of Gemma, pre-installed and ready to use.

What's Installed

When you select the Gemma Marketplace image, Database Mart automatically installs and configures:

Core Operating Environment

• Operating System: Ubuntu Server 24 LTS, pre-optimized for AI workloads
• GPU Drivers & Acceleration: NVIDIA drivers, CUDA 12.x, and cuDNN pre-installed for GPU acceleration
• Python Environment: Python 3.10+ with a dedicated virtual environment (Conda or venv) for dependency management

Gemma Core Components & Inference Tools

• Deep Learning Frameworks: PyTorch 2.0+, Transformers library, and other essential libraries
• Model Weights & Configurations: Pre-trained model weights and configuration files (config.json, tokenizer.json) downloaded automatically based on the selected model (Gemma 3 27B IT)
• Inference Engines: High-performance frameworks like vLLM, Ollama, Transformers, and Gemma.cpp for efficient inference across different hardware
• Quantization Support: Model quantization (INT4, INT8) included to reduce VRAM usage — for example, Gemma 3 27B drops from 54GB (BF16) to just 14.1GB (INT4)

Hardware & GPU Support

• Multimodal Capabilities: Support for text and image input processing
• FlashAttention: Included for optimized attention computation

Installation Process

During server provisioning, Database Mart automatically performs the following steps:

1. Deploys the selected Linux operating system (Ubuntu Server 24 LTS).

2. Installs the selected Gemma model version (Gemma 3 27B IT) with all pre-trained weights and configuration files.

3. Installs and configures all required components, including:

   • Python 3.10+ with a dedicated virtual environment (Conda or venv)
   • PyTorch 2.0+ and the Transformers library
   • vLLM or Ollama for high-performance inference
   • CUDA and cuDNN for GPU acceleration
   • FlashAttention for optimized attention computation
   • Model weights downloaded from Hugging Face
   • Quantization tools (INT4, INT8) for reduced memory footprint
   • REST API server (OpenAI-compatible endpoint) for easy integration

Everything is fully automated—your DeepSeek server is ready to use as soon as deployment is complete.

Getting Started After Deployment

After your server has been provisioned, follow these steps to begin using Gemma.

Step 1: Access the Server

Once deployment is complete, connect to your server via SSH using the credentials provided by Database Mart:

ssh administrator@your-server-ip

The Gemma model files are pre-installed in the designated directory (typically /opt/gemma or a similar location).

Step 2: Verify the Installation

Check that all components are properly installed and the model is ready:

# Verify CUDA and GPU availability
nvidia-smi

# Check Python environment
python3 --version

# Navigate to the Gemma directory
cd /opt/gemma

# List model files
ls -la

The model weights should be present in the model directory. For Gemma 3 27B, the BF16 model requires approximately 54GB VRAM, while the INT4 quantized version requires only 14.1GB.

Step 3: Start the Inference Service

The server comes with vLLM pre-configured for high-performance inference. Start the service using:

# For Gemma 3 27B
vllm serve "unsloth/gemma-3-27b-pt-unsloth-bnb-4bit" --host 0.0.0.0 --port 8000

Important flags explained:

• --host 0.0.0.0: Binds the server to all network interfaces
• --port 8000: Specifies the API port
• --tensor-parallel-size <N>: Enables distributed inference across N GPUs

The API server will start and be accessible at http://your-server-ip:8000/v1.

Step 4: Test the API with a Simple Request

Once the service is running, you can test it using the OpenAI-compatible API endpoint:

curl -X POST "http://localhost:8000/v1/completions" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "unsloth/gemma-3-27b-pt-unsloth-bnb-4bit",
    "prompt": "Once upon a time,",
    "max_tokens": 512,
    "temperature": 0.5
  }'

Using the OpenAI Python client:

from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",  # Not required for local deployment
    base_url="http://localhost:8000/v1"
)

response = client.completions.create(
    model="unsloth/gemma-3-27b-pt-unsloth-bnb-4bit",
    prompt="The future of AI is",
    max_tokens=100,
    temperature=0.7
)

print(response.choices[0].text)

Step 5: Use Chat Completions

For instruction-tuned models like Gemma 3 27B IT:

response = client.chat.completions.create(
    model="google/gemma-3-27b-it",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in simple terms."}
    ],
    max_tokens=500,
    temperature=0.7
)

print(response.choices[0].message.content)

Step 6: Use Ollama for Local-Style Inference

If your image includes Ollama, you can also run Gemma using:

# Run Gemma 3 27B
ollama run gemma3:27b

# Or run the default 4B model
ollama run gemma3

Step 7: Process Multimodal Inputs

Gemma 3 supports vision-language inputs, handling both text and images. The model accepts images normalized to 896x896 resolution, encoded to 256 tokens each, with a total input context of 128K tokens for the 27B version.

Notes

Multilingual Support

Gemma 3 supports over 140 languages, making it ideal for global applications. The training dataset includes content in over 140 languages, and benchmarks show strong performance on multilingual tasks including MGSM, Global-MMLU-Lite, and XQuAD.

Context Window

Gemma 3 27B features a 128K token context window for the 4B, 12B, and 27B sizes, and 32K tokens for the 1B size.

Key Capabilities

• Multimodal: Handles text and image input, generates text output
• Function Calling: Supports structured outputs and function calling
• Reasoning: Strong performance on math and reasoning benchmarks
• Coding: Excellent code generation capabilities on HumanEval, MBPP, and Natural2Code

Frequently Asked Questions

Q: What Gemma models are available from Database Mart?

Database Mart offers the Gemma 3 27B IT (Instruction-Tuned) model, the most capable variant that can run on a single GPU. It supports text and image inputs and is optimized for sophisticated applications.

Q: What are the hardware requirements for Gemma 3 27B?

BF16 precision: Requires ~54GB VRAM (e.g., NVIDIA H100)

INT4 quantization: Requires ~14.1GB VRAM (fits on RTX 3090/4090 with 24GB)

Q: What inference engines are supported?

The pre-installed image comes with vLLM and Ollama pre-configured. Gemma also supports:

• Hugging Face Transformers: Flexible, code-level control
• Gemma.cpp: Lightweight inference for resource-constrained environments
• Keras, JAX, MediaPipe, Google Cloud: Official integration options

Q: Does Gemma 3 support image inputs?

Yes. Gemma 3 27B is a multimodal model that handles both text and image input and generates text output. Images are normalized to 896x896 resolution and encoded to 256 tokens each.

Q: What languages does Gemma 3 support?

Gemma 3 supports over 140 languages with native multilingual capabilities. It has been evaluated on Global-MMLU-Lite, WMT24++, FloRes, XQuAD, and other multilingual benchmarks.