AI/ML

Deploy Mistral 7B in Docker with Ollama on Google Cloud - Step by Step Guide

Download Mistral 7B for free - Follow our Step by Step Guide here!

Overview

Mistral 7B is an advanced AI model designed for high performance text generation. Hosting it in a Docker container on a Google Cloud VM instance allows for a flexible and scalable setup.

Prerequisites

Ensure you have the following ready:

A Google Cloud VM instance (Ubuntu preferred) with Docker installed.
SSH access to the VM.

Step 1: Connect to Your Google Cloud VM

Use the following command to SSH into your VM instance:

gcloud compute ssh your-vm-instance-name --zone=your-zone

Once inside, update your packages:

sudo apt update && sudo apt upgrade -y

Step 2: Deploy the Ollama Container

Start an Ollama container with persistent storage:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Step 3: Enter the Running Container

Access the Ollama container shell with:

docker exec -it ollama /bin/bash

Step 4: Download Mistral 7B Model

Pull the Mistral 7B model inside the container:

ollama pull mistral:7b

Step 5: Run the Model

Start the model with the following command:

ollama run mistral:7b

Test it by entering a prompt:

>>> What is the capital of France?

Step 6: Set Up a Web UI for Easier Access

To interact with Mistral 7B via a browser, deploy Open WebUI:

docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=http://<YOUR-VM-IP>:11434 -vopen-webui:/app/backend/data --name open-webui --restart alwaysghcr.io/open-webui/open-webui:main

Now, visit http://<YOUR-VM-IP>:3000 to use the model through an intuitive web interface.

Conclusion

Deploying Mistral 7B on Google Cloud using Docker and Ollama provides a scalable and efficient way to run AI models. This setup enables seamless interaction while maintaining an isolated and reproducible environment.

Ready to elevate your business with cutting edge AI and ML solutions? Contact us today to harness the power of our expert technology services and drive innovation.