Locally Running LLMs Using Ollama

July 8, 2024

  • Tutorial
Thumbnail

In this tutorial, I'll guide you through running LLMs locally using Ollama. Ollama is a robust tool that enables local LLM execution, simplifying the development and testing of your applications. We'll cover setting up Ollama and using it to interact with an LLM.

Ollama

Step 1: Visit the Ollama Website and Click 'Download'

First, visit the Ollama website and click the 'Download' button to go to the download page.

Ollama Website

Step 2: Select 'Download for macOS'

Click the 'Download for macOS' button to download the Ollama installer for macOS.

Download for macOS

Step 3: Uncompress the Zip File

After the download completes, uncompress the file to access the Ollama installer.

Uncompress the Zip File

Step 4: Run the Installer

Double-click the installer and follow the on-screen instructions to complete the installation.

Run the Installer
Run the Installer
Run the Installer
Run the Installer

Step 5: Open 'Terminal' (Using Hyper Terminal)

Open the 'Terminal' application, found in the 'Utilities' section of your macOS menu.

Open 'Terminal' (Using Hyper)

Step 6: Run the Command

Type the command ollama run gemma:2b and press enter to pull and run the gemma:2b model.

Run the Command

Gemma is a family of lightweight, state-of-the-art open models from Google, developed using the same research and technology behind the Gemini models. These text-to-text, decoder-only large language models are available in English with open weights, pre-trained variants, and instruction-tuned variants. Gemma models excel in various text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes them deployable on resource-limited environments like laptops, desktops, or personal cloud infrastructure, democratizing access to cutting-edge AI and fostering innovation.

Learn more about Gemma at huggingface.co/google/gemma-2b.

Step 7: Send a Message

Once the model is running, send a message to the LLM by typing it in and pressing enter. You'll see a response from the LLM.

Send a Message

Congratulations! You have successfully run an LLM locally using Ollama. You can now interact with the LLM and explore the possibilities AI offers.