Ollama

KaibanJS allows you to integrate Ollama's powerful language models into your applications. This integration enables you to run various open-source models locally, providing flexibility and control over your AI capabilities.

Overview

Ollama is a tool that allows you to run open-source large language models locally. By integrating Ollama with KaibanJS, you can leverage these models in your agents, enabling offline operation and customization of your AI assistants.

Supported Models

Ollama supports a wide range of open-source models, including but not limited to:

Llama 2 (7B, 13B, 70B)
Code Llama (7B, 13B, 34B)
Mistral (7B)
Phi-2
Falcon (7B, 40B)
Orca 2
Vicuna
Etc

For the most up-to-date list of available models and their capabilities, please refer to the official Ollama model library.

Integration Steps

To use an Ollama model in your KaibanJS agent, follow these steps:

Install Ollama: First, ensure you have Ollama installed on your system. Follow the installation instructions on the Ollama website.
Install LangChain's Ollama Integration: Install the necessary package:
```
npm install @langchain/ollama
```

Import and Configure the Model: In your KaibanJS project, import and configure the Ollama model:

import { ChatOllama } from "@langchain/ollama";

const ollamaModel = new ChatOllama({
  model: "llama3.1",  // or any other model you've pulled with Ollama
  temperature: 0.7,
  maxRetries: 2,
  // Other Langchain-supported parameters
});

Create the Agent: Use the configured Ollama model in your KaibanJS agent:

const agent = new Agent({
  name: 'Ollama Agent',
  role: 'Assistant',
  goal: 'Provide assistance using locally run open-source models.',
  background: 'AI Assistant powered by Ollama',
  llmInstance: ollamaModel
});

Configuration Options and Langchain Compatibility

KaibanJS uses Langchain under the hood, which means you can use all the parameters that Langchain's Ollama integration supports. This provides extensive flexibility in configuring your language models.

Here's an example of using advanced configuration options:

import { ChatOllama } from "@langchain/ollama";

const ollamaModel = new ChatOllama({
  model: "llama2",
  temperature: 0.7,
  maxRetries: 2,
  baseUrl: "http://localhost:11434",
  // Any other Langchain-supported parameters...
});

const agent = new Agent({
  name: 'Advanced Ollama Agent',
  role: 'Assistant',
  goal: 'Provide advanced assistance using locally run open-source models.',
  background: 'AI Assistant powered by Ollama with advanced configuration',
  llmInstance: ollamaModel
});

For a comprehensive list of available parameters and advanced configuration options, please refer to the official Langchain documentation:

Langchain Ollama Integration Documentation

Best Practices

Model Selection: Choose an appropriate model based on your task requirements and available system resources.
Resource Management: Be aware of your system's capabilities when running larger models locally.
Updates: Regularly update your Ollama installation to access the latest models and improvements.
Experiment with Parameters: Adjust temperature, top_p, and other parameters to fine-tune model output for your specific use case.

Limitations

Performance depends on your local hardware capabilities.
Some larger models may require significant computational resources.
Ensure you comply with the licensing terms of the open-source models you use.

Further Resources

We Love Feedback!

Is there something unclear or quirky in the docs? Maybe you have a suggestion or spotted an issue? Help us refine and enhance our documentation by submitting an issue on GitHub. We're all ears!

Overview​

Supported Models​

Integration Steps​

Configuration Options and Langchain Compatibility​

Best Practices​

Limitations​

Further Resources​