安装

Ollama is distributed as a self-contained binary. Download it to a directory in your PATH:Ollama 作为独立的二进制文件分发。将其下载到 PATH 中的目录：

sudo curl -L https://ollama.com/download/ollama-linux-amd64 -o /usr/bin/ollama
sudo chmod +x /usr/bin/ollama

Adding Ollama as a startup service (recommended)将 Ollama 添加为启动服务（推荐）

[](https://github.com/ollama/ollama/blob/main/docs/linux.md#adding-ollama-as-a-startup-service-recommended)

Create a user for Ollama:为 Ollama 创建用户：

sudo useradd -r -s /bin/false -m -d /usr/share/ollama ollama

Create a service file in /etc/systemd/system/ollama.service:在以下位置创建 /etc/systemd/system/ollama.service 服务文件：

[Unit]
Description=Ollama Service
After=network-online.target

[Service]
Environment="OLLAMA_MODELS=/www/wwwroot/ollama;OLLAMA_HOST=0.0.0.0:11434"
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3

[Install]
WantedBy=default.target

Then start the service:然后启动服务：

sudo systemctl daemon-reload
sudo systemctl enable ollama
sudo systemctl start ollama

导入自定义gguf

Llama3-8b中文版（GGUF） · 模型库 (modelscope.cn)

下载别人微调好的，上传或者直接wget

创建文件
llama3chinese .modelfile

FROM llama3-zh.Q4_K_M.gguf
TEMPLATE """[INST] <>{{ .System }}<>

{{ .Prompt }} [/INST]
"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER stop "<>"
PARAMETER stop "<>"

ollama create llama3chinese --file llama3chinese.modelfile

测试

curl http://localhost:11434/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "llama3chinese",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Hello!Who are you?"}
        ],
        "stream": true
    }'