Setting up Ollama and OpenWebUI

It's been quite a bit since I posted this article https://elmariouribe.blogspot.com/2024/02/setup-privategpt-on-fresh-ubuntu-2204.html.  If I had waited a few more weeks, it would have been two years since. This, in reality, is a continuation of those learnings.  I will focus more on the cybersecurity approach of using AI. As a couple of examples, analysing code and running vulnerability scans. This might quickly get out of proportion, and I may need to break some of these postings up or choose a different channel. 

We'll be using an old Laptop for this with a freshly installed version of Ubuntu 24.04. 

- sudo apt update

- sudo apt upgrade

• Install Ollama 
• curl -fsSL https://ollama.com/install.sh | sh

• # systemctl edit ollama.service

• [Service]
  Environment="OLLAMA_HOST=0.0.0.0"

• Restart Ollama by : 

• # systemctl restart ollama
  

• This will make it so Ollama listens on all IPs
• Pull an LLM
• ollama pull qwen2.5:7b
• ollama pull qwen2.5:7b
• Install Docker
• Run Container;

• docker run -d --network=host --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
  
• Install Go