Hidden Profile Notebook / macOS

Local AI Agents and Open Models on macOS

A practical installation path for running local open-source models and common AI agents on Apple Silicon Macs. The default route is Ollama or LM Studio for model serving, then Continue, Aider, Open WebUI, AnythingLLM, Dify, or OpenHands on top.

Mac PrerequisitesBase

Install the common developer base first

xcode-select --install /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" brew install git git-lfs python@3.12 pipx node pnpm cmake rust pipx ensurepath
Recommended machine: Apple Silicon Mac with 16 GB unified memory for 7B/8B models, 32 GB for comfortable 14B models, and 64 GB or more for larger local experiments.

Install Docker Desktop only when you need OpenHands, Dify, Open WebUI, or full self-hosted stacks. For pure local chat and coding, Ollama or LM Studio is usually enough.

Model ChoiceLLM

Good default open models for macOS

Use caseModels to try
General local assistantLlama 3.1/3.2 8B, Qwen2.5 7B/14B, Mistral 7B
Coding agentQwen2.5-Coder 7B/14B, DeepSeek-Coder V2 Lite, CodeLlama
Longer contextQwen2.5 14B, Llama 3.1 8B with higher context settings
Small fast modelPhi-3.5 Mini, Gemma 2 2B/9B, Llama 3.2 3B

Choose 4-bit quantized GGUF models for laptops. They are much easier to run than full-precision weights.

OllamaRunner

Fastest CLI path

brew install ollama brew services start ollama ollama pull qwen2.5-coder:7b ollama pull llama3.1:8b ollama run qwen2.5-coder:7b

Ollama exposes a local API at http://localhost:11434. Many tools can point to this endpoint directly.

LM Studio / Jan / GPT4AllDesktop

Desktop model apps

brew install --cask lm-studio brew install --cask jan brew install --cask gpt4all

Use LM Studio when you want an easy GUI for downloading GGUF models and starting an OpenAI-compatible local server. Use Jan or GPT4All when you prefer a local-first chat desktop with simpler workspace management.

llama.cppRuntime

Direct GGUF runtime

brew install llama.cpp llama-cli -m ./models/model.gguf -p "Explain local RAG in one paragraph" llama-server -m ./models/model.gguf --host 127.0.0.1 --port 8080

Use llama.cpp when you want low-level control, direct GGUF testing, or a small local server without a desktop application.

MLXApple Silicon

Apple-native experiments

pipx install mlx-lm mlx_lm.generate --model mlx-community/Qwen2.5-Coder-7B-Instruct-4bit --prompt "Write a zsh function"

MLX is useful when you want Apple Silicon optimized model experiments. It is more developer-oriented than Ollama or LM Studio.

ContinueCoding

VS Code coding assistant with local models

ollama pull qwen2.5-coder:7b # Install the Continue extension in VS Code # Provider: Ollama # Model: qwen2.5-coder:7b # API base: http://localhost:11434

Continue is the easiest local coding assistant route inside VS Code. Pair it with Qwen Coder for chat and autocomplete.

AiderCoding Agent

Terminal coding agent

pipx install aider-chat ollama pull qwen2.5-coder:14b cd /path/to/git/repo aider --model ollama/qwen2.5-coder:14b

Aider works best inside a Git repository. It edits files through patches and keeps the conversation grounded in version control.

OpenHandsSoftware Agent

Local software engineering agent

# Install Docker Desktop first docker pull docker.all-hands.dev/all-hands-ai/openhands:latest docker run -it --rm \ -e LLM_MODEL="ollama/qwen2.5-coder:14b" \ -e LLM_BASE_URL="http://host.docker.internal:11434" \ -v /var/run/docker.sock:/var/run/docker.sock \ -p 3000:3000 \ docker.all-hands.dev/all-hands-ai/openhands:latest

OpenHands is heavier than Aider, but it can run commands, inspect files, and work through multi-step coding tasks in a browser UI.

Open WebUIChat UI

Local ChatGPT-like interface

ollama pull llama3.1:8b docker run -d -p 3001:8080 \ -v open-webui:/app/backend/data \ -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \ --name open-webui \ ghcr.io/open-webui/open-webui:main

Open http://localhost:3001. This is a good local team-style chat surface on top of Ollama.

AnythingLLMLocal RAG

Private documents and workspaces

brew install --cask anythingllm # In the app: # LLM provider: Ollama # Base URL: http://localhost:11434 # Model: llama3.1:8b or qwen2.5:7b

Use AnythingLLM when the goal is local documents, workspace-level memory, and simple private RAG without building a full stack.

Dify / FlowiseAgent Builder

Local visual agent workflows

# Dify git clone https://github.com/langgenius/dify.git cd dify/docker cp .env.example .env docker compose up -d # Flowise pnpm add -g flowise flowise start

Dify is better for app-like workflows, datasets, and RAG products. Flowise is lighter for node-based LangChain-style experiments.

Recommended PathSetup

A pragmatic macOS installation order

  1. Install Homebrew, Git, Python, Node, Docker Desktop.
  2. Install Ollama and pull qwen2.5-coder:7b plus llama3.1:8b.
  3. Install LM Studio for visual model discovery and fallback serving.
  4. Install Continue in VS Code and point it at Ollama.
  5. Install Aider for terminal-based repository edits.
  6. Add Open WebUI or AnythingLLM when you need local chat or local documents.
  7. Add OpenHands or Dify only when you need heavier browser-based agents or workflow products.
Copyleft© Ciel Z • CC BY-NC-SA 4.0