Choose a model and a private boundary
Choose a model and a private boundary — Local AI with Ollama. Local versus cloud; Inventory before installation; Verify the listener. Practical assignment: Lab evidence.
Demo edition. The materials are undergoing editorial review.
Topics covered
- Local versus cloud
- Inventory before installation
- Verify the listener
Local versus cloud
Choose a local text model for a fictional workshop summary. Check its model card, license, size and required features before downloading. Avoid cloud-labeled tags for this exercise; a localhost URL alone does not prove every selected backend runs locally.
Keep a small lab notebook containing the operating system, Ollama version, exact model tag, download size and task. A model tag identifies a particular available artifact, not a promise about quality or a universal memory requirement. Start with harmless invented text. Read the model license before redistributing weights or using them commercially. Keep installation and inference costs separate.
Inventory before installation
On macOS use the official application download; on Windows use the official installer; on Linux inspect the official installation instructions before executing them. Record available memory and disk first. The shell examples here use POSIX syntax; Windows learners can use WSL.
The listener address and the selected model are two separate decisions. A loopback listener can still forward to a cloud model, while a local model can be exposed by a public listener. Your note must therefore record both and explain how each was checked.
Run the example only after the previous check succeeds. The commands below target the local service; they do not authorize publishing a network port. Preserve the response and exit status before editing parameters. Change one input at a time so a later improvement can be attributed to an actual change. An empty response, HTTP error and slow generation require different diagnoses.
uname -a
df -h
# After official installation:
ollama --version
Verify the listener
Keep the service bound to loopback. In an isolated manual Linux session, use the command below only when another Ollama server is not running. An address-in-use error means inspect the existing process; it is not a reason to expose a different interface.
Measure a cold request separately from a second request with the model already loaded. Record whether the answer is correct against the supplied fictional facts, rather than judging only fluency. Keep private files out of prompts until the endpoint and logging behavior are verified. Local inference reduces one data transfer path, but downloads, integrations, backups and browser extensions still deserve inspection.
OLLAMA_HOST=127.0.0.1:11434 ollama serve
Practical assignment
Lab evidence
Continue to learning