Make Ollama Start on Boot: A systemd Service That Works

My home server runs headless. I set up Ollama, everything worked, I rebooted it a week later during a kernel update, and the service came back without GPU support and without any of my environment variables. The port was listening, so monitoring said green. Inference had quietly moved to CPU and was eight times slower.

The official install script does create a systemd unit, and it is fine on a desktop. On a headless machine with a dedicated GPU and a firewall, there are four things it does not handle. Here is the unit I run now and why each line is there.

Check what you currently have

Related: Why Ollama's First Response Is Slow (Cold Start Fix) →

systemctl status ollama
systemctl cat ollama

If systemctl cat prints nothing, the install script did not create a service — likely because you installed manually or with Homebrew. If it prints a unit, note the Environment= lines. On most installs there are none, which is the root of the problem: every setting you exported in your shell disappears when systemd starts the daemon.

The unit file

Related: Ollama Filled My Disk: How I Moved the Models Directory Safely →

Write this to /etc/systemd/system/ollama.service:

[Unit]
Description=Ollama Service
After=network-online.target
Wants=network-online.target

[Service] Type=simple ExecStart=/usr/local/bin/ollama serve User=ollama Group=ollama Restart=always RestartSec=3 TimeoutStopSec=30

Environment="HOME=/usr/share/ollama" Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" Environment="OLLAMA_HOST=0.0.0.0:11434" Environment="OLLAMA_MODELS=/srv/ollama/models" Environment="OLLAMA_KEEP_ALIVE=24h" Environment="OLLAMA_MAX_LOADED_MODELS=2" Environment="OLLAMA_NUM_PARALLEL=1" Environment="CUDA_VISIBLE_DEVICES=0"

[Install] WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl enable --now ollama

The pieces that matter:

After=network-online.target, not network.target. On a server with a bonded interface or a DHCP lease, network.target fires before an address exists. Binding to 0.0.0.0 still works, but anything you do at startup that resolves a hostname will fail. The Wants= line is required for network-online.target to actually be pulled in.

HOME set explicitly. Ollama stores models under $HOME/.ollama unless told otherwise. systemd services do not inherit your shell's HOME, so without this line a service running as the ollama user may look in a directory that does not exist and re-download everything.

OLLAMA_HOST=0.0.0.0:11434. The default binds to localhost only. If you reach the server from another machine, you need this — and you need to think about who else can now reach it. Details on locking that down are in accessing Ollama from another computer on the LAN.

Restart=always with RestartSec=3. A crash during model load should not leave the daemon down until you notice. Three seconds is long enough to avoid a tight restart loop and short enough that clients retry successfully.

CUDA_VISIBLE_DEVICES. If the box has more than one GPU, or an integrated adapter alongside a discrete card, device ordering can change across reboots. Pinning the index means the service always uses the card you intended.

Give the model directory to the service user

Related: Ollama Not Using My NVIDIA GPU in WSL2: The Fix That Finally Worked →

The most common failure after switching to a dedicated user is a permissions error that reads like a network problem. Set ownership on whatever path you used for OLLAMA_MODELS:

sudo mkdir -p /srv/ollama/models
sudo chown -R ollama:ollama /srv/ollama
sudo chmod 755 /srv/ollama

If your models are on a separate mount, add an ordering dependency so the service does not start before the disk is available:

[Unit]
RequiresMountsFor=/srv/ollama

Without that line, a reboot can start Ollama against an empty mount point, it will find no models, and pulling one will write to the underlying directory that later gets shadowed by the real mount. That produced a very confusing hour for me.

Verify it survived, properly

Related: Ollama Pull Fails with "max retries exceeded" or EOF: How I Get Downloads to Finish →

Rebooting and seeing a listening port is not a test. Check the three things that actually broke:

sudo reboot

After it comes back:

systemctl is-active ollama
systemctl show ollama --property=Environment
ollama list
ollama run llama3.1:8b "say ok" --verbose

The --verbose output prints evaluation rate. If it reports single-digit tokens per second on a machine with a discrete GPU, CUDA was not picked up. Confirm with:

journalctl -u ollama -b | grep -i -E "cuda|gpu|library"

You want to see the CUDA library being loaded and the GPU detected. If it says no compatible GPUs were discovered, the driver loaded after the service; adding After=nvidia-persistenced.service or enabling the persistence daemon usually resolves it:

sudo systemctl enable --now nvidia-persistenced

Adding an override instead of replacing the unit

Related: DeepSeek-R1 Shows Its <think> Tags in the Output — Here Is How I Strip Them →

If the install script's unit already works and you only need environment variables, do not edit it in place — package updates will overwrite your changes. Use a drop-in:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_KEEP_ALIVE=24h"

That writes to /etc/systemd/system/ollama.service.d/override.conf and survives upgrades. Reload and restart afterwards.

Logs worth knowing

Related: Ollama Answers Get Cut Off Mid-Sentence: num_ctx vs num_predict Explained →

journalctl -u ollama -f
journalctl -u ollama --since "10 min ago" -p err

The messages I check for most often:

The address-in-use case is worth a note: if you previously ran Ollama in Docker, that container may still be starting on boot and grabbing the port before systemd does. Stop and disable it, or the service will restart forever. That specific conflict is one of the causes listed in connection refused on port 11434.

If you put it behind a reverse proxy

Once the service is reliable, exposing it beyond the LAN should go through TLS rather than a raw open port. Bind Ollama to localhost, keep the systemd unit as above with OLLAMA_HOST=127.0.0.1:11434, and let nginx terminate the connection — the configuration is in putting Ollama behind an nginx HTTPS reverse proxy.

With the unit above my server has been through five reboots and two driver updates without intervention, and more importantly, when it does come back it comes back with the GPU. Monitoring a port tells you the process is alive; checking the evaluation rate tells you it is useful.