Tuesday, September 15, 2026

How to setup and use a local ollama server running on a windows 11 machine, from any machine in a local area network

 ## 1. Install Ollama on the Windows 11 host

Download and install Ollama on the machine with the GPU/CPU you want to use. Pull a model and verify it locally:

```powershell

ollama pull llama3.2

ollama run llama3.2

```

Ollama normally listens only on `127.0.0.1:11434`. To permit LAN clients, configure it to listen on all interfaces with `OLLAMA_HOST=0.0.0.0:11434`. [1,2]

## 2. Configure `OLLAMA_HOST`

On the Windows host:

1. Open **Settings → System → About → Advanced system settings**.

2. Select **Environment Variables**.

3. Under **User variables**, select **New**.

4. Enter:

   - **Variable name:** `OLLAMA_HOST`

   - **Variable value:** `0.0.0.0:11434`

5. Click **OK** through all dialogs.

6. Exit Ollama completely from the system tray, then relaunch it.

If Ollama runs as a service rather than as the desktop application, restart that service after changing the variable.

`OLLAMA_ORIGINS` is only relevant to browser-based cross-origin requests; it does not make the server reachable over the network. Avoid setting it to `*` unless you specifically need browser access from arbitrary origins.

## 3. Find the Windows host’s LAN IP address

On the Windows machine, run:

```powershell

ipconfig

```

Find the **IPv4 Address** for the active Wi-Fi or Ethernet adapter. It will usually look like:

```text

192.168.1.50

```

Use this address from other machines—not `localhost`, `127.0.0.1`, or `0.0.0.0`.

It is useful to reserve this address in your router’s DHCP settings so it does not change.

## 4. Allow TCP port 11434 through Windows Firewall

Open **PowerShell as Administrator** and create a rule limited to private networks:

```powershell

New-NetFirewallRule `

  -DisplayName "Ollama LAN API" `

  -Direction Inbound `

  -Protocol TCP `

  -LocalPort 11434 `

  -Action Allow `

  -Profile Private

```

Make sure the network is classified as **Private**, not Public:

**Settings → Network & internet → Wi-Fi/Ethernet → your network → Network profile → Private**

Do not expose port 11434 through your router or enable WAN/Internet port forwarding. Ollama’s local API generally should not be placed directly on the public Internet.

## 5. Test the server locally

On the Windows host:

```powershell

curl http://127.0.0.1:11434/api/tags

```

You should receive JSON listing installed models.

You can also test using the host’s LAN address:

```powershell

curl http://192.168.1.50:11434/api/tags

```

Replace `192.168.1.50` with the actual address.

## 6. Test from another LAN computer

From another machine on the same network:

```bash

curl http://192.168.1.50:11434/api/tags

```

A successful response confirms that the remote machine can reach Ollama and see its models. Both machines must be on the same reachable LAN; guest Wi-Fi and client-isolation settings can block this connection.

## 7. Call the model remotely

Example using the Ollama REST API:

```bash

curl http://192.168.1.50:11434/api/generate \

  -H "Content-Type: application/json" \

  -d '{

    "model": "llama3.2",

    "prompt": "Explain photosynthesis in two sentences.",

    "stream": false

  }'

```

For chat requests:

```bash

curl http://192.168.1.50:11434/api/chat \

  -H "Content-Type: application/json" \

  -d '{

    "model": "llama3.2",

    "messages": [

      {

        "role": "user",

        "content": "Write a short welcome message."

      }

    ],

    "stream": false

  }'

```

## 8. Use it from applications

Many Ollama-compatible applications have a **custom endpoint**, **base URL**, or **Ollama host** setting. Enter:

```text

http://192.168.1.50:11434

```

For OpenAI-compatible applications, the base URL is often:

```text

http://192.168.1.50:11434/v1

```

The exact setting depends on the application. In Python, an Ollama-compatible client commonly uses:

```python

from ollama import Client

client = Client(host="http://192.168.1.50:11434")

response = client.chat(

    model="llama3.2",

    messages=[

        {"role": "user", "content": "Give me three project ideas."}

    ],

)

print(response["message"]["content"])

```

## Troubleshooting

- **Local test works, remote test fails:** check the Windows Firewall rule and confirm the network is set to Private.

- **`curl` connects to `127.0.0.1` but not the LAN IP:** Ollama did not restart after setting `OLLAMA_HOST`, or it is still bound only to loopback.

- **Check what is listening:**

  ```powershell

  netstat -ano | findstr :11434

  ```

  You want to see `0.0.0.0:11434` or the machine’s LAN address, rather than only `127.0.0.1:11434`.

- **Connection times out:** check that both devices are on the same subnet and that Wi-Fi client isolation is disabled.

- **Model not found:** pull the model on the Windows host:

  ```powershell

  ollama pull llama3.2

  ```

- **Using WSL2 or Docker:** networking may require additional port forwarding because WSL2 uses a virtualized network adapter; a normal Windows installation is simpler for LAN access. [1,3]

The essential configuration is:

```text

OLLAMA_HOST=0.0.0.0:11434

```

Then connect from other machines using:

```text

http://<Windows-LAN-IP>:11434

```

Read More

[1] [WSL 2] Exposing ollama via 0.0.0.0 on local network #1431 - GitHub — https://github.com/ollama/ollama/issues/1431

[2] Impossible to connect to ollama locally from another pc #6571 - GitHub — https://github.com/ollama/ollama/issues/6571

[3] [WSL2] Ollama on Windows host unreachable from Docker ... - GitHub — https://github.com/ollama/ollama/issues/14947

Most affordable and practical new apple max studio for AI and creative work recommendation

 Assuming U.S. pricing, buy the **new Mac Studio with M5 Max, 18-core CPU, 40-core GPU, 64GB unified memory, and 1TB SSD**.

It is the most practical balance for AI-assisted creative work:

- **AI:** local image generation, transcription, coding assistants, and medium-sized local language models

- **Creative work:** 4K/8K video editing, Photoshop, Lightroom, audio production, motion graphics, and many professional apps

- **Memory:** 64GB gives substantially more headroom than the base 36GB configuration

- **Storage:** 1TB is usable; keep large media libraries and project archives on a fast external Thunderbolt SSD

- **GPU:** the 40-core option is worthwhile for video effects, 3D, and generative AI

Apple’s current M5 Max Mac Studio starts at **$2,499** with 36GB memory and 512GB storage; the 40-core GPU upgrade starts with 48GB memory and costs more. Apple lists M5 Max configurations up to 128GB unified memory. [1,2,3]

### What I would buy

| Component | Recommendation |

|---|---|

| Chip | M5 Max |

| CPU/GPU | 18-core CPU, 40-core GPU |

| Unified memory | 64GB if available; 48GB if keeping costs down |

| Storage | 1TB internal SSD |

| External storage | 2–4TB Thunderbolt SSD for active media |

| Display | Use your existing monitor initially |

**Budget-conscious option:** M5 Max, 32-core GPU, 36GB memory, 512GB SSD. It is adequate for general creative work and cloud-based AI, but I would avoid it if you expect to run local AI models regularly.

**Step up to 128GB** only if you specifically plan to run larger local language models, several demanding apps simultaneously, very large video projects, or heavy 3D scenes. The M5 Max supports up to 128GB, while the M5 Ultra goes much higher, but Ultra pricing is difficult to justify for ordinary creative work. [1,2]

One important limitation: Apple silicon Macs do not support NVIDIA CUDA, so choose a Windows/Linux workstation with an NVIDIA GPU instead if your AI workflow depends on CUDA-specific software or training large models locally. For Apple-optimized tools such as MLX, Core ML, image generation, video enhancement, and creative applications, the M5 Max is a strong practical choice. Apple reports major M5 Max gains over M4 Max in local-model prompt processing and AI image generation, though those are manufacturer test results. [2,3]

Read More

[1] Shop Mac Studio — https://www.apple.com/shop/buy-mac/mac-studio

[2] Mac Studio — https://www.apple.com/mac-studio/

[3] M5 Max Mac Studio vs. M4 Max Mac Studio: Specs, cost — https://appleinsider.com/articles/26/08/26/m5-max-mac-studio-vs-m4-max-mac-studio-faster-more-expensive


Tuesday, September 1, 2026

LLM Models Benchmark Results

 Below are the models l currently have under ollama, on my workstation which runs the AMD Ryzen 9 3900X 12-Core processor at a base speed of 3.8Ghz running at around 4.12Ghz with 64Gb DDR4 RAM and has an Nvidia RTX 3090 GPU with 24Gb VRAM.







Benchmark python code:









Results:











Run #2 results




Wednesday, August 5, 2026

Running Gemma 4 Locally - Notes 2

 Running Gemma 4 Locally - Gathered Notes 2


With 32GB of system RAM and an 8GB VRAM RTX 2060, the best overall options for your setup are gemma4:12b for a completely smooth, GPU-accelerated experience, or gemma4:26b if you want to push for max intelligence using partial RAM offloading.

Recommended Models for Your Specs

Model Tag

Download Size

Best For

Speed on Your Setup

gemma4:e4b

~9.6 GB

Ultimate speed & daily tasks

Fastest (~35+ tokens/sec) — Fits completely into your 8GB VRAM.

gemma4:12b

~7.6 GB

Best Balance (Coding & Logic)

Fast (~25 tokens/sec) — Squeezes almost entirely into VRAM with minimal RAM spillover.

gemma4:26b

~18 GB

High-tier reasoning & writing

Moderate (~8–12 tokens/sec) — Uses partial offloading to your 32GB system RAM.


Why These Models Fit Your Hardware

1. The Sweet Spot: gemma4:12b

This model size bridges the gap perfectly. A quantized 4-bit version only requires about 6.7 GB of memory. This leaves just enough room for your 8GB VRAM to handle the model weights, giving you high-speed performance along with advanced multimodal and coding abilities. 

ollama run gemma4:12b

2. Squeezing Out Max Quality: gemma4:26b

The 26B model uses a Mixture of Experts (MoE) design. It has the knowledge of a massive model but only activates 4 billion parameters per token. While its 18 GB footprint exceeds your 8GB VRAM, Ollama will automatically split the layers—putting ~6GB onto your GPU and offloading the rest to your 32GB system RAM. Because it only evaluates 4B parameters at a time, it generates text much faster than traditional large dense models. 

ollama run gemma4:26b

3. The Pure Speed Option: gemma4:e4b

If you need instant responses for simple tasks, lookups, or automation, the e4b (Effective 4B) variant fits entirely inside your VRAM with plenty of room left over for giant conversation histories (context windows). 

ollama run gemma4:e4b

With 64GB of system RAM and a 24GB VRAM RTX 3090, your system is a top-tier local AI workstation. You have enough memory to run the flagship gemma4:31b or gemma4:26b completely within your GPU's VRAM for lightning-fast, uncompromised performance.

Recommended Models for Your Specs

Model Tag

Download Size

Best For

Speed on Your Setup

gemma4:31b

~17.5 GB

Flagship Choice (Max Reasoning/Coding)

Blazing Fast (~50+ tokens/sec) — Fits 100% in VRAM with room for long context.

gemma4:26b

~14.4 GB

High-throughput agent workflows

Extremely Fast (~80+ tokens/sec) — Uses ultra-fast MoE sparse activation.

gemma4:12b

~7.6 GB

Low-latency audio & vision multitasking

Max Speed (~110+ tokens/sec) — Instantly processes huge context windows.


Why These Models Fit Your Hardware

1. The Flagship Experience: gemma4:31b

The 31B dense model is the workstation crown jewel of the Gemma 4 lineup. At its default 4-bit quantization, it only occupies 17.5 GB of memory. Because this fits entirely within your 24GB VRAM with ~6.5GB to spare, you avoid slow system RAM bottlenecks completely. This model handles deep architectural coding, advanced logic parsing, and multi-turn complex reasoning effortlessly.

ollama run gemma4:31b

2. High-Throughput Efficiency: gemma4:26b

The 26B model uses a sparse Mixture of Experts (MoE) design that only activates 4 billion parameters per token. The full model fits seamlessly inside your VRAM at 14.4 GB. Since it only computes a fraction of the parameters per token while retaining the full knowledge of a 26B model, it delivers a massive speed jump over dense models, routinely pushing 80+ tokens per second on a 3090. 

ollama run gemma4:26b

3. Extra Capabilities Note: gemma4:12b

While the 31B and 26B variants excel at raw intelligence, text reasoning, and image parsing, the 12b Unified model includes native end-to-end audio encoders. If you need a model that can natively listen to or process live speech alongside massive context windows (up to 256K tokens), keep the 12B variant in your toolkit. 

ollama run gemma4:12b


To handle both complex software development and heavy document analysis on your RTX 3090 (24GB VRAM) and 64GB RAM workstation, gemma4:31b is your best primary model, paired with gemma4:26b as a high-speed alternative.

When analyzing massive PDFs and coding full repositories, Context Window (KV Cache) memory exhaustion becomes your biggest bottleneck. If a model spills over your 24GB VRAM into system RAM, text generation speeds instantly tank. 

Hardware Breakdown for Context & Coding

Model

Size (Q4)

Safe Context Window on 24GB VRAM

Use Case Strength

gemma4:31b

~17.5 GB

Up to ~32K–64K tokens

Heavy logic, deep software architecture, code debugging.

gemma4:26b (MoE)

~14.4 GB

Up to 128K+ tokens

Gigantic multi-PDF analysis and high-speed iterative coding.


Strategic Recommendation Based on Your Tasks

1. For Deep Architecture & Complex Coding: gemma4:31b

The flagship 31B dense variant is the most intelligent model in the family. Its advanced reasoning excels at tracking complex code dependency trees, syntax generation, and logic structures. Because the model base takes up 17.5 GB, it leaves you about 6.5 GB of free VRAM. This remaining space comfortably holds a solid context history (plenty for medium-to-large code scripts) while maintaining maximum GPU execution speeds. 

ollama run gemma4:31b

2. For Massive PDFs & Repository Analysis: gemma4:26b

If you need to feed the AI entire source code repos or hundreds of pages of documentation at once, use the 26B Mixture of Experts (MoE) model. It only requires 14.4 GB of base VRAM, leaving nearly 10 GB of free VRAM purely for your context window. Since it only activates a portion of its network (4B parameters) per token, it reads through giant inputs at lightning speeds, allowing you to stretch deep into Gemma 4's massive native context limits without suffering hardware slowdowns. 

ollama run gemma4:26b

Pro-Tip for Document Processing

When running your documents through Ollama, make sure to adjust the environment variables or configuration files to maximize the num_ctx setting (e.g., set it to 32768 or 65536) to match the size of your files, ensuring the model reads the entire document rather than cutting it off.

Community Sentiment on the 26B MoE

Developers running this tier of hardware find the architectural trade-offs of the MoE variant highly effective:

“The 26B MoE variant is the best pick for most developers. It activates only 4B parameters per token, runs at near-4B speeds, but delivers quality close to a 13B model.”


Tuesday, July 28, 2026

Gemma 4

 Gemma 4 is a family of open-weight, multimodal models by Google DeepMind, available in pre-trained and instruction-tuned variants across five sizes (E2B, E4B, 12B, 26B A4B, and 31B) in both Dense and Mixture-of-Experts (MoE) architectures. Built to run on everything from mobile devices to high-end servers, Gemma 4 is optimized for reasoning, coding, agentic workflows, and text generation.

Key Features

Multimodal Inputs: Supports text and variable-resolution images across all sizes, plus native video and audio on E2B, E4B, and 12B. Output is text-only.

Expanded Context & Multilingual: Up to 256K context window (128K on smaller models) with support for over 140 languages.

Reasoning & Agents: Features configurable thinking modes, native function calling for autonomous agents, and native system prompt support.

Hybrid Attention: Interleaves local sliding window and full global attention with Proportional RoPE (p-RoPE) and unified Keys/Values to optimize memory and speed for long-context tasks.









Source:  https://huggingface.co/google/gemma-4-E4B-it


Building an LLM from scratch

 Great video from Syntax Youtube channel on building an LLM from scratch.


Here is a link to a copy/fork of the code he developed for this exercise.

Tuesday, July 21, 2026

Nvidia Jetson Orin

 Below are some great videos on using the Nvidia Jetson Orin

1)

NVIDIA Jetson Orin Nano: Secret to Running Ollama on the GPU on Jetpack 7.2, by Paul McWhorter of toptechboy.com



2)

How to Run YOLO 26 Detection Models on the NVIDIA Jetson Orin Nano, by Edje Electronics Channel




How to setup and use a local ollama server running on a windows 11 machine, from any machine in a local area network

 ## 1. Install Ollama on the Windows 11 host Download and install Ollama on the machine with the GPU/CPU you want to use. Pull a model and v...