Code l use most of the time:
python -m venv myenv
pip install matplotlib
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu130
pip install tensorflow
pip install tqdm
python.exe -m pip install --upgrade pip
Wednesday, October 7, 2026
Tuesday, September 15, 2026
How to setup and use a local ollama server running on a windows 11 machine, from any machine in a local area network
## 1. Install Ollama on the Windows 11 host
Download and install Ollama on the machine with the GPU/CPU you want to use. Pull a model and verify it locally:
```powershell
ollama pull llama3.2
ollama run llama3.2
```
Ollama normally listens only on `127.0.0.1:11434`. To permit LAN clients, configure it to listen on all interfaces with `OLLAMA_HOST=0.0.0.0:11434`. [1,2]
## 2. Configure `OLLAMA_HOST`
On the Windows host:
1. Open **Settings → System → About → Advanced system settings**.
2. Select **Environment Variables**.
3. Under **User variables**, select **New**.
4. Enter:
- **Variable name:** `OLLAMA_HOST`
- **Variable value:** `0.0.0.0:11434`
5. Click **OK** through all dialogs.
6. Exit Ollama completely from the system tray, then relaunch it.
If Ollama runs as a service rather than as the desktop application, restart that service after changing the variable.
`OLLAMA_ORIGINS` is only relevant to browser-based cross-origin requests; it does not make the server reachable over the network. Avoid setting it to `*` unless you specifically need browser access from arbitrary origins.
## 3. Find the Windows host’s LAN IP address
On the Windows machine, run:
```powershell
ipconfig
```
Find the **IPv4 Address** for the active Wi-Fi or Ethernet adapter. It will usually look like:
```text
192.168.1.50
```
Use this address from other machines—not `localhost`, `127.0.0.1`, or `0.0.0.0`.
It is useful to reserve this address in your router’s DHCP settings so it does not change.
## 4. Allow TCP port 11434 through Windows Firewall
Open **PowerShell as Administrator** and create a rule limited to private networks:
```powershell
New-NetFirewallRule `
-DisplayName "Ollama LAN API" `
-Direction Inbound `
-Protocol TCP `
-LocalPort 11434 `
-Action Allow `
-Profile Private
```
Make sure the network is classified as **Private**, not Public:
**Settings → Network & internet → Wi-Fi/Ethernet → your network → Network profile → Private**
Do not expose port 11434 through your router or enable WAN/Internet port forwarding. Ollama’s local API generally should not be placed directly on the public Internet.
## 5. Test the server locally
On the Windows host:
```powershell
curl http://127.0.0.1:11434/api/tags
```
You should receive JSON listing installed models.
You can also test using the host’s LAN address:
```powershell
curl http://192.168.1.50:11434/api/tags
```
Replace `192.168.1.50` with the actual address.
## 6. Test from another LAN computer
From another machine on the same network:
```bash
curl http://192.168.1.50:11434/api/tags
```
A successful response confirms that the remote machine can reach Ollama and see its models. Both machines must be on the same reachable LAN; guest Wi-Fi and client-isolation settings can block this connection.
## 7. Call the model remotely
Example using the Ollama REST API:
```bash
curl http://192.168.1.50:11434/api/generate \
-H "Content-Type: application/json" \
-d '{
"model": "llama3.2",
"prompt": "Explain photosynthesis in two sentences.",
"stream": false
}'
```
For chat requests:
```bash
curl http://192.168.1.50:11434/api/chat \
-H "Content-Type: application/json" \
-d '{
"model": "llama3.2",
"messages": [
{
"role": "user",
"content": "Write a short welcome message."
}
],
"stream": false
}'
```
## 8. Use it from applications
Many Ollama-compatible applications have a **custom endpoint**, **base URL**, or **Ollama host** setting. Enter:
```text
http://192.168.1.50:11434
```
For OpenAI-compatible applications, the base URL is often:
```text
http://192.168.1.50:11434/v1
```
The exact setting depends on the application. In Python, an Ollama-compatible client commonly uses:
```python
from ollama import Client
client = Client(host="http://192.168.1.50:11434")
response = client.chat(
model="llama3.2",
messages=[
{"role": "user", "content": "Give me three project ideas."}
],
)
print(response["message"]["content"])
```
## Troubleshooting
- **Local test works, remote test fails:** check the Windows Firewall rule and confirm the network is set to Private.
- **`curl` connects to `127.0.0.1` but not the LAN IP:** Ollama did not restart after setting `OLLAMA_HOST`, or it is still bound only to loopback.
- **Check what is listening:**
```powershell
netstat -ano | findstr :11434
```
You want to see `0.0.0.0:11434` or the machine’s LAN address, rather than only `127.0.0.1:11434`.
- **Connection times out:** check that both devices are on the same subnet and that Wi-Fi client isolation is disabled.
- **Model not found:** pull the model on the Windows host:
```powershell
ollama pull llama3.2
```
- **Using WSL2 or Docker:** networking may require additional port forwarding because WSL2 uses a virtualized network adapter; a normal Windows installation is simpler for LAN access. [1,3]
The essential configuration is:
```text
OLLAMA_HOST=0.0.0.0:11434
```
Then connect from other machines using:
```text
http://<Windows-LAN-IP>:11434
```
Read More
[1] [WSL 2] Exposing ollama via 0.0.0.0 on local network #1431 - GitHub — https://github.com/ollama/ollama/issues/1431
[2] Impossible to connect to ollama locally from another pc #6571 - GitHub — https://github.com/ollama/ollama/issues/6571
[3] [WSL2] Ollama on Windows host unreachable from Docker ... - GitHub — https://github.com/ollama/ollama/issues/14947
Most affordable and practical new apple max studio for AI and creative work recommendation
Assuming U.S. pricing, buy the **new Mac Studio with M5 Max, 18-core CPU, 40-core GPU, 64GB unified memory, and 1TB SSD**.
It is the most practical balance for AI-assisted creative work:
- **AI:** local image generation, transcription, coding assistants, and medium-sized local language models
- **Creative work:** 4K/8K video editing, Photoshop, Lightroom, audio production, motion graphics, and many professional apps
- **Memory:** 64GB gives substantially more headroom than the base 36GB configuration
- **Storage:** 1TB is usable; keep large media libraries and project archives on a fast external Thunderbolt SSD
- **GPU:** the 40-core option is worthwhile for video effects, 3D, and generative AI
Apple’s current M5 Max Mac Studio starts at **$2,499** with 36GB memory and 512GB storage; the 40-core GPU upgrade starts with 48GB memory and costs more. Apple lists M5 Max configurations up to 128GB unified memory. [1,2,3]
### What I would buy
| Component | Recommendation |
|---|---|
| Chip | M5 Max |
| CPU/GPU | 18-core CPU, 40-core GPU |
| Unified memory | 64GB if available; 48GB if keeping costs down |
| Storage | 1TB internal SSD |
| External storage | 2–4TB Thunderbolt SSD for active media |
| Display | Use your existing monitor initially |
**Budget-conscious option:** M5 Max, 32-core GPU, 36GB memory, 512GB SSD. It is adequate for general creative work and cloud-based AI, but I would avoid it if you expect to run local AI models regularly.
**Step up to 128GB** only if you specifically plan to run larger local language models, several demanding apps simultaneously, very large video projects, or heavy 3D scenes. The M5 Max supports up to 128GB, while the M5 Ultra goes much higher, but Ultra pricing is difficult to justify for ordinary creative work. [1,2]
One important limitation: Apple silicon Macs do not support NVIDIA CUDA, so choose a Windows/Linux workstation with an NVIDIA GPU instead if your AI workflow depends on CUDA-specific software or training large models locally. For Apple-optimized tools such as MLX, Core ML, image generation, video enhancement, and creative applications, the M5 Max is a strong practical choice. Apple reports major M5 Max gains over M4 Max in local-model prompt processing and AI image generation, though those are manufacturer test results. [2,3]
Read More
[1] Shop Mac Studio — https://www.apple.com/shop/buy-mac/mac-studio
[2] Mac Studio — https://www.apple.com/mac-studio/
[3] M5 Max Mac Studio vs. M4 Max Mac Studio: Specs, cost — https://appleinsider.com/articles/26/08/26/m5-max-mac-studio-vs-m4-max-mac-studio-faster-more-expensive
Tuesday, September 1, 2026
LLM Models Benchmark Results
Below are the models l currently have under ollama, on my workstation which runs the AMD Ryzen 9 3900X 12-Core processor at a base speed of 3.8Ghz running at around 4.12Ghz with 64Gb DDR4 RAM and has an Nvidia RTX 3090 GPU with 24Gb VRAM.
Benchmark python code:
Results:
Run #2 results
Wednesday, August 5, 2026
Running Gemma 4 Locally - Notes 2
Running Gemma 4 Locally - Gathered Notes 2
With 32GB of system RAM and an 8GB VRAM RTX 2060, the best
overall options for your setup are gemma4:12b for a completely smooth,
GPU-accelerated experience, or gemma4:26b if you want to push for max
intelligence using partial RAM offloading.
Recommended Models for Your Specs
|
Model Tag |
Download Size |
Best For |
Speed on Your Setup |
|
gemma4:e4b |
~9.6 GB |
Ultimate speed & daily tasks |
Fastest (~35+ tokens/sec) — Fits completely into
your 8GB VRAM. |
|
gemma4:12b |
~7.6 GB |
Best Balance (Coding & Logic) |
Fast (~25 tokens/sec) — Squeezes almost entirely
into VRAM with minimal RAM spillover. |
|
gemma4:26b |
~18 GB |
High-tier reasoning & writing |
Moderate (~8–12 tokens/sec) — Uses partial
offloading to your 32GB system RAM. |
Why These Models Fit Your Hardware
1. The Sweet Spot: gemma4:12b
This model size bridges the gap perfectly. A quantized 4-bit
version only requires about 6.7 GB of memory. This leaves just enough room for
your 8GB VRAM to handle the model weights, giving you high-speed performance
along with advanced multimodal and coding abilities.
ollama run gemma4:12b
2. Squeezing Out Max Quality: gemma4:26b
The 26B model uses a Mixture of Experts (MoE) design. It has
the knowledge of a massive model but only activates 4 billion parameters per
token. While its 18 GB footprint exceeds your 8GB VRAM, Ollama will
automatically split the layers—putting ~6GB onto your GPU and offloading the
rest to your 32GB system RAM. Because it only evaluates 4B parameters at a
time, it generates text much faster than traditional large dense models.
ollama run gemma4:26b
3. The Pure Speed Option: gemma4:e4b
If you need instant responses for simple tasks, lookups, or
automation, the e4b (Effective 4B) variant fits entirely inside your VRAM with
plenty of room left over for giant conversation histories (context windows).
ollama run gemma4:e4b
With 64GB of system RAM and a 24GB VRAM RTX 3090, your
system is a top-tier local AI workstation. You have enough memory to run the
flagship gemma4:31b or gemma4:26b completely within your GPU's
VRAM for lightning-fast, uncompromised performance.
Recommended Models for Your Specs
|
Model Tag |
Download Size |
Best For |
Speed on Your Setup |
|
gemma4:31b |
~17.5 GB |
Flagship Choice (Max Reasoning/Coding) |
Blazing Fast (~50+ tokens/sec) — Fits 100% in VRAM
with room for long context. |
|
gemma4:26b |
~14.4 GB |
High-throughput agent workflows |
Extremely Fast (~80+ tokens/sec) — Uses ultra-fast
MoE sparse activation. |
|
gemma4:12b |
~7.6 GB |
Low-latency audio & vision multitasking |
Max Speed (~110+ tokens/sec) — Instantly processes
huge context windows. |
Why These Models Fit Your Hardware
1. The Flagship Experience: gemma4:31b
The 31B dense model is the workstation crown jewel of the
Gemma 4 lineup. At its default 4-bit quantization, it only occupies 17.5 GB
of memory. Because this fits entirely within your 24GB VRAM with ~6.5GB to
spare, you avoid slow system RAM bottlenecks completely. This model handles
deep architectural coding, advanced logic parsing, and multi-turn complex
reasoning effortlessly.
ollama run gemma4:31b
2. High-Throughput Efficiency: gemma4:26b
The 26B model uses a sparse Mixture of Experts (MoE) design
that only activates 4 billion parameters per token. The full model fits
seamlessly inside your VRAM at 14.4 GB. Since it only computes a
fraction of the parameters per token while retaining the full knowledge of a
26B model, it delivers a massive speed jump over dense models, routinely
pushing 80+ tokens per second on a 3090.
ollama run gemma4:26b
3. Extra Capabilities Note: gemma4:12b
While the 31B and 26B variants excel at raw intelligence,
text reasoning, and image parsing, the 12b Unified model includes native
end-to-end audio encoders. If you need a model that can natively listen to or
process live speech alongside massive context windows (up to 256K tokens), keep
the 12B variant in your toolkit.
ollama run gemma4:12b
To handle both complex software development and heavy
document analysis on your RTX 3090 (24GB VRAM) and 64GB RAM workstation, gemma4:31b
is your best primary model, paired with gemma4:26b as a high-speed
alternative.
When analyzing massive PDFs and coding full repositories, Context
Window (KV Cache) memory exhaustion becomes your biggest bottleneck. If a
model spills over your 24GB VRAM into system RAM, text generation speeds
instantly tank.
Hardware Breakdown for Context & Coding
|
Model |
Size (Q4) |
Safe Context Window on 24GB VRAM |
Use Case Strength |
|
gemma4:31b |
~17.5 GB |
Up to ~32K–64K tokens |
Heavy logic, deep software architecture, code debugging. |
|
gemma4:26b (MoE) |
~14.4 GB |
Up to 128K+ tokens |
Gigantic multi-PDF analysis and high-speed iterative
coding. |
Strategic Recommendation Based on Your Tasks
1. For Deep Architecture & Complex Coding: gemma4:31b
The flagship 31B dense variant is the most intelligent model
in the family. Its advanced reasoning excels at tracking complex code
dependency trees, syntax generation, and logic structures. Because the model
base takes up 17.5 GB, it leaves you about 6.5 GB of free VRAM. This remaining
space comfortably holds a solid context history (plenty for medium-to-large
code scripts) while maintaining maximum GPU execution speeds.
ollama run gemma4:31b
2. For Massive PDFs & Repository Analysis: gemma4:26b
If you need to feed the AI entire source code repos or
hundreds of pages of documentation at once, use the 26B Mixture of Experts
(MoE) model. It only requires 14.4 GB of base VRAM, leaving nearly 10 GB of
free VRAM purely for your context window. Since it only activates a portion
of its network (4B parameters) per token, it reads through giant inputs at
lightning speeds, allowing you to stretch deep into Gemma 4's massive native
context limits without suffering hardware slowdowns.
ollama run gemma4:26b
Pro-Tip for Document Processing
When running your documents through Ollama, make sure to
adjust the environment variables or configuration files to maximize the num_ctx
setting (e.g., set it to 32768 or 65536) to match the size of your files,
ensuring the model reads the entire document rather than cutting it off.
Community Sentiment on the 26B MoE
Developers running this tier of hardware find the
architectural trade-offs of the MoE variant highly effective:
“The 26B MoE variant is the best pick for most developers.
It activates only 4B parameters per token, runs at near-4B speeds, but delivers
quality close to a 13B model.”
Tuesday, July 28, 2026
Gemma 4
Gemma 4 is a family of open-weight, multimodal models by Google DeepMind, available in pre-trained and instruction-tuned variants across five sizes (E2B, E4B, 12B, 26B A4B, and 31B) in both Dense and Mixture-of-Experts (MoE) architectures. Built to run on everything from mobile devices to high-end servers, Gemma 4 is optimized for reasoning, coding, agentic workflows, and text generation.
Key Features
Multimodal Inputs: Supports text and variable-resolution images across all sizes, plus native video and audio on E2B, E4B, and 12B. Output is text-only.
Expanded Context & Multilingual: Up to 256K context window (128K on smaller models) with support for over 140 languages.
Reasoning & Agents: Features configurable thinking modes, native function calling for autonomous agents, and native system prompt support.
Hybrid Attention: Interleaves local sliding window and full global attention with Proportional RoPE (p-RoPE) and unified Keys/Values to optimize memory and speed for long-context tasks.
Source: https://huggingface.co/google/gemma-4-E4B-it
Building an LLM from scratch
Great video from Syntax Youtube channel on building an LLM from scratch.
Here is a link to a copy/fork of the code he developed for this exercise.
Subscribe to:
Posts (Atom)
-
Welcome to Moments In Digital Ink! This blog is a space where everyday experiences and reflections come to life through the power of words....
-
From my tests as shown in the screenshots below, the more realistic images generated by AI are all dependent on how detailed and clear the ...
