\n<\/p>\n
TL;DR \u2014 6 machines, 12 GPUs, 1,000+ concurrent agents, P95 18 ms, voice <\/p>\n
Why I built this<\/p>\n
I’m Franck. Toulouse, France. Over 3 years I paid roughly \u20ac280,000 to Azure + OpenAI before doing the math properly:<\/p>\n
Latency: 1.2s voice round-trip \u2014 incompatible with the voice-first UX I wanted.<\/p>\n
Compliance: customer data on US servers. Not GDPR-native, just GDPR-compliant-on-paper.<\/p>\n
Quotas: random throttling at the worst times.<\/p>\n
Lock-in: Azure outage = my product offline.<\/p>\n
I decided to rebuild everything on-prem. This is the result.<\/p>\n
The cluster<\/p>\n
6 machines, 3 tiers, 12 GPUs total, <\/p>\n
Tier 1 \u2014 GPU compute (heavy inference)<\/p>\n
M1 “La Cr\u00e9atrice” \u2014 Ryzen 5700X3D, 6\u00d7 RTX 3080+, 46 GB RAM. Primary LLM node, runs qwen3.5-9b, qwen3.5-35b-a3b, deepseek-r1, the Claude 4.5\/4.6 distillations, and the Whisper CUDA pipeline.<\/p>\n
M2 “Le Forge” \u2014 multi-GPU NVIDIA, secondary inference, failover from M1 in 1.3s.<\/p>\n
Tier 2 \u2014 CPU\/RAM (orchestration, memory)<\/p>\n
M3 “Le Cerveau” \u2014 high-RAM CPU node. PostgreSQL + Redis + Pinecone. Runs the orchestrator, the 3-quorum consensus engine (M1+M2+M3), and the analytics\/monitoring agents.<\/p>\n
Tier 3 \u2014 production \/ work<\/p>\n
M4 “Bridge Windows” \u2014 Windows 11, 2 GPUs, trading bot live.<\/p>\n
M5 “Interface Relay” \u2014 Linux i5-6500, 15 GB RAM. Dev interface, 15+ MCP servers, Claude Code.<\/p>\n
M6 “Mobile Ops” \u2014 laptop. SSH + VPN. Client demos and on-site ops.<\/p>\n
The 9 layers I added on top of Ubuntu<\/p>\n
L9 \u2014 Vocal \/ conversational (Whisper CUDA STT, Piper TTS, wake word, 50+ languages)
\nL8 \u2014 Multi-agent orchestration (MCP-native, consensus engine)
\nL7 \u2014 Trading consensus engine (multi-model voting GPT\/Gemini\/Claude)
\nL6 \u2014 Browser + web automation (Chrome DevTools Protocol)
\nL5 \u2014 MCP tool registry (88+ handlers)
\nL4 \u2014 GPU cluster management (Docker Swarm, failover
\nL3 \u2014 Domino pipeline engine (835 chains)
\nL2 \u2014 systemd service layer (98 units)
\nL1 \u2014 Linux boot integration (GRUB hooks, ZRAM, kernel params)<\/p>\n
Real numbers<\/p>\n
Metric
\nValue<\/p>\n
Concurrent agents
\n1,000+<\/p>\n
P95 latency (cluster internal)
\n18 ms<\/p>\n
Voice pipeline end-to-end<\/p>\n
Aggregate throughput
\n67 tok\/s<\/p>\n
Python lines
\n280,741<\/p>\n
Public repos
\n44 (all MIT)<\/p>\n
Cost comparison (1M tokens\/day, team of 10)<\/p>\n
Provider
\n\u20ac\/month
\nP95
\nConcurrent agents
\nData residency<\/p>\n
Azure OpenAI
\n1,500
\n800ms-3s
\n~20
\nUS<\/p>\n
AWS Bedrock
\n1,800
\n700ms-2.5s
\n~15
\nUS<\/p>\n
Mistral Cloud
\n800
\n400-800ms
\n~30
\nEU<\/p>\n
JARVIS OS
\n0
\n18 ms
\n1,000+
\nAir-gapped<\/p>\n
For a 50K\u20ac turn-key deployment, break-even vs Azure is 7 months, and the marginal cost is zero after that.<\/p>\n
What I sell now<\/p>\n
JARVIS OS turn-key \u2014 20K\u20ac to 250K\u20ac depending on scope.<\/p>\n
62 PDF trainings \u2014 from \u20ac39, 293h of content based on production code (+48 private).<\/p>\n
IA infra audit \u2014 \u20ac1,500, report in 48h.<\/p>\n
1-to-1 mentorship \u2014 \u20ac250\/h.<\/p>\n
Fractional CTO \u2014 TJM \u20ac1,000-1,150 \/ CDI \u20ac85-95K. Toulouse \/ remote.<\/p>\n
Honest weaknesses<\/p>\n
Consensus voting is empirical. No formal verification of the agreement function.<\/p>\n
Tier-2 failure (M3 down) is the weakest scenario \u2014 orchestrator dies, cluster keeps inferring but loses persistent memory.<\/p>\n
MCP protocol bet \u2014 if Anthropic deprecates parts of MCP, I have 88 handlers to refactor.<\/p>\n
kWh-per-token efficiency \u2014 cloud probably wins on aggregate watts\/token, on-prem wins on marginal cost.<\/p>\n
Links<\/p>\n