\n
\n Gemma 4 Challenge: Build With Gemma 4 Submission<\/p>\n
This is a submission for the Gemma 4 Challenge: Build with Gemma 4\n<\/code><\/pre>\nWhat I Built<\/p>\n
GemmaOrch is a skill-based AI agent orchestrator: you define what an agent knows by dropping Markdown files into a folder, assign those skills to a named agent, and chat with it. The agent powered by Gemma 4 will only answer within the boundaries of those files \u2014 it refuses anything outside scope with a precise phrase, never hallucinates expertise it wasn’t given.<\/p>\n
The core idea: agent behavior lives in .md files, not in code. No prompts hardcoded in the application. No domain logic baked into the service layer. The skill files arethe agent.<\/p>\n
What it solves: building specialized AI assistants usually means either fine-tuning a model (expensive, slow to iterate) or writing complex prompt engineering into your codebase (brittle, hard to maintain). GemmaOrch separates the two concerns \u2014 the orchestration logic stays in Java, the expertise lives in plain Markdown that anyone can read and edit.<\/p>\n
Key features:<\/p>\n
Skill-driven agents \u2014 each agent’s system prompt is built entirely from its assigned skill files at runtime.
\nGitHub skill importer \u2014 paste a public GitHub folder URL and GemmaOrch fetches every .md file recursively, creating the skill locally.
\nStreaming chat \u2014 token-by-token streaming via Spring WebFlux, rendered as Markdown client-side.
\nMCP server \u2014 every agent is automatically exposed as a JSON-RPC 2.0 tool on POST \/mcp, callable from Claude Code, Cursor, or any MCP-compatible IDE.
\nREST API \u2014 POST \/api\/chat\/{agentId} for integrating agents into external services, with a one-click “Copy curl” button in the UI.
\nZero infrastructure \u2014 H2 file-based database, no external services required beyond the AI Studio API key.<\/p>\n
Built with: Java 25 \u00b7 Spring Boot 3.5 \u00b7 Spring AI 1.1.5 \u00b7 Thymeleaf \u00b7 HTMX 2.0<\/p>\n
Demo<\/p>\n
The app runs locally \u2014 see the Quick Start in the repo or the Docker section to spin it up in two commands.<\/p>\n
Dashboard \u2014 3-panel layout (skills \u00b7 main \u00b7 agents):<\/p>\n
Creating and importing skills:<\/p>\n
Chatting with a skill-scoped agent:<\/p>\n
API Access panel \u2014 copy the curl command directly from the agent detail view:<\/p>\n
Agents as MCP tools in the IDE:<\/p>\n
Code<\/p>\n
Repository: Bzaid94\/gemmorch-agents<\/p>\n
How I Used Gemma 4<\/p>\n
I used the gemma-4-31b-it model \u2014 the 31B dense instruction-tuned variant \u2014 via Google AI Studio through Spring AI’s spring-ai-starter-model-google-genai.<\/p>\n
Why the 31B dense, specifically:<\/p>\n
The project enforces a hard constraint: agents must refuse anything outside their assigned skills and must do so with an exact phrase. This is a correctness requirement, not a quality preference \u2014 if the constraint breaks, the product doesn’t work.<\/p>\n
I tested smaller variants first. The 4B model followed the constraint most of the time, but would occasionally drift: offering “related” information outside its skills, or partially revealing the system prompt when directly asked. With the 31B dense, these failures essentially disappeared. The constraint held reliably across multi-turn conversations and adversarial inputs.<\/p>\n
Two specific things the 31B unlocked that smaller models couldn’t deliver consistently:<\/p>\n
Long-context constraint adherence. A single agent’s system prompt can carry 10,000+ tokens of skill content (multiple skill files, each with reference documents). The 31B model kept the opening STRICT CONSTRAINTS block in effect even with extensive context following it \u2014 smaller models would silently “forget” early instructions as contextgrew.
\nRole disambiguation. Many skill files written for Claude Code or agentic CLI tools contain dispatch instructions like “invoke subagent X” or “request tool Y.” Injected directly into a system prompt, smaller models would sometimes output those templates literally. The 31B correctly understood the meta-instruction \u2014 “you are the agent being invoked, not the orchestrator invoking agents” \u2014 and applied the skill knowledge directly instead of outputting workflow templates.<\/p>\n
Why not the 26B MoE? The MoE variant optimizes for throughput across concurrent requests. GemmaOrch is a single-tenant orchestrator where precision per response matters morethan requests-per-second. The dense model’s full parameter activation per token is worth the inference cost for this use case.<\/p>\n
Why not the 4B? For a general assistant or creative tool, the 4B is genuinely capable and would be my first choice to keep costs and latency low. But when “breaking the constraint” is a correctness failure \u2014 not just a quality degradation \u2014 the extra capacity of the 31B is justified.<\/p>\n
The open-weights advantage: Gemma 4 is open. The application is architected so the model is an environment variable \u2014 swap AI Studio for a local Ollama instance and nothing else changes. For users with sensitive skill content (internal knowledge bases, proprietary processes), self-hosting is a real deployment path, not a future promise.<\/p>\n
Switch from AI Studio to self-hosted in one line:<\/p>\n
spring.ai.google.genai.chat.options.model=gemma-4-31b-it<\/p>\n
Or run locally with Ollama:<\/p>\n
ollama run gemma4:31b<\/p>\n
Source: https:\/\/github.com\/Bzaid94\/gemma-agents-orchestrator.git \u00b7 License: Apache 2.0<\/p>\n