\n
\nGive text-only LLMs vision through a tiny OpenAI-compatible proxy.
\nVisionBridge sits between your chat UI and your models. It lets a reasoning model
\nfrom LM Studio, Ollama, vLLM, Z.ai, OpenRouter, or any OpenAI-compatible backend
\ninspect images by asking a separate vision model targeted questions.
\nNo training. No weights. Bring your own reasoning model and your own vision model.<\/p>\n
OpenWebUI \/ LibreChat \/ app
\n |
\n v
\nVisionBridge at \/v1\/chat\/completions
\n |
\n +– reasoning model: LM Studio \/ Ollama \/ vLLM \/ GLM \/ DeepSeek
\n |
\n +– vision model: LLaVA \/ Qwen-VL \/ GLM-V \/ GPT-4o-mini \/ any VLM<\/p>\n
VisionBridge receives an OpenAI-style chat request. If the request contains
\nimages, it stores the images, rewrites the prompt for the reasoning model, and
\nlets the reasoning model call tools:<\/p>\n
look(image_id, question) for targeted visual inspection
\nocr(image_id) for text extraction
\nscan(image_id) for a tiled full-document text sweep of dense pages
\ncrop_and_look(image_id, box, question) for zooming into a region
\n(small crops are automatically upscaled before the VLM sees them)
\ncompare(image_id_a, image_id_b, question) for judging two images together<\/p>\n
The vision model does the seeing. The reasoning model does the thinking.
\nBefore the loop starts, every image gets a one-shot scene description from
\nthe vision model (in parallel, cached by content), so the reasoning model
\nbegins with global context and spends its tool budget on details. Multiple
\ntool calls in one turn run concurrently, and repeated images across a
\nconversation are served from a content-addressed cache instead of being
\nre-fetched and re-decoded.
\nTool calling uses the backend’s native OpenAI function calling when
\navailable. In the default TOOL_MODE=auto, VisionBridge probes the reasoning
\nbackend once and transparently falls back to a prompt-JSON protocol for
\nbackends and models that don’t support tools \u2014 so it works with strong hosted
\nmodels and weak local ones alike.
\nThe loop is built to survive imperfect models: hallucinated image ids, bad
\ncrop boxes, malformed JSON, and vision-backend hiccups are fed back to the
\nreasoning model as observations so it can correct itself, and hitting the
\ninspection limit triggers a final best-effort answer instead of an error.<\/p>\n
docker run -p 8080:8080 \\
\n -e REASONING_BASE_URL=http:\/\/host.docker.internal:1234\/v1 \\
\n -e REASONING_API_KEY=lm-studio \\
\n -e REASONING_MODEL=local-model \\
\n -e VISION_BASE_URL=http:\/\/host.docker.internal:11434\/v1 \\
\n -e VISION_API_KEY=ollama \\
\n -e VISION_MODEL=llava:13b \\
\n ghcr.io\/eekosystems\/visionbridge:latest
\nThen point your OpenAI-compatible client at:
\nBase URL: http:\/\/localhost:8080\/v1
\nAPI key: anything
\nModel: visionbridge<\/p>\n
cd visionbridge
\npython -m venv .venv
\nsource .venv\/bin\/activate # Windows: .venv\\Scripts\\activate
\npip install -e “.(dev)”
\ncp .env.example .env # Windows: copy .env.example .env
\nvisionbridge serve –host 0.0.0.0 –port 8080
\nCheck your backend configuration any time with:<\/p>\n
It verifies both backends are reachable, that the configured models exist, and
\nthat a test completion succeeds.<\/p>\n
All settings come from environment variables (or .env):<\/p>\n
Variable
\nDefault
\nPurpose<\/p>\n
REASONING_BASE_URL
\nhttp:\/\/localhost:1234\/v1
\nReasoning backend<\/p>\n
REASONING_API_KEY
\nlocal
\nReasoning API key<\/p>\n
REASONING_MODEL
\nlocal-model
\nReasoning model id<\/p>\n
VISION_BASE_URL
\nhttp:\/\/localhost:11434\/v1
\nVision backend<\/p>\n
VISION_API_KEY
\nlocal
\nVision API key<\/p>\n
VISION_MODEL
\nllava:13b
\nVision model id<\/p>\n
BRIDGE_MODEL_NAME
\nvisionbridge
\nModel name the proxy advertises<\/p>\n
TOOL_MODE
\nauto
\nauto, native, or prompt tool calling<\/p>\n
MAX_VISION_CALLS
\n8
\nReasoning turns before a forced final answer<\/p>\n
REQUEST_TIMEOUT_SECONDS
\n120
\nPer-backend-call timeout<\/p>\n
BACKEND_MAX_RETRIES
\n2
\nRetries on 429\/5xx\/transport errors<\/p>\n
SCENE_PRIMING
\ntrue
\nDescribe each image once before the loop starts<\/p>\n
ALLOW_FILE_URLS
\nfalse
\nAllow file:\/\/ image URLs<\/p>\n
ALLOW_PRIVATE_URLS
\nfalse
\nAllow image fetches from private\/internal hosts<\/p>\n
MAX_IMAGE_BYTES
\n20971520
\nMax bytes per image<\/p>\n
MAX_IMAGES_PER_REQUEST
\n16
\nMax images per request<\/p>\n
IMAGE_CACHE_ENTRIES
\n64
\nContent-addressed image cache size<\/p>\n
ENABLE_TRACES
\ntrue
\nKeep tool-loop traces for \/v1\/traces\/{id}<\/p>\n
BRIDGE_API_KEYS
\n(empty)
\nComma-separated keys; when set, \/v1\/* requires one<\/p>\n
EXTRA_MODELS
\n(empty)
\nJSON map of extra model names to backend pairs<\/p>\n
LOG_JSON
\nfalse
\nEmit JSON log lines<\/p>\n
Serving multiple model pairs
\nOne instance can serve several reasoning+vision pairs under different model
\nnames. Unspecified fields inherit from the default pair:
\nEXTRA_MODELS={“visionbridge-fast”: {“reasoning”: {“model”: “qwen3:8b”}, “vision”: {“model”: “llava:7b”}}}
\n(or point EXTRA_MODELS_FILE at a JSON file). All names appear in
\nGET \/v1\/models, and clients select a pair with the model field.<\/p>\n
Open LM Studio.
\nLoad your reasoning model.
\nGo to the Developer tab and start the local server.
\nUse the default base URL:<\/p>\n
REASONING_BASE_URL=http:\/\/localhost:1234\/v1
\nREASONING_API_KEY=lm-studio
\nREASONING_MODEL=local-model
\nLM Studio can also be used as the vision backend if you load a vision-capable
\nmodel and expose it through the same OpenAI-compatible server.<\/p>\n
REASONING_BASE_URL=http:\/\/localhost:11434\/v1
\nREASONING_API_KEY=ollama
\nREASONING_MODEL=qwen3:32b<\/p>\n
VISION_BASE_URL=http:\/\/localhost:11434\/v1
\nVISION_API_KEY=ollama
\nVISION_MODEL=llava:13b<\/p>\n
REASONING_BASE_URL=http:\/\/localhost:8000\/v1
\nREASONING_API_KEY=token-abc123
\nREASONING_MODEL=Qwen\/Qwen3-32B<\/p>\n
VISION_BASE_URL=http:\/\/localhost:8001\/v1
\nVISION_API_KEY=token-abc123
\nVISION_MODEL=Qwen\/Qwen3-VL-8B-Instruct<\/p>\n
In OpenWebUI:
\nAdmin Panel -> Settings -> Connections -> OpenAI-compatible
\nBase URL: http:\/\/host.docker.internal:8080\/v1
\nAPI Key: anything
\nModel: visionbridge<\/p>\n
VisionBridge intentionally exposes the small subset most clients need:<\/p>\n
GET \/health (add ?probe=true to verify both backends are reachable)
\nGET \/v1\/models
\nPOST \/v1\/chat\/completions
\nGET \/v1\/traces\/{completion_id} \u2014 the full tool-loop trace for a recent
\nrequest: every tool call, its arguments, the vision model’s observation, and
\ntimings. Invaluable for answering “why did it say that?”<\/p>\n
Responses include a usage block aggregated across every reasoning and vision
\ncall made for the request, and an X-Request-ID header for correlating logs.
\nStreaming: image-free requests are streamed token-by-token straight from
\nthe reasoning backend. Requests with images stream too: SSE heartbeats keep
\nthe connection alive while tools run, and in native tool-calling mode the
\nfinal answer is forwarded token-by-token as the reasoning model produces it
\n(prompt-JSON mode buffers the loop and streams the answer in chunks).
\nErrors: backend failures return HTTP 502, invalid inputs 400.
\nAuth: set BRIDGE_API_KEYS=key1,key2 to require a Bearer key on \/v1\/*
\nendpoints (\/health stays open). For anything internet-facing, still prefer a
\nreverse proxy with TLS in front.<\/p>\n
Image URL fetching is hardened by default: hosts resolving to private,
\nloopback, or link-local addresses are rejected (SSRF guard), redirects are
\nvalidated hop-by-hop, downloads are size-capped, and payloads must decode as
\nreal images. Run VisionBridge behind your own reverse proxy \/ auth layer if
\nyou expose it beyond localhost \u2014 it deliberately ships without built-in auth.<\/p>\n
benchmarks\/ contains a harness that scores VisionBridge against a direct
\nVLM (relaxed accuracy + ANLS) on DocVQA-style JSONL task sets. See
\nbenchmarks\/README.md.<\/p>\n
Actively developed. See CHANGELOG.md for what’s new in 0.2.0.
\nNext up:<\/p>\n
published benchmark numbers against caption-only prompting
\npersistent on-disk image\/primer cache
\nOpenWebUI and LibreChat screenshots<\/p>\n
Contributions welcome \u2014 see CONTRIBUTING.md.<\/p>\n