Readable internals

Use the app first. Inspect the workflow when you are curious.

Token Optimizer is meant to feel simple up front and transparent underneath. This page explains the source layout, agent responsibilities, adaptive routes, typed contracts, generated files, and contribution path.

Back to workspace GitHub repo Explore code map

Architecture in plain English

The user sees one simple workspace. Behind it, a worker preflights the prompt, a system runner owns job state, and the optimizer core chooses the leanest contract workflow.

UIWorkspace

Collects the messy prompt once, shows live status, and keeps final output collapsed until requested.

WorkerBackground preflight

Estimates tokens, finds constraints, builds a local contract preview, and recommends a route while the user types.

SystemRun manager

Creates run IDs, tracks stages, queues local work, and returns stable snapshots for the live status UI.

CoreAdaptive workflow

Runs intake, route selection, contract building, provider adapters, execution, verification, fallback, and token reporting in one shared module.

StoreBrowser records

Saves history, audit logs, session stats, and usage charts locally in the browser.

DeployHosted compatibility

Vercel uses the same system snapshot shape, with inline completion because serverless memory is not a durable queue.

Source map

These are the first files to open when someone wants to understand or contribute.

outputs/workspace.html

Main app experience: prompt box, live status, run buttons, results, history, and audit writes.

outputs/system-worker.js

Browser background worker for fast token estimates, constraint detection, contract preview, and route hints.

optimizer-system.cjs

System runner for run IDs, stages, background local execution, and shared run snapshot formatting.

optimizer-core.cjs

Public entry point that re-exports the optimizer core API from the focused modules under core/.

core/*.cjs

Provider adapters, adaptive routing, handoff preparation, prompt templates, secret redaction, usage accounting, and the workflow runners.

server.cjs

Thin local host for static routing, provider APIs, system-run endpoints, and run polling.

api/system-runs.js

Hosted system-run endpoint that returns the same snapshot shape as local background runs.

api/*.js

Vercel function entrypoints for hosted optimize, generate, provider status, and compatibility routes.

outputs/agent-structure.html

Agent roles and the hub-and-spoke graph that explains flow of information.

graphify-out

Graphify's interactive repository map, architecture report, and machine-readable code graph.

extensions/gemini-token-optimizer

Manifest V3 side-panel MVP for capturing, optimizing, and inserting prompts on Gemini.

README.md

Local run steps, environment setup, project overview, and contribution notes.

Explore the real code graph

Graphify maps the repository locally from its syntax trees, then publishes an inspectable graph that coding assistants can query without repeatedly loading every file.

Graphify

Files, functions, calls, and dependencies in one map

Search for a symbol, select a node to inspect its source file and neighbors, or filter communities to focus on the optimizer core, workspace, APIs, extension, tests, and deployment routes.

Open interactive map Open graph data Read architecture report

339code nodes

652relations

18communities

Background processing model

The system does useful work before and during a run without making the workspace feel busy.

PreflightRuns while typing

The worker updates token estimates, route hints, and compact contract previews without blocking the page.

Local runQueued by ID

The Node server starts a background job and the UI polls `/api/system-runs/:id` for stage snapshots.

Hosted runInline but compatible

Vercel completes the run inside one request and returns the same `run` object the local queue uses.

Typed contract

This is the heart of the token-saving system: compact state that workflow nodes can pass safely.

{
  "contract_id": "optimizer.contract_workflow.v2",
  "goal": "Complete the user's requested task.",
  "facts": [
    "Important context extracted from the messy prompt",
    "Relevant file, platform, or output requirements"
  ],
  "constraints": [
    "Preserve intent",
    "Avoid repeated context",
    "Do not include secrets"
  ],
  "decisions": [
    "Use raw input only during intake",
    "Use direct route for simple prompts",
    "Pass compact state to downstream nodes when structure is needed"
  ],
  "sources": ["user_input"],
  "open_questions": [],
  "next_action": "Run the optimized workflow and return the best result.",
  "required_payload": [
    "goal",
    "facts",
    "constraints",
    "decisions",
    "sources",
    "open_questions",
    "next_action"
  ]
}

Why it helpsNo repeated transcript

Downstream nodes receive only the useful state for their job.

Why it is openEasy to inspect

The contract shape is readable, copyable, and simple to compare between runs.

Generated file directions

The optimizer can produce files that tell users exactly where to put each artifact.

Markdowninstructions.md

Use for human-readable steps, agent instructions, and project notes.

YAMLhandoff.yaml

Use for nested workflow state, agent responsibilities, adapter routes, and config-like data.

JSONapi-payload.json

Use for API calls, structured contracts, validation fixtures, and provider-agnostic routing.

Sidecar wrapper layer

Background mode keeps the app useful beside active LLMs and IDE agents without making users think about token mechanics.

BrowserSide-panel workflow

Copy the compact sidecar prompt into the active chat tab, then return only if verification is needed.

GoogleWeb LLM wrapper

Use split-screen or a future extension to compress before pasting into a Google-style prompt box.

IDEAgent coding sidecar

Generate `.token-optimizer` files so coding agents can receive concise task state.

Gemini MVPChrome extension package

The repo now includes a local unpacked extension that opens Token Optimizer as a Gemini side panel.

Contributor checklist

Keep the user path simple. Put complexity behind tabs, details, or this page.

Product ruleDo not make users manage tokens manually

The app should decide the compact route and explain what happened after the run.

Open ruleMake every agent responsibility inspectable

If a step changes state, it should appear in prompts, contracts, graph, or audit log.