Browser or Cloud? A Founder Framework for Hybrid AI Apps

“Run the AI locally” sounds like an architecture decision. For a founder, it is really a product promise.

It may mean a customer’s document never leaves the browser. It may mean a feature still works on a train with bad connectivity. It may mean the first result appears without a trip to a distant server. It can also mean asking a user to download a model, consuming their memory and battery, failing on an older phone, and giving your team fewer clues when something goes wrong.

Cloud inference has the opposite shape. It gives you access to larger models, consistent hardware, centralized updates, and better operational visibility. It also creates network latency, usage cost, data-transfer obligations, provider dependencies, and a larger security boundary.

Neither side is automatically more responsible. The useful question is narrower:

For this specific task, on this specific device, with this specific data, where should inference happen—and what happens when the preferred path is unavailable?

Google’s release of LiteRT.js makes that question more practical for web products. The runtime can execute converted models in a browser through WebAssembly, WebGPU, and emerging WebNN support. ONNX Runtime Web, Transformers.js, TensorFlow.js, and browser-managed AI APIs offer other routes. But a new runtime does not remove the product work. It makes the routing decision more important.

This guide gives non-technical founders a way to make that decision without pretending every AI feature needs an enormous language model or that every user has a new laptop.

Start With the Job, Not the Model

A common mistake is to choose a model, become excited that it can run in a browser, and then search for product features to justify it. Reverse the order.

Write the job in one sentence:

When the user does X, the product should produce Y within Z time, while data class D stays inside boundary B.

For example:

When a user selects a receipt image, the product should detect whether it contains a likely card number within one second, while the image stays on the device.

That is a good candidate for local inference. The input is sensitive, the task is narrow, a small specialist model may be sufficient, and a fast answer improves the interaction.

Now compare this job:

When a founder uploads research notes, the product should produce a sourced market analysis, check current public information, and preserve a reviewable evidence trail.

That belongs primarily in the cloud. It needs current retrieval, orchestration, more capable reasoning, and durable logs. Trying to force it into a downloaded browser model may reduce quality while creating a false privacy story.

The unit of architecture is the job, not the entire product. One page can use local language detection, cloud retrieval, local redaction, and cloud generation in a single responsible flow.

The Five-Question Routing Test

Score each AI job against five questions before choosing a runtime.

Question	Favors browser/device	Favors cloud
How sensitive is the raw input?	Raw personal, medical, financial, or private workspace data need not leave the device	Data is already public, approved for processing, or must be shared with a server workflow
How bounded is the task?	Classification, detection, embeddings, transcription chunks, simple rewrite, or fixed extraction	Open-ended research, complex planning, long-context reasoning, or multi-step tool use
How strict is the interaction latency?	Per-keystroke or camera-frame feedback where a network round trip feels broken	Background work where seconds or minutes are acceptable
How predictable are user devices?	Controlled desktop fleet or a known supported browser/device segment	Wide consumer audience including older phones, restricted browsers, and low-memory devices
How important are centralized updates and audit?	Stable model with limited consequence and local verification	Fast-changing policy, regulated decisions, abuse monitoring, or outcomes requiring a durable trace

Do not turn this into a fake numerical formula. A single hard boundary can override four softer preferences. If raw data is contractually prohibited from leaving the device, server convenience does not win. If a safety-critical result needs a centrally auditable decision process, a small opaque local model does not win merely because it is fast.

The table is a conversation tool. It reveals which assumption must be tested.

What Should Usually Stay in the Browser

Local inference works best when the task is frequent, narrow, latency-sensitive, and possible with a compact model.

Good candidates include:

detecting language before routing text;
flagging likely personal information before an upload;
generating embeddings for local search over private notes;
classifying images, sounds, or gestures in an interactive interface;
providing lightweight autocomplete or rewrite suggestions;
extracting a predictable set of fields from a known document type;
maintaining a limited offline mode;
creating a low-cost preview before the user requests a higher-quality cloud result.

These are not universal recommendations. “Local redaction” is only trustworthy if you measure what it misses. “Local embeddings” still expose data if your application later uploads the vectors or surrounding text. “Offline mode” is not real if the model was never downloaded or the browser evicted it.

The privacy claim must describe the whole data path, not just where one inference call runs.

What Usually Belongs in the Cloud

Cloud inference is the better default when the job needs capabilities or controls that cannot be delivered consistently across user devices.

Typical cases include:

research that needs current web or private company sources;
long-context synthesis across many documents;
workflows that call third-party tools or update shared records;
decisions that need centralized policy enforcement and audit logs;
jobs that need a large model, substantial memory, or sustained compute;
asynchronous work that should continue after the tab closes;
high-value results that need human review, versioning, and replay;
experiences where all supported users must receive a consistent capability.

Cloud does not excuse careless collection. Send the minimum necessary input, separate product data from diagnostics, define retention, and make provider behavior part of the product’s data map. A local pre-processing step can reduce what the server receives, but it should not create a misleading “all on-device” badge.

Hybrid Is a Routing Policy, Not Two Implementations Glued Together

The most resilient design is often hybrid: a local path for eligible tasks and a cloud path for tasks that need greater capability or broader compatibility.

A useful routing sequence is:

Classify the data boundary. Can this input leave the device under the user’s settings and your policy?
Check capability. Is the required API, runtime, model, memory, and accelerator actually available?
Check readiness. Is the model downloaded and compiled, or would the wait violate the interaction promise?
Run the local task when eligible. Record only privacy-safe operational signals.
Measure confidence or validate the output. Low confidence should narrow the feature, ask for confirmation, or use an allowed escalation path.
Fall back deliberately. Use cloud inference only when policy permits it and the user experience makes the transition clear.

The wrong implementation quietly tries local execution, catches any error, and sends the same raw data to a server. That is technically resilient and ethically brittle. A user who chose local processing may experience the fallback as a data breach.

Treat fallback as a product state with three explicit outcomes:

Local required: wait, reduce capability, or stop; do not upload.
Local preferred: explain that cloud processing is available and ask when the data is sensitive.
Cloud allowed: fall back automatically only if your existing notice genuinely covers it.

This routing policy should be readable by product, engineering, support, and legal teams. If it exists only as branching code, the company will eventually describe it incorrectly.

The Hidden Costs of Browser AI

Local inference removes some server cost, but it moves work into places founders do not normally monitor.

1. The model is part of the page experience

A model download may be much larger than the JavaScript for the feature around it. The first use can include downloading, decompressing, compiling, warming up, and allocating memory. Chrome’s guidance for built-in AI explicitly recommends showing download progress. ONNX Runtime’s deployment documentation similarly emphasizes artifact size, load time, caching, and memory.

Do not hide this behind an endless spinner. Tell users what is happening, let them cancel, and avoid starting a large download before clear intent. Test on a fresh browser profile, not only on a developer machine where the model is already cached.

2. “Supported browser” does not mean “supported user”

WebGPU availability is broader than it used to be, but browser version, operating system, GPU driver, memory pressure, enterprise policy, and hardware still affect the result. WebNN remains experimental in the current LiteRT.js setup guidance. A runtime may fall back to CPU, turning an acceptable 200-millisecond interaction into several seconds while heating a laptop.

Capability detection must happen at runtime. Your marketing page should not convert a best-case compatibility table into a universal promise.

3. Conversion is not validation

LiteRT.js can accept models converted from frameworks such as PyTorch, JAX, and TensorFlow, but conversion can fail because of dynamic shapes, unsupported operations, tensor types, or memory limits. A model that loads can still produce meaningfully different outputs after conversion or quantization.

Test three things separately:

compatibility: does it load and execute on each supported backend?
performance: what are cold start, warm latency, peak memory, and sustained battery/CPU cost?
quality: does the converted model still pass product-specific cases, including rare and adversarial inputs?

The runtime’s model tester can catch execution problems. It cannot decide whether the output is good enough for your users.

4. The model is delivered to the user

If your application downloads model weights, assume motivated users can inspect and copy them. Do not ship a model whose secrecy is essential to your business or whose license does not permit client distribution. Do not place credentials, private prompts, internal rules, or customer-derived secrets inside downloadable artifacts.

Local execution protects input transit. It does not protect your model from the device owner.

5. Local privacy needs local observability

If you log every input and output to understand failures, you have undone the privacy benefit. If you log nothing, you may not know that one browser backend is failing for 30 percent of users.

Collect bounded operational signals: runtime selected, model version, download state, initialization result, latency bucket, error category, and an opt-in quality signal. Avoid raw content, embeddings, or reconstructed user data unless the user knowingly submits a case for support.

A Founder-Grade Launch Gate

Before exposing a browser AI feature to everyone, run a small launch gate.

Product boundary

The job and promised response time are written in plain language.
The interface says when processing is local, cloud-based, or unavailable.
Local-required and local-preferred modes behave differently.
The feature remains understandable when AI is disabled.

Device matrix

Test the oldest phone and laptop you actually support.
Test Chrome, Safari, Firefox, and Edge where relevant.
Test with GPU acceleration available and unavailable.
Test low-memory conditions, a cold cache, slow network, and an interrupted download.
Verify what happens when the tab is backgrounded or closed.

Model quality

Compare original and converted model outputs on a fixed task set.
Include messy, multilingual, low-quality, and adversarial inputs.
Define a reject or escalation state instead of forcing an answer.
Version the model and keep a rollback path.

Privacy and security

Inspect network traffic to confirm which data leaves the device.
Confirm analytics does not capture raw sensitive inputs.
Review model and dataset licenses for browser distribution.
Serve the app over HTTPS and apply a restrictive content security policy.
Test untrusted model files and dependencies outside production first.

Operations

Measure cold start separately from warm inference.
Track readiness, backend, latency, and failure category without user content.
Set thresholds for disabling a backend or model version.
Give support a way to identify the runtime path without seeing the user’s data.

Ship to a small percentage of eligible users first. Device diversity creates failures that a local lab cannot reproduce. A staged release is not only for large companies; it is how a small team avoids turning every customer into an involuntary hardware tester.

A Concrete Example: Private Document Intake

Imagine a founder building an AI assistant that reviews business documents.

The tempting design uploads the entire file to a powerful model and asks it to identify the document, extract metadata, summarize it, and suggest actions. The flow is simple, but every document crosses the server boundary before the product knows whether that was necessary.

A hybrid flow is more deliberate:

The browser checks file type and size with ordinary code.
A small local model classifies the document and flags likely sensitive regions.
The interface shows what was detected and asks what the user wants to share.
If the user chooses a local-only action, the product performs limited extraction on device.
If the user requests deep analysis, the app sends only the approved content to a cloud workflow.
The cloud result includes sources or document references and enters a reviewable project history.

This design does not claim that local classification makes the whole product private. It gives the user a meaningful boundary before upload. It also has a clear limitation: if the local detector misses sensitive content, the user can still share it. The interface must not present the detector as a guarantee.

For an early MVP, even this may be too much. If the team cannot validate the local model across target devices, it may be more honest to launch a cloud-only beta with strict data minimization, clear retention, and a narrow audience. Hybrid architecture is valuable only when the team can operate both routes.

When Not to Build On-Device AI Yet

Do not add a browser model merely to make the product sound advanced or private.

Delay it when:

ordinary deterministic code can perform the job more reliably;
the local model does not meet the product’s quality threshold;
most users lack compatible hardware or browsers;
the download is larger than the value of the feature;
the task changes so often that coordinating model updates becomes the main work;
your team cannot test, observe, and roll back multiple execution paths;
a cloud-only beta will teach you more about whether users want the workflow.

The cheapest inference is the inference you do not need. A regular expression, schema validator, search index, or explicit user choice may be faster, easier to explain, and more reliable than a model.

The Decision to Keep

The durable lesson from LiteRT.js is not that every web app should run AI in WebGPU. It is that web products now have a more credible spectrum of execution choices.

Use the browser for bounded work where locality creates a real user benefit. Use the cloud for capability, consistency, shared workflows, and auditable long-running work. Connect them with an explicit routing policy that respects the data boundary even when the preferred runtime fails.

For a recovering content or product brand, that restraint matters. Trust is not built by attaching “private,” “local,” or “AI-powered” to a feature. It is built by stating what happens, testing the uncomfortable devices and failure paths, and refusing to promise more than the architecture can deliver.

References