ONNX Runtime Web

ONNX Runtime Web  runs .onnx models in the browser. Its WebNN execution provider (EP) hands the parts of a model that WebNN can run to the browser’s WebNN implementation, which can use the CPU, GPU or NPU. Operators that WebNN cannot run fall back to the WebAssembly CPU EP, so most models still run.

You select the WebNN EP with executionProviders: ['webnn'] when you create an InferenceSession. Everything else, such as tensors, inputs and outputs, works the same as for the wasm EP.

The WebNN API and its browser support are still evolving. Always keep 'wasm' as a fallback, and see Installation for the browser setup.

When to choose ONNX Runtime Web

Choose ONNX Runtime Web when you have an ONNX model and want direct control over sessions, shapes and tensors. Transformers.js uses ONNX Runtime Web underneath, so what this page says about shapes and the WebNN EP also applies there.

How the WebNN EP works

When a session is created with the webnn EP:

1. ONNX Runtime Web creates an MLContext with navigator.ml.createContext(), using your deviceType and powerPreference.
2. The EP reads the operator and data type limits of that context, and marks the nodes it can build.
3. The graph is split into partitions. Each partition of supported nodes is compiled into a WebNN graph. Everything else stays on the Wasm CPU EP.
4. On each run(), partitions execute through WebNN and the remaining nodes execute on Wasm.

Every hand-off between WebNN and Wasm costs time. A model split into many partitions can run slower than on wasm alone, so check the partition count, see Verify that WebNN is used.

Prerequisites

• A browser with WebNN enabled. Follow Installation.
• A secure context : HTTPS or localhost.
• An .onnx model, preferably with fixed input shapes, see Static shapes.

Install

npm install onnxruntime-web

The package exposes several entry points in its package.json exports. For WebNN, the ONNX Runtime documentation uses onnxruntime-web/all, and a JSPI build is available as onnxruntime-web/jspi.

With a script tag, load ort.all.min.js instead of ort.min.js:

<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/ort.all.min.js"></script>

Pin the version in production, for example onnxruntime-web@<version> with the exact version you tested, so that the JavaScript bundle and the Wasm files always match. The nightly onnxruntime-web@dev build can have newer WebNN fixes.

Use the WebNN EP

import * as ort from 'onnxruntime-web/all';

ort.env.wasm.wasmPaths = '/ort/';

const session = await ort.InferenceSession.create('/models/mobilenetv2-12.onnx', {
  executionProviders: [
    { name: 'webnn', deviceType: 'gpu', powerPreference: 'default' },
    'wasm',
  ],
  // The key must match the free dimension name stored in your model.
  freeDimensionOverrides: { batch_size: 1 },
});
 
console.log('Inputs:', session.inputNames, 'Outputs:', session.outputNames);

const data = new Float32Array(1 * 3 * 224 * 224);
const input = new ort.Tensor('float32', data, [1, 3, 224, 224]);
 
const results = await session.run({ [session.inputNames[0]]: input });
const scores = results[session.outputNames[0]].data; // a Float32Array
console.log('Number of scores:', scores.length);

await session.release();

Execution provider options

These are ONNX Runtime Web options, passed in the webnn entry of executionProviders. They are not the same as the WebNN API’s MLContextOptions, which no longer has deviceType. ONNX Runtime Web uses deviceType to configure the graph builders and to request the context.

The string form, 'webnn', creates the context with default options. numThreads also appears in the option type, but the session code in the sources does not use it, so do not rely on it.

These are hints. The browser makes the final device choice, which can differ from your preference. See the device selection diagram.

Two session options also matter for WebNN:

Static shapes

WebNN compiles graphs with fixed tensor shapes. The WebNN EP rejects a node whose tensors have a free dimension, and logs, at verbose level, Dynamic shape is not supported, use sessionOptions.FreeDimensionOverrides to set a fixed shape. That node, and often most of the graph, then runs on Wasm.

Fix this by giving each free dimension a number with freeDimensionOverrides. The key is the dimension name stored in the ONNX model, such as batch_size:

const session = await ort.InferenceSession.create(modelUrl, {
  executionProviders: ['webnn'],
  freeDimensionOverrides: { batch_size: 1, sequence_length: 128 },
});

Look up the dimension names in the model with a viewer such as Netron . See Free Dimension Overrides for a full walkthrough, and Performance issues if a model is slower than expected.

Complete example: image classification

This example classifies an image with an ONNX MobileNet whose input is [1, 3, 224, 224] in float32, normalized with the ImageNet mean and standard deviation. It tries WebNN on the GPU, then the NPU, then falls back to Wasm. Adjust the model URL, size and normalization to match your model.

import * as ort from 'onnxruntime-web/all';
 
const SIZE = 224;
const MEAN = [0.485, 0.456, 0.406];
const STD = [0.229, 0.224, 0.225];
 
ort.env.wasm.wasmPaths = '/ort/';
 
// Try each execution provider list in turn until a session can be created.
export async function createSession(modelUrl) {
  const bytes = new Uint8Array(await (await fetch(modelUrl)).arrayBuffer());
  const candidates = [
    [{ name: 'webnn', deviceType: 'gpu' }],
    [{ name: 'webnn', deviceType: 'npu' }],
    ['wasm'],
  ];
 
  for (const executionProviders of candidates) {
    try {
      const session = await ort.InferenceSession.create(bytes, {
        executionProviders,
        freeDimensionOverrides: { batch_size: 1 },
      });
      console.log('Session created with', JSON.stringify(executionProviders));
      return session;
    } catch (error) {
      console.warn('Could not use', JSON.stringify(executionProviders), error);
    }
  }
  throw new Error('No execution provider could create a session.');
}
 
// Convert an image (HTMLImageElement, ImageBitmap, canvas) to a normalized NCHW tensor.
function preprocess(image) {
  const canvas = new OffscreenCanvas(SIZE, SIZE);
  const context = canvas.getContext('2d', { willReadFrequently: true });
  context.drawImage(image, 0, 0, SIZE, SIZE);
  const { data } = context.getImageData(0, 0, SIZE, SIZE); // RGBA, row by row
 
  const chw = new Float32Array(3 * SIZE * SIZE);
  for (let i = 0; i < SIZE * SIZE; i++) {
    for (let channel = 0; channel < 3; channel++) {
      chw[channel * SIZE * SIZE + i] = (data[i * 4 + channel] / 255 - MEAN[channel]) / STD[channel];
    }
  }
  return new ort.Tensor('float32', chw, [1, 3, SIZE, SIZE]);
}
 
// Softmax over the raw scores, then return the k most likely labels.
function topK(scores, labels, k = 5) {
  const max = Math.max(...scores);
  const exps = Array.from(scores, (score) => Math.exp(score - max));
  const sum = exps.reduce((a, b) => a + b, 0);
  return exps
    .map((value, index) => ({ label: labels[index], probability: value / sum }))
    .sort((a, b) => b.probability - a.probability)
    .slice(0, k);
}
 
export async function classify(session, image, labels) {
  const input = preprocess(image);
  const results = await session.run({ [session.inputNames[0]]: input });
  return topK(results[session.outputNames[0]].data, labels);
}

import { createSession, classify } from './classify.js';
 
const labels = await (await fetch('/models/imagenet_labels.json')).json();
const session = await createSession('/models/mobilenetv2-12.onnx');
 
const image = document.querySelector('img');
console.table(await classify(session, image, labels));

For a full walkthrough, including the page setup, see Image Classification.

Fallback behavior

There are two levels of fallback:

• Per operator. Nodes that WebNN cannot build, for example an unsupported operator, data type or shape on this device, run on the Wasm CPU EP. The session is still created.
• Per session. If InferenceSession.create() throws, for example because WebNN is not available, retry with executionProviders: ['wasm'], as the example above does.

Quantized models are handled specially: with the webnn EP, ONNX Runtime Web turns off QDQ fusion and QDQ constant folding, so DequantizeLinear and QuantizeLinear nodes reach the WebNN EP as individual operators.

MLTensor and IO binding

By default, inputs and outputs live in CPU memory, and with gpu or npu the data is copied to the device and back on every run. With MLTensor, data can stay on the device. This helps most when outputs are fed back as inputs, for example in transformer decoding loops. The sources confirm ort.Tensor.fromMLTensor() and the 'ml-tensor' output location.

Per the ONNX Runtime guide, IO binding needs one shared MLContext, created up front and passed as the context option of the EP together with deviceType.

// ONNX Runtime Web still needs a deviceType next to the context.
const deviceType = 'gpu';
const mlContext = await navigator.ml.createContext({ powerPreference: 'high-performance' });
 
const session = await ort.InferenceSession.create(modelUrl, {
  executionProviders: [{ name: 'webnn', context: mlContext, deviceType }],
  freeDimensionOverrides: { batch_size: 1 },
  preferredOutputLocation: 'ml-tensor',
});
 
// An input that already lives on an MLTensor
const mlTensor = await mlContext.createTensor({
  dataType: 'float32',
  shape: [1, 3, 224, 224],
  writable: true,
});
mlContext.writeTensor(mlTensor, new Float32Array(1 * 3 * 224 * 224));
const input = ort.Tensor.fromMLTensor(mlTensor, { dataType: 'float32', dims: [1, 3, 224, 224] });
 
const results = await session.run({ [session.inputNames[0]]: input });
const output = results[session.outputNames[0]];
const values = await output.getData(); // downloads from the MLTensor to the CPU
console.log('Number of values:', values.length);
 
output.dispose(); // ONNX Runtime created this MLTensor, so release it
mlTensor.destroy(); // you created this one, so you destroy it

preferredOutputLocation also accepts an object keyed by output name, for example { output_0: 'cpu', output_1: 'ml-tensor' }.

• A tensor from Tensor.fromMLTensor() is owned by you. Keep it valid during run() and destroy the MLTensor when done. Running with a destroyed MLTensor fails.
• An output that ONNX Runtime created owns its MLTensor. Read it with getData() and release it with dispose().
• The MLTensor part of WebNN is still changing, so check your browser version if this path fails.

Useful ort.env flags

Set these before creating the first session. They are defined in the env type of ONNX Runtime Web.

numThreads and proxy configure the Wasm runtime and the fallback nodes. They do not change what runs on WebNN. Tensors on the 'ml-tensor' location are not serializable across the proxy worker, so avoid combining env.wasm.proxy with IO binding.

Run in a Web Worker

Running inference in a worker keeps session creation and execution off the main thread. The Web AI Run  inference worker, which runs ONNX Runtime Web with WebNN, uses this pattern:

• It is a classic worker, not a module worker, because the Wasm loader calls importScripts(), which module workers do not have.
• It loads the JSPI bundle ort.jspi.min.mjs with a dynamic import(), then sets ort.env.wasm.wasmPaths to the folder that holds the Wasm files.

// Pin an exact version in production instead of 'latest'.
const BASE = 'https://cdn.jsdelivr.net/npm/onnxruntime-web@latest/dist/';
let session;
 
self.onmessage = async (event) => {
  if (event.data.type === 'init') {
    const ort = await import(BASE + 'ort.jspi.min.mjs');
    ort.env.wasm.wasmPaths = BASE;
 
    session = await ort.InferenceSession.create(event.data.modelUrl, {
      executionProviders: [{ name: 'webnn', deviceType: 'gpu' }, 'wasm'],
      freeDimensionOverrides: event.data.freeDimensionOverrides,
    });
    self.postMessage({ type: 'ready' });
  }
};

Replace latest with the exact version you use, and create the worker with new Worker('/inference-worker.js'), without { type: 'module' }.

Verify that WebNN is used

A session that is created is not necessarily running on your GPU or NPU. Check it like this:

1. Set ort.env.logLevel = 'verbose' before creating the session.
2. Find the line from the WebNN EP, WebNNExecutionProvider::GetCapability, number of partitions supported by WebNN: ... number of nodes in the graph: ... number of nodes supported by WebNN: .... One partition that covers nearly all nodes means the model is almost fully on WebNN. The EP writes this line as a warning when there is more than one partition, and as info otherwise.
3. Look for verbose messages about unsupported shapes or data types, such as the dynamic-shape message above.

To confirm that the hardware is busy, use the browser and OS tools described in Debug.

Supported operators and data types

The WebNN EP builds ONNX operators with the builders in onnxruntime/core/providers/webnn/builders. Support also depends on the browser, because the EP reads the supported data types and limits from the context at run time. The maintained operator list is webnn-operators.md .

Troubleshooting

For more, see Common errors, Framework errors and Backend errors.

Limitations

• Shapes must be static for WebNN. Models with dynamic shapes need freeDimensionOverrides.
• Operator and data type support depends on the browser and device. Unsupported nodes run on Wasm.
• ONNX Runtime Web runs the model only. Pre- and post-processing is your code.
• WebNN and MLTensor are not available in every browser yet.

Next steps

• Follow the end-to-end Image Classification tutorial.
• Learn about Free Dimension Overrides.
• Compare with LiteRT.js and Transformers.js.
• Read the WebNN API reference.
• Official docs: Get started with ONNX Runtime Web , Using the WebNN EP , and env flags and session options .