---
title: "ASIC and Tokenizer Hardware"
description: "A feasibility study on compiling LLM tokenizers into streaming circuits that match the software exactly. We measure cost before any silicon, and chip design notes run alongside the study."
canonical: "https://laboetie.io/lab/tokenizer-hw"
language: en
---
# ASIC and Tokenizer Hardware

A feasibility study on compiling LLM tokenizers into streaming circuits that match the software exactly. We measure cost before any silicon, and chip design notes run alongside the study.

Status: Research, software baselines. Access: In house. 16,843,008 strings enumerated per encoding.

## Capabilities

- The reference definition fixes the exact function over Unicode and raw bytes.
- We enumerate every byte string up to length 3.
- The multi category test corpus includes pathological input.
- We benchmark existing tokenizers in a harness and compare their output token by token.
- Our research notes cover the chip design flow from specification to tapeout.