A great chip with mediocre software is a mediocre chip
It's a familiar pattern in the accelerator industry: a chip ships with genuinely strong theoretical performance, and real-world benchmarks come in well below expectations because the compiler doesn't map workloads onto the hardware efficiently, or the runtime introduces overhead the silicon spec sheet never accounted for. Peak theoretical throughput is a hardware property. Delivered throughput is a hardware and software property together, and software is usually where the gap opens up.
What the stack actually has to do
Between a trained model and useful inference sits a full stack: a compiler that has to translate model operations into instructions the hardware can execute efficiently, a runtime that schedules and manages execution, an SDK that developers actually interact with, and integration with the AI frameworks the rest of the industry already builds on. Every layer in that stack is an opportunity to either preserve the hardware's theoretical performance or quietly erode it.
Co-design means neither side is a moving target for the other
The alternative to bolting software onto finished silicon after the fact is designing them together, so the compiler and runtime are built with specific knowledge of how the hardware actually executes operations, and hardware decisions account for how the software stack will need to use them. That's the structure behind BERNIONE's approach: Silicon, Runtime, Compiler, SDK, and AI Frameworks designed as one system rather than a chip released first and a software stack scrambling to catch up to it.
Why this matters more for inference than it did for training
Training workloads have historically tolerated more software friction because the cost is amortized over long runs and the field has had over a decade to mature training software stacks. Inference is younger as a distinct optimization target, latency-sensitive, and running in production paths where every millisecond of software overhead shows up directly in the numbers that matter, tokens per second, tokens per watt, tokens per dollar. Hardware-aware software optimization built in at every layer is what lets AI models actually reach the full depth of an architecture, not just its peak theoretical throughput.
Read more about the stack behind BERNIONE X1 in the Hardware + Software Co-Design section of our homepage.