The key performance indicator of hardware for running Large Language Models (LLMs) is memory bandwidth: for processing Tokens, or fragments of words, an LLM needs to read from memory the Billions of weights, or probabilities, which it is made of.

The larger the model, the more important this becomes for effective interaction with LLMs.

The Synogate LLM Accelerator is a drop-in replacement for dedicated hardware like Graphics Processing Units (GPUs), using memory bandwidth much more efficiently. It is the ideal hardware to run large LLMs like ChatGPT and Llama 2 on-premise.

Our solutions offer the best Token/s per USD on the market, and can be scaled to reach extreme performance: a configurable design that can run on cost-effective, compact solutions, or very powerful 1U servers.

For cloud and enterprise-scale solutions, these servers can also be clustered into racks, optimizied to work in perfect synchronization.

LLMs are different from previous forms of artificial intelligence in many ways. From a hardware perspective, the shift from compute-bound to memory-bound is the most significant. This means that the limiting factor of hardware performance is no longer the processing speed, but the speed at which models can be read from memory. Custom digital circuit design offers precise control over signal flow, allowing for extremely efficient memory bandwidth usage. With massively synchronized calculations, we can produce stunning performance, and scale very efficiently due to precise concertation between devices.