Meet the world's smallest AI supercomputer — it packs 'doctorate-level intelligence', its makers say, and can fit into your pocket

A U.S. startup has developed what it claims is the world’s smallest artificial intelligence (AI) supercomputer. Packed with high-performance hardware and plenty of RAM, company representatives say it can run "Ph.D. intelligence" AI models — despite being compact enough to tuck into your pocket. This means they're capable of autonomous problem solving, abstract reasoning and strategic planning.

The "AI Pocket Lab," as its creators at Tiiny AI have branded the device, is capable of running a complex 120-billion-parameter large language model (LLM) locally, without any reliance on internet connectivity. You would ordinarily need data-center-class infrastructure to run these systems, and it opens up the possibility of local expert-level coding capabilities, document assessment and refinement, or multi-step reasoning.

It's built around a 12-core ARM processor, of the kind commonly found in smartphones, laptops and tablets. Despite its tiny frame — the device measures just 5.59 × 3.15 × 1.00 inches (14.2 × 8 × 2.53 cm) — it packs 80 GB of LPDDR5X RAM. Most current laptops come with between 8 GB and 32 GB RAM, by way of comparison.

A massive 48 GB of the Pocket Lab's RAM is also reserved exclusively for the neural processing unit (NPU), a chip optimized for AI-related computations. Both Intel and AMD have been manufacturing processors for a few years that include dedicated NPUs to handle AI workloads and to meet Microsoft's 40 trillion operations per second (TOPS) threshold to run AI features on Windows 11.

The Pocket Lab qualifies as a supercomputer (rather than a standard mini-PC or workstation) because of its computational power, capable of running workloads — specifically local inference on 100 billion-plus parameter language models — that normally require multi-GPU, data-center-class systems. Current models the device can run include GPT-OSS 120B, large Phi models and high-parameter Llama family models.

This is part of a recent push towards edge computing for AI, in an attempt to reduce some of the power constraints and environmental impact of distributed AI processing.

Pocket power

While it's a far cry from rivaling the world's most powerful supercomputers, the AI Pocket Lab is capable of delivering 190 TOPS of computing power between its NPU and CPU. It represents another step towards miniaturization in the wake of Nvidia's recently announced Project Digits mini PC. While it doesn't pack the same horsepower as the Nvidia project, it's a fraction of the size.

To pack so much power into such an unassuming chassis, the Tiiny AI team leaned on a number of technologies and optimizations. Key among them was something the company calls TurboSparse — an innovation that allows massive LLMs to run faster on more limited hardware by ensuring a system only calls on the parts of a model that it needs at any given moment. While traditional models use every parameter for each word of processing/output, a TurboSparse model only uses specific parameters per step.

Another important feature is PowerInfer, which allows for heterogeneous scheduling of the device’s CPU, GPU and NPU. This means that each processor is only given the workload that it's most capable of handling, which makes the entire system more efficient overall and reduces power draw. PowerInfer also includes intelligent power management, deciding when full power is necessary and when it's possible to use less, in part by eliminating unnecessary calculations.

The implications of a miniature AI supercomputer go beyond reducing our reliance on environmentally harmful data centers. It's a boon to privacy, with users able to deploy the power of a sophisticated LLM without being connected to the internet and without their data being processed in the cloud by third parties, while enabling AI access in fieldwork situations such as remote research stations, or on ships or aircraft out of connectivity range.