Tiny Corp Nvidia eGPU Driver Approved by Apple for Arm Macs, Enabling LLM Inference

Breakthrough: External Nvidia GPUs Now Work on Apple Silicon via Signed Driver

Tiny Corp has achieved a landmark milestone by getting Apple to approve a driver that enables Nvidia eGPUs to work with Arm-based Macs, opening the door for LLM inference on Apple hardware with Nvidia GPUs.

How It Works

• The driver is developed by Tiny Corp not Nvidia directly
• Users compile the driver with Docker rather than plug-and-play installation
• The driver is specifically designed for LLM workloads
• Apple is allowing the driver to get signed eliminating the need to disable System Integrity Protection (SIP)

Significance

This development is significant because:

• Apple has historically restricted Nvidia GPU support on Macs dating back years
• The removal of SIP disablement requirement makes it accessible to mainstream users
• It bridges the gap between Apple Silicon efficiency and Nvidia GPU compute power
• Enables developers to use Nvidia CUDA ecosystem on Mac hardware

Use Case

The primary use case is LLM inference acceleration. Developers can now combine Mac usability with Nvidia GPU performance for running large language models, potentially offering a cost-effective alternative to dedicated AI workstations.

Context

Tiny Corp led by George Hotz is known for its tinygrad framework and hardware projects. The company has been working to democratize AI compute by making powerful hardware accessible through open-source software.

Source: The Verge via Tiny Corp https://www.theverge.com/tech