🔍 Read the full analysis: Getting Started With MPS In PyTorch 2.15 on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
PyTorch 2.15 adds FP8 tensor support and performance updates to its Metal Performance Shaders backend for Apple silicon. Hugging Face reports large gains in selected model and video-decoding tests, but results depend on the workload; the release requires macOS 15 or later and Python 3.11 or later.
PyTorch 2.15 adds FP8 tensor support and new performance paths to its Metal Performance Shaders (MPS) backend for Apple silicon Macs. The PyTorch team at Hugging Face reports faster decoding for two tested language models and substantially lower peak memory in a Wan2.2 video-decoding test, while warning that results vary by workload and setup, as described in the original analysis.
On an M3 Ultra using torch.compile, Hugging Face measured Qwen3.6-35B-A3B decoding at 61.3 tokens per second with PyTorch 2.15, compared with 9.4 in version 2.14. For LFM2.5-8B-A1B, it reported 123.6 tokens per second, up from 23.6. These measurements apply to the named models and test configuration; they do not establish similar performance on other Macs or models.
The team also reports a 1.23× geometric-mean speedup across 96 BF16 layer shapes taken from real models. The changes route eligible single-row linear layers through specialized matrix-vector kernels, fuse some bias additions, and add a native grouped matrix multiplication path for mixture-of-experts decoding. Hugging Face says the work also reduces GPU-to-CPU synchronization and makes vocabulary softmax about eight times faster in its measurements.
Version 2.15 supports torch.float8_e4m3fn tensors on MPS. FP8 values use one byte each, half the storage of BF16 values, and the release supports scaled matrix multiplication with tensorwise or rowwise scaling. Other reported MPS work includes complex LU and triangular solves, QR decomposition, convolutions, padding, dropout, interpolation backward, and reductions.
Faster Local Inference on Macs
The update targets two practical limits for running models locally: per-token response speed and available memory. Faster decoding can make interactive use feel more responsive, while lower memory use may let some workflows run with less pressure on a Mac’s unified memory. FP8’s smaller storage footprint may also help applications that can use the new format and scaling operations effectively.
Hugging Face says its Wan2.2 tiled VAE decoding test peaked at about 8 GiB in PyTorch 2.15, compared with 270 GiB in its earlier test. It also reports that an image-padding change improved a Qwen-Image-2.1 encode-decode round trip from 10.06 dB to 29.95 dB PSNR. These results suggest the work addresses memory and output-correctness concerns as well as raw speed, but they remain specific to the reported pipelines. Developers should measure their own memory use, latency, and output quality before relying on the gains.
Apple Silicon Mac compatible external GPU
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Building on Earlier MPS Kernels
MPS is PyTorch’s backend for using Apple’s Metal GPU framework. Hugging Face’s PyTorch team says that some models previously encountered unsupported operations, performance drops, or unexpected memory growth on MPS. The team’s stated approach has been to test models from the Hugging Face Hub and address backend gaps with PyTorch maintainers and community contributors, including by replacing some MPSGraph paths with dedicated Metal kernels.
PyTorch 2.15 extends work from version 2.14, which had introduced matrix-vector kernels for torch.mm. The new release connects eligible F.linear calls to those kernels, targeting single-row inputs commonly used during language-model decoding. The minimum requirements stated in the release material are macOS 15 and Python 3.11. MPS availability checks also give a reason when the operating system is too old or a Metal GPU is unavailable.
“The goal is to make your Mac a first-class place to run and train PyTorch models.”
— PyTorch team at Hugging Face
PyTorch MPS optimized MacBook accessories
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Benchmark Scope and Reproducibility
The supplied announcement does not include enough detail to independently reproduce every result, including full software versions, run counts, and measurement conditions. The language-model figures name an M3 Ultra and torch.compile, but do not show how performance changes across other Apple silicon chips, model configurations, or generation settings.
The Wan2.2 result is also tied to one reported tiled VAE decoding test. The material does not establish whether the same memory reduction holds across resolutions, pipeline settings, or other video-generation workflows. Similarly, the Qwen-Image-2.1 PSNR figures do not provide a broad assessment across images or use cases. FP8 support is confirmed, but the accuracy and memory effects in a particular application depend on its quantization and scaling choices.
FP8 tensor support for machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Testing 2.15 and Tracking 2.16
Developers considering the update should first check compatibility: the stated requirements are macOS 15 or later and Python 3.11 or later. They can then compare PyTorch 2.15 with their existing environment on the models and workflows they actually use, recording latency, peak memory, and output quality rather than assuming the published figures will transfer directly.
Hugging Face’s team previewed further work for PyTorch 2.16, including moving attention more fully onto Metal, additional migrations from MPSGraph, and faster LLM decoding. Those are future plans, not features confirmed as part of 2.15. Their availability and measured effects remain to be established in a later release.
As an affiliate, we earn on qualifying purchases.
Key Questions
What does PyTorch 2.15 change for MPS?
It adds FP8 tensor support and performance paths for operations including eligible linear layers and mixture-of-experts decoding. The release also includes changes to other operations, such as convolutions, reductions, and padding.
How much faster are the tested language models?
On an M3 Ultra with torch.compile, Hugging Face reports Qwen3.6-35B-A3B at 61.3 tokens per second, compared with 9.4 on PyTorch 2.14, and LFM2.5-8B-A1B at 123.6, compared with 23.6. These are results for the specified tests, not a general performance guarantee.
What are the minimum requirements?
The release material lists macOS 15 or later and Python 3.11 or later for the stated MPS setup.
Does FP8 automatically reduce memory in every model?
No. FP8 values use half the storage of BF16 values, but practical memory and accuracy effects depend on how an application uses the format, including its scaling choices and model implementation.
What is planned for PyTorch 2.16?
Hugging Face previewed plans for attention to run more fully on Metal, further MPSGraph migrations, and faster LLM decoding. These remain planned work, not confirmed PyTorch 2.15 features.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
