From The Creator Of Redis; Run LLM Locally With Ds4

TL;DR

Prime Big Deal Days · Oct 6–7Offer from Amazon

Get tech for your team delivered free — and shop member deals

• Fast, free delivery on millions of items
• Access to Prime Big Deal Days deals on October 6–7
• Prime Video, Amazon Music and more included

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

Salvatore Sanfilippo, the creator of Redis, has released ds4, a MIT-licensed C inference engine designed to run selected large open-weight models on high-memory Macs and CUDA- or ROCm-equipped systems. Its project site describes features including asymmetric 2-bit quantization, SSD-backed KV caching and local APIs; performance figures are project-reported benchmarks, not independent tests.

Salvatore Sanfilippo, the creator of Redis, has introduced DwarfStar 4 (ds4), a MIT-licensed inference engine written in C for running a limited set of large open-weight language models locally. The project targets high-memory Macs and CUDA- or ROCm-equipped machines, and combines a command-line interface, local APIs and a coding agent in one stack.

The ds4 project site lists support for DeepSeek V4 and V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next, including text and vision models. The engine supports Apple Metal, NVIDIA CUDA and AMD ROCm backends. The site characterizes ds4 as a narrow, model-specific system rather than a general-purpose GGUF runner: it says supported model layouts are validated end to end, and users should download the project’s prepared model files rather than assume generic GGUF files will work.

Its central compression method is asymmetric 2-bit quantization, which the project says compresses routed experts in mixture-of-experts models while keeping selected shared paths more precise. The site says this approach is intended to make supported model builds practical on high-memory computers. It also describes a KV cache that can save long prompt prefixes to SSD and resume them by prompt hash, so a restart need not always require processing the entire prefix again.

Users can interact with the same engine through ./ds4 for chat, ./ds4-server for local APIs and ./ds4-agent for persistent coding sessions. The server is described as supporting OpenAI- and Anthropic-style APIs, allowing compatible local clients to connect through a base URL. The project provides a download script and build instructions, but actual model fit and performance depend on the selected weights, hardware, context length and backend.

Local Access to Large Models

ds4 addresses a practical barrier to local AI use: large models can require substantial memory and computing resources, and running them on a personal machine can involve more than downloading weights. By combining model-specific compression, several hardware backends and familiar API interfaces, the project aims to give developers a route to test and use selected models without sending prompts to a remote inference service.

That may matter for users who want local control of prompts and model sessions, or who want to connect coding tools to an engine running on their own hardware. It does not, by itself, establish that local inference is cheaper, faster or more private in every setup. Those outcomes depend on the machine, configuration, model and the user’s security practices. The project’s narrow support list also means ds4 is not presented as a universal engine for arbitrary model files.

The performance numbers offer an initial guide to the intended hardware range, but should be read as project-published measurements. For example, the site reports 39.4 generated tokens per second on an M5 Max with 128 GB of memory at a 2,048-token context for a Q2 configuration. It reports 27.6 tokens per second at 65,536 tokens on that machine. These are separate workload measurements, not a guarantee of results on other systems.

A Focused Inference Stack

The project describes its design around a 284-billion-parameter DeepSeek V4 Flash mixture-of-experts model, presenting local execution as an alternative to the more usual remote-serving approach. In a mixture-of-experts model, different parts of the network are routed for different inputs; ds4 says its quantization strategy focuses compression on routed experts while preserving selected paths at greater precision. The source material does not provide independent validation of the resulting quality or trade-offs.

The system brings together model files, the C engine, a cache that can use RAM and SSD, and interfaces for interactive use or software clients. The project’s examples include a 128 GB M5 Max and NVIDIA DGX Spark. Its displayed benchmark table reports prefill and generation separately: on the M5 Max at a 2,048-token context, it lists 790.2 tokens per second for prefill and 39.4 for generation; at 65,536 tokens, it lists 398.5 and 27.6, respectively. For the DGX Spark, it lists 825.8 and 18.1 at 2,048 tokens, and 823.0 and 13.8 at 65,536 tokens.

These figures come from the ds4 benchmark table and are not accompanied in the supplied material by independent replication or a full test methodology. The site also lists fit guidance for particular model quantizations and memory sizes, including a claim that Qwen can run on 64 GB. Such guidance describes project-supported configurations, not a general minimum for all models or workloads.

“Run frontier open weights locally with ds4.”

— The ds4 project site

Limits of the Published Results

The supplied material does not specify when ds4 was released, how long it has been available, or whether the listed model and hardware support has been independently tested. The benchmarks are attributed to the project; independent measurements and full test conditions are not provided here. Results may vary with software versions, model files, settings and system configuration.

It is also not clear from the source material how model output quality compares with less-compressed versions, what trade-offs arise from the quantization choices, or how much SSD caching changes performance across different workloads. The project lists support for specific models and prepared layouts, but does not establish support for every version or file format associated with those model families. Details about the native agent’s capabilities and security protections are also not set out in the supplied description.

What Users Can Test Next

The project directs users to clone its GitHub repository, download a supported model using its script, build for their chosen backend and start the CLI or server. Developers interested in connecting a coding tool can test the local API using a compatible client and the server’s base URL. They will need to check the project’s hardware guidance and model-specific instructions before selecting a configuration.

Further evidence will come from hands-on testing and additional published benchmarks, particularly measurements that disclose settings and compare model output quality alongside speed and memory use. The source material does not announce a future release date or a next formal milestone, so it remains unclear when more models, hardware guidance or independently verified results might appear.

Key Questions

What is ds4?

ds4, short for DwarfStar 4, is a MIT-licensed inference engine written in C. Its project site says it is designed to run selected open-weight models locally on high-memory Macs and CUDA- or ROCm-equipped computers.

Which models does ds4 list as supported?

The project lists DeepSeek V4 and V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next, with text and vision support. It says users should use supported model layouts and project-provided weights; arbitrary GGUF files are not its target.

Does ds4 require a high-memory computer?

The project is aimed at high-memory systems and provides fit guidance for particular model configurations. The memory needed depends on the model, quantization, context length and execution setup; the supplied material does not give one minimum that applies to all uses.

Are the ds4 benchmark numbers independently verified?

The figures in the supplied material are presented in the ds4 project’s benchmark table. No independent replication or complete testing methodology is included, so they should be treated as project-reported results rather than independently verified performance.

Can other applications connect to ds4?

The project says ds4-server supports OpenAI- and Anthropic-style APIs and can be connected to compatible local clients through a base URL. Compatibility and performance will depend on the client and configuration.

Source: hn