Rendered at 14:55:48 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
erikkahler 18 hours ago [-]
To clarify, this MIT-licensed app is from the very same dev, 'Prince Canuma', who maintains the popular MLX-VLM library (https://github.com/Blaizzy/mlx-vlm). MLX-VLM is a long-time dependency of the excellent LM Studio and others because it can provide faster inference on Apple devices than llama.cpp. Historically, MLX is a smaller community than CUDA, but has some of the fastest updates upon the release of new models, particularly in models with modalities beyond text-in, text-out (vision, STT, TTS, video gen). See also (https://github.com/Blaizzy/mlx-audio-swift). Would be totally unsurprised if those modalities and models get integrated into this UI.
Possibly vibe-coded landing page notwithstanding, the app is mostly written in Swift language. That suggests it will be easy to port this inference stack to iPad and iPhone.
walrus01 12 hours ago [-]
I would also note that for people who want to download these models, you can find MLX versions of just about everything popular on huggingface these days. For instance go look at the "main" page for Qwen 3.6 35B-A3B and then follow the link to quantizations, and pick one of the more popular/reputable MLX variants.
I was so excited when I saw "blaizzy" in the domain, because Prince Canuma's work around MLX has been of such uniquely high quality.
T0mSIlver 5 hours ago [-]
[flagged]
justatdotin 12 hours ago [-]
Hi Simon, sorry to spam your comments but 7 months ago you asked me for a media report to back a claim I made and this week it finally arrived. perhaps a little late to be very useful to anyone who maintains that gating function, but here nonetheless:
Would be totally unsurprised if those modalities and models get integrated into this UI.
Yup, the GitHub repo says:
Support for dedicated audio-only and image-generation-only models is coming soon.
Prince Canuma is super-responsive on X and GitHub issues, and I use mlx-audio almost daily with mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16 (for voice cloning).
trollbridge 18 hours ago [-]
Thanks for that. My first question was “What does this do that Unsloth doesn’t?”
layla5alive 5 hours ago [-]
Prince CacheAsNUMA? Nice :)
recursivegirth 15 hours ago [-]
LM Studio is trash on Windows / Linux... guess that makes sense...
carterschonwald 13 hours ago [-]
its a wrapper around mlx, so thats gonna be the portability bottleneck
Der_Einzige 4 hours ago [-]
Switching to mlx-vlm is basically harmful since it (like vllm and sglang) have such garbage support for modern samplers. To be clear, I am one of the authors on the min_p paper, and if min_p is the best you have (when llamacpp supports the far superior top-n-sigma), than I have no reason to switch even if you are somehow faster.
Thanks for bringing this up. You're 100% right. But most people, even technical ones are oblivious to how much of a difference modern samplers and higher quality quantization algorithms make for on-device LLM inference and are stuck with good old top-p, top-k samplers and RTN quantization.
TBF mlx-vlm does support min-p sampling, but none of the other modern samplers that you list. Ollama and LM Studio are even worse with only top-p and top-k samplers.
moontear 19 hours ago [-]
Is „frontier“ overused? I thought frontier models were the best-of-the-best such as Fable right now. I assume you can’t host these models yourself since you would need many GB of RAM and expensive GPU of is my thinking of „frontier models“ wrong?
oersted 14 hours ago [-]
I was confused too, but I believe that it refers to the Pareto Frontier: the best set of solutions to a multi-objective problem.
Look it up, it’s a bit difficult to explain concisely in words but it is intuitive visually.
If we are thinking of intelligence and price, a model will be in the Pareto Frontier if there’s no cheaper model of the same or higher intelligence. Or if there’s no more intelligent model for that price or lower.
So for example DeepSeek V4 Pro can be considered a frontier model because there's no cheaper model that is as intelligent.
For any solution in the Pareto Frontier, there no "no-brainer" alternative, in the sense that there's no other option that is better in some way without giving up something else. It's the best of its "weight-class".
That’s a good visualization, although I am a bit mistrustful of Arena’s scores. It does get around the fact that models are getting trained for the benchmarks, but the methodology of letting random people compare outputs side-by-side is a very shallow judgement method in my opinion.
EDIT: Indeed looking at the overall rankings for text again, the list is rather strange, a lot more about writing style than intelligence.
moontear 9 hours ago [-]
Very interesting rabbit hole for today. Thanks for mentioning Pareto frontier!
andsoitis 12 hours ago [-]
The subtitle on their website is: "Nativ puts frontier intelligence on your desk".
Also, the title is "Run AI models
locally on your Mac," not "Run frontier open models locally on your Mac."
dofm 16 hours ago [-]
I don't know that I agree with this specific use of frontier because it is confusable as you say.
But (off on a tangent) I do think that there are multiple frontiers generally — and I also think the open weights, small local model frontier is by far the most important and exciting one.
I keep mucking about with what Gemma 4 12B can do and every time I do I find myself thinking that all the energies in the AI world are going in entirely the wrong direction, because it is small, clever, efficient and remarkable.
If all of that research money were to be spent on improving AI models that fit inside a 16GB RAM machine with unified memory, I think really important progress could be made.
I enjoy using the Qwen 3.6 models (and BottleCap's new fine tune of the 27B) but the small Gemma 4 models are impressive in a way that I think is going quite unreported.
So while I don't think this website should use the word "frontier" here, even referring to Qwen 3.6 27B which is weirdly close, I think it could.
lukevp 16 hours ago [-]
I’ve got a 32 gig m1 MacBook Pro. how would I go about trying Gemma like you mentioned? Would it run at an acceptable speed, and what could I do? Coding?
“docker run open-webui:ollama” is a simple way to start.
Don’t expect much for coding. But it’s great for general knowledge, rubber ducking, image classification…
dofm 15 hours ago [-]
I figure it might be quite a competent general coding teacher for more, er, consumer programming languages, for want of a better word — python, PHP, JS. Seems to be pretty solid on WP knowledge too.
And I find it curiously interesting when talking about photography. I've been finding it intriguing to ask it about my own photos and make suggestions about other images to research. I just showed it three of my own photos, and asked it to analyse them and recommend photographers I should research. It recommended someone amazing I have never heard of before. But it also recommended a 19th century British photographer who happens to be my lifelong photographic hero — someone whose broad characteristics inform what I do without me slavishly copying them. Bit of a jaw-dropping moment for it to have picked up their influence in subject matter that they would never have approached.
I'm still suggesting it more to people for them to see what the small-model future might look like, because it's so much more capable than one might expect.
dofm 15 hours ago [-]
On an M1 Max I have been using either Unsloth Studio (which is basically a web app) or LM Studio (nicer app on the Mac). You can use the Google AI Edge Gallery to play with the smaller Gemma models (but at the moment the QAT variants don't seem to be there unless I am missing something).
I think it's likely the 26B QAT model won't fit in your machine — you may be able to fit one of the UD_Q3 or UD_Q2 variants but whether you'll be able to run other things you want at the same time, I don't know.
(The QAT models are "quantization aware training" — AIUI the model weights have been assigned during training to survive four-bit quantization with less loss.)
(I don't think the M1 really gets much benefit from MLX, in case you were wondering, though I could be wrong)
My interest in this model is largely to really get to grips with what small models can actually do, especially with tool calling, because I think it helps comprehend what the value proposition of the cloud models is.
I have been very surprised by the quality and clarity of its answers. It's also helped me understand that much of a typical harness system prompt is likely to be unnecessary now; Gemma 4 seems to be pretty sensible out of the box.
You're absolutely not going to be able to get it to go off and build whole apps from a long prompt; it is not that good, but it does tool calling and thinking, and you should be able to explore pointing a coding harness at it if you turn on LM Studio or Unsloth Studio's API server. You could also use the Llama system tray app (formerly LlamaBarn) or just use llama-server from the llama.cpp distribution.
Probably Pi is going to be a better harness because it can have a minimal system prompt, though I've not tested it with Pi myself.
It seems to know PHP and SQL to a fairly decent depth (and I suspect JS and Python). It also has a unified vision model (it doesn't need a separate mmproj sidecar thingy) that is fairly fast, and it is quite impressive at image analysis.
So you could probably use it to generate image descriptions and tags, summarise text, generate wordpress snippets, that sort of thing.
It can capably answer questions like "Can you characterise this image and suggest further similar images I might like?" — I am currently using this to provoke me to take photos again.
Have a play with the E4B edge model, too — again, much more interesting than I expected.
dragonwriter 18 hours ago [-]
The frontier is a multidimensional space defined by the “best” combination of traits a model can have in multiple dimensions: parameter size [smaller is better], various task metrics, and relative token generation speed on like hardware [faster is better], and active memory requirements [smaller is better] are all possible dimensions, and a model on the frontier of the current options space is one where getting better on one of those measures cannot be done without getting worse on at least one of the others.
That is one sense, but it is also use to refer to the frontier of capability. Especially in this context.
IshKebab 19 hours ago [-]
That's not what people are normally referring to when they say "frontier models". It means the most capable models full stop. Not the most capable that you can run locally.
barnabee 5 hours ago [-]
I’ve always seen the frontier shown as a curve on a plot. Frontier ≠ SOTA
zarzavat 3 hours ago [-]
Frontier = SOTA.
Pareto frontier ≠ frontier.
chrisweekly 16 hours ago [-]
true but tfa's title says "frontier open models"
wmf 14 hours ago [-]
Frontier open models are Kimi K3, GLM 5.2, DeepSeek V4 Pro, etc. They're all too big to fit on most Macs.
andsoitis 12 hours ago [-]
> tfa's title says "frontier open models"
Does it? I read: "Run AI models locally on your Mac."
IshKebab 8 hours ago [-]
The submission is "Nativ: Run frontier open models locally on your Mac" and on the page it says "Nativ puts frontier intelligence on your desk."
andsoitis 1 hours ago [-]
> puts frontier intelligence on your desk
Which of the models it puts on your desk do you consider frontier intelligence?
bnfcl 19 hours ago [-]
Had the same though. The gap between open-source and the frontier is closing in, especially with Kimi K3, but that is like >2T parameters. The Gemma 4 and other models you can actually run on an average Mac, is not in the same league.
lucideer 16 hours ago [-]
It might be overused elsewhere but I don't think its use here is inappropriate given it's qualified: it doesn't say "frontier models", it says "frontier open models".
jdthedisciple 9 hours ago [-]
No, your thinking is 100% correct. It's called clickbait.
44za12 19 hours ago [-]
+1 came here to say this, I opened the link expecting some technical breakthrough. Misleading click bait title.
Nekorosu 2 hours ago [-]
I really don't like the marketing texts.
"Why we’re open source when nobody else is."
I'm using oMLX which is open source and seems to be doing everything Nativ offers. I'd rather see the comparison with existing "non-existing" open source competitors.
giancarlostoro 2 hours ago [-]
What they likely mean is, why options like LM Studio are not open source.
kmike84 33 minutes ago [-]
omlx is quite similar to LM Studio, so there are "options like LM Studio" which are open source
D13Fd 19 hours ago [-]
I'm surprised that their home page basically acts as if LM Studio and others don't already do this. It's not clear what the difference is from a glance.
It also omits Open WebUI. I've been running Deepseek V4 Flash locally on my Macbook Pro for weeks using Open WebUI + DS4.
moostii 17 hours ago [-]
LM studio is closed source software built ON TOP OF code released by the author of Nativ.
a3w 7 hours ago [-]
jan.ai is the F/LOSS alternative to it. If you do want a gui.
kzrdude 18 hours ago [-]
LM Studio seems to do the same thing, except that LM Studio is not open source. So they have a point, they do something more.
wmf 14 hours ago [-]
"The other “local AI” apps you’ve heard of? They’re proprietary shells built on top of open-source engines they don’t own."
This is a roundabout way of addressing LM Studio.
lylejohnson 18 hours ago [-]
I was wondering the same and assuming I'd overlooked something.
syntaxing 13 hours ago [-]
What spec is your macbook? I want to run Deepseek V4 Flash but its too slow for agents on my Strix Halo.
JosNun 17 hours ago [-]
Genuinely curious: what are people using these smaller local models for? They are getting decently capable, but they are still small enough that I don't trust them for "real" work outside of a handful of fun toy projects.
Are people actually using them in coding agents? Or are they mostly using them for other things?
efficax 6 minutes ago [-]
I use a gemma4 model locally to extract content from messages to a personal agent I'm building for its memory graph (to break the message up into the topic, source (assistant or owner), facts, entities, etc. in the message content (all getting thrown into a magma-esque graph using NLEmbeddings for memory search). This is for a custom personal agent that targets deepseek-v4 flash. The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well, saving API usage on every chat turn.
kgeist 17 hours ago [-]
We've shipped some code generated by Qwen3.6 27B to production (under OpenCode). It lacks the breadth of knowledge of models like Opus, but if a change is fully inferable from the prompt and the surrounding code, it works very well. It won't be able to write something from scratch that requires niche knowledge (say, a performant inference engine tailored to Blackwell GPUs), but if it's just a PR adding a new use case to an existing project (which is usually just "load from the DB, do some invariant checks, modify the entities, store them back"), it works as well as Sonnet (provided you have the correct configuration, like recommended temperature and top-p settings, the model isn't over-quantized, you have at least 150k tokens of context available, etc.).
netghost 12 hours ago [-]
I don't use them as coding agents, but they can be very useful for things like text transformation, summarizing, or text extraction.
That said, if you have a subscription to a paid model already, you're not necessarily winning out on anything except perhaps privacy, which isn't nothing.
jiqiren 17 hours ago [-]
there is plenty of grunt work these smaller models can do. update dependencies, fix merge conflicts, write --help, markdown, or readme files for existing code. etc.
sometimes they fail but undo is just a "git restore" or if automated, rejecting a PR and having a better model take a crack at it.
mips_avatar 15 hours ago [-]
Qwen35ba3b can do a huge amount of data cleaning work on pretty modest hardware. Already have run about 100 billion tokens on it using 2x3090 gpus.
febed 9 hours ago [-]
What exactly do you mean by data cleaning
noja 3 hours ago [-]
are you using it differently to OpenRefine?
teaearlgraycold 17 hours ago [-]
They're great at helping me look up web dev stuff when I don't have internet access.
ActorNightly 13 hours ago [-]
[flagged]
invictati 9 hours ago [-]
Apple would not waste money astroturfing on HN to sell O(1000) Macs.
kridsdale1 12 hours ago [-]
Wow what a hater. You know what else is thousands of dollars and doesn’t even include a monitor? An ATX case with 2 3090s in it. And that will use a kilowatt or more to do what the Studio (which is excellent) does with about 250 W.
shireboy 3 hours ago [-]
What is the “middlest” Mac one could get for this? I’m in the market but keep going back and forth between a 64gb m5 pro or “lower end m5 air and screw it I’ll just pay for cloud tokens”. At current prices the 2-3k diff to try to run something local that isn’t as powerful could buy a lot of tokens.
froobius 3 hours ago [-]
You can run a lot of these on e.g. M1 max 64gb
mfro 1 hours ago [-]
I don't love that this starts an API server on launch and has no option to disable it...
n8henrie 46 minutes ago [-]
Any reason to use this over omlx?
shitcoder 17 hours ago [-]
Looking forward to giving this a try. I have tried MLX using Rapid MLX however the LLM (Qwen) would always have hiccups and get stuck repeating itself.
Moving onto llama.cpp I was able to get faster tokens with MTP and a more reliable llm.
I wonder what other people's experiences are using MLX vs llama.cpp
dofm 14 hours ago [-]
FWIW on my M1 Max I have not really seen any advantage at all from MLX.
I am fully prepared to believe the benefits accrue more to the M3 and up (because of changes to the Apple Neural Engine).
But with the models I've tested, unless I am missing something, the performance of GGUFs in llama.cpp has been better in some cases.
I still have not had results from Gemma 4's MTP be really worth it, to be honest; but with the Qwen 3.6 MoE it is measurable. Maybe with newer kit it is more meaningful.
(There is every chance that the above is not the experience of anyone who really deeply knows what they are doing; it feels like I am a perpetual novice at this stuff)
regexorcist 5 hours ago [-]
Same here. Tried MLX twice at different times after reading the claims here but it always does considerably worse for me than llamacpp.
c4pt0r 18 hours ago [-]
I'm a bit curious why not running DeepSeek V4 on top of https://github.com/antirez/ds4. I think the results could be really good.
Archit3ch 15 hours ago [-]
I assume Nativ doesn't support SSD streaming like DwarfStar.
themihai 19 minutes ago [-]
What’s the BS with “Open models from teams we trust.”?
cootsnuck 2 hours ago [-]
So should I ditch ollama for this?
isomorphic 19 hours ago [-]
This looks like the Prism folks, who are making binary/ternary versions of popular edge models, so that those models will fit on constrained devices like phones. E.g., their Bonsai model derived from Qwen:
Perhaps they got tired of LM Studio, etc., not being able to run their models properly.
dofm 19 hours ago [-]
Is it? The developer page for the github repo suggests he works at Arcee. Maybe he moved?
(I initially thought the same because of the website appearance)
Daunk 17 hours ago [-]
Is Gemma 4 E2B actually "usable"?
I've been running Gemma 4 12B and it handles everything very well!
But the second I've moved down to E4B it's been unable to perform the simplest of tasks.
So I can't even imagine how E2B would do...
Or am I doing something wrong?
jsomedon 8 hours ago [-]
I can't really tell why would I use this over like lm studio, jan.ai and such?
satvikpendem 19 hours ago [-]
Only interesting thing about this vibe coded runner is the MLX support, as that's still annoying to use in other ones, most still use GGUFs. Unsloth Studio which is an OSS runner I use is still in progress with MLX support although it's still a ways away.
saagarjha 5 hours ago [-]
I'm a little confused why the page lists "UNIVERSAL · APPLE SILICON (M1+)". This seems like an oxymoron
woadwarrior01 18 hours ago [-]
Ironic that the app is named Nativ(e) and yet bundles a full Python runtime. Nonetheless, still less bloated than LM Studio, which bundles a full Python runtime and electron.js (which in turn bundles a whole browser runtime).
bigyabai 17 hours ago [-]
A lot of macOS apps are statically-linked, even interpreted programs. It's still a native app for going that route.
khurs 5 hours ago [-]
Flagged this, as it is not 'frontier models' in the title of the linked page so not sure why has been titled that way here.
It's a further way to run mlx models
mlx are apple specific format for M3 or later CPUs, and some benchmarks show mlx are not always better than just running generic ones.
noja 4 hours ago [-]
Can you add support to download gated models?
flyingcapabara 7 hours ago [-]
If you are on Linux check out Box, it runs models locally , has img gen etc and many more features, I will be releasing the source soon just polishing out last bug's, help porting to other distributions is welcomed
Github.com/jegly/b0x
TechSquidTV 12 hours ago [-]
The server wont start for me.
"ERROR: Application startup failed. Exiting."
"mlx-vlm-server stopped with status 3"
diimdeep 2 hours ago [-]
I am still rocking Sequoia and this targets Tahoe purely from UI constraints, no like.
sajithdilshan 18 hours ago [-]
Has anyone found a model that can run on a normal macbook? I have an M3 Pro with 18GB of memory and whenever I try to run even a basic model the fans goes off and the mac starts to get heated up and becomes so laggy.
I just run Gemma4 12B MLX via Ollama and it's been doing fantastic work!
kzrdude 18 hours ago [-]
If you go small enough it should be no problem. For example Gemma 4 E4B in Q6 or Q4 quantization should run well on your laptop. It shouldn't be too taxing, but would still want to eat 7-9 GB of VRAM or so.
Now that model is mostly useful for writing or chatting.
b3ing 16 hours ago [-]
You need more RAM, plus the models take up a lot of space. 32gb min but I’d recommend 48/64gb, you won’t get close to frontier but it’s still fun to play with, images are very good
fl0id 16 hours ago [-]
heating up is normal, that cannot be avoided. it should become laggy, but you just have very little RAM (I assume 16 GB?) So most models are too big with other stuff running.
sixtyj 18 hours ago [-]
Tbh, the only reason to run a model locally is when you want to be completely safe.
From productivity point of view, it doesn’t make sense to have any notebook running a local LLM.
Something nobody needs is pointless hot air like "Everything you need. Nothing you don't."
thejazzman 19 hours ago [-]
You’d be surprised how hard this actually is. I spent 3 days iterating on a marketing site, where I had very explicit / “well written” copy, and it would just repeatedly rewrite it back to the most awful slop. Over and over again! Ended up adding various AGENTS rules telling it to leave the copy alone
jmpz 19 hours ago [-]
Definitely easier and better than just writing the copy..
lantry 19 hours ago [-]
I think they're saying that they _had_ written the copy themselves, but were using the clanker for other tasks, and it kept going off course to "improve" the copy.
I did. And then it rewrote it. Over and over again. And I kept restoring it. And it kept taking agency to change it to some other neutral slop.
That’s the point.
para_parolu 18 hours ago [-]
I don’t think anyone ever looked at result. Website if full of overflow bugs. It’s pure slop.
sajithdilshan 18 hours ago [-]
true, it doesn't render properly on mobile. Although I quite like the design. It's a new AI design guide (font/color choice) I haven't seen in in other AI designed pages
15 hours ago [-]
jarek83 18 hours ago [-]
Ok, so we're at the point that even design are just one-shot by AI. This is exact look and feel any time I ask it to present a HTML doc about anything.
moostii 17 hours ago [-]
The webpage isn't the point. The software the webpage refers to is the point.
rvz 20 hours ago [-]
Looks like my call [0] for more competitors to Ollama has been answered.
We need more like this as well as llama.app, which also has a native mac app.
Agreed. Just based on this not being Ollama, so I will give it a try.
0gs 19 hours ago [-]
my thing is kind of an Ollama competitor (surrogate?) too. more for prose/text planning, not so much for coding, at least the harness, but i'm sure someone could set it up to do that: github.com/0gsd/enough
Atlas_Smith 7 hours ago [-]
[flagged]
troygentic 6 hours ago [-]
[flagged]
VaporJournalAPP 15 hours ago [-]
[flagged]
troygentic 8 hours ago [-]
[flagged]
audioh4cker 18 hours ago [-]
[flagged]
flyingcapabara 7 hours ago [-]
[dead]
brcmthrowaway 19 hours ago [-]
[dead]
iAMkenough 20 hours ago [-]
How does this compare to LM Studio Bionic?
asqueella 20 hours ago [-]
Bionic is an agent; this appears to be an open-source competitor to lmstudio. The initial commit is just a few hours ago though, so …
calumcl 18 hours ago [-]
The maintainer works on mlx-vlm so he does have pedigree in the scene, I'm don't know if this is just going to end up as unmaintained slop. I haven't tried this but I would personally just recommend oMLX for a currently more complete and fleshed out package - loads of features, provides the same MLX support and changelogs + commits are actually detailed.
jdiff 17 hours ago [-]
I use oMLX and I'm tentatively going to be giving this a shot. oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures and fallbacks that are only visible buried deep inside logs when they should be announced out loud.
The maintainer of mlx-vlm being behind this as well is the main thing kicking me over into trying it, even if it is incredibly young. I'm confused and unenthused to see it chomping on a whole GB of disk, but the Swift makes it feel much more refined even if it's not yet as feature rich. It automatically picked up the existing models I was using with mlx_vm directly, which was nifty.
crefiz 3 hours ago [-]
Why do ppl downvote this comment? HN makes no sense sometimes...
Possibly vibe-coded landing page notwithstanding, the app is mostly written in Swift language. That suggests it will be easy to port this inference stack to iPad and iPhone.
https://huggingface.co/Qwen/Qwen3.6-35B-A3B
https://ntindependent.com.au/scientist-says-ministers-pole-f...
(https://arxiv.org/abs/2411.07641)
And if you do care to support modern samplers, you can start with the following:
1. https://arxiv.org/abs/2509.23234
2. https://arxiv.org/abs/2509.02510
3. https://arxiv.org/abs/2604.11012
TBF mlx-vlm does support min-p sampling, but none of the other modern samplers that you list. Ollama and LM Studio are even worse with only top-p and top-k samplers.
Look it up, it’s a bit difficult to explain concisely in words but it is intuitive visually.
If we are thinking of intelligence and price, a model will be in the Pareto Frontier if there’s no cheaper model of the same or higher intelligence. Or if there’s no more intelligent model for that price or lower.
EDIT: See this chart from Artificial Analysis: https://artificialanalysis.ai/#intelligence-comparison-tabs
So for example DeepSeek V4 Pro can be considered a frontier model because there's no cheaper model that is as intelligent.
For any solution in the Pareto Frontier, there no "no-brainer" alternative, in the sense that there's no other option that is better in some way without giving up something else. It's the best of its "weight-class".
https://arena.ai/leaderboard/text/pareto
EDIT: Indeed looking at the overall rankings for text again, the list is rather strange, a lot more about writing style than intelligence.
Also, the title is "Run AI models locally on your Mac," not "Run frontier open models locally on your Mac."
But (off on a tangent) I do think that there are multiple frontiers generally — and I also think the open weights, small local model frontier is by far the most important and exciting one.
I keep mucking about with what Gemma 4 12B can do and every time I do I find myself thinking that all the energies in the AI world are going in entirely the wrong direction, because it is small, clever, efficient and remarkable.
If all of that research money were to be spent on improving AI models that fit inside a 16GB RAM machine with unified memory, I think really important progress could be made.
I enjoy using the Qwen 3.6 models (and BottleCap's new fine tune of the 27B) but the small Gemma 4 models are impressive in a way that I think is going quite unreported.
So while I don't think this website should use the word "frontier" here, even referring to Qwen 3.6 27B which is weirdly close, I think it could.
Don’t expect much for coding. But it’s great for general knowledge, rubber ducking, image classification…
And I find it curiously interesting when talking about photography. I've been finding it intriguing to ask it about my own photos and make suggestions about other images to research. I just showed it three of my own photos, and asked it to analyse them and recommend photographers I should research. It recommended someone amazing I have never heard of before. But it also recommended a 19th century British photographer who happens to be my lifelong photographic hero — someone whose broad characteristics inform what I do without me slavishly copying them. Bit of a jaw-dropping moment for it to have picked up their influence in subject matter that they would never have approached.
I'm still suggesting it more to people for them to see what the small-model future might look like, because it's so much more capable than one might expect.
I think it's likely the 26B QAT model won't fit in your machine — you may be able to fit one of the UD_Q3 or UD_Q2 variants but whether you'll be able to run other things you want at the same time, I don't know.
(The QAT models are "quantization aware training" — AIUI the model weights have been assigned during training to survive four-bit quantization with less loss.)
So what I would recommend trying is this model:
https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF
Try UD Q4_K_M maybe.
(I don't think the M1 really gets much benefit from MLX, in case you were wondering, though I could be wrong)
My interest in this model is largely to really get to grips with what small models can actually do, especially with tool calling, because I think it helps comprehend what the value proposition of the cloud models is.
I have been very surprised by the quality and clarity of its answers. It's also helped me understand that much of a typical harness system prompt is likely to be unnecessary now; Gemma 4 seems to be pretty sensible out of the box.
You're absolutely not going to be able to get it to go off and build whole apps from a long prompt; it is not that good, but it does tool calling and thinking, and you should be able to explore pointing a coding harness at it if you turn on LM Studio or Unsloth Studio's API server. You could also use the Llama system tray app (formerly LlamaBarn) or just use llama-server from the llama.cpp distribution.
Probably Pi is going to be a better harness because it can have a minimal system prompt, though I've not tested it with Pi myself.
It seems to know PHP and SQL to a fairly decent depth (and I suspect JS and Python). It also has a unified vision model (it doesn't need a separate mmproj sidecar thingy) that is fairly fast, and it is quite impressive at image analysis.
So you could probably use it to generate image descriptions and tags, summarise text, generate wordpress snippets, that sort of thing.
It can capably answer questions like "Can you characterise this image and suggest further similar images I might like?" — I am currently using this to provoke me to take photos again.
Have a play with the E4B edge model, too — again, much more interesting than I expected.
Pareto frontier ≠ frontier.
Does it? I read: "Run AI models locally on your Mac."
Which of the models it puts on your desk do you consider frontier intelligence?
It also omits Open WebUI. I've been running Deepseek V4 Flash locally on my Macbook Pro for weeks using Open WebUI + DS4.
This is a roundabout way of addressing LM Studio.
Are people actually using them in coding agents? Or are they mostly using them for other things?
That said, if you have a subscription to a paid model already, you're not necessarily winning out on anything except perhaps privacy, which isn't nothing.
sometimes they fail but undo is just a "git restore" or if automated, rejecting a PR and having a better model take a crack at it.
Moving onto llama.cpp I was able to get faster tokens with MTP and a more reliable llm.
I wonder what other people's experiences are using MLX vs llama.cpp
I am fully prepared to believe the benefits accrue more to the M3 and up (because of changes to the Apple Neural Engine).
But with the models I've tested, unless I am missing something, the performance of GGUFs in llama.cpp has been better in some cases.
I still have not had results from Gemma 4's MTP be really worth it, to be honest; but with the Qwen 3.6 MoE it is measurable. Maybe with newer kit it is more meaningful.
(There is every chance that the above is not the experience of anyone who really deeply knows what they are doing; it feels like I am a perpetual novice at this stuff)
https://news.ycombinator.com/item?id=48910545
Perhaps they got tired of LM Studio, etc., not being able to run their models properly.
(I initially thought the same because of the website appearance)
It's a further way to run mlx models
mlx are apple specific format for M3 or later CPUs, and some benchmarks show mlx are not always better than just running generic ones.
Github.com/jegly/b0x
That said, your computer will still get hot!
Now that model is mostly useful for writing or chatting.
From productivity point of view, it doesn’t make sense to have any notebook running a local LLM.
We have one life. We should spend it wisely.
Just state the information you want to communicate in the plainest and most straightforward way possible.
0: https://en.wikipedia.org/wiki/Performative_contradiction
That’s the point.
We need more like this as well as llama.app, which also has a native mac app.
[0] https://news.ycombinator.com/item?id=48968898
The maintainer of mlx-vlm being behind this as well is the main thing kicking me over into trying it, even if it is incredibly young. I'm confused and unenthused to see it chomping on a whole GB of disk, but the Swift makes it feel much more refined even if it's not yet as feature rich. It automatically picked up the existing models I was using with mlx_vm directly, which was nifty.