I lament the comments saying this in any way redeems Meta (the company).
The researchers releasing this stuff have almost nothing to do with Meta other than being bankrolled by the slaughterhouse.
You aren't the customer, you are the pawn in big tech's game of thrones. Your good will is a commodity to be traded, almost literally. It will be used against you the moment it's convenient. This is open weights because Meta couldn't monetize it in any other way than to cloud developer's judgement of their reputation.
But I guess most people just don't care.
I'm glad it's open. It does not make me think any better of Meta.
It’s rather amusing to me to read comments like this, and then simultaneously whenever a Chinese company or team releases open-weight models or whatever there is a giant round of applause, America is so behind, and there’s nothing but positive things to say about the intelligent, creative, and well-intentioned Chinese engineers (which is true, America certainly doesn’t have a monopoly on great people). Don’t you know? Only China can release good, open weight models and American companies can’t compete. Oh by the way all the spend is for nothing because China alone can release open-weight models thus destroying American AI.
When an American company does anything? Doom. And. Gloom. The engineers? Taken to the slaughterhouse! America? Behind! The public? Bamboozeled!
> This is open weights because Meta couldn't monetize it in any other way than to cloud developer's judgement of their reputation.
I’ve been told over and over this doesn’t matter. Just needs to be cheap and open. Or maybe that’s only when Chyna is involved?
Sorry this post is a bit snarky but it really is something to behold. And certainly I don’t know the OP’s opinions on Chinese open weight models. Perhaps they agree with me.
Good points, I personally believe that if/when China takes the lead, they will immediately stop releasing model weights. It only makes sense as a strategy to counterbalance (current) American labs' monopoly on frontier models.
Holding both those positions would be hypocritical all right, but are you sure it's the same people commenting/voting in both cases? I don't think there's a strong consensus on Hacker News. Even something like the time of day an article is posted might get different engagement depending on who is active in which time zones.
> I don't think there's a strong consensus on Hacker News. Even something like the time of day an article is posted might get different engagement depending on who is active in which time zones.
Based on my own experience and reading, I do think there's a general consensus on this site but I could certainly be wrong about that. I'm less concerned about hypocrisy per se, it's more that the arguments that are used, even if by a minority, seem to apply in only circumstances in which China releases open-weight models.
I am aligned with your viewpoint as well. And I've repeatedly argued it. If China were to take the lead the US can then just release open-weight models. Folks say having the lead doesn't matter because China releases cheaper open-weight models. We can just let them take the lead and then do it back to them.
I see it as a strategy to increase capital costs for American frontier labs. By eroding the expected ROI of frontier labs, you deter private investment into them, which slows OpenAI and Anthropic in particular, giving space for the laggards to catch up.
I don’t think Meta is bad for releasing open models, but are you really going to ignore all the terrible things they’ve done over the years just because of that?
As for DeepSeek or any other Chinese lab, I’m not aware of any practices that would make me consider them a bad actor. Can you say the same about OpenAI, Meta or Anthropic?
There is a big astroturfing going on social media platforms by the chinese. Did you notice 'day in a life of unmarried 30 yr old lady' videos flooding usa social media.
Regular ppl in the west now hold mildly positive views of the ccp and how 'advanced' china is than usa.
Then there are europeans who now are looking for china to give them the technology handout now that relationship with usa has soured.
Meta can never be redeemed, but it's still valid to admit that FB at one point had a very badass engineering culture.
They're one of 2 companies I would absolutely never work for (weapons etc aside). FB's recruiters hounded me so often I requested that they blackball me. The day they became Meta, I learned this by checking my email to see that they started trying to reach out again. I once again requested that they blackball me. This by extention taints OAI, the other company I'll never work for.
After a few hours with Glimmer I'm pretty impressed. It's better than the benchmark scores seem to indicate compared to Qwen 3.6 27B. I'm very excited for 3.8
Meta doesn't need to be "redeemed". They have two of the most popular social media apps in the world. And theyll prob survive without ever having you work there
I'd also argue this is the case for any company releasing open weights. They're not righteous, they're marketing. That's not necessarily a bad thing! They're releasing some great stuff for free and we benefit from that. Every company doing this has a motivation to not release these for free.
Alibaba, Google, Moonshot, Thinking Machines, etc are not releasing their models for free because they love to. They want to grab market share. I'll take it.
I still will not use a hosted Meta product, but damn this model looks solid.
Muse Glimmer doesn't redeem Meta, but it's a contribution to the commons and the Apache 2.0 licensing is an improvement from the restricted licenses attached to Llama. If even Meta can use a permissive license for its model weights, so can any other company.
Thomas Bayes would say that the population of people who hate Facebook is really big and the population of people who are scrupulous about whether or not their comments are specific to the matter at hand is relatively small
Unfortunately there are a few topics that short circuit some terminally only people. One of them being anything related to meta. Few others recently emerging is Flock or Musk. It's really exhausting since you can't have a discussion relating to anything that may be adjacent to said topics. It's like a black hole.
This is par for the course, HN is far worse than reddit on balance, especially involving upvoting/downvoting decorum.
Go vibecode something to auto upvote all downvoted posts, call it "Antiechochamber.HN" or something, and if enough people used it this website might improve a bit.
To be honest the main issue with meta has never been around open/closed software. They've also done react, Cassandra and some other bits. But this, like their open weights is like a feather pressing down on the scale compared to things like promoting genocide in Myanmar, enabling Cambridge analytica, creating a huge closed ecosystem which dominate(s/d) local community communication, mandating doxxed communication, trying to replace actual community communication with algorithmic nonsense etc.
Any retort to do this like “but why would they just openly release this”[1] pretty much answers itself. Public relations.
If a company can spend money to redeem itself then, well, it can (game theoretically or whatever) do whatever it wants in the future and then spend money to wipe the slate clean.
[1] By which I mean: the very act of being prompted to ask such a question, of planting a seed like hmm, Meta might have some aspects which are good for us. You don’t have to be convinced of it. Just the seed itself can pay for itself.
Meta and its products, as a whole, is a threat to your kids, your mental health, your community's health and the planet as a whole. It is just sad and very repulsive everyone fell so easily addicted to their social drug. Yes - it is a drug, and it is hard to get off from.
Nothing redeems them at this point of time, they are doing exactly ZERO to redeem. Tossing open weight models (not opensource!!) is not a basis for redemption, and does not constitute remorse in any way. Trying to portray it as such is complicity to META's crimes against humanity.
Based on the benchmarks, it seems that Muse Glimmer barely edges out against Qwen3.6 27B, except for tool-calling skills (MCP, etc.). I wouldn't be surprised if they released it now because they are afraid they wouldn't beat Qwen3.8 27B.
Do AI companies make release plans based on upcoming other models like this? I would think all the processes that go into the repository and weight infrastructure pre-training, checkpointing, knowledge distillation, model compression, post training pipeline, ecosystem integrations, inference API, benchmarking, human eval/safety/alignment, docs, etc... all that dictates the release schedule.
Any company working in a competitive industry is generally aware of what their competitors are doing. PR is an important aspect to market success, so it factors into release schedule. It may not be the dominant factor given engineering constraints, but yea, it’s certainly a factor, and a large one at that.
Yes, not every model release is reactionary to other labs. Either they had hints for the release of other models or they cut efforts in late stage testing of the models to hit these earlier release dates. There’s always some flexibility. And there’s certainly the incentive to cannibalize the news cycles for competitor models.
I could imagine pulling out all the stops to get a release over the finish line a week early if you're worried about being surpassed by another release
Yeah but you can probably have everything ready and then accelerate as necessary. Meta itself did this when releasing Llama 4, it was a really botched release right when they were feeling the heat from DeepSeek and others.
There has been a long history of AI model releases made shortly before or after a major planned release by another company. Almost always to upstage or steal thunder.
Just recently, Minimax H3 released as open weights on the eve of Seedance 2.5 global availability. It's not as good, but it's good enough and it's completely open.
Flux 3, which is nowhere near as good as either, suddenly announced their release once news of these other two became public. They knew if they waited they'd be ignored. It didn't really help them much, unfortunately.
The LLM releases are even more rivalrous.
And don't forget all of the competing launches planned before Google IO or major release events.
Companies like to eat into the news and press cycle of their rivals.
I've seen it here on HN (it's particularly noticeable via the /active page) multiple times. If Google, OpenAI or Anthropic release something significant, odds are good you'll see a headline from one of the others.
If you start counting since WaveNet or BERT, it's been ages. Especially when it feels like decades of advancements happen every single year, and rival labs are always trying to one up each other.
the last few items there (benchmarking, human evaluation, docs) can be rushed or skipped by leadership if they want to beat comp. they probably spend a few weeks on those things normally
One window that can be shortened is working with software ecosystem and upstream partners; think day 0 on together, fireworks, Unsloth, etc. That obviously happens from partners getting embargoed weights early.
The tokenizers are included in the open s̶o̶u̶r̶c̶e̶ weights releases; you wouldn’t be able to use the weights without the corresponding encoder/decoder, in fact.
I've been using Qwen3.6 35B A3B, and with reasoning turned on, I'd say 2/3 (give or take) of the tokens for a response are thinking tokens. Which at 70+ tps locally, that isn't that awful. I run an 80k context across 4-10 "agents" for my solo TTRPG, where Qwen is the GM, each NPC at a location, the director, and the narrator.
Each turn is about 45-60 seconds to generate all of the various responses. The GM and director have reasoning on, and the NPCs/Location/Narrator do not.
It's a fairly good "engine" for that. I'm not sure how a denser Qwen would do here regarding speed.
Just to play devil’s advocate: you can’t compare Qwen to a (proprietary/closed source) hosted model and deduce that Qwen is overthinking, as Qwen gives you the full reasoning/thinking trace while all the proprietary models now give you only a summary “to prevent distillation”, making it hard to properly compare apples to apples here.
Quantization awareness doesn’t change the size of the weights, just means it won’t degrade when quantized. QAT = quantization aware training. They will both be very similar in size at the same quant.
You're mixing up sizes of different quants. The 60GB is unquantized, and Qwen's unquantized size is around 54GB. Their sizes as like quantization levels are similar.
Considering how all the big players are playing fast [1] and loose [2] with limits, billing [3] and adding undisclosed changes that burn your tokens on autopilot [4], it can't happen soon enough.
Not to mention all the other ways they can screw you:
- Middle of the day, servers busy? Swap to Sonnet while pretending it's still Opus. Many people won't notice, and nobody can prove anything if they suspect.
- Middle of the night, server load is light? Put it into extra thinky mode so it burns more tokens to ramp up the bills. Flip the switch where it gets really pedantic about writing lots of extra test cases and verifying against documentation.
- Demand increases, but don't feel like running more hardware? Switch to low bit quants, but have a monitor model swap back to quality if it can tell you're running a benchmark.
Assuming model capability plateaus (I think it will), token providers will be in a race to the bottom to maximize profits at the expense of quality that's very difficult to measure.
It all sounds like having to rely on a dodgy housing contractor that wants to steal from you, take shortcuts AND choose the gold-plated options from their supplier friends, and will start doing this the minute you are not on site supervising.
You don't do it yourself (because the contractor is faster and stronger than you in many ways) but you can't leave, so you're stuck on the worksite just watching them.
It's worse though, because you can't really watch them at all. It's very difficult to get quantitative numbers for quality. Even within the same model family, same tokenizer, and complete control over the weights and logits, perplexity and KL-divergence isn't really what you want. Now put it behind an HTTP endpoint, and it's just opaque.
I've seen local models recognize when the task I'm asking them for is likely to be an artificial benchmark.
And any smart company is going to use lightweight models to monitor your sessions. If their sentiment analysis suspects you're close to cancelling, they'll up the knob for a few days until you calm down. Or worse, their accounting tells them that you're getting too much value from your fixed price subscription, so they turn the knob down to encourage you to cancel.
In the short term, the "frontier" models are too good to ignore. But if (when?) that plateaus, I don't see how anyone could trust a non-local model. When you pay an ISP to serve your web site, you can tell if they over-compress your images to save storage and bandwidth. With LLMs, it's just JSON with more errors and pointing to the fine print that models are not deterministic.
I've got nothing but hand-waving, but after you've extracted all the smarts from every piece of text ever created, how do you get more?
Alpha Go had a game where the models could compete against each other. That let it become super human. What's the intelligence game we can create for LLMs? Even if you invent something, will it make the model smarter in a way the market values enough?
Then there's a race to use the weights more efficiently, or to offload information that shouldn't be in the weights in the first place (Karpathy's Cognitive Core). I like to imagine we train the models in something like Lojban, have a lightweight model translate from human language to that, and you can update the Sqlite or Postgres store it uses for knowledge.
And there's no barrier to entry for agent harnesses. So whatever loops or recursive orchestrated council of elders idea comes up, that won't protect the monopolies (duopolies).
Anyways, depending on your definitions, I think we'll hit AGI, but I don't think we're getting a Singularity this time around. Again though, this is all just hand-waving.
Why do you say API based llms looking iffy at best? Do you just mean current profitability due to market pressures from some companies’ subsidized investor money?
Surely, even if you’re just using open weights models, it should theoretically be cheaper to use them in a highly optimized cloud architecture(even with vendor markups) rather than each person serving their own models from much less efficient (and more importantly, much less consistent volume) self-owned “server under your desk”?
What do you mean iffy? The major AI labs are gross profitable when selling access to inference. In addition, the best models have trillions of parameters and are most efficiently served on large, expensive clusters and served to many concurrent users.
They make money on each token when you look at the electricity and interconnect fees, but no, I don’t think they’ve turned a profit on their Capex, even a little bit
Can I ask, do you feel the pain of the level of abstraction? I haven't tried local in a few months, but last time I tried, I felt like I was directing a coding exercise - whereas with a frontier model, it feels more like directing a product building.
"I need this feature", vs "write code to do this in this file".
That’s part of it. There’s also just a natural back-and-forth between what I call “time sharing” and “personal.” When the thing you want is expensive, you share it remotely, but as soon as costs fall, everyone wants it under their desk.
With how expensive consumer hardware is and will continue getting (due to LLM demand), good luck getting a "server under your desk" for something less than an arm, leg, and first born.
Until A100 prices are reliably under 1.70$ an hour, there is no GPU/AI bubble and Michael Burry doesn't know anything about GPUs.
There are lots of points in a spectrum of choices. DGX Sparks, Strix Halos, and the surviving Mac Studios can easily run these 30B class models, just not as fast. So maybe just the leg, but you can keep the arm and first born.
And super noteworthy is that a 27B model (Qwen 3.6 27B) from this year is a huge improvement over a 120B model (gpt-oss:120b) from last year. The goal posts are moving, but at some point "good enough" is good enough for the kind programming I like to do.
Some interesting findings from the chat template designs:
1. The template name is Onyx ATEM as found in the tool call exception message
2. It appears to be following a harmony-style chat template. But the tool use seems to be a xml like :<atem:function_calls> / <atem:invoke> / <atem:parameter>
The XML tags are similar to <antml:xxx>, which is obviously Anthropic ML (or ANTrophic xML).
I think it’s likely 3; meta in reverse. While tokenisers and preprocessing can catch it, you want your special tokens to be unique and not present in the original corpus. <meta: is likely too common.
It is interesting but it does look like a careful distillation of (Spark and) biggers open-weight models.
The progress compared to Qwen3.6 27B is good, not that impressive, it's a 4 months old model. (kuto to them to compare to 27B dense and not 35B MoE, it's more fair to do so).
It is very probable that Qwen3.8 27B will crush Glimmer-30B on most benchmarks.
I'm so excited about these two new models. Qwen 3.6 27B has been my sweet spot so I cannot wait to try 3.8. Glimmer looks really strong, I'm encouraged that Meta compared it to 3.6 in the model card! Exciting times!
"... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..."
This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no problem being #1.
Still needs 32-64GB memory to run it locally. 64GB Macbook pro with an M5 chip costs more than 4k Euros in Germany. A more practical model would be a language specific (e.g Python or JVM language) and excellent at tool calling and reasoning. Maybe that way they can shrink it even more.
I think if there's going to be advantages to making smaller, more targeted models, those advantages will probably come from targeting specific domains, not from targeting specific languages.
I think that if an LLM can't abstract over the differences between Python and C++, it probably will have an even harder time abstracting over the differences between writing code that manages a webserver, and writing code that does aerodynamic simulations.
There is no good reason to believe language-specific models are going to be any meaningfully smaller, just worse. Same as English-only models vs those trained on a multilingual corpus.
I would love to see any good research projects about it but i have the feeling that Frontier with MoE is making too fast of a progress so that a customized model would always be worse and that the MoE part is actually going somehow in this direction.
On the other hand, at the GTC was a talk about coding in different lanugage (like spanish) and explaining that the quality between spanish and english is relevant different.
But i have not found a good article about the impact of learning data with practical experiments or even if the order of the learning data matters.
At least I think i remember that Meta mentioned having better and less data can be better than more data with lower quality.
As long as these models can explain to you facts about any other topics, its still overfitted for the task though.
Capability in LLM's is distributed throughout the manifold in subspaces. Even worse, the subspaces exist in superposition.
That is to say, there is no single 'python' part of the model. The python bit is spread throughout the entire model and overlaps with other pieces that have similar, but unrelated, capabilities. For example the python subpspace might be partially in superposition with cupcake recipes, Esperanto, and calculus. We need calculus in a coding agent but not the other two. However, separating them cleanly is almost impossible, and even identifying them is tough.
Internally the manifolds are highly inefficient and nothing like you would imagine something humans built would be designed. It's more like something that evolved in nature.
With MOE you train a router designed to select which parts to activate. The router itself is a trained neural network and the 'experts' are usually not really things like 'python'. They're just the functional subspaces I described above.
Again, those subspaces are all somehow inextricably correlated and live in complex superposition spread throughout the manifold. The router doesn't know (or care) WHY those sections get lit up it just learns which ones to activate to optimize it's own reward function. So maybe it learns to activate "logic", "python" and "cupcake recipes in esperanto" whenever it see's something that kind of looks like python. It's not the best answer, it's just the best answer the tiny router could figure out.
It's all wildly complicated and inefficient, and works nothing like any reasonable human would imagine that it SHOULD operate.
There was some paper about routing at training bio-knowledge into a particular region of the model, which you then can cutoff when serving. But you probably lose some efficiency since maybe you sized that region too small/too big.
I'm sooo happy I pulled the trigger on upgrading and getting a new laptop (with 64 GB RAM) last summer. Feels like it was just in time before the exponential price jumps.
I was about a week away from buying a very tricked out MacBook Pro with 128 GB RAM, but was on vacation and worried about it arriving while I was away, and then the price hikes went into effect. Grumble. Oh, well. Serves me right.
That commenter you're replying to knows that. The original commenter before them wrote "pulled the plug" which is different and doesn't quite apply here (actually implies the opposite of what they meant to say).
If you can’t do it cheaper on your own hardware it does make you wonder how much of the cost of inference those large LLM providers are eating? Datacenter hardware isn’t magic.
Your personal hardware probably isn't running useful tasks 24/7.
If you spend 60% of your 8h work day on full on agentic work, then your hardware is paying off for itself only 20% of available time.
I feel like we’ve had this discussion before. From what I remember, specialized models rarely do that much better than general ones, hence no mode Codex models.
Even if you had a 64GB machine: Are you willing to reserve 90% of your memory to run a LLM? With dirt cheap models like deepseek-v4-flash that will run "forever" on $10, the answer for me is clearly: no.
I'm waiting for the speed/quality per dollar metric to go down a little bit further and then I will def run it at home.
Its not just that you send a sentence to an API endpoint, you always send EVERYTHING to that agent as a context.
You want to analyse your spending history? You now send everything to someone.
Either no one cares but understands this implication on how easy it is to really capture you or no one really things about it.
But i'm a lot more diligent on what I send. I disabled the gemini activity feature for example because google started telling me that my stuff could be reviwed by humans.
Yeah, it does feel a bit silly with my encrypted disks, encrypted backups, unique passwords, advanced router, etc, while I send everything I do in plain text to anthropic.
Deepseek flash is open weight, this means we can download and run that model without any connection to deepseek, no data/tokens/usage data ever reaches them. They cannot make us their product.
I see many people saying deepseek and other chinese providers have always been profitable. Also they show their training costs publicly. Can't say for sure since I have not used it personally, but I think they'll for sure outlive the western SOTAs.
OpenAI apparently runs a profitable inference business with 40% gross margin, but their advertising budget is nutso and their real costs are pretraining and research. I suspect Deepseek's comp is not predicated on capturing the lightcone of all future value, some googling insinuates their top pay is $212K US which would support that suspicion. Compare and contrast with the $1.35M and up at OpenAI.
Ah yes, I'm sure Trovalds and Stallman are harvesting my data through free software, aren't they? This argument is used by boomers who were fed cold war era propoganda that surely everybody is selfish, and you're always at fault.
Instead of saying "I have a MBP with 64gb of RAM" you'll hear people say: "I'm subscribed to Model 9.x11B" and others will comment: "Oh dang, that's a nice model!"
I don't understand the desire to run own AI models for programming locally. No laptop is ever going to be as powerful and energy efficient to run anything close to OpenAI, Anthropic or Google models. A model you can run on a loptop is simply not going to work as well as it's needed for programming. Small models for linguistic work fine, but anything more sophisticated simply won't provide enough resources or power. Or models would need to be significantly dumbed down - then why use them at all? So far the idea of carrying a "thin" or "thin"-like device looks more reasonable to me, while running AI on your own server.
I’m quite optimistic about the long-term future of local LLMs for privacy and cost control reasons. An LLM running on my own hardware, even if it’s not a laptop but a home server, is one where I don’t need to worry about token limits, token fees, privacy, and “rug-pulling” from the vendor.
In the short term, the big challenge is being able to afford hardware that can run a ~30B model. Last month I got to experiment with LLMs on a NVIDIA RTX 6000 Ada Generation as a visiting researcher during my summer break. I see the power of local LLMs for agentic coding; they’re no Claude, but they are quite useful. I wish I had gotten into local LLMs before hardware has gotten prohibitively expensive and in some cases unavailable; Apple discontinued certain Mac Minis and Mac Studios with high amounts of RAM due to the RAM shortage.
Hopefully high RAM prices don’t become a new normal, though the next year or two doesn’t look good.
> A model you can run on a loptop is simply not going to work as well as it's needed for programming
The models you can run on a high-spec laptop today are approximately where frontier models were 12-18mo ago (albeit at a lower tok/s rate). If you scan back through hn comments from that era, you’ll find plenty of people saying “this is powerful enough to massively increase my productivity”.
Not always! I get 80-100 tok/s from Qwen 3.6 35B-A3B on a MacBook Pro thanks to MTP. With long contexts that dips to around 50-60. However, prefill is much slower than API models. So it becomes really, really, really critical to not have cache misses.
I've never done it but would be interested because it cuts out the burden of worrying about costs. Maybe I'm mistaken on energy cost here. There's a constant raincloud that follows me around regarding limits, and it would be nice to shake that.
I've been able to accomplish incredible feats (for myself) since GPT-4, so model intelligence is secondary.
For some companies there might be a need to run them locally. For instance, Apple decided to run LLMs on the phone locally. I guess it depends on how important latency and privacy are. Perhaps Meta is looking at how much interest for those local models is there.
That's a bit amusing - not that I have the hardware to run it, but officially it's not available in Hong Kong. Not that getting it would be much of a problem with a help of a VPN either, but I'll assume mainland China is also restricted. Certainly not a competition for Chinese open weight models... in China.
What I think would be perfect is a model that could run on a single DGX spark and be competitive with DSV4 Flash 731. Flash is already a game changer. Hopefully meta plans on this, like the old 70b. V4 flash is smart enough for any use but slightly too big. 27b-30b isn’t intelligent enough.
This model I think will be too slow for that on Spark, even at 4 bit quant.
It's a dense model, not MoE like e.g. Qwen 35b or Gemma 4 26B A4B. On a Spark it will be memory bandwidth limited
I haven't tried yet (working on it) but back of the napkin estimate puts it at around 15tok/s even after converting to NVFP4. Prefill would be much higher though. That 15tok/sec is pretty typical for dense models of this size:
NVFP4 Q/K/V/O and MLP projections: ~13 GB/token
BF16 attention gates: ~3 GB/token
BF16 LM head: ~2.5 GB/token
Total: ~18.9 GB/token
At 273 GB/s, that gives a bandwidth-only ceiling of about 14.5 tok/s; actual performance would be lower.
You're right. I'm getting ~33tok/sec w/ dflash on it, even bursts up to 60tok/sec, using my personal home-built-for-Spark inference engine (not vLLM or llama.cpp based)
That's pretty respectable.
Still working on optimizing and cleaning up before I push it.
For the same price as a DGX Spark here (A$8499) I can buy roughly 544GB of DDR5-5200MHz from retail; which on a quad channel platform would deliver ~160gb/s real world; and ~320gb/s with octa channels (Xeon, Threadripper Pro).
If you can afford it or somehow find a used unit, you can go Epyc for 12 channels.
8/12 channel DDR5 will beat DGX Spark in inference/decode even without a GPU of any kind, as it’s memory bandwidth bound, and the Spark tops out at ~240gb/s real world.
With some optimisation and maths, it’s entirely plausible to ach
You are paying an extraordinary amount of money for the convenience of a super small unit, with still mediocre software support, but at least a community. Expect to be crawling through forum posts regularly, as SM121/Spark has many quirks and ecosystem issues still.
Please don’t pay another 70-80% gross margins on top of already inflated DRAM prices unless you need. The Spark IS really nice if you want to test out ConnectX or if you really need something small and compact and quiet.
Also consider: used Adas or even Ampere NVIDIA workstation GPUs can come with a lot of VRAM and be “reasonable”, with CUDA.
Been investigating these multichannel AMD based platforms last year and seem like none of them can in real scenarios utilize anywhere close to their theoretical bandwidth.
Optimizing speed is really the way to go.
Yet 24GB is not what everyone can afford.
Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.
Unfortunately this is just the entry price for LLMs. With the exception of the Qwen 27B models, I personally haven’t found a ton of use cases for models less than 200B. With the right setup, fine tuning, etc, you can make small models do cool things, but hard to please everyone given the insane hardware costs at the moment and the comparably cheap API costs.
Small models are still great for lots of “simple intelligence” use cases, like annotating or summarising files and media; or even just basic chat when given web search tools.
My local NAS is private and I’m not going to send it off to APIs for captioning or metadata; but even Qwen3VL 8B does an excellent job at this, despite being quite old.
They are also really excellent for fine tuning. Unsloth and Tinker (from Mira’s TML) are great places to start.
If your use case is narrower than “coding agent for everything”, you can probably match frontier performances on that narrow domain with ~30b and exceed it with ~100b+.
Three of these landed in the same week. Mistral's Shieldstral is a 3B safety
classifier that matches models 7x its size, and Google shipped Gemma Translator
which runs entirely offline. Different problems, same shape. Small open weights,
local, no API call.
some support already merged, and I verified in a local build that it runs (cannot get MTP params working tho, about ~40 tok/s on my beefy 800GB/s 7900XT w/ 20GB VRAM). https://github.com/ggml-org/llama.cpp/pull/26841
The least they could do, after ruthlessly bombarding my employer's servers with requests, ignoring the robots.txt, scraping everything, and incurring significant Google Maps costs for us in the process.
How are you handling the tradeoff between quantization for device fit and accuracy loss on tool calling? That's where local agents typically break down in production.
Another candidate for the 7900XT (20GB VRAM) I got sitting around. I pulled latest llama.cpp (targeting vulkan during build) after seeing a muse PR merged a few hours ago, and unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL runs on my 7900XT barely (and with no MTP). Sits at 19GB VRAM w/ 4 parallel 113k context slots, all layers on GPU, and at 700 tok/s prompt, and ~36 tok/s generation.
Waiting on Q3 to download to check speed + do my usual anecdotes. I generate beefy code snippets and poems, and also ingest my HOA declaration and answer nuanced questions.
edit: i should've prefaced this somewhere with: This card ballparks at 800GB/s IO, which I can't seem to find easily on the market anymore. Kinda the ideal card for this model, if I just had a _little_ more VRAM (XTX is 24GB).
edit2: not mtp, this is dflash model (param in child comment). I'm up to ~60 tok/s generation and sitting at 19GB VRAM (i added --no-mmproj (makes it text-only i believe) because I'm used to speculative decoding wanting more VRAM and I'm already close to the limit :sweat_smile:)
Q3 results: unsloth/Muse-Glimmer-30B-GGUF:UD-Q3_K_XL gets down to 15.6GB VRAM and full context (131k) on the 4 parallel slots. Prompt/generation speeds about the same. Overall feeling like a nicer-fitting Qwen 3.6 27B, but want to test out MTP generation speeds once I can.
edit: My favorite bit of reasoning I saw go by in my "generate me a beautiful code snippet" anecdote: 'Could give a snippet of beautiful code: the "hello world" in brainfuck? No.'
edit2: my first dflash speculative model! no mtp. I'm up to ~60 tok/s on empty context with `--spec-type draft-dflash`
> Some have tried to frame distillation as harmful, but I think it is important to protect the principle that you can learn from anything you can observe.
Meta did not abandon opensource. I would love to see a smaller distill, or a moe of this size but the benchmarks seems competetive as long as it isnt benchmaxed witch i would not be suprosed if it is.
I don't think outside of the Big 3 (Ant, OAI, GDM), given the strong competition from China, any other Lab has a chance at capturing the coding market if they aren't open weights (save for xAI whose latest Grok looks every bit good & will probably rely on Cursor for distribution instead of going open weights). There's literally no other selling point, as the capabilities have mostly converged by now among the chasing pack.
It’s not completely open source, but they actually release their pretraining and post-training datasets with some redactions for (cough) pirated content.
They also have very good code and playbooks for actually doing a fine-tune, CPT, etc.
Even if you’re not tuning a Nemotron model, its mixes are very excellent for your replay data slice; or general experiments. Way better curation and quality than Dolma, etc; or other large huggingface data mixes I tested.
Yes, I second Nemotron. I'm using Ultra remotely and Super locally, and I find them very useful for RAG-like problems. I wouldn't really use them for coding.
There was a good discussion yesterday on the DeepSeek Flash release thread about this.
There's a large market, very large, who want the best regardless of what it costs. Probably a large enough market to keep that domain of research afloat (as opposed to shifting research manpower to cost cutting).
The reasoning is just that the marginal cost of AI is very secondary to fixed costs of the businesses themselves; it's not an excuse to sacrifice performance.
Those two offer MoE variants, this doesn't seem to.
Dense model makes it dog slow on anything without HBM. Max 15tok/sec on decode on DDR5 systems like a Spark or a Strix Halo -- and that's at 4 bit quant.
Dense models run at a very usable speed (Qwen 3.6 was running at ~50t/s last I looked) on my dual 7900 XTX desktop. (And before anyone brings it up, I did not buy them for this purpose, so the up-front cost is irrelevant in my case.)
If there is anything meta can do to regain hearts other than owning up their evil deeds, radically change their business model and paying up for taxes and damages, then the world is truly fucked and corporations will continue to win.
The more open weight models get released the greater the market for personal and small business oriented hardware to run these models. This will drive lower cost hardware, which has stagnated in recent years due to most software not needing the performance and capacity.
I like this class of model. Multi-token prediction makes it viable to run dense models at not-too-far-off speeds as MoE models with much better intelligence.
The submission’s title (open weights 30B local coding model) is luckily wrong: This is meant to be a general agentic model.
It even comes pre-quantized and with a MTP/drafter model. Looking good!
Let’s hope they aren’t dishonest with the benchmarks this time …
"Meta Muse" immediately made me think of Metamucil.
Product teams really need to hire at least one or two people with a 12-year-old's sense is humor. They need to winnow all the potential stupid jokes out of their product namings.
As an industry, I wish we would stop calling these things "open weight" because it is too easy to confuse with actual "open source", which they are not.
Photoshop source code+ OSI license = open source
Photoshop binary you can run on your own computer = open weight
Photoshop SaaS web app = closed, proprietary (Opus, GPT, etc.)
"Open weight" models are still just binary blobs that are completely inscrutable. It's like bringing home a dog from the rescue and just hoping that it doesn't have a tendency to bite kids in the face. You just can't know. The only thing that you can do is try to add more training (fine tuning) telling it not to bite kids.
I don't think the FOSS community has ever accepted this, but somehow we're feeling like it is okay now.
Photoshop binary you can run on your own computer = open weight
I don't think this is a correct analogy. You are not allowed to distribute modified versions of the Photoshop binary. Most open weight model licenses allow you to make and distribute your own finetunes, etc.
Given an open weights model trained to sometimes bite kids, we can’t train it to not bite kids, even though billions of dollars of research have been thrown at this open problem.
Given an open weights model trained to never bite kids, you can get it to bite kids with 10 prompts and a linear projection, the known simple algorithm doesn’t even need a backwards pass.
I believe that comparing LLMs with traditional deterministic software is fundamentally misleading. It is extremely difficult to truly interpret what LLMs do internally, and as of now, nobody fully understands it. Even if you trained the LLM yourself, there is no source code you can simply read and learn from.
Sure, having information about how these models were trained is helpful for reproducibility, but it is basically impossible for anyone without substantial capital and access to the same (likely copyrighted) data to reproduce the model. For normal users, owning the model weights essentially means owning 100% of the model, you can inspect and study the weights in much the same way as the lab that produced the model can, you can modify the weights, and you can use and distribute them if the license allows you to
It is useful to indicate you can run the weights on your own hardware. That’s categorically different from most other commercial offerings. It’s as if your adobe example ignores the reality that would exist had photoshop been invented in 2019: cloud only.
I am extremely well aware of how rescues evaluate dogs. And I'm also fully aware that they do not know the full history of the dog. They go through a limited set of testing and interrogation to evaluate the safety of the dog. That's it.
The researchers releasing this stuff have almost nothing to do with Meta other than being bankrolled by the slaughterhouse.
You aren't the customer, you are the pawn in big tech's game of thrones. Your good will is a commodity to be traded, almost literally. It will be used against you the moment it's convenient. This is open weights because Meta couldn't monetize it in any other way than to cloud developer's judgement of their reputation.
But I guess most people just don't care.
I'm glad it's open. It does not make me think any better of Meta.
When an American company does anything? Doom. And. Gloom. The engineers? Taken to the slaughterhouse! America? Behind! The public? Bamboozeled!
> This is open weights because Meta couldn't monetize it in any other way than to cloud developer's judgement of their reputation.
I’ve been told over and over this doesn’t matter. Just needs to be cheap and open. Or maybe that’s only when Chyna is involved?
Sorry this post is a bit snarky but it really is something to behold. And certainly I don’t know the OP’s opinions on Chinese open weight models. Perhaps they agree with me.
Holding both those positions would be hypocritical all right, but are you sure it's the same people commenting/voting in both cases? I don't think there's a strong consensus on Hacker News. Even something like the time of day an article is posted might get different engagement depending on who is active in which time zones.
Based on my own experience and reading, I do think there's a general consensus on this site but I could certainly be wrong about that. I'm less concerned about hypocrisy per se, it's more that the arguments that are used, even if by a minority, seem to apply in only circumstances in which China releases open-weight models.
I am aligned with your viewpoint as well. And I've repeatedly argued it. If China were to take the lead the US can then just release open-weight models. Folks say having the lead doesn't matter because China releases cheaper open-weight models. We can just let them take the lead and then do it back to them.
As for DeepSeek or any other Chinese lab, I’m not aware of any practices that would make me consider them a bad actor. Can you say the same about OpenAI, Meta or Anthropic?
https://en.wiktionary.org/wiki/Goomba_fallacy
Regular ppl in the west now hold mildly positive views of the ccp and how 'advanced' china is than usa.
Then there are europeans who now are looking for china to give them the technology handout now that relationship with usa has soured.
They're one of 2 companies I would absolutely never work for (weapons etc aside). FB's recruiters hounded me so often I requested that they blackball me. The day they became Meta, I learned this by checking my email to see that they started trying to reach out again. I once again requested that they blackball me. This by extention taints OAI, the other company I'll never work for.
After a few hours with Glimmer I'm pretty impressed. It's better than the benchmark scores seem to indicate compared to Qwen 3.6 27B. I'm very excited for 3.8
Perpetually kneecapped by one of the worst management cultures I've ever seen
Alibaba, Google, Moonshot, Thinking Machines, etc are not releasing their models for free because they love to. They want to grab market share. I'll take it.
I still will not use a hosted Meta product, but damn this model looks solid.
Unfortunately there are a few topics that short circuit some terminally only people. One of them being anything related to meta. Few others recently emerging is Flock or Musk. It's really exhausting since you can't have a discussion relating to anything that may be adjacent to said topics. It's like a black hole.
One can do no right regardless, the other can do no wrong.
At least in HN.
Go vibecode something to auto upvote all downvoted posts, call it "Antiechochamber.HN" or something, and if enough people used it this website might improve a bit.
If a company can spend money to redeem itself then, well, it can (game theoretically or whatever) do whatever it wants in the future and then spend money to wipe the slate clean.
[1] By which I mean: the very act of being prompted to ask such a question, of planting a seed like hmm, Meta might have some aspects which are good for us. You don’t have to be convinced of it. Just the seed itself can pay for itself.
Nothing redeems them at this point of time, they are doing exactly ZERO to redeem. Tossing open weight models (not opensource!!) is not a basis for redemption, and does not constitute remorse in any way. Trying to portray it as such is complicity to META's crimes against humanity.
Perhaps none of the AI companies are shining examples of high ethics, but basically all of them have ethical high ground over Meta.
At least Anthropic isn’t sending private videos from pervert glasses to contract workers in Africa. It’s a low bar but it’s a bar nonetheless.
I’m liking it, and I don’t see a personal moral contradiction here. Do you use React for frontend for example?
I also wish this HN post is a bit more focused on the release, and less noise around Meta.
EDIT: An open weight version of Muse Spark 1.2 is going to be released as well:
https://x.com/alexandr_wang/status/2086756152034066792
https://xcancel.com/alexandr_wang/status/2086756152034066792
Just recently, Minimax H3 released as open weights on the eve of Seedance 2.5 global availability. It's not as good, but it's good enough and it's completely open.
Flux 3, which is nowhere near as good as either, suddenly announced their release once news of these other two became public. They knew if they waited they'd be ignored. It didn't really help them much, unfortunately.
The LLM releases are even more rivalrous.
And don't forget all of the competing launches planned before Google IO or major release events.
Companies like to eat into the news and press cycle of their rivals.
Seems a bit premature of a statement lol
Even if I did, we’re talking barely a decade
https://x.com/osanseviero/status/2086107547535122767
</div> is four Gemma4 tokens, but one Qwen3.6 token.
Each turn is about 45-60 seconds to generate all of the various responses. The GM and director have reasoning on, and the NPCs/Location/Narrator do not.
It's a fairly good "engine" for that. I'm not sure how a denser Qwen would do here regarding speed.
Try a system prompt requiring it to think in Mandarin, while still delivering the response in the user’s language.
Surprising that Meta don't host this model, even as rate-limited free-tier.
> open weight version of Muse Spark 1.2
Wait. Is this "version" different from what Meta serves?
UPD. was wrong on smaller, it's actually much larger
UPD, NVM, got misled by comments here. It is actually almost 60 GB so much larger
[1]: Limits may change without notice, including due to capacity constraints. - https://support.google.com/gemini/answer/16275805?sjid=14713....
[2]: "standard limits" are never defined - https://support.google.com/gemini/answer/16275805?sjid=14713...
[3]: https://tobyonfitnesstech.com/blog/anthropic-refund-scam/
[4]: https://news.ycombinator.com/item?id=48947776
- Middle of the day, servers busy? Swap to Sonnet while pretending it's still Opus. Many people won't notice, and nobody can prove anything if they suspect.
- Middle of the night, server load is light? Put it into extra thinky mode so it burns more tokens to ramp up the bills. Flip the switch where it gets really pedantic about writing lots of extra test cases and verifying against documentation.
- Demand increases, but don't feel like running more hardware? Switch to low bit quants, but have a monitor model swap back to quality if it can tell you're running a benchmark.
Assuming model capability plateaus (I think it will), token providers will be in a race to the bottom to maximize profits at the expense of quality that's very difficult to measure.
I've seen local models recognize when the task I'm asking them for is likely to be an artificial benchmark.
And any smart company is going to use lightweight models to monitor your sessions. If their sentiment analysis suspects you're close to cancelling, they'll up the knob for a few days until you calm down. Or worse, their accounting tells them that you're getting too much value from your fixed price subscription, so they turn the knob down to encourage you to cancel.
In the short term, the "frontier" models are too good to ignore. But if (when?) that plateaus, I don't see how anyone could trust a non-local model. When you pay an ISP to serve your web site, you can tell if they over-compress your images to save storage and bandwidth. With LLMs, it's just JSON with more errors and pointing to the fine print that models are not deterministic.
Alpha Go had a game where the models could compete against each other. That let it become super human. What's the intelligence game we can create for LLMs? Even if you invent something, will it make the model smarter in a way the market values enough?
Then there's a race to use the weights more efficiently, or to offload information that shouldn't be in the weights in the first place (Karpathy's Cognitive Core). I like to imagine we train the models in something like Lojban, have a lightweight model translate from human language to that, and you can update the Sqlite or Postgres store it uses for knowledge.
And there's no barrier to entry for agent harnesses. So whatever loops or recursive orchestrated council of elders idea comes up, that won't protect the monopolies (duopolies).
Anyways, depending on your definitions, I think we'll hit AGI, but I don't think we're getting a Singularity this time around. Again though, this is all just hand-waving.
So many ways for enshittification here.
Surely, even if you’re just using open weights models, it should theoretically be cheaper to use them in a highly optimized cloud architecture(even with vendor markups) rather than each person serving their own models from much less efficient (and more importantly, much less consistent volume) self-owned “server under your desk”?
Do you have a good source for this?
Until A100 prices are reliably under 1.70$ an hour, there is no GPU/AI bubble and Michael Burry doesn't know anything about GPUs.
And super noteworthy is that a 27B model (Qwen 3.6 27B) from this year is a huge improvement over a 120B model (gpt-oss:120b) from last year. The goal posts are moving, but at some point "good enough" is good enough for the kind programming I like to do.
1. The template name is Onyx ATEM as found in the tool call exception message
2. It appears to be following a harmony-style chat template. But the tool use seems to be a xml like :<atem:function_calls> / <atem:invoke> / <atem:parameter>
3. atem: a internal joke of meta in reverse?
https://huggingface.co/meta-models/Muse-Glimmer-30B/blob/mai...
I think it’s likely 3; meta in reverse. While tokenisers and preprocessing can catch it, you want your special tokens to be unique and not present in the original corpus. <meta: is likely too common.
The progress compared to Qwen3.6 27B is good, not that impressive, it's a 4 months old model. (kuto to them to compare to 27B dense and not 35B MoE, it's more fair to do so). It is very probable that Qwen3.8 27B will crush Glimmer-30B on most benchmarks.
"... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..."
This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no problem being #1.
Fair on size, but the headline numbers are against a model a generation back
> We quantize weights to ~4-bit, bringing the LM under 20 GB. We validated minimal to no degradation on agentic tasks under compression.
https://www.reddit.com/r/LocalLLaMA/comments/1vkgsum/introdu...
I think that if an LLM can't abstract over the differences between Python and C++, it probably will have an even harder time abstracting over the differences between writing code that manages a webserver, and writing code that does aerodynamic simulations.
On the other hand, at the GTC was a talk about coding in different lanugage (like spanish) and explaining that the quality between spanish and english is relevant different.
But i have not found a good article about the impact of learning data with practical experiments or even if the order of the learning data matters.
At least I think i remember that Meta mentioned having better and less data can be better than more data with lower quality.
As long as these models can explain to you facts about any other topics, its still overfitted for the task though.
That is to say, there is no single 'python' part of the model. The python bit is spread throughout the entire model and overlaps with other pieces that have similar, but unrelated, capabilities. For example the python subpspace might be partially in superposition with cupcake recipes, Esperanto, and calculus. We need calculus in a coding agent but not the other two. However, separating them cleanly is almost impossible, and even identifying them is tough.
Internally the manifolds are highly inefficient and nothing like you would imagine something humans built would be designed. It's more like something that evolved in nature.
Again, those subspaces are all somehow inextricably correlated and live in complex superposition spread throughout the manifold. The router doesn't know (or care) WHY those sections get lit up it just learns which ones to activate to optimize it's own reward function. So maybe it learns to activate "logic", "python" and "cupcake recipes in esperanto" whenever it see's something that kind of looks like python. It's not the best answer, it's just the best answer the tiny router could figure out.
It's all wildly complicated and inefficient, and works nothing like any reasonable human would imagine that it SHOULD operate.
https://alignment.anthropic.com/2025/selective-gradient-mask...
I corrected it.
Or just use Luna honestly. Worth considering if you’re ok with hosted APIs.
128gb hardly runs deepseek v4 flash which is almost free via api pricing.
Sure, if you want the latest and almost* greatest. You can pick up an M1 Max 64GB for ~1k.
* I guess 128GB also exists
Its not just that you send a sentence to an API endpoint, you always send EVERYTHING to that agent as a context.
You want to analyse your spending history? You now send everything to someone.
Either no one cares but understands this implication on how easy it is to really capture you or no one really things about it.
But i'm a lot more diligent on what I send. I disabled the gemini activity feature for example because google started telling me that my stuff could be reviwed by humans.
When it's free, you are the product.
In the short term, the big challenge is being able to afford hardware that can run a ~30B model. Last month I got to experiment with LLMs on a NVIDIA RTX 6000 Ada Generation as a visiting researcher during my summer break. I see the power of local LLMs for agentic coding; they’re no Claude, but they are quite useful. I wish I had gotten into local LLMs before hardware has gotten prohibitively expensive and in some cases unavailable; Apple discontinued certain Mac Minis and Mac Studios with high amounts of RAM due to the RAM shortage.
Hopefully high RAM prices don’t become a new normal, though the next year or two doesn’t look good.
The models you can run on a high-spec laptop today are approximately where frontier models were 12-18mo ago (albeit at a lower tok/s rate). If you scan back through hn comments from that era, you’ll find plenty of people saying “this is powerful enough to massively increase my productivity”.
Not always! I get 80-100 tok/s from Qwen 3.6 35B-A3B on a MacBook Pro thanks to MTP. With long contexts that dips to around 50-60. However, prefill is much slower than API models. So it becomes really, really, really critical to not have cache misses.
Privacy. Security. Not bulk uploading your trade secrets and intellectual property to Sam and Dario’s servers.
I've been able to accomplish incredible feats (for myself) since GPT-4, so model intelligence is secondary.
It's a dense model, not MoE like e.g. Qwen 35b or Gemma 4 26B A4B. On a Spark it will be memory bandwidth limited
I haven't tried yet (working on it) but back of the napkin estimate puts it at around 15tok/s even after converting to NVFP4. Prefill would be much higher though. That 15tok/sec is pretty typical for dense models of this size:
NVFP4 Q/K/V/O and MLP projections: ~13 GB/token
BF16 attention gates: ~3 GB/token
BF16 LM head: ~2.5 GB/token
Total: ~18.9 GB/token
At 273 GB/s, that gives a bandwidth-only ceiling of about 14.5 tok/s; actual performance would be lower.
That's pretty respectable.
Still working on optimizing and cleaning up before I push it.
If you can afford it or somehow find a used unit, you can go Epyc for 12 channels.
8/12 channel DDR5 will beat DGX Spark in inference/decode even without a GPU of any kind, as it’s memory bandwidth bound, and the Spark tops out at ~240gb/s real world.
With some optimisation and maths, it’s entirely plausible to ach
You are paying an extraordinary amount of money for the convenience of a super small unit, with still mediocre software support, but at least a community. Expect to be crawling through forum posts regularly, as SM121/Spark has many quirks and ecosystem issues still.
Please don’t pay another 70-80% gross margins on top of already inflated DRAM prices unless you need. The Spark IS really nice if you want to test out ConnectX or if you really need something small and compact and quiet.
Also consider: used Adas or even Ampere NVIDIA workstation GPUs can come with a lot of VRAM and be “reasonable”, with CUDA.
I don't know why MSL released this, but it is very nice that they did.
My local NAS is private and I’m not going to send it off to APIs for captioning or metadata; but even Qwen3VL 8B does an excellent job at this, despite being quite old.
They are also really excellent for fine tuning. Unsloth and Tinker (from Mira’s TML) are great places to start.
If your use case is narrower than “coding agent for everything”, you can probably match frontier performances on that narrow domain with ~30b and exceed it with ~100b+.
MoE will be faster because it will read less memory for sure, you still have to have it though.
Waiting on Q3 to download to check speed + do my usual anecdotes. I generate beefy code snippets and poems, and also ingest my HOA declaration and answer nuanced questions.
edit: i should've prefaced this somewhere with: This card ballparks at 800GB/s IO, which I can't seem to find easily on the market anymore. Kinda the ideal card for this model, if I just had a _little_ more VRAM (XTX is 24GB).
edit2: not mtp, this is dflash model (param in child comment). I'm up to ~60 tok/s generation and sitting at 19GB VRAM (i added --no-mmproj (makes it text-only i believe) because I'm used to speculative decoding wanting more VRAM and I'm already close to the limit :sweat_smile:)
edit: My favorite bit of reasoning I saw go by in my "generate me a beautiful code snippet" anecdote: 'Could give a snippet of beautiful code: the "hello world" in brainfuck? No.'
edit2: my first dflash speculative model! no mtp. I'm up to ~60 tok/s on empty context with `--spec-type draft-dflash`
- Mark Zuckerberg
https://www.meta.com/thefutureisforeveryone
That's a modern gaming laptop; cheapest I see in the US with 24GB is $3.5k.
Should be quite a bit faster than the new M5 MacBook Pro, and you can run Linux on it!
Open weights*
I don't think outside of the Big 3 (Ant, OAI, GDM), given the strong competition from China, any other Lab has a chance at capturing the coding market if they aren't open weights (save for xAI whose latest Grok looks every bit good & will probably rely on Cursor for distribution instead of going open weights). There's literally no other selling point, as the capabilities have mostly converged by now among the chasing pack.
It’s not completely open source, but they actually release their pretraining and post-training datasets with some redactions for (cough) pirated content.
They also have very good code and playbooks for actually doing a fine-tune, CPT, etc.
Even if you’re not tuning a Nemotron model, its mixes are very excellent for your replay data slice; or general experiments. Way better curation and quality than Dolma, etc; or other large huggingface data mixes I tested.
There's a large market, very large, who want the best regardless of what it costs. Probably a large enough market to keep that domain of research afloat (as opposed to shifting research manpower to cost cutting).
The reasoning is just that the marginal cost of AI is very secondary to fixed costs of the businesses themselves; it's not an excuse to sacrifice performance.
Dense model makes it dog slow on anything without HBM. Max 15tok/sec on decode on DDR5 systems like a Spark or a Strix Halo -- and that's at 4 bit quant.
I like this class of model. Multi-token prediction makes it viable to run dense models at not-too-far-off speeds as MoE models with much better intelligence.
The submission’s title (open weights 30B local coding model) is luckily wrong: This is meant to be a general agentic model.
It even comes pre-quantized and with a MTP/drafter model. Looking good!
Let’s hope they aren’t dishonest with the benchmarks this time …
https://xcancel.com/alexandr_wang/status/2086756152034066792
It's correct. See the OpenCode demo. Generic models are good enough for coding without necessarily being designed specifically for coding.
Product teams really need to hire at least one or two people with a 12-year-old's sense is humor. They need to winnow all the potential stupid jokes out of their product namings.
Photoshop source code+ OSI license = open source
Photoshop binary you can run on your own computer = open weight
Photoshop SaaS web app = closed, proprietary (Opus, GPT, etc.)
"Open weight" models are still just binary blobs that are completely inscrutable. It's like bringing home a dog from the rescue and just hoping that it doesn't have a tendency to bite kids in the face. You just can't know. The only thing that you can do is try to add more training (fine tuning) telling it not to bite kids.
I don't think the FOSS community has ever accepted this, but somehow we're feeling like it is okay now.
Photoshop binary you can run on your own computer = open weight
I don't think this is a correct analogy. You are not allowed to distribute modified versions of the Photoshop binary. Most open weight model licenses allow you to make and distribute your own finetunes, etc.
Given an open weights model trained to never bite kids, you can get it to bite kids with 10 prompts and a linear projection, the known simple algorithm doesn’t even need a backwards pass.
yay asymmetry!
Sure, having information about how these models were trained is helpful for reproducibility, but it is basically impossible for anyone without substantial capital and access to the same (likely copyrighted) data to reproduce the model. For normal users, owning the model weights essentially means owning 100% of the model, you can inspect and study the weights in much the same way as the lab that produced the model can, you can modify the weights, and you can use and distribute them if the license allows you to