I vibecoded https://punkx.org/apl/learn.html [1] its good slow entry into gnu-apl, at least it got me to be able to make tictactoe, vibecoded or not, it helped me to start.
I am amazed how quickly APL changed the way I think.
Also strongly recommend watching Aaron Hsu on youtube.
There is no better time to re-learn programming, try APL, Forth, LISP, z80 machine code, UXN TAL, just try new things.
"Men are born soft and supple; dead they are stiff and hard. Plants are born tender and pliant; dead, they are brittle and dry. Thus whoever is stiff and inflexible is a disciple of death. Whoever is soft and yielding is a disciple of life. The hard and stiff will be broken. The soft and supple will prevail."
FYI, your "8‿64 •bit._cast -⟜@ raw" in bqn/util.bqn results in passing to •bit._cast integers in the range 0..255, whereas it expects integers ¯128..127 (and does arbitrary platform-specific things on integers outside that for the int16→int8 narrow); removing the "-⟜@" makes it work properly, and 2.5-5x faster while at it from not going to intermediate int16 (allowing the bitcast to be O(1) by itself). (•FBytes → •file.MapBytes also probably improves things, at the cost of the obvious mmap problem of issues if the file changes, as BQNs arrays are immutable). This is also what caused the problem at https://github.com/ClickHouse/ClickBench/pull/939/changes/40... (there's been some discussion about what to do about invalid •bit._cast before, considering it's rather perf-sensitive; it's the only place in CBQN with such quiet very-wrong results on invalid inputs)
The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, but mostly only AVX2 on mainline; and no arm SVE)
That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway.
(didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing; and, of course, if some of the columns are bools/int8/int16/int32, it'd be beneficial to store & load them as such instead of float64)
BQN & CBQN are absolutely wonderful pieces of code.
L is mac/lin but linux is avx512 only specifically to try to deal with that problem. The compute on compressed algos helps fit more in those cache lines! https://lv1.sh/blog/compute-on-compressed/
Closed source (and vibe-coded) software, especially for a language runtime, is a non-starter for many people including myself. But proprietary license is actually common in the APL and K family of languages. They are fascinating though, and plenty of free/open implemenations exist, for example: https://ktye.github.io/
the author of the site makes no secret that the website is vibecoded. You're unfairly extrapolating that the product itself is vibe-coded (hard to do unless he is a time-traveller). And you're missing the point of the unique advantages that L offers wrt computation on compressed data.
yup website is Claude Design for prototype. For the core ... AI has been less helpful than I hoped - I believe largely because array style languages have so little source in training? But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those (although it was hard to go ASM>C or ASM>rust). E.g. lots of small mistakes would have been missed by not having restrict/const in places. Claude was great at compile all functions, analyze ASM, suggest optimized ASM (and ASM2vec was helpful as well for finding any "similar" code paths that could be combined (e.g. var/dev/cov are just moments)
Unfortunately the code itself is in a style of C many find difficult to read. I blame my upbringing. ATW open sourced examples and it was not really helpful. More recently others are doing a step by step in more standard C https://github.com/ardentsia-cgs/kparser/
Quite cool, but for a new runtime of an existing language it might make sense to compare to, y'know, the other[0] runtimes of that language? Even if one has to omit the best, closed ones for lack of access / permission to benchmark?
that is fast! Are you going to do the STAC-M3 benchmarks? Kx use them as evidence that they are the fastest, which is a fair claim as nobody else participates in that particular test :-)
Not sure why all the hate (sure site may be vibecoded, not all of us are front-end masters and it's at least not spartan)... I've always found k to be fascinating, and cool idea to try and roll a new one. Wish it were wholly open source, but cool to have a new variant that seems to bench well.
benchmarks at https://github.com/l-labs
unlike klong/ngn/bqn et al (which are GREAT) this has the goal of full production database compatibility (and full language compatibility).
We'd like to see some benchmarks against open and closed k interpreters please! I'm curious how well a vibe coded k/q interpreter stands up to Shakti or whatever Mr. Whitney is letting out the door right now.
q is the programming language underpinning kdb+ by Kx Systems - often claimed (by Kx) to be "the fastest database in the world". Possibly also the most expensive?
L is an independent implementation of a q interpreter. To those who reluctantly commercially license kdb+ - this will be a welcome alternative.
and fusion (e.g. f g h x has no intermediates e.g. mutates in place) is new
as is compute on compressed vectors (very helpful performance unlock) https://lv1.sh/blog/compute-on-compressed/
[1] seems to have the actual details on compressed vectors. looks like for each page you're storing min&max, and the values come as offsets from the min (smaller->less bits per value), did I get that right?
simple example at https://lv1.sh/blog/compute-on-compressed/
But in general compression is reducing the bit width of the input through an encoder (FOR or Frame of Reference is an old and good example). So we store the base in an offset location then the large payload is a much smaller size. E.g. i64 can goto i16. Then simd gets more #'s per cycle on the smaller, and the base is added to the scratch in stack (for sum). avg is similar (since it is just sum / count)
if you k you k ;-)
for the uninitiated, this looks like some wallstreet quant's new startup. Initially I thought it was the rebrand of shakti, Arthur Whites most recent rewrite of an array language. It's purpose built tooling for computing tick data for financial markets, but the best way I can describe it is codegolf for experienced programmers who don't want to give up the keyboard. these tools combine dataaccess and the ability to compute against that data with as few abstractions as possible.
As someone who does not know what k4, qSQL, or q are, reading through the landing page of this website was giving me mild schizophrenia. And then I tried to search for these things in the old way, and received incredibly dry technical sites that still don’t tell me what it is, and all these names are wildly SEO unfriendly. So I had Claude give me context and it’s apparently the database Wall Street uses for tick data. Sounds cool but, jeez.
K4 is the K programming language, Q is a language built on top of K. They are the practical over achieving members of the array family of programming languages.
Flagged, lots of unexplained grandiose thinking like “k and q made the vector the unit of thought.” K and q are unexplained and unlinked and “unit vector of thought” is pseudoscientific language
Extremely likely to be AI, though I’m not sure that matters for rules re: submissions
The language and its ideas come from Ken Iverson's famous paper "Notation as a Tool of Thought". This is a common understanding for people in the APL, K/J/Q etc array language communities, who are likely this website and product's intended audience.
> lots of unexplained grandiose thinking like “ k and q made the vector the unit of thought.”
That much makes sense in context: K is an array language, like APL, J, etc. From what little experience I have with J, ‘vector as a unit of thought’ seems like a reasonable description.
That’s fair: The most fair-to-submitter reading of your comment is my concerns are unfounded. If you didn’t intend that, that’s fine.
For moderators, I’d suggest that for the community, it’s spam. That’s one example, there are many more like it. The individual statement may be defensible but is still pseudoscientific language. This sort of content is a massive burden to community. Unanswerable anrguments about AI writing, whether the ability for an individual to have a parse-able reading is the same as writing being parable. The net effect is negative experiences for many and copy-editing for someone who did not do copy editing.
The 'unit of thought' thing is a nod to the Ken Iverson paper on APL that won him the Turing Award ("Notation as a Tool of Thought"), also referenced by Dyalog's tag line "The tool of thought for software solutions"). Variations of that phrase are endemic in the array languages space.
I see how jarring it is for you, but I want to comment-vouch against your flag. If you slow down a little, you can spot some meaning there. Its oddly phrased, but it does make sense that vectors (as in: simd vectors) are the building blocks of execution (to put it closer to how I might say it).
I’ve had similar ideas in the past: clearly simd is the way to get the most out of your cpu. Can we design a language where all operations are automatically simd, and it takes effort to do anything in scalars?
And I guess these array languages are what you might get.
It’s not ‘unit vector of thought,’ btw, which is weirder than what it says.
>If a story is spam or off-topic, flag it. Don't feed egregious comments by replying; flag them instead. If you flag, please don't also comment that you did.
I am amazed how quickly APL changed the way I think.
Also strongly recommend watching Aaron Hsu on youtube.
There is no better time to re-learn programming, try APL, Forth, LISP, z80 machine code, UXN TAL, just try new things.
"Men are born soft and supple; dead they are stiff and hard. Plants are born tender and pliant; dead, they are brittle and dry. Thus whoever is stiff and inflexible is a disciple of death. Whoever is soft and yielding is a disciple of life. The hard and stiff will be broken. The soft and supple will prevail."
[1]: https://github.com/jackdoe/gnu-apl-wasm source of the repl and learn playground and also how to compile gnu apl to wasm (vibecoded)
If anyone is curious how queries in this language look, you can see it here: https://github.com/ClickHouse/ClickBench/pull/939/changes#di...
The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, but mostly only AVX2 on mainline; and no arm SVE)
That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway.
(didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing; and, of course, if some of the columns are bools/int8/int16/int32, it'd be beneficial to store & load them as such instead of float64)
Columnar databases are array languages, after all.
[0]: https://k.miraheze.org/wiki/Running_K
L used the two open ones that are easy to replicate: H2O.ai (great bench) https://github.com/l-labs/db-benchmark TSBS (less great but useful) https://github.com/l-labs/tsbs
If there are others (will do ClickBench) they'll go there as well
[1]https://lv1.sh/blog/compression-memory-wall/
Did you mean APL?
https://en.wikipedia.org/wiki/Array_programming
EDIT: Did someone downvoted this because it is wrong or is someone I am arguing with rage-downvoted this ?
Extremely likely to be AI, though I’m not sure that matters for rules re: submissions
The language and its ideas come from Ken Iverson's famous paper "Notation as a Tool of Thought". This is a common understanding for people in the APL, K/J/Q etc array language communities, who are likely this website and product's intended audience.
That much makes sense in context: K is an array language, like APL, J, etc. From what little experience I have with J, ‘vector as a unit of thought’ seems like a reasonable description.
> Extremely likely to be AI
I had the same thought though.
For moderators, I’d suggest that for the community, it’s spam. That’s one example, there are many more like it. The individual statement may be defensible but is still pseudoscientific language. This sort of content is a massive burden to community. Unanswerable anrguments about AI writing, whether the ability for an individual to have a parse-able reading is the same as writing being parable. The net effect is negative experiences for many and copy-editing for someone who did not do copy editing.
I’ve had similar ideas in the past: clearly simd is the way to get the most out of your cpu. Can we design a language where all operations are automatically simd, and it takes effort to do anything in scalars?
And I guess these array languages are what you might get.
It’s not ‘unit vector of thought,’ btw, which is weirder than what it says.
https://news.ycombinator.com/newsguidelines.html