Logs: liberachat/#haskell
| 2025-11-13 12:12:50 | → | deptype_ joins (~deptype@2406:b400:3a:73c2:4944:536a:6ac6:1483) |
| 2025-11-13 12:14:58 | Googulator89 | is now known as Googulator |
| 2025-11-13 12:16:40 | → | lambdabot joins (~lambdabot@haskell/bot/lambdabot) |
| 2025-11-13 12:16:40 | ChanServ | sets mode +v lambdabot |
| 2025-11-13 12:23:35 | <kuribas`> | merijn: how would you query my keys, like ("foo", *, 3) in sqlite? With a CTE? |
| 2025-11-13 12:28:08 | × | tromp quits (~textual@2001:1c00:3487:1b00:7d:cf52:961a:9343) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2025-11-13 12:30:07 | <kuribas`> | "SELECT val FROM my_patterns WHERE (string_key == "foo" OR string_key IS NULL) AND (..) ORDER BY (string_key, string_key2, int_key) ASC NULLS LAST" |
| 2025-11-13 12:30:13 | <kuribas`> | LIMIT 1 |
| 2025-11-13 12:30:30 | <kuribas`> | Assuming NULL means "star" here... |
| 2025-11-13 12:32:27 | × | merijn quits (~merijn@77.242.116.146) (Ping timeout: 252 seconds) |
| 2025-11-13 12:32:32 | → | tromp joins (~textual@2001:1c00:3487:1b00:7d:cf52:961a:9343) |
| 2025-11-13 12:32:39 | × | deptype_ quits (~deptype@2406:b400:3a:73c2:4944:536a:6ac6:1483) (Remote host closed the connection) |
| 2025-11-13 12:33:00 | → | deptype_ joins (~deptype@2406:b400:3a:73c2:ffa7:a8a3:7599:8304) |
| 2025-11-13 12:34:20 | <kuribas`> | Storing * as NULL is a bit inelegant... |
| 2025-11-13 12:34:42 | → | __monty__ joins (~toonn@user/toonn) |
| 2025-11-13 12:35:07 | × | qqe_ quits (~qqq@185.54.21.203) (Quit: Lost terminal) |
| 2025-11-13 12:35:28 | × | annamalai quits (~annamalai@157.33.249.99) (Ping timeout: 255 seconds) |
| 2025-11-13 12:35:41 | <lucabtz> | [exa] yeah i was speaking about something distinct from multimap |
| 2025-11-13 12:36:09 | <kuribas`> | Basically I have "data Wild a = Specific a | Wildcard", and map "Map (Wild key1, Wild key2, ...) val" |
| 2025-11-13 12:37:26 | <[exa]> | kuribas`: how do you evaluate a wildcard on the hashmaps btw |
| 2025-11-13 12:37:54 | <[exa]> | kuribas`: select everything with "foo" at position 1, everything with 3 at position 3, and intersect? |
| 2025-11-13 12:37:56 | <kuribas`> | [exa]: I have "Hashable key => Hashable (Wild key)" |
| 2025-11-13 12:38:35 | <[exa]> | so the * is essentially another constant? (I thought it's a query like "anything goes here") |
| 2025-11-13 12:38:51 | → | merijn joins (~merijn@77.242.116.146) |
| 2025-11-13 12:39:06 | <kuribas`> | [exa]: basically yes. Wildcard means match anything. |
| 2025-11-13 12:39:22 | [exa] | confused |
| 2025-11-13 12:40:08 | <kuribas`> | [exa]: wildcard is only allowed when storing the keys, not when looking up. |
| 2025-11-13 12:41:09 | <[exa]> | so these are keys that match all possible queries, as in, if you query for ("foo","haha",3) that thing with * is going to show up, right? |
| 2025-11-13 12:41:39 | <kuribas`> | exactly |
| 2025-11-13 12:42:14 | <kuribas`> | but ordered by * last. So first the specific thing, only * if no specific thing exists. |
| 2025-11-13 12:42:49 | <[exa]> | so your query algorithm basically checks 2 hash values for each position, right? |
| 2025-11-13 12:43:03 | <kuribas`> | [exa]: 3 |
| 2025-11-13 12:43:13 | <kuribas`> | each position can have a wildcard. |
| 2025-11-13 12:43:57 | <[exa]> | ah yes I meant that to get the result for ("foo","haha",3) you need to hash an dlookup the tuple as is, and then you need to hash and lookup (*, "haha", 3) and ("foo", *, 3), etc... |
| 2025-11-13 12:43:59 | × | merijn quits (~merijn@77.242.116.146) (Ping timeout: 256 seconds) |
| 2025-11-13 12:44:13 | <lucabtz> | kuribas` so when setting say (*, *, *) you would modify the whole table? |
| 2025-11-13 12:44:28 | <[exa]> | (no other idea on how to map it to hashmaps) |
| 2025-11-13 12:44:56 | → | merijn joins (~merijn@77.242.116.146) |
| 2025-11-13 12:45:19 | <kuribas`> | lucabtz: if you insert ((*, *, *) -> val), every lookup will succeed, but val only if everything else fails. |
| 2025-11-13 12:45:34 | <merijn> | kuribas`: sorry, got distracted so missed half the context except the one question |
| 2025-11-13 12:45:44 | <lucabtz> | ah i see okay |
| 2025-11-13 12:45:51 | <[exa]> | kuribas`: is this RDF by the way? |
| 2025-11-13 12:45:55 | <merijn> | kuribas`: Also, note that you can actually have SQLite run arbitrary custom functions mid query |
| 2025-11-13 12:45:57 | <lucabtz> | the wildcard is like a fallbacl |
| 2025-11-13 12:46:07 | <kuribas`> | [exa]: I have (HashMap (Wild a) (HashMap (Wild b) (HashMap (Wild c) val))) |
| 2025-11-13 12:46:24 | <kuribas`> | [exa]: no, something we use for configuration. |
| 2025-11-13 12:47:17 | <[exa]> | ok I see, so whenever the `a` is not matched you try also the wildcard at that level, which also gives ordering to wildcards |
| 2025-11-13 12:47:56 | <kuribas`> | yeah |
| 2025-11-13 12:48:48 | × | CiaoSen quits (~Jura@2a02:8071:64e1:da0:5a47:caff:fe78:33db) (Ping timeout: 256 seconds) |
| 2025-11-13 12:49:34 | <[exa]> | tree version: make * the lowest-ordering item, first select exactly, if that fails select minimum bound of ("foo","haha",whatever) to see if the wildcard is on the 3rd position (you can re-use the partial failed select!), then of ("foo", whatever, whatever) to see if it's on the 2nd position, ... |
| 2025-11-13 12:49:51 | <kuribas`> | Maybe I can put an index at "a", and "b", since "c" (and d...) are sparse. |
| 2025-11-13 12:50:51 | <kuribas`> | [exa]: why lowest? |
| 2025-11-13 12:51:07 | <[exa]> | kuribas`: can be highest; any predictable position is OK |
| 2025-11-13 12:51:42 | <kuribas`> | True, putting * in the dictionary is less efficient. |
| 2025-11-13 12:52:58 | <[exa]> | lucene version: make occurence inverted indexes for all key parts that you have, select Very Very Quickly by intersecting the skiplists. (That also gives you infinite wildcard queries for free.) |
| 2025-11-13 12:53:47 | <[exa]> | as in, in the 3-level map model, querying for (whatever, "bar", 3) requires traversing the whole first map; with inversion this is free |
| 2025-11-13 12:54:10 | <kuribas`> | hmm, interesting... |
| 2025-11-13 12:55:38 | <[exa]> | (PS. skiplists are magical) |
| 2025-11-13 12:57:00 | <merijn> | something, something, lattice theory |
| 2025-11-13 12:57:01 | <kuribas`> | [exa]: So you have three maps in parallel, then you intersect? |
| 2025-11-13 12:59:47 | <[exa]> | yeah there's a very fast intersection algorithm for many skiplists |
| 2025-11-13 13:01:35 | → | acarrico joins (~acarrico@pppoe-209-99-223-51.greenmountainaccess.net) |
| 2025-11-13 13:02:32 | → | CiaoSen joins (~Jura@2a02:8071:64e1:da0:5a47:caff:fe78:33db) |
| 2025-11-13 13:10:59 | <kuribas`> | Anyway, my current dict implementation is fast enough. For comparison, reading all the configuration files, and constructing this map in python takes less than a second, while in our legacy (lisp) system, it takes several minutes to load and store in the lisp ORM. |
| 2025-11-13 13:19:57 | <__monty__> | That must be some implementation. |
| 2025-11-13 13:21:11 | <kuribas`> | __monty__: it's a whole lisp system with its own type system, an ORM based on a triple store, our own (graphql like) xml query engine, timeseries computations based on linked lists, etc... |
| 2025-11-13 13:22:10 | <kuribas`> | Also self-documenting values that are real-time inspectable in a UI (which hasn't been used for a decade now). |
| 2025-11-13 13:22:37 | <kuribas`> | Also the system is being kept up to date by keeping a running system, then patching it as we go. |
| 2025-11-13 13:22:54 | <kuribas`> | Since restarting the system takes an hour, to populate all the data in the ORM. |
| 2025-11-13 13:23:23 | <__monty__> | The ORM loads all the data into memory? |
| 2025-11-13 13:23:51 | <kuribas`> | __monty__: yes, everything except time series data. |
| 2025-11-13 13:24:12 | <kuribas`> | And store in mysql on a triple store. |
| 2025-11-13 13:24:47 | × | Lycurgus quits (~juan@user/Lycurgus) (Quit: alsoknownas.renjuan.org ( juan@acm.org )) |
| 2025-11-13 13:25:30 | <kuribas`> | I hope we get rid of it as soon as possible, since it's maintained by a single guy, who is the only one to understand the whole system. |
| 2025-11-13 13:33:49 | <kuribas`> | If he leaves, the company will be in trouble. |
| 2025-11-13 13:34:07 | → | annamalai joins (~annamalai@157.33.224.236) |
| 2025-11-13 13:34:35 | ← | kuribas` parts (~user@ip-188-118-57-242.reverse.destiny.be) (ERC 5.5.0.29.1 (IRC client for GNU Emacs 29.3)) |
| 2025-11-13 13:49:19 | → | spew joins (~spew@user/spew) |
| 2025-11-13 13:59:20 | × | chromoblob quits (~chromoblo@user/chromob1ot1c) (Ping timeout: 244 seconds) |
| 2025-11-13 14:00:29 | → | chromoblob joins (~chromoblo@user/chromob1ot1c) |
| 2025-11-13 14:00:39 | → | kuribas joins (~user@ip-188-118-57-242.reverse.destiny.be) |
| 2025-11-13 14:02:22 | → | karenw joins (~karenw@user/karenw) |
| 2025-11-13 14:02:45 | × | karenw quits (~karenw@user/karenw) (Remote host closed the connection) |
| 2025-11-13 14:05:51 | → | karenw joins (~karenw@user/karenw) |
| 2025-11-13 14:07:04 | × | lbseale_ quits (~quassel@user/ep1ctetus) (Quit: https://quassel-irc.org - Chat comfortably. Anywhere.) |
| 2025-11-13 14:07:12 | × | weary-traveler quits (~user@user/user363627) (Remote host closed the connection) |
| 2025-11-13 14:07:30 | → | lbseale joins (~quassel@user/ep1ctetus) |
| 2025-11-13 14:08:59 | × | merijn quits (~merijn@77.242.116.146) (Ping timeout: 256 seconds) |
| 2025-11-13 14:13:27 | → | ystael joins (~ystael@user/ystael) |
| 2025-11-13 14:14:12 | × | chromoblob quits (~chromoblo@user/chromob1ot1c) (Read error: Connection reset by peer) |
| 2025-11-13 14:14:32 | → | chromoblob joins (~chromoblo@user/chromob1ot1c) |
| 2025-11-13 14:15:07 | <__monty__> | That's not how ORMs work generally, no? Don't they keep everything in the database rather memory? |
| 2025-11-13 14:15:58 | <kuribas> | __monty__: yes, but our ORM does both. |
| 2025-11-13 14:17:41 | × | deptype_ quits (~deptype@2406:b400:3a:73c2:ffa7:a8a3:7599:8304) (Remote host closed the connection) |
| 2025-11-13 14:17:54 | → | deptype_ joins (~deptype@2406:b400:3a:73c2:8859:e21d:8034:1343) |
| 2025-11-13 14:20:05 | × | biberu quits (~biberu@user/biberu) (Read error: Connection reset by peer) |
| 2025-11-13 14:20:53 | → | merijn joins (~merijn@77.242.116.146) |
| 2025-11-13 14:22:48 | <gentauro> | iirc, wasn't one of the op's of this channel also an op at Libera? |
| 2025-11-13 14:27:15 | → | biberu joins (~biberu@user/biberu) |
| 2025-11-13 14:27:55 | <__monty__> | Think so, why? |
All times are in UTC.