Logs: freenode/#haskell
| 2021-04-01 14:55:00 | × | Alleria__ quits (~textual@zrcout.mskcc.org) (Quit: Textual IRC Client: www.textualapp.com) |
| 2021-04-01 14:55:13 | <merijn> | shapr: Also, isn't that just conduit? |
| 2021-04-01 14:56:55 | × | Raito_Bezarius quits (~Raito@unaffiliated/raito-bezarius/x-8764578) (Ping timeout: 258 seconds) |
| 2021-04-01 14:57:15 | × | gzj quits (~gzj@unaffiliated/gzj) (Read error: Connection reset by peer) |
| 2021-04-01 14:57:35 | → | gzj joins (~gzj@unaffiliated/gzj) |
| 2021-04-01 14:58:01 | × | kawzeg quits (unknown@2a01:7e01::f03c:92ff:fee2:ec34) (Quit: WeeChat 1.9.1) |
| 2021-04-01 14:58:34 | → | Alleria__ joins (~textual@mskresolve-a.mskcc.org) |
| 2021-04-01 14:58:38 | <shapr> | merijn: is it? Does conduit send a batch of multiple I/O requests for different parts of the file? |
| 2021-04-01 14:59:06 | <merijn> | Ah, no |
| 2021-04-01 14:59:24 | <shapr> | I've been reading https://www.tbray.org/ongoing/When/202x/2021/03/27/Topfew-and-Amdahl and wondering if I can reproduce the same parallel file reading behavior described |
| 2021-04-01 15:01:18 | × | conal quits (~conal@64.71.133.70) (Ping timeout: 240 seconds) |
| 2021-04-01 15:01:56 | <shapr> | merijn: background reading implies sending multiple requests for different parts of the file is 'faster' when the file can be processed in parallel |
| 2021-04-01 15:02:36 | <merijn> | shapr: That seems reasonable |
| 2021-04-01 15:02:46 | <shapr> | My reading implies drives are designed to handle a deeper/larger queue of requests |
| 2021-04-01 15:02:50 | <merijn> | shapr: But you could just do that using explicit reads on handles? |
| 2021-04-01 15:03:09 | <merijn> | shapr: You're familiar with DMA? |
| 2021-04-01 15:03:13 | <shapr> | yup |
| 2021-04-01 15:03:30 | → | conal joins (~conal@192.145.118.200) |
| 2021-04-01 15:03:45 | <shapr> | merijn: but not sure how that connects here, explain? |
| 2021-04-01 15:03:47 | <merijn> | shapr: Seems that alone is enough to get better parallel results |
| 2021-04-01 15:04:50 | <L29Ah> | 17:54:21]<shapr> Is there a file reading library that reads multiple chunks of a file at the same time? |
| 2021-04-01 15:04:51 | <L29Ah> | mmap + readahead? |
| 2021-04-01 15:05:00 | <merijn> | shapr: If the compute time for 1 page of data is bigger than the transfer time (because you have an SSD or something) then DMA 4 pages, processing those in parallel should be enough to get speedup? |
| 2021-04-01 15:05:17 | × | gzj quits (~gzj@unaffiliated/gzj) (Remote host closed the connection) |
| 2021-04-01 15:05:25 | × | gehmehgeh quits (~ircuser1@gateway/tor-sasl/gehmehgeh) (Quit: Leaving) |
| 2021-04-01 15:05:29 | <L29Ah> | or madvise |
| 2021-04-01 15:05:37 | → | gzj joins (~gzj@unaffiliated/gzj) |
| 2021-04-01 15:05:48 | <merijn> | shapr: Since DMA takes 0 CPU, if you queue a bunch of DMA accesses and then start processing you'd expect all loads (except the first N) to be free due to overlapping with compute |
| 2021-04-01 15:05:57 | <merijn> | shapr: Although, admittedly I haven't read your link |
| 2021-04-01 15:06:12 | <shapr> | merijn: sounds reasonable, thanks for the explanation |
| 2021-04-01 15:06:41 | <shapr> | L29Ah: I don't know, Tim Bray tried mmap and it wasn't any faster |
| 2021-04-01 15:07:01 | <merijn> | shapr: mmap and file reads are most likely the same |
| 2021-04-01 15:07:17 | <merijn> | shapr: mmap is just more explicit about what kinda behaviour you want |
| 2021-04-01 15:08:30 | <L29Ah> | shapr: anyway, are you sure you aren't hitting the ceiling of your device throughput w/o any hacks? can you show the numbers you got? |
| 2021-04-01 15:09:10 | → | kawzeg joins (unknown@2a01:7e01::f03c:92ff:fee2:ec34) |
| 2021-04-01 15:09:12 | <L29Ah> | OS caches aren't very dumb |
| 2021-04-01 15:09:45 | <merijn> | See also: https://queue.acm.org/detail.cfm?id=1814327 :) |
| 2021-04-01 15:09:50 | → | Raito_Bezarius joins (~Raito@unaffiliated/raito-bezarius/x-8764578) |
| 2021-04-01 15:10:22 | → | LKoen joins (~LKoen@65.250.88.92.rev.sfr.net) |
| 2021-04-01 15:13:02 | × | rond_ quits (59402019@89-64-32-25.dynamic.chello.pl) (Ping timeout: 240 seconds) |
| 2021-04-01 15:13:20 | → | tzh joins (~tzh@c-24-21-73-154.hsd1.or.comcast.net) |
| 2021-04-01 15:15:19 | <shapr> | L29Ah: Hi! I haven't seen you around here before, are you new to Haskell? |
| 2021-04-01 15:15:30 | <shapr> | I was the official greeter of #haskell for awhile :-) |
| 2021-04-01 15:16:34 | <L29Ah> | i'm poking haskell for >15y, but still haven't committed my code monkey career to it |
| 2021-04-01 15:17:06 | × | hololeap quits (~hololeap@unaffiliated/hololeap) (Ping timeout: 240 seconds) |
| 2021-04-01 15:17:10 | <shapr> | ah, neato! |
| 2021-04-01 15:18:42 | → | Sgeo joins (~Sgeo@ool-18b98aa4.dyn.optonline.net) |
| 2021-04-01 15:18:45 | → | DTZUZU_ joins (~DTZUZO@207.81.119.43) |
| 2021-04-01 15:20:46 | × | elfets quits (~elfets@ip-37-201-23-96.hsi13.unitymediagroup.de) (Ping timeout: 260 seconds) |
| 2021-04-01 15:20:47 | × | DTZUZU quits (~DTZUZO@205.ip-149-56-132.net) (Ping timeout: 268 seconds) |
| 2021-04-01 15:20:52 | × | raym quits (~ray@115.187.32.14) (Quit: leaving) |
| 2021-04-01 15:23:53 | × | gzj quits (~gzj@unaffiliated/gzj) (Ping timeout: 268 seconds) |
| 2021-04-01 15:24:02 | → | hexreel joins (~hr@2600:1700:28e2:14d0:89fc:b839:4e34:e1f2) |
| 2021-04-01 15:24:30 | <shapr> | L29Ah: do you build fun projects in Haskell? I think my most recent fun thing is https://github.com/shapr/bloohm |
| 2021-04-01 15:25:12 | <L29Ah> | https://github.com/l29ah?tab=repositories&q=&type=source&language=haskell not sure if fun |
| 2021-04-01 15:25:40 | × | zebrag quits (~inkbottle@aaubervilliers-654-1-98-245.w86-212.abo.wanadoo.fr) (Read error: Connection reset by peer) |
| 2021-04-01 15:25:59 | → | zebrag joins (~inkbottle@aaubervilliers-654-1-98-245.w86-212.abo.wanadoo.fr) |
| 2021-04-01 15:26:16 | × | chele quits (~chele@ip5b40237d.dynamic.kabel-deutschland.de) (Remote host closed the connection) |
| 2021-04-01 15:26:19 | → | Gurkenglas joins (~Gurkengla@unaffiliated/gurkenglas) |
| 2021-04-01 15:26:46 | × | o1lo01ol1o quits (~o1lo01ol1@bl7-88-181.dsl.telepac.pt) (Remote host closed the connection) |
| 2021-04-01 15:27:50 | → | Vadrigar_ joins (~Vadrigar@ip5b417208.dynamic.kabel-deutschland.de) |
| 2021-04-01 15:28:04 | × | enoq quits (~textual@194-208-146-143.lampert.tv) (Quit: Textual IRC Client: www.textualapp.com) |
| 2021-04-01 15:28:21 | <peanut_> | shapr, I like the "(no idea if this works in bash)" |
| 2021-04-01 15:29:32 | → | ezrakilty joins (~ezrakilty@97-126-95-37.tukw.qwest.net) |
| 2021-04-01 15:30:03 | × | stree quits (~stree@68.36.8.116) (Ping timeout: 268 seconds) |
| 2021-04-01 15:30:21 | × | jpds quits (~jpds@gateway/tor-sasl/jpds) (Ping timeout: 240 seconds) |
| 2021-04-01 15:31:16 | <shapr> | peanut_: turns out it does not! |
| 2021-04-01 15:31:23 | → | plutoniix joins (~q@node-um3.pool-125-24.dynamic.totinternet.net) |
| 2021-04-01 15:31:39 | × | xff0x_ quits (~xff0x@dynamic-046-114-038-234.46.114.pool.telefonica.de) (Ping timeout: 252 seconds) |
| 2021-04-01 15:31:48 | <peanut_> | thats how I can tell you had fun making that |
| 2021-04-01 15:32:54 | → | Mrbuck joins (~Mrbuck@gateway/tor-sasl/mrbuck) |
| 2021-04-01 15:33:28 | → | xff0x_ joins (~xff0x@2001:1a81:532d:a700:e532:2773:4b16:d5dc) |
| 2021-04-01 15:35:24 | → | DTZUZU joins (~DTZUZO@205.ip-149-56-132.net) |
| 2021-04-01 15:37:20 | → | heatsink joins (~heatsink@108-201-191-115.lightspeed.sntcca.sbcglobal.net) |
| 2021-04-01 15:38:05 | × | DTZUZU_ quits (~DTZUZO@207.81.119.43) (Ping timeout: 265 seconds) |
| 2021-04-01 15:40:11 | × | plutoniix quits (~q@node-um3.pool-125-24.dynamic.totinternet.net) (Ping timeout: 240 seconds) |
| 2021-04-01 15:41:50 | × | heatsink quits (~heatsink@108-201-191-115.lightspeed.sntcca.sbcglobal.net) (Ping timeout: 260 seconds) |
| 2021-04-01 15:42:50 | → | stree joins (~stree@68.36.8.116) |
| 2021-04-01 15:43:34 | → | __minoru__shirae joins (~shiraeesh@109.166.58.221) |
| 2021-04-01 15:48:02 | → | plutoniix joins (~q@node-um3.pool-125-24.dynamic.totinternet.net) |
| 2021-04-01 15:49:47 | → | xiinotulp joins (~q@node-ul9.pool-125-24.dynamic.totinternet.net) |
| 2021-04-01 15:50:46 | × | __minoru__shirae quits (~shiraeesh@109.166.58.221) (Ping timeout: 240 seconds) |
| 2021-04-01 15:50:49 | × | zigapeda1 quits (~zigapeda@185.169.233.10) (Remote host closed the connection) |
| 2021-04-01 15:53:04 | → | s00pcan joins (~chris@075-133-056-178.res.spectrum.com) |
| 2021-04-01 15:53:17 | × | plutoniix quits (~q@node-um3.pool-125-24.dynamic.totinternet.net) (Ping timeout: 252 seconds) |
| 2021-04-01 15:53:41 | → | Ariakenom joins (~Ariakenom@2001:9b1:efb:fc00:887c:d703:31b0:ee74) |
| 2021-04-01 15:55:42 | → | codygman__ joins (~user@209.251.131.98) |
| 2021-04-01 15:56:03 | → | bitmapper joins (uid464869@gateway/web/irccloud.com/x-uvvubdxwbzbtugry) |
| 2021-04-01 15:57:38 | × | codygman__ quits (~user@209.251.131.98) (Client Quit) |
| 2021-04-01 15:57:51 | × | Franciman quits (~francesco@host-79-53-62-46.retail.telecomitalia.it) (Quit: Leaving) |
| 2021-04-01 15:59:17 | → | geowiesnot joins (~user@i15-les02-ix2-87-89-181-157.sfr.lns.abo.bbox.fr) |
| 2021-04-01 16:00:29 | → | Rudd0 joins (~Rudd0@185.189.115.103) |
| 2021-04-01 16:04:37 | → | __minoru__shirae joins (~shiraeesh@109.166.58.221) |
| 2021-04-01 16:04:49 | × | jamm_ quits (~jamm@unaffiliated/jamm) (Remote host closed the connection) |
| 2021-04-01 16:05:07 | → | jpds joins (~jpds@gateway/tor-sasl/jpds) |
| 2021-04-01 16:05:47 | × | mice-user quits (b4960d8b@180-150-13-139.b4960d.syd.static.aussiebb.net) (Quit: Connection closed) |
| 2021-04-01 16:05:52 | → | jamm_ joins (~jamm@unaffiliated/jamm) |
| 2021-04-01 16:09:11 | × | merijn quits (~merijn@83-160-49-249.ip.xs4all.nl) (Ping timeout: 240 seconds) |
| 2021-04-01 16:10:56 | → | merijn joins (~merijn@83-160-49-249.ip.xs4all.nl) |
| 2021-04-01 16:12:31 | → | geekosaur joins (82650c7a@130.101.12.122) |
All times are in UTC.