Logs: liberachat/#haskell
| 2021-07-19 17:45:46 | <geekosaur> | iirc ghc's performance gets worse the more cores you have :( (but maybe this has improved, that was around 7.x) |
| 2021-07-19 17:46:06 | <geekosaur> | mostly because of stop-the-world gc, though |
| 2021-07-19 17:46:39 | × | favonia quits (~favonia@user/favonia) (Ping timeout: 255 seconds) |
| 2021-07-19 17:46:51 | <euouae> | Okay interesting |
| 2021-07-19 17:46:59 | <davean> | It gets harder to make it perform well with more cores, it doesn't inhernety not perform well. |
| 2021-07-19 17:47:06 | <maerwald> | dmj`: why are you using xml :> |
| 2021-07-19 17:47:08 | <davean> | But no super computing task is single process |
| 2021-07-19 17:47:21 | <euouae> | Davean my knowledge is limited |
| 2021-07-19 17:47:22 | <davean> | I mean if you could fit it on one system it wouldn't really be using a modern supercomputer |
| 2021-07-19 17:47:50 | <euouae> | I am not great at Haskell and never programmed for supercomputers before |
| 2021-07-19 17:48:57 | → | favonia joins (~favonia@user/favonia) |
| 2021-07-19 17:49:49 | <euouae> | But Haskell is really attractive for math |
| 2021-07-19 17:49:52 | <davean> | I think I've heard of some super computer Haskell use. I'd say everything to do with super computers is pretty definitionally specialized |
| 2021-07-19 17:50:17 | <euouae> | Yeah basically you need to be very knowledgeable in the domain davean |
| 2021-07-19 17:50:25 | <euouae> | And I’m just trying to get by |
| 2021-07-19 17:50:30 | <maerwald> | is this about bitcoin mining? |
| 2021-07-19 17:50:33 | <davean> | The domain, but also that specific computer |
| 2021-07-19 17:50:38 | <davean> | maerwald: I sure hope not! |
| 2021-07-19 17:50:46 | <euouae> | No lol it’s research |
| 2021-07-19 17:50:50 | <dmj`> | maerwald: AWS S3 still uses XML, https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html |
| 2021-07-19 17:51:17 | <euouae> | Bitcoin stuff is so scummy o want nothing with. It |
| 2021-07-19 17:52:26 | <davean> | maerwald: Many code bugs kill a large number of fractional people |
| 2021-07-19 17:52:28 | → | mnrmnaugh joins (~mnrmnaugh@pool-96-252-87-182.bstnma.fios.verizon.net) |
| 2021-07-19 17:53:01 | <euouae> | Davean I don’t understand how situational it could be though |
| 2021-07-19 17:53:22 | <euouae> | what I have is some pure math computations applied to many different numbers (say) |
| 2021-07-19 17:53:54 | <davean> | euouae: The structure of the super computers differ, and the primary issue with super computers is communication |
| 2021-07-19 17:54:03 | <davean> | You optimize communication, not compute |
| 2021-07-19 17:54:11 | <euouae> | For now I’m on a single node |
| 2021-07-19 17:54:14 | <davean> | The network structures are all different for different optimalities |
| 2021-07-19 17:54:26 | <davean> | Yes, and theres no modern single node super computers |
| 2021-07-19 17:54:54 | <euouae> | It’s not necessarily a supercomputer since it’s single node but I’m starting from that |
| 2021-07-19 17:54:55 | <davean> | even on a single processor, you have to deal with inter-core communication issues, they're just less |
| 2021-07-19 17:55:07 | <euouae> | Inter core? |
| 2021-07-19 17:55:22 | <euouae> | Are you talking about worker communication? |
| 2021-07-19 17:55:27 | → | oso joins (~oso@2601:58c:c080:a950:f275:2530:b398:680b) |
| 2021-07-19 17:55:28 | <euouae> | I’m not following |
| 2021-07-19 17:56:41 | → | benin0369 joins (~benin@106.198.95.190) |
| 2021-07-19 17:57:54 | <geekosaur> | things like shared memory access |
| 2021-07-19 17:58:13 | <euouae> | Why is that important ? |
| 2021-07-19 17:58:29 | <euouae> | I think in my problem no memory is shared |
| 2021-07-19 17:58:48 | × | wroathe quits (~wroathe@c-68-54-25-135.hsd1.mn.comcast.net) (Ping timeout: 255 seconds) |
| 2021-07-19 17:58:54 | <euouae> | It’s like map f xs but parallel |
| 2021-07-19 17:59:09 | <geekosaur> | if you're handling out subproblems to different threads/cores to be worked on, then there is at least some shared memory |
| 2021-07-19 17:59:12 | <euouae> | More or less, maybe with a stop condition |
| 2021-07-19 17:59:36 | <euouae> | Okay but this is the most basic form of it right? |
| 2021-07-19 17:59:40 | → | peterhil_ joins (~peterhil@dsl-hkibng32-54fb52-57.dhcp.inet.fi) |
| 2021-07-19 18:00:20 | → | benin03694 joins (~benin@183.82.177.8) |
| 2021-07-19 18:01:06 | → | merijn joins (~merijn@83-160-49-249.ip.xs4all.nl) |
| 2021-07-19 18:02:10 | × | benin0369 quits (~benin@106.198.95.190) (Ping timeout: 252 seconds) |
| 2021-07-19 18:02:10 | benin03694 | is now known as benin0369 |
| 2021-07-19 18:02:19 | × | peterhil quits (~peterhil@mobile-access-b04801-219.dhcp.inet.fi) (Ping timeout: 268 seconds) |
| 2021-07-19 18:04:02 | <euouae> | Anyway those issues are part of the algorithm right? |
| 2021-07-19 18:04:14 | → | Obo joins (~roberto@70.pool90-171-81.dynamic.orange.es) |
| 2021-07-19 18:04:14 | <euouae> | Not some other aspect of the peoblrm |
| 2021-07-19 18:04:20 | <euouae> | problem* |
| 2021-07-19 18:04:39 | × | raehik quits (~raehik@cpc95906-rdng25-2-0-cust156.15-3.cable.virginm.net) (Ping timeout: 255 seconds) |
| 2021-07-19 18:04:39 | → | burnsidesLlama joins (~burnsides@dhcp168-011.wadham.ox.ac.uk) |
| 2021-07-19 18:04:54 | <davean> | I mean your algorithm isn't determined |
| 2021-07-19 18:05:02 | <davean> | you design one to optimize for the HW |
| 2021-07-19 18:05:06 | <geekosaur> | you'd best hope they are, but that would mean it's been specialized to your supercomputer |
| 2021-07-19 18:05:27 | → | eggplantade joins (~Eggplanta@108-201-191-115.lightspeed.sntcca.sbcglobal.net) |
| 2021-07-19 18:05:51 | <davean> | what alg. is best depends on what your counting, and what the various costs are |
| 2021-07-19 18:06:02 | <davean> | along with your expected problem parameters |
| 2021-07-19 18:06:45 | <euouae> | So it’s really hard or are you just talking about the optimal case? |
| 2021-07-19 18:07:14 | <monochrom> | I would think "really hard" and "the optimal case" go well together. |
| 2021-07-19 18:07:55 | <davean> | Depends on the problem |
| 2021-07-19 18:08:14 | <davean> | if you're scaling it up past a very small case though I'm talking about what it usually takes to do a servicable job at it |
| 2021-07-19 18:08:49 | <euouae> | Davean are you speaking from experience with industry or research |
| 2021-07-19 18:08:59 | → | derelict joins (~derelict@user/derelict) |
| 2021-07-19 18:09:12 | <euouae> | obviously any optimization means more profit in industry I imagine it’s really right |
| 2021-07-19 18:09:15 | <davean> | yes, though my research wasn't particularly compute intensive even if it was "AI" |
| 2021-07-19 18:09:43 | × | burnsidesLlama quits (~burnsides@dhcp168-011.wadham.ox.ac.uk) (Ping timeout: 268 seconds) |
| 2021-07-19 18:09:50 | <euouae> | But in research there can be diminishing returns because your problem isn’t computationally feasible anyway after n=10 etc |
| 2021-07-19 18:10:22 | <euouae> | Okay… hmmm. Is it possible for me to get into this at all or am I hopeless? |
| 2021-07-19 18:10:36 | <davean> | euouae: I mean theres a huge difference between Floyd-Warshall and A* |
| 2021-07-19 18:10:43 | <davean> | euouae: but they both sorta solve the same problem space |
| 2021-07-19 18:10:44 | <geekosaur> | nobody here can tell that as we don't know your problem |
| 2021-07-19 18:10:59 | <euouae> | I don’t know those davean |
| 2021-07-19 18:11:31 | × | Obo quits (~roberto@70.pool90-171-81.dynamic.orange.es) (Ping timeout: 250 seconds) |
| 2021-07-19 18:11:55 | <euouae> | Okay maybe I’ll ask again if I have something more to show |
| 2021-07-19 18:11:56 | <davean> | ANd oh no, Floyd-Warshall is O(|V|^3), and A* is only O(|E|)! Clearly A* is always better? Except not at all |
| 2021-07-19 18:12:22 | → | fendor joins (~fendor@77.119.223.215.wireless.dyn.drei.com) |
| 2021-07-19 18:12:45 | <euouae> | Big constants davean? |
| 2021-07-19 18:12:45 | <davean> | well |E| can be ~ |V|^2, and what if you want the best paths, not approximations, but you're metric isn't admissible? And you want it for all of them? Or even just one side of those? |
| 2021-07-19 18:12:51 | <davean> | euouae: no, nothing to do with constants |
| 2021-07-19 18:13:25 | × | dunkeln quits (~dunkeln@188.71.231.232) (Ping timeout: 268 seconds) |
| 2021-07-19 18:13:40 | × | chris_ quits (~chris@81.96.113.213) (Remote host closed the connection) |
| 2021-07-19 18:13:41 | <davean> | Algrithmicly Floyd-Warshall is the same complexity for some use cases, because while they solve the same-ish problems, they do it for different uses |
| 2021-07-19 18:13:57 | <davean> | if you do something |V|^2 |V times, thats |V|^3 |
| 2021-07-19 18:14:17 | <davean> | But Floyd-Warshall requires more communication |
| 2021-07-19 18:14:27 | <davean> | A* is the "embarasingly" paralleler case |
| 2021-07-19 18:14:53 | <davean> | so if you're CPUs are very seperated, maybe you want to work more like that again, except not you can go back and plan the communication of F-W ... |
| 2021-07-19 18:15:40 | → | wroathe joins (~wroathe@c-68-54-25-135.hsd1.mn.comcast.net) |
| 2021-07-19 18:16:14 | <euouae> | Oh man I’m not following |
| 2021-07-19 18:17:29 | × | mnrmnaugh quits (~mnrmnaugh@pool-96-252-87-182.bstnma.fios.verizon.net) (Quit: Leaving) |
| 2021-07-19 18:18:10 | × | lavaman quits (~lavaman@98.38.249.169) (Ping timeout: 240 seconds) |
| 2021-07-19 18:18:36 | → | chris_ joins (~chris@81.96.113.213) |
| 2021-07-19 18:20:08 | × | cfricke quits (~cfricke@user/cfricke) (Ping timeout: 252 seconds) |
| 2021-07-19 18:20:36 | × | wroathe quits (~wroathe@c-68-54-25-135.hsd1.mn.comcast.net) (Ping timeout: 265 seconds) |
| 2021-07-19 18:21:41 | × | euandreh quits (~euandreh@191.181.59.160) (Quit: WeeChat 3.2) |
All times are in UTC.