Logs: liberachat/#haskell
| 2025-08-21 09:47:42 | × | tromp quits (~textual@2001:1c00:3487:1b00:1542:a509:831e:a9dc) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2025-08-21 09:58:49 | → | tromp joins (~textual@2001:1c00:3487:1b00:1542:a509:831e:a9dc) |
| 2025-08-21 10:03:49 | × | vgtw_ quits (~vgtw@user/vgtw) (Server closed connection) |
| 2025-08-21 10:04:41 | → | vgtw joins (~vgtw@user/vgtw) |
| 2025-08-21 10:06:21 | → | tremon joins (~tremon@83.80.159.219) |
| 2025-08-21 10:06:42 | <Athas> | Under which circumstances will GHC encode characters read from a handle as '?'? |
| 2025-08-21 10:06:55 | <Athas> | I am trying to debug a test bug that appears only in CI, and which I cannot reproduce locally. |
| 2025-08-21 10:07:26 | <Athas> | From what I can see, anything non-ASCII is turned into '?'. |
| 2025-08-21 10:09:26 | × | pointlessslippe1 quits (~pointless@62.106.85.17) (Read error: Connection reset by peer) |
| 2025-08-21 10:09:49 | × | pavonia quits (~user@user/siracusa) (Quit: Bye!) |
| 2025-08-21 10:10:39 | → | fp joins (~Thunderbi@wireless-86-50-140-217.open.aalto.fi) |
| 2025-08-21 10:10:52 | × | lortabac quits (~lortabac@2a01:e0a:541:b8f0:55ab:e185:7f81:54a4) (Quit: WeeChat 4.5.2) |
| 2025-08-21 10:11:32 | <ncf> | see https://hackage.haskell.org/package/base-4.21.0.0/docs/System-IO.html#g:22 |
| 2025-08-21 10:11:46 | trickard_ | is now known as trickard |
| 2025-08-21 10:11:46 | <ncf> | tldr your CI's environment is probably not using a unicode locale, try setting LC_ALL=C.UTF-8 |
| 2025-08-21 10:14:13 | <Athas> | I am pretty sure I already tried setting LC_ALL before (and LANG is already C.UTF-8), but let me try again. |
| 2025-08-21 10:15:07 | → | pointlessslippe1 joins (~pointless@62.106.85.17) |
| 2025-08-21 10:17:56 | × | xff0x quits (~xff0x@fsb6a9491c.tkyc517.ap.nuro.jp) (Ping timeout: 244 seconds) |
| 2025-08-21 10:19:32 | <merijn> | Athas: Are you using text/bytestring? |
| 2025-08-21 10:19:45 | <Athas> | Yeah, no difference. |
| 2025-08-21 10:20:21 | <Athas> | merijn: this is in the haskeline test suite. I am using a combination of ByteString and raw C strings to communicate with another process. |
| 2025-08-21 10:20:24 | <merijn> | With explicit decoding from bytestring to text you can specify how to handle parse errors |
| 2025-08-21 10:20:41 | <Athas> | That's part of my debugging challenge: not only can I not reproduce outside of CI, but I also don't know exactly where the encoding gets fucked up. |
| 2025-08-21 10:20:42 | <merijn> | Athas: Oh, sadly Haskeline isn't very principled in its API |
| 2025-08-21 10:20:59 | <merijn> | At least, it wasn't last I used it (admittedly a while ago) |
| 2025-08-21 10:21:21 | <merijn> | I assume the decoding happening inside it is doing that? |
| 2025-08-21 10:21:53 | <Athas> | Or maybe the encoding - I am also sending non-ASCII to the other process. |
| 2025-08-21 10:22:15 | <Athas> | The encoding is explicitly done with encodeUtf8 so that is fine, but something is turning all non-ASCII bytes into question marks. |
| 2025-08-21 10:22:54 | <Athas> | The test suite explicitly sets LANG to en_US.UTF-8, so it should not be sensitive to the system environment. |
| 2025-08-21 10:23:26 | <Athas> | This captures pretty well what I am looking at: https://github.com/haskell/haskeline/actions/runs/17124026707/job/48571413738?pr=203#step:23:39 |
| 2025-08-21 10:25:50 | <Athas> | Oh, the writes are also done by turning the bytestring into a C array and writing the raw bytes to a handle. There should be no non-explicit encoding/decoding anywhere! |
| 2025-08-21 10:26:23 | <jreicher> | Athas: are you writing to a file? |
| 2025-08-21 10:26:50 | <Athas> | jreicher: no, a pipe connected to a subprocess. |
| 2025-08-21 10:27:25 | <Athas> | Well, actually a handle produced by openPseudoTerminal. |
| 2025-08-21 10:27:38 | <Athas> | But I guess it is a pipe at the kernel level. |
| 2025-08-21 10:27:53 | × | merijn quits (~merijn@77.242.116.146) (Ping timeout: 245 seconds) |
| 2025-08-21 10:29:21 | <Athas> | This is the program that is the other side of the pipe: https://github.com/haskell/haskeline/blob/master/examples/Test.hs Now I am trying to figure out if that is doing something wrong. It certainly has all the encoding/decoding machinery in operation. |
| 2025-08-21 10:37:45 | <jreicher> | Can you capture a copy of the bytes before they go into the pipe? |
| 2025-08-21 10:38:23 | <Athas> | I don't see why not; it's just tedious to debug with such a high roundtrip latency. |
| 2025-08-21 10:39:09 | <jreicher> | It's just to cover off the possibility that the bytes are fine and the problem is somewhere else. |
| 2025-08-21 10:39:43 | <jreicher> | For example I'm always suspicious of seeing things like ? in a terminal. I don't know if it's the terminal emulator doing that or if it's the actual process output. |
| 2025-08-21 10:39:50 | <jreicher> | So I send to a file and examine the file. |
| 2025-08-21 10:40:25 | → | merijn joins (~merijn@77.242.116.146) |
| 2025-08-21 10:42:37 | <Athas> | It's not the terminal emulator for sure, since there is none. |
| 2025-08-21 10:43:18 | <Athas> | But now I also found another case of flakiness, as it is apparently system-dependent whether a line is terminated by \r\n or \ESCE... |
| 2025-08-21 10:43:29 | <Athas> | God damn do I dislike the Unix tty system. |
| 2025-08-21 10:43:55 | <Athas> | I have a hunch the haskeline tests never actually worked reliably, but just happened to work on the original author's machine. |
| 2025-08-21 10:47:10 | × | merijn quits (~merijn@77.242.116.146) (Ping timeout: 256 seconds) |
| 2025-08-21 10:49:17 | <jreicher> | I was worried the use of tty might play into it, but I actually don't have any real knowledge about that system. Is there a binary mode that might cut out having an EOL convention at all? |
| 2025-08-21 10:49:28 | → | merijn joins (~merijn@77.242.116.146) |
| 2025-08-21 10:50:46 | <Athas> | Yes, and from what I can see all the tests that use files work fine. |
| 2025-08-21 10:51:03 | <Athas> | It is definitely some tty thing. |
| 2025-08-21 10:52:07 | <[exa]> | Athas: tbh I'd expect some very weird tty settings on the github CI side |
| 2025-08-21 10:53:20 | <Athas> | [exa]: I am also seeing different tty behaviour between my system and some department server. And that's just NixOS vs RHEL, and with the same environments from my .profile... |
| 2025-08-21 10:53:56 | <Athas> | I am increasingly confident I don't want to spend the time needed to figure this one out. Haskeline does actually work fine; it's just that its tests don't. |
| 2025-08-21 10:54:39 | <[exa]> | if I got it right, the test is throwing random valid unicode at the terminal and expects it to get through right? |
| 2025-08-21 10:54:45 | <Athas> | Yes. |
| 2025-08-21 10:55:15 | <Athas> | But that is not actually all; *something* is also *sometimes* turning linebreaks into \ESCE, but I expect that to be some Haskeline-logic based on what it thinks the terminal might be like. |
| 2025-08-21 10:56:12 | <[exa]> | can you run the whole thing with TERM= set to something very safe? |
| 2025-08-21 10:56:34 | <[exa]> | also the warnings that setlocale can't set locale don't spawn much confidence |
| 2025-08-21 10:58:04 | <Athas> | I have also tried with locales (C.UTF-8) that didn't trigger that warning, with no apparent effect. TERM=dumb and TERM=xterm has no apparent effect. |
| 2025-08-21 11:00:04 | × | caconym747 quits (~caconym@user/caconym) (Quit: bye) |
| 2025-08-21 11:00:43 | → | athan joins (~athan@syn-047-132-161-157.res.spectrum.com) |
| 2025-08-21 11:03:54 | → | caconym747 joins (~caconym@user/caconym) |
| 2025-08-21 11:06:25 | <Athas> | Actually that was a useful clue, because after changing the haskeline test suite to explicit set C-UTF-8 (instead of en_US.UTF-8), those encoding errors seem to go away. |
| 2025-08-21 11:06:40 | <Athas> | But it didn't warn about those; only when I hacked in some explicit environments in the CI config. Annoying. |
| 2025-08-21 11:07:02 | <Athas> | Now there is just some tab completion problem left, which is probably related to working directories. |
| 2025-08-21 11:09:30 | → | lortabac joins (~lortabac@2a01:e0a:541:b8f0:55ab:e185:7f81:54a4) |
| 2025-08-21 11:18:35 | → | xff0x joins (~xff0x@2405:6580:b080:900:5c56:788a:c6b4:ed46) |
| 2025-08-21 11:19:09 | × | trickard quits (~trickard@cpe-58-98-47-163.wireline.com.au) (Ping timeout: 260 seconds) |
| 2025-08-21 11:19:28 | × | merijn quits (~merijn@77.242.116.146) (Ping timeout: 256 seconds) |
| 2025-08-21 11:20:58 | → | merijn joins (~merijn@77.242.116.146) |
| 2025-08-21 11:23:41 | × | fp quits (~Thunderbi@wireless-86-50-140-217.open.aalto.fi) (Quit: fp) |
| 2025-08-21 11:24:22 | → | fp joins (~Thunderbi@2001:708:150:10::72df) |
| 2025-08-21 11:24:57 | × | Lord_of_Life quits (~Lord@user/lord-of-life/x-2819915) (Ping timeout: 252 seconds) |
| 2025-08-21 11:28:33 | → | Lord_of_Life joins (~Lord@user/lord-of-life/x-2819915) |
| 2025-08-21 11:30:30 | → | Lycurgus joins (~juan@user/Lycurgus) |
| 2025-08-21 11:31:10 | → | trickard_ joins (~trickard@cpe-61-98-47-163.wireline.com.au) |
| 2025-08-21 11:31:38 | trickard_ | is now known as trickard |
| 2025-08-21 11:33:00 | × | Frostillicus quits (~Frostilli@pool-71-174-119-69.bstnma.fios.verizon.net) (Remote host closed the connection) |
| 2025-08-21 11:33:23 | → | Frostillicus joins (~Frostilli@pool-71-174-119-69.bstnma.fios.verizon.net) |
| 2025-08-21 11:37:09 | → | Square2 joins (~Square4@user/square) |
| 2025-08-21 11:40:00 | × | tromp quits (~textual@2001:1c00:3487:1b00:1542:a509:831e:a9dc) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2025-08-21 11:50:01 | × | trickard quits (~trickard@cpe-61-98-47-163.wireline.com.au) (Read error: Connection reset by peer) |
| 2025-08-21 11:50:14 | → | trickard_ joins (~trickard@cpe-61-98-47-163.wireline.com.au) |
| 2025-08-21 11:52:53 | → | poscat0x04 joins (~poscat@user/poscat) |
| 2025-08-21 11:54:39 | × | poscat quits (~poscat@user/poscat) (Ping timeout: 252 seconds) |
| 2025-08-21 11:55:52 | × | Frostillicus quits (~Frostilli@pool-71-174-119-69.bstnma.fios.verizon.net) (Ping timeout: 255 seconds) |
| 2025-08-21 12:02:49 | → | weary-traveler joins (~user@user/user363627) |
| 2025-08-21 12:08:45 | → | Frostillicus joins (~Frostilli@pool-71-174-119-69.bstnma.fios.verizon.net) |
| 2025-08-21 12:11:33 | × | liskin quits (~liskin@xmonad/liskin) (Server closed connection) |
| 2025-08-21 12:11:51 | → | liskin joins (~liskin@xmonad/liskin) |
| 2025-08-21 12:15:53 | → | marinelli joins (~weechat@gateway/tor-sasl/marinelli) |
| 2025-08-21 12:15:54 | × | trickard_ quits (~trickard@cpe-61-98-47-163.wireline.com.au) (Read error: Connection reset by peer) |
| 2025-08-21 12:18:10 | → | trickard_ joins (~trickard@cpe-61-98-47-163.wireline.com.au) |
| 2025-08-21 12:26:18 | × | trickard_ quits (~trickard@cpe-61-98-47-163.wireline.com.au) (Read error: Connection reset by peer) |
| 2025-08-21 12:28:47 | → | trickard_ joins (~trickard@cpe-61-98-47-163.wireline.com.au) |
| 2025-08-21 12:42:45 | → | haritz joins (~hrtz@2a01:4b00:bc2e:7000:d5af:a266:ca31:5ef8) |
| 2025-08-21 12:42:45 | × | haritz quits (~hrtz@2a01:4b00:bc2e:7000:d5af:a266:ca31:5ef8) (Changing host) |
| 2025-08-21 12:42:45 | → | haritz joins (~hrtz@user/haritz) |
All times are in UTC.