Strange characters on remove (rm) command

[RagingRaven]

Hello everybody,

I'm having a strange issue which I've never seen before and can't seem to find any info about on google.

When I do a file remove, the confirmation show the file with strange characters in front and at the end of the filename.
For example if I try to remove the file 'test.txt':
rm test.txt
I get:
rm: remove regular file âtest.txtâ?
if I enter 'y/n' it works normally though, so it's not really an issue, but I was wondering if this is something that's a bug or if I made a mistake somewhere.

edit: Did a bit more testing, when doing the above locally on the machine, I get the same response but the 'â' is a black questionmark in a white rounded background.
Before I did it through ssh, but I'm just guessing it's just the char it shows when it doesn't know which character it should be.
It looks to me like an encoding issue and it should be quotes or double-quotes or something it should show there.

edit 2: locale -a shows:

Code: Select all

C
CP1251
en_AG
en_AG.utf8
en_AU
en_AU.iso88591
en_AU.utf8
en_AU.UTF-8
en_BE
en_BE@euro
en_BE.iso88591
en_BE.utf8
en_BE.UTF-8
en_BW
en_BW.iso88591
en_BW.utf8
en_BW.UTF-8
en_CA
en_CA.iso88591
en_CA.utf8
en_CA.UTF-8
en_DK
en_DK.iso88591
en_DK.utf8
en_DK.UTF-8
en_GB
en_GB.iso88591
en_GB.utf8
en_GB.UTF-8
en_HK
en_HK.iso88591
en_HK.utf8
en_HK.UTF-8
en_IE
en_IE@euro
en_IE.iso88591
en_IE.utf8
en_IE.UTF-8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ
en_NZ.iso88591
en_NZ.utf8
en_NZ.UTF-8
en_PH
en_PH.iso88591
en_PH.utf8
en_PH.UTF-8
en_SG
en_SG.iso88591
en_SG.utf8
en_SG.UTF-8
en_US
en_US.iso88591
en_US.utf8
en_US.UTF-8
en_ZA
en_ZA.iso88591
en_ZA.utf8
en_ZA.UTF-8
en_ZM
en_ZM.utf8
en_ZW
en_ZW.iso88591
en_ZW.utf8
en_ZW.UTF-8
ISO-8859-1
ISO-8859-13
ISO-8859-14
ISO-8859-15
ISO-8859-2
ISO-8859-3
ISO-8859-4
ISO-8859-5
ISO-8859-7
ISO-8859-9
KOI8-R
KOI8-U
nl_AW
nl_AW.utf8
nl_BE
nl_BE@euro
nl_BE.iso88591
nl_BE.utf8
nl_BE.UTF-8
nl_NL
nl_NL@euro
nl_NL.iso88591
nl_NL.utf8
nl_NL.UTF-8
POSIX
UTF-8

[doktor5000]

Please show the output of as the affected user of

Code: Select all

locale
localectl status

Also, what Mageia version do you use?

[martinw]

I've not seen this when using ssh, but have seen it when using a local text login (pseudo-tty). For me, changing the environment variable LC_CTYPE from "en_GB.UTF-8" to "en_GB" fixes the problem. The 'rm' command is trying to use quote characters from the extended Unicode character set, but the pseudo-tty does not handle UTF-8 coding.

It would happen via ssh as well if the terminal you are running in does not support UTF-8.

(for doktor5000, this happens in both Mageia-4 and Mageia-5)

[doktor5000]

What is $TERM set to when this happens?

[RagingRaven]

locale shows:

Code: Select all

LANG=en_US.UTF-8
LC_CTYPE=en_US.UTF-8
LC_NUMERIC=nl_NL.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE=en_US.UTF-8
LC_MONETARY=nl_NL.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_PAPER=nl_NL.UTF-8
LC_NAME=nl_NL.UTF-8
LC_ADDRESS=nl_NL.UTF-8
LC_TELEPHONE=nl_NL.UTF-8
LC_MEASUREMENT=nl_NL.UTF-8
LC_IDENTIFICATION=nl_NL.UTF-8
LC_ALL=

localectl status shows:

Code: Select all

   System Locale: n/a

       VC Keymap: n/a
      X11 Layout: n/a

I have Mageia release 4.
I use putty to connect through ssh.

echo $TERM shows:
xterm

[doktor5000]

You need to configure putty to show UTF-8, and you may also need to use a different font:
http://thegreyblog.blogspot.de/2009/08/ ... utf-8.html
http://serverfault.com/questions/475925 ... characters
http://superuser.com/questions/393834/h ... characters

[RagingRaven]

You need to configure putty to show UTF-8, and you may also need to use a different font:
http://thegreyblog.blogspot.de/2009/08/ ... utf-8.html
http://serverfault.com/questions/475925 ... characters
http://superuser.com/questions/393834/h ... characters

Ok, thanks, that works for putty, but what about the local terminal?

btw the 'strange chars' appear to be ‘ and ’ so not normal single quotes ( ' ):
rm: remove regular file ‘test.txt’? y

[doktor5000]

What exactly do you mean by local terminal? Switching to or logging in via a tty via Ctrl+Alt+[F1-F8] ?
It is probably also a font issue, or maybe about the terminal type.

Does it work if you do

Code: Select all

export LC_ALL=C

before the rm?

Otherwise, you could always use rm -f so you won't see those characters again as you won't be asked ...

[RagingRaven]

I mean on the machine itself.
I have mageia installed withouth automatically starting the x environment, so it starts in the console/terminal/tty (or whatever it's called) where I have to enter 'startx' to start the gui.

That is the console I'm referring to.
I also get back to that when I choose to log out.

But it doesn't happen in the console I start from gui: application launcher menu -> tools -> konsole.

If I do 'export LC_ALL=C' and then do rm it shows correctly.
But I have to do this again when I do 'su -l' and after a reboot it's no longer working. (but I'm guessing that's normal as the export command is only temporary for the current session?)

I also tried to 'reset' the locale settings in the gui through mageia control center, but this has no effect on the console/terminal when I logout.

Otherwise, you could always use rm -f so you won't see those characters again as you won't be asked ...

Yup I could as said it's not really that big of a deal, but perhaps it's a bug, so I thought it would be nice to know

[doktor5000]

I'll try to play around with this, as some characters from my prompt also don't show up in a tty login.

Code: Select all

┌─[doktor5000@Mageia5]─[17:51:47]─[~]
└──╼

would show up as

Code: Select all

┌─[doktor5000@Mageia5]─[17:51:47]─[~]
└──�

[RagingRaven]

Cool, I'll await your findings.

[doktor5000]

Actually, it turns out to be pretty easy. You just need to change the font to one that includes a unicode map. See https://wiki.archlinux.org/index.php/Fo ... sole_fonts
For Mageia, the console fonts are located below /lib/kbd/consolefonts. I filtered out the latin ones including a unicode map:

Code: Select all

ll /lib/kbd/consolefonts/ | grep -i lat | grep psfu

I tried Lat2-Terminus16 via

Code: Select all

setfont Lat2-Terminus16

and now rm shows " instead of â around the filename.

To make that persistent, put it into /etc/vconsole.conf as shown in abovemention arch wiki page.
This file is read on boot by systemd-vconsole-setup.

Related, please make sure that the terminal is UTF-8 enabled. stty should display iutf8 if that is the case. If it's not the case, run

Code: Select all

stty iutf8

and reload the font via setfont as mentioned above.

[RagingRaven]

Good, I'll give it a try when I'm back at work tomorrow.

Just curious...is this because I set some locale variables to dutch or should this be something that should work by default?

[doktor5000]

Seems the default font does not contain a unicode map, also seems default font is not properly set.
This is explained by the absence of /etc/vconsole.conf in a default installation, where the font should be defined.

[RagingRaven]

Changing the font seems to be the fix indeed, everything appears to work as expected now.

Should I mark this thread as solved ? Seeing as the underlying problem is not yet fixed ?
In other words should there be a bug report, because I guess a default installation should have the correct font?

Thanks for all the help though.

Edit: I thought everything worked, but I can't get /etc/vconsole.conf to work.
My /etc/vconsole.conf file looks like this now:

Code: Select all

FONT=Lat2-Terminus16

I also tried:

Code: Select all

LANG=en-US.UTF-8
FONT=Lat2-Terminus16

Does /etc/vconsole.conf need specific permissions/owner?

I can change the font manually with setfont, but after a reboot, it appears the vconsole.conf file isn't read.

[RagingRaven]

Could the vconsole.conf issue I have be related to: viewtopic.php?f=7&t=5637&p=38346&hilit=vconsole.conf#p38346 ?

[RagingRaven]

From the topic mentioned, I gathered I could change /etc/sysconfig/i18n.
So I made a copy of that file (just to make sure I could restore it if I screwed things up), edited the file and changed the sysfont to use the Terminus font:

Code: Select all

SYSFONT=Lat2-Terminus16

This appears to work, but I'm not sure if this is actually the correct way to do things?

[doktor5000]

Well, we're currently in a transient state between the "old way to do things" which would be partly /etc/sysconfig/i18n vs. the "new systemd way" which would be vconsole.conf.
It it works for you that's great
Although you should note that down as a reminder for the next installation to recheck this. vconsole.conf should be the proper way to do things,
but we may need to iron some wrinkles out there, you probably stumbled upon one of those wrinkles ...
Maybe related: https://bugs.freedesktop.org/show_bug.cgi?id=88373 and maybe one of those: https://bugs.mageia.org/buglist.cgi?qui ... t_id=45835

What does

Code: Select all

systemctl status systemd-vconsole-setup -a
systemctl is-enabled systemd-vconsole-setup

show as root?

[RagingRaven]

systemctl status systemd-vconsole-setup -a:

Code: Select all

systemd-vconsole-setup.service - Setup Virtual Console
   Loaded: loaded (/usr/lib/systemd/system/systemd-vconsole-setup.service; static)
   Active: active (exited) since Sat 2015-04-25 09:09:02 CEST; 1min 8s ago
     Docs: man:systemd-vconsole-setup.service(8)
           man:vconsole.conf(5)
  Process: 758 ExecStart=/usr/lib/systemd/systemd-vconsole-setup (code=exited, status=0/SUCCESS)
 Main PID: 758 (code=exited, status=0/SUCCESS)

Apr 25 09:09:02 hoofdserver.sunteam.nl systemd[1]: Starting Setup Virtual Console...
Apr 25 09:09:02 hoofdserver.sunteam.nl systemd[1]: Started Setup Virtual Console.

systemctl is-enabled systemd-vconsole-setup:

Code: Select all

static

[doktor5000]

Hmmm, seems vconsole.conf only exists in initrd, and is only used during early boot.
Check e.g. https://bugs.mageia.org/show_bug.cgi?id=5266 and the linked https://bugs.mageia.org/show_bug.cgi?id=5192

I can verify that it exists in the initrd, together with the font I've chosen:

Code: Select all

┌─[doktor5000@Mageia5]─[19:19:10]─[~]
└──╼ sudo lsinitrd /boot/initrd.img | grep console
crw-r--r--   1 root     root       5,   1 Apr 18 23:15 dev/console
-rw-r--r--   1 root     root          168 Jun 30  2014 etc/udev/rules.d/10-console.rules
-rw-r--r--   1 root     root           78 Apr 18 23:15 etc/vconsole.conf
drwxr-xr-x   1 root     root            0 Apr 18 23:15 usr/lib/kbd/consolefonts
-rw-r--r--   1 root     root         4949 Oct 17  2014 usr/lib/kbd/consolefonts/lat0-16.psfu
-rw-r--r--   1 root     root         9836 Oct 17  2014 usr/lib/kbd/consolefonts/LatArCyrHeb-16.psfu
-rwxr-xr-x   1 root     root         1722 Jun 30  2014 usr/lib/udev/console_init

[doktor5000]

Well, found one alternative, at least for the ttys: http://www.freedesktop.org/wiki/Software/kmscon/ / https://wiki.archlinux.org/index.php/KMSCON
Offers full unicode support by default and looks nicer and brighter then the default classic tty terminal.

Will try to investigate further for the vconsole issue.

[RagingRaven]

That one looks nice indeed, I'll await your findings.

[doktor5000]

For mga5 I could attach the packages here for preliminary tests ... so ping me if you want them

[RagingRaven]

Sorry was bit busy last couple of days.
I'm not using mga5 yet and i18n is working for me now, so for now that's fine.
I'll just wait till either this issue gets fixed in mga4 or wait for mga5 to get an official stable release

[doktor5000]

I'll just wait till either this issue gets fixed in mga4 or wait for mga5 to get an official stable release

Well, there won't ne a fix if there's no bugreport for this. And the issue still exists in Mageia 5.