zsh backspace headache

2007-10-2, 19:20(+00), kj:
[...]

Post by kj
Our system just underwent a major overhaul, and now on some terminals,
and *only under zsh*, backspace no longer erases to the left, but
instead it erases under the cursor. Under bash, however, on every
terminal (including those with the bad backspace action under zsh)
backspace still erases to the left. Given this, it seems fair to
hypothesize that the root of the problem has to do with the
configuration of zsh in the new system.
Unfortunately, I have not been able to find anything relevant to
this problem in the zsh documentation...
Out of desperation I tried
export INPUTRC=/etc/inputrc
and I also tried copying /etc/inputrc to ~/.inputrc. None of this
had the slightest effect, as far as I could tell.
How does zsh deal with the problems that bash uses /etc/inputrc or
~/.inputrc for?

[...]

inputrc is for the readline library that zsh thanksfully doesn't
use. See info -f zsh -n 'Zsh Line Editor' to learn about zsh's
line editor that is a lot more capable than GNU readline. key
binding is done through the bindkey and zle zsh builtins.

What do you see when you press <Ctrl-V><Backspace> on your
terminal?

What's the output of:

bindkey | grep delete

and

echo "$TERM"
infocmp -1 | grep -e dch1 -e bs

I'd suspect some /etc/zshrc or some file called by it actually
binds the character output by backspace to delete-char for some
reason. It might be because of a broken terminfo entry, hence
the question about infocmp.

--
Stéphane

2007-10-05 17:37:43 UTC

What do you see when you press <Ctrl-V><Backspace> on your
terminal?

The output of <Ctrl-V><Backspace> *looks* like this:

% ^[[3~

(where the leading " % " stands for my shell prompt.)

Post by Stephane CHAZELAS
bindkey | grep delete

"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

Post by Stephane CHAZELAS
and
echo "$TERM"

xterm-color

(or screen)

Post by Stephane CHAZELAS
infocmp -1 | grep -e dch1 -e bs

kbs=^H,

(No matches for dch1.)

Post by Stephane CHAZELAS
I'd suspect some /etc/zshrc or some file called by it actually
binds the character output by backspace to delete-char for some
reason. It might be because of a broken terminfo entry, hence
the question about infocmp.

The terminal program I'm using is OS X Terminal, which I use to
connect to a new 64-bit Linux server; it is this connection that
is giving me headaches now. In contrast, the connection to the
32-bit Linux server we were using until recently has no problems.
BTW, if I carry out the same diagnostics on the connection to the
old 32-bit server, the results are identical except that the output
for <Ctrl-V><Backspace> looks like this:

% ^?

The best solution I've found so far (though I have not followed
your advice yet) is to set the "Delete key sends backspace" option
in Terminal's Window Settings...>Keyboard. This seems to solve
the problem with backspace (though I don't understand why this is
now required whereas it wasn't before the server upgrade).

Still, even though now my backspace key is working as I want it
to, the display often becomes garbled whenenver I navigate a command
line that wraps around. And some man pages looked a bit screwed
up...

Anyway, thanks for your advice. I'll learn more about zsh's line
editor.

kj

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Stephane CHAZELAS

2007-10-05 18:06:51 UTC

What do you see when you press <Ctrl-V><Backspace> on your
terminal?

% ^[[3~
(where the leading " % " stands for my shell prompt.)

That sequence shouldn't be sent by the backspace key but by the
delete key.

Post by Stephane CHAZELAS
bindkey | grep delete

"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

So zsh is fine, that's your terminal or possibly ssh/sshd,
telnet/telnetd, rlogin/rlogind, screen (depending on how you
connect to your remote hosts) that's doing some wrong
translations.

Post by Stephane CHAZELAS
and
echo "$TERM"

xterm-color
(or screen)

hmmm, you're using GNU screen?

Post by Stephane CHAZELAS
infocmp -1 | grep -e dch1 -e bs

kbs=^H,
(No matches for dch1.)

I would have expected a kdch1, that is the sequence that delete
is supposed to send.

It would seem your terminfo database for that value of $TERM
doesn't match your actual terminal.

Post by kj
The terminal program I'm using is OS X Terminal, which I use to
connect to a new 64-bit Linux server; it is this connection that
is giving me headaches now. In contrast, the connection to the
32-bit Linux server we were using until recently has no problems.
BTW, if I carry out the same diagnostics on the connection to the
old 32-bit server, the results are identical except that the output
% ^?

[...]

With the same terminal?

Could you please clarify what you're doing.

You're starting an "OS X Terminal" application on machine A, is
that right? At this point what are the diagnostics (<Ctrl-C>,
$TERM, infocmp ... | grep)?

Then, you're starting "screen" in that same terminal? Then, what
do you get inside screen windows?

Then you're connecting to machine B (64bit) from within a screen
window? Is it via ssh, rlogin, telnet? Or are you starting
another emulator on that machine? And what are the diagnosis?

Same thing for machine C (32bit)?

--
Stéphane

2007-10-06 17:50:33 UTC

Hi. Many thanks for your reply. I give the results of all the
diagnostics below.

What do you see when you press <Ctrl-V><Backspace> on your
terminal?

% ^[[3~
(where the leading " % " stands for my shell prompt.)

That sequence shouldn't be sent by the backspace key but by the
delete key.

But note that in the data shown below, there is one case in which
backspace sends ^? and yet it still fails to work.

Post by Stephane CHAZELAS
bindkey | grep delete

"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

So zsh is fine, that's your terminal or possibly ssh/sshd,
telnet/telnetd, rlogin/rlogind, screen (depending on how you
connect to your remote hosts) that's doing some wrong
translations.

But if so, I don't understand why bash works. For example, if I
connect to this server from the local Mac's Terminal app via ssh (no
screen at all), my login shell is zsh, and my backspace key doesn't do
anything. If this problem were due to ssh, one would expect that
switching to bash on the same ssh session would make no difference,
but this is not so: if within that very same ssh/zsh session I execute
the command 'bash', now, in the resulting bash shell interaction, the
backspace key works fine. It was this piece of evidence that lead me
to suspect zsh in the first place.

Post by Stephane CHAZELAS
Could you please clarify what you're doing.

My normal sequence is to start the Terminal app in my iMac workstation
(machine A) and immediately to ssh to our main Linux server, which, as
of very recently, is a 64-bit machine running Ubuntu (machine B), but
used to be a 32-bit machine running SuSE (machine C). At any point I
can toggle the setting of the "Delete key sends backspace" option in
the Terminal app, which becomes effective immediately for the
currently active Terminal window. Also, I should note that the shell
for machines B and C is zsh, but for machine A the shell is bash.

Below I summarize the results of the diagnostics I ran, according to
your suggestions, plus a couple more of my own.

This summary is divided in two parallel sections which differ only in
the setting of the "Delete key sends backspace" option in the Terminal
app. The first of these two sections (in which this option is
unchecked) corresponds to the setting I used without any difficulties
until we started using machine B as our main linux host.

In each group of lines, the first line indicates the machine (A, B, or
C) under which the diagnostics were run.

The second line shows the two outputs I see when I hit
<Ctrl-V><Backspace>.

The third and fourth lines are the outputs of "echo $TERM" and
"infocmp -1 | grep -e dhc1 -e bs", respectively.

The fifth and sixth lines describe the behavior of the backspace key
on the command line and within a regular (i.e. non-shell) Emacs
buffer, respectively. On these lines, by "OK" I mean that hitting the
backspace key causes the cursor to move to the left and delete one
character; i.e. the desired behavior. In the case of Emacs, when the
this behavior is not "OK", I give the string that Emacs displays in
the "echo area" at the bottom of the screen. (Also, note the line
that I've marked with ***; under these conditions backspace fails, and
yet <Ctrl-V><Backspace> produces ^? .)

The seventh line indicates whether TERMCAP has been defined. (Usually
it isn't, but the value is the same whenever it is, and I give it at
the end of this post. BTW, I don't know for sure what sets this
variable, but I suppose it's GNU screen.)

The remaining lines (which are present only for machines B and C,
since bindkey is not available in bash) is the output of "bindkey |
grep delete".

=========================================================
"Delete key sends backspace" option in Terminal UNCHECKED
=========================================================

A
^?
xterm-color
kbs=^H,
backspace on CL: OK
backspace on Emacs: OK
TERMCAP?: NO

B
^?
xterm-color
kbs=^H,
backspace on CL: nothing happens ***
backspace on Emacs: OK
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^?" delete-char

B
^[[3~
screen
kbs=\177,
backspace on CL: nothing happens
backspace on Emacs: "End of buffer" (beep)
TERMCAP?: YES
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

C
^?
xterm-color
kbs=^H,
backspace on CL: OK
backspace on Emacs: OK
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

C
^?
screen
kbs=\177,
backspace on CL: OK
backspace on Emacs: OK
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

=======================================================
"Delete key sends backspace" option in Terminal CHECKED
=======================================================

A
^H
xterm-color
kbs=^H,
backspace on CL: OK
backspace on Emacs: OK

B
^H
xterm-color
kbs=^H,
backspace on CL: OK
backspace on Emacs: "C-h (Type ? for further options)-"
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^?" delete-char

B
^H
screen
kbs=\177,
backspace on CL: OK
backspace on Emacs: "C-h (Type ? for further options)-"
TERMCAP?: YES
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

C
^H
xterm-color
kbs=^H,
backspace on CL: OK
backspace on Emacs: "C-h (Type ? for further options)-"
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

C
^H
screen
kbs=\177,
backspace on CL: OK
backspace on Emacs: "C-h (Type ? for further options)-"
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

Also, in most cases $TERMCAP is not defined, but in those where it is, its value is:

SC|screen|VT 100/ANSI X3.64 virtual terminal:\
:DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:UP=\E[%dA:bs:bt=\E[Z:\
:cd=\E[J:ce=\E[K:cl=\E[H\E[J:cm=\E[%i%d;%dH:ct=\E[3g:\
:do=^J:nd=\E[C:pt:rc=\E8:rs=\Ec:sc=\E7:st=\EH:up=\EM:\
:le=^H:bl=^G:cr=^M:it#8:ho=\E[H:nw=\EE:ta=^I:is=\E)0:\
:li#40:co#126:am:xn:xv:LP:sr=\EM:al=\E[L:AL=\E[%dL:\
:cs=\E[%i%d;%dr:dl=\E[M:DL=\E[%dM:dc=\E[P:DC=\E[%dP:\
:im=\E[4h:ei=\E[4l:mi:IC=\E[%d@:ks=\E[?1h\E=:\
:ke=\E[?1l\E>:vi=\E[?25l:ve=\E[34h\E[?25h:vs=\E[34l:\
:ti=\E[?1049h:te=\E[?1049l:us=\E[4m:ue=\E[24m:so=\E[3m:\
:se=\E[23m:md=\E[1m:mr=\E[7m:me=\E[m:ms:\
:Co#8:pa#64:AF=\E[3%dm:AB=\E[4%dm:op=\E[39;49m:AX:\
:as=\E(0:ae=\E(B:\
:ac=\140\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:\
:k0=\E[10~:k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:k5=\E[15~:\
:k6=\E[17~:k7=\E[18~:k8=\E[19~:k9=\E[20~:k;=\E[21~:\
:F1=\E[23~:F2=\E[24~:F3=\E[25~:F4=\E[26~:F5=\E[28~:\
:F6=\E[29~:F7=\E[31~:F8=\E[32~:F9=\E[33~:FA=\E[34~:\
:kb=^H:kh=\E[1~:@1=\E[1~:kH=\E[4~:@7=\E[4~:kN=\E[6~:\
:kP=\E[5~:kI=\E[2~:kD=\E[3~:ku=\EOA:kd=\EOB:kr=\EOC:\
:kl=\EOD:km:

TIA,

kj

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Stephane CHAZELAS

2007-10-06 18:20:18 UTC

2007-10-6, 17:50(+00), kj:
[...]

Post by kj
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

So zsh is fine, that's your terminal or possibly ssh/sshd,
telnet/telnetd, rlogin/rlogind, screen (depending on how you
connect to your remote hosts) that's doing some wrong
translations.

And does <Ctrl-V><Backspace> show you ^[[3 in bash as well?

That would mean that bash maps ^[[3~ to the "del-to-the-left"
action, which wouldn't make sense, why would bash do such a
thing out of the blue while no terminal is meant to send that
sequence upon backspace. Unless you've configure bash to do
so...

[...]

[...]

Ok, so you observe the wrong behavior in screen only. It now
makes sense. Your terminfo entry is telling screen that your
terminal sends ^? upon <Backspace>. In that case, it's not
correct, as your terminal happens to be sending a ^? (\177) upon
<Backspace> but screen has no way to know about that. So screen
assumes that ^H means <Backspace> and ^? means <Delete>

screen is another terminal emulator. xterm uses X keypress
events that it translates into character sequences sent to the
application. screen does the same except that it sits inside
another terminal emulator, so that for screen, the keypress
events are sequences of characters received from the outer
termial.

screen implements a specific type of terminal that is described
to the applications in the screen terminfo entry. In that
terminfo entries, it says that the key for <Backspace> is ^? and
the key for <Delete> is ^[[3~. So screen will send ^[[3~ upon
the <Delete> key press event. And as the outer terminal claims
that its <Backspace> key sends event, ^? can't be the even
meaning <Backspace>, so screen assumes it's a delete (as there's
no kdch1 cap in the terminfo entry).

So, the problem is that you terminfo entry for xterm-color
doesn't match the actual behavior of your terminal. So, the
solution would be to align them. That is either make sure your
terminal sends ^H upon <Backspace>, or pick another $TERM that
matches better your terminal. I don't know about Apple
terminals, but you should be able to find the correct terminfo
entry for it.

You can also configure screen to tell it that terminfo is lying
to it, by correcting the wrong capabilities in your ~/.screenrc.
See the manual, I don't remember the details.

--
Stéphane

2007-10-06 22:34:44 UTC

[...]
Ok, so you observe the wrong behavior in screen only.

No, I also observe a wrong behavior without screen. In fact, I now
regret having mentioned screen at all, since it is really a red
herring (see below). I'm sure that screen is making matters worse,
but they were already messed up right after ssh'ing in, as shown in

Post by kj
B
^?
xterm-color
kbs=^H,
backspace on CL: nothing happens ***
backspace on Emacs: OK
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^?" delete-char

Backspace doesn't work on the shell command line in this case (though
it seems to be fine in Emacs). As a reminder, these are the
diagnostics that I ran on the 64-bit server right after ssh'ing in,
without running screen. Also, if from this point I run bash,
backspace works correctly, both on the command line and within Emacs.

BTW, this the case is the one I was referring to when I wrote in my

Post by kj
But note that in the data shown ..., there is one case in which
backspace sends ^? and yet it still fails to work.
(Also, note the line that I've marked with ***; under these conditions
backspace fails, and yet <Ctrl-V><Backspace> produces ^? .)

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Stephane CHAZELAS

2007-10-07 09:11:36 UTC

2007-10-6, 17:50(+00), kj:
[...]

Post by kj
B
^?
xterm-color
kbs=^H,
backspace on CL: nothing happens ***
backspace on Emacs: OK
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^?" delete-char

You've got ^? mapped to delete-char, so it deletes the character
under the cursor, not to the left of the cursor.

Normally, zsh by default maps both ^H and ^?, so you must have
something in /etc/zshrc or in ~/.zshrc that maps ^? to
delete-char. And the other entry below suggests that it is done
via something like:

(($+terminfo[kdch1])) && bindkey $terminfo[kdch1] delete-char

So I suspect that you should get this result:
$ infocmp -1 xterm-color | grep kdch1
kdch1=\177,

which is what I get here on a debian system.

So the problem again is the mismatch between the actual
behavior of your terminal and the way terminfo describes that
behavior to application. So, the wrong terminfo entry on B
confuses both screen (below) and /etc/zshrc. bash is probably
not affected because it doesn't try to be smart (that is
query the terminfo database) in finding what key sends what. So,
I'd suggest you use another value for $TERM. Alternatively, you
can /correct/ the xterm-color terminfo entry on B, by doing:

infocmp xterm-color > file
edit the file and change kbs and kdch1 to respectively \177 and
\E[3~. Then:

export TERMINFO=~/.terminfo
mkdir $TERMINFO
tic file

And add to your ~/.zlogin:
export TERMINFO=~/.terminfo

But there may be other caps that are incorrect in B's
xterm-color terminfo entry as everything suggests that this
terminfo entry is not meant to describe the MacOS X terminal.
So, as I said earlier, best is probably to adjust $TERM to a
terminal name whose terminfo entry matches your terminal. For
instance, on a debian system here, "xterm" seems to match better
your terminal.

Post by kj
B
^[[3~
screen
kbs=\177,
backspace on CL: nothing happens
backspace on Emacs: "End of buffer" (beep)
TERMCAP?: YES
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char
C
^?
xterm-color
kbs=^H,
backspace on CL: OK
backspace on Emacs: OK
TERMCAP?: NO
"^D" delete-char-or-list
"^H" backward-delete-char
"^[[3~" delete-char
"^?" backward-delete-char

That suggests that C has a different terminfo entry for
xterm-color from B's, or a different /etc/zshrc that hardcodes
^[[3~ for <Delete>

[...]

Post by kj
B
^H
xterm-color
kbs=^H,
backspace on CL: OK
backspace on Emacs: "C-h (Type ? for further options)-"

[...]

That's one of the many reasons I stopped using emacs a long time
ago. However, what that suggests above is the pseudo-terminal
driver has ^? for erase, as emacs/xemacs seem to query the
terminal driver (by doing the equivalent of "stty -a") to know
what character sends <Backspace>.

So, you'll find that if you do "stty erase '^H'", that is tell
the pseudo-terminal driver that your backspace actually sends
^H, the emacs problem will go away (but you won't be able to
access the help).

So, if you're an emacs user, I'd suggest you stick to ^? (that
also happens to be the default value for Linux terminal driver
"erase"), and make sure all the terminfo entries for the value
of $TERM on all the systems you're logging to are correct. In
particular, you don't want kdch1=\177 as this will cause
applications to thing that your <Backspace> key is a <Delete>
key.

--
Stéphane

2007-10-07 19:51:58 UTC

StÃ©phane, thank you very much for your patience and your helpful
posts. It'll take me a while to digest the last one. There's a
lot of info there that is entirely above my head at the moment,
but I intend to remedy that...

kj

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Stephane CHAZELAS

2007-10-07 20:32:18 UTC

Post by kj
B

[...]

Post by kj
"^H" backward-delete-char
"^?" delete-char

You've got ^? mapped to delete-char, so it deletes the character
under the cursor, not to the left of the cursor.
Normally, zsh by default maps both ^H and ^?, so you must have

Sorry, I meant:

Normally, zsh by default maps both ^H and ^? to
backward-delete-char, so...

Post by Stephane CHAZELAS
something in /etc/zshrc or in ~/.zshrc that maps ^? to
delete-char. And the other entry below suggests that it is done

[...]

--
Stéphane

a***@gwikzone.org

2007-10-30 12:24:56 UTC

I've posted the solution here :

http://blog.gwikzone.org/articles/2007/10/30/enable-backspace-when-ssh-into-a-debian-machine-with-zsh

Antonin.

Christian Schneider

2007-10-30 16:13:40 UTC