Discussion:
[PATCH] optional fatter cat(1)
(too old to reply)
enh
2014-12-19 03:38:01 UTC
Permalink
This basically brings the catv code into cat (but optionally enabled,
like the 'big' variant of sort) and adds the missing-from-both -n
option. There are a couple of options in coreutils cat still missing,
and a few more still in BSD's cat, but they're pretty useless. (I've
commented the missing coreutils options on the assumption that someone
might care, but haven't even done that for the missing BSD options.)

diff --git a/toys/posix/cat.c b/toys/posix/cat.c
index 3644c4f..758248a 100644
--- a/toys/posix/cat.c
+++ b/toys/posix/cat.c
@@ -4,7 +4,7 @@
*
* See http://opengroup.org/onlinepubs/9699919799/utilities/cat.html

-USE_CAT(NEWTOY(cat, "u", TOYFLAG_BIN))
+USE_CAT(NEWTOY(cat, USE_CAT_FAT("AETentv")"u", TOYFLAG_BIN))

config CAT
bool "cat"
@@ -16,10 +16,33 @@ config CAT
Filename "-" is a synonym for stdin.

-u Copy one byte at a time (slow).
+
+config CAT_FAT
+ bool "cat [-AETentv]"
+ default y
+ depends on CAT
+ help
+ usage: cat [-AETentv] [file...]
+
+ Display nonprinting characters as escape sequences. Use M-x for
+ high ascii characters (>127), and ^x for other nonprinting chars.
+
+ -A Show all non-printable (same as -vET).
+ -E Show $ at line ends.
+ -T Show ^I for tabs.
+ -e Show non-printable and line ends (same as -vE).
+ -n Number all lines.
+ -t Show non-printable and tabs (same as -vT).
+ -v Show non-printable with ^x or M-x, except for line ends and tabs.
*/

+#define FOR_cat
#include "toys.h"

+// Still missing:
+// -b (number non-blank lines only)
+// -s (suppress repeated blank lines)
+
static void do_cat(int fd, char *name)
{
int len, size=toys.optflags ? 1 : sizeof(toybuf);
@@ -32,7 +55,58 @@ static void do_cat(int fd, char *name)
}
}

+#if CFG_CAT_FAT
+static void do_fat_cat(int fd, char *name)
+{
+ int line = 1, at_line_start = 1;
+ for (;;) {
+ int i, len;
+
+ len = read(fd, toybuf, sizeof(toybuf));
+ if (len < 0) toys.exitval = EXIT_FAILURE;
+ if (len < 1) break;
+ for (i=0; i<len; i++) {
+ char c=toybuf[i];
+
+ if (at_line_start && toys.optflags & FLAG_n) {
+ printf("% 6d ", line++);
+ at_line_start = 0;
+ }
+ if (c > 126 && (toys.optflags & FLAG_v)) {
+ if (c > 127) {
+ printf("M-");
+ c -= 128;
+ }
+ if (c == 127) {
+ printf("^?");
+ continue;
+ }
+ }
+ if (c < 32) {
+ if (c == 10) {
+ if (toys.optflags & FLAG_E) xputc('$');
+ at_line_start = 1;
+ } else if (toys.optflags & (c==9 ? FLAG_T : FLAG_v)) {
+ printf("^%c", c+'@');
+ continue;
+ }
+ }
+ xputc(c);
+ }
+ }
+}
+#endif
+
void cat_main(void)
{
- loopfiles(toys.optargs, do_cat);
+ if ((toys.optflags & ~FLAG_u) == 0) {
+ loopfiles(toys.optargs, do_cat);
+ } else {
+#if CFG_CAT_FAT
+ if (toys.optflags & FLAG_A) toys.optflags |= FLAG_v|FLAG_E|FLAG_T;
+ if (toys.optflags & FLAG_e) toys.optflags |= FLAG_v|FLAG_E;
+ if (toys.optflags & FLAG_t) toys.optflags |= FLAG_v|FLAG_T;
+ loopfiles(toys.optargs, do_fat_cat);
+#endif
+ }
}
Rob Landley
2014-12-19 19:22:12 UTC
Permalink
Post by enh
This basically brings the catv code into cat (but optionally enabled,
like the 'big' variant of sort) and adds the missing-from-both -n
option. There are a couple of options in coreutils cat still missing,
and a few more still in BSD's cat, but they're pretty useless.
Do you have an actual use case for these?

I'm curious because when I did catv as a separate command in busybox in
2006, I did so waiting for somebody to actually complain that it broke a
script. Technically, I'm still waiting, although I don't follow the
busybox list quite so closely these days.

Posix only defined the -u option, and LSB is so _awesomely_ craptacular
a standard that not only do they not mention "cat" at all, but they
deprecated "ar" (presumably because of Ulrich Dr. Pepper's personal
dislike of static linking).

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/cat.html

http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html

In implementing catv, I was really responding to original bell labs unix
developer Rob Pike's 1983 usenix paper telling the BSD guys they were
full of it (popularly called "cat -v considered harmful"):

http://harmful.cat-v.org/cat-v/unix_prog_design.pdf

(I saw
http://www.all-things-android.com/content/pros-and-cons-busybox-android
but he wasn't actually using it for anything either.)

Again, the above is not a "no", just a request for more information. If
we already implement catv, then having it available in cat (via config
option) isn't a big stretch. I'm just curious what use case finally
actually needed it after 8 years...

Rob
enh
2014-12-19 19:54:55 UTC
Permalink
Post by Rob Landley
Post by enh
This basically brings the catv code into cat (but optionally enabled,
like the 'big' variant of sort) and adds the missing-from-both -n
option. There are a couple of options in coreutils cat still missing,
and a few more still in BSD's cat, but they're pretty useless.
Do you have an actual use case for these?
i use both cat -v and -n (separately at least). i've never used -e/-E
or -t/-T; i just brought them across from catv. one reason i didn't
add -b/-s (pun intended) is that i didn't even know they existed. (or
the BSD -l and -B.)
Post by Rob Landley
I'm curious because when I did catv as a separate command in busybox in
2006, I did so waiting for somebody to actually complain that it broke a
script. Technically, I'm still waiting, although I don't follow the
busybox list quite so closely these days.
i only use them interactively. a script can see non-printable
characters just fine anyway, and although i feel like a barbarian if i
have to count lines by hand, in a script it's probably easier to
maintain a count than it is to parse one.

that's also the reason why i don't care that BSD and toybox have a
byte-by-byte implementation of the options --- they're only useful
(imho) if there's a human reader, and byte-by-byte is plenty fast
enough for that.
Post by Rob Landley
Posix only defined the -u option,
(an option i've never used, nor known anyone use.)
Post by Rob Landley
and LSB is so _awesomely_ craptacular
a standard that not only do they not mention "cat" at all, but they
deprecated "ar" (presumably because of Ulrich Dr. Pepper's personal
dislike of static linking).
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/cat.html
http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html
In implementing catv, I was really responding to original bell labs unix
developer Rob Pike's 1983 usenix paper telling the BSD guys they were
http://harmful.cat-v.org/cat-v/unix_prog_design.pdf
(I saw
http://www.all-things-android.com/content/pros-and-cons-busybox-android
but he wasn't actually using it for anything either.)
interesting. i hadn't seen that. some things are definitely wrong
(well, assuming busybox is like toybox/coreutils anyway) and others
are out of date, and the most interesting thing to me is "which
missing commands are actually causing pain for developers?". 360
symlinks is enough that people will start to notice :-) right now the
toybox binary is slightly bigger than the toolbox binary was at its
peak, though it offers significantly more functionality, and internal
developers have already been pleased to have find, od (!), and
truncate.

(sadly the most common request seems to be "ooh, does this mean we can
have less now?".)
Post by Rob Landley
Again, the above is not a "no", just a request for more information. If
we already implement catv, then having it available in cat (via config
option) isn't a big stretch. I'm just curious what use case finally
actually needed it after 8 years...
for me, "| cat -v" has always been the least obtrusive "is this
misbehaving for reasons i can't see?". yes, there's always hd, but
that's a lot noisier and -- going back to -e/-E/-t/-T -- the fact that
it thinks that line ends and tabs are non-printable is a disadvantage
for that kind of thing.

i know there's catv and nl already in toybox, but part of my goal with
moving us to toybox is that people's muscle memory should just work.
the guy who found find found it by accident, for example. i haven't
sent out any kind of announcement yet, and may not even do so if
people continue to just be pleasantly surprised (or don't even
notice!). http://en.wikipedia.org/wiki/Boiling_frog

--elliott
Rob Landley
2014-12-19 20:05:10 UTC
Permalink
Post by enh
i know there's catv and nl already in toybox, but part of my goal with
moving us to toybox is that people's muscle memory should just work.
A good argument. :)

Rob
Rich Felker
2014-12-19 21:05:12 UTC
Permalink
Post by Rob Landley
Post by enh
i know there's catv and nl already in toybox, but part of my goal with
moving us to toybox is that people's muscle memory should just work.
A good argument. :)
Could the cat toy just detect the -v option via a flag and jump to the
catv toy instead when it's set? This would avoid polluting its code
with that mess. The -v option and the code to jump could just be
omitted when catv is not included in the build.

Rich
enh
2014-12-19 21:48:11 UTC
Permalink
I want to remove catv though.
Post by Rich Felker
Post by Rob Landley
Post by enh
i know there's catv and nl already in toybox, but part of my goal with
moving us to toybox is that people's muscle memory should just work.
A good argument. :)
Could the cat toy just detect the -v option via a flag and jump to the
catv toy instead when it's set? This would avoid polluting its code
with that mess. The -v option and the code to jump could just be
omitted when catv is not included in the build.
Rich
Rich Felker
2014-12-19 21:50:41 UTC
Permalink
Post by enh
I want to remove catv though.
Is there a reason you need to, though? This seems like purely an
aesthetic consideration to be balanced with the aesthetic
consideration of not having "catv" code in the main cat.

Rich
enh
2014-12-19 22:41:30 UTC
Permalink
like i said earlier, my aim is be roughly like coreutils. catv is some
weird made-up thing. and i _will_ get push back about adding too many
commands, so i intend to make them all count.
Post by Rich Felker
Post by enh
I want to remove catv though.
Is there a reason you need to, though? This seems like purely an
aesthetic consideration to be balanced with the aesthetic
consideration of not having "catv" code in the main cat.
Rich
--
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Java i18n/JNI/NIO, or bionic questions? Mail me/drop by/add me as a reviewer.
Marc André Tanner
2014-12-19 23:15:36 UTC
Permalink
Post by enh
like i said earlier, my aim is be roughly like coreutils. catv is some
weird made-up thing. and i _will_ get push back about adding too many
commands, so i intend to make them all count.
The irony of course is that Rob Pike, Russ Cox and a lot of other
Plan 9 folks nowadays work at Google.
--
Marc André Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0
David Seikel
2014-12-20 00:30:34 UTC
Permalink
Post by enh
(sadly the most common request seems to be "ooh, does this mean we can
have less now?".)
Actually I made a start on less, and more, and a bunch of editors, and
got them working at a basic level. Robs just been too busy to get my
generic editor infrastructure in. So that stalled.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
enh
2014-12-20 00:43:51 UTC
Permalink
i was thinking about writing a trivial termcap implementation that
just returns the xterm-color answers and using the BSD less, but i
didn't realize that even the BSDs use the GNU less.

i switched Android over to the toybox more from the mksh shell
function, but people still complain they really want less. (afaict,
mostly for "the ability to go backwards" and "search".)
Post by David Seikel
Post by enh
(sadly the most common request seems to be "ooh, does this mean we can
have less now?".)
Actually I made a start on less, and more, and a bunch of editors, and
got them working at a basic level. Robs just been too busy to get my
generic editor infrastructure in. So that stalled.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
--
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Java i18n/JNI/NIO, or bionic questions? Mail me/drop by/add me as a reviewer.
David Seikel
2014-12-20 00:52:48 UTC
Permalink
Post by enh
i was thinking about writing a trivial termcap implementation that
just returns the xterm-color answers and using the BSD less, but i
didn't realize that even the BSDs use the GNU less.
i switched Android over to the toybox more from the mksh shell
function, but people still complain they really want less. (afaict,
mostly for "the ability to go backwards" and "search".)
Post by David Seikel
Post by enh
(sadly the most common request seems to be "ooh, does this mean we
can have less now?".)
Actually I made a start on less, and more, and a bunch of editors,
and got them working at a basic level. Robs just been too busy to
get my generic editor infrastructure in. So that stalled.
https://github.com/onefang/boxes if you want to look at it. It may
have bit rotted though.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rob Landley
2014-12-21 01:30:01 UTC
Permalink
Post by David Seikel
Post by enh
i was thinking about writing a trivial termcap implementation that
just returns the xterm-color answers and using the BSD less, but i
didn't realize that even the BSDs use the GNU less.
i switched Android over to the toybox more from the mksh shell
function, but people still complain they really want less. (afaict,
mostly for "the ability to go backwards" and "search".)
Post by David Seikel
Post by enh
(sadly the most common request seems to be "ooh, does this mean we
can have less now?".)
Actually I made a start on less, and more, and a bunch of editors,
and got them working at a basic level. Robs just been too busy to
get my generic editor infrastructure in. So that stalled.
https://github.com/onefang/boxes if you want to look at it. It may
have bit rotted though.
I'm working on it. Your infrastructure required some adaptation to do
what I want it to do in toybox.

I'm trying to clean up pending as fast as I can but I'm worried that
people using pending and relying on stuff in there might get broken when
I do the cleanup and promotion...

Speaking of promotion, I think I may get sed promoted today! (Yay!
Implementing 'l' now and the testing pass has made it to the kernel
build of the aboriginal native build. I've still got to run it through a
linux from scratch build, but it hasn't glitched in a while...)

Then I have to finish putting df back together with the -h output, and
then I promised Ashwini sharma a gzip compression implementation, and
then I can poke at boxes again...

Rob

(I also wrote a very long email at dreamhost again about the web
archive. They're working on it. I think I may finally have managed to
explain that regenerating the web index doesn't fix a message delivery
problem. We'll see. The techs working on it aren't subscribed to the
list so they dunno what "success" looks like...)
David Seikel
2014-12-21 01:55:45 UTC
Permalink
Post by Rob Landley
Post by David Seikel
Post by enh
i was thinking about writing a trivial termcap implementation that
just returns the xterm-color answers and using the BSD less, but i
didn't realize that even the BSDs use the GNU less.
i switched Android over to the toybox more from the mksh shell
function, but people still complain they really want less. (afaict,
mostly for "the ability to go backwards" and "search".)
Post by David Seikel
Post by enh
(sadly the most common request seems to be "ooh, does this mean
we can have less now?".)
Actually I made a start on less, and more, and a bunch of editors,
and got them working at a basic level. Robs just been too busy to
get my generic editor infrastructure in. So that stalled.
https://github.com/onefang/boxes if you want to look at it. It may
have bit rotted though.
I'm working on it. Your infrastructure required some adaptation to do
what I want it to do in toybox.
You can always tell me what those adaptations are, and I can see if I
can make them.
Post by Rob Landley
I'm trying to clean up pending as fast as I can but I'm worried that
people using pending and relying on stuff in there might get broken
when I do the cleanup and promotion...
Speaking of promotion, I think I may get sed promoted today! (Yay!
Implementing 'l' now and the testing pass has made it to the kernel
build of the aboriginal native build. I've still got to run it
through a linux from scratch build, but it hasn't glitched in a
while...)
Your new sed implementation would have some infrastructure that might
be shared by boxes. Lots of the editing stuff in a few of the editors
is essentially identical to sed, using the same commands. Any editor
will have similar infrastructure for dealing with files, lines, actual
editing, and such.

So maybe during the Christmas break (new $DAYJOB means I get paid
holidays for a change), I'll go over toybox sed and see what I can
reuse for boxes.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
James McMechan
2014-12-21 21:07:02 UTC
Permalink
Post by enh
Post by Rob Landley
Post by David Seikel
Post by enh
i was thinking about writing a trivial termcap implementation
that
Post by Rob Landley
Post by David Seikel
Post by enh
just returns the xterm-color answers and using the BSD less, but
i
Post by Rob Landley
Post by David Seikel
Post by enh
didn't realize that even the BSDs use the GNU less.
fortunately n-curses devolves tremendously when you don't try to
support every terminal type under the sun efficiently...
Post by enh
Post by Rob Landley
Post by David Seikel
Post by enh
i switched Android over to the toybox more from the mksh shell
function, but people still complain they really want less.
(afaict,
Post by Rob Landley
Post by David Seikel
Post by enh
mostly for "the ability to go backwards" and "search".)
Thinking about it, an old school more without ncurses could have the
last few bits of less tacked on top: searching, backwards movement, any
other bits of interest?
If we yield on fine cursor control and efficiency we could just print
the screen...
It is not like we are trying to be efficient on a ADM-3 or a teletype
over a slow serial connection...
Post by enh
Post by Rob Landley
Post by David Seikel
Post by enh
Post by David Seikel
Post by enh
(sadly the most common request seems to be "ooh, does this mean
we can have less now?".)
Actually I made a start on less, and more, and a bunch of
editors,
Post by Rob Landley
Post by David Seikel
Post by enh
Post by David Seikel
and got them working at a basic level. Robs just been too busy
to
Post by Rob Landley
Post by David Seikel
Post by enh
Post by David Seikel
get my generic editor infrastructure in. So that stalled.
https://github.com/onefang/boxes if you want to look at it. It
may
Post by Rob Landley
Post by David Seikel
have bit rotted though.
I'm working on it. Your infrastructure required some adaptation to
do
Post by Rob Landley
what I want it to do in toybox.
You can always tell me what those adaptations are, and I can see if I
can make them.
Post by Rob Landley
I'm trying to clean up pending as fast as I can but I'm worried that
people using pending and relying on stuff in there might get broken
when I do the cleanup and promotion...
Speaking of promotion, I think I may get sed promoted today! (Yay!
Implementing 'l' now and the testing pass has made it to the kernel
build of the aboriginal native build. I've still got to run it
through a linux from scratch build, but it hasn't glitched in a
while...)
Your new sed implementation would have some infrastructure that might
be shared by boxes. Lots of the editing stuff in a few of the editors
is essentially identical to sed, using the same commands. Any editor
will have similar infrastructure for dealing with files, lines, actual
editing, and such.
So maybe during the Christmas break (new $DAYJOB means I get paid
holidays for a change), I'll go over toybox sed and see what I can
reuse for boxes.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Personally I have a strong bias toward simplicity, a major reason I
like aboriginal and toybox and would like to see qcc ( also being a 60X
faster is nice too 14s vs 14m from tar file to executable compiler :)

I am would suggest something like GNU_QUIRKS and POSIX_QUIRKS config
symbols to add in the odd features like gnu sed directly disagreeing
with POSIX or POSIX's umm interesting xargs without -0 quoting
conventions and any other "why did they do it that way" I know POSIX is
nice to adhere too, but some of the stuff is just odd, and I at least
could live without complex code that does not do anything that I want,
and just complain for example on xargs without -0 and weird filenames.
Hopefully nobody depends on e.g. the sed N;N working like gnu choses, I
am not fond of bug to bug compatibility.

Just like I would prefer to modify glob to output "ls ./-c" for "ls *"
instead of modifying the kernel to not accept "-c" as
a legal file name or try to escape it in the kernel like wheeler wrote
about.

This will be my first email sent from balsa, hotmail improved things
again. So now that I have downloaded my 3G of email in 71314 messages
through IMAP4/SSL, hotmail's server was both slow and goes bonkers
after only 60min of downloading and errors out, so I have been
downloading in a loop.

Jim McMechan
enh
2014-12-21 23:42:01 UTC
Permalink
On Sun, Dec 21, 2014 at 1:07 PM, James McMechan
Post by David Seikel
Post by Rob Landley
Post by David Seikel
Post by enh
i was thinking about writing a trivial termcap implementation that
just returns the xterm-color answers and using the BSD less, but i
didn't realize that even the BSDs use the GNU less.
fortunately n-curses devolves tremendously when you don't try to support
every terminal type under the sun efficiently...
(very off-topic at this point, but vim doesn't use ncurses under any
circumstances. moreover, it turns out that vim actually has all kinds
of different fallbacks and as long as you don't misconfigure it like i
did, it builds and runs fine out of the box on Android. it has a
minimal tgetstr/tgoto/tputs implementation *and* a few built-in
termcap subsets, including xterm. it's an 8.3MiB binary unstripped or
2.1MiB stripped for aarch64, though, so probably not of much interest
to you guys :-) )
Post by David Seikel
Post by Rob Landley
Post by David Seikel
Post by enh
i switched Android over to the toybox more from the mksh shell
function, but people still complain they really want less. (afaict,
mostly for "the ability to go backwards" and "search".)
Thinking about it, an old school more without ncurses could have the last
few bits of less tacked on top: searching, backwards movement, any other
bits of interest?
If we yield on fine cursor control and efficiency we could just print the
screen...
It is not like we are trying to be efficient on a ADM-3 or a teletype over a
slow serial connection...
Post by David Seikel
Post by Rob Landley
Post by David Seikel
Post by enh
Post by David Seikel
Post by enh
(sadly the most common request seems to be "ooh, does this mean
we can have less now?".)
Actually I made a start on less, and more, and a bunch of editors,
and got them working at a basic level. Robs just been too busy to
get my generic editor infrastructure in. So that stalled.
https://github.com/onefang/boxes if you want to look at it. It may
have bit rotted though.
I'm working on it. Your infrastructure required some adaptation to do
what I want it to do in toybox.
You can always tell me what those adaptations are, and I can see if I
can make them.
Post by Rob Landley
I'm trying to clean up pending as fast as I can but I'm worried that
people using pending and relying on stuff in there might get broken
when I do the cleanup and promotion...
Speaking of promotion, I think I may get sed promoted today! (Yay!
Implementing 'l' now and the testing pass has made it to the kernel
build of the aboriginal native build. I've still got to run it
through a linux from scratch build, but it hasn't glitched in a
while...)
Your new sed implementation would have some infrastructure that might
be shared by boxes. Lots of the editing stuff in a few of the editors
is essentially identical to sed, using the same commands. Any editor
will have similar infrastructure for dealing with files, lines, actual
editing, and such.
So maybe during the Christmas break (new $DAYJOB means I get paid
holidays for a change), I'll go over toybox sed and see what I can
reuse for boxes.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Personally I have a strong bias toward simplicity, a major reason I like
aboriginal and toybox and would like to see qcc ( also being a 60X faster is
nice too 14s vs 14m from tar file to executable compiler :)
I am would suggest something like GNU_QUIRKS and POSIX_QUIRKS config symbols
to add in the odd features like gnu sed directly disagreeing with POSIX or
POSIX's umm interesting xargs without -0 quoting conventions and any other
"why did they do it that way" I know POSIX is nice to adhere too, but some
of the stuff is just odd, and I at least could live without complex code
that does not do anything that I want, and just complain for example on
xargs without -0 and weird filenames. Hopefully nobody depends on e.g. the
sed N;N working like gnu choses, I am not fond of bug to bug compatibility.
Just like I would prefer to modify glob to output "ls ./-c" for "ls *"
instead of modifying the kernel to not accept "-c" as
about.
This will be my first email sent from balsa, hotmail improved things again.
So now that I have downloaded my 3G of email in 71314 messages through
IMAP4/SSL, hotmail's server was both slow and goes bonkers after only 60min
of downloading and errors out, so I have been downloading in a loop.
Jim McMechan
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
dmccunney
2014-12-22 02:10:21 UTC
Permalink
Post by enh
(very off-topic at this point, but vim doesn't use ncurses under any
circumstances. moreover, it turns out that vim actually has all kinds
of different fallbacks and as long as you don't misconfigure it like i
did, it builds and runs fine out of the box on Android. it has a
minimal tgetstr/tgoto/tputs implementation *and* a few built-in
termcap subsets, including xterm. it's an 8.3MiB binary unstripped or
2.1MiB stripped for aarch64, though, so probably not of much interest
to you guys :-) )
As it happens, I have a version of Vim running on my generic Android
tablet with dual core ARM7 CPU.

VimTouch is a version optimized for touch screen operation:
https://play.google.com/store/apps/details?id=net.momodalo.app.vimtouch&hl=en

You can also get the optional full Vim runtime, which gives you things
like ex mode:
https://play.google.com/store/apps/details?id=net.momodalo.app.vimtouch.fullruntime

Source is at https://github.com/momodalo/vimtouch

Works fine here in limited testing, though an external keyboard is
really preferable to touch nscreen and onscreen virtual keyboard.
______
Dennis
https://plus.google.com/u/0/105128793974319004519
David Seikel
2014-12-22 03:40:07 UTC
Permalink
Post by enh
On Sun, Dec 21, 2014 at 1:07 PM, James McMechan
Post by James McMechan
Post by enh
i was thinking about writing a trivial termcap implementation
that just returns the xterm-color answers and using the BSD
less, but i didn't realize that even the BSDs use the GNU
less.
fortunately n-curses devolves tremendously when you don't try to
support every terminal type under the sun efficiently...
(very off-topic at this point, but vim doesn't use ncurses under any
circumstances. moreover, it turns out that vim actually has all kinds
of different fallbacks and as long as you don't misconfigure it like i
did, it builds and runs fine out of the box on Android. it has a
minimal tgetstr/tgoto/tputs implementation *and* a few built-in
termcap subsets, including xterm. it's an 8.3MiB binary unstripped or
2.1MiB stripped for aarch64, though, so probably not of much interest
to you guys :-) )
Post by James McMechan
Post by enh
i switched Android over to the toybox more from the mksh shell
function, but people still complain they really want less.
(afaict, mostly for "the ability to go backwards" and
"search".)
Thinking about it, an old school more without ncurses could have
the last few bits of less tacked on top: searching, backwards
movement, any other bits of interest?
If we yield on fine cursor control and efficiency we could just
print the screen...
It is not like we are trying to be efficient on a ADM-3 or a
teletype over a slow serial connection...
This is what boxes does. In theory most things still in use today use
some basic and ordinary ANSI escape sequences, so the complexity of
ncurses and friends is not really needed. Boxes is a generic editor
frame work, with the simple stuff of editing / paging generalised and
wrappers around it for emacs, less, more, vi, WordStar, and a few
others. And yes, it does just print the entire screen when stuff
changes, though Rob mentioned more intelligent redraws might be a good
addition. I left it out for simplicities sake for now, better to get
the basic infrastructure in place first. Apparently Rob has had to
deal with slow serial ports recently.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rob Landley
2014-12-23 04:21:26 UTC
Permalink
Post by enh
On Sun, Dec 21, 2014 at 1:07 PM, James McMechan
Post by enh
i was thinking about writing a trivial termcap implementation that
just returns the xterm-color answers and using the BSD less, but i
didn't realize that even the BSDs use the GNU less.
fortunately n-curses devolves tremendously when you don't try to support
every terminal type under the sun efficiently...
(very off-topic at this point, but vim doesn't use ncurses under any
circumstances. moreover, it turns out that vim actually has all kinds
of different fallbacks and as long as you don't misconfigure it like i
did, it builds and runs fine out of the box on Android. it has a
minimal tgetstr/tgoto/tputs implementation *and* a few built-in
termcap subsets, including xterm. it's an 8.3MiB binary unstripped or
2.1MiB stripped for aarch64, though, so probably not of much interest
to you guys :-) )
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. (It's easy to
knock out a simple 90% implementation. It's really hard to make
something do everything right in all the cases people are going to throw
at it.)

There are some others (the kernel build requires bc now, if I'm doing
"less" and "vi" I should be able to do "screen", and "rsync" is really
useful...) but right now I'm focused on the list for 1.0, and there are
a lot of smaller commands (and the giant backlog of pending cleanups)
that could get knocked off the list faster...

By the way, the generic line editing infrastructure is useful not just
for shell command line editing, and obvious stuff like less, screen, and
vi, but even "watch" needs it to work quite right. (And if you're
supporting unicode, the difference between doing "more" and "less"
_right_ isn't as big as you'd think...)

If "less" is a priority, that actually helps prioritize the rest of
them. Once I've written the basic "navigate a line" infrastructure (with
querying screen size via ansi probe fallback, and reassembling escape
sequences that got decoupled going over serial line; I fixed both of
these in busybox several years ago), then stacking them isn't quite as
big a deal.

I do follow a security researcher on twitter (@0xabad1dea) and for a
while she had as her handle:

echo -e "Melissa \xe2\x80\xae ασσιλέΜ \xe2\x80\xed"

So it would be nice to get the reversal codes to work. But given that
xfce's terminal doesn't handle it either...

(Dalias wrote a terminal that probably does. I should compile/install
that and try it out...)

Rob
Rich Felker
2014-12-23 05:37:17 UTC
Permalink
Post by Rob Landley
If "less" is a priority, that actually helps prioritize the rest of
them. Once I've written the basic "navigate a line" infrastructure (with
querying screen size via ansi probe fallback, and reassembling escape
sequences that got decoupled going over serial line; I fixed both of
these in busybox several years ago), then stacking them isn't quite as
big a deal.
I hope you'll have an option just to rely on TIOCGWINSZ. The escapes
that cause the terminal to "echo back" a response are considered a
security misdesign by some people (myself included) and they're also
problematic from the standpoint that you don't know if they'll be
supported and that you can't distinguish between a terminal that
doesn't support them and one which is just slow to respond. I realize
you don't have any other option on serial terminals, but thankfully
lots of us don't have to deal with serial terminals.
Post by Rob Landley
echo -e "Melissa \xe2\x80\xae ασσιλέΜ \xe2\x80\xed"
So it would be nice to get the reversal codes to work. But given that
xfce's terminal doesn't handle it either...
(Dalias wrote a terminal that probably does. I should compile/install
that and try it out...)
Mine doesn't. Bidirectional text is the main (only?) multilingual
feature uuterm is missing.

Rich
Rob Landley
2014-12-23 16:23:43 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
If "less" is a priority, that actually helps prioritize the rest of
them. Once I've written the basic "navigate a line" infrastructure (with
querying screen size via ansi probe fallback, and reassembling escape
sequences that got decoupled going over serial line; I fixed both of
these in busybox several years ago), then stacking them isn't quite as
big a deal.
I hope you'll have an option just to rely on TIOCGWINSZ. The escapes
that cause the terminal to "echo back" a response are considered a
security misdesign by some people (myself included) and they're also
problematic from the standpoint that you don't know if they'll be
supported and that you can't distinguish between a terminal that
doesn't support them and one which is just slow to respond. I realize
you don't have any other option on serial terminals, but thankfully
lots of us don't have to deal with serial terminals.
Lots of us _do_ have to deal with serial terminals.

If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Post by Rich Felker
Post by Rob Landley
echo -e "Melissa \xe2\x80\xae ασσιλέΜ \xe2\x80\xed"
So it would be nice to get the reversal codes to work. But given that
xfce's terminal doesn't handle it either...
(Dalias wrote a terminal that probably does. I should compile/install
that and try it out...)
Mine doesn't. Bidirectional text is the main (only?) multilingual
feature uuterm is missing.
Even for just "more", the fontmetrics of combining characters is... fun.

I _mostly_ punt this to the display layer, but this piece of
infrastructure probably should care.
Post by Rich Felker
Rich
Rob
Rich Felker
2014-12-23 19:44:19 UTC
Permalink
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
If "less" is a priority, that actually helps prioritize the rest of
them. Once I've written the basic "navigate a line" infrastructure (with
querying screen size via ansi probe fallback, and reassembling escape
sequences that got decoupled going over serial line; I fixed both of
these in busybox several years ago), then stacking them isn't quite as
big a deal.
I hope you'll have an option just to rely on TIOCGWINSZ. The escapes
that cause the terminal to "echo back" a response are considered a
security misdesign by some people (myself included) and they're also
problematic from the standpoint that you don't know if they'll be
supported and that you can't distinguish between a terminal that
doesn't support them and one which is just slow to respond. I realize
you don't have any other option on serial terminals, but thankfully
lots of us don't have to deal with serial terminals.
Lots of us _do_ have to deal with serial terminals.
Yes, I know.
Post by Rob Landley
If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Exporting ROWS and COLUMNS precludes runtime size changes though. The
correct way to handle size is TIOCGWINSZ. Perhaps you could use
TIOCGWINSZ for device numbers that indicate virtual terminal or pty
and only fall back to the ugly methods for serial ports?
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
echo -e "Melissa \xe2\x80\xae ασσιλέΜ \xe2\x80\xed"
So it would be nice to get the reversal codes to work. But given that
xfce's terminal doesn't handle it either...
(Dalias wrote a terminal that probably does. I should compile/install
that and try it out...)
Mine doesn't. Bidirectional text is the main (only?) multilingual
feature uuterm is missing.
Even for just "more", the fontmetrics of combining characters is... fun.
I _mostly_ punt this to the display layer, but this piece of
infrastructure probably should care.
Programs writing to a terminal have no need to care specially about
RTL characters; it's all in the presentation layer. On the other hand
they _might_ need to know about RTL/LTR overrides (but probably not)
and they probably need to know about bidi _nesting_ (if you care to
support that, but it's probably useless).

Rich
enh
2014-12-23 19:48:42 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
If "less" is a priority, that actually helps prioritize the rest of
them. Once I've written the basic "navigate a line" infrastructure (with
querying screen size via ansi probe fallback, and reassembling escape
sequences that got decoupled going over serial line; I fixed both of
these in busybox several years ago), then stacking them isn't quite as
big a deal.
I hope you'll have an option just to rely on TIOCGWINSZ. The escapes
that cause the terminal to "echo back" a response are considered a
security misdesign by some people (myself included) and they're also
problematic from the standpoint that you don't know if they'll be
supported and that you can't distinguish between a terminal that
doesn't support them and one which is just slow to respond. I realize
you don't have any other option on serial terminals, but thankfully
lots of us don't have to deal with serial terminals.
Lots of us _do_ have to deal with serial terminals.
Yes, I know.
Post by Rob Landley
If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Exporting ROWS and COLUMNS precludes runtime size changes though. The
correct way to handle size is TIOCGWINSZ. Perhaps you could use
TIOCGWINSZ for device numbers that indicate virtual terminal or pty
and only fall back to the ugly methods for serial ports?
annoyingly, adb (Android Debug Bridge) doesn't currently pass window
size changes through, so until someone fixes that Android users count
as serial port users too :-(
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
echo -e "Melissa \xe2\x80\xae ασσιλέΜ \xe2\x80\xed"
So it would be nice to get the reversal codes to work. But given that
xfce's terminal doesn't handle it either...
(Dalias wrote a terminal that probably does. I should compile/install
that and try it out...)
Mine doesn't. Bidirectional text is the main (only?) multilingual
feature uuterm is missing.
Even for just "more", the fontmetrics of combining characters is... fun.
I _mostly_ punt this to the display layer, but this piece of
infrastructure probably should care.
Programs writing to a terminal have no need to care specially about
RTL characters; it's all in the presentation layer. On the other hand
they _might_ need to know about RTL/LTR overrides (but probably not)
and they probably need to know about bidi _nesting_ (if you care to
support that, but it's probably useless).
Rich
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
--
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Java i18n/JNI/NIO, or bionic questions? Mail me/drop by/add me as a reviewer.
Rich Felker
2014-12-23 19:53:31 UTC
Permalink
Post by enh
Post by Rich Felker
Post by Rob Landley
Lots of us _do_ have to deal with serial terminals.
Yes, I know.
Post by Rob Landley
If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Exporting ROWS and COLUMNS precludes runtime size changes though. The
correct way to handle size is TIOCGWINSZ. Perhaps you could use
TIOCGWINSZ for device numbers that indicate virtual terminal or pty
and only fall back to the ugly methods for serial ports?
annoyingly, adb (Android Debug Bridge) doesn't currently pass window
size changes through, so until someone fixes that Android users count
as serial port users too :-(
From the Android side, does adb use a pty? Or is it a usb serial
port device? My impression was that it's the latter but I'm not very
familiar with it.

Rich
enh
2014-12-23 20:12:00 UTC
Permalink
Post by Rich Felker
Post by enh
Post by Rich Felker
Post by Rob Landley
Lots of us _do_ have to deal with serial terminals.
Yes, I know.
Post by Rob Landley
If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Exporting ROWS and COLUMNS precludes runtime size changes though. The
correct way to handle size is TIOCGWINSZ. Perhaps you could use
TIOCGWINSZ for device numbers that indicate virtual terminal or pty
and only fall back to the ugly methods for serial ports?
annoyingly, adb (Android Debug Bridge) doesn't currently pass window
size changes through, so until someone fixes that Android users count
as serial port users too :-(
From the Android side, does adb use a pty? Or is it a usb serial
port device? My impression was that it's the latter but I'm not very
familiar with it.
i'm not an expert -- or i'd have fixed all the stuff worked around by
https://github.com/facebook/fb-adb and we wouldn't be having this
conversation :-) -- but there's definitely some use of ptys in there.
Rob Landley
2014-12-23 21:20:29 UTC
Permalink
Post by enh
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
If "less" is a priority, that actually helps prioritize the rest of
them. Once I've written the basic "navigate a line" infrastructure (with
querying screen size via ansi probe fallback, and reassembling escape
sequences that got decoupled going over serial line; I fixed both of
these in busybox several years ago), then stacking them isn't quite as
big a deal.
I hope you'll have an option just to rely on TIOCGWINSZ. The escapes
that cause the terminal to "echo back" a response are considered a
security misdesign by some people (myself included) and they're also
problematic from the standpoint that you don't know if they'll be
supported and that you can't distinguish between a terminal that
doesn't support them and one which is just slow to respond. I realize
you don't have any other option on serial terminals, but thankfully
lots of us don't have to deal with serial terminals.
Lots of us _do_ have to deal with serial terminals.
Yes, I know.
Post by Rob Landley
If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Exporting ROWS and COLUMNS precludes runtime size changes though. The
correct way to handle size is TIOCGWINSZ. Perhaps you could use
TIOCGWINSZ for device numbers that indicate virtual terminal or pty
and only fall back to the ugly methods for serial ports?
annoyingly, adb (Android Debug Bridge) doesn't currently pass window
size changes through, so until someone fixes that Android users count
as serial port users too :-(
And the qemu -nographic option hooks serial up to stdin and stdout of
the qemu process, which is a very convenient place to park /dev/console.

And my sh2-on-a-stick xylinx fpga device (and my armv4t on a stick "tin
can tools nail board") both have serial consoles. Even though both are
USB plugging straight into the computer and powered by it, they care
what baud rate you set them to. (Really! Not just emulated serial,
there's apparently a full milimeter of wire in there somewhere that an
actual serial signal is going down.)

Rob
Rob Landley
2014-12-23 21:17:12 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Exporting ROWS and COLUMNS precludes runtime size changes though. The
correct way to handle size is TIOCGWINSZ. Perhaps you could use
TIOCGWINSZ for device numbers that indicate virtual terminal or pty
and only fall back to the ugly methods for serial ports?
Look at the current code:

http://landley.net/hg/toybox/file/45fb262230c9/lib/lib.c#l588

It checks the size. If it get zeroes, it checks the environment
variables. If those are blank _then_ you do the ascii probe.

Except the way I've set it up a wrapper function calls that and then
does the ascii probe if it returns 0, because the ascii probe isn't
always appropriate. (But the line editing thing can do the ascii probe,
because any time it gets the reply escape sequence it can asynchronously
update its idea of the screen size and where the cursor is, since a
proper probe should probably query both because your shell prompt isn't
guaranteed to start at the left edge of the screen if the output of the
previous program didn't end with a newline.)
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
echo -e "Melissa \xe2\x80\xae ασσιλέΜ \xe2\x80\xed"
So it would be nice to get the reversal codes to work. But given that
xfce's terminal doesn't handle it either...
(Dalias wrote a terminal that probably does. I should compile/install
that and try it out...)
Mine doesn't. Bidirectional text is the main (only?) multilingual
feature uuterm is missing.
Even for just "more", the fontmetrics of combining characters is... fun.
I _mostly_ punt this to the display layer, but this piece of
infrastructure probably should care.
Programs writing to a terminal have no need to care specially about
RTL characters; it's all in the presentation layer. On the other hand
they _might_ need to know about RTL/LTR overrides (but probably not)
and they probably need to know about bidi _nesting_ (if you care to
support that, but it's probably useless).
I need proper utf8/unicode test cases. Right now I'm thinking of looking
at the backscroll of @fakeapstylebook from December 23 last year for
tweets like
https://twitter.com/FakeAPStylebook/status/415263262629171201 and
https://twitter.com/FakeAPStylebook/status/415142987778117632

But better test cases would be nice...

Rob
Rich Felker
2014-12-24 02:05:00 UTC
Permalink
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
If you export $ROWS and $COLUMNS it'll do that instead of probe, and I
can add a compile-time config symbol to switch it off if you care that much.
Exporting ROWS and COLUMNS precludes runtime size changes though. The
correct way to handle size is TIOCGWINSZ. Perhaps you could use
TIOCGWINSZ for device numbers that indicate virtual terminal or pty
and only fall back to the ugly methods for serial ports?
http://landley.net/hg/toybox/file/45fb262230c9/lib/lib.c#l588
It checks the size. If it get zeroes, it checks the environment
variables. If those are blank _then_ you do the ascii probe.
Except the way I've set it up a wrapper function calls that and then
does the ascii probe if it returns 0, because the ascii probe isn't
always appropriate. (But the line editing thing can do the ascii probe,
because any time it gets the reply escape sequence it can asynchronously
update its idea of the screen size and where the cursor is, since a
proper probe should probably query both because your shell prompt isn't
guaranteed to start at the left edge of the screen if the output of the
previous program didn't end with a newline.)
Sounds reasonable.
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
echo -e "Melissa \xe2\x80\xae ασσιλέΜ \xe2\x80\xed"
So it would be nice to get the reversal codes to work. But given that
xfce's terminal doesn't handle it either...
(Dalias wrote a terminal that probably does. I should compile/install
that and try it out...)
Mine doesn't. Bidirectional text is the main (only?) multilingual
feature uuterm is missing.
Even for just "more", the fontmetrics of combining characters is... fun.
I _mostly_ punt this to the display layer, but this piece of
infrastructure probably should care.
Programs writing to a terminal have no need to care specially about
RTL characters; it's all in the presentation layer. On the other hand
they _might_ need to know about RTL/LTR overrides (but probably not)
and they probably need to know about bidi _nesting_ (if you care to
support that, but it's probably useless).
I need proper utf8/unicode test cases. Right now I'm thinking of looking
tweets like
https://twitter.com/FakeAPStylebook/status/415263262629171201 and
https://twitter.com/FakeAPStylebook/status/415142987778117632
But better test cases would be nice...
I wouldn't really worry about it. From the standpoint of an
application writing to a terminal, all you need to know is wcwidth(),
i.e. the number of logical columns the character occupies. This
doesn't need fancy test cases; just one character of each width (0, 1,
2, or negative==unprintable) is all you need.

Rich
David Seikel
2014-12-23 06:36:39 UTC
Permalink
Post by Rob Landley
Post by enh
On Sun, Dec 21, 2014 at 1:07 PM, James McMechan
No idea how the quoting in your reply got built, nothing I actually said
is quoted.
Post by Rob Landley
Post by enh
Post by James McMechan
Post by enh
i was thinking about writing a trivial termcap implementation
that just returns the xterm-color answers and using the BSD
less, but i didn't realize that even the BSDs use the GNU less.
fortunately n-curses devolves tremendously when you don't try to
support every terminal type under the sun efficiently...
(very off-topic at this point, but vim doesn't use ncurses under any
circumstances. moreover, it turns out that vim actually has all
kinds of different fallbacks and as long as you don't misconfigure
it like i did, it builds and runs fine out of the box on Android.
it has a minimal tgetstr/tgoto/tputs implementation *and* a few
built-in termcap subsets, including xterm. it's an 8.3MiB binary
unstripped or 2.1MiB stripped for aarch64, though, so probably not
of much interest to you guys :-) )
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. (It's easy to
knock out a simple 90% implementation. It's really hard to make
something do everything right in all the cases people are going to
throw at it.)
There are some others (the kernel build requires bc now, if I'm doing
"less" and "vi" I should be able to do "screen", and "rsync" is really
useful...) but right now I'm focused on the list for 1.0, and there
are a lot of smaller commands (and the giant backlog of pending
cleanups) that could get knocked off the list faster...
"Rsync" isn't in my scope, but I agree, it being in toybox would be
very useful, I hope someone works on that.
Post by Rob Landley
By the way, the generic line editing infrastructure is useful not just
for shell command line editing, and obvious stuff like less, screen,
and vi, but even "watch" needs it to work quite right. (And if you're
supporting unicode, the difference between doing "more" and "less"
_right_ isn't as big as you'd think...)
If "less" is a priority, that actually helps prioritize the rest of
them. Once I've written the basic "navigate a line" infrastructure
(with querying screen size via ansi probe fallback, and reassembling
escape sequences that got decoupled going over serial line; I fixed
both of these in busybox several years ago), then stacking them isn't
quite as big a deal.
Even less of a big deal since I've already done a lot of these things
for toybox, and hanging out for things to filter through Robs TODO list
of DOOM so that we can complete them. "Screen" / "tmux" (I prefer tmux
at this stage) is something my boxes infrastructure could handle as
well, so it's on my TODO to look at.

I've done the "no need for curses / termcap" thing. I've done the
"basic navigate a line" infrastructure. I've done the "figure out
screen size with fallbacks" thing. I've done the "generic line and
screen editing" thing. I've tied it all together. I've done the basic
emacs, less, joe, less, mcedit, more, nano, vi thing. I've done a
brian dead sh to show how my infrastructure would fit that use case.
I've done the research for the non basic editing stuff that I've not
implemented yet. I've even implemented a few non basic editing
commands.

It's been going on for a long time now (over two years) can you at least
put the boxes stuff into pending please, so that we can all work on it?
Then people wont keep bringing up less, vi, and all those others that I
have a basic working implementation of, as if there's been no work done
on them. Would be nice if we all worked on one implementation, instead
of investigating them / working on them separately.

"Boxes is good enough for pending" means we can move on. "Boxes is
shit" means I can give up and find other things to do, someone else
can work on them. "Boxes is not quite good enough for pending and Rob's
not been telling me what needs to be fixed for years" means it's in
limbo for two years and I'm getting frustrated with that lack of
progress. I know others where waiting on some progress here two years
ago. They might have given up waiting.

OK Rob, you told me one thing that was wrong, it was too big for you to
grok in one sitting. It has to be, it covers a lot of ground. I broke
of the smallest stand alone part I could and submitted that, and still
no movement with it.

Please put boxes (or at least that bite sized handlekeys bit) in
pending, or put it out of it's misery. We can help you write this
stuff if you let us Rob. Delegate responsibility to me if you like,
or not. Let me and any one else that wants to work on these things,
work on it in pending, bitch at us if our idea of perfection (or even
our temporary horrid hack) doesn't meet your idea of perfection. Maybe
in another two years it will be good enough for moving out of pending,
but at least progress happens, people can use it, people can file bug
reports, or fix it. This current limbo isn't working for anyone.

Especially now that you think sed is almost good enough. Now would be
a great time for me to read through the way you handle files and lines
in sed, with a view to reusing that infrastructure in boxes. Then the
next bite sized bit could be file / line handling, and could happen
soon.

It's especially frustrating watching other peoples submissions going
in easily, and pretty much nothing happening for boxes for over two
years.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Jason Spiro
2014-12-29 03:12:03 UTC
Permalink
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. [...]
There are some others (the kernel build requires bc now, if I'm doing
"less" and "vi" I should be able to do "screen", and "rsync" is really
useful...) but right now I'm focused on the list for 1.0, and there are
a lot of smaller commands (and the giant backlog of pending cleanups)
that could get knocked off the list faster...
In this message, I will tell you why I think toybox should emulate tmux
instead of GNU Screen.

Let me begin.

GNU Screen, a "terminal multiplexer", is very useful. Still, I've since
switched to tmux. It's a newer terminal multiplexer, and is BSD-licensed.
It is true that GNU Screen has now started making releases again for the
first time in half a decade. Still, I like tmux so much that I don't plan
to switch back to Screen.

tmux makes certain operations easier.[1] For example: It ships with
preconfigured keybindings (<C-b> 0, <C-b> 1, ..., <C-b> 9) which let you
jump to low-numbered windows in just a few keystrokes. Another example: To
renumber a window, you need only hit five keys (<C-b> . 9 <RET>), instead of
nine (<C-a> : n u <TAB> 9 <RET>).

tmux is also easier to learn. For example: It shows a status line (tab
bar) by default, instead of forcing users to mess with complex configuration
options just to get a status line. See screenshot[2].

tmux is included in the software repositories of Ubuntu, Debian "stable",
and other distros.

tmux's basic keybindings are fairly similar to Screen's. But, instead of
Ctrl+A, tmux's default prefix is Ctrl+B. (This is reconfigurable.)

You can find a tmux reference card[3] on the Web.

Dear Rob: I know you mention Screen in your todo.txt file[4]. But please
consider instead mentioning tmux. Those who are familiar only with good old
Screen can either adapt to the nicer user interface that tmux provides, or
can download and install Screen themselves.

Cheers,
--Jason

P.S. I thank everyone on this list for maintaining Toybox. My next Android
device will be a more pleasant to work with, thanks to you all.

^ [1]. http://www.techrepublic.com/blog/linux-and-open-source/is-tmux-the-
gnu-screen-killer/
^ [2]. Loading Image...
^ [3]. http://www.mechanicalkeys.com/files/os/notes/tm.html
^ [4]. http://www.landley.net/code/toybox/todo.txt
David Seikel
2014-12-29 03:50:03 UTC
Permalink
On Mon, 29 Dec 2014 03:12:03 +0000 (UTC) Jason Spiro
Post by Jason Spiro
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've
been debugging sed against real-world data for _weeks_ now. [...]
There are some others (the kernel build requires bc now, if I'm
doing "less" and "vi" I should be able to do "screen", and "rsync"
is really useful...) but right now I'm focused on the list for 1.0,
and there are a lot of smaller commands (and the giant backlog of
pending cleanups) that could get knocked off the list faster...
In this message, I will tell you why I think toybox should emulate
tmux instead of GNU Screen.
Well, if I can go ahead with my plans for boxes for toybox, which will
eventually include a screen / tmux type system, then no one has to
convince me to make it more like tmux than screen. I fully agree with
you, tmux is much better, it's what I use. I've configured tmux to be
friendly to screen users in the past, and convinced one of my clients
to switch to it. Actually, he convinced himself with the phrase
"ooooh, you can resize the window panes with your mouse". Mind you,
he was resizing MY window panes from another computer at the time,
which annoyed me watching from the other side of the city. lol
Post by Jason Spiro
Let me begin.
GNU Screen, a "terminal multiplexer", is very useful. Still, I've
since switched to tmux. It's a newer terminal multiplexer, and is
BSD-licensed. It is true that GNU Screen has now started making
releases again for the first time in half a decade. Still, I like
tmux so much that I don't plan to switch back to Screen.
From my point of view tmux seems to be better designed.
Post by Jason Spiro
tmux makes certain operations easier.[1] For example: It ships with
preconfigured keybindings (<C-b> 0, <C-b> 1, ..., <C-b> 9) which let
you jump to low-numbered windows in just a few keystrokes. Another
example: To renumber a window, you need only hit five keys (<C-b> .
9 <RET>), instead of nine (<C-a> : n u <TAB> 9 <RET>).
tmux is also easier to learn. For example: It shows a status line
(tab bar) by default, instead of forcing users to mess with complex
configuration options just to get a status line. See screenshot[2].
tmux is included in the software repositories of Ubuntu, Debian
"stable", and other distros.
tmux's basic keybindings are fairly similar to Screen's. But,
instead of Ctrl+A, tmux's default prefix is Ctrl+B. (This is
reconfigurable.)
I do generally configure the hot key and a few others for screen
compatibility. So that if I ever have to use screen, then I know what
to do, and others used to screen can still use my tmux installs.

Tmux is also built for scripting, which is hard for screen. I have a
script that, for instance, connects to a tmux session, switches through
particular windows in that session, issuing commands to each window, to
automate some backups, running from a cron job. The other sysadmin for
that server had originally used screen, coz she didn't know about tmux,
and completely failed to get screen to be able to do anything like that.
Post by Jason Spiro
You can find a tmux reference card[3] on the Web.
Dear Rob: I know you mention Screen in your todo.txt file[4]. But
please consider instead mentioning tmux. Those who are familiar only
with good old Screen can either adapt to the nicer user interface
that tmux provides, or can download and install Screen themselves.
In the end, I'll likely do what I did for the boxes based editors, make
it generic, and allow people to build their own variations easily. So
screen and tmux users will both be happy. After all, the boxes based
editors already support all the editors / pagers on the toybox roadmap
(except sed), and a couple more.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Robert Thompson
2014-12-29 07:58:37 UTC
Permalink
Once upon a time, the serial issue would have kept me using screen. But,
these days I use picoterm as the minimal pty<-->serial connector, and that
works as well with tmux as screen's own serial handling worked with screen.
It also works well without any multiplexer, just letting your (virtual)
terminal handle whatever escape codes are coming, so my smaller utility
images only have to include the tiny picocom binary rather than the much
bigger screen (or tmux+picocom) binary.

Picoterm itself is unfortunately GPL, but it's also pretty simple;
basically just abstracts the serial-pty and the console-pty stuff and plugs
them together with a limited set of command keys for exit, baud-up/down,
etc, and for launching rzsz-style file-transfer programs.

For people who use serial devices on a regular basis, this is very useful.
I am *very* carefully resisting the urge to ask that it be in toybox, at
least until after everything already on the list is done :)

If boxes has a picocom toy, I'll be very happy... if not, I'll eventually
make the time to try to contribute one. Since terminal-side and serial-side
are pretty much a matter of doing the same thing for different reasons, it
might not even significantly change the size of the binary...
Post by David Seikel
On Mon, 29 Dec 2014 03:12:03 +0000 (UTC) Jason Spiro
Post by Jason Spiro
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've
been debugging sed against real-world data for _weeks_ now. [...]
There are some others (the kernel build requires bc now, if I'm
doing "less" and "vi" I should be able to do "screen", and "rsync"
is really useful...) but right now I'm focused on the list for 1.0,
and there are a lot of smaller commands (and the giant backlog of
pending cleanups) that could get knocked off the list faster...
In this message, I will tell you why I think toybox should emulate
tmux instead of GNU Screen.
Well, if I can go ahead with my plans for boxes for toybox, which will
eventually include a screen / tmux type system, then no one has to
convince me to make it more like tmux than screen. I fully agree with
you, tmux is much better, it's what I use. I've configured tmux to be
friendly to screen users in the past, and convinced one of my clients
to switch to it. Actually, he convinced himself with the phrase
"ooooh, you can resize the window panes with your mouse". Mind you,
he was resizing MY window panes from another computer at the time,
which annoyed me watching from the other side of the city. lol
Post by Jason Spiro
Let me begin.
GNU Screen, a "terminal multiplexer", is very useful. Still, I've
since switched to tmux. It's a newer terminal multiplexer, and is
BSD-licensed. It is true that GNU Screen has now started making
releases again for the first time in half a decade. Still, I like
tmux so much that I don't plan to switch back to Screen.
From my point of view tmux seems to be better designed.
Post by Jason Spiro
tmux makes certain operations easier.[1] For example: It ships with
preconfigured keybindings (<C-b> 0, <C-b> 1, ..., <C-b> 9) which let
you jump to low-numbered windows in just a few keystrokes. Another
example: To renumber a window, you need only hit five keys (<C-b> .
9 <RET>), instead of nine (<C-a> : n u <TAB> 9 <RET>).
tmux is also easier to learn. For example: It shows a status line
(tab bar) by default, instead of forcing users to mess with complex
configuration options just to get a status line. See screenshot[2].
tmux is included in the software repositories of Ubuntu, Debian
"stable", and other distros.
tmux's basic keybindings are fairly similar to Screen's. But,
instead of Ctrl+A, tmux's default prefix is Ctrl+B. (This is
reconfigurable.)
I do generally configure the hot key and a few others for screen
compatibility. So that if I ever have to use screen, then I know what
to do, and others used to screen can still use my tmux installs.
Tmux is also built for scripting, which is hard for screen. I have a
script that, for instance, connects to a tmux session, switches through
particular windows in that session, issuing commands to each window, to
automate some backups, running from a cron job. The other sysadmin for
that server had originally used screen, coz she didn't know about tmux,
and completely failed to get screen to be able to do anything like that.
Post by Jason Spiro
You can find a tmux reference card[3] on the Web.
Dear Rob: I know you mention Screen in your todo.txt file[4]. But
please consider instead mentioning tmux. Those who are familiar only
with good old Screen can either adapt to the nicer user interface
that tmux provides, or can download and install Screen themselves.
In the end, I'll likely do what I did for the boxes based editors, make
it generic, and allow people to build their own variations easily. So
screen and tmux users will both be happy. After all, the boxes based
editors already support all the editors / pagers on the toybox roadmap
(except sed), and a couple more.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
David Seikel
2014-12-29 08:51:07 UTC
Permalink
On Mon, 29 Dec 2014 07:58:37 +0000 Robert Thompson
Post by Robert Thompson
Once upon a time, the serial issue would have kept me using screen.
But, these days I use picoterm as the minimal pty<-->serial
connector, and that works as well with tmux as screen's own serial
handling worked with screen. It also works well without any
multiplexer, just letting your (virtual) terminal handle whatever
escape codes are coming, so my smaller utility images only have to
include the tiny picocom binary rather than the much bigger screen
(or tmux+picocom) binary.
Picoterm itself is unfortunately GPL, but it's also pretty simple;
basically just abstracts the serial-pty and the console-pty stuff and
plugs them together with a limited set of command keys for exit,
baud-up/down, etc, and for launching rzsz-style file-transfer
programs.
For people who use serial devices on a regular basis, this is very
useful. I am *very* carefully resisting the urge to ask that it be in
toybox, at least until after everything already on the list is done :)
If boxes has a picocom toy, I'll be very happy... if not, I'll
eventually make the time to try to contribute one. Since
terminal-side and serial-side are pretty much a matter of doing the
same thing for different reasons, it might not even significantly
change the size of the binary...
I have no plans to add a picocom / picoterm toy, or anything similar. I
don't know what Rob's thoughts on that are. I have no objection to it,
could be useful for embedded development.
Post by Robert Thompson
Post by David Seikel
On Mon, 29 Dec 2014 03:12:03 +0000 (UTC) Jason Spiro
Post by Jason Spiro
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the
four realy big commands required by posix (sed, awk, sh, vi)
and I've been debugging sed against real-world data for _weeks_
now. [...]
There are some others (the kernel build requires bc now, if I'm
doing "less" and "vi" I should be able to do "screen", and
"rsync" is really useful...) but right now I'm focused on the
list for 1.0, and there are a lot of smaller commands (and the
giant backlog of pending cleanups) that could get knocked off
the list faster...
In this message, I will tell you why I think toybox should emulate
tmux instead of GNU Screen.
Well, if I can go ahead with my plans for boxes for toybox, which
will eventually include a screen / tmux type system, then no one
has to convince me to make it more like tmux than screen. I fully
agree with you, tmux is much better, it's what I use. I've
configured tmux to be friendly to screen users in the past, and
convinced one of my clients to switch to it. Actually, he
convinced himself with the phrase "ooooh, you can resize the window
panes with your mouse". Mind you, he was resizing MY window panes
from another computer at the time, which annoyed me watching from
the other side of the city. lol
Post by Jason Spiro
Let me begin.
GNU Screen, a "terminal multiplexer", is very useful. Still, I've
since switched to tmux. It's a newer terminal multiplexer, and is
BSD-licensed. It is true that GNU Screen has now started making
releases again for the first time in half a decade. Still, I like
tmux so much that I don't plan to switch back to Screen.
From my point of view tmux seems to be better designed.
Post by Jason Spiro
tmux makes certain operations easier.[1] For example: It ships
with preconfigured keybindings (<C-b> 0, <C-b> 1, ..., <C-b> 9)
which let you jump to low-numbered windows in just a few
keystrokes. Another example: To renumber a window, you need
only hit five keys (<C-b> . 9 <RET>), instead of nine (<C-a> : n
u <TAB> 9 <RET>).
tmux is also easier to learn. For example: It shows a status
line (tab bar) by default, instead of forcing users to mess with
complex configuration options just to get a status line. See
screenshot[2].
tmux is included in the software repositories of Ubuntu, Debian
"stable", and other distros.
tmux's basic keybindings are fairly similar to Screen's. But,
instead of Ctrl+A, tmux's default prefix is Ctrl+B. (This is
reconfigurable.)
I do generally configure the hot key and a few others for screen
compatibility. So that if I ever have to use screen, then I know
what to do, and others used to screen can still use my tmux
installs.
Tmux is also built for scripting, which is hard for screen. I have
a script that, for instance, connects to a tmux session, switches
through particular windows in that session, issuing commands to
each window, to automate some backups, running from a cron job.
The other sysadmin for that server had originally used screen, coz
she didn't know about tmux, and completely failed to get screen to
be able to do anything like that.
Post by Jason Spiro
You can find a tmux reference card[3] on the Web.
Dear Rob: I know you mention Screen in your todo.txt file[4].
But please consider instead mentioning tmux. Those who are
familiar only with good old Screen can either adapt to the nicer
user interface that tmux provides, or can download and install
Screen themselves.
In the end, I'll likely do what I did for the boxes based editors,
make it generic, and allow people to build their own variations
easily. So screen and tmux users will both be happy. After all,
the boxes based editors already support all the editors / pagers on
the toybox roadmap (except sed), and a couple more.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rob Landley
2015-01-01 04:19:47 UTC
Permalink
Post by Robert Thompson
Once upon a time, the serial issue would have kept me using screen. But,
these days I use picoterm as the minimal pty<-->serial connector, and
that works as well with tmux as screen's own serial handling worked with
screen. It also works well without any multiplexer, just letting your
(virtual) terminal handle whatever escape codes are coming, so my
smaller utility images only have to include the tiny picocom binary
rather than the much bigger screen (or tmux+picocom) binary.
Picoterm itself is unfortunately GPL, but it's also pretty simple;
basically just abstracts the serial-pty and the console-pty stuff and
plugs them together with a limited set of command keys for exit,
baud-up/down, etc, and for launching rzsz-style file-transfer programs.
I've used netcat -f /dev/ttyUSB0 as a terminal, the netcat help points
out you need to use stty first to set up the terminal:

Use "stty 115200 -F /dev/ttyS0 && stty raw -echo -ctlecho" with
netcat -f to connect to a serial port.

On my todo list is gluing the two of those together into a nanoterm but
I need to cleanup and promote stty first and that command is... complicated.

Um, correction. Looks like I need to implement it. (Not in pending.
might be in ashwini's patch pile...)

Rob
Isaac Dunham
2015-01-01 05:14:01 UTC
Permalink
Post by Rob Landley
I've used netcat -f /dev/ttyUSB0 as a terminal, the netcat help points
Use "stty 115200 -F /dev/ttyS0 && stty raw -echo -ctlecho" with
netcat -f to connect to a serial port.
On my todo list is gluing the two of those together into a nanoterm but
I need to cleanup and promote stty first and that command is... complicated.
Um, correction. Looks like I need to implement it. (Not in pending.
might be in ashwini's patch pile...)
To save you the trouble of looking:
I haven't seen an stty patch (that was actually my complaint about reset).

HTH,
Isaac Dunham
Rob Landley
2015-01-01 07:10:29 UTC
Permalink
Post by Isaac Dunham
Post by Rob Landley
I've used netcat -f /dev/ttyUSB0 as a terminal, the netcat help points
Use "stty 115200 -F /dev/ttyS0 && stty raw -echo -ctlecho" with
netcat -f to connect to a serial port.
On my todo list is gluing the two of those together into a nanoterm but
I need to cleanup and promote stty first and that command is... complicated.
Um, correction. Looks like I need to implement it. (Not in pending.
might be in ashwini's patch pile...)
I haven't seen an stty patch (that was actually my complaint about reset).
stty isn't hard, it's just long.

Lemme rephrase that: stty has 8 gazillion config options that aren't
always obvious how to implement.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/stty.html

The entire historical unix layer is one of those things that evolved
from teletype machines with 6 bit serial interfaces driving a daisy
wheel (_5_ bit serial back in the 1930's, they had a shift code between
two pages, yes really:
http://www.textfiles.com/bitsavers/pdf/teletype/144_Model15_Descr_Feb31.pdf
).

For most of toybox all I really care about is "raw mode vs cooked mode",
but stty has to flip all the switches individually...

It's on the todo list.
Post by Isaac Dunham
HTH,
Isaac Dunham
Rob
Marc André Tanner
2014-12-29 11:42:30 UTC
Permalink
Post by Jason Spiro
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. [...]
There are some others (the kernel build requires bc now, if I'm doing
"less" and "vi" I should be able to do "screen", and "rsync" is really
useful...) but right now I'm focused on the list for 1.0, and there are
a lot of smaller commands (and the giant backlog of pending cleanups)
that could get knocked off the list faster...
In this message, I will tell you why I think toybox should emulate tmux
instead of GNU Screen.
While I agree that tmux is an improvement over screen I think it is
already on the bloated side of things. I think session management and
terminal multiplexing are two distinct features which should not be
intermingled.

Therefore I personally prefere abduco[0] in combination with dvtm[1].

In general I would like to see toybox focus on clean, POSIX compilant
replacements of the basic utilities (thing core/find utils, sed, awk etc)
instead of duplicating stuff like tmux.

[0] http://www.brain-dump.org/projects/abduco/
[1] http://www.brain-dump.org/projects/dvtm/
--
Marc André Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0
Jason Spiro
2014-12-30 01:45:01 UTC
Permalink
Post by Marc André Tanner
In general I would like to see toybox focus on clean, POSIX compilant
replacements of the basic utilities (thing core/find utils, sed, awk etc)
instead of duplicating stuff like tmux.
Agreed. The only reason why I mentioned tmux is because Rob mentioned[1],
in passing, the idea of putting screen into Toybox. If he puts in any
terminal multiplexer, I'd prefer tmux instead of screen. But, in truth, I
too would be happier if toybox would focus on the basics rather than on
terminal multiplexers. :)

^ [1]. http://thread.gmane.org/gmane.linux.toybox/1621/focus=1654
Rob Landley
2014-12-30 02:25:23 UTC
Permalink
Post by Jason Spiro
Post by Marc André Tanner
In general I would like to see toybox focus on clean, POSIX compilant
replacements of the basic utilities (thing core/find utils, sed, awk etc)
instead of duplicating stuff like tmux.
Agreed. The only reason why I mentioned tmux is because Rob mentioned[1],
in passing, the idea of putting screen into Toybox. If he puts in any
terminal multiplexer, I'd prefer tmux instead of screen. But, in truth, I
too would be happier if toybox would focus on the basics rather than on
terminal multiplexers. :)
I have a list of things I might add after the 1.0 release. For 1.0, the
existing roadmap is _plenty_.

Rob
David Seikel
2014-12-30 02:51:13 UTC
Permalink
Post by Rob Landley
Post by Jason Spiro
Post by Marc André Tanner
In general I would like to see toybox focus on clean, POSIX
compilant replacements of the basic utilities (thing core/find
utils, sed, awk etc) instead of duplicating stuff like tmux.
Agreed. The only reason why I mentioned tmux is because Rob
mentioned[1], in passing, the idea of putting screen into Toybox.
If he puts in any terminal multiplexer, I'd prefer tmux instead of
screen. But, in truth, I too would be happier if toybox would
focus on the basics rather than on terminal multiplexers. :)
I have a list of things I might add after the 1.0 release. For 1.0,
the existing roadmap is _plenty_.
My guess is that neither screen nor tmux would be added to toybox, but
something a bit more generic, engineered from the ground up, and with
screen and tmux compatible front ends (probably even abduco and dvtm
as well). None of them has any sort of standard as far as I know. And
yeah, post 1.0 release.

I note that nohup is on the roadmap. Along with my boxes stuff, I
suspect that would be half way there.


(Quoting Rob from a different email ...)
Post by Rob Landley
Possibly at some point I'll need to implement a midnight commander
variant, but building it into screen seems odd. (The "windowing
system" vs "desktop" dichotomy. You seem to be objecting that screen
_isn't_ the textmode version of a desktop.)
If you recall, I was all gung ho about doing midnight commander as
well. Part of my boxes infrastructure was designed with that in mind.

Implementing midnight commander - need to split the terminal into
independent rectangular areas. Implementing screen / tmux / dvtm -
need to split the terminal into arbitrary, independent rectangular
areas. Implementing the usual editors - need to split the terminal into
arbitrary, independent rectangular areas. This is why I started
writing boxes in this way. B-)

They all also need to deal with readline style input, and figuring out
their terminal size, including size changes, and do terminal escapes
without a curses style library. All of this I put into boxes two years
ago, mostly been waiting since then for bits of it to pass through Robs
TODO from hell, so I can continue. We probably could have had most of
this by now.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
stephen Turner
2014-12-30 05:01:54 UTC
Permalink
Post by David Seikel
Post by Rob Landley
Post by Jason Spiro
Post by Marc André Tanner
In general I would like to see toybox focus on clean, POSIX
compilant replacements of the basic utilities (thing core/find
utils, sed, awk etc) instead of duplicating stuff like tmux.
I agree this is best, atleast for 1.0
Post by David Seikel
Post by Rob Landley
Post by Jason Spiro
Agreed. The only reason why I mentioned tmux is because Rob
mentioned[1], in passing, the idea of putting screen into Toybox.
If he puts in any terminal multiplexer, I'd prefer tmux instead of
screen. But, in truth, I too would be happier if toybox would
focus on the basics rather than on terminal multiplexers. :)
I have a list of things I might add after the 1.0 release. For 1.0,
the existing roadmap is _plenty_.
My guess is that neither screen nor tmux would be added to toybox, but
something a bit more generic, engineered from the ground up, and with
screen and tmux compatible front ends (probably even abduco and dvtm
as well). None of them has any sort of standard as far as I know. And
yeah, post 1.0 release.
Laugh, cry, or shake your heads. I just found out about
screens/tmux/multiplexers from this email string and all I have to say is
ctrl-a S and ctrl-a | are awsome! This reinforces my love for cli and i
cant wait to start making use of ssh screen -dR just as soon as i figure
out why my laptop hates it. If it does make its way to toybox im in favor
of these features.
Rob Landley
2014-12-30 01:23:51 UTC
Permalink
Post by Jason Spiro
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. [...]
There are some others (the kernel build requires bc now, if I'm doing
"less" and "vi" I should be able to do "screen", and "rsync" is really
useful...) but right now I'm focused on the list for 1.0, and there are
a lot of smaller commands (and the giant backlog of pending cleanups)
that could get knocked off the list faster...
In this message, I will tell you why I think toybox should emulate tmux
instead of GNU Screen.
Let me begin.
If there are obvious differences beyond key bindings, then I'm probably
only interested in the common subset between them. (You can install the
big fluffy one if you really need it. I'm not trying to eliminate every
other implementation, I'm just trying to establish a simple reliable
understandable common base set.)

If vi and less can give you navigable screen output (scrolling and
redrawing the screen, keeping track of your cursor position, etc), then
multi-page support shouldn't be a big deal. That's why it's worth doing,
something else we're _already_ doing gets us 80% of the way there.
Post by Jason Spiro
GNU Screen, a "terminal multiplexer", is very useful. Still, I've since
switched to tmux. It's a newer terminal multiplexer, and is BSD-licensed.
And toybox is zero clause BSD/public domain.

I'm not using the _source_ of gnu screen. I'm not even looking at the
source. (Ick.) I might run the other one under strace to see how it
implemented backgrounding/reconnect (network socket? named pipe? Hand
off a tty/pty filehandle to the other process through that magic unix
domain socket stuff? There's a bunch of ways to do it, I'd rather not
have it fail on a system where they forgot to "ifconfig 127.0.0.1 up"
nor do I want to have to reserve a magic port number, so network
socket's probably out...)
Post by Jason Spiro
It is true that GNU Screen has now started making releases again for the
first time in half a decade. Still, I like tmux so much that I don't plan
to switch back to Screen.
When I say "tar" I mean the command ubuntu has installed. The fact gnu
tar went 5 years between releases (even though everybody in the world
was patching -j support in for bzip they hadn't bothered, until the
fsf's website got broken into and they got kicked awake, and yes I'm the
one who notified slashdot way back when:

http://developers-beta.slashdot.org/story/03/08/13/1530239/fsf-ftp-site-cracked-looking-for-md5-sums

The whole "gee, there should be another implementation of this command I
can use instead, one which isn't maintained by crazy people" thing is
something I come by naturally. Protesting "but the other implementation
sucks", my response is "yes, I know"...
Post by Jason Spiro
tmux makes certain operations easier.[1] For example: It ships with
preconfigured keybindings (<C-b> 0, <C-b> 1, ..., <C-b> 9) which let you
jump to low-numbered windows in just a few keystrokes.
Screen on unbuntu has ctrl-a 1, ctrl-a 2, ctrl-a 3...

The first time I used "screen" was on the sun workstations back at
college. I have no idea what implementation they were using, and don't
really care. As with vi, I learned a half-dozen keybindings and I was
going to wait for people to complain about the others missing and use
that as a guide of what to add. (Ok, I might read the man page for low
hanging fruit. And if there's an actual standard, I implement as much of
the standard as is remotely reasonable to do. But I am not adding ebcdic
support to dd, I don't care what posix says.)
Post by Jason Spiro
Another example: To
renumber a window, you need only hit five keys (<C-b> . 9 <RET>), instead of
nine (<C-a> : n u <TAB> 9 <RET>).
There may be a way to do that in screen. I never bothered learning how.
Post by Jason Spiro
tmux is also easier to learn. For example: It shows a status line (tab
bar) by default, instead of forcing users to mess with complex configuration
options just to get a status line. See screenshot[2].
I.E. it eats screen space and screws up your terminal size, so it has to
intercept the ansi escape sequence querying that stuff instead of
letting it naturally pass to your xterm and let _that_ do it.

Despite the wheel reimplementing inherent in the project itself, "I can
avoid doing that" still gives me a warm glow. This TLS nonsense makes me
grumpy because "avoiding doing that" has developed significant downsides
because the internet is changing out from under me (thank you SO much
insane NSA voyeurism bugging every single backbone router without a
warrant).

Possibly at some point I'll need to implement a midnight commander
variant, but building it into screen seems odd. (The "windowing system"
vs "desktop" dichotomy. You seem to be objecting that screen _isn't_ the
textmode version of a desktop.)
Post by Jason Spiro
tmux is included in the software repositories of Ubuntu, Debian "stable",
and other distros.
Trying to remember if screen was installed by default on ubuntu, or if I
had to apt-get it.
Post by Jason Spiro
tmux's basic keybindings are fairly similar to Screen's. But, instead of
Ctrl+A, tmux's default prefix is Ctrl+B. (This is reconfigurable.)
This is an argument against screen?
Post by Jason Spiro
You can find a tmux reference card[3] on the Web.
Dear Rob: I know you mention Screen in your todo.txt file[4]. But please
consider instead mentioning tmux. Those who are familiar only with good old
Screen can either adapt to the nicer user interface that tmux provides, or
can download and install Screen themselves.
Can you summarize the user interface you want? What keybindings do what
and how does "detach" work in tmux?

With screen I've learned ctrl-a ctrl-a to cycle through windows, ctrl-a
c to create a new window, ctrl-a k to kill a window, ctrl-a d to detach,
ctrl-a " to get a window list, ctrl-a ESC to let me scroll back up
through the window's output above the top of the screen (and then ESC
again to exit that), and then to reattach to a named session "screen -dR
NAME".

Connecting to my server, I generally do:

ssh 192.168.1.12 -t screen -dR walrus

Which will even bump a previous screen instance if I've got it and give
me my persistent stack of running processes. And then when I ctrl-a d
(or just suspend my netbook) it detaches and continues running in the
background.

Presumaly this is all horrible and tmux is better. Could you explain how?
Post by Jason Spiro
Cheers,
--Jason
P.S. I thank everyone on this list for maintaining Toybox. My next Android
device will be a more pleasant to work with, thanks to you all.
I'm so sorry I'm behind on Anhwini and enh's todo lists (and the whole
of pending). I sat down yesterday to clear backlog and hit the fact that
the reason I hadn't merged Ashwini's kill patch is kill -l wasn't
working, and the reason it wasn't working is when I merged it into
killall5 I hit the darn command proessing glitch where kill is working
in killall5's flag context an when killall5 is disabled (ala
scripts/single.sh) all those FLAG_x macros become 0.

I blogged about it:

http://landley.net/notes.html#18-10-2014
http://landley.net/notes.html#05-11-2014

and now I have to deal with it...

But before I could deal with THAT today I tried to swap in the linux
3.18 kernel in aboriginal and build i686 and SOMEBODY ADDED PERL BACK as
a build preprequisite. (I literally spent YEARS getting that removed
last time, and they added it back in 3.18-rc6. Right at the END of the
build cycle. Honestly...)

Bisected it to:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e6023367d779

And now I have to figure out what (?:) means in perl regex context (I
think it means "skip this parenthetical when filling out \1 and \2 and
so on), but I need to _confirm_ that to write a reliable replacement in
something other than perl? I eventually found it under
http://perldoc.perl.org/perlre.html#Extended-Patterns ...
Post by Jason Spiro
^ [1]. http://www.techrepublic.com/blog/linux-and-open-source/is-tmux-the-
gnu-screen-killer/
http://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines
Post by Jason Spiro
^ [2]. http://tmux.sf.net/tmux3.png
^ [3]. http://www.mechanicalkeys.com/files/os/notes/tm.html
#3 is a decent cheat sheet...
Post by Jason Spiro
^ [4]. http://www.landley.net/code/toybox/todo.txt
That todo is from like 2011. The roadmap.html file is much more recent.

Rob
Jason Spiro
2014-12-30 04:04:42 UTC
Permalink
Post by Rob Landley
And toybox is zero clause BSD/public domain.
I was oversimplifying before. Really, tmux uses the ISC license, which is
the same license which OpenBSD is now using for new code.
Post by Rob Landley
I'm not using the _source_ of gnu screen. I'm not even looking at the
source. (Ick.)
tmux's source is much cleaner.
Post by Rob Landley
When I say "tar" I mean the command ubuntu has installed.
:)
Post by Rob Landley
The whole "gee, there should be another implementation of this command I
can use instead, one which isn't maintained by crazy people" thing is
something I come by naturally. Protesting "but the other implementation
sucks", my response is "yes, I know"...
Post by Jason Spiro
Another example: To
renumber a window, you need only hit five keys (<C-b> . 9 <RET>), instead of
nine (<C-a> : n u <TAB> 9 <RET>).
There may be a way to do that in screen. I never bothered learning how.
Most people don't bother learning how to make keystroke-saving configuration
changes like that.
Post by Rob Landley
Post by Jason Spiro
tmux is also easier to learn. For example: It shows a status line (tab
bar) by default, instead of forcing users to mess with complex configuration
options just to get a status line. See screenshot[2].
I.E. it eats screen space and screws up your terminal size, so it has to
intercept the ansi escape sequence querying that stuff instead of
letting it naturally pass to your xterm and let _that_ do it.
Even in tmux, it's easy to hide the status line. But tmux enables it by
default, since many people like it.
Post by Rob Landley
Trying to remember if screen was installed by default on ubuntu, or if I
had to apt-get it.
Screen was included with all Ubuntu versions released before 26 Nov 2010.[*]

^ [*]. Source:
http://changelogs.ubuntu.com/changelogs/pool/main/u/ubuntu-meta/ubuntu-meta_1.327/changelog
Post by Rob Landley
Post by Jason Spiro
tmux's basic keybindings are fairly similar to Screen's. But, instead of
Ctrl+A, tmux's default prefix is Ctrl+B. (This is reconfigurable.)
This is an argument against screen?
No. I'm explaining that screen and tmux are pretty similar, though the
prefix key is different.
Post by Rob Landley
Can you summarize the user interface you want? What keybindings do what
and how does "detach" work in tmux?
With screen I've learned ctrl-a ctrl-a to cycle through windows, ctrl-a
c to create a new window, ctrl-a k to kill a window, ctrl-a d to detach,
ctrl-a " to get a window list, ctrl-a ESC to let me scroll back up
through the window's output above the top of the screen (and then ESC
again to exit that), and then to reattach to a named session "screen -dR
NAME".
If you add a terminal multiplexer to toybox, then I would like ctrl-b p and
ctrl-b n to switch windows, ctrl-b 0 through ctrl-b 9 to jump to a window,
ctrl-b c to create a new window, and ctrl-b x to kill a window. If you
want, you can also implement ctrl-b d to detach, "tmux attach" to reattach,
C-b [ to enter scrollback mode, and ESC to exit scrollback mode. tmux
supports named sessions, but I don't use them.
Post by Rob Landley
Presumaly this is all horrible and tmux is better. Could you explain how?
Screen is good. It's just that tmux is even better. :)
Post by Rob Landley
I'm so sorry I'm behind on Anhwini and enh's todo lists (and the whole
of pending).
Have you ever considered taking on a co-maintainer, or at least taking on a
helper who will maintain "pending" for you?

Alternatively, have you thought of creating a "toybox-experimental" branch,
and accepting "boxes" into it, and encouraging people to test it and send
feedback?
Post by Rob Landley
But before I could deal with THAT today I tried to swap in the linux
3.18 kernel in aboriginal and build i686 and SOMEBODY ADDED PERL BACK as
a build preprequisite. (I literally spent YEARS getting that removed
last time, and they added it back in 3.18-rc6. Right at the END of the
build cycle. Honestly...)
I know you strongly dislike Perl. But the Linux kernel maintainers don't
mind it so much, and they probably don't care about Aboriginal Linux at all.
Why don't you want to add Perl to Aboriginal Linux? Perl isn't that big,
and if you delete some or all of the modules which come with it, it's even
smaller.
Post by Rob Landley
Post by Jason Spiro
^ [4]. http://www.landley.net/code/toybox/todo.txt
That todo is from like 2011. The roadmap.html file is much more recent.
Ah, okay.
David Seikel
2014-12-30 05:15:46 UTC
Permalink
On Tue, 30 Dec 2014 04:04:42 +0000 (UTC) Jason Spiro
Post by Jason Spiro
Post by Rob Landley
And toybox is zero clause BSD/public domain.
I was oversimplifying before. Really, tmux uses the ISC license,
which is the same license which OpenBSD is now using for new code.
Post by Rob Landley
I'm not using the _source_ of gnu screen. I'm not even looking at
the source. (Ick.)
tmux's source is much cleaner.
Post by Rob Landley
When I say "tar" I mean the command ubuntu has installed.
:)
Post by Rob Landley
The whole "gee, there should be another implementation of this
command I can use instead, one which isn't maintained by crazy
people" thing is something I come by naturally. Protesting "but the
other implementation sucks", my response is "yes, I know"...
Post by Jason Spiro
Another example: To
renumber a window, you need only hit five keys (<C-b> . 9 <RET>),
instead of
Post by Rob Landley
Post by Jason Spiro
nine (<C-a> : n u <TAB> 9 <RET>).
There may be a way to do that in screen. I never bothered learning how.
Most people don't bother learning how to make keystroke-saving
configuration changes like that.
Post by Rob Landley
Post by Jason Spiro
tmux is also easier to learn. For example: It shows a status
line (tab bar) by default, instead of forcing users to mess with
complex
configuration
Post by Rob Landley
Post by Jason Spiro
options just to get a status line. See screenshot[2].
I.E. it eats screen space and screws up your terminal size, so it
has to intercept the ansi escape sequence querying that stuff
instead of letting it naturally pass to your xterm and let _that_
do it.
Even in tmux, it's easy to hide the status line. But tmux enables it
by default, since many people like it.
Actually I use the tmux status line to add tabs, then remove the tabs
from the terminal emulator. It gives me back some screen space. No
need for all those pixels being eaten up by fancy graphics around each
tab.
Post by Jason Spiro
Post by Rob Landley
Trying to remember if screen was installed by default on ubuntu, or
if I had to apt-get it.
Screen was included with all Ubuntu versions released before 26 Nov 2010.[*]
http://changelogs.ubuntu.com/changelogs/pool/main/u/ubuntu-meta/ubuntu-meta_1.327/changelog
Post by Rob Landley
Post by Jason Spiro
tmux's basic keybindings are fairly similar to Screen's. But,
instead of Ctrl+A, tmux's default prefix is Ctrl+B. (This is
reconfigurable.)
This is an argument against screen?
No. I'm explaining that screen and tmux are pretty similar, though
the prefix key is different.
Post by Rob Landley
Can you summarize the user interface you want? What keybindings do
what and how does "detach" work in tmux?
With screen I've learned ctrl-a ctrl-a to cycle through windows,
ctrl-a c to create a new window, ctrl-a k to kill a window, ctrl-a
d to detach, ctrl-a " to get a window list, ctrl-a ESC to let me
scroll back up through the window's output above the top of the
screen (and then ESC again to exit that), and then to reattach to a
named session "screen -dR NAME".
If you add a terminal multiplexer to toybox, then I would like ctrl-b
p and ctrl-b n to switch windows, ctrl-b 0 through ctrl-b 9 to jump
to a window, ctrl-b c to create a new window, and ctrl-b x to kill a
window. If you want, you can also implement ctrl-b d to detach,
"tmux attach" to reattach, C-b [ to enter scrollback mode, and ESC to
exit scrollback mode. tmux supports named sessions, but I don't use
them.
Well, if I add it using the boxes infrastructure, it already supports
reconfiguring keystrokes. Coz many of the editors do, plus it's how I
do the upper compatibility layers.

I do use named tmux sessions. Helps with scripts to tell them which
sessions they should be working on.
Post by Jason Spiro
Post by Rob Landley
Presumaly this is all horrible and tmux is better. Could you
explain how?
Screen is good. It's just that tmux is even better. :)
Post by Rob Landley
I'm so sorry I'm behind on Anhwini and enh's todo lists (and the
whole of pending).
Have you ever considered taking on a co-maintainer, or at least
taking on a helper who will maintain "pending" for you?
Alternatively, have you thought of creating a "toybox-experimental"
branch, and accepting "boxes" into it, and encouraging people to test
it and send feedback?
That works for me. Actually anything that gets it out there in a way
that people can test it, report back to me, or help develop it, works
for me. The thing I don't like is it sitting in limbo for years.
Post by Jason Spiro
Post by Rob Landley
But before I could deal with THAT today I tried to swap in the linux
3.18 kernel in aboriginal and build i686 and SOMEBODY ADDED PERL
BACK as a build preprequisite. (I literally spent YEARS getting
that removed last time, and they added it back in 3.18-rc6. Right
at the END of the build cycle. Honestly...)
I know you strongly dislike Perl. But the Linux kernel maintainers
don't mind it so much, and they probably don't care about Aboriginal
Linux at all. Why don't you want to add Perl to Aboriginal Linux?
Perl isn't that big, and if you delete some or all of the modules
which come with it, it's even smaller.
Post by Rob Landley
Post by Jason Spiro
^ [4]. http://www.landley.net/code/toybox/todo.txt
That todo is from like 2011. The roadmap.html file is much more recent.
Ah, okay.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rich Felker
2014-12-30 06:12:45 UTC
Permalink
Post by Jason Spiro
Post by Rob Landley
But before I could deal with THAT today I tried to swap in the linux
3.18 kernel in aboriginal and build i686 and SOMEBODY ADDED PERL BACK as
a build preprequisite. (I literally spent YEARS getting that removed
last time, and they added it back in 3.18-rc6. Right at the END of the
build cycle. Honestly...)
I know you strongly dislike Perl. But the Linux kernel maintainers don't
mind it so much, and they probably don't care about Aboriginal Linux at all.
Why don't you want to add Perl to Aboriginal Linux? Perl isn't that big,
and if you delete some or all of the modules which come with it, it's even
smaller.
Um, I think Perl is about the size of all of Aboriginal combined.
Doubling the size of the package for the sake of gratuitous (i.e.
otherwise useless) crap needed for the kernel build system does not
make sense.

Rich
Rob Landley
2015-01-01 07:17:08 UTC
Permalink
Post by Rich Felker
Post by Jason Spiro
Post by Rob Landley
But before I could deal with THAT today I tried to swap in the linux
3.18 kernel in aboriginal and build i686 and SOMEBODY ADDED PERL BACK as
a build preprequisite. (I literally spent YEARS getting that removed
last time, and they added it back in 3.18-rc6. Right at the END of the
build cycle. Honestly...)
I know you strongly dislike Perl. But the Linux kernel maintainers don't
mind it so much, and they probably don't care about Aboriginal Linux at all.
Why don't you want to add Perl to Aboriginal Linux? Perl isn't that big,
and if you delete some or all of the modules which come with it, it's even
smaller.
Um, I think Perl is about the size of all of Aboriginal combined.
Doubling the size of the package for the sake of gratuitous (i.e.
otherwise useless) crap needed for the kernel build system does not
make sense.
I got the previous perl removal patches upstream eventually (after about
7 years of trying), I just had to reenact John Cleese's part of the Dead
Parrot skit:

http://lkml.iu.edu/hypermail/linux/kernel/1302.3/02061.html

I should submit the new perl removal patch to linux-kernel. I already
checked it in to Aboriginal:

http://landley.net/hg/aboriginal/rev/1717

I just... have little enthusiasm left for dealing with the linux-kernel
community.

Rob
Marc André Tanner
2014-12-29 12:45:33 UTC
Permalink
Post by Rob Landley
Post by enh
On Sun, Dec 21, 2014 at 1:07 PM, James McMechan
fortunately n-curses devolves tremendously when you don't try to support
every terminal type under the sun efficiently...
I think using raw escape sequences or using a home made translation
layer like tmux and vim do it is not optimal. I would prefer a
clean and simple curses library most likely based on the one from
NetBSD.
Post by Rob Landley
Post by enh
(very off-topic at this point, but vim doesn't use ncurses under any
circumstances. moreover, it turns out that vim actually has all kinds
of different fallbacks and as long as you don't misconfigure it like i
did, it builds and runs fine out of the box on Android. it has a
minimal tgetstr/tgoto/tputs implementation *and* a few built-in
termcap subsets, including xterm. it's an 8.3MiB binary unstripped or
2.1MiB stripped for aarch64, though, so probably not of much interest
to you guys :-) )
I don't care that much about binary size but vim's source is a bloated,
unmaintainable, #ifdef littered mess, designed to run on all broken
systems ever envisioned by mankind.
Post by Rob Landley
I plan to implement vi over the next year
I agree that there is a need for a clean vi(m) like editor. Based on my
experience I got from writing my own work in progress clone[0], the
problem is that while the basic editing functionality is fairly easy to
provide, users will also expect more advance stuff like (visual block mode,
code folding, auto completion etc). If you don't implement these then
they won't use it as their main editor, at which point the whole endeavor
becomes kind of pointless ...

I don't think providing such a full featured editor should be the focus
for toybox, but unfortunatley anything less will be of little interest
for most people.

[0] https://github.com/martanne/vis
--
Marc André Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0
Roy Tam
2014-12-29 15:42:23 UTC
Permalink
Post by Marc André Tanner
Post by Rob Landley
Post by enh
On Sun, Dec 21, 2014 at 1:07 PM, James McMechan
fortunately n-curses devolves tremendously when you don't try to support
every terminal type under the sun efficiently...
I think using raw escape sequences or using a home made translation
layer like tmux and vim do it is not optimal. I would prefer a
clean and simple curses library most likely based on the one from
NetBSD.
Post by Rob Landley
Post by enh
(very off-topic at this point, but vim doesn't use ncurses under any
circumstances. moreover, it turns out that vim actually has all kinds
of different fallbacks and as long as you don't misconfigure it like i
did, it builds and runs fine out of the box on Android. it has a
minimal tgetstr/tgoto/tputs implementation *and* a few built-in
termcap subsets, including xterm. it's an 8.3MiB binary unstripped or
2.1MiB stripped for aarch64, though, so probably not of much interest
to you guys :-) )
I don't care that much about binary size but vim's source is a bloated,
unmaintainable, #ifdef littered mess, designed to run on all broken
systems ever envisioned by mankind.
Post by Rob Landley
I plan to implement vi over the next year
I agree that there is a need for a clean vi(m) like editor. Based on my
experience I got from writing my own work in progress clone[0], the
problem is that while the basic editing functionality is fairly easy to
provide, users will also expect more advance stuff like (visual block mode,
code folding, auto completion etc). If you don't implement these then
they won't use it as their main editor, at which point the whole endeavor
becomes kind of pointless ...
I don't think providing such a full featured editor should be the focus
for toybox, but unfortunatley anything less will be of little interest
for most people.
For me, why not using something that is already targeting toybox?
Let's get David's boxes in toybox repo!
Post by Marc André Tanner
[0] https://github.com/martanne/vis
--
Marc André Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
dmccunney
2014-12-29 21:06:00 UTC
Permalink
Post by Marc André Tanner
Post by Rob Landley
I plan to implement vi over the next year
I agree that there is a need for a clean vi(m) like editor. Based on my
experience I got from writing my own work in progress clone[0], the
problem is that while the basic editing functionality is fairly easy to
provide, users will also expect more advance stuff like (visual block mode,
code folding, auto completion etc). If you don't implement these then
they won't use it as their main editor, at which point the whole endeavor
becomes kind of pointless ...
Hardly pointless.

I first learned *nix in the 80's, beginning with AT&T System V Release
2, when Linux was not yet a gleam in Linus Torvald's eye. I learned
vi because it was the standard editor offered with Unix systems, and
while other things like emacs were around, vi was pretty much
guaranteed to be on the machine you had to work on. I came to Unix
from an IBM mainframe environment, and vi was a rude shock after
screen editing on a 3270 block mode terminal with IBM SPF, but once I
understood the design concepts I grew to like it.

These days I use Linux, and Linux tends to come with Vim. I have Vim
under Linux and Windows, but for the use I make of it, it might as
well *be* original vi. I've never had a need to use the myriad of
features Vim added to the original design.

Toybox needs a decent basic editor, and one modeled after vi is a
reasonable design choice. If it can also emulate others like emacs,
and it's essentially a matter of mapping different keystrokes to
underlying editor functions, and so much the better.

On a Toybox based system, I'd likely use its editor by default, but I
wouldn't be concerned with things like syntax highlighting, code
folding, and auto-completion, unless those could be implemented on top
of the existing framework with a minimum of code and trouble. If I
really need an editor that does all the fancy stuff, I'll install one.
Lord knows there are enough to choose from.

Even when I have more powerful editors available, I've often used vi
from a command line for editing because most of the editing I need to
do is quick-and-dirty - create a script, edit a config file - and a
full featured programmer's development editor with all bells and
whistles is massive overkill. Speed trumps function, and I can invoke
vi or the like, do my edits, save my changes and exit far faster than
I can in a major editor.

Assuming that an editor that might not be used as the main editor is
pointless to implement is a misguided notion. The editor needs to be
good enough to handle the basic chores and give the users something to
use until they can install whatever they normally prefer if they want
more than it offers. It shouldn't *try* to compete with what the user
might otherwise use except by accident.
______
Dennis
https://plus.google.com/u/0/105128793974319004519
Rob Landley
2015-01-01 04:26:17 UTC
Permalink
Post by dmccunney
Post by Rob Landley
I plan to implement vi over the next year
...
Post by dmccunney
Toybox needs a decent basic editor, and one modeled after vi is a
reasonable design choice.
It's also specified in posix-2008:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html

Emacs, joe, nano and so on aren't. (That's presumably why busybox
implemented vi but not the others.)

That said, Linus Torvalds' personal primary editor is a microemacs fork
he's maintained for years:

https://git.kernel.org/cgit/editors/uemacs/uemacs.git/

Which I admit is of interest as well. :)

Rob
Rich Felker
2014-12-29 18:00:10 UTC
Permalink
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. (It's easy to
knock out a simple 90% implementation. It's really hard to make
something do everything right in all the cases people are going to throw
at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.

Rich
dmccunney
2014-12-29 21:10:07 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. (It's easy to
knock out a simple 90% implementation. It's really hard to make
something do everything right in all the cases people are going to throw
at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.
You might look at Albrecht Kliene's e3. e3 is that Linux rarity -
code written mostly in Assembler. (There is also a C version for
other systems.)

By default, e3 implements a subset of the WordStar command set, but
can emulate emacs, ne, pico and vi. It's the same executable -
symlink it under the desired name and it assumes the desired
personality. Puppy Linux bundled e3 as vi. See
http://texteditors.org/cgi-bin/wiki.pl?E3

(There is also a Puppy build of "Traditional vi", built from source
released by Caldera, which I prefer under Puppy. See
http://ex-vi.sourceforge.net/ for source. It's under a BSD license.)
Post by Rich Felker
Rich
______
Dennis
https://plus.google.com/u/0/105128793974319004519
David Seikel
2014-12-30 00:18:19 UTC
Permalink
On Mon, 29 Dec 2014 16:10:07 -0500 dmccunney
Post by dmccunney
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've
been debugging sed against real-world data for _weeks_ now. (It's
easy to knock out a simple 90% implementation. It's really hard to
make something do everything right in all the cases people are
going to throw at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.
Mg is one I haven't heard of. Got a URL?
Post by dmccunney
You might look at Albrecht Kliene's e3. e3 is that Linux rarity -
code written mostly in Assembler. (There is also a C version for
other systems.)
By default, e3 implements a subset of the WordStar command set, but
can emulate emacs, ne, pico and vi. It's the same executable -
symlink it under the desired name and it assumes the desired
personality. Puppy Linux bundled e3 as vi. See
http://texteditors.org/cgi-bin/wiki.pl?E3
(There is also a Puppy build of "Traditional vi", built from source
released by Caldera, which I prefer under Puppy. See
http://ex-vi.sourceforge.net/ for source. It's under a BSD license.)
E3 was my inspiration for boxes. Though I do have a soft spot for
assembler, I'd prefer using C for this sort of thing. Or perhaps C +
Lua, but Rob decided a while ago to not use Lua.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Roy Tam
2014-12-30 00:31:43 UTC
Permalink
Post by David Seikel
On Mon, 29 Dec 2014 16:10:07 -0500 dmccunney
Post by dmccunney
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've
been debugging sed against real-world data for _weeks_ now. (It's
easy to knock out a simple 90% implementation. It's really hard to
make something do everything right in all the cases people are
going to throw at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.
Mg is one I haven't heard of. Got a URL?
Here it is.
http://homepage.boetes.org/software/mg/
Post by David Seikel
Post by dmccunney
You might look at Albrecht Kliene's e3. e3 is that Linux rarity -
code written mostly in Assembler. (There is also a C version for
other systems.)
By default, e3 implements a subset of the WordStar command set, but
can emulate emacs, ne, pico and vi. It's the same executable -
symlink it under the desired name and it assumes the desired
personality. Puppy Linux bundled e3 as vi. See
http://texteditors.org/cgi-bin/wiki.pl?E3
(There is also a Puppy build of "Traditional vi", built from source
released by Caldera, which I prefer under Puppy. See
http://ex-vi.sourceforge.net/ for source. It's under a BSD license.)
E3 was my inspiration for boxes. Though I do have a soft spot for
assembler, I'd prefer using C for this sort of thing. Or perhaps C +
Lua, but Rob decided a while ago to not use Lua.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
dmccunney
2014-12-30 00:52:59 UTC
Permalink
Post by David Seikel
On Mon, 29 Dec 2014 16:10:07 -0500 dmccunney
Post by dmccunney
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've
been debugging sed against real-world data for _weeks_ now. (It's
easy to knock out a simple 90% implementation. It's really hard to
make something do everything right in all the cases people are
going to throw at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.
Mg is one I haven't heard of. Got a URL?
Mg was originally MicroGnuEmacs, a tiny emacs implementation based on
Dave Conroy's microemacs, intended to preserve the Gnu Emacs key
assignments. It was written by Mike Meyer, and appears in Craig
Finseth's list of Emacs implementations:
http://www.finseth.com/emacs.html#66 The name ran into some objection
from the FSF folks, and got changed to mg.

There is no current URL, and hasn't been for ages, but it's findable
in various repositories. Source for an OpenBSD versions is here:
http://homepage.boetes.org/software/mg/

Julie Melbin's NotGNU editor seems to be an mg derivative:
http://www.notgnu.org/
Post by David Seikel
Post by dmccunney
You might look at Albrecht Kliene's e3. e3 is that Linux rarity -
code written mostly in Assembler. (There is also a C version for
other systems.)
<...>
Post by David Seikel
E3 was my inspiration for boxes. Though I do have a soft spot for
assembler, I'd prefer using C for this sort of thing. Or perhaps C +
Lua, but Rob decided a while ago to not use Lua.
Albrecht apparently wanted to write the smallest most efficient code
possible, and assembler was the way to get it. He does also provide C
source for those wishing to build on other OSes. The Win9X binary
from the C code is all of 20KB here. (He also provides batch files
for the other emulations, but since NTFS5 supports actual links, I use
a binary and symlinks as under Linux on Win7.)

I have a soft spot for Lua, but I can understand Rob not wanting to go
that route in Toybox. If you want to go the Lua route, you may like
TextAdept, an editor based on the Scintilla edit control with Lua for
scripting and dynamic lexing: http://foicica.com/textadept/
______
Dennis
https://plus.google.com/u/0/105128793974319004519
Rich Felker
2014-12-30 05:58:54 UTC
Permalink
Post by David Seikel
On Mon, 29 Dec 2014 16:10:07 -0500 dmccunney
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've
been debugging sed against real-world data for _weeks_ now. (It's
easy to knock out a simple 90% implementation. It's really hard to
make something do everything right in all the cases people are
going to throw at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.
Mg is one I haven't heard of. Got a URL?
http://en.wikipedia.org/wiki/Mg_%28editor%29
http://homepage.boetes.org/software/mg/

It's the only "emacs clone" I've ever seen that actually feels like
real emacs (except the lack of slowness) for real editing tasks. The
rest only look like emacs superficially, although jmacs (from joe) and
qemacs (Fabrice Bellard) are somewhat close but still feel like
semi-emacs-like bindings on a foreign editor.

Rich
David Seikel
2014-12-30 06:08:55 UTC
Permalink
Post by Rich Felker
Post by David Seikel
On Mon, 29 Dec 2014 16:10:07 -0500 dmccunney
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the
four realy big commands required by posix (sed, awk, sh, vi)
and I've been debugging sed against real-world data for
_weeks_ now. (It's easy to knock out a simple 90%
implementation. It's really hard to make something do
everything right in all the cases people are going to throw at
it.)
It would be awesome to have mg too or a similar mg-like emacs
clone but with working unicode support.
Mg is one I haven't heard of. Got a URL?
http://en.wikipedia.org/wiki/Mg_%28editor%29
http://homepage.boetes.org/software/mg/
It's the only "emacs clone" I've ever seen that actually feels like
real emacs (except the lack of slowness) for real editing tasks. The
rest only look like emacs superficially, although jmacs (from joe) and
qemacs (Fabrice Bellard) are somewhat close but still feel like
semi-emacs-like bindings on a foreign editor.
I can add slowness as an option if you like. B-)
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rob Landley
2015-01-01 04:44:22 UTC
Permalink
Post by Rich Felker
Post by David Seikel
On Mon, 29 Dec 2014 16:10:07 -0500 dmccunney
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've
been debugging sed against real-world data for _weeks_ now. (It's
easy to knock out a simple 90% implementation. It's really hard to
make something do everything right in all the cases people are
going to throw at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.
Mg is one I haven't heard of. Got a URL?
http://en.wikipedia.org/wiki/Mg_%28editor%29
http://homepage.boetes.org/software/mg/
It's the only "emacs clone" I've ever seen that actually feels like
real emacs (except the lack of slowness) for real editing tasks. The
rest only look like emacs superficially, although jmacs (from joe) and
qemacs (Fabrice Bellard) are somewhat close but still feel like
semi-emacs-like bindings on a foreign editor.
I'm not a native user of emacs. (I did wordstar bindings from Turbo C
through qedit until the Sun Workstations at Rutgers kept getting really
confused by ctrl-s even though joe claimed it was putting the terminal
in raw mode when it didn't segfault. Then I switched to vi in
self-defense, but have never actually recommended it.)

A very, very long time ago I used microemacs on the Amiga for about 6
months, but haven't poked at it since. When we get around to terminal
stuff I'd like somebody comfortable with emacs to tell me how the
keybindings/behavior are wrong, I can't dogfood that.

Rob
David Seikel
2015-01-01 06:39:42 UTC
Permalink
Post by Rob Landley
Post by Rich Felker
Post by David Seikel
On Mon, 29 Dec 2014 16:10:07 -0500 dmccunney
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the
four realy big commands required by posix (sed, awk, sh, vi)
and I've been debugging sed against real-world data for _weeks_
now. (It's easy to knock out a simple 90% implementation. It's
really hard to make something do everything right in all the
cases people are going to throw at it.)
It would be awesome to have mg too or a similar mg-like emacs
clone but with working unicode support.
Mg is one I haven't heard of. Got a URL?
http://en.wikipedia.org/wiki/Mg_%28editor%29
http://homepage.boetes.org/software/mg/
It's the only "emacs clone" I've ever seen that actually feels like
real emacs (except the lack of slowness) for real editing tasks. The
rest only look like emacs superficially, although jmacs (from joe)
and qemacs (Fabrice Bellard) are somewhat close but still feel like
semi-emacs-like bindings on a foreign editor.
I'm not a native user of emacs. (I did wordstar bindings from Turbo C
through qedit until the Sun Workstations at Rutgers kept getting
really confused by ctrl-s even though joe claimed it was putting the
terminal in raw mode when it didn't segfault. Then I switched to vi in
self-defense, but have never actually recommended it.)
A very, very long time ago I used microemacs on the Amiga for about 6
months, but haven't poked at it since. When we get around to terminal
stuff I'd like somebody comfortable with emacs to tell me how the
keybindings/behavior are wrong, I can't dogfood that.
I've been programming computers since the mid '70s, programming every
class of computing device except supercomputers (though I did work for
the company that turned bunches of PS3's into supercomputers). During
the first couple of decades of that, I would often end up using what
ever random editors were available on what ever random computer systems
I had to work on that day. Often several in a single day. I've used
OMR cards (punch cards with black marks instead of holes), Martian
hieroglyph editors (APL was my first language), front panel switches,
line editors, hex editors (even on machines that only had a hex
keypad), screen editors, terminal based IDE's, and full blown GUI
IDE's. I've even used beads, matchboxes, and glue to program a
computer. I came very close to using a magnetised pin once for
editing, but never used butterflies. I've used slide rules and
abacuses.

I've _been_ a text editor. For my first real job, on a mining site
(only one computer within hundreds of kilometres, I built it from
discrete components and sheet metal), one of the bosses would sit in his
room with source code printouts. He would scribble line editor
commands all over these printouts, then hand them to me. Late at night
when the computer was not being used by the office staff, I'd sit in
front of it and enter his editor commands. Was very educational for a
young coder. The client later bought Wordstar for the secretaries, a
big step up.

I have a simple test to decide if I like an editor as a result of these
decades of random editor usage. If I can't sit down with the editor
and figure out how to do basic editing and saving in less than a
minute (sans documentation), then in my opinion it's a crap editor.
Both TECO and vi fail this test miserably, though oddly enough I have a
soft spot for TECO.

I have memorised one command to deal with vi, since it does tend to be
the default editor in unix style systems, thus I accidentally end up
inside it sometimes without asking for it -

killall -KILL vi

OK, yes I know about :qa as well, killall is much more satisfying. B-)

Out of all the editors that I have implemented in boxes, I have used
them all professionally, including the line editors buried inside emacs
and vi. These days I prefer to use mc and mcedit for everything though,
which is why boxes includes mcedit. So I'll dogfood mcedit, the rest
I'll have to rely on others to tell me what I get wrong.

A traditional trick to try with TECO is to type in your name, to see
what it does. The first two letters of my name delete the rest of the
file, so mine is kinda boring. I have no plans to implement TECO in
boxes, vi and ed are already there.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
dmccunney
2015-01-01 19:04:41 UTC
Permalink
Post by David Seikel
I have a simple test to decide if I like an editor as a result of these
decades of random editor usage. If I can't sit down with the editor
and figure out how to do basic editing and saving in less than a
minute (sans documentation), then in my opinion it's a crap editor.
Both TECO and vi fail this test miserably, though oddly enough I have a
soft spot for TECO.
These days, the general assumption is that you can open a file in an
editor with "<editor> <filename>", and that once up, cursor keys can
be used to move around in the file and that text can be added where
desired by typing it at the cursor location and deleted with Backspace
or Delete keys.

Vi originated in the days when some of those assumptions might not be
true. Some early terminals on Unix systems didn't *have* cursor keys
or F-keys. The vi command set and separation between input and
command modes was a result.

Given where Toybox will be used, it's a reasonable assumption folks
running the included editor will already know enough of the basics to
deal with vi, even if they may prefer something else, so "Can they
figure out basic editing and saving without docs" is a non-issue.
Post by David Seikel
killall -KILL vi
OK, yes I know about :qa as well, killall is much more satisfying. B-)
You *could* simply remove vi from your system. :-)
Post by David Seikel
Out of all the editors that I have implemented in boxes, I have used
them all professionally, including the line editors buried inside emacs
and vi. These days I prefer to use mc and mcedit for everything though,
which is why boxes includes mcedit. So I'll dogfood mcedit, the rest
I'll have to rely on others to tell me what I get wrong.
We will. The big issue I can see is preserving "moded" editing in vi
and dispensing with it in the others.
Post by David Seikel
I have no plans to implement TECO in boxes, vi and ed are already there.
C source is available for a port of TECO if anyone wants to build it,
but I don't see a need in Toybox. Stallman used ITS TECO as the
implementation language for Editing MACroS, the lineal ancestor of Gnu
Emacs, because it was what was available. It was a language in which
he could implement an editor. It took over as the standard editor at
MIT's AI labs, and he realized how successful his efforts had been
when he no longer remembered how to do things in raw TECO. We have
other languages in which you can write editors now.
______
Dennis
https://plus.google.com/u/0/105128793974319004519
David Seikel
2015-01-02 00:02:53 UTC
Permalink
Post by dmccunney
Post by David Seikel
I have a simple test to decide if I like an editor as a result of
these decades of random editor usage. If I can't sit down with the
editor and figure out how to do basic editing and saving in less
than a minute (sans documentation), then in my opinion it's a crap
editor. Both TECO and vi fail this test miserably, though oddly
enough I have a soft spot for TECO.
These days, the general assumption is that you can open a file in an
editor with "<editor> <filename>", and that once up, cursor keys can
be used to move around in the file and that text can be added where
desired by typing it at the cursor location and deleted with Backspace
or Delete keys.
Vi originated in the days when some of those assumptions might not be
true. Some early terminals on Unix systems didn't *have* cursor keys
or F-keys. The vi command set and separation between input and
command modes was a result.
Given where Toybox will be used, it's a reasonable assumption folks
running the included editor will already know enough of the basics to
deal with vi, even if they may prefer something else, so "Can they
figure out basic editing and saving without docs" is a non-issue.
Boxes already has other editors in it that do pass my test. So having
to deal with vi is a non-issue. Choice is a good thing, we have it.
B-)
Post by dmccunney
Post by David Seikel
killall -KILL vi
OK, yes I know about :qa as well, killall is much more satisfying.
B-)
You *could* simply remove vi from your system. :-)
While that's true, I can't do that to clients systems. I think one of
them actually likes vi.
Post by dmccunney
Post by David Seikel
Out of all the editors that I have implemented in boxes, I have used
them all professionally, including the line editors buried inside
emacs and vi. These days I prefer to use mc and mcedit for
everything though, which is why boxes includes mcedit. So I'll
dogfood mcedit, the rest I'll have to rely on others to tell me
what I get wrong.
We will. The big issue I can see is preserving "moded" editing in vi
and dispensing with it in the others.
Actually, other editors have modes to. So also not an issue, and
already solved.
Post by dmccunney
Post by David Seikel
I have no plans to implement TECO in boxes, vi and ed are already there.
C source is available for a port of TECO if anyone wants to build it,
but I don't see a need in Toybox. Stallman used ITS TECO as the
implementation language for Editing MACroS, the lineal ancestor of Gnu
Emacs, because it was what was available. It was a language in which
he could implement an editor. It took over as the standard editor at
MIT's AI labs, and he realized how successful his efforts had been
when he no longer remembered how to do things in raw TECO. We have
other languages in which you can write editors now.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rob Landley
2015-01-02 04:41:22 UTC
Permalink
Post by dmccunney
Post by David Seikel
I have a simple test to decide if I like an editor as a result of these
decades of random editor usage. If I can't sit down with the editor
and figure out how to do basic editing and saving in less than a
minute (sans documentation), then in my opinion it's a crap editor.
Both TECO and vi fail this test miserably, though oddly enough I have a
soft spot for TECO.
These days, the general assumption is that you can open a file in an
editor with "<editor> <filename>", and that once up, cursor keys can
be used to move around in the file and that text can be added where
desired by typing it at the cursor location and deleted with Backspace
or Delete keys.
Vi originated in the days when some of those assumptions might not be
true. Some early terminals on Unix systems didn't *have* cursor keys
or F-keys. The vi command set and separation between input and
command modes was a result.
Indeed.

However, ubuntu's decision to only allow you to cursor around in insert
mode when you call "vim" and to _disable_ that when you call it as "vi"
(so the cursor keys instead crap B[ and such all over your text) is
insane and stupid. And the fix is to delete /etc/vim/vimrc.tiny and make
it a symlink to just "vimrc" in the same directory. And the fact you
_need_ to do that on each new ubuntu install is just one more way that
Mark Shuttleworth is trying to cram his personal preferences down
people's throats.

(Redirecting /bin/sh to point to dash instead of bash was still a dumber
move, though.)
Post by dmccunney
Given where Toybox will be used, it's a reasonable assumption folks
running the included editor will already know enough of the basics to
deal with vi, even if they may prefer something else, so "Can they
figure out basic editing and saving without docs" is a non-issue.
Post by David Seikel
killall -KILL vi
OK, yes I know about :qa as well, killall is much more satisfying. B-)
You *could* simply remove vi from your system. :-)
Post by David Seikel
Out of all the editors that I have implemented in boxes, I have used
them all professionally, including the line editors buried inside emacs
and vi. These days I prefer to use mc and mcedit for everything though,
which is why boxes includes mcedit. So I'll dogfood mcedit, the rest
I'll have to rely on others to tell me what I get wrong.
We will. The big issue I can see is preserving "moded" editing in vi
and dispensing with it in the others.
The design I worked out back in 2006 was to have a line editor function
that handles cursor left/right and inserting, but returns to its caller
if you hit cursor up or cursor down (or page up...), and then a calling
function could stack lines for a text editor, or handle command history
instead if that's what the caller was implementing. (It gets fiddly when
"a line" can be multiple terminal lines because it's wrapping a long
line, but you could call the function with an argument telling it
whether or not to do that and have "I expanded to another line" or "I
shrank a line" be return codes so the caller can adjust stack views and
such".)

That way vi would have a line stack (possibly with last line's display
truncated because "you can show X vertical lines of content" could be an
argument when it's called), and then when you go into editing mode you
call it telling it to display on the bottom line and only one line of
text (scroll left/right as necessary to have cursor on screen). That
mode would similarly be what "less" uses, and "watch" would basically
use it in reverse (the line that isn't wrapped content would be at the
top instead of the bottom).

I was going to wait until actually _implementing_ it to fluff out the
design much more than that, because plans don't survive contact with
reality and infrastructure in search of a user is bad.

My plan was to start with a function to fetch a key given an array of
possible understood escape sequences and return the index in the array
of the escape sequence it saw, or the raw character if it was none of
that. (That way I could use it to reimplement menuconfig, interpreting
the cursor keys for me.) Then build the line editor function, and use
the line editor function to implement command history for toysh. Then
start on "watch" and "less" (stacking output in a display-only manner),
then work my way up to vi.

The boxes code wasn't quite layered the way I had in mind. (I tried to
chop out the "read one escape sequence or character" function, but could
never _find_ it as a separable entity.) I need to read through it more
and see what bits of it I can adapt to what I was thinking or if I need
to give up on what I was thinking, or what.

And that's why it's still on the todo list, because wrapping my head
around a design is not something I can do in 15 minute increments. It
takes me at least 15 minutes to swap back in "where did I leave off with
this" when I start poking at it again after a fairly short break, let
alone months.

But "sed" was a higer priority because when it comes to replacing
busybox in a self-bootstrapping development environment, it was far and
away the most frequently called remaining command toybox didn't
implement, the top ten being:

http://landley.net/notes-2013.html#13-08-2014

10 busybox dd
12 busybox gzip
46 busybox bzip2
81 busybox tar
85 busybox find
682 busybox diff
1041 busybox awk
1942 busybox sh
2024 busybox install
3202 busybox tr
16603 busybox expr
130099 busybox sed

(Also why it's so hard to debug. 130k different invocations to match the
output of, and sometimes when I fix one it causes regressions elsewhere...)
Post by dmccunney
Post by David Seikel
I have no plans to implement TECO in boxes, vi and ed are already there.
C source is available for a port of TECO if anyone wants to build it,
but I don't see a need in Toybox. Stallman used ITS TECO as the
implementation language for Editing MACroS, the lineal ancestor of Gnu
Emacs, because it was what was available. It was a language in which
he could implement an editor. It took over as the standard editor at
MIT's AI labs, and he realized how successful his efforts had been
when he no longer remembered how to do things in raw TECO. We have
other languages in which you can write editors now.
I've read rather a lot of history of vi/emacs/teco (some of it on
http://landley.net/history/mirror/ some not) and my rsponse to ALL of it
is the same as Vogons Grandmothers and Bugblatter Beasts: "In brief: avoid."

vi is interesting because it's ubiquitous and standardized by posix.
Microemacs is interesting because it's ubiquitous and Linus Torvalds'
personal editor. Last "intro to unix" class I taught at the local
community college (many moons ago) I did the textbook's chapter on vi
out of the syllabus I was given and then introduced everybody to nano.

I have todo items like "look up what keybindings eclipse uses" because
there's an entire generation of rebadged cobol developers filling out
forms and calling it programming for the fortune 500 using that. (Java's
flash in the pan popularity spike coincided with the "throw money at the
problem" stage of Y2K last minute panicking, so Java became the new
cobol and is now fossilized into enterprise procedure specification
certification buzzword compliant whatsises.)

If vi/uemacs/joe/nano are trivial extensions of the same basic
infrastructure (sort of true modulo vi command mode), I have no problem
implementing lots of sets of keybindings. But the first target is vi
because it's the only one actually in posix.

Rob
Rich Felker
2015-01-04 04:07:55 UTC
Permalink
Post by Rob Landley
Post by dmccunney
Post by David Seikel
I have a simple test to decide if I like an editor as a result of these
decades of random editor usage. If I can't sit down with the editor
and figure out how to do basic editing and saving in less than a
minute (sans documentation), then in my opinion it's a crap editor.
Both TECO and vi fail this test miserably, though oddly enough I have a
soft spot for TECO.
These days, the general assumption is that you can open a file in an
editor with "<editor> <filename>", and that once up, cursor keys can
be used to move around in the file and that text can be added where
desired by typing it at the cursor location and deleted with Backspace
or Delete keys.
Vi originated in the days when some of those assumptions might not be
true. Some early terminals on Unix systems didn't *have* cursor keys
or F-keys. The vi command set and separation between input and
command modes was a result.
Indeed.
However, ubuntu's decision to only allow you to cursor around in insert
mode when you call "vim" and to _disable_ that when you call it as "vi"
(so the cursor keys instead crap B[ and such all over your text) is
insane and stupid. And the fix is to delete /etc/vim/vimrc.tiny and make
it a symlink to just "vimrc" in the same directory. And the fact you
_need_ to do that on each new ubuntu install is just one more way that
Mark Shuttleworth is trying to cram his personal preferences down
people's throats.
Yes this is idiotic.
Post by Rob Landley
(Redirecting /bin/sh to point to dash instead of bash was still a dumber
move, though.)
I fail to see how this was dumb. It made shellshock a non-issue and
massively reduced the memory requirements (and probably increased the
speed) of portable shell scripts. Bash-specific scripts should always
be using #!/bin/bash.
Post by Rob Landley
If vi/uemacs/joe/nano are trivial extensions of the same basic
infrastructure (sort of true modulo vi command mode), I have no problem
implementing lots of sets of keybindings. But the first target is vi
because it's the only one actually in posix.
My guess is that this is not so easy, and that attempting to do it
this way would have a lot of subtle failures that would just annoy
users. But it might be a lot less annoying than being stuck with
nothing but vi...

Rich
David Seikel
2015-01-04 04:24:19 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
If vi/uemacs/joe/nano are trivial extensions of the same basic
infrastructure (sort of true modulo vi command mode), I have no
problem implementing lots of sets of keybindings. But the first
target is vi because it's the only one actually in posix.
My guess is that this is not so easy, and that attempting to do it
this way would have a lot of subtle failures that would just annoy
users. But it might be a lot less annoying than being stuck with
nothing but vi...
It was actually quite easy once I had done my homework. I don't see
how it "would have a lot of subtle failures that would just annoy
users." Though my design wasn't "start with vi and bend it to look
like other editors", it was "research all the targeted editors /
pagers, figure out what's common and what's not, then come up with a
generic infrastructure that can handle them all".
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rich Felker
2015-01-04 04:31:38 UTC
Permalink
Post by David Seikel
Post by Rich Felker
Post by Rob Landley
If vi/uemacs/joe/nano are trivial extensions of the same basic
infrastructure (sort of true modulo vi command mode), I have no
problem implementing lots of sets of keybindings. But the first
target is vi because it's the only one actually in posix.
My guess is that this is not so easy, and that attempting to do it
this way would have a lot of subtle failures that would just annoy
users. But it might be a lot less annoying than being stuck with
nothing but vi...
It was actually quite easy once I had done my homework. I don't see
how it "would have a lot of subtle failures that would just annoy
users."
The how/why is just past experience, e.g. joe -- jmacs feels nothing
like emacs and it's frustrating because your brain has to be in "emacs
mode" to use it, but then nothing actually works like emacs. The only
emacs workalike I've seen that actually fits my emacs workflow is mg,
and even then only until I have a non-ASCII file, in which case it
breaks horribly...
Post by David Seikel
Though my design wasn't "start with vi and bend it to look
like other editors", it was "research all the targeted editors /
pagers, figure out what's common and what's not, then come up with a
generic infrastructure that can handle them all".
That might work a lot better. I'd love to see it work out. I'm just
not holding my breath.. :-)

Rich
David Seikel
2015-01-04 04:49:16 UTC
Permalink
Post by Rich Felker
Post by David Seikel
Post by Rich Felker
Post by Rob Landley
If vi/uemacs/joe/nano are trivial extensions of the same basic
infrastructure (sort of true modulo vi command mode), I have no
problem implementing lots of sets of keybindings. But the first
target is vi because it's the only one actually in posix.
My guess is that this is not so easy, and that attempting to do it
this way would have a lot of subtle failures that would just annoy
users. But it might be a lot less annoying than being stuck with
nothing but vi...
It was actually quite easy once I had done my homework. I don't see
how it "would have a lot of subtle failures that would just annoy
users."
The how/why is just past experience, e.g. joe -- jmacs feels nothing
like emacs and it's frustrating because your brain has to be in "emacs
mode" to use it, but then nothing actually works like emacs. The only
emacs workalike I've seen that actually fits my emacs workflow is mg,
and even then only until I have a non-ASCII file, in which case it
breaks horribly...
Wellll, my research didn't include "how does this feel to experienced
users of these editors". Not actually being an emacs user these days,
I'll have to rely on people like you to report how you think the "feel"
might need adjusting.

Rob has mentioned a few times about Linus Torvalds maintaining his own
emacs fork, and how it would be great if the toybox emacs could convert
Linus. Certainly feedback from him be welcome as also.
Post by Rich Felker
Post by David Seikel
Though my design wasn't "start with vi and bend it to look
like other editors", it was "research all the targeted editors /
pagers, figure out what's common and what's not, then come up with a
generic infrastructure that can handle them all".
That might work a lot better. I'd love to see it work out. I'm just
not holding my breath.. :-)
I stopped holding my breath two years ago. Though with this fresh
discussion, there might be some movement on this soonish. In fact I
just had an idea I'll rush off to add to the boxes TODO right now.
--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
Rob Landley
2015-01-04 06:54:59 UTC
Permalink
Post by David Seikel
Post by Rich Felker
Post by David Seikel
Post by Rich Felker
Post by Rob Landley
If vi/uemacs/joe/nano are trivial extensions of the same basic
infrastructure (sort of true modulo vi command mode), I have no
problem implementing lots of sets of keybindings. But the first
target is vi because it's the only one actually in posix.
My guess is that this is not so easy, and that attempting to do it
this way would have a lot of subtle failures that would just annoy
users. But it might be a lot less annoying than being stuck with
nothing but vi...
It was actually quite easy once I had done my homework. I don't see
how it "would have a lot of subtle failures that would just annoy
users."
The how/why is just past experience, e.g. joe -- jmacs feels nothing
like emacs and it's frustrating because your brain has to be in "emacs
mode" to use it, but then nothing actually works like emacs. The only
emacs workalike I've seen that actually fits my emacs workflow is mg,
and even then only until I have a non-ASCII file, in which case it
breaks horribly...
Wellll, my research didn't include "how does this feel to experienced
users of these editors".
It's pretty important. Not having proper guinea pigs lined up _and_ a
way to do fast turnaround on their fixes is part of the reason this is
on the back burner.
Post by David Seikel
Not actually being an emacs user these days,
I'll have to rely on people like you to report how you think the "feel"
might need adjusting.
I remember when Wolfgang Denk, the maintainer of u-boot, tried
Aboriginal Linux a few years ago. He broke it 3 completely unique ways
in like the first 2 days, and when I fixed all that (adding things like
the environment variable whitelist and the check whether your host had
static libraries installed) he'd lost all interest in the project
because it didn't work for him and he had better things to do with his time.

His input was hugely valuable to me, but I couldn't sustain his interest.
Post by David Seikel
Rob has mentioned a few times about Linus Torvalds maintaining his own
emacs fork, and how it would be great if the toybox emacs could convert
Linus. Certainly feedback from him be welcome as also.
Hah. Not gonna happen.

He wrote code and I get to study that code and figure out WHY ever weird
little crease in it exists. Best of all he posted a repository with
individual commits and I get to look at each commit and try to work out
the _problem_ he solved when he did it.

Joel Spolsky wrote a marvelous piece about extracting implicit knowledge
from existing code almost 15 years ago:

http://www.joelonsoftware.com/articles/fog0000000069.html

In my case I'm mostly using other sources of knowledge to implement
posix commands (posix, man pages, almost 20 years of using these
commands, the giant pile of use case that is aboriginal linux
bootstrapping itself and building linux from scratch under it, my
history maintaining busybox...) but I'm aware how _much_ of it there is.

With making a comfortable microemacs UI... I think I may have to get
back in touch with Eric Raymond (who used to maintain the emacs lisp
library back in the 80's and is THE biggest emacs user I know) and see
if I can get him to use the result?

(This is complicated by the fact that gnu emacs is actually a lisp
application framework, it is to lisp what browsers are to javascript and
what Eric rally does is code in lisp, so the _editing_ functions are
secondary to him. That said, "this runs javascript" and "this is a
comfortable browser to use to read web pages" are sort of almost
seperable tasks maybe? Ok, it won't run google maps but is it rendering
HTML properly and does the back button work in a non-crazy manner?)

(Or see if Dalias can stomach using it for a week or two, and tell me
what's wrong and fix it. Possibly if Dalias and I wind up at CELF or
Ohio LinuxFest again and he can actually show me in person what's
screwed up...)

But again, this takes focus and is a can of worms that must be seen
through once opened. As I learned with the shell, if you let it sit long
enough sufficiently unfinished, you more or less have to redo it when
you come back. (Or maybe that's just me.)

Rob
Rob Landley
2015-01-04 07:39:21 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
Post by dmccunney
Post by David Seikel
I have a simple test to decide if I like an editor as a result of these
decades of random editor usage. If I can't sit down with the editor
and figure out how to do basic editing and saving in less than a
minute (sans documentation), then in my opinion it's a crap editor.
Both TECO and vi fail this test miserably, though oddly enough I have a
soft spot for TECO.
These days, the general assumption is that you can open a file in an
editor with "<editor> <filename>", and that once up, cursor keys can
be used to move around in the file and that text can be added where
desired by typing it at the cursor location and deleted with Backspace
or Delete keys.
Vi originated in the days when some of those assumptions might not be
true. Some early terminals on Unix systems didn't *have* cursor keys
or F-keys. The vi command set and separation between input and
command modes was a result.
Indeed.
However, ubuntu's decision to only allow you to cursor around in insert
mode when you call "vim" and to _disable_ that when you call it as "vi"
(so the cursor keys instead crap B[ and such all over your text) is
insane and stupid. And the fix is to delete /etc/vim/vimrc.tiny and make
it a symlink to just "vimrc" in the same directory. And the fact you
_need_ to do that on each new ubuntu install is just one more way that
Mark Shuttleworth is trying to cram his personal preferences down
people's throats.
Yes this is idiotic.
Post by Rob Landley
(Redirecting /bin/sh to point to dash instead of bash was still a dumber
move, though.)
I fail to see how this was dumb. It made shellshock a non-issue
When they switched to dash I _segfaulted_ the thing multiple times. It
was a buggy pile of crap for _years_.

It's interesting you think shellshock was a non-issue, presumably that's
why it didn't make the news or anything? (I'm consistently amused that
ESR wrote The Cathedral and the Bazaar pointing out that the Linux
Bazaar worked better than the FSF's Cathedral and then people assumed
that "free software" codebases benefitted the same way from "open
source" codebases despite the giant cultural gap. The variable export
bug was 20 years old when it was found. FSF code does not get reviewed.
OpenSSL did not get reviewed either. Contributions from outsiders were
not welcome, but everybody got complacent because _other_ pieces of open
source software were being thoroughly reviewed...)
Post by Rich Felker
and massively reduced the memory requirements (and probably increased the
speed) of portable shell scripts.
Linus Torvalds implemented unix system calls in his 386 assembly
terminal program so it would run bash. Specifically, the program bash.
(It's in "Just For Fun and early interviews".) This means bash was the
standard shell in Linux _before_ 0.0.1, and continued to be right up
until ubuntu suddenly changed it.

When ubuntu made the change, among other things it broke the kernel
build. Linux _itself_ wasn't saying #!/bin/bash (or running with a
non-bash #!/bin/sh), and nobody had ever actually _noticed_ before
shuttleworth decided he knew better because nobody was doing that.
That's how ubiquitous this was on Linux.

Portable linux shell scripts back then meant portable between
distributions, and bash was one of the few things linux distributions
agreed on. To the point LSB 1.2 (in 2002) had a page that boiled down to
"yeah, everybody actualy uses bash but we're trying to be a standards
document here":

http://refspecs.linuxfoundation.org/LSB_1.2.0/gLSB/stdshellrat.html

There were unix scripts ported from other operating systems, but linux
had always had bash as its default shell on all distributions up until
shuttleworth decided to change that. (If somebody like Jorg Schilling
was forced to use a linux box he'd change #!/bin/sh as part of his
personal customization while waiting for the cyanide to kick in, but it
was one of the few things all the linux distros agreed on.)
Post by Rich Felker
Bash-specific scripts should always
be using #!/bin/bash.
If his goal had been "we should be more standard", that would have been
one thing. But that explicitly _wasn't_ his goal. The part that makes
this not just epic hubris but REALLY POOR JUDGEMENT is that his stated
reasons for doing it were a goal it clearly failed to achieve:

https://wiki.ubuntu.com/DashAsBinSh

I.E. "Init scripts are too slow, but having the init scripts to say
#!/bin/dash is too hard, so let's change the global symlink that's been
consistent across distros for 20 years, that won't cause any problems!"

(And see how the above link said "the default login shell remains bash",
so it was never about _simplifying_ anything. Instead of removing code
from the system they adding a second shell in parallel so some stuff was
done with one and some with another, and made the system depend on BOTH
of them. This was not laudable, this was insane.)

I repeat: their explicit _reason_ for redirecting the #!/bin/sh symlink
because changing the individual init scripts to say #!/bin/dash was
considered too big and intrusive a change.

No really.

And then since this didn't accomplish its stated purpose (boots were
still too slow and single threaded with lots of blocking waits,
especially since they kept adding more crap to their boot scripts rather
than simplifying them), they implemented upstart thus rendering the
switch more or less moot for its original purpose, and never REVERTED
the /bin/sh change because that would have been admitting they made a
mistake.

THAT is why this is a sign that Ubuntu's technical judgement is on the
far side of questionable. (Even before attempting to do Unity and
Upstart and Mir and their pay cloud storage thing and their music
service and sending all your desktop searches to <strike>the
NSA</strike> amazon all at the same time.)

And that's without getting into dash itself's "not ready for prime time"
nature when they made the switch:

http://landley.net/notes-2007.html#17-11-2007
http://landley.net/notes-2007.html#22-10-2007
http://landley.net/notes-2007.html#08-05-2007

Heh, I forgot that when you forked a background process with & and then
hit ctrl-c, dash would kill the background process:

http://landley.net/notes-2007.html#21-03-2007
http://landley.net/notes-2007.html#18-03-2007

and so on...

Really, I feel strongly about this one. Ubuntu can screw up at a
technical level, and that was a big one. The scope of the change they
made and the results they expected to get out of it were utterly
disproportionate, and as a distro they crammed a lot of pain down their
users throats for no actual benefit.

Rob
Rich Felker
2015-01-04 11:43:19 UTC
Permalink
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
Post by dmccunney
Post by David Seikel
I have a simple test to decide if I like an editor as a result of these
decades of random editor usage. If I can't sit down with the editor
and figure out how to do basic editing and saving in less than a
minute (sans documentation), then in my opinion it's a crap editor.
Both TECO and vi fail this test miserably, though oddly enough I have a
soft spot for TECO.
These days, the general assumption is that you can open a file in an
editor with "<editor> <filename>", and that once up, cursor keys can
be used to move around in the file and that text can be added where
desired by typing it at the cursor location and deleted with Backspace
or Delete keys.
Vi originated in the days when some of those assumptions might not be
true. Some early terminals on Unix systems didn't *have* cursor keys
or F-keys. The vi command set and separation between input and
command modes was a result.
Indeed.
However, ubuntu's decision to only allow you to cursor around in insert
mode when you call "vim" and to _disable_ that when you call it as "vi"
(so the cursor keys instead crap B[ and such all over your text) is
insane and stupid. And the fix is to delete /etc/vim/vimrc.tiny and make
it a symlink to just "vimrc" in the same directory. And the fact you
_need_ to do that on each new ubuntu install is just one more way that
Mark Shuttleworth is trying to cram his personal preferences down
people's throats.
Yes this is idiotic.
Post by Rob Landley
(Redirecting /bin/sh to point to dash instead of bash was still a dumber
move, though.)
I fail to see how this was dumb. It made shellshock a non-issue
When they switched to dash I _segfaulted_ the thing multiple times. It
was a buggy pile of crap for _years_.
It's interesting you think shellshock was a non-issue, presumably that's
why it didn't make the news or anything? (I'm consistently amused that
I don't think shellshock was a non-issue. I just think it was mostly a
non-issue for systems that don't use bash as /bin/sh or their login
shell for restricted accounts (forced command in authorized_keys
file), and this is one reason I think it was a good move not to make
/bin/sh be a link to bash.

Yes some people write crappy scripts that depend on bash but don't use
#!/bin/bash. I don't and I don't like paying the penalty for people
who do this. They should fix their broken scripts, and preferably they
should write portable shell script instead of bash. I suspect you'll
be more agreeable to this position once more bash-specific scripts
start depending on post-GPLv3 versions of bash... :-)

Rich
Rob Landley
2015-01-04 18:39:25 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
Post by dmccunney
Post by David Seikel
I have a simple test to decide if I like an editor as a result of these
decades of random editor usage. If I can't sit down with the editor
and figure out how to do basic editing and saving in less than a
minute (sans documentation), then in my opinion it's a crap editor.
Both TECO and vi fail this test miserably, though oddly enough I have a
soft spot for TECO.
These days, the general assumption is that you can open a file in an
editor with "<editor> <filename>", and that once up, cursor keys can
be used to move around in the file and that text can be added where
desired by typing it at the cursor location and deleted with Backspace
or Delete keys.
Vi originated in the days when some of those assumptions might not be
true. Some early terminals on Unix systems didn't *have* cursor keys
or F-keys. The vi command set and separation between input and
command modes was a result.
Indeed.
However, ubuntu's decision to only allow you to cursor around in insert
mode when you call "vim" and to _disable_ that when you call it as "vi"
(so the cursor keys instead crap B[ and such all over your text) is
insane and stupid. And the fix is to delete /etc/vim/vimrc.tiny and make
it a symlink to just "vimrc" in the same directory. And the fact you
_need_ to do that on each new ubuntu install is just one more way that
Mark Shuttleworth is trying to cram his personal preferences down
people's throats.
Yes this is idiotic.
Post by Rob Landley
(Redirecting /bin/sh to point to dash instead of bash was still a dumber
move, though.)
I fail to see how this was dumb. It made shellshock a non-issue
When they switched to dash I _segfaulted_ the thing multiple times. It
was a buggy pile of crap for _years_.
It's interesting you think shellshock was a non-issue, presumably that's
why it didn't make the news or anything? (I'm consistently amused that
I don't think shellshock was a non-issue. I just think it was mostly a
non-issue for systems that don't use bash as /bin/sh or their login
shell for restricted accounts (forced command in authorized_keys
file), and this is one reason I think it was a good move not to make
/bin/sh be a link to bash.
Yes some people write crappy scripts that depend on bash but don't use
#!/bin/bash. I don't and I don't like paying the penalty for people
who do this. They should fix their broken scripts, and preferably they
should write portable shell script instead of bash. I suspect you'll
be more agreeable to this position once more bash-specific scripts
start depending on post-GPLv3 versions of bash... :-)
No, I'm implementing the non-crazy bash extensions people actually use
in toysh. (Toybox should eventually be able to build itself including
its own "sed" invocations and its own shell to run scripts/*.sh.) This
is why I went to the "advanced bash scripting" talk at Texas LinuxFest,
to see what tricks that speaker felt were worth knowing/using (and thus
implementing).

Coming up with a bash replacement, yay. But dash didn't even bother to
implement "source" as an alias for ".", or implement the "function"
keyword (it's basically a comment). Those were trivial bash flavoring
things that would be like 5 minutes coding, and they explicitly chose
_not_ to do that because they did not _want_ compatibility. (Stuff like
implementing path/to/{one,two,three} curly bracket support would be more
like a weekend of coding, but still: that not being there broke stuff.)

I repeat: ubuntu made a bad technical decision, gratuitously breaking
compatibility for its existing userbase for a stated reason that was
inadequate to justify the fallout, one which could easily have been
accomplished a different way without end user impact, and fairly quickly
_was_ because it didn't accomplish its stated goal but the change was
retained anyway.

Rob
Rich Felker
2015-01-04 19:04:55 UTC
Permalink
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
(Redirecting /bin/sh to point to dash instead of bash was still a dumber
move, though.)
I fail to see how this was dumb. It made shellshock a non-issue
When they switched to dash I _segfaulted_ the thing multiple times. It
was a buggy pile of crap for _years_.
It's interesting you think shellshock was a non-issue, presumably that's
why it didn't make the news or anything? (I'm consistently amused that
I don't think shellshock was a non-issue. I just think it was mostly a
non-issue for systems that don't use bash as /bin/sh or their login
shell for restricted accounts (forced command in authorized_keys
file), and this is one reason I think it was a good move not to make
/bin/sh be a link to bash.
Yes some people write crappy scripts that depend on bash but don't use
#!/bin/bash. I don't and I don't like paying the penalty for people
who do this. They should fix their broken scripts, and preferably they
should write portable shell script instead of bash. I suspect you'll
be more agreeable to this position once more bash-specific scripts
start depending on post-GPLv3 versions of bash... :-)
No, I'm implementing the non-crazy bash extensions people actually use
in toysh. (Toybox should eventually be able to build itself including
its own "sed" invocations and its own shell to run scripts/*.sh.) This
is why I went to the "advanced bash scripting" talk at Texas LinuxFest,
to see what tricks that speaker felt were worth knowing/using (and thus
implementing).
Nothing wrong with that; if it helps eliminate bash from more systems
I'm quite happy to see it happen. But...
Post by Rob Landley
Coming up with a bash replacement, yay. But dash didn't even bother to
implement "source" as an alias for ".", or implement the "function"
keyword (it's basically a comment). Those were trivial bash flavoring
things that would be like 5 minutes coding, and they explicitly chose
_not_ to do that because they did not _want_ compatibility.
I see those things as roughly equivalent to <sys/poll.h> and
<sys/fcntl.h> in musl -- or maybe even <sys/cdefs.h> which musl
doesn't have. These are not useful features but coddling of spelling
mistakes made in non-portable code. And it's a matter of opinion,
where both sides have valid points, whether it's better to pressure
people to fix those spelling mistakes or simply work around them.
Post by Rob Landley
(Stuff like
implementing path/to/{one,two,three} curly bracket support would be more
like a weekend of coding, but still: that not being there broke stuff.)
Additional features like that also cause incorrect behavior in
non-bash scripts unless they're off-by-default.
Post by Rob Landley
I repeat: ubuntu made a bad technical decision, gratuitously breaking
compatibility for its existing userbase for a stated reason that was
inadequate to justify the fallout, one which could easily have been
accomplished a different way without end user impact, and fairly quickly
_was_ because it didn't accomplish its stated goal but the change was
retained anyway.
I think the security and runtime-cost benefits were more than
sufficient to justify the "fallout". The broken scripts are easily
remedied just by fixing the #! line, or they can be made portable if
that was the intent to begin with.

If there were cases where dash really was technically worse than bash
(like crashing, memleaks, etc.) then that's another matter, but I
think they just indicate that the switch was made prematurely rather
than that it was wrong in concept.

Rich
Rob Landley
2015-01-05 00:20:05 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
Coming up with a bash replacement, yay. But dash didn't even bother to
implement "source" as an alias for ".", or implement the "function"
keyword (it's basically a comment). Those were trivial bash flavoring
things that would be like 5 minutes coding, and they explicitly chose
_not_ to do that because they did not _want_ compatibility.
I see those things as roughly equivalent to <sys/poll.h> and
<sys/fcntl.h> in musl -- or maybe even <sys/cdefs.h> which musl
doesn't have.
http://landley.net/hg/aboriginal/rev/540

I want stuff to _work_. Throw warnings sure, but if something is
actually _using_ it...
Post by Rich Felker
These are not useful features but coddling of spelling
mistakes made in non-portable code. And it's a matter of opinion,
where both sides have valid points, whether it's better to pressure
people to fix those spelling mistakes or simply work around them.
Indeed. I come down on the side of having a dos box you can confine old
code to so it still works, and have -Wall to let you fix new stuff.

You've seen the lengths I went to in aboriginal to do host/path and
more/record-commands.sh and forensically determine exactly what was
going _on_ in the builds and the specific requirements down to the bare
metal. Heck, I worked out the minimal kernel .config that had the system
calls I needed.

But then what I _ship_ includes linux from scratch chroot tarballs
because other people reproducing my minimal environment is science, but
other people starting with a lfs tarball is engineering.
Post by Rich Felker
Post by Rob Landley
(Stuff like
implementing path/to/{one,two,three} curly bracket support would be more
like a weekend of coding, but still: that not being there broke stuff.)
Additional features like that also cause incorrect behavior in
non-bash scripts unless they're off-by-default.
Except that on Linux, "non-bash scripts" were almost nonexistent before
2006 (ported from other OSes and you know where to find ksh or zsh if it
says #!/bin/zsh), and even since then it's still a minority and not very
_interesting_.

I view posix sh -> bash much the way I System V -> Linux. (System V
didn't have procfs. I'm writing stuff that uses procfs. There are
compatability modes that implement that in BSD, and limiting yourself
to /etc/mtab not being a symlink to /proc/mounts means you're not
dealing with reality on modern system. You _can't_ get that right
entirely in userspace these days.)
Post by Rich Felker
Post by Rob Landley
I repeat: ubuntu made a bad technical decision, gratuitously breaking
compatibility for its existing userbase for a stated reason that was
inadequate to justify the fallout, one which could easily have been
accomplished a different way without end user impact, and fairly quickly
_was_ because it didn't accomplish its stated goal but the change was
retained anyway.
I think the security and runtime-cost benefits were more than
sufficient to justify the "fallout".
I see no benefits.
Post by Rich Felker
The broken scripts are easily
remedied just by fixing the #! line, or they can be made portable if
that was the intent to begin with.
Every script I know changed the #! line, but that was a bug workaround
for ubuntu breaking the /bin/sh symlink.
Post by Rich Felker
If there were cases where dash really was technically worse than bash
(like crashing, memleaks, etc.) then that's another matter, but I
think they just indicate that the switch was made prematurely rather
than that it was wrong in concept.
This was not an either/or situation, it was both.
Post by Rich Felker
Rich
Rob
Rich Felker
2015-01-05 00:31:59 UTC
Permalink
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
Coming up with a bash replacement, yay. But dash didn't even bother to
implement "source" as an alias for ".", or implement the "function"
keyword (it's basically a comment). Those were trivial bash flavoring
things that would be like 5 minutes coding, and they explicitly chose
_not_ to do that because they did not _want_ compatibility.
I see those things as roughly equivalent to <sys/poll.h> and
<sys/fcntl.h> in musl -- or maybe even <sys/cdefs.h> which musl
doesn't have.
http://landley.net/hg/aboriginal/rev/540
I want stuff to _work_. Throw warnings sure, but if something is
actually _using_ it...
Post by Rich Felker
These are not useful features but coddling of spelling
mistakes made in non-portable code. And it's a matter of opinion,
where both sides have valid points, whether it's better to pressure
people to fix those spelling mistakes or simply work around them.
Indeed. I come down on the side of having a dos box you can confine old
code to so it still works, and have -Wall to let you fix new stuff.
You've seen the lengths I went to in aboriginal to do host/path and
more/record-commands.sh and forensically determine exactly what was
going _on_ in the builds and the specific requirements down to the bare
metal. Heck, I worked out the minimal kernel .config that had the system
calls I needed.
But then what I _ship_ includes linux from scratch chroot tarballs
because other people reproducing my minimal environment is science, but
other people starting with a lfs tarball is engineering.
Post by Rich Felker
Post by Rob Landley
(Stuff like
implementing path/to/{one,two,three} curly bracket support would be more
like a weekend of coding, but still: that not being there broke stuff.)
Additional features like that also cause incorrect behavior in
non-bash scripts unless they're off-by-default.
Except that on Linux, "non-bash scripts" were almost nonexistent before
2006 (ported from other OSes and you know where to find ksh or zsh if it
says #!/bin/zsh), and even since then it's still a minority and not very
_interesting_.
Every single practical configure script (autoconf generated or written
by hand) is a non-bash script. That's quite a huge portion of shell
scripts. And they've been around since the beginning on Linux.
Post by Rob Landley
I view posix sh -> bash much the way I System V -> Linux. (System V
didn't have procfs. I'm writing stuff that uses procfs. There are
compatability modes that implement that in BSD, and limiting yourself
to /etc/mtab not being a symlink to /proc/mounts means you're not
dealing with reality on modern system. You _can't_ get that right
entirely in userspace these days.)
These are all implementation details that matter only to a few
boot-time utilities and such. They're all irrelevant to applications.
It's stupid and inappropriate for applications to assume procfs or
mtab or even the concept of mounts, which won't exist on future
systems when Linux is considered a dinosaur... This is why the
standards omit them. Standards are about what an application can
expect to see, not how it gets done under the hood. They're about
moving us forward into compatibility and interoperability, not
backwards into lock-in to outdated implementation details.
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
I repeat: ubuntu made a bad technical decision, gratuitously breaking
compatibility for its existing userbase for a stated reason that was
inadequate to justify the fallout, one which could easily have been
accomplished a different way without end user impact, and fairly quickly
_was_ because it didn't accomplish its stated goal but the change was
retained anyway.
I think the security and runtime-cost benefits were more than
sufficient to justify the "fallout".
I see no benefits.
2x VSZRW, 5x actual ram usage (dirty pages), per instance.

And anecdotally (I don't have figures) performance is considerably
better.

And of course, most importantly, complete lack of function exports and
other dangerous code paths processing potentially untrusted data that
should never have existed in the first place.
Post by Rob Landley
Post by Rich Felker
The broken scripts are easily
remedied just by fixing the #! line, or they can be made portable if
that was the intent to begin with.
Every script I know changed the #! line, but that was a bug workaround
for ubuntu breaking the /bin/sh symlink.
Every script you know? No, some small set of bash scripts that were
wrongly labeled as sh scripts. The vast majority of the scripts you
know are configure scripts and most of them even run on ancient
pre-POSIX shells.

Rich
Roy Tam
2015-01-05 00:40:48 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
Coming up with a bash replacement, yay. But dash didn't even bother to
implement "source" as an alias for ".", or implement the "function"
keyword (it's basically a comment). Those were trivial bash flavoring
things that would be like 5 minutes coding, and they explicitly chose
_not_ to do that because they did not _want_ compatibility.
I see those things as roughly equivalent to <sys/poll.h> and
<sys/fcntl.h> in musl -- or maybe even <sys/cdefs.h> which musl
doesn't have.
http://landley.net/hg/aboriginal/rev/540
I want stuff to _work_. Throw warnings sure, but if something is
actually _using_ it...
Post by Rich Felker
These are not useful features but coddling of spelling
mistakes made in non-portable code. And it's a matter of opinion,
where both sides have valid points, whether it's better to pressure
people to fix those spelling mistakes or simply work around them.
Indeed. I come down on the side of having a dos box you can confine old
code to so it still works, and have -Wall to let you fix new stuff.
You've seen the lengths I went to in aboriginal to do host/path and
more/record-commands.sh and forensically determine exactly what was
going _on_ in the builds and the specific requirements down to the bare
metal. Heck, I worked out the minimal kernel .config that had the system
calls I needed.
But then what I _ship_ includes linux from scratch chroot tarballs
because other people reproducing my minimal environment is science, but
other people starting with a lfs tarball is engineering.
Post by Rich Felker
Post by Rob Landley
(Stuff like
implementing path/to/{one,two,three} curly bracket support would be more
like a weekend of coding, but still: that not being there broke stuff.)
Additional features like that also cause incorrect behavior in
non-bash scripts unless they're off-by-default.
Except that on Linux, "non-bash scripts" were almost nonexistent before
2006 (ported from other OSes and you know where to find ksh or zsh if it
says #!/bin/zsh), and even since then it's still a minority and not very
_interesting_.
Every single practical configure script (autoconf generated or written
by hand) is a non-bash script. That's quite a huge portion of shell
scripts. And they've been around since the beginning on Linux.
Post by Rob Landley
I view posix sh -> bash much the way I System V -> Linux. (System V
didn't have procfs. I'm writing stuff that uses procfs. There are
compatability modes that implement that in BSD, and limiting yourself
to /etc/mtab not being a symlink to /proc/mounts means you're not
dealing with reality on modern system. You _can't_ get that right
entirely in userspace these days.)
These are all implementation details that matter only to a few
boot-time utilities and such. They're all irrelevant to applications.
It's stupid and inappropriate for applications to assume procfs or
mtab or even the concept of mounts, which won't exist on future
systems when Linux is considered a dinosaur... This is why the
standards omit them. Standards are about what an application can
expect to see, not how it gets done under the hood. They're about
moving us forward into compatibility and interoperability, not
backwards into lock-in to outdated implementation details.
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
I repeat: ubuntu made a bad technical decision, gratuitously breaking
compatibility for its existing userbase for a stated reason that was
inadequate to justify the fallout, one which could easily have been
accomplished a different way without end user impact, and fairly quickly
_was_ because it didn't accomplish its stated goal but the change was
retained anyway.
I think the security and runtime-cost benefits were more than
sufficient to justify the "fallout".
I see no benefits.
2x VSZRW, 5x actual ram usage (dirty pages), per instance.
And anecdotally (I don't have figures) performance is considerably
better.
And of course, most importantly, complete lack of function exports and
other dangerous code paths processing potentially untrusted data that
should never have existed in the first place.
Post by Rob Landley
Post by Rich Felker
The broken scripts are easily
remedied just by fixing the #! line, or they can be made portable if
that was the intent to begin with.
Every script I know changed the #! line, but that was a bug workaround
for ubuntu breaking the /bin/sh symlink.
Every script you know? No, some small set of bash scripts that were
wrongly labeled as sh scripts. The vast majority of the scripts you
know are configure scripts and most of them even run on ancient
pre-POSIX shells.
IIRC busybox sh even has some bash extensions. But form my point of
view, using mksh(Android default shell since ICS) seems to be better
since Thorsten Glaser keeps fixing bugs.
Post by Rich Felker
Rich
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Rob Landley
2015-01-05 13:34:40 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
Except that on Linux, "non-bash scripts" were almost nonexistent before
2006 (ported from other OSes and you know where to find ksh or zsh if it
says #!/bin/zsh), and even since then it's still a minority and not very
_interesting_.
Every single practical configure script (autoconf generated or written
by hand) is a non-bash script. That's quite a huge portion of shell
scripts. And they've been around since the beginning on Linux.
You're using autoconf as a positive example to _support_ your argument?
Post by Rich Felker
Post by Rob Landley
I view posix sh -> bash much the way I System V -> Linux. (System V
didn't have procfs. I'm writing stuff that uses procfs. There are
compatability modes that implement that in BSD, and limiting yourself
to /etc/mtab not being a symlink to /proc/mounts means you're not
dealing with reality on modern system. You _can't_ get that right
entirely in userspace these days.)
These are all implementation details that matter only to a few
boot-time utilities and such. They're all irrelevant to applications.
It's stupid and inappropriate for applications to assume procfs or
mtab or even the concept of mounts, which won't exist on future
systems when Linux is considered a dinosaur...
A "best practices" gui application from 20 years ago would either be
using raw xlib calls or a widget toolkit like motif. It would be IPv4
only, couldn't have supported UTF-8, any encryption or hash algorithm
would be _hilarious_ before you even got into export controls, it would
be written in c89 for single processor 32 bit systems. If you're lucky
it might have understood http 1.0 instead of gopher, and implement the
html "blink" or "marquee" tags (but would predate css or javascript). It
would still predate mp3 by a few months, could expect 800x600 screen
resolution at 256 colors (but should scale to 1024x768 with 65k colors
at the high end), and let's just ignore fonts entirely for the sake of
sanity... As or compression algorithms, pkzip 2.0 introduced deflate in
1993 so it's at least be possible instead of one of the dozen other now
largely forgotten variants, but the deflate rfc wasn't for another year.
This means no png (gif was 256 colors)...

You're implying there's a way to write code that's so good nobody would
ever need to maintain or port it again. Your example would be what,
xmms? Or are you saying nobody's ever managed this before but we should
start now?

Back then I personally was doing OS/2 code, because I didn't see the
point of Sun workstations. (Which were themselves switching from SunOS
to Solaris while I was poking at them.) I'm not saying Linux is eternal,
I'm saying portability to the unknown is another variant of
"infrastructure in search of a user". It's not hard to write new code in
future, and if an existing codebase of nontrivial size isn't already
actively maintained nobody's going to inherit OpenWatcom and go "yes,
that, I'll learn it and make it mine" rather than writing a new one,
because reading code isn't going to _stop_ being harder than writing
code any time soon. (Yes Erik Anderson did repurpose busybox and uClibc
instead of starting over, but A) he was unusual, B) each of those
codebases had only _existed_ for about 5 years when he started.)
Post by Rich Felker
This is why the standards omit them.
Clearly IBM paying to have OS/360 be posix compliant and Microsoft
paying to have NT be certified posix complaint had nothing to do with
this subset selection.
Post by Rich Felker
Standards are about what an application can
expect to see, not how it gets done under the hood.
Posix removed cpio after 2001, and never standardized the 8 digit
(rather than 6 digit) variant of it used by rpm and initramfs. Posix
standardized pax (which almost nobody uses outside Sun), but not tar
(which is almost ubiquitous outside of windows, which uses zip, which is
not mentioned in posix. There's an IETF RFC on zip, but there are over
7000 other RFCs most of which are useless).

Sure, you can mix and match various standards bodies to describe what
you're doing. Or, in the case of large corporations, sponsor a standards
body to certify your description of what you're already doing. Nobody
ever games that:

http://en.wikipedia.org/wiki/Standardization_of_Office_Open_XML

I remember the _year_ a company I once worked at spent trying to
implement a realplayer compatible server using the RTSP "standard"
they'd put forth:

http://www.ietf.org/rfc/rfc2326.txt

(Spoiler: it couldn't be done. There was a required encoding layer,
somewhere between encryption and pure obfuscation, that wasn't
documented but was totally required to actually talk to their servers or
clients with the protocol they'd published as a "standard".)
Post by Rich Felker
They're about
moving us forward into compatibility and interoperability, not
backwards into lock-in to outdated implementation details.
You're trying to predict the future in a way that will prevent you from
having to respond to it. I'm expecting to have to constantly change to
adapt to the future as it arrives.

I throw out waaay more code than I ever ship. The stuff I ship is the
best I can do _currently_, but I just ripped out and redid a chunk of
the flag parsing logic yet again because I needed it to do something
else, and what took me so long is I didn't want to add another layer on
top but figure out how to _remove_ old code that was no longer relevant
(I.E. generated/oldtoys.h).

If circumstances change in future to render current code obsolete, than
I (or somebody else) will write new code. This is how it works. I'm not
that tied to the existing implementation of _anything_, and that
includes the current version of the standards documents. (It wasn't
_that_ long ago I was looking into chucking the existing codebase and
rewriting it in LUA. That turned out not to be viable, but "written in
C" is a pretty fundamental assumption for a project like toybox and I
gave it a go of not doing that _after_ making patch.c actually work with
real-world data despite posix still to this day not having the concept
of a unified diff.)
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
Post by Rob Landley
I repeat: ubuntu made a bad technical decision, gratuitously breaking
compatibility for its existing userbase for a stated reason that was
inadequate to justify the fallout, one which could easily have been
accomplished a different way without end user impact, and fairly quickly
_was_ because it didn't accomplish its stated goal but the change was
retained anyway.
I think the security and runtime-cost benefits were more than
sufficient to justify the "fallout".
I see no benefits.
2x VSZRW, 5x actual ram usage (dirty pages), per instance.
And anecdotally (I don't have figures) performance is considerably
better.
There's a variant of Gates' Law that applies specifically to the FSF.
Anything they maintain rapidly turns into a pile of crap, they have a
hugely corrosive effect on any software they're involved with and this
is nothing new.

But: when you inevitably write a new one (the way busybox and android
and klibc did, or the various "yet another blah" or
"somethingorother-ng" projects), you target compatability with what
people are using, not an ideal nobody currently actually conforms to.

We didn't get chromium and firefox from people trying to implement
standards documents. We got them from people throwing ever more
real-world data at their programs and iteratively fixing what didn't
work (and then going back and submitting changes to the standards
documents to make them less sad).

That's not what dash did, running real-world scripts and fixing their
shell. They shipped a shell based on some idealistic vision in a
standards document and broke everybody's projects because they were big
enough to throw their weight around. Just like Microsoft used to be able
to do.

Trying to shove dash down people's throats was the same impulse as
making "vi" and "vim" behave differently, or unity as a common desktop
between phones and laptops. It was Mark Shuttleworth imposing policy on
his userbase.
Post by Rich Felker
And of course, most importantly, complete lack of function exports and
other dangerous code paths processing potentially untrusted data that
should never have existed in the first place.
So you're saying they predicted heartbleed 8 years in advance, and
that's why they shipped something you could trivially segfault that
didn't get basic signal handling right and which assumed that if you
didn't have a controlling terminal you were never in interactive mode
and thus shouldn't get a prompt? (On _top_ of "hey, you broke the kernel
build, this is news to the kernel development community, but it's not
like "compiling linux" is an activity common enough to linux systems
that people would have noticed it breaking over the previous 15 years...)
Post by Rich Felker
Post by Rob Landley
Post by Rich Felker
The broken scripts are easily
remedied just by fixing the #! line, or they can be made portable if
that was the intent to begin with.
Every script I know changed the #! line, but that was a bug workaround
for ubuntu breaking the /bin/sh symlink.
Every script you know? No, some small set of bash scripts that were
wrongly labeled as sh scripts. The vast majority of the scripts you
know
Thanks for telling me what I know. It's good that you're more of an
authority on it than I am. I had previously been unaware of that.
Post by Rich Felker
are configure scripts and most of them even run on ancient
pre-POSIX shells.
The output of autoconf is only coincidentally a shell script. It could
just as easily have been C.

The largest non-autoconf configure script I have lying around at the
moment is the one in qemu. I just checked out the last version from 2006
(git 388c45084d54), back before ubuntu broke #!/bin/sh, and tried to run
it on current ubuntu:

$ ./configure
WARNING: "gcc" looks like gcc 4.x
Looking for gcc 3.x
./configure: 358: ./configure: Syntax error: Bad fd number
***@driftwood:~/qemu/qemu$

So:

A) hardwired gcc 3.x dependency that they made go away by rewriting
their entire code generator (replacing dyngen with tcg) because you can
do that: writing new code in future is always an option.

B) the "bad fd number" was because despite saying #!/bin/sh at the top
this script had only ever been tested with bash because THAT IS MY POINT.

C) these guys care about portability to _windows_ enough to use ptrtype
macros for the insane win64 LLP64 stuff. (Not running windows clients,
running on windows _hosts_. Why? Because IBM, git cc3ac9c4a6fd for example.)
Post by Rich Felker
Rich
Rob
Daniel Cegiełka
2015-01-04 12:10:42 UTC
Permalink
Post by Rich Felker
Post by Rob Landley
I plan to implement vi over the next year, but it's one of the four
realy big commands required by posix (sed, awk, sh, vi) and I've been
debugging sed against real-world data for _weeks_ now. (It's easy to
knock out a simple 90% implementation. It's really hard to make
something do everything right in all the cases people are going to throw
at it.)
It would be awesome to have mg too or a similar mg-like emacs clone
but with working unicode support.
http://git.kernel.org/cgit/editors/uemacs/uemacs.git/tree/

I have a small patch, because uemacs freezes when ctrl-s.
but the license... :/
Post by Rich Felker
Rich
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Daniel Cegiełka
2015-01-04 12:58:33 UTC
Permalink
uemacs version with small fixes - does not freeze when ctrl-s (sorry
for packaging... f*** gmail).

but:

1) uemacs isn't properly compatible with emacs
2) http://git.kernel.org/cgit/editors/uemacs/uemacs.git/tree/README

Original statement of copying policy:

MicroEMACS 3.9 can be copied and distributed freely for any
non-commercial purposes. MicroEMACS 3.9 can only be incorporated
into commercial software with the permission of the current author
[Daniel M. Lawrence].

3) http://www.clapperfuneralservices.com/memsol.cgi?user_id=212445

In my opinion it is better to invest in mg (support for utf-8) eg:

http://git.kernel.org/cgit/editors/uemacs/uemacs.git/log/?qt=grep&q=utf

Daniel

Rich Felker
2014-12-22 00:41:19 UTC
Permalink
Post by James McMechan
I am would suggest something like GNU_QUIRKS and POSIX_QUIRKS config
symbols to add in the odd features like gnu sed directly disagreeing
with POSIX or POSIX's umm interesting xargs without -0 quoting
conventions and any other "why did they do it that way" I know POSIX
There is nothing unusual or quirky about the way xargs without -0
works. It's the way it's always been on all historical systems, and
it's the only way xargs can actually process arbitrary lists of
filenames. Mistakenly treating the input as a newline-delimited list
of literals rather than a whitespace-delimited list of shell-quoted
strings is a toybox quirk that's a bug and just needs to be fixed.

Rich
James McMechan
2015-01-02 03:39:32 UTC
Permalink
Post by Rob Landley
Post by Isaac Dunham
Post by Rob Landley
I've used netcat -f /dev/ttyUSB0 as a terminal, the netcat help points

   Use "stty 115200 -F /dev/ttyS0 && stty raw -echo -ctlecho" with
   netcat -f to connect to a serial port.

On my todo list is gluing the two of those together into a nanoterm but
I need to cleanup and promote stty first and that command is... complicated.

Um, correction. Looks like I need to implement it. (Not in pending.
might be in ashwini's patch pile...)

I haven't seen an stty patch (that was actually my complaint about reset).
stty isn't hard, it's just long.
Lemme rephrase that: stty has 8 gazillion config options that aren't
always obvious how to implement.
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/stty.html
I thought it seemed fairly straight forward.
Tedious if you want the whole set but straight forward, since a lot of it can be implementation defined
if you do something mutually exclusive the results are unspecified. a bunch of the commands include if possible.  The reason the current version in busybox is such a mess is poor organization and an ifdef forest to support a bunch of very old OSes that were missing parts of the standard.
9 control character settings
35 toggles
+7 one of severals (cs5 cs6 cs7 cs8) for 18 symbols
and the combo modes that are combos of the above

only two required options
-a : output all the current settings
-g : output all the current settings in unspecified format acceptable as input to stty and not requiring shell escapes
Post by Rob Landley
The entire historical unix layer is one of those things that evolved
from teletype machines with 6 bit serial interfaces driving a daisy
wheel (_5_ bit serial back in the 1930's, they had a shift code between
http://www.textfiles.com/bitsavers/pdf/teletype/144_Model15_Descr_Feb31.pdf
).
Hey, I always liked the model 15 it was built like a brick, and they built them for over 50 years, how many other pieces of complex high tech gear will be built in the same model for that long ;)

The 5-level code is usually called Baudot after the inventor of the first version in 1874 is still in use today, still at 60wpm or 45.45 baud (22ms/bit), after all on amateur radio the FCC did not even permit ASCII until 1980.
The code evolved from 5 keys played like a piano with a code chosen to minimize operator fatigue through electro mechanical version where the bit patterns were chosen to minimize wear in sort of a combination of Morse code and a typewriter.  The reason it is called a shift code is that it literally shifted the carriage up and down just like in a mechanical typewriter.

The daisy-wheel was used in some of the first experimental systems but most of the teletypes were more like a mechanical typewriter. Some of the last models made were more like the IBM Selectric typewriter. The 1.42 stop bits were used to resynchronize the clutch, in the mechanical serial to parallel converter.  The original daisy-wheels tended to use synchronous motors and getting everything synced up was hard.  Asynchronous was tolerant of minor variations in equipment timing and still worked.

The only six-level code I know of was for the Lin-o-type machines in a teletypesetter. Where lowercase letters, ligatures like fi, fl, oe and quadding (blank space on left, right or both sides) and mechanical automatic justification was required and used in newspapers to cast the lead slugs of a entire line of type.
Post by Rob Landley
For most of toybox all I really care about is "raw mode vs cooked mode",
but stty has to flip all the switches individually...
And if you don't require it to succeed at processing all those other options then that looks very simple, load one set of constants or the other and call tcsetattr
Post by Rob Landley
It's on the todo list.
Post by Isaac Dunham
HTH,
Isaac Dunham
Rob
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Loading...