Discussion:
[PATCH] Seperate 'userdel' from testing syntax
(too old to reply)
Yeongdeok Suh
2015-01-30 07:56:14 UTC
Permalink
Hi, all,

When I test toybox with toybox/tests/*.test scripts,
I got many false FAILs from it. So, I tried to fix useradd.test file.
What I fixed are as below.

1. I added the checking routine at the head of file whether it is run by
root.
useradd.test should be PASSed only by root user.

2. I Separated 'userdel' from 'testing' syntax.
in existing code,

*testing "adduser user_name (text)" "useradd toyTestUser $arg ||*
* grep '^toyTestUser:' /etc/passwd $arg && test -d /home/toyTestUser &&*
* userdel toyTestUser $arg && rm -rf /home/toyTestUser && echo 'yes'" \*
* "yes\n" "" "$pass" *

*userdel toyTestUser $arg && rm -rf /home/toyTestUser* is not related with
useradd directly. But if userdel or rm occur error, this test case is
FAILed. So I moved this part out of testing syntax.

Could you share your opinion about this patch?


Yeongdeok
Rob Landley
2015-01-30 22:53:59 UTC
Permalink
Post by Yeongdeok Suh
Hi, all,
When I test toybox with toybox/tests/*.test scripts,
I got many false FAILs from it. So, I tried to fix useradd.test file.
What I fixed are as below.
This week has been entirely eaten by $DAYJOB having me do kernel stuff.
I hope to catch up a bit this weekend...
Post by Yeongdeok Suh
1. I added the checking routine at the head of file whether it is run by
root.
useradd.test should be PASSed only by root user.
Existing tests (ifconfig, losetup, chgrp, chown) output "SKIPPED"
instead of FAIL. (Not running the test as root isn't a failure per say,
it just means we couldn't perform this test in this context. That way
when we run "make test" to test all the commands enabled in the current
config, we don't get spurious failures.)
Post by Yeongdeok Suh
2. I Separated 'userdel' from 'testing' syntax.
in existing code,
/testing "adduser user_name (text)" "useradd toyTestUser $arg ||/
/ grep '^toyTestUser:' /etc/passwd $arg && test -d /home/toyTestUser &&/
/ *userdel toyTestUser $arg && rm -rf /home/toyTestUser &&* echo 'yes'" \/
/ "yes\n" "" "$pass" /
*/userdel toyTestUser $arg && rm -rf /home/toyTestUser/* is not related
with useradd directly. But if userdel or rm occur error, this test case
is FAILed. So I moved this part out of testing syntax.
It's a cleanup step, yes.
Post by Yeongdeok Suh
Could you share your opinion about this patch?
useradd and userdel are still in pending, in part because Android uses a
different userlist mechanism than /etc/passwd files and I need to do
some research on the correct way to handle that.

Oddly, enh (the Android guy) sent us this patch:

https://www.mail-archive.com/***@lists.landley.net/msg01664.html

Which reaches out and touches /etc/passwd for testing chown. It's on my
list of things to think about in a lot more depth than I've had time to yet.

There are some minor hiccups with your patch (&> redirects both stdout
and stderr, and then you redirect stderr again?) but it's better than
what was there. I'll probably apply it this weekend. (Bured by $DAYJOB
right now...)
Post by Yeongdeok Suh
Yeongdeok
Thanks,

Rob
enh
2015-01-30 23:55:02 UTC
Permalink
Post by Rob Landley
Post by Yeongdeok Suh
Hi, all,
When I test toybox with toybox/tests/*.test scripts,
I got many false FAILs from it. So, I tried to fix useradd.test file.
What I fixed are as below.
This week has been entirely eaten by $DAYJOB having me do kernel stuff.
I hope to catch up a bit this weekend...
Post by Yeongdeok Suh
1. I added the checking routine at the head of file whether it is run by
root.
useradd.test should be PASSed only by root user.
Existing tests (ifconfig, losetup, chgrp, chown) output "SKIPPED"
instead of FAIL. (Not running the test as root isn't a failure per say,
it just means we couldn't perform this test in this context. That way
when we run "make test" to test all the commands enabled in the current
config, we don't get spurious failures.)
Post by Yeongdeok Suh
2. I Separated 'userdel' from 'testing' syntax.
in existing code,
/testing "adduser user_name (text)" "useradd toyTestUser $arg ||/
/ grep '^toyTestUser:' /etc/passwd $arg && test -d /home/toyTestUser &&/
/ *userdel toyTestUser $arg && rm -rf /home/toyTestUser &&* echo 'yes'" \/
/ "yes\n" "" "$pass" /
*/userdel toyTestUser $arg && rm -rf /home/toyTestUser/* is not related
with useradd directly. But if userdel or rm occur error, this test case
is FAILed. So I moved this part out of testing syntax.
It's a cleanup step, yes.
Post by Yeongdeok Suh
Could you share your opinion about this patch?
useradd and userdel are still in pending, in part because Android uses a
different userlist mechanism than /etc/passwd files and I need to do
some research on the correct way to handle that.
Which reaches out and touches /etc/passwd for testing chown. It's on my
list of things to think about in a lot more depth than I've had time to yet.
oops! sorry to have caused confusion...

right now i don't have any good way to run your tests on Android. i
need to make a "simon says"-style mess with "adb shell" (at least at
run time; hopefully not actually in the test scripts themselves), and
haven't got round to doing that yet. so right now if i make a code
change, i actually run the tests on the host (with either bionic or
glibc, depending on what exactly i'm trying to test). on the device i
usually just rely on manual testing. (in this specific case, i rewrote
the file using "root" and "shell" for Android.)

as i've said, Android doesn't have /etc/passwd (or /etc/group), and it
doesn't have setpwent/getpwent/endpwent (or equivalents). you *can* do
uid/gid or name lookups though, because they do make some degree of
sense. (so id(1) works.) i've considered implementing getpwent so that
it would cycle through the well known users, but we don't actually
have an example of anything that would use it, and it's really not
obvious that we'd be doing anyone any favors --- code calling getpwent
that wants to run on Android needs to think long and hard about
exactly what it means by that, and whether it makes any sense at all.
so going back to your "in pending ... because Android", i think you
should just ignore us as far as adduser/userdel are concerned. i don't
think there's anything useful you can do.

the fact that Android tends to be pretty locked down with SELinux
means it's going to be tricky to test a lot of toybox things, but
getting to a point where i can run the test suite is definitely a
goal. just not a priority (because i can run the tests on the desktop
with bionic).
Post by Rob Landley
There are some minor hiccups with your patch (&> redirects both stdout
and stderr, and then you redirect stderr again?) but it's better than
what was there. I'll probably apply it this weekend. (Bured by $DAYJOB
right now...)
Post by Yeongdeok Suh
Yeongdeok
Thanks,
Rob
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Rich Felker
2015-01-31 01:41:14 UTC
Permalink
Post by enh
as i've said, Android doesn't have /etc/passwd (or /etc/group), and it
doesn't have setpwent/getpwent/endpwent (or equivalents). you *can* do
uid/gid or name lookups though, because they do make some degree of
sense. (so id(1) works.) i've considered implementing getpwent so that
it would cycle through the well known users, but we don't actually
have an example of anything that would use it, and it's really not
obvious that we'd be doing anyone any favors --- code calling getpwent
that wants to run on Android needs to think long and hard about
exactly what it means by that, and whether it makes any sense at all.
The main application I've seen that makes _any_ sense is
tab-completion of usernames in the shell.
Post by enh
the fact that Android tends to be pretty locked down with SELinux
means it's going to be tricky to test a lot of toybox things, but
getting to a point where i can run the test suite is definitely a
goal. just not a priority (because i can run the tests on the desktop
with bionic).
Is there any guide to getting bionic built and working on a desktop
system, short of doing a complete android build? I tried a while back
with the gentoobionic repo which was setup to work without the android
build system, but didn't get anywhere with it. This is not something I
have a lot of time to spend on, but I'd like it if I could get
something working to evaluate differences between musl and bionic.

Rich
enh
2015-01-31 01:53:55 UTC
Permalink
Post by Rich Felker
Post by enh
as i've said, Android doesn't have /etc/passwd (or /etc/group), and it
doesn't have setpwent/getpwent/endpwent (or equivalents). you *can* do
uid/gid or name lookups though, because they do make some degree of
sense. (so id(1) works.) i've considered implementing getpwent so that
it would cycle through the well known users, but we don't actually
have an example of anything that would use it, and it's really not
obvious that we'd be doing anyone any favors --- code calling getpwent
that wants to run on Android needs to think long and hard about
exactly what it means by that, and whether it makes any sense at all.
The main application I've seen that makes _any_ sense is
tab-completion of usernames in the shell.
yeah, which wouldn't be very useful on a system that's so locked down
by SELinux that users are very much not interchangeable.
Post by Rich Felker
Post by enh
the fact that Android tends to be pretty locked down with SELinux
means it's going to be tricky to test a lot of toybox things, but
getting to a point where i can run the test suite is definitely a
goal. just not a priority (because i can run the tests on the desktop
with bionic).
Is there any guide to getting bionic built and working on a desktop
system, short of doing a complete android build? I tried a while back
with the gentoobionic repo which was setup to work without the android
build system, but didn't get anywhere with it. This is not something I
have a lot of time to spend on, but I'd like it if I could get
something working to evaluate differences between musl and bionic.
not really because we always need the whole tree anyway, but "cd
bionic ; mma -j32" is probably a good place to start to only build the
dependencies. (this is after the usual "source build/envsetup.sh ;
lunch aosp_x86_64-eng".) there's a build target in bionic/tests to run
the tests on the host that will set up /system so that Android
binaries will run (because the toolchain hardcodes the linker's path,
and that's /system/bin/linker64). everything else can be diverted with
environment variables, which that build target also does.

details of how to submit patches is at
https://source.android.com/source/submit-patches.html :-)
Post by Rich Felker
Rich
Rob Landley
2015-02-18 19:10:39 UTC
Permalink
Post by enh
Post by Rob Landley
Post by Yeongdeok Suh
Hi, all,
When I test toybox with toybox/tests/*.test scripts,
I got many false FAILs from it. So, I tried to fix useradd.test file.
What I fixed are as below.
This week has been entirely eaten by $DAYJOB having me do kernel stuff.
I hope to catch up a bit this weekend...
Post by Yeongdeok Suh
1. I added the checking routine at the head of file whether it is run by
root.
useradd.test should be PASSed only by root user.
Existing tests (ifconfig, losetup, chgrp, chown) output "SKIPPED"
instead of FAIL. (Not running the test as root isn't a failure per say,
it just means we couldn't perform this test in this context. That way
when we run "make test" to test all the commands enabled in the current
config, we don't get spurious failures.)
Post by Yeongdeok Suh
2. I Separated 'userdel' from 'testing' syntax.
in existing code,
/testing "adduser user_name (text)" "useradd toyTestUser $arg ||/
/ grep '^toyTestUser:' /etc/passwd $arg && test -d /home/toyTestUser &&/
/ *userdel toyTestUser $arg && rm -rf /home/toyTestUser &&* echo 'yes'" \/
/ "yes\n" "" "$pass" /
*/userdel toyTestUser $arg && rm -rf /home/toyTestUser/* is not related
with useradd directly. But if userdel or rm occur error, this test case
is FAILed. So I moved this part out of testing syntax.
It's a cleanup step, yes.
Post by Yeongdeok Suh
Could you share your opinion about this patch?
useradd and userdel are still in pending, in part because Android uses a
different userlist mechanism than /etc/passwd files and I need to do
some research on the correct way to handle that.
Which reaches out and touches /etc/passwd for testing chown. It's on my
list of things to think about in a lot more depth than I've had time to yet.
oops! sorry to have caused confusion...
right now i don't have any good way to run your tests on Android. i
need to make a "simon says"-style mess with "adb shell" (at least at
run time; hopefully not actually in the test scripts themselves), and
haven't got round to doing that yet.
I need to upgrade android toysh to the point where both the toybox build
and the toybox test suite can run under the toybox shell.

(Always a goal, just... kind of a big job.)

I poked at the mirbsd guys online, but their license doesn't allow their
code to be merged into toybox as is (yes, I'm a stickler for this sort
of thing), and they have no interest in offering different license terms
to the code. (Alas, would have saved some time to rebase and start
fixing things up from there.)

I've recently been poking at Aboriginal Linux (since part of my toybox
release testing is building linux from scratch under that) and I
recently found out that the bash 2.05b I've been using in my Aboriginal
Linux project won't run the new parallel toybox build scripts (where
"new" is like 6 months ago now). So the "rebuild itself under itself"
thing is broken until I either fix up bash or get toysh to the point I
can swap it in.
Post by enh
so right now if i make a code
change, i actually run the tests on the host (with either bionic or
glibc, depending on what exactly i'm trying to test).
I need to set up a bionic host test environment. I want to add that to
my standard regression testing. Alas, its build is an android.mk and I
dunno how to run that outside of the giant AOSP build blob.
Post by enh
on the device i
the file using "root" and "shell" for Android.)
You could also add a hook to scripts/runtest.sh replacing the "eval" on
line 86 with a function similar to the do_loudly function starting on
scripts/make.sh line 19.

What does an adb remote invocation look like? Anything like testing via
ssh? I've vaguely pondered adding a "passwordly ssh to remote machine
and run the test there" thing so I could set up an aboriginal linux
powerpc instance, do a static cross compile build and run the tests
remotely inside the emulated instance with the host's test script
driving things.

(Alas, a generic solution would also require a shared filesystem since
the test script is doing mkdir and such to set up temporary files to
test on. But I could probably botch something up...)
Post by enh
as i've said, Android doesn't have /etc/passwd (or /etc/group), and it
doesn't have setpwent/getpwent/endpwent (or equivalents). you *can* do
uid/gid or name lookups though, because they do make some degree of
sense. (so id(1) works.) i've considered implementing getpwent so that
it would cycle through the well known users, but we don't actually
have an example of anything that would use it, and it's really not
obvious that we'd be doing anyone any favors --- code calling getpwent
that wants to run on Android needs to think long and hard about
exactly what it means by that, and whether it makes any sense at all.
This I'm still trying to wrap my head around. Most of the android
educational resources I find assume you're programming an app in java.

Even if all your uid/gid pairs are dynamically allocated, can't you have
an app that requests and tracks uid/gid pairs, and assigns names for
them? Ok, they wouldn't be sequential (although the containers
infrastructure can remap them so they could _appear_ sequential if it
really mattered because somebody wanted to run a cifs fileserver or
something)...
Post by enh
so going back to your "in pending ... because Android", i think you
should just ignore us as far as adduser/userdel are concerned. i don't
think there's anything useful you can do.
I've heard that before. :)

It still sounds fixable. Containers are awesome. Pity the crazy systemd
guys keep trying to hog the infrastructure over in legacy linux land,
but one big advantage android has is that the systemd being gpl keeps it
_off_ of android, so you don't have to get any of "devfsd-tng" on you
and can await the train wreck with popcorn...
Post by enh
the fact that Android tends to be pretty locked down with SELinux
means it's going to be tricky to test a lot of toybox things, but
getting to a point where i can run the test suite is definitely a
goal. just not a priority (because i can run the tests on the desktop
with bionic).
I'm all for having a billion unadministered full posix systems with
broadband access being tightly locked down, but I'm far more a fan of
the containers approach (make the entire system nest so you can contain
the damage) than SELinux's whack-a-mole complexity. It would be nice if
there could be some kind of migration path eventually, but again I would
know where to start...
Post by enh
Post by Rob Landley
There are some minor hiccups with your patch (&> redirects both stdout
and stderr, and then you redirect stderr again?) but it's better than
what was there. I'll probably apply it this weekend. (Bured by $DAYJOB
right now...)
Did I remember to do that?

Sigh. No, I did not. Ok, now it's in.

Thanks,

Rob
enh
2015-02-20 17:46:37 UTC
Permalink
Post by Rob Landley
Post by enh
Post by Rob Landley
Post by Yeongdeok Suh
Hi, all,
When I test toybox with toybox/tests/*.test scripts,
I got many false FAILs from it. So, I tried to fix useradd.test file.
What I fixed are as below.
This week has been entirely eaten by $DAYJOB having me do kernel stuff.
I hope to catch up a bit this weekend...
Post by Yeongdeok Suh
1. I added the checking routine at the head of file whether it is run by
root.
useradd.test should be PASSed only by root user.
Existing tests (ifconfig, losetup, chgrp, chown) output "SKIPPED"
instead of FAIL. (Not running the test as root isn't a failure per say,
it just means we couldn't perform this test in this context. That way
when we run "make test" to test all the commands enabled in the current
config, we don't get spurious failures.)
Post by Yeongdeok Suh
2. I Separated 'userdel' from 'testing' syntax.
in existing code,
/testing "adduser user_name (text)" "useradd toyTestUser $arg ||/
/ grep '^toyTestUser:' /etc/passwd $arg && test -d /home/toyTestUser &&/
/ *userdel toyTestUser $arg && rm -rf /home/toyTestUser &&* echo 'yes'" \/
/ "yes\n" "" "$pass" /
*/userdel toyTestUser $arg && rm -rf /home/toyTestUser/* is not related
with useradd directly. But if userdel or rm occur error, this test case
is FAILed. So I moved this part out of testing syntax.
It's a cleanup step, yes.
Post by Yeongdeok Suh
Could you share your opinion about this patch?
useradd and userdel are still in pending, in part because Android uses a
different userlist mechanism than /etc/passwd files and I need to do
some research on the correct way to handle that.
Which reaches out and touches /etc/passwd for testing chown. It's on my
list of things to think about in a lot more depth than I've had time to yet.
oops! sorry to have caused confusion...
right now i don't have any good way to run your tests on Android. i
need to make a "simon says"-style mess with "adb shell" (at least at
run time; hopefully not actually in the test scripts themselves), and
haven't got round to doing that yet.
I need to upgrade android toysh to the point where both the toybox build
and the toybox test suite can run under the toybox shell.
(Always a goal, just... kind of a big job.)
I poked at the mirbsd guys online, but their license doesn't allow their
code to be merged into toybox as is (yes, I'm a stickler for this sort
of thing), and they have no interest in offering different license terms
to the code. (Alas, would have saved some time to rebase and start
fixing things up from there.)
I've recently been poking at Aboriginal Linux (since part of my toybox
release testing is building linux from scratch under that) and I
recently found out that the bash 2.05b I've been using in my Aboriginal
Linux project won't run the new parallel toybox build scripts (where
"new" is like 6 months ago now). So the "rebuild itself under itself"
thing is broken until I either fix up bash or get toysh to the point I
can swap it in.
Post by enh
so right now if i make a code
change, i actually run the tests on the host (with either bionic or
glibc, depending on what exactly i'm trying to test).
I need to set up a bionic host test environment. I want to add that to
my standard regression testing. Alas, its build is an android.mk and I
dunno how to run that outside of the giant AOSP build blob.
Post by enh
on the device i
the file using "root" and "shell" for Android.)
You could also add a hook to scripts/runtest.sh replacing the "eval" on
line 86 with a function similar to the do_loudly function starting on
scripts/make.sh line 19.
What does an adb remote invocation look like? Anything like testing via
ssh? I've vaguely pondered adding a "passwordly ssh to remote machine
and run the test there" thing so I could set up an aboriginal linux
powerpc instance, do a static cross compile build and run the tests
remotely inside the emulated instance with the host's test script
driving things.
so that's the first problem... you give adb more credit than it
deserves. of its many misfeatures, pertinent ones here are: it won't
work in a pipe, it won't return the exit status of the command it ran,
and it will add a \r to every line of output. i need to fix all of
those, but none of them have become the most urgent fire yet, and no
one's sent completely convincing patches. facebook wrote their own adb
client (but it's an extended subset and doesn't support Windows [or
the Mac?] so it's not really a replacement).

anyway... something like this lets you run stuff like the pwd tests on
a connected Android device. ($ANDROID_SERIAL is how you tell adb which
Android device to talk to if there are several connected, without
having to specify it on every command line. since you'd want to set
that anyway before running the tests, that seemed reasonable.)

diff --git a/scripts/runtest.sh b/scripts/runtest.sh
index 8da1089..c4244a8 100644
--- a/scripts/runtest.sh
+++ b/scripts/runtest.sh
@@ -83,8 +83,16 @@ testing()

echo -ne "$3" > expected
echo -ne "$4" > input
- echo -ne "$5" | eval "$2" > actual
- RETVAL=$?
+ if [ -n "$ANDROID_SERIAL" ]
+ then
+ adb shell "echo -ne '$5' | eval '$2' > /data/local/tmp/actual ;
echo $? > /data/local/tmp/retval"
+ adb pull /data/local/tmp/actual .
+ adb pull /data/local/tmp/retval .
+ RETVAL=`cat retval`
+ else
+ echo -ne "$5" | eval "$2" > actual
+ RETVAL=$?
+ fi

cmp expected actual > /dev/null 2>&1
if [ $? -ne 0 ]
Post by Rob Landley
(Alas, a generic solution would also require a shared filesystem since
the test script is doing mkdir and such to set up temporary files to
test on. But I could probably botch something up...)
...but, yeah, this is a problem. any test script that does some file
system setup is a problem.

one possibility i've wondered about is factoring that out into a
script of its own. this would have two advantages: 1. only one place
to fix all this stuff for Android/ssh/whatever and 2. less boilerplate
in each test script. if there was some known file system tree for
commands to work on.
Post by Rob Landley
Post by enh
as i've said, Android doesn't have /etc/passwd (or /etc/group), and it
doesn't have setpwent/getpwent/endpwent (or equivalents). you *can* do
uid/gid or name lookups though, because they do make some degree of
sense. (so id(1) works.) i've considered implementing getpwent so that
it would cycle through the well known users, but we don't actually
have an example of anything that would use it, and it's really not
obvious that we'd be doing anyone any favors --- code calling getpwent
that wants to run on Android needs to think long and hard about
exactly what it means by that, and whether it makes any sense at all.
This I'm still trying to wrap my head around. Most of the android
educational resources I find assume you're programming an app in java.
Even if all your uid/gid pairs are dynamically allocated, can't you have
an app that requests and tracks uid/gid pairs, and assigns names for
them? Ok, they wouldn't be sequential (although the containers
infrastructure can remap them so they could _appear_ sequential if it
really mattered because somebody wanted to run a cifs fileserver or
something)...
no, there's no API for any of that. i doubt non-system apps even have
the ability to read the relevant databases.
Post by Rob Landley
Post by enh
so going back to your "in pending ... because Android", i think you
should just ignore us as far as adduser/userdel are concerned. i don't
think there's anything useful you can do.
I've heard that before. :)
It still sounds fixable. Containers are awesome. Pity the crazy systemd
guys keep trying to hog the infrastructure over in legacy linux land,
but one big advantage android has is that the systemd being gpl keeps it
_off_ of android, so you don't have to get any of "devfsd-tng" on you
and can await the train wreck with popcorn...
i'm not saying the implementation won't ever change, but afaik no one
is working on it. your best hope is for someone to decide that there's
a big enough security advantage to make it worth the disruption.
Post by Rob Landley
Post by enh
the fact that Android tends to be pretty locked down with SELinux
means it's going to be tricky to test a lot of toybox things, but
getting to a point where i can run the test suite is definitely a
goal. just not a priority (because i can run the tests on the desktop
with bionic).
I'm all for having a billion unadministered full posix systems with
broadband access being tightly locked down, but I'm far more a fan of
the containers approach (make the entire system nest so you can contain
the damage) than SELinux's whack-a-mole complexity. It would be nice if
there could be some kind of migration path eventually, but again I would
know where to start...
Post by enh
Post by Rob Landley
There are some minor hiccups with your patch (&> redirects both stdout
and stderr, and then you redirect stderr again?) but it's better than
what was there. I'll probably apply it this weekend. (Bured by $DAYJOB
right now...)
Did I remember to do that?
Sigh. No, I did not. Ok, now it's in.
Thanks,
Rob
Rob Landley
2015-02-20 23:12:19 UTC
Permalink
Post by enh
Post by Rob Landley
You could also add a hook to scripts/runtest.sh replacing the "eval" on
line 86 with a function similar to the do_loudly function starting on
scripts/make.sh line 19.
What does an adb remote invocation look like? Anything like testing via
ssh? I've vaguely pondered adding a "passwordless ssh to remote machine
and run the test there" thing so I could set up an aboriginal linux
powerpc instance, do a static cross compile build and run the tests
remotely inside the emulated instance with the host's test script
driving things.
so that's the first problem... you give adb more credit than it
deserves. of its many misfeatures, pertinent ones here are: it won't
work in a pipe,
I've wrapped stuff in pipe to pty translation before to make something
that really cared about having a controlling terminal think it had one.
I've also done ratelimiting for crap like qemu's powerpc serial
emulation where incoming data bigger than the serial buffer size gets
discarded.
Post by enh
it won't return the exit status of the command it ran,
You'll notice the "test" command isn't checking exit status either,
which is why all the "|| echo yes" and such.
Post by enh
and it will add a \r to every line of output.
| dos2unix
Post by enh
i need to fix all of
those, but none of them have become the most urgent fire yet, and no
one's sent completely convincing patches. facebook wrote their own adb
client (but it's an extended subset and doesn't support Windows [or
the Mac?] so it's not really a replacement).
Lovely.

What's the adb transport, anyway? Toybox netcat has a "-f" mode that
connects to a file instead of stdin/stdout, which I've used as
half-assed microcom before. (I believe the help text describes how. A
todo item is making a _proper_ microcom alias for it, but that involves
writing stty first.)

(Alas the recent netcat patch to change the command line option parsing
broke server mode. I have a todo item to fix that before the release but
I've been fighting with my aboriginal linux test environment for a
couple weeks now and that's a blocker for doing a release of either
because the lfs-bootstrap with current toybox on a current kernel is my
main regression test.)
Post by enh
anyway... something like this lets you run stuff like the pwd tests on
a connected Android device. ($ANDROID_SERIAL is how you tell adb which
Android device to talk to if there are several connected, without
having to specify it on every command line. since you'd want to set
that anyway before running the tests, that seemed reasonable.)
diff --git a/scripts/runtest.sh b/scripts/runtest.sh
index 8da1089..c4244a8 100644
--- a/scripts/runtest.sh
+++ b/scripts/runtest.sh
@@ -83,8 +83,16 @@ testing()
echo -ne "$3" > expected
echo -ne "$4" > input
- echo -ne "$5" | eval "$2" > actual
- RETVAL=$?
+ if [ -n "$ANDROID_SERIAL" ]
+ then
+ adb shell "echo -ne '$5' | eval '$2' > /data/local/tmp/actual ;
echo $? > /data/local/tmp/retval"
+ adb pull /data/local/tmp/actual .
+ adb pull /data/local/tmp/retval .
+ RETVAL=`cat retval`
+ else
+ echo -ne "$5" | eval "$2" > actual
+ RETVAL=$?
+ fi
Is there a way to hook that up to the emulator?

(I have two old phones with android I keep eyeing as test devices, but
both are their respective ancient vendor installs from before I
upgraded. My current Nexus 5 is t-mobile stock because I use it as a
phone. Although the recent upgrade to version 5.0.1 has made me
seriously reconsider that policy...)
Post by enh
cmp expected actual > /dev/null 2>&1
if [ $? -ne 0 ]
Post by Rob Landley
(Alas, a generic solution would also require a shared filesystem since
the test script is doing mkdir and such to set up temporary files to
test on. But I could probably botch something up...)
...but, yeah, this is a problem. any test script that does some file
system setup is a problem.
Did I mention that I have the first 1/3 or so of a 9p fileserver here,
and that when I sat next to a Samba developer at texas linuxfest last
year he told me that writing an smb server that _just_ served the newest
protocol version in "posix" mode was probably a simpler protocol than
9p2000.L? (He sent me links and everything.)

It's on the todo list. Also, Apple apparently wrote their own samba
replacement as part of their "great GPL purge":

http://www.osnews.com/story/24572/Apple_Ditches_SAMBA_in_Favour_of_Homegrown_Replacement

http://meta.ath0.com/2012/02/05/apples-great-gpl-purge/
Post by enh
one possibility i've wondered about is factoring that out into a
script of its own. this would have two advantages: 1. only one place
to fix all this stuff for Android/ssh/whatever and 2. less boilerplate
in each test script. if there was some known file system tree for
commands to work on.
Factoring which out into its own script, setting up a shared filesystem?

The long-term solution is for me to get a proper shell on android, which
means making toysh work well enough to build toybox, aboriginal linux,
linux from scratch, and eventually the AOSP build.

But not this week...
Post by enh
Post by Rob Landley
Post by enh
as i've said, Android doesn't have /etc/passwd (or /etc/group), and it
doesn't have setpwent/getpwent/endpwent (or equivalents). you *can* do
uid/gid or name lookups though, because they do make some degree of
sense. (so id(1) works.) i've considered implementing getpwent so that
it would cycle through the well known users, but we don't actually
have an example of anything that would use it, and it's really not
obvious that we'd be doing anyone any favors --- code calling getpwent
that wants to run on Android needs to think long and hard about
exactly what it means by that, and whether it makes any sense at all.
This I'm still trying to wrap my head around. Most of the android
educational resources I find assume you're programming an app in java.
Even if all your uid/gid pairs are dynamically allocated, can't you have
an app that requests and tracks uid/gid pairs, and assigns names for
them? Ok, they wouldn't be sequential (although the containers
infrastructure can remap them so they could _appear_ sequential if it
really mattered because somebody wanted to run a cifs fileserver or
something)...
no, there's no API for any of that. i doubt non-system apps even have
the ability to read the relevant databases.
Takes some of the fun out of it.

This only comes up because I don't know how much traditional linux
context the AOSP build expects the build machine to have. If I can dig
out from under the current todo and $DAYJOB stuff, I need to try to
bootstrap the AOSP build natively under Aboriginal Linux (because it's
the simplest most stripped down build environment I've managed to get
working, so I should only ever have to _add_ stuff to it, and there
should never be a thing in it that another development environment
wouldn't have. If aboriginal has it, that's because you can't build a
self-bootstrapping system without it, so no missed implicit dependencies.)

But just working out "these packages build in this order" and then
triggering the builds one at a time is probably going to be a bit of a
job. (Let alone mapping the actual prerequisites. But hey, I've done it
before...)
Post by enh
Post by Rob Landley
Post by enh
so going back to your "in pending ... because Android", i think you
should just ignore us as far as adduser/userdel are concerned. i don't
think there's anything useful you can do.
I've heard that before. :)
It still sounds fixable. Containers are awesome. Pity the crazy systemd
guys keep trying to hog the infrastructure over in legacy linux land,
but one big advantage android has is that the systemd being gpl keeps it
_off_ of android, so you don't have to get any of "devfsd-tng" on you
and can await the train wreck with popcorn...
i'm not saying the implementation won't ever change, but afaik no one
is working on it.
Then me working on it wouldn't interfere with anybody. :)

(Although by the time I get the chance to, somebody else might be so I
need to try to stay up to date. Presumably there's some mailing list I
should be on somewhere...)
Post by enh
your best hope is for someone to decide that there's
a big enough security advantage to make it worth the disruption.
Oh I'm not trying to disrupt. I was suggesting creating a new thing off
to one side that could run builds, not changing the context your
existing stuff runs in.

If the system layer could reserve a UID/GID range that will never be
used by existing apk installs (a thousand of each should be plenty,
somewhere up in the sixty thousands maybe), then you could have a system
tool that creates a new LXC style container that maps those UID/GID
values down to the first 1000 (so inside the container you see UID 0 and
outside the container you see UID 60000 for the same resources), and the
tool runs these processes in a chroot (pivot_root actually) in which
there's a conventional /etc/password and /etc/groups for those however
many of those first 1000 values are actually in use by the container.

That container wouldn't need to have any _other_ UID/GID values
accessible (map the rest of the range to the same "nobody" value), so
you could have multiple users without the rest of the system
particularly having to care.

So the _option_ to add a process context with more traditional unix user
semantics isn't necessarily that intrusive a change.

That said, the bigger question is whether the AOSP build actually
_needs_ multiple users. If it doesn't (and any build that runs as a
non-root user shouldn't), then we can worry about it later.

Rob

Yeongdeok Suh
2015-02-03 02:51:11 UTC
Permalink
Post by Rob Landley
Post by Yeongdeok Suh
Hi, all,
When I test toybox with toybox/tests/*.test scripts,
I got many false FAILs from it. So, I tried to fix useradd.test file.
What I fixed are as below.
This week has been entirely eaten by $DAYJOB having me do kernel stuff.
I hope to catch up a bit this weekend...
I appreciate your efforts:)
I fixed some thing what you said.
Post by Rob Landley
Post by Yeongdeok Suh
1. I added the checking routine at the head of file whether it is run by
root.
useradd.test should be PASSed only by root user.
Existing tests (ifconfig, losetup, chgrp, chown) output "SKIPPED"
instead of FAIL. (Not running the test as root isn't a failure per say,
it just means we couldn't perform this test in this context. That way
when we run "make test" to test all the commands enabled in the current
config, we don't get spurious failures.)
I fixed the "FAIL" to "SKIPPED"
Post by Rob Landley
Post by Yeongdeok Suh
2. I Separated 'userdel' from 'testing' syntax.
in existing code,
/testing "adduser user_name (text)" "useradd toyTestUser $arg ||/
/ grep '^toyTestUser:' /etc/passwd $arg && test -d /home/toyTestUser
&&/
Post by Yeongdeok Suh
/ *userdel toyTestUser $arg && rm -rf /home/toyTestUser &&* echo
'yes'" \/
Post by Yeongdeok Suh
/ "yes\n" "" "$pass" /
*/userdel toyTestUser $arg && rm -rf /home/toyTestUser/* is not related
with useradd directly. But if userdel or rm occur error, this test case
is FAILed. So I moved this part out of testing syntax.
It's a cleanup step, yes.
Post by Yeongdeok Suh
Could you share your opinion about this patch?
useradd and userdel are still in pending, in part because Android uses a
Post by Rob Landley
different userlist mechanism than /etc/passwd files and I need to do
some research on the correct way to handle that.
Which reaches out and touches /etc/passwd for testing chown. It's on my
list of things to think about in a lot more depth than I've had time to yet.
It is good idea. But I didn't apply it to new patch.
I gonna test it on my ARM machines too.
Post by Rob Landley
There are some minor hiccups with your patch (&> redirects both stdout
and stderr, and then you redirect stderr again?) but it's better than
what was there. I'll probably apply it this weekend. (Bured by $DAYJOB
right now...)
Oops! I fixed them to $args ("&>/dev/null").
Post by Rob Landley
Post by Yeongdeok Suh
Yeongdeok
Thanks,
Rob
Thanks,

Yeongdeok
Loading...