[Toybox] macOS sitrep

Discussion:

Rob Landley

2018-12-01 00:41:10 UTC

so, if you merge this afternoon's patches, you can build toybox for
1. i'm only talking about a very limited .config, that's basically
CONFIG_TOYBOX_ICONV=y
CONFIG_TOYBOX_UTMPX=y
CONFIG_TOYBOX_FORK=y
CONFIG_BASENAME=y
CONFIG_CAT=y
CONFIG_CHMOD=y
CONFIG_CMP=y
CONFIG_COMM=y
CONFIG_CUT=y
CONFIG_DIRNAME=y
CONFIG_DU=y
CONFIG_ECHO=y
CONFIG_FALSE=y
CONFIG_FILE=y
CONFIG_GREP=y
CONFIG_EGREP=y
CONFIG_FGREP=y
CONFIG_HEAD=y
CONFIG_ID=y
CONFIG_WHOAMI=y
CONFIG_LN=y
CONFIG_MKDIR=y
CONFIG_OD=y
CONFIG_PASTE=y
CONFIG_PATCH=y
CONFIG_PWD=y
CONFIG_RM=y
CONFIG_RMDIR=y
CONFIG_SED=y
CONFIG_SLEEP=y
CONFIG_SLEEP_FLOAT=y
CONFIG_SORT=y
CONFIG_SORT_BIG=y
CONFIG_SORT_FLOAT=y
CONFIG_TAIL_SEEK=y
CONFIG_TEE=y
CONFIG_TRUE=y
CONFIG_UNAME=y
CONFIG_UNIQ=y
CONFIG_WC=y
CONFIG_XARGS=y
CONFIG_DOS2UNIX=y
CONFIG_UNIX2DOS=y
CONFIG_HELP=y
CONFIG_HELP_EXTRAS=y
CONFIG_READLINK=y
CONFIG_REALPATH=y
CONFIG_SETSID=y
CONFIG_TIMEOUT=y
CONFIG_XXD=y
CONFIG_HOSTNAME=y
CONFIG_MD5SUM=y
CONFIG_SHA1SUM=y
CONFIG_MKTEMP=y
CONFIG_TOYBOX=y
CONFIG_TOYBOX_SUID=y
CONFIG_TOYBOX_LSM_NONE=y
CONFIG_TOYBOX_FLOAT=y
CONFIG_TOYBOX_HELP=y
CONFIG_TOYBOX_HELP_DASHDASH=y
CONFIG_TOYBOX_I18N=y

Easiest thing might be to put that in a file and have a "make mac_defconfig"?

i'm not sure what we want to do about that long-term. add `depends on
TOYBOX_LINUX` to all the stuff like dmesg that's never going to work?
(and probably to the "too much work for anyone to care" stuff like ps
too.)

I might reopen the ps can of worms for bsd, or if darwin became an actual
runnable thing, but probably not for a proprietary OS. :)

2. lib/getmountlist.c doesn't compile. i have #ifndef __APPLE__ ...
#endif around the whole file locally. macos seems to have a wholly
different API, getmntinfo(3).

Thank you _ever_ so much, Posix.

if/when i get around to porting toybox
stat(1) to macOS i might have to fix this properly, but for now i'm
punting. (df isn't used by AOSP and mount/umount seem to be in the
"probably not worth it" territory mentioned in #1.)

I ranted about that in the posix section of roadmap.html. I tried to figure out
a proper way to do it and there isn't one. But there's a reason it _doesn't_
return struct mntent, and that's also why I broke commas.c out from that file
when it started getting more generically used.

It might be best to just move the remaining 3 mntent functions into portability.c...

things like struct stat and struct statfs turn out to be quite
different on macOS. but that's a problem for another day.
______________________________________________

In theory, stat has a posix subset. Dunno if dirtree adheres to it, though. (But
openat() is posix-2008!) And there's a posix statvfs too:

http://pubs.opengroup.org/onlinepubs/009604599/basedefs/sys/statvfs.h.html

I've tried not to be _gratuitously_ linux-specific. But there are large chunks
of things that just haven't got portable APIs. :(

Rob

Rob Landley

2018-12-05 04:05:47 UTC

Permalink

okay, so now toybox builds out of the box (with a custom .config) i
added a bunch more stuff to my .config. the following were all
ascii base64 basename cal cat catv chgrp chmod chown cksum clear cmp
comm count cpio crc32 cut dirname dos2unix du echo egrep env expand
factor false fgrep file flock fmt ftpget ftpput grep groups head help
hexedit hostname iconv id kill killall5 link ln logname md5sum microcom
mkdir mkfifo mktemp nc netcat nice nl nohup od paste patch printenv
printf pwd pwdx readlink realpath renice rev rm rmdir sed seq setsid
sha1sum sha224sum sha256sum sha384sum sha512sum sleep sort split strings
tac tee timeout true truncate tty uname uniq unix2dos unlink usleep
uudecode uuencode uuidgen w watch wc which who whoami xargs xxd yes

Could you send me your .config? I can check in a "make macos_defconfig".

(i haven't even bothered trying anything from pending.)

If you get the urge for specific commands out of there, lemme know. I just
finished redoing test.c and I'm squinting at tftpd.c now (using a dhcp/bootp
thing at $DAYJOB)...

I'm aware android's building bc, dd, diff, expr, getfattr, gzip, lsof, modprobe,
more, stty, tar, tr, and traceroute. But most of that isn't low hanging fruit
for cleanup. Although I'm close on the compression side of gzip, and since
watch.c I think I know how to tackle less/more now (although I need to go _back_
to watch and add ansi color support, which I also need to do in grep although
_that's_ a genericiation of the -o logic), and getfattr _would_ be low hanging
fruit if I had any sort of test case for it (what files on ubuntu normally have
xattrs I can examine?) Last time I tried to tackle expr. I spent all the
time/energy I had reading a chunk of that pdf on the precedence climbing thing
and the window got buried and my machine eventually rebooted...

Mostly when I look at pending it's ls -lS toys/pending, but vi.c is a stub,
groupdel is part of the userdel/useradd/groupadd cluster, getfattr.c I haven't
got any real test cases for (what on ubuntu has attributes I can look at?),
ipcrm I've never really used (and it's in suite with ipcs, need to read up on
both), and that brings us to fold which is basically "not fmt".

Chipping away at it as I can...

Rob

Rob Landley

2018-12-08 03:36:14 UTC

Permalink

Post by Rob Landley

Could you send me your .config? I can check in a "make macos_defconfig".

with the two extra patches i just sent to the list, you can build a

Applied.

ascii base64 basename cal cat catv chgrp chmod chown cksum clear cmp
comm count cpio crc32 cut date dirname dos2unix du echo egrep env
expand factor false fgrep file find flock fmt ftpget ftpput getconf
grep groups head help hexedit hostname iconv id kill killall5 link
ln logger logname ls md5sum microcom mkdir mkfifo nc netcat nice nl
nohup od paste patch printenv printf pwd pwdx readlink realpath renice
rev rm rmdir sed seq setsid sha1sum sha224sum sha256sum sha384sum
sha512sum sleep sort split strings tac tee time timeout touch tr true
truncate tty uname uniq unix2dos unlink usleep uudecode uuencode uuidgen
w watch wc which who whoami xargs xxd yes

Which means it's _not_ building.... (rummage rummage)

acpi blkid blockdev bunzip2 bzcat chattr chroot chrt chvt cp df dmesg eject
fallocate free freeramdisk fsfreeze fstype fsync halt hwclock i2cdetect i2cdump
i2cget i2cset ifconfig inotifyd insmod install ionice iorenice iotop killall
login losetup lsattr lsmod lspci lsusb makedevs mix mknod mkpasswd mkswap mktemp
modinfo mount mountpoint mv nbd-client netstat nproc oneit partprobe passwd
pgrep pidof ping ping6 pivot_root pkill pmap poweroff prlimit ps readahead
reboot reset rfkill rmmod setfattr shred stat su swapoff swapon switch_root sync
sysctl tail taskset test top tunctl ulimit umount uptime vconfig vmstat

Hmmm... bunzip2 and bzcat are easy, reset, sync, mktemp, and test might already
work, shred is trivial but silly on journaled filesystems (it was a request),
dunno how much work stat is, tail has two codepaths and one's probably easy...

The cp/install/mktemp/mv cluser... dunno what's missing? IN theory the openat()
stuff is posix-2008?

ps/top/iotop/pkill/pgrep are probably a nightmare because of the /proc stuff and
I have no idea what a macos equivalent would look like.

Most of the rest probably doesn't apply conceptually. (I mean... maaaaaybe
blkid? Dunno what macos does for mount...)

i've basically ignored pending for now, because i was only originally
hoping to answer the question "is this feasible?" and i think we have
a pretty strong "yes" that includes most of the things we'd need for a
mac AOSP build already.

Cool.

i might try to get the xattr portability wrapper done before i have to
give the mac back, but otherwise i'm hanging the "mission
accomplished" banner for now and will worry about actually, you know,
_using_ this stuff some other day. probably after i've finished
getting linux AOSP builds switched over, which is much more
interesting to me anyway (and especially in the short term).

I wonder how horrible building this subset under https://midipix.org and/or the
Windows linux binary running thingy interface is?

i've attached my config. note that i've included TODOs explaining why
a few of the main failures don't build. tail's inotify is
interesting/annoying.

It's all for FLAG_f. I could add a compile-time probe with a config symbol and a
portability.h stub? (Or move the sys/inotify.h #include into portability.h so
#ifdef APPLE could #else it...)

Post by Rob Landley

(i haven't even bothered trying anything from pending.)

If you get the urge for specific commands out of there, lemme know. I just
finished redoing test.c and I'm squinting at tftpd.c now (using a dhcp/bootp
thing at $DAYJOB)...

of the stuff in pending, AOSP builds would use dd, diff, egrep, grep,
tar, and tr.

grep isn't in pending, but I need to add --color support.

all of which were on your "non-trivial" list. (we're
_building_ toybox's grep family, but we're still using the BSD ones.
Q/master is currently using toybox dd [P had BSD dd].)

Lemme see what I can get done on grep --color this weekend...

The macos regex() implementation is still a bit different from any of the
linuxes, might still cause a problem there...

Rob