Toybox bc's transcendental and irrational functions produce incorrect results, memory leaks and exhibit brute force algorithmic complexity.

Discussion:

(too old to reply)

Sasha Toltec

2018-08-27 09:15:21 UTC

Permalink

Upon inspection of the mathematics core of the toybox bc it appears to
have multiple systemic issues and is potentially unsalvagable.

All transcendental functions produce out of bounds memory accesses.

All builtin functions produce gratuitous and insecure memory errors.

Excessive or incorrect memory consumption for invocations of
transcendental functions result in total memory exhaustion consuming
about 1 GB for every few seconds of run time.

The algorithms used are of the most naive possible variety and balk
under even the smallest inputs taking hundreds of times longer than
similar implementations.

All builtin transcendental and irrational functions produce incorrect results.

Functions that should be complete in a fraction of a second are
getting bogged down in naive algorithmic complexity.

I had read that a team of mathematicians and systems programmers were
working on the
new toybox bc implementation. What exactly happened here?

Sasha Toltec
Toltec Enterprises
***@gmail.com

Sasha Toltec

2018-08-27 13:28:58 UTC

Permalink

We ran across these errors while doing "make check" on the hebimath
bignum library on an alpine linux system running against toybox's
`bc'. Hebimath uses POSIX bc as part of its test suite, Perhaps it
could be used to drive the core arithmetic of your POSIX bc
implementation as it is MIT licensed and is fast and accurate.

https://github.com/suiginsoft/hebimath

Sasha Toltec
Toltec Enterprises

Post by Sasha Toltec
Upon inspection of the mathematics core of the toybox bc it appears to
have multiple systemic issues and is potentially unsalvagable.
All transcendental functions produce out of bounds memory accesses.
All builtin functions produce gratuitous and insecure memory errors.
Excessive or incorrect memory consumption for invocations of
transcendental functions result in total memory exhaustion consuming
about 1 GB for every few seconds of run time.
The algorithms used are of the most naive possible variety and balk
under even the smallest inputs taking hundreds of times longer than
similar implementations.
All builtin transcendental and irrational functions produce incorrect results.
Functions that should be complete in a fraction of a second are
getting bogged down in naive algorithmic complexity.
I had read that a team of mathematicians and systems programmers were
working on the
new toybox bc implementation. What exactly happened here?
Sasha Toltec
Toltec Enterprises

Gavin Howard

2018-08-27 20:39:26 UTC

Permalink

Hi Sasha,

Thank you for bringing to my attention these potentially serious
issues. I will certainly investigate each of them and make the
appropriate changes. I've included an associate who is familiar
with the origins and development of this `bc' to help me address
each point below, as you make some strong assertions.

To your message [1] "Toybox bc run away interactive process",

When attempting to use ctrl-c to kill the toybox `bc' implementation
while in non-interactive mode a bug results in it entering interactive
mode while the program continues to run and then the user loses
control of both interactive and non-interactive modes.
echo "s(131231)" | ./bc -l .. and hit ctrl-c
This consumes many GB of memory before finally exhausting the system.

On a glibc-based x86_64 system I was not able to reproduce this
issue as stated above, however one of your later messages stated
that you encountered the issues on an Alpine Linux (musl-based)
system. On an Alpine 3.8 system I still could not reproduce it,
both in the Toybox environment and the stand-alone upstream [5].

My associate was able to reproduce this using '131231' and '0.5'
on Alpine, even using a fixed scale, e.g., "scale=50", using the
stand-alone version.

As to your second message [2] "Toybox bc's transcendental and
irrational functions produce incorrect results, memory leaks and
exhibit brute force algorithmic complexity.",

Upon inspection of the mathematics core of the toybox `bc' it appears
to have multiple systemic issues and is potentially unsalvagable.

Let me begin by saying that I'm grateful for your insightful and
detailed suggestions. If you have any more suggestions that may
be helpful please feel free to write those to the mailing list.
Several subscribers are definitely interested in this material.

All transcendental functions produce out of bounds memory accesses.
All builtin functions produce gratuitous and insecure memory errors.

We've tested `bc' using three static analysis tools, two memory
scanners (Google Address Sanitizer and Valgrind), in addition to
manual review, and have not found "gratuitous and insecure
memory errors". There may be a single leak in the IO module, and
we are presently looking into this.

Valgrind does indicate excess (but not illegal) memory allocation in
some of the transcendental functions, with certain inputs. We've
yet to isolate specific classes of inputs that cause the issue.

Excessive or incorrect memory consumption for invocations of
transcendental functions result in total memory exhaustion consuming
about 1 GB for every few seconds of run time.

Would you please explain how we might be able to reproduce this?
Is there a specific scenario or condition where this occurs, and
on which particular platform(s) does this occur?

The algorithms used are of the most naive possible variety and balk
under even the smallest inputs taking hundreds of times longer than
similar implementations.

Typically, "naive" algorithms perform better for smaller inputs
and more "advanced" algorithms are desired for their lower bound
which is realized as the input grows large. To hear that "even
the smallest inputs [are] taking hundreds of times longer" than
similar implementations is ... odd.

Would you please share your benchmarks and testing methods? It
will be tremendously helpful. Reference benchmarks to "similar"
implementations (and what those are) would be even better.

For larger problems, you are certainly correct. This `bc' hasn't
been optimized for performance; it's been optimized for size. It
implements a complete language, virtual machine and math library
in under 6000 LOC.

All builtin transcendental and irrational functions produce incorrect results.

Would you please provide an example (ideally for each) showing
an instance where incorrect results are produced?

Functions that should be complete in a fraction of a second are
getting bogged down in naive algorithmic complexity.

See my above remark re: algorithmic complexity, and please offer
an example or two. There had been some discussion regarding just
how "precise" the computation needs to be: an exact solution for
all digits except the last? We will refer to the specification
for clarification on this matter.

I had read that a team of mathematicians and systems programmers were
working on the new toybox bc implementation. What exactly happened
here?

I was not aware of such efforts/arrangements. The Toybox mailing
list on 2018-03-09 by a Christopher M. Graff states that he had
asked a few people, including Gavin D. Howard ("gdh") who wrote
the `bc' in question [5], to assist him with his implementation.
Who are you referring to? That message doesn't reference either,
but perhaps we could collaborate with the mathematicians to help
in achieving lower bounds?

In your third message [4],

This is a helpful suggestion (to use the 'hebimath' test suite);
thank you Sasha. In December 2017 my associate had discussed the
'hebimath' library in a Freenode channel with the aforementioned
CM Graff and determined it a reasonable implementation, but that
it was rather large and would not "fit" into the Toybox project
(and size is the limiting factor).

Anyway, we'll review and make the necessary improvements.

Regards,

Gavin D. Howard Zach van Rijn
"gdh" "Thalheim"

[1]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009626.html

[2]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009628.html

[3]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-March/009403.html

[4]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009629.html

[5]: https://github.com/gavinhoward/bc

We ran across these errors while doing "make check" on the hebimath
bignum library on an alpine linux system running against toybox's
`bc'. Hebimath uses POSIX bc as part of its test suite, Perhaps it
could be used to drive the core arithmetic of your POSIX bc
implementation as it is MIT licensed and is fast and accurate.
https://github.com/suiginsoft/hebimath
Sasha Toltec
Toltec Enterprises

_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Sasha Toltec

2018-08-28 05:35:38 UTC

Permalink

Wait.. what?

Am I really reading an email signed by two people?

First off, you used subtractive chunking for division. Calling this a
naive algorithm is giving it more credit than it is worth. The only
time subtractive chunking is used is by school children.

I cloned your repository (van Rijn and Howard?)

https://github.com/gavinhoward/bc/

You (both?) seem to have no idea how to force errors out of a badly
scaled irrational function so I supplied an example (though judging
from the first example I gave, you (both?) will just lie/boast and say
this one works on your system too!):

echo "sqrt(.0000000000000000000000000000123)" | ./bc -l
.0000000000000035071539807692832
echo "sqrt(.0000000000000000000000000000123)" | bc -l
.0000000000000035071355833500363

As you can see, the "solution" is not even close.

It is very hard to produce test results from such a badly made
library, as it has actually taken my computer offline twice now (by
overusing memory and other strange bugs), but I will give it another
go:

This command takes 25 seconds using your math (if you want to call it
that), but only .01 seconds using a correct implementation:

echo "sqrt(123123^123)" | time ./bc -l
11372609275300462201446509717277573045371910522125088219567947999131\
66658519961107066259363279984633201203915727943358917486581704249214\
07062970046980106751948432347504473036486754487763304174402241913448\
76069500421950114801423888362557755902898498344596580006258740796194\
555554490321132814795761735241438589907716.72501611371543803908
25.30user

echo "sqrt(123123^123)" | time bc -l
11372609275300462201446509717277573045371910522125088219567947999131\
66658519961107066259363279984633201203915727943358917486581704249214\
07062970046980106751948432347504473036486754487763304174402241913448\
76069500421950114801423888362557755902898498344596580006258740796194\
555554490321132814795761735241438589907716.72501611371543803908
0.01user

It is easy to scale this to take more time than the lifetime of the
universe while other implementations are still instantaneous.

I revved your repository back using git checkout and discovered a
sordid history. Including an entire fast arbitrary precision
implementation written by other authors (appears to be CM Graff and
Bao Hexing) which was stripped, renamed and then badly reimplemented.

Please just fix your fix your "implementation" (if one can really call
it that) and stop wasting everyone's time.

Sasha Toltec
Toltec Enterprises

Post by Gavin Howard
Hi Sasha,
Thank you for bringing to my attention these potentially serious
issues. I will certainly investigate each of them and make the
appropriate changes. I've included an associate who is familiar
with the origins and development of this `bc' to help me address
each point below, as you make some strong assertions.
To your message [1] "Toybox bc run away interactive process",

On a glibc-based x86_64 system I was not able to reproduce this
issue as stated above, however one of your later messages stated
that you encountered the issues on an Alpine Linux (musl-based)
system. On an Alpine 3.8 system I still could not reproduce it,
both in the Toybox environment and the stand-alone upstream [5].
My associate was able to reproduce this using '131231' and '0.5'
on Alpine, even using a fixed scale, e.g., "scale=50", using the
stand-alone version.
As to your second message [2] "Toybox bc's transcendental and
irrational functions produce incorrect results, memory leaks and
exhibit brute force algorithmic complexity.",

Upon inspection of the mathematics core of the toybox `bc' it appears
to have multiple systemic issues and is potentially unsalvagable.

All transcendental functions produce out of bounds memory accesses.
All builtin functions produce gratuitous and insecure memory errors.

We've tested `bc' using three static analysis tools, two memory
scanners (Google Address Sanitizer and Valgrind), in addition to
manual review, and have not found "gratuitous and insecure
memory errors". There may be a single leak in the IO module, and
we are presently looking into this.
Valgrind does indicate excess (but not illegal) memory allocation in
some of the transcendental functions, with certain inputs. We've
yet to isolate specific classes of inputs that cause the issue.

Excessive or incorrect memory consumption for invocations of
transcendental functions result in total memory exhaustion consuming
about 1 GB for every few seconds of run time.

Would you please explain how we might be able to reproduce this?
Is there a specific scenario or condition where this occurs, and
on which particular platform(s) does this occur?

The algorithms used are of the most naive possible variety and balk
under even the smallest inputs taking hundreds of times longer than
similar implementations.

Typically, "naive" algorithms perform better for smaller inputs
and more "advanced" algorithms are desired for their lower bound
which is realized as the input grows large. To hear that "even
the smallest inputs [are] taking hundreds of times longer" than
similar implementations is ... odd.
Would you please share your benchmarks and testing methods? It
will be tremendously helpful. Reference benchmarks to "similar"
implementations (and what those are) would be even better.
For larger problems, you are certainly correct. This `bc' hasn't
been optimized for performance; it's been optimized for size. It
implements a complete language, virtual machine and math library
in under 6000 LOC.

All builtin transcendental and irrational functions produce incorrect results.

Would you please provide an example (ideally for each) showing
an instance where incorrect results are produced?

Functions that should be complete in a fraction of a second are
getting bogged down in naive algorithmic complexity.

I had read that a team of mathematicians and systems programmers were
working on the new toybox bc implementation. What exactly happened
here?

I was not aware of such efforts/arrangements. The Toybox mailing
list on 2018-03-09 by a Christopher M. Graff states that he had
the `bc' in question [5], to assist him with his implementation.
Who are you referring to? That message doesn't reference either,
but perhaps we could collaborate with the mathematicians to help
in achieving lower bounds?
In your third message [4],

This is a helpful suggestion (to use the 'hebimath' test suite);
thank you Sasha. In December 2017 my associate had discussed the
'hebimath' library in a Freenode channel with the aforementioned
CM Graff and determined it a reasonable implementation, but that
it was rather large and would not "fit" into the Toybox project
(and size is the limiting factor).
Anyway, we'll review and make the necessary improvements.
Regards,
Gavin D. Howard Zach van Rijn
"gdh" "Thalheim"
[1]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009626.html
[2]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009628.html
[3]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-March/009403.html
[4]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009629.html
[5]: https://github.com/gavinhoward/bc

We ran across these errors while doing "make check" on the hebimath
bignum library on an alpine linux system running against toybox's
`bc'. Hebimath uses POSIX bc as part of its test suite, Perhaps it
could be used to drive the core arithmetic of your POSIX bc
implementation as it is MIT licensed and is fast and accurate.
https://github.com/suiginsoft/hebimath
Sasha Toltec
Toltec Enterprises

_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Rob Landley

2018-08-28 17:51:18 UTC

Permalink

Post by Sasha Toltec
Wait.. what?

Commands in pending/ default to "n" in defconfig. Some of them are nearly
finished and just need more review, some are stubs that don't do anything
useful, and some need complete rewrites.

This is normal for toybox.

Post by Sasha Toltec
Am I really reading an email signed by two people?

People keep emailing me about drama behind the scenes in this command's
development, interchanging handles and names of people I barely know and can't
really track. I'm trying to stay out of it, but don't personally have the domain
expertise to write this command myself, nor the 3-ish months to come up to speed
in this area enough to use existing algorithms explained on wikipedia and similar.

I'm told the main source of drama is somebody named "Graff" who got banned from
#musl, and Rich is _way_ better at dealing with people than I am. I've been
tempted to reject the bc implementation just because it's not worth the drama
(and already deleted an earlier version from pending rather than wait for an
update as I usually do) but if it's just one troll sock puppeting that's nobody
else's fault.

Post by Sasha Toltec
First off, you used subtractive chunking for division. Calling this a
naive algorithm is giving it more credit than it is worth. The only
time subtractive chunking is used is by school children.

And drama. How nice.

Have you seen my factor implementation? It's naive. The largest 64 bit prime
number is 2^64-59, and ubuntu's factor returns that it's prime basically
instantly (even on my tiny little 5 year old $200-when-it-was-new netbook), but
my version:

$ time ./factor 18446744073709551557
^C

I got tired of waiting after about 3 minutes.

My code, which I authored, sucks. I knew that when I checked it in. Patches
welcome. (That said, the 80/20 rule applies: small and simple and works for the
common case, then wait for people to show up with personal use cases in the
remaining 20%...)

Post by Sasha Toltec
I cloned your repository (van Rijn and Howard?)

van Rijn is the guy who hosts the musl-cross-make toolchain binaries at
http://b.zv.io/mcm/bin/ and he's Thalheim on the IRC channel. He's always been
polite and has a history of helping out. I trust his judgement.

(According to Girl Genius he created the nine muses, but his other projects are
his business.)

Post by Sasha Toltec
https://github.com/gavinhoward/bc/
You (both?) seem to have no idea how to force errors out of a badly
scaled irrational function

I don't either, for the record.

Post by Sasha Toltec
so I supplied an example (though judging
from the first example I gave, you (both?) will just lie/boast and say
echo "sqrt(.0000000000000000000000000000123)" | ./bc -l
.0000000000000035071539807692832
echo "sqrt(.0000000000000000000000000000123)" | bc -l
.0000000000000035071355833500363
As you can see, the "solution" is not even close.

Nineteen digits of precision is not even close?

What I usually do is add failing test suite entries to remind me of issues I
need to fix. Happy to merge patches against tests/bc.test. Let's see... Huh.

***@driftwood:~/toybox/toy3$ TEST_HOST=1 time make test_bc
...
598.95vuser 257.87vsystem 14:14.24velapsed

The _host_ version takes almost 12 minutes to run this test. Most of which is
spent in "generating", and spits out a couple "(standard_in) 1: syntax error"
lines. That should probably be fixed somehow (running "make tests" to test
_everything_ should be a feasible thing to do on a regularish basis, this one
test can't take this long, maybe it should have a "short" version or
something... Another item for the todo heap.)

Anyway, adding _this_ test looks like... commit e579e69656b3

Historically, this bc implementation has had one troll, who sock puppets.

Post by Sasha Toltec
This command takes 25 seconds using your math (if you want to call it

I'm ok with that. Patches welcome, but that's not a blocker for me.

Post by Sasha Toltec
It is easy to scale this to take more time than the lifetime of the
universe while other implementations are still instantaneous.

See "factor", above. Which I wrote on a long bus ride while reading
https://www.muppetlabs.com/~breadbox/txt/rsa.html which used "factor", and I
went "that looks easy enough"...

Post by Sasha Toltec
I revved your repository back using git checkout and discovered a
sordid history.

A) One troll, with a history of sock puppetry.

B) Toybox just checked the results into pending. If toybox's maintainer doesn't
care about the history before that (other than confirming it's under the right
license, and the fun bonus that people who know about it are hanging around to
do more work on it as necessary), why would you?

Post by Sasha Toltec
Including an entire fast arbitrary precision
implementation written by other authors (appears to be CM Graff and
Bao Hexing) which was stripped, renamed and then badly reimplemented.

Hello Graff. You can stop now, you've been made.

Post by Sasha Toltec
Please just fix your fix your "implementation" (if one can really call
it that) and stop wasting everyone's time.
Sasha Toltec
Toltec Enterprises

Who does not exist according to Google.

Do I need to switch on list moderation for new subscribers?

Rob

Gavin Howard

2018-08-28 18:22:26 UTC

Permalink

Sasha,

Post by Sasha Toltec
Wait.. what?
Am I really reading an email signed by two people?

Yes; its a common format in professional writing. Not so common
is the strong sarcasm and derision that you display in yours.

Our previous email explained that the motivating factor was code
size. We are free to write faster implementations.

Post by Sasha Toltec
I cloned your repository (van Rijn and Howard?)
https://github.com/gavinhoward/bc/
You (both?) seem to have no idea how to force errors out of a badly
scaled irrational function so I supplied an example (though judging
from the first example I gave, you (both?) will just lie/boast and say
echo "sqrt(.0000000000000000000000000000123)" | ./bc -l
.0000000000000035071539807692832
echo "sqrt(.0000000000000000000000000000123)" | bc -l
.0000000000000035071355833500363
As you can see, the "solution" is not even close.

This example is appreciated. I cannot, however, find your first
example where an irrational function produces incorrect results.

Did you mistakenly send it to the group of "mathematicians and
systems programmers" who were working also on another `bc' ? [1]

Post by Sasha Toltec
It is very hard to produce test results from such a badly made
library, as it has actually taken my computer offline twice now (by
overusing memory and other strange bugs), but I will give it another
This command takes 25 seconds using your math (if you want to call it
echo "sqrt(123123^123)" | time ./bc -l
11372609275300462201446509717277573045371910522125088219567947999131\
66658519961107066259363279984633201203915727943358917486581704249214\
07062970046980106751948432347504473036486754487763304174402241913448\
76069500421950114801423888362557755902898498344596580006258740796194\
555554490321132814795761735241438589907716.72501611371543803908
25.30user
echo "sqrt(123123^123)" | time bc -l
11372609275300462201446509717277573045371910522125088219567947999131\
66658519961107066259363279984633201203915727943358917486581704249214\
07062970046980106751948432347504473036486754487763304174402241913448\
76069500421950114801423888362557755902898498344596580006258740796194\
555554490321132814795761735241438589907716.72501611371543803908
0.01user
It is easy to scale this to take more time than the lifetime of the
universe while other implementations are still instantaneous.

We are looking into this.

Post by Sasha Toltec
I revved your repository back using git checkout and discovered a
sordid history. Including an entire fast arbitrary precision
implementation written by other authors (appears to be CM Graff and
Bao Hexing) which was stripped, renamed and then badly reimplemented.
Please just fix your fix your "implementation" (if one can really call
it that) and stop wasting everyone's time.

Now, this is enough. As far as commits go, I (Gavin D. Howard)
and one other person ("rofl0r") are the only two authors:

$ git log --all --format='%aN <%cE>' | sort -u
Gavin Howard <***@gmail.com>
rofl0r <***@gmail.com>

Background for anyone unfamiliar: as I cited [2] in my previous
email, this `bc' was developed at the same time as this CM Graff
was working on a separate bignum library. The plan was to use
that library behind the `bc' language which I was implementing.

The code you mention, by authors "CM Graff" and "Bao Hexing" was
not in a shape to be integrated with my `bc' at the time that I
had finished my own implementation. This is explained on-list.

You, "Sasha", must have been poring _extremely_ carefully over
each commit (or have advance knowledge of this) to be able to
find "an entire fast arbitrary precision implementation" lying
around, which _only_ existed between Jan. 03 and Jan. 24, 2018,
and of which was not touched or used in any way except residing
in the project directory adjacent to my code for a few weeks.

On January 15, 2018, CM Graff added my name (Gavin D. Howard) to
the LICENSE file for the mathematics library in question. I am
an author. Bao Hexing was added on January 16, 2018.

To accuse me of "stripping", "renaming", and/or "reimplementing"
that code is simply not true. Those claims are not supported by
the commit histories of either project.

One more thing...

I could not find any mention of you or your "consulting company"
online (including reviews or even a business registration in any
of the 50 states in a relevant industry). So if you're charging
money for your services you may wish to resolve that.

For such small developer communities, highly-specific projects,
mention of "mathematicians and systems programmers", `hebimath',
and the two other individuals, it seems -- suspicious -- that
you, "Sasha Toltec" (of whom nobody has heard), suddenly come
into the picture, seemingly so eager to revile the efforts of a
single, strong first implementation for Toybox?

What more is there to say?

Oh yeah: hi, Graff.

Gavin D. Howard

[1]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009628.html

[2]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-March/009403.html

Post by Sasha Toltec

Post by Sasha Toltec
Wait.. what?

Post by Sasha Toltec
Am I really reading an email signed by two people?

People keep emailing me about drama behind the scenes in this command's
development, interchanging handles and names of people I barely know and can't
really track. I'm trying to stay out of it, but don't personally have the domain
expertise to write this command myself, nor the 3-ish months to come up to speed
in this area enough to use existing algorithms explained on wikipedia and similar.
I'm told the main source of drama is somebody named "Graff" who got banned from
#musl, and Rich is _way_ better at dealing with people than I am. I've been
tempted to reject the bc implementation just because it's not worth the drama
(and already deleted an earlier version from pending rather than wait for an
update as I usually do) but if it's just one troll sock puppeting that's nobody
else's fault.

And drama. How nice.
Have you seen my factor implementation? It's naive. The largest 64 bit prime
number is 2^64-59, and ubuntu's factor returns that it's prime basically
instantly (even on my tiny little 5 year old $200-when-it-was-new netbook), but
$ time ./factor 18446744073709551557
^C
I got tired of waiting after about 3 minutes.
My code, which I authored, sucks. I knew that when I checked it in. Patches
welcome. (That said, the 80/20 rule applies: small and simple and works for the
common case, then wait for people to show up with personal use cases in the
remaining 20%...)

Post by Sasha Toltec
I cloned your repository (van Rijn and Howard?)

van Rijn is the guy who hosts the musl-cross-make toolchain binaries at
http://b.zv.io/mcm/bin/ and he's Thalheim on the IRC channel. He's always been
polite and has a history of helping out. I trust his judgement.
(According to Girl Genius he created the nine muses, but his other projects are
his business.)

Post by Sasha Toltec
https://github.com/gavinhoward/bc/
You (both?) seem to have no idea how to force errors out of a badly
scaled irrational function

I don't either, for the record.

Nineteen digits of precision is not even close?
What I usually do is add failing test suite entries to remind me of issues I
need to fix. Happy to merge patches against tests/bc.test. Let's see... Huh.
...
598.95vuser 257.87vsystem 14:14.24velapsed
The _host_ version takes almost 12 minutes to run this test. Most of which is
spent in "generating", and spits out a couple "(standard_in) 1: syntax error"
lines. That should probably be fixed somehow (running "make tests" to test
_everything_ should be a feasible thing to do on a regularish basis, this one
test can't take this long, maybe it should have a "short" version or
something... Another item for the todo heap.)
Anyway, adding _this_ test looks like... commit e579e69656b3

Historically, this bc implementation has had one troll, who sock puppets.

Post by Sasha Toltec
This command takes 25 seconds using your math (if you want to call it

I'm ok with that. Patches welcome, but that's not a blocker for me.

Post by Sasha Toltec
It is easy to scale this to take more time than the lifetime of the
universe while other implementations are still instantaneous.

See "factor", above. Which I wrote on a long bus ride while reading
https://www.muppetlabs.com/~breadbox/txt/rsa.html which used "factor", and I
went "that looks easy enough"...

Post by Sasha Toltec
I revved your repository back using git checkout and discovered a
sordid history.

A) One troll, with a history of sock puppetry.
B) Toybox just checked the results into pending. If toybox's maintainer doesn't
care about the history before that (other than confirming it's under the right
license, and the fun bonus that people who know about it are hanging around to
do more work on it as necessary), why would you?

Hello Graff. You can stop now, you've been made.

Post by Sasha Toltec
Please just fix your fix your "implementation" (if one can really call
it that) and stop wasting everyone's time.
Sasha Toltec
Toltec Enterprises

Who does not exist according to Google.
Do I need to switch on list moderation for new subscribers?
Rob

Sasha Toltec

2018-08-28 19:28:12 UTC

Permalink

I have no idea what you all are talking about.

And no "19 digits of accuracy" is not enough for a multiple-precision
mathematics implementation. I think the issue is just that you need to
find a mathematician to work on the project instead of trying to
busywork the problem type with patches.

Sasha Toltec
Toltec Enterprises

Post by Gavin Howard
Sasha,

Post by Sasha Toltec
Wait.. what?
Am I really reading an email signed by two people?

Yes; its a common format in professional writing. Not so common
is the strong sarcasm and derision that you display in yours.

Our previous email explained that the motivating factor was code
size. We are free to write faster implementations.

This example is appreciated. I cannot, however, find your first
example where an irrational function produces incorrect results.
Did you mistakenly send it to the group of "mathematicians and
systems programmers" who were working also on another `bc' ? [1]

We are looking into this.

Now, this is enough. As far as commits go, I (Gavin D. Howard)
$ git log --all --format='%aN <%cE>' | sort -u
Background for anyone unfamiliar: as I cited [2] in my previous
email, this `bc' was developed at the same time as this CM Graff
was working on a separate bignum library. The plan was to use
that library behind the `bc' language which I was implementing.
The code you mention, by authors "CM Graff" and "Bao Hexing" was
not in a shape to be integrated with my `bc' at the time that I
had finished my own implementation. This is explained on-list.
You, "Sasha", must have been poring _extremely_ carefully over
each commit (or have advance knowledge of this) to be able to
find "an entire fast arbitrary precision implementation" lying
around, which _only_ existed between Jan. 03 and Jan. 24, 2018,
and of which was not touched or used in any way except residing
in the project directory adjacent to my code for a few weeks.
On January 15, 2018, CM Graff added my name (Gavin D. Howard) to
the LICENSE file for the mathematics library in question. I am
an author. Bao Hexing was added on January 16, 2018.
To accuse me of "stripping", "renaming", and/or "reimplementing"
that code is simply not true. Those claims are not supported by
the commit histories of either project.
One more thing...
I could not find any mention of you or your "consulting company"
online (including reviews or even a business registration in any
of the 50 states in a relevant industry). So if you're charging
money for your services you may wish to resolve that.
For such small developer communities, highly-specific projects,
mention of "mathematicians and systems programmers", `hebimath',
and the two other individuals, it seems -- suspicious -- that
you, "Sasha Toltec" (of whom nobody has heard), suddenly come
into the picture, seemingly so eager to revile the efforts of a
single, strong first implementation for Toybox?
What more is there to say?
Oh yeah: hi, Graff.
Gavin D. Howard
[1]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009628.html
[2]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-March/009403.html

Post by Sasha Toltec

Post by Sasha Toltec
Wait.. what?

Post by Sasha Toltec
Am I really reading an email signed by two people?

People keep emailing me about drama behind the scenes in this command's
development, interchanging handles and names of people I barely know and can't
really track. I'm trying to stay out of it, but don't personally have the domain
expertise to write this command myself, nor the 3-ish months to come up
to
speed
in this area enough to use existing algorithms explained on wikipedia and similar.
I'm told the main source of drama is somebody named "Graff" who got
banned
from
#musl, and Rich is _way_ better at dealing with people than I am. I've been
tempted to reject the bc implementation just because it's not worth the drama
(and already deleted an earlier version from pending rather than wait for an
update as I usually do) but if it's just one troll sock puppeting that's nobody
else's fault.

And drama. How nice.
Have you seen my factor implementation? It's naive. The largest 64 bit prime
number is 2^64-59, and ubuntu's factor returns that it's prime basically
instantly (even on my tiny little 5 year old $200-when-it-was-new netbook), but
$ time ./factor 18446744073709551557
^C
I got tired of waiting after about 3 minutes.
My code, which I authored, sucks. I knew that when I checked it in. Patches
welcome. (That said, the 80/20 rule applies: small and simple and works for the
common case, then wait for people to show up with personal use cases in the
remaining 20%...)

Post by Sasha Toltec
I cloned your repository (van Rijn and Howard?)

van Rijn is the guy who hosts the musl-cross-make toolchain binaries at
http://b.zv.io/mcm/bin/ and he's Thalheim on the IRC channel. He's always been
polite and has a history of helping out. I trust his judgement.
(According to Girl Genius he created the nine muses, but his other projects are
his business.)

Post by Sasha Toltec
https://github.com/gavinhoward/bc/
You (both?) seem to have no idea how to force errors out of a badly
scaled irrational function

I don't either, for the record.

Nineteen digits of precision is not even close?
What I usually do is add failing test suite entries to remind me of
issues
I
need to fix. Happy to merge patches against tests/bc.test. Let's see... Huh.
...
598.95vuser 257.87vsystem 14:14.24velapsed
The _host_ version takes almost 12 minutes to run this test. Most of
which
is
spent in "generating", and spits out a couple "(standard_in) 1: syntax error"
lines. That should probably be fixed somehow (running "make tests" to test
_everything_ should be a feasible thing to do on a regularish basis, this one
test can't take this long, maybe it should have a "short" version or
something... Another item for the todo heap.)
Anyway, adding _this_ test looks like... commit e579e69656b3

Historically, this bc implementation has had one troll, who sock puppets.

Post by Sasha Toltec
This command takes 25 seconds using your math (if you want to call it

I'm ok with that. Patches welcome, but that's not a blocker for me.

Post by Sasha Toltec
It is easy to scale this to take more time than the lifetime of the
universe while other implementations are still instantaneous.

See "factor", above. Which I wrote on a long bus ride while reading
https://www.muppetlabs.com/~breadbox/txt/rsa.html which used "factor", and I
went "that looks easy enough"...

Post by Sasha Toltec
I revved your repository back using git checkout and discovered a
sordid history.

A) One troll, with a history of sock puppetry.
B) Toybox just checked the results into pending. If toybox's maintainer doesn't
care about the history before that (other than confirming it's under the right
license, and the fun bonus that people who know about it are hanging around to
do more work on it as necessary), why would you?

Hello Graff. You can stop now, you've been made.

Post by Sasha Toltec
Please just fix your fix your "implementation" (if one can really call
it that) and stop wasting everyone's time.
Sasha Toltec
Toltec Enterprises

Who does not exist according to Google.
Do I need to switch on list moderation for new subscribers?
Rob

Sasha Toltec

2018-08-28 19:28:32 UTC

Permalink

Post by Sasha Toltec
I have no idea what you all are talking about.
And no "19 digits of accuracy" is not enough for a multiple-precision
mathematics implementation. I think the issue is just that you need to
find a mathematician to work on the project instead of trying to
busywork the problem type with patches.
Sasha Toltec
Toltec Enterprises

Post by Gavin Howard
Sasha,

Post by Sasha Toltec
Wait.. what?
Am I really reading an email signed by two people?

Yes; its a common format in professional writing. Not so common
is the strong sarcasm and derision that you display in yours.

Our previous email explained that the motivating factor was code
size. We are free to write faster implementations.

This example is appreciated. I cannot, however, find your first
example where an irrational function produces incorrect results.
Did you mistakenly send it to the group of "mathematicians and
systems programmers" who were working also on another `bc' ? [1]

We are looking into this.

Post by Sasha Toltec

Post by Sasha Toltec
Wait.. what?

Post by Sasha Toltec
Am I really reading an email signed by two people?

People keep emailing me about drama behind the scenes in this command's
development, interchanging handles and names of people I barely know and can't
really track. I'm trying to stay out of it, but don't personally have
the
domain
expertise to write this command myself, nor the 3-ish months to come up
to
speed
in this area enough to use existing algorithms explained on wikipedia
and
similar.
I'm told the main source of drama is somebody named "Graff" who got
banned
from
#musl, and Rich is _way_ better at dealing with people than I am. I've been
tempted to reject the bc implementation just because it's not worth the drama
(and already deleted an earlier version from pending rather than wait
for
an
update as I usually do) but if it's just one troll sock puppeting that's nobody
else's fault.

And drama. How nice.
Have you seen my factor implementation? It's naive. The largest 64 bit prime
number is 2^64-59, and ubuntu's factor returns that it's prime basically
instantly (even on my tiny little 5 year old $200-when-it-was-new netbook), but
$ time ./factor 18446744073709551557
^C
I got tired of waiting after about 3 minutes.
My code, which I authored, sucks. I knew that when I checked it in. Patches
welcome. (That said, the 80/20 rule applies: small and simple and works for the
common case, then wait for people to show up with personal use cases in the
remaining 20%...)

Post by Sasha Toltec
I cloned your repository (van Rijn and Howard?)

van Rijn is the guy who hosts the musl-cross-make toolchain binaries at
http://b.zv.io/mcm/bin/ and he's Thalheim on the IRC channel. He's
always
been
polite and has a history of helping out. I trust his judgement.
(According to Girl Genius he created the nine muses, but his other projects are
his business.)

Post by Sasha Toltec
https://github.com/gavinhoward/bc/
You (both?) seem to have no idea how to force errors out of a badly
scaled irrational function

I don't either, for the record.

Nineteen digits of precision is not even close?
What I usually do is add failing test suite entries to remind me of
issues
I
need to fix. Happy to merge patches against tests/bc.test. Let's see... Huh.
...
598.95vuser 257.87vsystem 14:14.24velapsed
The _host_ version takes almost 12 minutes to run this test. Most of
which
is
spent in "generating", and spits out a couple "(standard_in) 1: syntax error"
lines. That should probably be fixed somehow (running "make tests" to test
_everything_ should be a feasible thing to do on a regularish basis,
this
one
test can't take this long, maybe it should have a "short" version or
something... Another item for the todo heap.)
Anyway, adding _this_ test looks like... commit e579e69656b3

Historically, this bc implementation has had one troll, who sock puppets.

Post by Sasha Toltec
This command takes 25 seconds using your math (if you want to call it

I'm ok with that. Patches welcome, but that's not a blocker for me.

Post by Sasha Toltec
It is easy to scale this to take more time than the lifetime of the
universe while other implementations are still instantaneous.

See "factor", above. Which I wrote on a long bus ride while reading
https://www.muppetlabs.com/~breadbox/txt/rsa.html which used "factor", and I
went "that looks easy enough"...

Post by Sasha Toltec
I revved your repository back using git checkout and discovered a
sordid history.

A) One troll, with a history of sock puppetry.
B) Toybox just checked the results into pending. If toybox's maintainer doesn't
care about the history before that (other than confirming it's under the right
license, and the fun bonus that people who know about it are hanging around to
do more work on it as necessary), why would you?

Hello Graff. You can stop now, you've been made.

Post by Sasha Toltec
Please just fix your fix your "implementation" (if one can really call
it that) and stop wasting everyone's time.
Sasha Toltec
Toltec Enterprises

Who does not exist according to Google.
Do I need to switch on list moderation for new subscribers?
Rob

Zach van Rijn

2018-08-28 19:31:23 UTC

Permalink

Post by Sasha Toltec
I have no idea what you all are talking about.

...

Post by Sasha Toltec
Sasha Toltec
Toltec Enterprises

^ Do you normally mis-type your email address each time
you write your emails? Standard email signatures are
usually consistent...

Sasha Toltec

2018-08-28 19:53:34 UTC

Permalink

I do type my signature out each time yes.

I think what we have here is a classic case of cargo culting. The
nature of the errors indicates that your implementations were copy
pasted without any clue as to how they work.

Otherwise, I am deleting this account. As it seems like you all aren't
interested in the memory error reports or reports of failed functions
and instead are only interested in wasting time.

Sasha Toltec
Toltec Enterprises

Post by Zach van Rijn

Post by Sasha Toltec
I have no idea what you all are talking about.

...

Post by Sasha Toltec
Sasha Toltec
Toltec Enterprises

^ Do you normally mis-type your email address each time
you write your emails? Standard email signatures are
usually consistent...
_______________________________________________
Toybox mailing list
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Zach van Rijn

2018-08-28 20:25:48 UTC

Permalink

Post by Sasha Toltec
I do type my signature out each time yes.
I think what we have here is a classic case of cargo culting.
The nature of the errors indicates that your implementations
were copy pasted without any clue as to how they work.

I'm not sure this even warrants a response. You haven't yet
explained what "nature of the errors" and "multiple systemic
issues" and "badly made library" and "sordid history" and "badly
reimplemented" and "wasting everyone's [who is everyone?] time"
and "what types of numbers put stress on a bignum library" even
means. And if you can't explain it, you don't know it.

Post by Sasha Toltec
Otherwise, I am deleting this account.

Why would you "delete your account"? Do you not offer consulting
services [1] or have former clients with whom you stay in touch?

Post by Sasha Toltec
As it seems like you all aren't interested in the memory error
reports or reports of failed functions and instead are only
interested in wasting time.

As stated previously, you have brought up valid points and they
are being reviewed/fixed. "Wasting time" is your hurling of
insults at the project and its author and not being clear about
the actual issues or how to reproduce them until we asked.

To treat others with respect, dignity, and patience, is arguably
more important than any technical know-how one could possess.
You would be wise to approach others in this manner.

Z

[1]: http://lists.landley.net/pipermail/toybox-landley.net
/2018-August/009635.html

Ivo van Poorten

2018-08-28 20:36:53 UTC

Permalink

Post by Zach van Rijn

Post by Sasha Toltec
Otherwise, I am deleting this account.

No idea about the difference between an account and a mailing-list
subscription?

Post by Zach van Rijn
Why would you "delete your account"? Do you not offer consulting
services [1] or have former clients with whom you stay in touch?

Expect ***@gmail.com. Or ***@gmail.com. If he really
deletes his "account" :)

--Ivo

Rob Landley

2018-08-28 21:50:22 UTC

Permalink

Post by Ivo van Poorten

Post by Zach van Rijn
Why would you "delete your account"? Do you not offer consulting
services [1] or have former clients with whom you stay in touch?

deletes his "account" :)

If necessary I can block Graff's current account and switch moderation on for
new accounts. I think he's made it clear by this point he's not interested in
(capable of?) being helpful.

Rob

Ivo van Poorten

2018-08-28 18:25:45 UTC

Permalink

On Tue, 28 Aug 2018 00:35:38 -0500 Sasha Toltec

Post by Sasha Toltec
naive algorithm

[...]

Post by Sasha Toltec
The only time subtractive chunking is used is by school children.

[...]

Post by Sasha Toltec
You (both?) seem to have no idea how to force errors out of a badly
scaled irrational function so I supplied an example (though judging
from the first example I gave, you (both?) will just lie/boast and say

[...]

Post by Sasha Toltec
This command takes 25 seconds using your math (if you want to call it

[...]

Post by Sasha Toltec
sordid history. [...] and then badly reimplemented.
Please just fix your fix your "implementation" (if one can really call
it that) and stop wasting everyone's time.

*Slow clap* nice way to express yourself and expect people to help you
out.

If out-of-memory causes your whole system to crash, you have other
problems.

Also, really good publicity for your "company"

Post by Sasha Toltec
Toltec Enterprises

--Ivo

Rob Landley

2018-08-28 19:26:20 UTC

Permalink

Post by Ivo van Poorten
If out-of-memory causes your whole system to crash, you have other
problems.

Out of memory is bad, and the kernel guys keep fiddling with the OOM killer so
it doesn't reliably trigger. (I had to power cycle my netbook over the weekend
because it had gone so deep into swap thrashing it wouldn't let me ctrl-alt-f1
to "pkill -f renderer" for over an hour. The mouse cursor position updated twice
while I was waiting, but that was it. Wheee.)

That said, "sed H /dev/zero" can blow up pretty big pretty fast. There's lots of
trivial ways to do that from the command line. (Anything that calls readline()
on /dev/null since NUL isn't a newline...)

The email itself said that you need to know a lot of the guts of the thing to
trigger a problem, I.E. it's high unlikely to hit the unaware. Yay fixing it,
but as priorities go? Pilot error and ulimit -m exists.

Rob