feat: define parse() function that takes a string to a Node tree #54
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "nodes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This POC is built on top of the #52 grammar. It adds a traversal of the concrete syntax tree produced directly by the parser_combinator grammar to translate it to a Node tree along the lines of mathjs parse().
This approach is going well, and shaking out some minor bugs in the grammar. Note that I am converting to a streamlined collection of nodes. For example, a conditional like
test ? yes : nois just converted to an ApplyNode ofcondto[test, yes, no], and ApplyNode itself is a hybrid of FunctionNode and OperatorNode (as has long been planned for mathjs).As of this comment, all that remains is to provide abstract syntax conversions for the Range and Assignment rules (and also, only single-entry Blocks are currently supported; we shall see how far we can get in the testing without that).
Ultimately, it should be just fine to convert an instance of Range to an ordinary ApplyNode, and even Assignment (e.g.
assign(x, 7)is a perfectly reasonable abstract syntax forx = 7, and I don't see why we can't also supportassign(f(x,y), x*y^2)as well. However, we will likely need a BlockNode for multi-entry Blocks, as there seems to be more structure going on than just a call to some notionalblock(expr1, expr2, expr3)function, at least insofar as collecting the return value.Once those are done, it would be nice to get through all of the tests in mathjs/test/unit-tests/expression/parse.test.js, but there is one oddity. That test file currently tests both the parsing and the evaluation of an extensive catalogue of expressions. It would actually be better to test these two aspects of expression handling separately, especially because of cases in which two different parse trees for the same expression could lead to the same value, but we actually only want one of the two parse trees. (This has explicitly come up in the review of PRs to mathjs, in which supplied tests didn't actually disambiguate the possible parses of some expressions.)
I don't think that there is a bank of expressions and corresponding parse trees in the mathjs test suite. Therefore, the approach would be make the
linearize()function on a Node that I've thrown together for the testing slightly less clunky, and then just go through the bank of expressions in the parse.test.js and write down all of their linearizations, so that we can test the parse trees directly going forward. That seems like worthwhile effort. It's a big job, we'll see if I get through it.Although I have Assignment, it doesn't do anything special for functions. So next I plan to switch to an explicit
lambda()operator for defining functions and make it the result of parsing(x,y) => x + 3*yfor example, and then make an assignment to a function call be syntactic sugar for assigning the corresponding lambda-expression to the symbol of the function name. That should be the last thing before adding Block. (This change will make the nanomath expression language a non-breaking extension of the current mathjs one.)The expression bank has uncovered a lexing error: the
#foocomment is getting lost when lexing"#foo\n#bar\n". It's because comments are not allowed on newline tokens, and then it gets overwritten. Instead, they should coalesce separated by a newline.Multiple comments now coalesce as desired
:toirangeto indicate inclusive rangeView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.