Comparisons

nomnoml

A system for creating UML diagrams from text representations, which could be a source of interface ideas. Noteworthy features:

• It has a very clean look.
• It maintains a text representation of the graphical view.
• The text representation can be effortlessly shown and hidden by mousing over the left and right sides of the screen.
  • This feature is tricky to discover, and can be frustrating until you figure out how it works.
• The text representation is declarative, and somewhat (although not completely) order-independent.

VSCode

A widely used Microsoft IDE. Noteworthy features:

• Integrated command palette and terminal views, which could inform the dyna3 command prompt.
• Excellent keyboard controls.

Keyboard controls

Glen's early ideas

Lightly edited by Aaron for organization and readability

I think having a solid foundation of synced declarative textual rep and graphical rep would then allow for additional interaction modes. For example, we could add keybindings, and maybe selection by typing name. Suppose there are spheres S and U, and you want to impose a tangency constraint on them. Maybe when the graphical view is focused, typing S selects that sphere. And then typing U adds that other sphere to the selection. And then maybe hitting Ctrl-T or Meta-T or something like that imposes the tangency constraint (which would then update the display and the text representation). If we have such keyboard shortcuts, they could be shown on/by the icons/menu items, or be shown when you hover over them, and maybe suggested by a floating tip when you type the command S tangent U.

Modifiers

Within a web app, marking keyboard shortcuts with modifiers like Ctrl and Meta can lead to conflicts with browser- and desktop-level shortcuts, which often use the same modifiers. As of pull request #138, we mark shortcuts with Shift. Glen has suggested that the most common shortcuts should be triggered without modifiers (which would make a mechanism for distinguishing when type-to-search and when action shortcuts are active critical, as pointed out in the next subsection).

Type to search

Typing to search for objects and commands can feel very fluid for users who find typing easier or more reflexive than pointing. Subjectively, it feels like It might demand less motion and less precision than pointing. VSCode uses shortcuts to initiate its various keyboard search features. For us, that could crucial for avoiding conflicts between keyboard searches and shortcuts, since web apps have fewer modifiers at their disposal than desktop apps like VSCode.

Feedback

It's critical that all keys/combinations that have a binding provide some noticeable feedback when triggered, even if they "fail" or or are not eligible to run for some reason, so that the person making the construction can distinguish between them and keys/combinations that simply have no binding (which likely should just silently do nothing).

Context

Likely some key bindings should be "global," but clearly others need to depend at least on what view is focused. The simplest example is that letter keys in the command listener need to type command names, whereas on the graphical view, maybe "p" adds a point, etc. There has also been the suggestion that in an outline view, arrow keys navigate the table of entities (possibly changing the selection as they do) whereas in a graphical display view, they might be used to manipulate the camera.

Possibly a binding could also depend on the selection (beyond just whether it is active or not), although I don't have a specific example use case for that at the moment.

Text representation

Glen's early ideas

Lightly edited by Aaron for organization and readability

It seems to me that any construction should have a declarative text representation, which it's easy to see and manipulate like "source code". There would be a way to create a (named) point or sphere etc. and a way to express constraints. There would also be a way to express that something is constructed/derived from something else: for example, M = midpoint(A,B) could be a shorthand for something vaguely like Point M; AM = BM = AB/2 (I am not proposing actual syntax at this point).

The idea is that the text representation and the current displayed construction would be kept in sync: there would be a system of tool icons/menus and mouse interactions/gestures that would allow you to create points, set constraints, etc. Anything you did this way would be reflected in the text representation; and of course any declarations you entered, or deleted, in the text would update the displayed construction. (In nomnoml, you can't manipulate the graphical representation to change the text, it only goes from text to graphics. So that's a really important extension of the idea in Dyna3.) We can choose whichever tool affordances we think are easier to use: some things might be accessible by icons while others by menus, or maybe we use just one or the other.

I think the text representation should be completely order-independent if that's possible, and indeed, there should likely be quick ways to sort it in a variety of ways: by order it was entered, by entity type, by entity name, etc. Or you should just be able to drag entries around as you like.

Some questions:

• Does the text representation also record current contingent aspects of the construction? E.g., the position of a point among its allowed positions, even if it is not constrained to that specific position? For example in GeoGebra, if you enter Q=(1, -2) in its text rep, then the point Q is plotted in the graphical view at that position, but it is fully draggable, and as you drag it, the text representation/definition of it changes in the text window. (I think it is choosing rational coordinates, but it is hard to be certain as they are displayed in decimal.) This has a very nice feel to it. If you want to create a point that is pinned to a specific location there's a different command you can write, e.g. F = Point({-1,1}) (there may be other possible commands). Then F is plotted at that location, but with different appearance, and it is not draggable. I don't know that I love those notations, but ideally we would have both of those possibilities in Dyna3.
• Or more generally, is the text representation the "save file" of the construction? I think if the answer is yes, then the answer to the previous question has to be "yes" as well. But then I think more has to be true: the styles/labels/visibility/etc. etc. of every element also must appear in the text representation, which further probably means that portions of the text representation have to be hide-able to keep it from becoming too busy.
• How do the derivations interact with the declarations? Continuing the example above, does M=midpoint(A,B) stay that way in the text, or does it just expand to Point M; AM = BM = AB/2? The latter would seem to make it harder to see what's going on, so it doesn't seem to be a good idea. But suppose you type out Point X; CX=DX=CD/2 -- does it automatically "collapse" to X = midpoint(C,D)? Or even to be crazier, suppose you do Point X; Segment s = AB; X on s; AX = XB -- does that also collapse to X = midpoint (A,B)? How far does Dyna3 go in trying to prove that X really is the midpoint of AB? And what happens if that's deduced from a bunch of other declarations and constraints, and so maybe then displayed as a midpoint derivation, but one of the necessary constraints is deleted, does the text representation flip back to some other description of X, and if so, exactly what? This seems like it could be a morass. Perhaps when something is defined as derived from other things, it has a slightly different status -- maybe it doesn't get its own variables, but is always computed on the fly, dragging it is implemented by temporarily adding a draggable thing with the right definition/constraints, which all evaporate after the drag, constraints on it are rephrased internally into constraints on the things it is derived from, etc. This is a perspective in which somehow M = midpoint(P,Q) makes M intrinsically the midpoint of PQ, rather than just currently the midpoint either by accident or even just by other individual constraints, some of which could later be removed, freeing up M to leave from its midpoint. If you use the derived form, then its the whole package, take it or leave it. If you decide you want M just to be equidistant from P and Q rather than the exact midpoint, you really have to redefine M; you can't just eliminate one condition from it. So in that sense a derivation wouldn't be just an abbreviation, it would be more strict than that. That approach seems less fraught.