Refinement to click selection and feedback #26

New Issue

glen · 2024-11-27T05:01:18Z

glen commented

2024-11-27 05:01:18 +00:00

After #25, you can click a graphical view of the assembly to select elements. However, the initial algorithm thereby implement intersects the ray from the camera through the click location with the elements, and (only) the one that intersects closest to the camera is the target of the selection. That can be frustrating at times, for example with the screen looking as in the accompanying capture, there is no way to click to select "Phobos" (the small teal sphere) or "Ursa Minor" (the medium-sized yellow-green sphere that intersects the orange one). My personal feeling is that a click near the apparent centroid of Phobos, certainly, would be intended to select it, and should be interpreted as such by the principle "there are so many other places to click if you wanted to select the big blue sphere ("Pollux") in front, why would you click right on Phobos if you actually wanted Pollux?". The situation with Ursa Minor is a bit more ambiguous, but I would still say that a click on the brightest triangle of Ursa Minor should be interpreted as selecting it.

These observations mean that we will have to experiment with other, more sophisticated selection inference algorithms. For example, one might try to have a map from pixels (or other 2D geometric regions) to all elements that are under or nearly under those regions (and hence are conceivable selection targets there), and choose based on something like bayesian inference: estimate for each element X the likelihood a visitor would click there if they wanted to select X, and choose the maximum likelihood X; this would likely boil down to some ratio of areas that they could conceivably click on for X as opposed to other elements Y that are also possibilities for the click being interpreted. There might also be more local strategies with similar advantages over the reasonable default that #25 implemented, such as take the list of all of the conceivable candidates on the click point, and then also intersect with families of points on rings surrounding the actual click point, and take the candidate that leaves the list of candidates on the ring of smallest radius. (Intuitively, the entity you had to be "most careful to click on" at the point of the actual click.) The advantage of this latter algorithm, if it is comfortable in actual use, is that it is likely to immediately deal well with points and lines when we add those as elements and thereby potential click targets.

Aaron raised the point that an advantage of a simple rule like the one in #25 is that a visitor may be more likely to pick up what the rule is and thereby be able to more easily predict the outcome of a given action (e.g., a click on a particular pixel). Glen provided the counterpoint that if you can find a more complicated algorithm that is "right" (in the sense of matching the visitor's intention) a sufficiently high percentage of the time, it doesn't matter how complicated the algorithm is, so there is benefit to trying to use the considerable processing power of the computer to model/adapt to the human's interaction mode and intentions.

To that discussion I would like to add that we could implement a feature that would help, possibly on both sides of the above point/counterpoint, which is to always "semi-highlight" in some fairly low-key but still distinguishable way, any element on mouseover; then there is zero surprise when you click: whatever element was "semi-highlighted" will be the target of the click. I think, if it doesn't make the display too visually busy, that this sort of mouseover feedback will enhance any method of selection, and should in any case be implemented.

After #25, you can click a graphical view of the assembly to select elements. However, the initial algorithm thereby implement intersects the ray from the camera through the click location with the elements, and (only) the one that intersects closest to the camera is the target of the selection. That can be frustrating at times, for example with the screen looking as in the accompanying capture, there is no way to click to select "Phobos" (the small teal sphere) or "Ursa Minor" (the medium-sized yellow-green sphere that intersects the orange one). My personal feeling is that a click near the apparent centroid of Phobos, certainly, would be intended to select it, and should be interpreted as such by the principle "there are so many other places to click if you wanted to select the big blue sphere ("Pollux") in front, why would you click right on Phobos if you actually wanted Pollux?". The situation with Ursa Minor is a bit more ambiguous, but I would still say that a click on the brightest triangle of Ursa Minor should be interpreted as selecting it. These observations mean that we will have to experiment with other, more sophisticated selection inference algorithms. For example, one might try to have a map from pixels (or other 2D geometric regions) to all elements that are under or nearly under those regions (and hence are conceivable selection targets there), and choose based on something like bayesian inference: estimate for each element X the likelihood a visitor would click there if they wanted to select X, and choose the maximum likelihood X; this would likely boil down to some ratio of areas that they could conceivably click on for X as opposed to other elements Y that are also possibilities for the click being interpreted. There might also be more local strategies with similar advantages over the reasonable default that #25 implemented, such as take the list of all of the conceivable candidates on the click point, and then also intersect with families of points on rings surrounding the actual click point, and take the candidate that _leaves_ the list of candidates on the ring of smallest radius. (Intuitively, the entity you had to be "most careful to click on" at the point of the actual click.) The advantage of this latter algorithm, if it is comfortable in actual use, is that it is likely to immediately deal well with points and lines when we add those as elements and thereby potential click targets. Aaron raised the point that an advantage of a simple rule like the one in #25 is that a visitor may be more likely to pick up what the rule is and thereby be able to more easily predict the outcome of a given action (e.g., a click on a particular pixel). Glen provided the counterpoint that if you can find a more complicated algorithm that is "right" (in the sense of matching the visitor's intention) a sufficiently high percentage of the time, it doesn't matter how complicated the algorithm is, so there is benefit to trying to use the considerable processing power of the computer to model/adapt to the human's interaction mode and intentions. To that discussion I would like to add that we could implement a feature that would help, possibly on both sides of the above point/counterpoint, which is to always "semi-highlight" in some fairly low-key but still distinguishable way, any element on mouseover; then there is zero surprise when you click: whatever element was "semi-highlighted" will be the target of the click. I think, if it doesn't make the display too visually busy, that this sort of mouseover feedback will enhance any method of selection, and should in any case be implemented.

glen added the

enhancement

label 2024-11-27 05:01:27 +00:00

Sign in to join this conversation.