Croquet Transactions and Steps

I’m currently in the process of trying to write a paper about the automatic tutorial generation work that Shannon and I completed this summer. So far progress has been going fairly well on the paper, but my user studies have me slightly worried. I currently do not have enough participants to see a clear trend with how differently users perform with an automatically generated tutorial. While I wait for the rest of the data to come in from the user study I’ve been trying to nail down how to describe Croquet in our paper. For those of you who are new to Looking Glass, Looking Glass is built with our own custom toolkit called Croquet. The Croquet toolkit is extremely aware of everything that is happening within an application because it records everything that the user has done. Further, Croquet is also aware of the relationship that different actions in the interface have, this is what enables us to produce automatically generated tutorials.

For this paper, it isn’t necessary to describe the entire design of Croquet. I only need to describe how we produce automatically generated tutorials. However, a big piece of generating tutorials is the transaction history. The transaction history is the log of everything that has happened while the application was running. This log is very detailed; it has enough information to completely repeat the exact same series of steps verbatim.

The problem I’m having with describing the transaction history in our paper is we have, what I perceive to be, inconsistent treatment of data in the transaction history. The transaction history has many transactions. Each transaction is a complete unit of work completed by the user. The unit of work is some change (interface state or user edit).  However, some transitions have prep steps, such as menu selection. Previously we defended this practice because the prep steps “aren’t really complete units work”, they lead up to a complete unit work or change, prep steps are really a dependency because you need them to complete the transaction. However if we look at how the transaction history is generated in practice the only thing that is a prep step is a menu selection and dragging. But if we look at a dialog box for variable declaration, we have three dependencies: type, name, initial value. Each of these is a nested transaction within the parent transaction. So these three pieced of data should be prep steps because they are the dependency to complete a single change. So then why is menu selection have prep steps but dialogs have nested transactions? The reason for both is the same… there is a dependency.

Further, almost all transactions, except for those with menu selection and dragging, only have one step. They each have a completion step. Even the variable declaration only has one step, but several nested transactions. I’m now fairly convinced that we can radically simplify how we described Croquet if we remove the complexity of steps from the transaction history. The transaction history need only have transactions, some nested to support dependencies. But I no longer see any reason why menu selection should be a prep step when it can fit into the model of nested transactions perfectly well. At this point prep steps and completion steps seem extra complexity that doesn’t add any value.

At this point, I feel pretty strongly about this. I think I can write about it fairly well in the paper too. The problem is I can’t make this type of decision on my own and I need to keep making progress on the paper. Plus, I know that making this type of change will be a difficult sell… even though I don’t really see the value in the extra complexity of the current design because it seems arbitrary when we use prep steps and when we use nested transactions. At this point prep steps really appear to be an exception to the rule, not the norm. Hopefully I can avoid menus in this paper that way I won’t need to explain this odd exception to the rule.