User testing the remix + tutorial process

For the past several weeks Shannon and I have been user testing Looking Glass at the St. Louis Science Center. We have updated Looking Glass to produce an automatically generated tutorial at the end of the remix process. The tutorial walks the user through the process of then adding the remixed animation into their own story. So far we have conducted 19 pilot studies to see if our remix process and the following tutorial work for middle school kids. This testing has been equally exciting and frustrating. There are quite a few problems with the remix process and tutorial that we will need to address and maybe some exciting new oportunities.

First, the remix process. From our testing, I'm beginning to think that the remix process is too out of context for our users. I've begun to think of ideas of how to better contextualize this process. One of these ideas leverages Michael's remixed animations that other users have already done. So if you are new to Looking Glass and remixing we give you the choice of using a prefab remix or a custom remix. Prefab remixes are simply the remixes that Michael has already captured from other users. The prefab remix interface (from my sketches) will then present the prefab remix and the user's current world side by side and allow for character selection, and then live replacement to show the users what to expect out of the process. I've also begun creating some sketches of the custom remix interface. From testing I think maybe time should play a more important role for our users. So the custom remix interface would have this timeline. The timeline (in my head) is like the current time slider with the addition of the right now pane. This timeline is pretty interactive too... since we noticed that many users do not replay the individual statements to find the right remix action... the timeline is all about playing selected statements. I'm still working on this idea... so maybe I'll have more to say about it at a later date.

Second, I want to talk about the tutorial. I'm really pleased that we were able to pull an automatically generated tutorial off for the study. But there are so many problems with the current design. We really need to re-think how dialogs and menus are handled as part of stencils based tutorials. I think this means that croquet components need to know more information about the tutorial and maybe even present themselves differently. I'm not exactly certain, at this point, what this means but I do know that having a stencil be the only thing that prevents users from making mistakes is simply not enough. I also think we need to not base our tutorial directly off the transactions. This seemed like a good idea weeks ago to make an automatically generated tutorial, now I realize that it is too limiting. It doesn't give me the flexibility to present different things to the user. It also limits how we can robustly automatically recover; recovery is done by adding a new transaction. But what if you have two recovery transactions, you don't care which happens first, right? Ah... but when the tutorial is based on transactions you cannot switch the order. So we really need to re-think how to make these tutorials as flexible and response as possible.