Simplifying

After presenting preliminary designs last week, I was struggling with the fact that this process can get kind of long and unweildy. While mentors will be domain experts in programming, if they don't understand the process or the intentions of the various steps, kids may not receive helpful suggestions. 

Arising from these problems, I decided to split the Mentoring system into two mini systems, both of which are hopefully simpler and easier for mentors to use.

In the first section, currently called "Mentor a Student"  (https://docs.google.com/open?id=0B3edPjRL_GgxUjhUYzd0cE5OMjA), domain experts who are linked to a specific student can view suggestions for their mentee's worlds and make their own suggestions. While viewing existing suggestions, the system will request that the mentor rank whether the suggestion is appropriate for their child's world, which will hopefully help the community to change those rules that send suggestions to the wrong students. 

In the second section (https://docs.google.com/open?id=0B3edPjRL_Ggxdmd6QmFHc3o4T1U), any domain experts can participate, whether they know a child using Looking Glass or not. In this phase, mentors can view suggestions, choose appropriate code modifications and then write and test rules for those suggestions. In theory, this section will rely on the wisdom of multiple mentors checking over rules to create really useful rules. The two main questions in this section are:1) How can we encourage mentors to create an effective community around rule-editing / what suggestions should mentors see? and 2) what information can we give mentors to effectively test their rules?

1) In order to create some sort of consistant community, from what we have seen in Wikipedia research, it seems like we would want a set of mentors to work on a growing set of suggestions that are related and that they are interested or experts in. But how can we encourage this type of community to grow? Showing mentors the most recent or a random assortment of suggestions probably isn't good enough. Although we aren't necessarily thinking about Badges this way yet, many suggestions do recommend skills that will fit into badge categories and this seems like a logical way to groups. It also seems like rules that have been vetoed by a lot of mentors should be either in their own section, or shown higher up in the sorting of rules within badge groups. Since these suggestions are currently being shown to kids and deemed wrong, it's important to get these rules fixed as soon as possible. 

2) Testing whether a rule is too general seems to be pretty straight forward. Showing mentors a set of worlds that receive the suggestion will show them cases where the rule passes, but the suggestion is not applicable. However, when a rule is too specific, it's harder to show the cases where a rule should have passed but didn't. Depending on how the rule is written (or how the API functions), we may be able to easily remove a constraint at a time and show those results, to see if the new worlds actually also fit the idea of the suggestion. If it can't be done automatically, we may also be able to allow mentors to do this process manually, or maybe there is even a better way to test a rule for being too specific.

Thoughts for far in the future: Although I'm focusing on the interface for the mentors at the moment, in the back of my mind, I sometimes wonder how we can present the suggestions in a way that middle school students will actually want to use. There are two issues: kids may not want to use hints that change the output (which is ok), but sometimes it might teach them a lot if they were motivated to be flexible and learn the suggestions. In other cases, where the mentor has improved the code quality but hasn't changed the output, kids might not care to take the time to make a change that doesn't seem necessary. Obviously, we don't want to create a recommendation system like clippy that just annoys our users. Thinking about badges, however, gave me a possible idea. I don't know if there are any current plans for badges in the IDE, but rather than have the suggestions seem kind of random, kids might be more excited if they were "hints" that helped them get a badge they want. Essentially, there would be something that shows a badge that is a level just above where they are and then has an option to get a hint for the badge. The hint would be a mentor suggestion that teaches them one of the skills they need for a certain badge. This would definitely need user-testing to determine if kids would use "hints" more than random suggestions, but I'd be interested in whether there are any other IDE plans for badges and whether it might make sense to add suggestions there.