#MozAcademy and Engagement

Greg McVerry

Greg McVerry is a teacher, researcher and scholar at Southern Connecticut State University.

You may also like...

4 Responses

  1. Adam Lofting says:

    Hi Greg,

    Thanks for reading my blog and for digging into so many of the details. It’s extremely valuable to have another pair of eyes on this.

    I think you’ve pulled together our overarching conversations in a really clear way and called out the main challenges we’re working on.

    I’ll comment on some of the metrics pieces, as that’s where my head is…

    * The Webmaker legacy tools work well in face-to-face teaching scenarios (i.e. in tandem with Webmaker events) but we also have many people who come to Webmaker independently (through Mozilla referrals and elsewhere). The existing retention metrics are so low because of the combination of these two things: (1) lots of people find Webmaker and think it’s a tool they can use on their own to make and learn and sign-up (2) that’s not what we have been building in the past. Making a remote-learning product ‘sticky’ is tough, but we’ve made retention (aka sustained engagement) a KPI for that reason. If we get this right it’s a way to engage deeply with hundred’s of thousands of people over a long period of time in the nuances that make the open web so great.

    * As you note, The Learning Networks data points are still being worked out (along with branding and naming challenges which make all of these conversations a bit more confusing in the meantime). But overall, the KPIs we’re optimising for are all about people, and building sustained relationships with them. The details we’re working out now for metrics are to ensure that what we’re counting meaningfully reflects what we care about. The KPIs are the ‘means’ rather than the ‘end’.

    * For social metrics. Actually, no, I’m not tracking this right now. There are things to learn from this data (I’ve worked a lot with it in the past) but it’s a secondary priority in relation to direct engagement through the tools and programs. In short, lots of people talking about the product won’t make up for people not using it, and there are lots of ways to game social metrics without having real impact on the goals. That’s not to dismiss it entirely, but just to say it’s further down my priority list right now. And on that note, the ‘metrics team’, for the Foundation at least, is just me 🙂

    I hope that’s all useful context, and think it’s already in line with your understanding. And I’d love to keep this conversation going throughout the year.

    • Greg McVerry says:

      Adam,

      Thank you for the comment. Your work is so fascinating. Especially the A/B testing post. Had no idea what a few words in a snippet could do.

      I was looking the metrics through the lens of the #MozAcademy so the view on retention is flavored by my last three years.

      The Webmaker legacy tools work well in face-to-face teaching scenarios (i.e. in tandem with Webmaker events) but we also have many people who come to Webmaker independently (through Mozilla referrals and elsewhere).

      It looks like then we need a plan to focus on the conversion rates of visitors to users. I saw that metric in the dashboard.

      The tools do work in tandem with the clubs. Learning was always at the center of the design. I love the new webmaker prototypes. They will get the mass market making. It will be up to us to figure out how to apply them to teaching. These seems like a fundemental shift.

      The existing retention metrics are so low because of the combination of these two things: (1) lots of people find Webmaker and think it’s a tool they can use on their own to make and learn and sign-up (2) that’s not what we have been building in the past.

      The legacy tools have issues. The gallery didn’t make learning pathways noticeable, the teaching kits weren’t easy to find and there were 14,000 different places to go. Every visit to webmaker could end in a different reader experience.

      I still love the tools. Thimble taught me everything I know about the web. A year ago, literally today, I tweeted the most basic question about HTML and CSS. Now I can mess up web pages in ways I never thought possible.

      I was able to learn by myself because I stumbled into a community. I can’t wait to see what we can do with the new tools. I think they need to be a major part of the UX in the Academy.

      Making a remote-learning product ‘sticky’ is tough, but we’ve made retention (aka sustained engagement) a KPI for that reason. If we get this right it’s a way to engage deeply with hundred’s of thousands of people over a long period

      Yes we will not be the first to try and make a global classroom. In fact MoFo has tried a few times already. I went back and read the GitHub issue on defining a club and a city. To count on the dashboard clubs will have to actively engaged. Once the definition is solidified I would add it to the dashboard.

      There are things to learn from this data (I’ve worked a lot with it in the past) but it’s a secondary priority in relation to direct engagement through the tools and programs

      I agree the KPIs make sense for now. Atleast until after the “softest of launches” and the breaking of ground on the Academy.

      I consider connections to be the most important content in any connected learning environment. I disagree with you that you can have folks talking about tools without using them. Tools shape human activity. More importantly I do not think they would be gained because I do not see these social metrics or marketing metrics. The impressions on social media are learning metrics.

      I believe #MozAcademy should be a distributed classroom and not a destination classroom. So I was pulling in your work to highlight a need in the Mozilla Learning Networks. In the 2-3 years I have been involved with #teachtheweb there have only been a handful of people (super deadicated people) networking. This may have just as much to do with low retention as it does with tools designed for too specific of a use case. People talking about tools, means people are using tools, which means higher retention. Twitter, Tumblr, Google+ are where people so to learn how to read, write, and participate on the web.

      I didn’t realize you are an army of one on the metrics team. Your squad rocks.

Leave a Reply

Your email address will not be published. Required fields are marked *