To many in higher education they see problems as nails and testing as the only hammer in our kit. Yet we stand at a time where we can re-mediate assessment using new technologies and old definitions of what it means to learn.
I serve on the Tech Fluency (TF) affinity group at Southern Connecticut State College. TF is a tier one competency in our liberal education program (our general ed program but with more hoops and loftier goals). The goals is to ensure all students have the minimum tech skills they will require after college. In our current (permanent) budget crisis we have been asked to review the effectiveness of LEP.
We developed a series of rubrics instructors could use in their classroom. The newly appointed LEP assessment committee decide our approach was “too subjective.” They suggested a common task, filling out a spreadsheet was their recommendation, that students could complete in a controlled and supervised environment.
This set me off. I responded (probably with not enough to delay and too much acerbic snark) with some of the following comments.
Objective Assessments are a Hoax
Subjective flickr photo by EVRT Studio shared under a Creative Commons (BY-NC-ND) license \
I am not anti-testing. Most of my research revolves around item design and testing. Yet I think what set me off was the belief that some assessments are objective. Those who rely on standard measures ignore the bias inherent in statistical models and deciding as “what counts as learning.” They look at rubric scored items as being “too subjective” yet ignore the error variance, the noise in their models. I say bring the noise. It is in outliers where we see interesting methods and learning.
Technology Assessment Lack Ecological Validity
We were asked to create a shared assessment. It’s just the task sounds like 1996 wants their Computer Applications textbook back. We have to move beyond, “These kids don’t know spreadsheets” as the only critique in our self-assessment. There is so much more in the competencies beyond the basics of Excel.
The idea that you do anything in tech under supervision and sitting alone in a crowded room is the wrong approach to assessment. What we are calling cheating will be required collaboration for anyone doing any thing with tech in any field.
This is why I think a digital credentialing platform is the correct path forward. If you begin by mapping pathways and rubrics similar to ours or better yet even more fine grained criteria we could develop a system where faculty still had the freedom to design (hopefully co-design) a pathway for students.
Students Should Drive Assessment
I think we should involve the students as stakeholders to a much higher degree in any assessment.We do this by helping stduents tell their story. This also shifts responsibility onto them to build the data trails we need.
Purpose of Assessment
I also took issue with the the purpose of our assessment. If the goal is to evaluate the effectiveness of our LEP program then why is our gut reaction to assess each student individually.
Its not that the approach of of a learning artifact (the spreadsheet assessment we were asked to develop) is the wrong path in terms of overall measurement.
Let us as faculty assess the individual and let machines surface the patterns at class, school, and system level.
I think the Academy long term should push off most system wide assessment onto machines. Its way more effective and correlates so highly with well trained human raters.
If we scored could score a batch with high inter-rater reliability once, and laser honed the criteria, much of this could be machine scored and credentialed with minimum faculty involvement. Faculty could build whatever assessments they wanted to on top of the task. It really wouldn’t matter to LEP assessment.
This isn’t fantasy. It’s usually $7-10 a user (for the scoring).
All measurement and all grades are subjective. Yet I think we have a chance to rethink the academy by empowering learners through assessment. Its time to kill the Carnegie Credit hour.
In fact across the state of Connecticut we have been discussing how to seamlessly transfer students between seventeen community colleges and four universities. Plus students would like to receive credit for work that would demonstrate competencies in our Tier One classes. If we really wanted to think about Transfer Articulation we would forget about tracking credit hours and think of each student as an API. If we had the matching criteria, or even a crosswalk of offerings, it would be a matter of plug and playing the assertions built into our credentialing platform.
This would also allow students to apply previous work they completed in high school or outside of school and get credit for meeting the technology fluency competencies. We can use the new endorsement feature in the Badge 2.0 specification so local schools, computer clubs, or even boot camps could vouch for the independence of student work. The learning analytics can help us with our programmatic review and tracking student knowledge growth.
There will be no numbers just criteria and evidence.
The Academic Blogger Pathway
Students accomplish the pathway by earning badges. Students must provide evidence of their growing capacity as a writer. There are nine badges to earn before a student can earn the academic blogger credential.
A “Hello World” credential with critical importance. It takes quite some time to onboard the entire class and teach them how to share a url to a post. About 50% will share a link to their WordPress profile.
This badge I want to get at the organization of thought and the the presenting of thinking. I will award to students as I see their work grow. When students begin to blog I may get one short paragraph and then a few post later one long paragraph. I am okay with this. Focus on the on boarding early. Overtime I will reword this badge when I see greater cohesion of work. Students can nominate each other for this badge,
This class fils a Writing Intensive requirement. Therefore revision (as ifit isn’t always important) is critical to your success. This is a self nominated badge after a student makes three substantial revisions.
You can’t teach writing without community. I will give this badge out (with scarcity) to those who contribute to meaningful conversation through commenting. This can be on a person’s blog or a class stream.
How Will You Apply?
I want the burden of proving excellence on my students. They should be nominating each other and them selves. The credentialing platform I use badgr.io doesn’t allow for submissions just the issuing of badges. So in the next few days I will create a Google to nominate people.
Many of us rely on the generosity of others as we search the web for all of the images used on our blogs, keynotes, and slide decks. Passionate artists and makers often share their work using a Creative Commons license that allows us to openly use and remix the work of others. As committed people supporting the open web we must provide recognition to these folks if we want to defend the Commons.
A community of photographers have blossomed on Flickr who share our mission and now millions of photos, openly licensed for re-use, exist for us to search. Luckily Alan Levine created the flickr cc attribution tool. Using this powerful bookmarklet tool you can search Flickr for Creative Commons images and then get the proper code (mark up) to embed images on the web. You also get a text based attribution when using the images in slide decks or for when you give image credits at the bottom of a web page.
It is a vision of the decentralized LMS I want. The website is part of the #Helloweb campaign, a Mozilla project to teach web literacy across rural India.
I like the site. It is actually super easy to do, but looks good. Just the grid view of embedded Twitter timeline. A stream that is pleasing to the eye. Pretty is important.
When I see it and think about other recent efforts I feel like I am getting close to the decentralized LMS I want.
I want to display Twitter cards, and Featured Images plus blog headlines through RSS. Maybe even annotations. Most importantly I want to empower learners to publish on their own site and control their privacy by deciding how to share. I want my students to own their data and learn on the open web.
The twitter timeline on #HelloWeb isn’t my only snapshot into possibilities.
Alan Levine is working on WordPress plug ins that will use Feedburner to suck up participants work:
Alan built a site with imple navigation to activities. Plus an active stream of blogs, videos, and Twitter. I would want a link to the blog stream and the comment stream.
I can’t wait to see what Alan is working on.
I also need a good teacher dashboard. What hypothes.is (I am beta testing their new Stream) built is pretty to close to all I need. I blacked out my student names but I can click through and see all that they are doing.
I now have access to a dashboard of my students. The navigation in the group features make teaching easier. I am finding hanging out in texts is sometime better than talking about texts in comments.
The new hypothes.is Stream is almost everything I need to teach in a decentralized classroom. In fact if you built your decentralized LMS around LMS you would have everything you need.
A few more thing I would like to have would be RSS driven. This would include include participants posts and comments.
I feel like what I envision as my perfect decentralized learning environment is here. I want the hub of any course to be dynamic but not a place that collects (beyond RSS) user data. People shouldn’t have to sign in.
I want them to own their own learning, or if they do not have a blog or website, be able to choose their own network to connect through.
However I do believe the chat stream to be an essential elemt to the modern LMS. Yet I do not want to dictate to people which river to choose. If your community is active on Twitter use that. If its Telegram or Facebook messenger that is fine too. Try Slack, Rocketchat, or gitter. Doesn’t matter.
I am intrigued about how AI and machine learning will interact with facilitators in course design. I do not fear these robots as my other educational technologists do. They will be able to surface cool patterns. Patterns that can be made into pretty graphs. people like pretty pictures.
What I still need:
I need to make the syndication of particiaptns blogs, websites and hashtags to be a one button push. For Twitter I use Martin Hawksey’s Twitter Archive for Google Sheets. Be cool to embed that.
For participant blogs I want a field to just enter in their url, (for the blog, category, or tag for advanced bloggers) and have them added to the stream and the faciltator profile that tracks number of posts, comments.
I want to add their hypothe.is ID. Really I may just get away with using the new hypothes.is profile as the only teacher dashboard I need for now.
In a recent post on my backstage blog (I use it to describe process or half baked thoughts) I explained my excitement for #HipHopEd in order to gather some song lyrics. I am pop culture deficient yet carry some knowledge of early hip hop. I got there on a strange path long after the original tracks dropped..but that is another story.
I was excited for the chat just to archive cool crowdsourced lyrics. I would have the chance to grow a collection from people who recognize the literacy practices of all youth while recognizing that education is central to social movements.
Our Gear Up students, are entering the eleventh grade–a crucial year for the college bound. This year I want to transition our text message outreach from parents (who will receive reminders and event updates) to students. We Use Remind. I will set up new classes for youth. I am thinking a motivational quote throwdown will be a fun activity.
Quotes make great memes and memes make great tools for learning basic HTML and CSS. In fact I shared a template I had remixed from Mozilla Learning. I hope to use these during #EDU106. o I decided to make on the fly. As I saw quotes I made a series of remixes for participants in the chat.
In each remix I used a creative commons image from Flickr. Using Alan Levine’s Flickr CC Attribution helper I was able to use a legally licensed image while providing some amazing artists credit. In the train picture I actually remixed two images. It isn’t much of a composition. I was constrained by time as I was trying to make a few memes while also participating in the chat.All I used was the eraser and the opacity tools…Still looks pretty cool. I may keep working on it.
I hope others in the #HipHopEd community takes up the call to #teachtheweb. Hip Hop has a long history of digital arts. We need to grow this legacy while getting mroe students of color to code. Do I believe every kid should code. No, but everyone should have the opportunity to explore their identities across the web while mastering skills that can open up opportunities that have been closed off for the first three decades of our online world.
This summer we hope to unleash a new wave of webmakers on the Elm City. For the first time our 11th grade Gear Up students will take college level credit bearing classes. We will start with EDU 106: New Literacies for Life Long Learning.
The hybrid class will focus on telling the story of us. I have a syllabus, but I am not a fan. I want our learning to unfold in real-time…to be user driven. In July we will have three days a week together for three and a half hours a day. The other two days students will attend college visits and arts based field trips. They will have to choose some digital documentation tools.
I want to create and curate curriculum with the class. Last fall in our Gear Up Academy we focused on Beyonce and Formation. Students learned about the local history of the Black Panther Party in New Haven and wrote letters to the Miami Fraternal Order of Police who called for a boycott of Beyonce.
The readings will be based on what I have chosen but I hope to add , or better yet have the class upvote, a reading each module. I am toying with just picking something from Medium each week.
I want to tell the story of us. More importantly I want our GearUp stories to share their story. Last year we worked on college essays for the common app. Each story was unique. A tail of trials and triumphs. Many shared stories of being first generation college students, or of trying to make it through high school while being the primary breadwinner and caregiver to their siblings. There were stories of surviving abuse and the horrors of growing up in the system.
Yet everyone endured. They have made it this far. They have taught me so much.
We will be using our new collaborative learning space at Southern. Our Dean Stephen Hegedus, came to campus and immediatley ripped out the computer lab. He replaced it with individual networked stations and a state of the art 360 camera system.
I also purchased a Tricaster mini and a few black magic cameras. Be awesome to get some podcasting going.
Every student will maintain a domain of their own. We have used Known as our basic platform. I think I will stick with the tool.
Elm City Webmakers
Leveling up the digital skills of the youth in New Haven is a passion of mine. If diversity in tech is an HR problem it is already too late. We have been learning basic html/css skills for the past three years as a Mozilla Club.
I recognize not everyone is a maker, especially a webmaker (Students can also choose a music production class to fill the technology fluency component). So the tech doesn’t really matter. I want the web to be the paintbrush students can use to paint their world.
If anyone in tech in the Greater New Haven area would like to get involved please reach out. Urban education is not a school thing. It is not a parent thing. It belongs to the community. We must work to make the city our campus.
The crowd beamed with pleasure as each team crossed the stage. One by one, cheering on each other, they took the microphone to share their projects. Four teams of young women at our first annual #femhack had two minutes to explain the problem they were trying to solve and how they would solve it.
We had twelve participants from around Connecticut.
We began the day with an icebreaker. It was one I poached from the We Are LRNG conference. You basically ask people to draw their social media profile on a post it note. It works and can be fun. Plus you get a sly little name tag next to participants.
Only one of the teams had any previous coding experience.
A few of the young women attended our YouthZone sessions the previous day where we used Mozilla’s X-Ray Goggles to “hack” school websites and Mozilla’s Thimble to make memes. Students learned about different HTML tags as they remixed the the images, headings, and paragraphs on he pages. During Youth Zone one participant remixed an animal shelter website where her parents used to work. She made them the world’s best animal rescuers.
The other participants had never peaked behind the curtains before so we started Saturday with Thimble and the Letters#2nextprez campaign. The participants all shared their ideas on how to get more women into tech.
They then had the rest of the day to work on their projects:
Diversity in tech will not happen at Google, Twitter, and Facebook. We will not solve this issue in Silicon Valley. The work must happen in our cities , schools, and community colleges. Our networks grown in churches and Girl Scouts. In programs like Gear Up. Once diversity is an HR issue its already too late.
We must remember that America NEVER scored the highest on any international assessments of learning. Expansion through genocide and centuries of slavery leaves long lasting scars…but the United States of America WAS always the place the best and the brightest wanted to be. We must preserve this uniquely American advantage. Yet the xenophobia rhetoric in todays’ politics is a direct threat to our economic future.
Most importantly they lit a path for the young women to see a future in tech where no voices are silenced or lost.
Campaign Core- This team created a place for students who want to get involved in different social justice campaigns. It is a database of social media accounts . The goal is to connect users to topics that matter. There are also lessons on running effective social media campaigns for change.
EZCode– This project is designed to teach kids basic HTML. There are flash cards that have questions one one side and directions on the other. There is also a directory of places you can learn to code.
RPGgeniuses- This was a role playing game that had kids learn HTML and CSS. Students were given a guide, completed scenarios where they learned lessons, and then would complete a quiz.
Danbury Girls Who Code– The Danbury Branch of “Girls Who Code” designed a website for their club to recruit members,
I think that leadership may be mostly a commitment to the constant mediation and care required by love, that place where both individuality and relationship must assert themselves and somehow walk and dance together.
I have spent the last few months thinking about the role leadership plays as we transition our learning into the Open. I keep coming back to this definition from Gardner. He was responding to a quote pulled from Leadership on the Line: Staying Alive Through the Dangers of Leading by Ronald A. Heifetz and Marty Linsky. In that quote the authors go on to describe many of the acts we see in open learning spaces:
And every day you must decide whether to put your contribution out there, or keep it to yourself to avoid upsetting anyone, and get through another day. You are right to be cautious. Prudence is a virtue. You disturb people when you take unpopular initiatives in your community, put provocative new ideas on the table in your organization, question the gap between colleagues’ values and behavior, or ask friends and relatives to face up to tough realities.
Leaders Who Teach Me
This made me realize when we create opportunities for open learning we create leadership engines. I see this in the actions of so many people I have come to know and respect through a variety of open learning initiatives. There are too many to list. So I will share the next three that come across Tweetdeck:
Maha really needs no introduction. I got to know her through connected courses, and rhizo15, and a variety of other online learning experiences. She, along with Rebecca Hogue, founded virtually connecting. This online space connects on the ground conference participants with virtual buddies. This has expanded conference access to thousands. Maha and Jennifer see this as an issue of advocacy and equity. Conference attendance comes from a place of privilege. The learning that occurs there should not.
Laura needs no introduction…if you are one of Google+’s seven users. I kid Laura, but in actuality no on has pushed my teaching in the open through RSS more than Laura (Jim Groom and Alan Levine aside). I met Laura first on Google+ and maybe through #ds106.
Kevin, much like Terry Elliot, has always pushed my thinking on the boundaries of storytelling. I actually met Kevin IRL before following his every move online. He has inspired me to rethink what is possible when we allow children to play with meaning.
I have met so many awesome leaders during my journey through open learning: Doug Belshaw, Verena Roberts, Karen Fasimpaur, Emma Irwin, Sarah Honeychurch, Anna Smith, Simon Esnor, Lee Sakallerup, Laura Hilliger and everyone in our Slack channels. They commit as leaders through learning. Through teaching they bring “the constant mediation and care required by love” to help ensure the Open Web remains a place where we can feel safe to put ourselves out there.
I try to make my little donation back to Open teaching through a collective project called #walkmyworld.
Teaching Leaders Teaching Me
I grow as a leader and learner through #walkmyworld. In this annual social media gathering we come together to explore the boundaries of texts and identities. We complete a series of ten learning events and share them with the world. What I have realized is we have also created a pathway to leadership development.
A great turn of events. I now sit back and learn while others lead. Many of our first participants now organize #walkmyworld.
Kate is one of the most amazing teachers I have ever met. She works in Australia and has participated in #walkmyworld since 2014. She has helped to organize the last two iterations. In looking back at #walkmyworld Kate notes:
I have always felt comfortable experimenting with new digital tools but participating in this project and becoming an organiser has certainly enriched my pedagogy and self-confidence as a teacher. I now have access to a number of ‘literacy experts’ and colleagues that I feel comfortable enough to be quite open and honest with. They support and validate what I am doing, not only with the project, but also with my class and the experiences I am creating for my students.
In her post Kate goes on to note that #walkmyworld has improved her classroom management and what Ryan Rish calls, “pedagogic imagination.”
Stephanie credits #walkmyworld in shaping her journey into a doctoral program. She took part as local node when Ryan Rish used some of the learning events in his class. She notes:
It was the first iteration of the alternative MOOC-like-thing. The learning events and poetry of Robert Hass made an impression, but for me, the connection between students and professors and teachers across the world was both fascinating and exciting. I barely recall the events of that year, but I do remember the people.
Fast forward several months to an unexpected invitation to collaborate with the creators for the second iteration. I was still working on my M.Ed., but suddenly I felt like a valued member of academia. Considering I was just over a year from being told that my background wasn’t good enough to teach high school, I was floored. And excited. I had something to offer that someone actually wanted. It doesn’t get much better than that for someone like me.
Katarina organizes #walkmyworld after being what she called a “rogue participant” last year. She is drawn to the scholarly aspect of an open research project:
I think technology and the multimodal composing that I’ve done in #Walk and in other projects since has been critical for me really getting to the heart of concepts like visual literacy, modal affordances, and where modes are present in everyday life without digital technology.
She goes on to explain that research community around #walkmyworld has lead to new creative activity:
I’m not teaching a class right now, but I’ve done a case study of the planning and implementation of #WalkMyWorld 2015 for both the LRA annual conference and a class paper around situated learning and communities of practice. I am thinking that #WalkMyWorld 2016 will pair nicely with my New Literacies class this upcoming semester!
Good open communities create pipelines of participation. Leadership emerges from learning. I know when I look back at #walkmyworld I am thankful for the lessons that people like Kate, Stephanie, and Katarina have taught me. I am proud to say we all walk and dance together.
The end of the semester, unlike the winter weather, has rolled on in. Similar to meteorologists we teachers have to use tools to gather data and make predictions. On their own these readings do not describe much. A climate is so much more than a thermometer reading. Taken together, however, we hope multiple measures of performance can demonstrate some growth in learning.
At the Mozfest session on Thimble, however, I felt we wanted to focus on the tools for measuring. Instead we need to consider the climate of space that allows learning to thrive. Many of the teachers who attended the session on Thimble immediately requested features that we could loosely label, “testing mode.”
Thimble should not have a testing mode. Any teacher can remix an activity or a teaching kit to have learners move from blank page to published product. I just wonder how much we miss when we base our writing assessments on products of learning and not the processes of learning.
My Semester with Thimble
I chose to do a portfolio assessment for my EDU 106: New Literacies: Digital Texts and Tools for Life long Learning. I don’t think “code quizzes” would be a better approach. There are awesome games to measure those types of learning outcomes. Play CSS Diner or Flexbox Froggy.
I would have missed so much learning if I just used a “final project” approach.
The student who taught half the class how to use Google fonts…would she receive credit?
How about the learners who pinged every corner of social media for help…does this effort not count?
How about the student who struggled but persisted to get an “MVP” published. Should his portfolio be judged on the same scale?
When you teach with Thimble so much of the learning happens as students exchange strategies and designs. When we only use tests or constrained projects as a thermometer of the class we miss so much of the climate.
My Portfolio Assessment
I created a task for the class. I tried to Mr. Miyagi my students. Almost everything to be included in the final should have been done. The students then had two templates to choose from. One portfolio uses the basic three page template included on thimble.mozilla.org and the second template was one I hacked together (a lot of broken JS and used CSS in my template).
I offered an optional design studio the last day of class. Many people went from their paper prototypes (I required this) to a basic portfolio by the end of the class. They all used the <3 project on thimble.mozilla.org.
You can check in and follow the portfolio submissions on our class stream.
We cannot use new tools to support outdated paradigms of measurement. We cannot rethink learning without challenging education. We will not develop opportunities for connected learning without rethinking assessment.
I am not saying all measurement is bad. In fact I am rather found of counting things and I am excited about the possibilities of machine learning. With Thimble, for example, I would love a summary pre/post of lines changed, unused CSS, elements changed. Basically I want to know how far a remix deviates from an original template.
Not sure how hard revision history would be, but that would make a powerful source of data for teachers.
This, however, would never be enough.
Assess by Learning
I am learning! I am assessing. The two can not be separated. I am also documenting my learning through a series of blog posts I share with the class. For example I talked about my struggles with the navigation and the creation of my media page.
Assess by Leading
In essence by sharing my work I am teaching by leading. I am also seeing leadership emerge as an important element in the connected learning classroom. There are folks people turned to as their skills leveled up. There were task managers who could keep their classmates on point. So many different roles emerged that helped to contribute to our climate.
I learn so much by volunteering my time as a teacher for Mozilla. This year I attended my first #MozFest. As a participation leader I want to help my team. We want to get you more involved as a contributor to Mozilla.
MozFest is like no other conference you have attended. Its hackability becomes apparent as soon as you walk into Ravensbourne College. We power our sessions with post it notes not Powerpoint.
Products are perceived, prototyped, and pushed in days. That is what struck me most. I watched Matt Thompson, Phillip Smith, and Darren Mothersele push GitDone from just an intro session on project management with Git. More so I participated in the design.
To me that is #Mozfest. We (re)Design future possibilities.
GitDone is a project management tool. I must admit I was a GitHub naysayer when MoFo (Mozilla Foundation) transitioned that way last year. I work with teachers who volunteer for Mozilla. Our time is limited. I want any time educators have to offer to be remixing curriculum not trying to figure out what a repo means.
Yet what I saw were developers recognizing a community need and solving the problem for non-developers like me.
This experience made me a better teacher.
I was also presenting for the first time at #Mozfest. I and a group of open scholars tried to hack together what it means to play in open spaces as teachers and learners.
It started with a PechaFlickr session. Slidedecks are so much better when you have no idea what will come next. Then we did a live demo on how not to make awful instructional videos.
So we tried to brainstorm what instructional design means from a #Mozfest perspective. This is what we determined would be relevant.
We first started with a discussion of traditional instructional design:
We then examined a bunch of Open classrooms such as #ds106, #walkmyworld, #rhizo15, and #clmooc. That lead us to this:
We took the lessons learned (such as RSS being the backbone of push/pull learning) and then tried to apply a #MozFest lens. This is what we came up with:
Passion. That’s the key difference with Mozilla. We want production based learning that drives people to the web because it is a place of passion.
My goal as a participation leader is to help improve the professional development and leadership training offered by Mozilla. This is a goal of both the Corporation (MoCo) and the Foundation (MoFo).
Step one is helping people tell there #MyMozFest story. We learn and teach best when we do it in the open and our open course safari noted that the most successful online classes empower people to learn on their own space and then they push/pull the participants across many different Web.
Step two (scratch that making this step one) is thanking the amazing people who made MozFest possible for the Participation Team.
I just met so many amazing people who want to change the world. It will take me quite some time to say thanks to them all.
Want to help?
Whether you were a remote or on the ground attendee tell your story about MozFest and share across the Web using the #mymozfest hashtag.