To many in higher education they see problems as nails and testing as the only hammer in our kit. Yet we stand at a time where we can re-mediate assessment using new technologies and old definitions of what it means to learn.
I serve on the Tech Fluency (TF) affinity group at Southern Connecticut State College. TF is a tier one competency in our liberal education program (our general ed program but with more hoops and loftier goals). The goals is to ensure all students have the minimum tech skills they will require after college. In our current (permanent) budget crisis we have been asked to review the effectiveness of LEP.
We developed a series of rubrics instructors could use in their classroom. The newly appointed LEP assessment committee decide our approach was “too subjective.” They suggested a common task, filling out a spreadsheet was their recommendation, that students could complete in a controlled and supervised environment.
This set me off. I responded (probably with not enough to delay and too much acerbic snark) with some of the following comments.
Objective Assessments are a Hoax
Subjective flickr photo by EVRT Studio shared under a Creative Commons (BY-NC-ND) license \
I am not anti-testing. Most of my research revolves around item design and testing. Yet I think what set me off was the belief that some assessments are objective. Those who rely on standard measures ignore the bias inherent in statistical models and deciding as “what counts as learning.” They look at rubric scored items as being “too subjective” yet ignore the error variance, the noise in their models. I say bring the noise. It is in outliers where we see interesting methods and learning.
Technology Assessment Lack Ecological Validity
Students Should Drive Assessment
Purpose of Assessment
I also took issue with the the purpose of our assessment. If the goal is to evaluate the effectiveness of our LEP program then why is our gut reaction to assess each student individually.
I admit I am guilty of questioning badges for low hanging fruit. During the Connected Educator Month last October you could get a badge for attending almost any event. I wondered, often out loud on Twitter, if this approach made sense.
Christina Cantrill of the National Writing Project often pushed backed. Christina made the argument that the evidence for the participatory bags is just that, evidence. She explained to me that different badges have different value and you could even have leveling up badges. As Christina explained it, rigor means nothing, it is relevance of the badge to the community that matters most.
As I have become more and more swayed towards Open Badges (due to the evangelizing efforts of Ian O’Byrne and Doug Belshaw) I keep returning back to this question: What do we get when badging for participation versus badging for competencies?
I found my answer yesterday during the The Badge Alliance Teacher Badges open call. Mark Otter and Julie Keane shared their learning platform that they use as part of their efforts in teacher preparation and global education.
The call was great. Seeing badges in the wild, specifically connected to teacher preparation and professional development, really helped to formulate my thinking. To date VIF International has awarded 592 badges to teachers in their community.
Behaviors versurs Competencies
I am sure this debate has played out int eh badging research community for quite some time. Like I said, I am a recent convert so I am stumbling into many of the lessons others have already learned.
Mark and Julie make a distinction between competencies and the behaviors that they would like to see in their community.Teachers can earn badges when they share evidence that demonstrates competencies. This often includes lesson plans.
To recognize behaviors that help support online communities and professional development Mark and Julie created a point system that translates into stars. So you can earn points for things such as logging in, posting to a forum, and commenting on another post. These are not behaviors that provide evidence of growth. Yet these behaviors are crucial to building critical mass in online learning spaces.
I think as I start to develop badges, specifically for summer Gear Up Programs I help run, I will try to bake in a similar approach. Lets keep badges for competencies and use other metrics to reinforce behavior.
This “just for fun” badge actually documents the successful work flow of a badge. Doug created the badge, explained the competencies being addressed, described the evidence needed. I then submitted my material. Doug reviewed it and left me this feedback:
I then went back and revised my original post. If you want to track my edits look at my source code I left the old version in html comment form.
Doug Belshaw issued a challenge. As part of the Mozilla Web Literacy Map roll out he encouraged folks to submit artifacts that would demonstrate competencies on three areas of the map,
I am submitting my badge under the working title of Wisdom, Virtue, Sincerity, Valor, and Austerity in Online Spaces.
- Accessing the web using the common features of web browsers– I try to teach students how to to explore the web by creating and remixing videos and think alouds of online data: I created videos of students reading online that I allow others to use
- Using hyperlinks to access a range of resources on the web- I try to find and share links to open education resources such as this one. I was looking for OER sources on the web, and put out a call on Twitter. This link came back to me.
- Reading, evaluating, and manipulating URLs I wrote a dissertation on differences in searching and evaluating online sources.
- Recognizing the visual cues in everyday web services- I make online minilessons to teach students how to search the internet and research ways to teach credibility.
- Using keywords, search operators, and keyboard shortcuts to make web searches more efficient –My dissertation research focused on improving search results.
- Finding real-time or time-sensitive information using a range of search techniques- I write about the need to teach and read socially complex texts.
- Researching authorship and ownership of websites and their content- I create online materials to teach students to focus on authorship.
- I use my brain as my virus and phishing detector.
Composing for the web
- Inserting hyperlinks into a web page- This post
- Embedding multimedia content into a web page-I can embed multimedia into posts.
- Creating web resources in ways appropriate to the medium/genre- I write in a variety of places using the norms of thos sites such as Medium.
- Identifying and using HTML tags- I used the comment tags so people could track the revision history on this post.
- Structuring a web page- My homepage
- Identifying and using openly-licensed work- The image I remixed for the header on this post used two openly licensed images.
- Combining multimedia resources and Creating something new on the web using existing resources- I make remixes using popcorn.
Design and Accessibility
- Iterating on a design after feedback from a target audience– I got feedback from Doug and then revised this post. This website is the rough draft of my life
- Improving the accessibility of a web page by modifying its color scheme and markup- I try hard to choose color schemes that allow those with red/green color blindness to differentiate.
- Demonstrating the difference between inline, embedded and external CSS- I spent this semester trying to use Thimble to teach myself CSS. I have created my first page that no longer uses HTML tables but relies on CSS containers. The actual page isn’t live yet (Department website but containers work
Coding and Scripting
- Sharing a resource using an appropriate tool and format for the audience- My ipad tutorial page
- Tracking changes made to co-created web resources– This is the first collaborative story I wrote in Gdocs with my 6th graders.
- Co-creating web resources- Ian and I edit the digital texts and tools page. Please join us and add your stuff.
- Configuring notifications to keep up to date with community spaces and interactions– Much to my wife’s chagrin as things chirp and beep all day long.
- Using synchronous and asynchronous tools to communicate with web communities, networks and groups– I use asychnronous and synchronous chat in my teaching.
- Encouraging participation in web communities– I encourage folks to be digital residences.
- Encouraging participation in web communities- I am involved in many twitter discussions.
- Using constructive criticism in a group or community setting -I use online communities on Google+ for Feedback.
- Defining different terminology used within online communities- I use the discourse of specific affinity spaces and use these spaces for learning.
- Configuring settings within tools used by online communities- I use a variety of webtools for communication.
- Identifying rights retained and removed through user agreements– I added the Creative Commons plug in to this site.
- Distinguishing between open and closed licensing- I use only open lecensed images on this site.
As educators we need to draw a map (of the territory such as the Web Literacy Map) using creativity and all means available to you. To [further] illustrate this point, when even the roads are unknown, enter the online spaces, and familiarize yourself with the languages and practices. Determine which areas have steep learning curves, which areas are wide open, and measure the width of roads to understanding.
So instead I discussed why Open Badges makes more sense for our University rather than a proprietary portfolio system.
Our Liberal Education Program
Calling our LEP classes, general ed, does a disservice to our students and those committed faculty who designed the program. Yes they are classes every student must take but the LEP builds on a foundational belief in the importance of liberal studies. The program contains 24 separate goals over three tiers of classes (click here for more information). These goals are broken into competencies, knowledge and values.
As a rough summary tier one classes include competencies, tier two include focus on knowledge and values, and tier three integrates all three elements into a final outcome.
Our tier one competencies briefly introduce the Areas of Knowledge and Experience and Discussions of Values while maintaining a focus on the development of intellectual skills, or competencies.
I volunteer on the technology fluency committee which is a core competency. In this tier students have to:
1. Common Tasks – Solving problems, accessing information, and communicating information and ideas using appropriate technologies
A. Students will be able to engage in electronic collaboration.
B. Students will be able to use and create structured electronic documents.
C. Students will be able to do technology-enhanced presentations.
D. Students will be able to use databases to manage information.
E. Students will be able to use spreadsheets to manage information.
F. Students will be able to use graphical and multimedia technologies.
2. Focus – Using emergent or recently developed technologies (hardware orsoftware) to address specialized tasks
A. Students will have the ability to perform basic operations in at least on ecurrent technology platform, or
B. Students will acquire advanced level skills in three out of six of the Common Tasks listed in (1).
3. Future Technological Change – Navigating and adapting to futuretechnological developments
A. Students will be able to use electronic tools to navigate, to compare orcontrast, to research and to know enough to evaluate the technology as a tool.
4. Broader Implications – Being cognizant of ethical and social implications of revolutionary technologies, including but not limited to their impact on security, privacy, censorship, intellectual property, and the reliability of information
A. Students will be familiar with major legal, ethical, privacy and security issues in information technology.
The purpose of tier one classes is to introduce and teach these competencies. They are then reinforced in tier two and tier three classes. This spiral of complexity in the curriculum design makes badging a better assessment system than portfolios.
The tech fluency and the embedded competency of information fluency represent a broad goal. To establish and display competency students need more than a GPA. Having a B in a class tells me nothing. Yet if I can look at a meta-level badge and know student earned badges in the three areas of tech fluency I can build off of that learning in my tier two classes.
So the badging system would involve different levels of badges. Students would have to earn the common task badge (I know need a better name) would have to point to urls that document each of the common tasks.
They could then submit requisite materials for the other three badges. Once those are all submitted the students could apply for the meta level tech fluency badge.
I believe the badges will then support the tier two classes. Teachers trying to put a larger focus on general knowledge, can look back at the badges students collected in tier one and cater projects to individual students.
Why Open Badges over Portfolios
Portfolios do not seem to work at our University. We make students buy into a program called TK20, but this is not in the support of learning. Its just a way to generate pretty tables for NCATE. At the end of the semester students upload the prerequisite documents, educators grade them, and there they sit collecting digital dust until we need the artifacts for NCATE.
Portfolios are often proprietary. We could use the system embedded into our overpriced LMS but why not ensure assessments are owned by students and our 100% portable.
Presence matters more than portfolios. Learning is a social event. I do not want to silo off engagement into a system. I would rather students be able to document their education where ever they learn.
Open badges would allow students to document learning in lieu of class. We have many students, especially in tech fluency, that believe they meet the requirements all ready. If we had an open badge system students could then submit documents, without being in a proprietary system, that demonstrate their competencies. If they could earn the meta level technology fluency badge they would not have to complete the course.
First I have to convince a slew of committees open badges makes sense and explain how it could be implemented into our liberal education program.
Then we would have create the road map of competencies for each badge.
We would hten design the layout of the badges.
Finally for this to work open badges require the help from our great IT team on building the back end support so we can use the open API.
The competency based liberal education program and badges make sense. We want to document students abilities for life long learning. Why not create a system that reflects these values in both practice and theory?