When we launched our Eli pilot test with several Michigan school districts late last year, we hit a painful roadblock almost immediately. We discovered that all of our pilot schools rely on Internet Explorer 7 (IE7) and only one school had access to a more modern browser (and had to jump through several hoops to use it). Eli was initially designed for modern web browsers (Firefox, Chrome, Safari, even Internet Explorer 8), but because IE7 is five years old it either lacks or makes support difficult for many newer standards and design methods (Wikipedia offers a thorough breakdown of IE7’s history).
Our pilot users immediately encountered problems when trying to use Eli. It was the worst-case scenario; assignments and reviews couldn’t be created, students had trouble responding to writing prompts, and data was lost during review activities. Calls were made to district technology directors about the possibility of upgrades, but it became clear that we’d be waiting a very long time if we were to rely on school districts to upgrade their browsers. We would have to find a way to get Eli to be backwards compatible with IE7.
Our major effort to overhaul Eli began with Eli’s master programmer, Red Cedar’s Vaughn Anderson, who scoured the code to the best of his ability and was able to solve many of Eli’s IE7 support issues. However, because we now know that many of our K-12 users will be using IE7, making sure we identified and solved all of our support problems required further testing that would put Eli through all of it’s possible uses and test for failure.
To that end, we designed a series of rigorous usability sessions that would put Eli through all of its paces and test all of its functionality in our worst-case scenario. They would involve using IE7 in two different environments: inside a virtual machine running an original version of Windows XP SP3, which would give us access to IE7 as it exists in the wild (see image below), and Internet Explorer 8 running in IE7 compatibility mode, which would give us access to more sophisticated debugging tools. We also scheduled the tests in a wide variety of computer labs across Michigan State’s campus, ranging from newer, more powerful computers to machines that haven’t been upgraded in many years. With this testing framework in place, we were confident that we’d be able to identify and eliminate all of Eli’s IE7 support problems.
Eli running in Internet Explorer 7 running inside a virtual machine (notice the two start menus open simultaneously).
We also took advantage of these tests to develop a set of resources that many of our pilot tests have been requesting: a fully-functional demo course inside Eli that would give users access not only to live review data but also test Eli’s functionality without the danger of exposing those tests to students. To accomplish this, we worked with one of our pilot testers, Dawn Reed of Okemos High, to develop a series of writing assignments that high school writers of various ages and skill levels might be expected to answer. To generate responses to those assignments, we decided to utilize volunteers from MSU and ask them to play the role of a high school writer; it was decided that college students adopting personas of high school writers would be easier than recruiting actual high school students because of the approvals that would require from parents and principals. We provided our testers/writers with personas, or roles to play, describing their character’s writing style, ability, and general attitude toward school, writing, and review.
Example of personas MSU volunteers used to respond to high school writing assignments.
We encountered a number of small errors during the first round of testing, but it seemed that Vaughn’s previous work to eliminate the major problems was successful; students were able to complete all of their writing and review work with no major difficulty. Vaughn was able to spend some time addressing those issues between tests, and the second and third tests went progressively better, with functionality and ease of use improving with each subsequent test.
Jeff Grabill leads his fake high school students through a writing assignment.
We still have a few display issues (fonts, positioning, etc) to work out, but after these usability tests, we feel confident in saying that Eli is fully capable of working inside Internet Explorer 7. We’ll be reaching out to those pilot schools that encountered serious problems in the fall and asking them to give Eli another go in their classrooms.
We’ll also have the data generated from our usability sessions converted into a fully-functional demo class for all instructor-level users in the very near future, so that instructors who want to test out assignments and reviews can do so without the worry of students seeing their tests.
Many thanks to all of our pilot testers who stuck with us as we’ve addressed these problems. Your patience and feedback continue to make Eli better and better!
