Last week’s “Deep Dive” analyzed quantity of tasks assigned in Eli Review over the past few years. This week’s post marks important milestones in the mix and moxy of our users.
By themselves, many of these numbers may seem unremarkable. But taken together, we are reminded of just how many students and teachers we’ve been lucky enough to work with so far. They come from all over the world and study at many kinds of places. We are very grateful for the chance to help them learn!
What Kinds of Institutions Use Eli?
We talk about Eli as a pedagogy, one that applies from third grade to professional organizations: All writers need feedback and revision cycles. Our userbase* shows that range. Although most users are at higher education institutions in the US, this year we’ve added universities outside the US as well as a workplace investing in raising the feedback skills of its employees through a hybrid course.
*Prior to 2012, institution was not recorded.
What are common class sizes in Eli?
Although it is far more likely for Eli to be used in small classes (fewer than 24 students), review and revision are possible in large classes. Out of the 972 courses enrolling 5+ students this year
- 62% enrolled fewer than 24 students,
- 36% enrolled between 25-44 students
- 2% enrolled more than 44 students.
The two largest enrollments were Chemistry 301 at Michigan State with 147 students and PHC 515 at University of Rhode Island with 124 students.
How many users have used Eli in total?
Since Eli first began recording user information in 2010, 54,122 distinct accounts have been created. Users can interact in Eli as either instructors (creating courses and assigning tasks) or students (joining courses and completing tasks):
- 993 Instructors
- 53,129 Students
How many instructors come back and use Eli again?
A learning technology enhances teacher work by providing evidence that allows them to coach students more effectively. It takes practice to fully benefit from the app, so year-over-year retention is critical. But, teachers’ schedules—especially in first-year composition, which constitutes the largest portion of our user base—and priorities also change year to year.
This year, about a third of instructors used Eli in the previous year and almost 40% had used Eli at some point in the past.
| Instructors | 2014-2015 | 2015-2016 | ||
| Count | Percent | Count | Percent | |
| Total | 343 | 100% | 400 | 100% |
| New | 261 | 76% | 250 | 63% |
| Returning from any previous use* | 82 | 23% | 150 | 38% |
| *Retained from prior year | 72 | 21% | 133 | 33% |
How many students use Eli in more than one course?
Students use Eli only if it is a required course material, so enrollment represents an instructor’s choice, not students’. When students are in multiple courses, we get a better sense of how Eli use is spreading throughout the curriculum, which is happening at Michigan State University and a several Michigan high schools.
| Students | 2014-2015 | 2015-2016 | ||
| Count | Percent | Count | Percent | |
| Total | 16,145 | 100% | 20,559 | 100% |
| Enrolled in 2+ courses | 2,092 | 13% | 2,313 | 11% |
How much work are users doing?
In the “Deep Dive” post, we focused on the percentage of instructors assigning 5+ reviews and 5+ revision plans per course per academic year as an indication that Eli’s pedagogy of rapid feedback cycles was spreading. We know that it takes a bit of experimentation and practice before instructors make feedback and revision in Eli a part of their class routines. This data accounts for all that work. Below, we report cumulative activity recorded in Eli since 2010.
Writing Tasks
- 14,963 task prompts by instructors
- 242,994* “Composed in Eli” submissions by students
- 95,654* Uploaded documents by students
Review Tasks
- 10,794 task prompts by instructors
- 529,438 completed submissions by students
Review Comments
- 8,811 review task prompts by instructors allowed for Contextual Comments
- 359,978 completed submissions by students to prompts including contextual comments
- 1,009,026 comments by students
- 37,086,554 total words in comments by students
- 334,924 (33.2%) rated by student recipients for helpfulness
- 38,127 (3.8%) endorsed by instructors
Revision Plans
- 3,032 revision plan task prompts by instructors
- 43,350 revision plans submitted by students in response to prompts
- 12,828 self-sponsored revision plans (completed by student when not assigned by instructor)
Revise and Resubmit Tasks
- 1,894 prompts* by instructors
*We are reporting this way to show the full write-review-revise cycle. Submissions for writing tasks and revise & resubmit tasks are treated identically in the app, so the number of “composed in Eli” submissions and uploaded documents include both types of tasks.
How does 2015-2016 academic year compare to previous years?
This year has been a banner year on several metrics:
- The number of participating institutions doubled over last year.
- About 10% more of active instructors this year are returning users compared to last year.
- Instructors assigned 12,571 tasks this year. That is about half of the total number of tasks assigned since. . . well, ever.
- Eli almost has 250,000 full-text writing task submissions by students.
- Eli captured the 1,000,000th comment left by a student reviewer for a student writer.
So what?
These milestones let us know that, from a system-wide perspective, users are busy giving feedback and making revisions. We can see the consequence of a stable, secure peer learning environment. The data allows us to follow student performance across write-review-revise tasks. In particular, we can describe comments based on count, word count, helpfulness, endorsement, and whether it was added to a revision plan. We are confident that students are not only giving feedback but also using it.
This matters beyond our own internal measures. It’s especially valuable to the schools we work with who want to understand student performance. We partner with individual instructors and entire institutions to research their records of peer learning.
Are you a current Eli Review instructors? Get in touch with us if you’d like to do a deep dive into your own data.
