Code.org's Impact on Computer Science Education: June 2014 - June 2015 Program Report
Code.org is is a nonprofit dedicated to expanding participation in computer science by making it available in more schools, and increasing participation by women and underrepresented students of color.
Code.org partners with Outlier Research & Evaluation at the University of Chicago, a third-party evaluator, to understand our impact and progress towards meeting our mission. This report summarizes our program evaluation work for June 2014 through June 2015, which focused on answering the following key questions:
- What do teachers take away from Code.org professional development?
- How successful have our district partnerships been in driving short- and long-term outcomes in each region?
- What learning outcomes have Code.org students achieved?
- How much do Code.org programs cost?
- How does learning Computer Science impact learning other subjects?
- What next steps can Code.org take to improve our programs?
Evaluation Design
We have gathered data from a number of sources -- internal data collected via Code Studio as well as external data collected by third parties -- in order to evaluate our impact and make decisions about future program direction. A full summary of Outlier's findings is available on their website. All other data cited in this summary are available upon request.
What do teachers take away from Code.org professional development?
From June 2014 through June 2015, over 11,000* teachers across K-12 attended Code.org workshops. We offered professional development through two models:
- a day-long, face-to-face workshop delivered to K-5 teachers via our Affiliate Program or via our District Partnership Program
- a multi-phase, year-long blended PD experience delivered to Middle and High School teachers via our District Partnership Program
Key Finding:
Through our professional development efforts, we are helping a large, diverse audience of K-12 educators feel more prepared to teach Computer Science using Code.org resources.
Table: Overall Participation and Satisfaction in Code.org Professional Development, June 1 2014 - June 15 2015
| Affiliate-Trained Teachers | Code.org-Trained Teachers | |
|---|---|---|
| Number of teachers trained | 9,290 | 500 |
| I would recommend the PD to others | 98% | 97% |
| This was the best PD ever | 66% | 81% |
| I feel prepared to teach CS | 94% | 88% |
Source: Affiliate-trained teacher data from Code.org Survey. Code.org-trained teacher data from Outlier survey.
- Note: In the 2014-15 teacher cohort, we trained approximately 10,000 teachers. We trained an additional 1,000 teachers for the 2015-16 school year.
How successful have our district partnerships been in driving short- and long-term outcomes in each region?
In 2014-15, Code.org partnered with 35 districts across the United States to bring curriculum, professional development and support to over 500 new CS teachers.
Key Finding:
Through our district partnerships, we are helping districts increase the priority of computer science locally in the face of some big challenges.
Additionally, Code.org studied the attitudes of administrators and counselors in an effort to understand the relationship between school leadership and program sustainability. Further study is warranted.
What learning outcomes have Code.org students achieved?
Key Finding:
Students across thousands of Code.org classrooms have demonstrated the ability to use computer science and computer programming concepts, starting as early as elementary school.
Code Studio, our online learning platform, is designed to host our interactive CS courses. All online courses are complemented by lesson plans teachers can deliver in the classroom. We saw phenomenal growth in the 2014-15 school year for both student and teacher account creation.
Table: Code Studio User Growth, June 1 2014 - June 30 2015
| Student Accounts Created | Teacher Accounts Created | |
|---|---|---|
| Prior to September 1 2014 | 1,440,227 | 35,976 |
| September 1 2014 through June 30 2015 | 4,466,810 | 111,684 |
| All time (as of August 1 2015) | 6,033,943 | 147,660 |
Source: Code Studio database
The diversity of the audience we are reaching is also interesting to note. Across all student accounts that reported demographic data, 43% self-identified as female and 37% were identified as African-American or Hispanic by their teacher.
By comparison, the demographic breakdown of students taking the most recent AP Computer Science exam in high school was 20% female students and 17% African American, Hispanic, or other historically underrepresented students.
Overall, Code Studio course participation is high, and basic measures of learning are encouraging. Students who are learning with the support of their teacher in a classroom setting complete courses more than those learning on their own, supporting Code.org’s focus on a mix of teacher development and self-guided material.
How much do Code.org programs cost?
Our first cohort of teachers finished a complete 24-month cycle of district partnership and teacher professional development, allowing Code.org to calculate our all-in cost per teacher, and to estimate our cost per student impacted.
How does learning computer science impact learning other subjects?
Although we are still early in terms of making a direct link between computer science course participation and learning gains in other subjects, a number of recent studies by our partners have shown interesting positive results connecting CS experience with performance in Math and other fields.
Next Steps: Implications for the 2015-16 School Year
Our 2015-16 evaluation plan, developed in concert with Outlier, covers:
- Student outcomes on curricular (part of the course materials) and extra-curricular assessments (general CS concepts test).
- Changes in student attitudes about the value and relevance of computer science, sense of belonging, and self-efficacy, using pre/post survey data.
- Program improvement based on classroom implementation studies and teacher feedback.
- Teacher pre/post attitudinal and pedagogical outcomes.
- Effect of CS in Algebra and K-5 courses on Math/Science/Reading scores.
- Additionally, Outlier will work with our district partners to develop sustainability plans.
Internally, through participant surveys and data science projects, we will continue to study:
- Workshop satisfaction (including online PD components)
- Student outcomes measured through Code Studio
Learn More About Our Approach
In this recorded session, you’ll hear from Pat Yongpradit, Chief Academic Officer at Code.org and Heather King, program manager at Outlier, about the results of our programs over the course of the 2014-15 school year. What did teachers take away from professional development? What did students learn? What did we learn from our district partnerships? Learn from our successes and challenges as we discuss how we might work together as a CS education community to increase access to CS education for all.
