Data Collection

We have worked collaboratively with Kansas City, Missouri School District (KCMSD), Miami-Dade County Public School (M-DCPS), Milwaukee Public Schools (MPS), and San Francisco Unified School District (SFUSD) to gather administrative data related to schools, staff and students as well as collect primary data in the form of observations, surveys and interviews.

We developed surveys for principals, assistant principals, and teachers in order to better understand the role of school leaders. All three of these surveys asks about the instructional climate of their schools, the role of the school principal, how appealing different aspects of the principalship are to them, how leadership responsibilities are distributed between themselves and other school leaders, and their preferences for different school characteristics. The principal survey specifically asks how effective they feel as a school leader, and their pathway to the principalship. In parallel to the principal survey, the assistant principal and teacher surveys asks about the effectiveness of their principal as a school leader, and their future plans.

We use teachers' survey responses to develop a measure of teacher satisfaction to augment the teacher turnover measure described above. The combination of the two serves as one of our key school outcome variables. Responses are linked to a unique individual identifier so that we can follow the respondents over time with future surveys, and the surveys are confidential but not anonymous so that we can link the responses to the district administrative datasets described above. Additionally, because parallel items are used for the assistant principal and teacher surveys, this study design allows for the calculation of inter-rater reliabilities within schools. Triangulating responses of principals, assistant principals and teachers also allows us to create school-level measures of aggregate school leadership capacity as it is distributed across school actors. This operationalization of school leadership capacity allows us to examine the relationship between this capacity and other school outcomes.

We have conducted time-use observations of school principals in M-DCPS, MPS, KCMSD, and SFUSD. In M-DCPS we have also collected time-use observation data of high school Assistant Principals.

For these time-use observations, trained Stanford researchers shadowed the principals for an entire school day, starting roughly 30 minutes before the start of school and generally ending when students were released at the end of the day. Researchers recorded the principals' tasks throughout the day at five-minute intervals. At each observation, researchers coded the task the principal was engaged in at the exact time of observation, the location of the principal, with whom the principal was interacting, and the nature of the activity (e.g., phone call, scheduled meeting, etc.). In cases where multiple codes were relevant, the more specific code was entered as the primary task, with other relevant codes listed as secondary activities. Between each five minute task assessment, the researchers recorded observer comments, including details about the nature of the task and notes about tasks that were completed within the five-minute intervals.

Principal actions were coded as one of a list of 48 activities as shown in Figure 1. We populated our list of task codes based on the categories for principal duties developed by Spillane, Camburn, and Pareja (2007). We further developed this task list through consultation with principals and district leadership in multiple states. Finally, we refined our expanded list through pilot shadowing of principals in local schools. Hard copy versions of the shadowing observation protocol were used until the 2011-2012 school year, when an electronic version of the protocol was used for data collection in both M-DCPS and KCMSD.

The tasks fall into three main leadership roles: administrative, instructional, and relationship building. Within each of these roles, there are two categories of tasks. Administrative leadership combines management and operations; instructional leadership combines day-to-day instructional tasks with instructional program planning; and, relationship building combines fostering internal relationships with brokering (i.e., building support and obtaining external resources).

To supplement the quantitative data described above, we conduct interviews of school leaders, including principals and assistant principals, following each time-use observation. These interviews are used to extend our analyses of the surveys, time-use data, and administrative data. They are conducted at the conclusion of the time-use observation by the shadower. Depending on the school leaders’ availabilities, the duration of the interviews range from 30 minutes to one hour. With the school leader’s consent, the interview is audio-recorded and later transcribed.

The school leader interview data supplements our quantitative analyses by providing insights into their personal goals and visions for their school, their perceptions of what role school leaders should play, and their personal path to the principalship. Because this research design provides us with quantitative time-use data and qualitative interview data from the same school leaders, we are able to triangulate these complementary sources of data.

Our school district partners have provided us with longitudinal school, staff, and student data. Data on students includes student performance on standardized tests in the core subjects, student demographic data (such as ethnicity and eligibility for subsidized lunch), and student course-taking and grades. School-level data include school type and aggregated responses to district-administered school climate surveys. Finally, the longitudinal data on district staff includes job placement, demographic information, certification status, experience, and education level.

The administrative data provided by the districts are particularly important for our analyses because they allow us to construct two important outcome measures: teacher turnover and student learning. Specifically, we use teacher personnel data to construct measures of teacher turnover – both transfer across schools and leaving the school district. We also create measures of teacher movement into leadership and administrative positions. We use the longitudinal data to gain statistical power in assessing the effects of leadership on attrition. Administrative data on student achievement test scores provide a primary measure of schooling outcomes and thus a key dependent variable for our analyses. In keeping with the No Child Left Behind Act each state tests all students in reading and mathematics in grades three through eight and in one grade in high school. We use the student-level results of these tests to construct our measures of student learning, including valued-added measures. In addition to providing the outcome measures of teacher attrition and student learning, the administrative data contributes to measures of teacher and leader attributes and school context. The longitudinal nature of this data allows us to describe and model career paths and estimate the effects of leadership attributes and other factors on career decisions.