Data Collection

We have worked collaboratively with Kansas City, Missouri School District (KCMSD), Miami-Dade County Public School (M-DCPS), Milwaukee Public Schools (MPS), and San Francisco Unified School District (SFUSD) to gather administrative data related to schools, staff and students as well as collect primary data in the form of observations, surveys and interviews. Each of the following data sources (and our data collection methods) is described in detail below:

Administrative Data

Our school district partners have provided us with longitudinal school, staff, and student data. Data on students includes student performance on standardized tests in the core subjects, student demographic data (such as ethnicity and whether they are eligible for subsidized lunch), and student course-taking and grades. School-level data include school type and aggregated responses to district-administered school climate surveys. Finally, the longitudinal data on district staff includes job placement, demographic information, certification status, experience, and education level.

The administrative data provided by the districts are particularly important for our analyses because they allow us to construct two important outcome measures: teacher turnover and student learning. Specifically, we use teacher personnel data to construct measures of teacher turnover – both transfer across schools and leaving the school district. We also create measures of teacher movement into leadership and administrative positions. We use the longitudinal data to gain statistical power in assessing the effects of leadership on attrition. Administrative data on student achievement test scores provide a primary measure of schooling outcomes and thus a key dependent variable for our analyses. In keeping with the No Child Left Behind Act each state tests all students in reading and mathematics in grades three through eight and in one grade in high school. We use the student-level results of these tests to construct our measures of student learning, including valued-added measures. In addition to providing the outcome measures of teacher attrition and student learning, the administrative data contributes to measures of teacher and leader attributes and school context. The longitudinal nature of this data allows us to describe and model career paths and estimate the effects of leadership attributes and other factors on career decisions.

Principal Observations

Our school district partners have provided us with longitudinal school, staff, and student data. Data on students includes student performance on standardized tests in the core subjects, student demographic data (such as ethnicity and whether they are eligible for subsidized lunch), and student course-taking and grades. School-level data include school type and aggregated responses to district-administered school climate surveys. Finally, the longitudinal data on district staff includes job placement, demographic information, certification status, experience, and education level.

Principal actions were coded as one of a list of 46 activities as shown in Figure 1. We populated our list of task codes based on the categories for principal duties developed by Spillane, Camburn, and Pareja (2007). We further developed this task list through consultation with principals and district leadership in multiple states. Finally, we refined our expanded list through pilot shadowing of principals in local schools.

The tasks fall into three main leadership roles: administrative, instructional, and relationship building. Within each of these roles, there are two categories of tasks. Administrative leadership combines management and operations; instructional leadership combines day-to-day instructional tasks with instructional program planning; and, relationship building combines fostering internal relationships with brokering (i.e., building support and obtaining external resources).

Surveys of Principals

We developed a survey to ask principals about the instructional climate of their school, their role as a principal, how appealing different aspects of the principalship are to them, how effective they feel as a school leader, how leadership responsibilities are distributed between themselves and other school leaders, their pathway to the principalship, and their preferences for different school characteristics. The surveys are confidential but not anonymous so that we can link the responses to the district administrative datasets described above. Additionally, because parallel items are used for the assistant principal and teacher surveys, this study design allows for the calculation of inter-rater reliabilities within schools. Triangulating responses of principals, assistant principals and teachers also allows us to create school-level measures of aggregate school leadership capacity as it is distributed across school actors. This operationalization of school leadership capacity allows us to examine the relationship between this capacity and other school outcomes.
District Date Number of Respondents Response Rate
M-DCPS May 2008 314 89%
MPS October 2008 137 93%
KCMSD Principal Survey Report 2010
M-DCPS Principal Survey Report 2008
MPS Principal Survey Report 2008

Surveys of Assistant Principals

To better understand assistant principals' experiences and perceptions of their principal as well as the principalship in general, we also developed a survey for assistant principals (APs). In parallel to the principal survey, this survey asked APs about the instructional climate of their school, the role of their current principal, how effective their principal is as a school leader, how school leadership responsibilities are distributed between themselves and other school leaders, how appealing different aspects of the principalship are to them, how prepared they feel to take on school leadership responsibilities, their future plans, and their preferences for different school characteristics.

Responses are linked to a unique individual identifier so that we can follow the respondents over time with future surveys, and the surveys are confidential but not anonymous so that we can link the responses to the district administrative datasets described above.
District Date Number of Respondents Response Rate
M-DCPS May 2008 585 85%
MPS October 2008 91 70%
M-DCPS Assistant Principal Survey Report 2008

Surveys of Teachers

To better understand teachers' perceptions of their current principal as well as the principalship in general, we also survey teachers. In parallel to the principal and AP surveys, this survey asks teachers about the instructional climate of their school, the role of their current principal, how appealing different aspects of the principalship are to them, how prepared they feel to take on school leadership responsibilities, their future plans, and their preferences for different school characteristics.

We use teachers' survey responses to develop a measure of teacher satisfaction to augment the teacher turnover measure described above. The combination of the two serves as one of our key school outcome variables.
District Date Number of Respondents Response Rate
M-DCPS May 2008 15,842 83%
MPS March 2009 1010 70%
M-DCPS Teacher Survey Report 2008

Interviews of Principals

To supplement the quantitative data described above, we conduct interviews of the principals following each time-use observation. These principal interviews are used to extend our analyses of the surveys, time-use data, and administrative data. They are conducted at the conclusion of the time-use observation by the shadower. Depending on the principals' availabilities, the duration of the interviews range from 30 minutes to one hour. With the principal's consent, the interview is audio-recorded and later transcribed. The principal interview data supplements our quantitative analyses by providing insights into principals' personal goals and visions for their school, their perceptions of what role principals do and should play, and their personal path to the principalship. Because this research design provides us with quantitative time-use data and qualitative interview data from the same principals, we are able to triangulate these complementary sources of data.