This is the final results of the Plan Quality Evaluation for Heat Resilience and Plan Integration for Resilience ScorecardTM for Heat as applied to the City of Tempe and City of Tucson, Arizona.
The Plan Quality Evaluation for Heat Resilience methodology follows best practices for plan content analysis. The assessment has two components: the Quality Principles Evaluation and the Heat Strategies Evaluation.
For the Quality Principles Evaluation, we assess whether plans meet 56 criteria spanning seven principles as outlined in Meerow and Woodruff (2020) and Keith and Meerow (2022): goals, fact base, strategies, implementation and monitoring, coordination, public participation, and uncertainty. We code each plan using a binary assessment for all criteria, 1 if the criterion is met in the plan, 0 if it is not.
The goals principle examines whether plans have a clear purpose, vision, objectives, and heat-specific aims. The fact base principle reflects the informational basis of the plans, including heat-relevant data. The strategies principle looks for heat-specific actions and whether the costs, co-benefits, and trade-offs of these actions are discussed. The implementation and monitoring principle assesses characteristics that make it likely actions are executed and tracked. The coordination principle examines which stakeholders are involved in the planning process, while the public participation principle focuses on how the broader public was engaged. The uncertainty principle considers whether plans account for future uncertainties, which is critical for climate change planning.
The Heat Strategies Evaluation includes 27 subcategory criteria spanning eight categories of heat mitigation and management strategies (Table A2, supplementary material; Keith and Meerow 2022). Criteria are scored a 0 if the plan does not discuss that strategy type, 1 if it does discuss it, and an additional 1 (2 total) if the strategy is explicitly linked to heat.
Two researchers independently coded each plan, and the full team then discussed any discrepancies. Following best practices for plan evaluation, we calculated intercoder reliability indicators, including the percent agreement and Krippendorf’s Alpha (Krippendorff 2004, Stevens et al 2014), all of which were well above Stevens et al.’s (2014) recommended minimums for plan evaluations with many, highly distributed items.
The Plan Integration for Resilience Scorecard™ (PIRS™) is a method for analyzing the policies, or specific action items, within a network of plans and spatially assessing their combined effect on hazard risks. PIRSTM was first developed for flood hazards by Berke et al (2015), refined and translated into practice by Malecha et al (2019), and has since been applied in a variety of communities in the US and internationally (Yu et al 2020), used in different research contexts, and endorsed by the American Planning Association (DeAngelis et al 2021). Keith et al. (2023b, 2022) first adapted PIRS™ to the specific challenges of heat hazards, piloting the resulting PIRS™ for Heat in five climatically diverse US cities.
To apply PIRS™ for Heat, two researchers independently read each plan and record all land use policies that 1) have the potential to mitigate or exacerbate heat; 2) include a recognizable policy tool, or means for implementation; and 3) are spatially explicit or mappable. These policies are then categorized based on the policy tool and heat mitigation strategy they most closely align with and scored based on their potential effect on heat risks. Policies are scored +1 if they have the potential to mitigate heat, -1 if they would likely exacerbate it, 0 if they would have a neutral effect, and ‘Unknown’ if they would likely affect heat, but it is impossible to determine whether it would be positive or negative with information included in the plan. Policies and scores are reconciled between the researchers and any discrepancies resolved by consensus of the full team.
Policy scores are then assigned to the census tracts where they would apply using the information in the plans and additional geospatial datasets. For example, if a policy states that vegetation will be added along a road, the research team would use a map to determine which census tracts that road runs through, and those tracts would receive a score of +1 from that policy. Some policies apply to the whole city and are assigned to all tracts. Positive and negative scores are then summed for each tract to determine the “net score,” revealing spatial patterns in heat-relevant policy attention.
(2024-02-27)