# Breeding Bird Survey statistical methods

The North American Breeding Bird Survey (BBS) is conducted annually in the United States, Canada, and, as of 2008, northern Mexico. The survey is designed to monitor trends in relative abundance of North American breeding birds at continental, national, and regional scales. Environment and Climate Change Canada's Canadian Wildlife Service and the United States Geological Survey's (USGS) Patuxent Wildlife Research Center jointly coordinate the BBS program.

## Data collection

BBS data are collected along randomly established roadside routes by skilled volunteer observers. Routes consist of 50 stops spaced 0.8 km apart along a 39.2 km length of all-weather secondary roads. The starting point and direction of routes are selected randomly within a given degree block (degree blocks are geographic areas 1 degree longitude by 1 degree latitude in size and are the basic sample design units for the BBS) in order to sample a range of habitats representative of the region. In Canada, the acceptable dates for running a BBS route are between 28 May and 7 July although most observations are made during the first three weeks of June. Observers are encouraged to run their routes for as many consecutive years as possible in order to reduce the effects of observer variability on trend estimates. Because the BBS is designed to monitor long-term changes in bird populations, changes in habitat along routes are expected. However, routes may be altered or abandoned if the road system is altered, increasing ambient noise interferes with an observer's ability to detect birds, or if traffic conditions are unsafe for the observer.

At each of the 50 stops, observers conduct a three-minute point count observation, during which they record the total number and species of all birds heard, and all birds seen within 0.4 km of their location. Start and finish times for the routes are recorded. Sky and wind conditions are recorded at the beginning of each 10-stop block and at the end of the route. The number of vehicles passing and other excess noise are recorded during each stop. Data are entered into the online database by observers, or are submitted to one of the BBS national offices for entry. All BBS data are stored in the North American BBS database housed at the USGS Patuxent Wildlife Research Center and are available on the USGS BBS website. Original data sheets for Canada are stored by Environment and Climate Change Canada's Canadian Wildlife Service. For a comprehensive description of the BBS field methods, see Robbins et al. (1986).

## Data screening

Several factors, in addition to changes in bird populations, contribute to variation within BBS data. These include changes in weather, date, starting and finishing times, as well as differences among observers. To help reduce the extraneous variability in the data, the data are screened to remove surveys run under unacceptable conditions (e.g., rain and/or high winds, outside of the range of acceptable dates, or outside of the range of acceptable times of day). All criteria are available on the USGS BBS website.

## Geographic analysis strata

The data are analyzed within geographic strata defined by the intersection of Bird Conservation Regions (BCRs) and the Provinces/Territories. For example, British Columbia is divided into four strata by BCRs 4, 5, 9, and 10. For a given species, only strata with sufficient data are included in the analysis. To be included, a stratum must contain at least 3 routes on which the species was observed, at least one of those routes was surveyed in 5 or more years, and at least one route on which the species has been observed in 3 or more years. Trends and estimates of relative abundance are modeled separately within each of these analytical strata to account for differences in a species' population status and trends among different regions, and to allow the stratum-level estimates to be combined into composite regional estimates based on BCRs or political units.

The geographic nature of these strata means that users of these results should carefully consider the spatial scope of the national and regional estimates of trends and annual indices from the BBS data. To this end, the trend and annual index estimates for every species and region are presented with an associated map showing the regions to which the estimates apply. The maps provide an at-a-glance assessment of the spatial scope of each trend and annual index estimate, and will be helpful in clarifying the following two potential sources of error in interpreting BBS results:

First, the estimates apply to the specific strata in which the data were collected and these strata may not completely cover a given region of interest. For example, we report "national" trends; however, the BBS does not cover Canada in its entirety, so some portion of most species' populations is not sampled by the BBS. The same can be said for many provinces, territories, and BCRs.

Second, the short-term and long-term estimates for a given species and region may be calculated based on data from different portions of the region. The earliest BBS routes in Canada were run in the Maritimes and Quebec in 1966. By the early 1970s, much of southern Canada had adequate coverage; however, some northern regions have only had adequate coverage for the last 20 years. Therefore, not all strata can be appropriately included in all long-term trend estimates (i.e., approximately 40 years). For example, there were very few BBS data in the Northwest Territories and much of the Yukon until the late 1980s or later, so trends in these areas cannot reasonably be extended back to 1970. For a species and region that includes these later-starting strata and strata with data from the full time-series (e.g., most national trends for species with some distribution in the boreal forest), the short and long-term results will differ because they represent different parts of the region. Although this complicates comparisons between short- and long-term trends, it allows for trend estimates for both the longest possible time period (the long-term trend for regions with data over the full time-series) and for the largest geographic area (the short-term trend for all regions with some data). In each case where the short- and long-term trends include different strata, lists of included and excluded strata will be supplied directly under the associated map, reflecting these differences.

## Analytical methods

The BBS is analyzed using a Hierarchical Bayesian model that accounts for the effects of variation among observers and routes, first-year observer effects, variations in trend and abundance among strata, and annual variation around a long-term trend. The observed counts are assumed to have an over-dispersed Poisson distribution. The model is fit using standard approaches to Bayesian modeling: Markov chain Monte Carlo (MCMC) methods with Gibbs sampling using WinBUGS software. The model used here is derived from the model described in Sauer and Link (2011), with some small changes to better suit the Canadian data. A detailed statistical description of the model and the model code in BUGS language is freely available in a scientific article published in the Canadian Field Naturalist (Smith et al. 2014).

The hierarchical structure of the model makes the most efficient use of the data and also means estimates are much less sensitive to random sampling error (e.g., variation in the number of routes run in a given year). The parameters that represent potential sources of error or variation in the data-over dispersion, variation among observers and routes, first-year observer effects, and the annual variation around the stratum trend-are modeled as random effects (i.e., parameters governed by hyperparameters higher up in the hierarchy). The individual parameters (e.g., the effect of a given observers' first year, or the departure from the stratum-specific long-term trend line in a given year) are drawn from a distribution of parameters with a common mean and variance: the hyperparameters. One particular benefit/consequence of the hierarchical structure is that the inter-annual variation in the annual indices partially reflects the quality of the underlying data. As a result, the annual indices will show patterns in the population's trajectory (i.e., well-defined, inter-annual variation) if the underlying data support that pattern, but in years with sparse or variable counts, the pattern in the trajectory is smoothed towards the trend line.

As of the 2014 data version, a hierarchical structure has been applied to the stratum-level estimates of long-term trend line (i.e., the log-linear slope parameters for each stratum). Therefore, these estimates of long-term slopes are partially shrunk towards the species' range-wide average slope, particularly in strata with relatively few data. Therefore estimated rates of change in regions with relatively sparse data are generally more precise and less extreme (relative to the species' range-wide average slope) than estimates from the previous model.

The relative influence of local data and data from other parts of the species' range on the estimates of the long-term slope depends on two factors: 1) the quality of the data in the region and 2) the relative magnitude of the local estimate, compared to the range-wide estimate (i.e., range-wide estimates have greater influence when local estimates are very imprecise and/or very different from the range-wide estimate). We believe this statistical assumption is reasonable because local populations of a given species are, to some extent, linked across the species' range, due to mixing of individuals through dispersal and migration. Also many users of BBS trend estimates are likely comfortable with the idea of considering trend estimates from other regions to inform their understanding of a species' status in areas where local trend estimates are particularly poor (i.e., low precision) or of an unexpected magnitude (e.g., increasing steeply when other regional trends suggest the species is stable or in decline). One important consequence is that BBS data from outside of Canada are now incorporated into the Canadian trend analyses. This is an acceptable, and arguably desirable, feature of this new approach because: 1) populations of bird species are linked across national borders; 2) it improves the precision of regional and national trend estimates; 3) for regions with high-quality data, it has little or no influence; 4) it reduces the likelihood of estimating anomalously extreme trends of very low quality (e.g., steeply increasing trends with exceptionally low precision); and 5) incorporating the status of a species in other parts of its range allows us to estimate trends for more species, particularly those with limited distributions within Canada. A new reliability measure ("local data weight") has been added that indexes the contribution of the local data to a regional estimate.

## Reliability measures

To quantify the reliability of the BBS trend estimates in describing overall population trends, three measures of reliability are calculated: **precision, coverage and local data weight**. In addition to these three credibility measures, one may also want to consider the following measures reported in the final columns of the trend data tables: the number of routes used to generate the trends, the estimates of the probabilities that a given population has decreased or increased, or the probabilities that the magnitude of the population change falls within one of the six population change categories. These probabilities provide a concise and intuitive assessment of the uncertainty around some commonly desired interpretations. For example, a relatively strong positive trend that is imprecisely estimated may in fact have a greater probability of being a decline (i.e., the trend was estimated in the wrong direction), than a stable or slightly positive, but precisely estimated trend.

**Precision:** This measure reflects the width of the 95% credible interval (CI). Smaller values indicate a more precise trend estimate. The 95% CI defines a range of trend values that have a 95% probability of including the true population trend value, given the data and the model. For example, the following statement is a valid interpretation of a hypothetical 95% CI with lower limit = 1.2 and upper limit = 2.3: given the BBS data and the accuracy of the model, there is a 95% probability that the average annual increase in the population is somewhere between 1.2%/year and 2.3%/year.

**Geographic coverage:** This measure indicates what proportion of the regional population had some probability of influencing the estimated trend. It is calculated as the estimated proportion of the regional population inside degree-blocks with BBS data. The reported value is a geometric average annual proportion covered during the time period of the trend, where degree blocks are considered covered for all years between the first and last years that contributing routes were run in the degree-block. Estimates of regional and degree-block specific population proportions were derived using maps of a species' relative density across its Canadian range. These maps were created by overlaying existing species range maps with estimates of relative population density within the species' range, which were derived using the average number of individuals on BBS routes run from 1997 through 2007, and comparable counts from other monitoring programs.

**Local data weight:** Indicates the proportional contribution of local data (i.e., data from routes in the stratum) to estimating the local slope parameter, which describe the average rate of population change over the long-term. These slope terms (i.e., the beta term in equation 1 of Smith et al. 2014) are a major component of the long-term trend estimates. However, the long-term trend estimates also include information on the annual fluctuations, which rely only on local data. Therefore, this reliability measure is a useful metric to gauge the influence of data from outside the local region, but is not a comprehensive measure of the proportional contribution of local data to the trend estimates. The metric is calculated using a hierarchical pooling factor (Gelman and Pardoe 2006) that measures the degree to which each stratum's slope estimate is "pooled" or "shrunk" towards the mean slope across all strata included in the species' analysis (including BBS strata in the United States). The local data weight reliability measure is calculated as 1-this pooling factor, so that values closer to 1 indicate less pooling and a greater contribution by the local data, and values closer to 0, indicate more pooling and a greater contribution by the survey-wide mean. For composite trends (i.e., trends for any regions that are composed of > 1 stratum), we report a weighted geometric mean of the stratum-level values, weighted by the geographic area of each stratum.

Precision, coverage, and local data weight are combined into an overall reliability score for each trend. These overall reliability scores provide an overview that alerts the user to potentially unreliable trends. However, specific measures are also available to make an informed decision about the uncertainty associated with particular conclusions based on the trends. For example, a given trend may have a low reliability score because it is relatively imprecise (i.e., large credible interval), but, if for example the magnitude of the trend estimate is sufficiently large, this imprecision may not affect an overall conclusion that the species' population has increased.

## Literature cited

- Gelman,A., and I. Pardoe. 2006. Bayesian measures of explained variance and pooling in multilevel (hierarchical) models. Technometrics 48:241-251
- Robbins, C.S., D. Bystrak, and P.H. Geissler. 1986. The Breeding Bird Survey: its first fifteen years, 1965-1979. U.S. Fish and Wildlife Service Resource Publication157.
- Sauer, J. R., and W. A. Link. 2011. Analysis of the North American Breeding Bird Survey using hierarchical models. The Auk 128:87-98
- Smith A.C., M-A.R. Hudson, C. Downes, and C.M. Francis. 2014. Estimating Breeding Bird Survey trends and annual indices for Canada: how do the new hierarchical Bayesian estimates differ from previous estimates? Canadian Field-Naturalist 128:119-134.