Refining national asthma indicators
Delphi survey and correlation analysis
July 2009

Please note that there is the potential for minor revisions of data in this report.
Please check the online version at <www.aihw.gov.au> for any amendments.

The Australian Institute of Health and Welfare is Australia’s national
health and welfare statistics and information agency. The Institute’s mission is
 better information and statistics for better health and wellbeing.

© Australian Institute of Health and Welfare 2009

This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced without prior written permission from the Australian Institute of Health and Welfare. Requests and enquiries concerning reproduction and rights should be directed to the Head, Media and Communications Unit, Australian Institute of Health and Welfare, GPO Box 570, Canberra ACT 2601.

A complete list of the Institute’s publications is available from the Institute’s website <www.aihw.gov.au>. 

ISSN 1448-7594
ISBN 978 1 74024 938 6
Suggested citation 
Australian Centre for Asthma Monitoring 2009. Refining national asthma indicators: Delphi survey and correlation analysis. Cat. no. ACM 15. Canberra: AIHW.
Australian Institute of Health and Welfare 
Board Chair
Hon. Peter Collins, AM, QC

Director 
Penny Allbon

Any enquiries about or comments on this publication should be directed to:
Australian Centre for Asthma Monitoring
PO Box M77
Missenden Road NSW 2050
Phone: (02) 9114 0467
Email: acam@asthmamonitoring.org

Published by the Australian Institute of Health and Welfare

Contents
Acknowledgments
Summary
1	Introduction
1.1 	Purpose
1.2 	Background
Delphi surveys
Correlation analysis
1.3	Aim and objectives
Specific objectives of the Delphi survey
Specific objectives of the correlation analysis
2	Methods
2.1 	Delphi survey
2.2 	Correlation analysis
3	Results
3.1 	Delphi survey
Respondents
Question 1: Rating of asthma indicators
Question 2: Ranking of asthma indicators for retention
Question 3: Listing of potential asthma indicators for exclusion
Question 4: Suggestions for additional asthma indicators
3.2 	Correlation analysis
Whole population analysis
Subpopulation analysis
4 	Discussion
5 	Conclusions
Appendix 1: 	The Delphi survey initial questionnaire
Appendix 2: 	Sample of results and feedback from round one circulated to respondents in round two of the Delphi survey
Abbreviations
References
List of tables and figures
Acknowledgments
Authorship
Paola Espinel, Leanne Poulos, Patricia Correll, Wei Xuan, Helen Reddel and Guy Marks from the Australian Centre for Asthma Monitoring (ACAM) were responsible for the preparation of this document.
Contributors
A steering committee was established within ACAM to oversee this project and monitor its progress. Valuable guidance and contributions were received from the members of the steering committee group: Helen Reddel, Brett Toelle, Guy Marks, Patricia Correll, Leanne Poulos, Wei Xuan, Elena Belousova and Teresa To (corresponding member).
A special thank you goes to Gunnar Unger and Elena Belousova from the Woolcock Institute of Medical Research for the creation and management of the database and web-based questionnaires for the Delphi survey. 
The authors also acknowledge the participation of asthma experts in the Delphi survey. Their expertise and time were greatly appreciated.
Preparation of this report was also guided by the steering committee of the Australian System for Monitoring Asthma (ASMA). In particular, we are grateful to Nicholas Glasgow and Christine Jenkins for reviewing the draft.
The authors would also like to acknowledge the helpful comments on the draft provided by Adrian Webster from the Australian Institute of Health and Welfare.
Funding
This publication was funded by the Australian Government Department of Health and Ageing through the National Asthma Management Program. 

Summary
This report forms part of a review undertaken by ACAM to refine and simplify Australia’s national asthma data monitoring system. 
Why refine the list of asthma indicators?
Currently 24 indicators are recommended for monitoring asthma at a population level to guide policy relating to the prevention and management of asthma in Australia. This is too many for an efficient monitoring program. The overall aim of this body of work was to define an efficient set of indicators for asthma that are useful for stakeholders, avoid redundancy and can be measured reliably and validly using population data. 
Methods used to refine the list of asthma indicators
Two separate methods were employed to assess whether the number of asthma indicators recommended for monitoring could be reduced: a Delphi survey and an analysis of correlation among the indicators. A Delphi survey is an established qualitative research method used to systematically assimilate information from people with knowledge of the topic. We used a web-based Delphi survey, conducted over two rounds, to seek input from relevant asthma experts on the importance of each of the asthma indicators. We also asked them to rate each indicator in terms of its value in providing information to policy makers about the status of asthma in Australia. In the second round, panellists were sent their own previous responses, pooled results and anonymised comments of other participants. With this information they were asked to consider refining their answers. In conjunction with this process, correlation analyses were conducted to investigate whether there was any potential redundancy among the indicators.
Refined list of core asthma indicators
In conjunction with the correlation analysis, the Delphi survey has helped to obtain consensus about the most important asthma indicators for monitoring asthma at a national level. These indicators are:
1) prevalence of current asthma 
2) deaths (all ages)
3) deaths (5 to 34 years)
4) hospitalisations 
5) asthma control 
6) general practice encounters
7) asthma action plans
8) quality of life
9) preventer use  
10) costs of asthma. 
This core set of indicators should be used to gain population-based information on asthma in Australia. However, this list does not preclude the use of other indicators where they are relevant to a specific purpose.

1	
1	Introduction
1.1 Purpose
This project forms part of a systematic review currently being undertaken by the Australian Centre for Asthma Monitoring (ACAM) to refine and simplify Australia’s national asthma data monitoring system. The primary purpose of this study was to review the currently recommended list of 24 national asthma indicators in order to identify a smaller set of core indicators, which provide the most information and which are more effective at signalling change, for future asthma monitoring activities.
1.2 Background
The Australian System for Monitoring Asthma (ASMA) has been developed to monitor asthma data and inform policies addressing asthma in Australia. It is an indicator-based monitoring system in which defined measures of interest (indicators) have been developed for data collection and monitoring. The current asthma monitoring system is based on 24 asthma indicators (see Table 1). 
These were initially developed at a workshop convened by the Australian Institute of Health and Welfare (AIHW) in 2000 (AIHW 2000) and subsequently refined by ACAM in 2004 (Baker et al. 2004). They cover the areas of disease prevalence, impact (quality of life, disability, disease severity and mortality), risk factors, health service use and management practices. 
In the report Australian asthma indicators: Five-year review of asthma monitoring in Australia (ACAM 2007), ACAM recommended a review of the national asthma indicators in order to simplify the asthma monitoring system by excluding indicators that were no longer relevant, or that were redundant or not feasible to measure in the foreseeable future. Two overlapping projects were proposed to revise the existing set of indicators. One of these projects employed an expert opinion approach, via the use of a Delphi survey, to obtain expert input on the relative merit of each asthma indicator. The other project was data-driven and involved the assessment of statistical correlations between data series for some asthma indicators to identify redundancy within the existing set of indicators. 
Delphi surveys
A Delphi survey is a qualitative research method used to gain consensus among a panel of individuals who have knowledge of the topic (Keeny et al. 2006). It employs a series of surveys (referred to as rounds) in which each subsequent survey provides summary feedback to the panellists from the responses to the previous survey and invites panellists to modify their responses based on the views of the group (Keeny et al. 2001). By this means, it implements a structured group communication process in which panellists can be influenced by the responses of others in the group. The goal of a Delphi survey is to organise a debate, collect and synthesise opinions and to achieve a degree of consensus and agreement among participants (Hader & Hader 1995). 
A key component of the methodology of the Delphi technique is that the identities of panellists and their individual responses are kept hidden from the rest of participants (Hasson et al. 2000). This is designed to avoid dominance in the group by more outspoken or prestigious participants. However, although the panellists remain anonymous to each other, they are not anonymous to the investigators who need to know their identities in order to feed back responses and to follow up panellists during rounds. A high retention rate along all survey rounds is crucial as the number of participants is usually small. 
A Delphi survey is a valuable tool for accessing the judgment of experts in circumstances where there are limits in the ability of empirical methods to ascertain information because knowledge is, by its nature, incomplete or unavailable (Powell 2003). This is relevant to the situation existing for most asthma indicators, and this method has the potential to provide helpful information for refining national asthma indicators.
Table 1: Current national asthma indicators
Indicator	Description	
1	Ever asthma	Reporting ever having doctor-diagnosed asthma	
2	Current asthma	Reporting doctor-diagnosed asthma plus symptoms of or treatment for asthma in the last 12 months	
3	Current wheeze	Reporting wheeze in the preceding 12 months	
4	Airway hyperresponsiveness	Proportion of the population who are diagnosed with airway hyperressponsiveness	
5	Deaths (all ages)	Deaths due to asthma in the population	
6	Deaths (age 5 to 34 years)	Deaths due to asthma among people aged 5 to 34 years	
7	Hospitalisations	Episodes of hospitalisation for asthma	
8	Hospital patient days	Patient days (or ‘bed days’) in hospital for asthma 	
9	Individual hospitalisations	People hospitalised for asthma 	
10	Hospital re-admissions	Re-admissions to hospital for asthma within 28 days of a previous admission for asthma 	
11	Emergency department attendances	Attendances at emergency departments in the population for asthma each year	
12	Re-attendances	Re-attendance at either hospital or an emergency department for asthma within 28 days of a previous attendance for asthma 	
13	General practice encounters	General practice encounters for asthma	
14	Urgent asthma visits	Total healthcare visits (hospital, emergency department and general practice) for asthma exacerbations or worsening asthma	
15	Asthma Cycle of Care uptake	Asthma Cycle of Care (formerly Asthma 3+ Visit Plan) Practitioner Incentive Program payments	
16	Asthma action plans	People with asthma who have a written asthma action plan	
17	Preventer use	People with asthma who use preventers (inhaled corticosteroids, leukotriene receptor antagonists or similar drugs) regularly	
18	Quality of life	People with asthma who report poor health-related quality of life 	
19	Smoking	Current smoking among people with asthma	
20	Children residing with smokers	Smoking in households where children with asthma reside	
21	Spirometry	People with asthma who have had spirometry within the last 12 months	
22	Asthma control	A composite indicator developed from measures of symptoms and medication use to impute the proportion of people with asthma who have poor clinical control	
23	Occupational asthma	Asthma caused by occupational exposure	
24	Costs of asthma	An index derived from expenditure and burden of disease data to examine the costs of asthma to individuals	
Note: These are the indicator descriptions provided to panellists involved in the Delphi survey. 

Correlation analysis 
Correlation is a measure of the strength of association between two continuous variables. Two variables that are highly correlated contain essentially the same information and, hence, one of them can be considered redundant. We applied this methodology to the analysis of indicators that had been measured using existing data sources to see if any combination of two indicators were highly correlated. If two items were strongly correlated we considered that one of them was redundant and could be excluded from the indicator list. 
Approaches such as the expert opinion and data-driven methods used here are consistent with those used in the creation and refinement of health indicator lists in other jurisdictions. In Scotland, a mixed approach was used to establish a robust set of mental health indicators for adults (NHS Health Scotland: Parkinson 2007; Parkinson 2006). In the United States, committee members were asked to rank-order lists of national health indicators according to their personal judgment (IOM 1999). A similar method was used in Canada, where a modified Delphi process was undertaken to develop a draft list of population health indicators at a consensus conference held in 1999 (Canadian Institute for Health Information (CIHI) 1999). Indicators were retained or rejected on the basis of average scores. A second consensus conference was held in 2004 to identify information gaps, validate the originally proposed indicators and identify potential new indicators (CIHI 2005).
1.3	Aim and objectives 
The overall aim of this body of work was to define an efficient set of indicators for asthma that are useful for stakeholders, avoid redundancy and can be measured reliably and validly using population data. 
Specific objectives of the Delphi survey
There were three main objectives of the Delphi survey:
1) 	To consult individuals with interest and expertise in asthma and data monitoring on the priority of individual indicators for monitoring asthma in Australia
2) 	To obtain consensus among asthma and data monitoring experts on which indicators are the most important in the asthma monitoring system
3) 	To engage asthma and data monitoring experts in the asthma monitoring process. 
Specific objectives of the correlation analysis
The objectives of the correlation analyses were:
1)	To examine correlations among asthma indicators that are classified by sex, age group, geographical remoteness and socioeconomic disadvantage of location of residence
2)	To use this information to identify a smaller set of indicators that provides independent information with minimal redundancy.

2	Methods
2.1 Delphi survey 
Pilot test
The initial questionnaire was designed as a web-based one and pilot-tested by five respondents prior to the commencement of the study. Minor changes to the wording of the participant information statement and questionnaire were made to ensure clarity. The web-based format was also adjusted to ensure simplicity and completion. 
Ethics
Approval was sought and given by the University of Sydney Human Research Ethics Committee on 30 July 2008 (Ref No. 11003). 
Sample 
A list of potential participants was identified at a national level by investigators, based on all of the following selection criteria: 
1) 	Currently practicing respiratory physician, paediatrician, general practitioner, asthma researcher, epidemiologist, asthma educator, policy maker, or representative from health departments or relevant interest groups
2) 	Highly knowledgeable about some aspects of asthma and data monitoring in Australia
3) 	Interest in monitoring the status of asthma in Australia using indicators.
ACAM employees and members of the project steering committee were excluded from participation in this project. 
Recruitment 
Potential participants were contacted and invited to participate by email. They received a Participant Information Statement with an explanation of the Delphi survey purposes and process, and the activities they would be asked to undertake as panellists (including the time it would take, the number of rounds and how the information would be used). Detailed instructions on how to complete the initial questionnaire were also provided. 
The identities of the panel members were not revealed to the panellists, and participants were reassured that their responses would be anonymous to the rest of the panel at all times. It was also made clear that their participation was voluntary and they could withdraw at any point without penalty. It was planned to include approximately 20 panellists. 
Email addresses were obtained from an existing list of stakeholders held at ACAM. This list of stakeholders has evolved since ACAM’s inception in 2002. It contains an extensive list of experts who have worked directly and indirectly with ACAM in a variety of capacities, and is regularly maintained. 
Questionnaire
A full copy of the questionnaire is provided in Appendix 1. Briefly, panellists were asked to rate the value of each of the asthma indicators and to identify between five and ten indicators that they recommended for retention. They were also asked to list up to three indicators they considered could be excluded. Panellists were encouraged to provide qualitative explanations for their rankings and selections, although these were optional. Finally, participants were given the opportunity to suggest additional asthma indicators they believed could be important in monitoring the status of asthma in Australia. The same questionnaire was used in the second round of the survey, but no free text was collected. 
Survey rounds 
The survey was designed to include up to three rounds of testing, although only two were conducted (see below). Those who completed the initial questionnaire were invited to participate in the second round of the survey. In the second round, respondents were given feedback including their own previous responses, pooled results from all respondents and anonymised comments of other participants (see Appendix 2). Participants were asked to consider refining their answers based on this feedback provided.
Data collection 
Surveys were administered as web-based questionnaires with communication to the panellists via email. Completed surveys were stored in a password protected database. Participants were given two weeks to complete the initial questionnaire, and three weeks to complete the second questionnaire in an effort to ensure a high retention rate. Non-responders received two follow-up emails as reminders to complete the survey.
2.2 Correlation analysis 
We investigated correlations among several of the asthma indicators to identify potential redundancy. In order to investigate correlation among indicators, it is necessary to aggregate the data for the indicator by population characteristics (age group and sex) and geographical characteristics (socioeconomic status and remoteness of the location of residence). Indicators that are not highly correlated with any other indicator offer independent information. On the other hand, highly correlated indicators may indicate redundancy and there may be opportunities to cull one of the indicators from the recommended list. Six of the 24 indicators were able to be aggregated by age group, sex, Socio-Economic Indexes For Areas (SEIFA) (providing an indication of socioeconomic status) and Australian Standard Geographical Classification (providing an indication of remoteness of residence). These were:
•	Ever asthma, defined as the proportion of ever having doctor diagnosed asthma per 100,000 resident population (data source: National Health Survey)
•	Current asthma, defined as the proportion of people with current asthma (ever being doctor diagnosed with asthma and still having it) per 100,000 resident population (data source: National Health Survey)
•	Asthma action plans, defined as the proportion of people with current asthma who had an asthma action plan (data source: National Health Survey)
•	Deaths (all ages and 5 to 34 years), defined as deaths with asthma listed as the underlying cause of death (ICD-10 codes J45 and J46 ) per 100,000 resident population (data source: National Mortality Database)
•	Hospitalisations, defined as hospital separations  with asthma listed as the principal cause (ICD-10-AM codes J45 and J46) per 100,000 resident population (data source: National Hospital Morbidity Database)
•	Hospital patient days, defined as the number of patient days in hospital due to asthma per 100,000 resident population (data source: National Hospital Morbidity Database).
Data for each of the six indicators were derived from the various data sources used to monitor these indicators on a regular basis, including the National Mortality Database, the National Hospital Morbidity Database and the National Health Survey. Data were aggregated by sex, age group (four levels), SEIFA (five levels) and remoteness (three levels). A correlation matrix was constructed to examine how these indicators were correlated to each other. For the purpose of this analysis, we defined redundancy as >80% shared variance, in other words, a correlation coefficient >0.9. 
Additionally, some subpopulation analyses were conducted if the graphical representation of the correlation analysis indicated the possibility of another factor, such as age or sex, having an important effect on the correlation. An investigation of the correlation between the trend in deaths (all ages) and deaths (5 to 34 years) indicators over time was also conducted. 

3	Results
3.1 	Delphi survey
Respondents
Sixty-two asthma experts from different disciplines across Australia were invited by email to participate in the Delphi survey. Thirty-two panellists (52%) completed the initial questionnaire and 27 (84%) of these completed the second questionnaire. Among the 32 initial responders, 12 (37.5%) were women. Panellists included eight currently practicing respiratory physicians, two paediatricians, three general practitioners, and one asthma educator. There were also asthma researchers (12), epidemiologists (9), and several representatives from health departments (6), Australian System for Monitoring Asthma steering committee (10), the Australian Institute of Health and Welfare (AIHW) (2), the Cooperative Research Centre for Asthma and Airways (Asthma CRC) (2) and Asthma Foundations around Australia (5). It should be noted that these categories are not mutually exclusive. Each of the states and territories were represented among the respondents.
In this report, results are presented in the same order as the questionnaire. In general, as shown in Figures 1 and 2, responses from the second survey were consistent with the results from the initial survey.
Question 1: Rating of asthma indicators 
Respondents were asked to rate and rank current asthma indicators in terms of their value in providing information to policy makers about the status of asthma in Australia. Figure 1 shows the group score range (minimum and maximum score), the number of responders who ranked each indicator at each rating, and the median rating for each indicator. It can be seen that there was a broad range of responses, but some extremes were only rated by one respondent. 
Current asthma, urgent asthma visits, deaths, hospitalisations and Emergency Department attendances were the top five indicators rated in the initial round. These same indicators were also the ones most strongly supported in the second round of the survey. 

Notes: These results arose from Question 1 of the Delphi survey where panellists were asked ‘Please rate each of the indicators in terms of its value in providing information to policy makers about the status of asthma in Australia, where 1 indicates it is most important and 5 indicates it is least important or redundant.’ Indicators are listed in order from lowest to highest median (i.e. most to least important from top to bottom). The grey boxes indicate the score range for the initial survey (n=32), and the blue boxes indicate the range for the second survey (n=27). Numbers inside the boxes indicate the number of respondents who rated each indicator at each score. The black vertical bar indicates the median.
Figure 1: Rating of asthma indicators in terms of their value in providing information to policy makers about the status of asthma in Australia	
Question 2: Ranking of asthma indicators for retention
Panellists were asked to select a limited number of indicators (5–10) for retention. 
A sum score was calculated for both survey rounds, based on the priority level for retention given by each respondent. A priority level of ‘1’ was awarded 10 points, a priority level of ‘2’ was awarded 9 points, and so on to 1 point for priority level of ‘10’. For each indicator, the sum score was calculated as the sum of all priority level scores divided by the total number of respondents.
The results of the first and second round were generally consistent with each other (Figure 2). Current asthma and deaths (all ages) were ranked as the most preferred indicators for retention in both survey rounds. Free text answers provided by some respondents were consistent with this ranking. For example, in relation to deaths, comments included: ‘sentinel event/outcome’ and ‘an important endpoint’ and ’death is the ultimate and unequivocal outcome and thus an excellent indication of whether outcomes are improving or worsening‘. In addition, hospitalisations, deaths (5 to 34 years) and costs of asthma were considered worthy of retention by respondents in the first round. On the other hand, there was a substantial increase between round one and round two in the percentage of respondents recommending inclusion for asthma control and urgent asthma visits. This may reflect the strongly positive sentiments expressed by other respondents in the first round and subsequently circulated with the round two survey, such as asthma control is ‘one of the key ultimate outcomes we are seeking. Good control provides a clear picture of whether strategies are working or not’ and ‘has been shown to be a rigorous yet flexible measure in primary care and is responsive to change’. In terms of urgent asthma visits, round one respondents described it as an ‘important outcome measure’ and ‘a good marker of whether asthma is well controlled’. A full list of all the free text answers is provided in Appendix 2.
Question 3: Listing of potential asthma indicators for exclusion
Asthma Cycle of Care uptake and airway hyperresponsiveness were consistently identified as potential indicators for exclusion during both survey rounds. Asthma Cycle of Care uptake was considered by round one participants as ‘complex’, ‘not a great indicator of activity’ and ‘not useful to measure’ while airway hyperresponsiveness was ‘rarely tested’, ‘probably not feasible’ and ‘difficult to measure and thus difficult to be representative of the population’. On the other hand, there were some alternative comments that supported the retention of these indicators. In the second round, there was a significant increase in the number of respondents recommending exclusion for occupational asthma, airway hyperresponsiveness and Asthma Cycle of Care uptake (Figure 2). Occupational asthma had been described as ‘too specific’ and as a ‘subgroup of asthma’. 

Notes: For each indicator, the bar at the top represents results from the initial survey and the bar below shows results for the second survey. Retention (question 2): Panellists were asked: ‘If we had to limit the number of asthma indicators, which ones, from the previous list of 24, would you choose to keep? Please rank a minimum of 5 indicators, starting from the most important. You can rank up to 10 indicators that you think are important.’ Dark blue shows the percentage of responders who thought the indicator should be retained and gave it a priority level of 1–5. Light blue shows the percentage of responders who thought the indicator should be retained and gave it a priority level of 6–10. Sample sizes were: initial survey n=30; second survey n=25. At the right of the bars a sum score has been calculated for both survey rounds, based on priority level for retention where priority level 1 awarded 10 points, priority level 2 awarded 9 points, and so on to 1 point for priority level 10. Indicators are listed in descending order of the sum score for the second round. Exclusion (question 3): Panellists were asked: ‘Please list up to five indicators from the previous list of 24 that you believe could be excluded from the asthma data monitoring system.’ Sample sizes were: initial survey n=27; second survey n=22. A full list of the voluntary free text answers provided as reasons for retention or exclusion is provided in Appendix 2.
Figure 2: Asthma indicators recommended for retention and exclusion	
Question 4: Suggestions for additional asthma indicators 
Seven respondents made suggestions about additional indicators for asthma monitoring or proposed subgroup analyses (see Appendix 2 for the full list of suggestions). The two suggested additional asthma indicators, which were circulated with the second survey round, were asthma self-management or health knowledge and frequency of use of relievers. Figure 3 shows the rating of these two additional indicators. Panellists agreed that the two additional indicators were important in monitoring the status of asthma in Australia. However, they did not rate as highly as some of the existing indicators. 

Notes: For the initial survey panellists were asked ‘If you have any further suggestions for indicators that you believe could be important in monitoring the status of asthma in Australia, please list below (optional).’ In the second survey panellists were asked to rate these extra indicators: ‘Please rate each of the indicators in terms of its value in providing information to policy makers about the status of asthma in Australia, where 1 indicates it is most important and 5 indicates it is least important or redundant.’ The blue boxes indicate the score range (n=27). Numbers inside the boxes indicate the number of respondents who rated each indicator at each score. The black vertical bar indicates the median.
Figure 3: Rating of additional asthma indicators proposed by panellists	
3.2 Correlation analysis
Whole population analysis
Using whole population analysis, prevalence of ever asthma and prevalence of current asthma were the two indicators that were most highly correlated with each other (r=0.82) (Table 2). Hospitalisations and hospital patient days were also highly correlated (r=0.80). Deaths due to asthma was moderately correlated to the ever asthma indicator (r=0.48) as well as the current asthma indicator (r=0.30).   Although some correlations were relatively strong, none were greater than 0.9 in this whole population analysis and, hence, none met the criterion for redundancy.
Table 2: Correlation matrix among the six asthma indicators
	Ever asthma (1)	Current asthma (2)	Asthma action plan (3)	Deaths (all ages) (4)	Hospitalisations
 for asthma (5)	
Ever asthma (1)	1.00					
Current asthma (2)	0.82	1.00				
Asthma action plan (3)	–0.07	–0.08	1.00			
Deaths (all ages) (4)	0.48	0.30	0.21	1.00		
Hospitalisations (5)	–0.31	–0.23	0.34	–0.22	1.00	
Hospital patient days (6)	0.06	–0.06	0.33	0.39	0.80	
Sources: ABS 2004–05 National Health Survey; AIHW National Mortality Database; AIHW National Hospital Morbidity Database; Australian Bureau of Statistics
When these correlations between indicators were presented graphically, with each point representing a defined subgroup of the population with a specific age, sex, socioeconomic status and remoteness, it was evident that in some cases, that subgroup analyses were appropriate (data not shown). 
Subpopulation analysis
We investigated the correlation between hospital patient days and hospital separations stratified by broad age groups, 0–14 years, 15–34 years and 35 years and over (Figure 5). It can be seen that in adults aged 35 years and over, the slope of the regression line is steeper than in the other two age groups, presumably due to the narrower range of hospital separations and wider range of hospital patient days, indicating longer length of stay. However, in children aged 0–14 years, both hospital separations and hospital patient days covered a wide range (Figure 5). The correlation coefficient between hospital separations and hospital patient days was very high within each age subgroup (0.972, 0.979 and 0.906 for age groups 0–14 years, 15–34 years and 35 years and over, respectively; Figure 5), while the overall correlation between these two indicators was lower (0.80; Table 2). Hence, the two indicators, hospital patient days and hospital separations, give different information across the full age range, even though they are highly correlated within each age group. These subgroup analyses suggest that it is not possible to use hospital patient days or hospital separations alone as national asthma indicators across the whole Australian population. Both give independent information and should be monitored. On the other hand, if the data are stratified by age group, into children (age 0–14 years), young adults (age 15–34 years) and older adults (age 35 years and over), it is only necessary to monitor either hospital patient days or hospital separations since both indicators provide very similar information.
We found a moderate correlation between the annual rates of deaths (all ages) and deaths (5 to 34 years) (0.84) in males and females over time (data not shown). However, the correlation was not high enough to meet our criterion for redundancy.

Sources: AIHW National Hospital Morbidity Dataset; Australian Bureau of Statistics
Figure 4: Correlation between hospital separations and hospital patient days by age group	
4 	Discussion 
Asthma data monitoring provides valuable information on changes in the impact of disease at a population level and for examining the effectiveness of health policy and management strategies over time. Currently, there are 24 national asthma indicators that measure selected aspects of the disease. It was believed that some of these indicators were no longer relevant or feasible; therefore a systematic review was recommended to simplify the asthma monitoring system. In this report we are presenting results from two independent processes: a Delphi survey and a correlation analysis conducted in order to reassess the current indicators and to recommend a priority list of core indicators that will provide high quality information for future asthma monitoring activities. 
Asthma experts around Australia took part in a two-round Delphi survey where asthma indicators were rated and ranked based on their value for guiding policy, prioritising strategies for effective management and providing information about effectiveness of these strategies. The similarity between the first and second rounds indicated that there was no need to have a third round, therefore the research team stopped the process after the second round. The results presented above revealed that there are some indicators preferred by the respondents and that others might be excluded from the monitoring system. 
The asthma indicators most strongly endorsed by panellists, and hence recommended by the research team for retention, were current asthma, deaths (all ages) and hospitalisations. Free text answers provided by some respondents supported these results. For instance, participants declared that hospitalisations provided a ‘measure of severity’ and ‘serious adverse event’ and it was claimed as ‘harder data’. Asthma control was also highly ranked by participants in both survey rounds. However, there were conflicting views in regards to asthma control as it was believed it could be ‘difficult to measure’ and might not be ‘policy relevant’. This variation may reflect varying levels of knowledge by different respondents about standardised methods for assessing asthma control. Finally, costs of asthma was given a low priority by respondents but quite a high preference for inclusion. Panellists commented that costs of asthma was relevant for ‘prioritising’ and for identifying ‘strategies to guide policy’. The research team also recommended the retention of the following indicators: general practice encounters, asthma action plans, quality of life and preventer use. Although these indicators did not score as highly as some of the other indicators in the sum score for retention, few people recommended them for exclusion. For people with persistent asthma, current guidelines recommend use of preventer medications and ownership of an asthma action plan. Hence, the research team felt that the preventer use and asthma action plans indicators needed to be retained. It should be noted that the Australian term ‘preventer’ corresponds to medications called ‘controllers’ in many other countries and clinical practice guidelines, so the meaning of the indicator ‘preventer use’ should be defined in any publication accessible to an international audience. General practice encounters enables the quantification of one of the main health service utilisation measures for asthma. Quality of life represents an holistic outcome not encompassed by other measures. For these reasons, general practice encounters, asthma action plans, quality of life and preventer use were recommended as indicators to be retained in the core indicator list.
Panellists who participated in the Delphi survey recommended the following indicators for exclusion: individual hospitalisations, re-attendances, occupational asthma, hospital patient days, airway hyperrresponsiveness, children residing with smokers, current wheeze, Asthma Cycle of Care uptake and spirometry. Although most respondents agreed that these indicators should be excluded, there were some alternative comments that advocated indicators for inclusion. For example, some panellists commented that individual hospitalisations determined ‘effectiveness of asthma control and severity’ and that current wheeze was ‘needed for population prevalence’. 
The whole population correlation analysis did not identify any of the six indicators assessed that could be eliminated on the basis of redundancy. In the case of the hospital patient days and hospitalisations indicators, the whole population results showed that the two indicators provide independent information, as shown by the low correlation coefficient. However, when the data were analysed separately by broad age group (0–14 years, 15–34 years, 35 years and over), the correlation coefficients met the criterion for redundancy and it was found that it was only necessary to measure either hospital patient days or hospitalisations since they provide similar information. Thus, based on the results from both the Delphi survey and the correlation analysis, hospital patient days was added to the list of indicators recommended for exclusion.
For many of the indicators, the results of the Delphi survey provided a clear indication of whether or not the indicator should be included or excluded from the final list. However, in some cases the decision was not immediately apparent. In these instances, the research team steering committee determined whether to include or exclude the indicator based on the results from the Delphi survey and the correlation analysis (where available), as well as by drawing on knowledge gained through monitoring the indicators in the past. 
The research team proposed to exclude Emergency Department attendances because of incomplete national coverage from existing datasets. Although urgent asthma visits is an accepted indicator at an international level, it is not feasible for use in Australia as the urgency of visits is not able to be assessed from currently available administrative datasets. Therefore, the research team recommended exclusion of urgent asthma visits from the core indicator list. The exclusion of individual hospitalisations and ever asthma was also recommended for the reasons that follow. The research team decided to exclude individual hospitalisations from the list of recommended indicators since there was a similar proportion of respondents in the Delphi survey that recommended retention of the indicator as there was recommending exclusion of the indicator. Furthermore, total hospitalisations would already provide a good indication of the burden of asthma in the health care system, and this had already been included in the core indicator list on the basis of the Delphi results. Ever asthma was excluded since the current asthma indicator provided similar (though not identical) information about the diagnosis of asthma but with the added value of taking into account recent symptoms or treatment. 
Deaths (age 5 to 34 years) was included in the final indicator list. Panellists consistently rated the indicator highly in the two rounds of the Delphi survey. Furthermore, information from the correlation analysis indicated that it provided independent information from the deaths (all ages) indicator. Free text answers from the Delphi survey process such as ‘simple non-controversial indicator’ and ‘deaths among young people for whom the asthma diagnosis is reasonably clear’ also supported the inclusion of the deaths (5 to 34 years) indicator.
Smoking was identified as ‘one public health factor proven to reduce the frequency and severity of asthma’ but was seen as ‘a separate issue’ by other respondents. Hospital re-admissions was declared as ‘an indication of potential for improvement in delivery of care’ and a ‘measure of quality of hospital and GP care’. However, there was no final agreement about these two indicators.  
The process of the Delphi survey also gained information on additional indicators that panellists felt had been excluded from the initial set of 24 asthma indicators. Relevant experts suggested that it may be important to monitor reliever use and asthma knowledge and panellists agreed that these were quite important asthma indicators in terms of their value in providing information to policy makers about the status of asthma in Australia.
While the aim of these analyses was to recommend a short list of core asthma indicators, it should be noted that monitoring some of the other indicators at a population level and in smaller areas should still be conducted where relevant and necessary. Furthermore, some of those recommended for exclusion from the core indicator list may still be useful for monitoring the outcomes of specific interventions that target these indicators. The list of core asthma indicators has been formulated on the basis of the analyses described in this report as a guide for indicators which will provide the most important information about asthma at a national level. The list is intended as an indication of the key areas that should be monitored if limited resources were available and a snapshot of the national burden of asthma was required. It may be necessary to further refine this list in the future, or to add new asthma indicators as the need arises.
5 	Conclusions
Based on the outcome of the Delphi survey and the correlation analysis reported here we recommend that the following be retained as core asthma indicators:
1)	prevalence of current asthma 
2)	deaths (all ages)
3)	deaths (5 to 34 years)
4)	hospitalisations
5)	asthma control 
6)	general practice encounters
7)	asthma action plans
8)	quality of life
9)	preventer use  
10)	costs of asthma.
On the basis of the same analyses, we recommend that the following be deleted from the core asthma indicator list:
1)	individual hospitalisations 
2)	re-attendances
3)	occupational asthma 
4)	hospital patient days
5)	airway hyperresponsiveness
6)	children residing with smokers 
7)	current wheeze
8)	Asthma Cycle of Care uptake
9)	spirometry
10)	Emergency Department attendances 
11)	urgent asthma visits 
12)	ever asthma. 
Several of the indicators recommended for exclusion from the core indicator list should still be monitored at a population level and in smaller areas when the need arises. For example, monitoring these indicators may still be useful for assessing the effectiveness of specific interventions that target these indicators.
Finally, we could not reach any conclusions about the following indicators on the basis of these data:
1)	smoking 
2)	hospital re-admissions.
Appendix 1: The Delphi survey initial questionnaire
Refining asthma indicators: Delphi survey
Initial questionnaire
Thank you for agreeing to participate in this Delphi survey on the priorities of national indicators for asthma data monitoring in Australia. The survey forms part of a review of asthma indicators used in national monitoring that is being carried out by the Australian Centre for Asthma Monitoring. 
This questionnaire round is the first of up to three rounds of the survey. Please try to answer all questions, even though we do not expect you to have in depth knowledge of all of them. You will have the opportunity to revise your answers with subsequent rounds of the survey. 
In these surveys, you will be asked to develop priorities among the current national asthma indicators. Most of the questions can be answered with only a single selection. Where appropriate, a space is also provided for you to comment on the underlying reasons for your responses.
•	In formulating your responses, you are not expected to assess the feasibility or cost of data collection for the indicators. 
Once we have received responses from all panellists, we will collate and summarise the findings and formulate the second questionnaire. You should receive this in the next month. 
We assure you that your participation in the survey and your individual responses will be strictly confidential to the research team and will not be divulged to any outside party, including other panellists. 

Username (email address) 
Password (pre-registered): ……………………..

1.	
Below, is a list of asthma indicators currently used to monitor asthma in Australia. Please rate each of the indicators in terms of its value in providing information to policy makers about the status of asthma in Australia, where 1 indicates it is most important and 5 indicates it is least important or redundant.
The following questions might be helpful in guiding your assessment of the value of each indicator:
•	Is the indicator useful for guiding policy that aims to reduce the burden of asthma?
•	Is the indicator helpful in prioritising strategies for the effective management of asthma?
•	Does the indicator provide information about whether the policies to manage asthma are working?

In formulating your responses, you are not expected to assess the feasibility or cost of monitoring the indicators.
	INDICATOR NAME AND DESCRIPTION	Rating (1=most important-5=least important)	
		1	2	3	4	5	
1	Ever asthma: Reporting ever having doctor-diagnosed asthma						
2	Current asthma: Reporting doctor-diagnosed asthma plus symptoms of or treatment for asthma in the last 12 months						
3	Current wheeze: Reporting wheeze in the preceding 12 months						
4	Airway hyperresponsiveness: Proportion of the population who are diagnosed with airway hyperressponsiveness						
5	Deaths (all ages): Deaths due to asthma in the population						
6	Deaths (age 5 to 34 years): Deaths due to asthma among people aged 5 to 34 years						
7	Hospitalisations: Episodes of hospitalisation for asthma						
8	Hospital days: Patient days (or “bed days”) in hospital for asthma 						
9	Individual hospitalisations: People hospitalised for asthma 						
10	Hospital re-admissions: Re-admissions to hospital for asthma within 28 days of a previous admission for asthma 						
11	Emergency department attendances: Attendances at emergency departments in the population for asthma each year						
12	Re-attendances: Re-attendance at either hospital or an emergency department for asthma within 28 days of a previous attendance for asthma 						
13	General practice encounters: General Practice encounters for asthma						
14	Urgent asthma visits: Total healthcare visits (hospital, emergency department and general practice) for asthma exacerbations or worsening asthma						
15	Cycle of Care uptake: Asthma Cycle of Care (formerly Asthma 3+ Visit Plan) Practitioner Incentive Program payments						
16	Asthma action plans: People with asthma who have a written asthma action plan						
17	Preventer use: People with asthma who use preventers (inhaled corticosteroids, leukotriene receptor antagonists or similar drugs) regularly						
18	Quality of life: People with asthma who report poor health-related quality of life 						
19	Smoking: Current smoking among people with asthma						
20	Children residing with smokers: Smoking in households where children with asthma reside						
21	Spirometry: People with asthma who have had spirometry within the last 12 months						
22	Asthma control: A composite indicator developed from measures of symptoms and medication use to impute the proportion of people with asthma who have poor clinical control						
23	Occupational asthma: Asthma caused by occupational exposure						
24	Costs of asthma: An index derived from expenditure and burden of disease data to examine the costs of asthma to individuals						

2.	 If we had to limit the number of asthma indicators, which ones, from the previous list of 24, would you choose to keep? Please rank a minimum of 5 indicators, starting from the most important. You can rank up 10 indicators that you think are important.
In making your decisions, please consider the guidelines provided in question 1:
•	Is the indicator useful for guiding policy that aims to reduce the burden of asthma?
•	Is the indicator helpful in prioritising strategies for the effective management of asthma?
•	Does the indicator provide information about whether the policies to manage asthma are working?

A space is provided for you to briefly explain the reason for your ranking if you wish. This additional information is optional, and could help us understand the reasons some indicators are valued over others
1.	Indicator (drop down menu)………………………..
Reason: ____________________________________________________________________
2.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
3.	Indicator (drop down menu)………………………
Reason: ____________________________________________________________________

4.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
5.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
6.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
7.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
8.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
9.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
10.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________

3.	Please list up to five indicators from the previous list of 24 that you believe could be excluded from the asthma data monitoring system.
Once again, in making your decisions, please consider the guidelines provided in question 1:
•	Is the indicator useful for guiding policy that aims to reduce the burden of asthma?
•	Is the indicator helpful in prioritising strategies for the effective management of asthma?
•	Does the indicator provide information about whether the policies to manage asthma are working?

1.	Indicator (drop down menu)………………………..
Reason: ____________________________________________________________________

2.	Indicator (drop down menu)……………………….
Reason: ____________________________________________________________________
3.	Indicator (drop down menu)……………………,
Reason: ____________________________________________________________________
4.	Indicator (drop down menu)……………………,
Reason: ____________________________________________________________________
5.	Indicator (drop down menu)……………………,
Reason: ____________________________________________________________________

4.	If you have any further suggestions for indicators that you believe could be important in monitoring the status of asthma in Australia, please list below (optional):
_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Appendix 2: Sample of results and feedback from round one circulated to respondents in round two of the Delphi survey
Refining Asthma Indicators: Delphi survey
Initial survey results and 
Invitation to complete second survey

Thanks for your responses to the first phase of the Delphi process. The process is designed, over two or three phases, to develop a consensus among the participants about the asthma indicators we should continue to use. With this in mind we now present you the summarised results from all 32 respondents to the first round of the survey, together with a reminder of your own responses. We have also included the free-text comments made by respondents to explain their rankings. Taking the responses from the first round into account, please review the material below and respond again to the questions in the Delphi survey.
After you have read the results from the first round, please follow the link to complete the second round of the short, secure, web-based questionnaire. Please consider whether, in the light of your colleagues’ responses, you would like to alter your answers.
Remember that your responses are strictly confidential to the research team and that your participation in this project is entirely voluntary. However, we would like to stress the importance of having everyone who responded to the first round to complete the remaining couple of rounds, when agreement and consistency in regards to redefining the asthma indicators is being sought.
Please find the link and instructions for the second survey at the end of this document (App’x 2).
If, at any time, you would like to discuss the Delphi survey or the review of Australian asthma indicators, please do not hesitate to contact Leanne Poulos, Acting Project Manager, Australian Centre for Asthma Monitoring on (02) 9114 0467 (telephone) or lmp@woolcock.org.au (email). 
Further information about the study is available at http://www.asthmamonitoring.org/PDF/Delphi%20Project%20Outline.pdf 
If  you would like to raise any concern or complaints regarding this study, please contact the Senior Ethics Officer, Ethics Administration, University of Sydney, (02) 9515 4811 (telephone); (02) 9351 6706 (facsimile) or gbriody@usyd.edu.au (email).

RESULTS FROM ROUND 1-Question 1. You were asked “Please rate each of the indicators in terms of its value in providing information to policy makers about the status of asthma in Australia, where 1 indicates it is most important and 5 indicates it is least important or redundant.” 
Responses from the 32 participants are shown in Figure 1

Figure 1. Rating of Asthma Indicators. 
Indicators are listed in order from lowest to highest median (i.e. most to least important).  The blue boxes indicate the group score range (minimum and maximum score), (I) indicates the median and (    ) indicate your individual responses  

RESULTS FROM ROUND 1-Question 2. You were asked: “If we had to limit the number of asthma indicators, which ones, from the previous list of 24, would you choose to keep? Please rank a minimum of 5 indicators, starting from the most important. You can rank up to 10 indicators that you think are important”. The results from 30 responders are shown in Figure 2 and Table 1

Figure 2. Asthma indicators to be kept.  
For each indicator, the bar shows the percentage of respondents who recommended that the indicator should be retained. The indicators are listed in order of the percentage of respondents recommending retention. Dark green shows the percentage of respondents who thought the indicator should be retained and gave it a priority level of 1-5, and light green shows the percentage of respondents who thought the indicator should be retained and gave it a priority level of 
6-10. Next to this, the median ranking that these respondents gave each indicator for its priority to be retained is provided, as well as your own ranking. 

Table 1. Reasons for keeping Asthma indicators. 
Indicators are listed in order from the lowest to the highest median. Ranking is classified into 2 columns (1-5 and 6-10). At the end of each comment, the ranking is given in brackets. 
Indicator	                                           Ranking 1– 5	                                          Ranking 6 –10	
Current asthma	“Improves accuracy wrt genuine asthma” (4)	“Extent of the impact on the population”(10)	
Hospitalisa-tions	“Measure of burden (severity and hosp usage) and possibly reflective of effectiveness of programs”(1)  “Index of community control”(2)  “It captures the severe end of the disease and once again a good indicator to assess the reduction in the burden of disease”(2)  “Trends in burden and effectiveness of management”(3)  “Major health care usage”(3)  “Harder data”(4)  “Measure of severity and of care by GPs”(4)  “Serious adverse event”(5)  “Information about the use of health care resources is an important measure of how well we're doing at addressing asthma in the community, and in assessing the overall cost of the condition”(5)	“Important re looking at avoidable admissions - could be use regionally”(6)  “Useful morbidity statistic in determining burden of asthma in the community and effectiveness of current strategy”(7)	
Costs of asthma	“Understanding of the break up of the burden very useful in directing strategies and efforts”(3)  “For prioritising”(5)  “It captures the over all cost to the community. The breakdown of different costs can identify the strategies to guide policy”(5)	“This is useful in assessing the burden, but only in comparison to the cost of other diseases”(6)   “A large public health problem, services and support available but not working”(7)  “Strong political value in getting action”(8)  “Economic impact and to judge value of new initiatives”(9)  “Important policy indicator”(9)  “Cost is an important factor in establishing policy priorities”(9)	
Deaths
(all ages)	“Overall mortality is a measure of how well we're doing in a policy sense in addressing problems such as asthma”(1)  “Fundamental epidemiological information. Death is the ultimate and unequivocal outcome and thus an excellent indication of whether outcomes are improving or worsening”(2)  “Sentinel event/outcome”(2)  “An important endpoint”(2) 
“Important final outcome”(3)  “Demonstrating impact”(3)  “Crucial re long assessment re policy”(3)  “Can assess age subgroups.”(3)  “Important trend information and covers 5 - 34 age group as well”(4)  “This statistic is useful in helping determine effectiveness of asthma control, in which age group this appears to be problematic and whether investigation is warranted”(5)	“This is valuable and extreme and reinforces the result if asthma is not well controlled. It has also been shown to fall in response to various strategies”(6)	
ED attendances	“May indicate either a) adequacy of control or b) adherence to action plan”(3)  “A major burden for the patient (waiting +++)”(3)  “Critical marker for burden of the disease”(4)
“Major health care usage"(4)  “An indicator of asthma that is out of control and needs help, also high cost for the community”(4)   “May reflect asthma severity or asthma treatment behaviour”(5)	“A measure of asthma control at community level (ecological)”(6)  “Information about the use of health care resources is an important measure of how well we're doing at addressing asthma in the community, and in assessing the overall cost of the condition”(6)  “Measure of loss of control”(8)
	
Asthma action plans	“It captures the practice that can be evaluated in relation to clinical guidelines. Could prioritise the strategies”(3)  “This is a self management strategy that is evidence based”(3)  “Prevention & control”(4)  “Important management strategy”(5)  “The question of adherence is important. can we test the number with and whether they follow”(5)  
“Difficult. But we need a national agreed measure re process of care. this is already suggested with AHIW and Qual and Safety Commission”(5)	No comments	
Indicator	                                           Ranking 1– 5	                                          Ranking 6 –10	
Ever asthma	“Important as asthma re-emerges in adult life in some asthmatics”(1)  “Needed for population prevalence”(1)  “Useful to have for statistical purposes to determine burden of disease in the community and whether it is changing”(1)  “Inter country comparisons; trends over time”(1)  “History of ever having had asthma”(1)  “Cumulative prevalence. Subject to recall bias. Used repeatedly so valuable for assessing changing prevalence.” (2)  “Prevalence data is a crucial input to our decision-making regarding whether to intervene (in a policy sense)”(4)	“Of limited value but still important. It is rough guide to assist in assessing asthma control when compared with current asthma”(6)	
Asthma control	“One of the key ultimate outcomes we are seeking. Good control provides clear picture of whether strategies are working or not”(1)  “Has been shown to be a rigorous yet flexible measure in primary care and is responsive to change i.e. can measure if patient gets better”(1)  “More specific control factors show where control needs to be targeted”(4) 
“Can be obtained by questionnaire and can be used to estimate burden of asthma”(4)  “Answers are we doing well in helping people manage”(4)  “Indication of control and to a certain extent management, should be used across all health care sectors”(4)	“Indicator of asthma severity”(10)	
Quality of life	“Will allow an increased focus in patient/consumer perspective”(1)  “This is what is important to the patient”(2)  “It captures what is important from community perspective which should be considered when looking at whether burden is reduced at a patient level”(4)  “Overall health status”(5)	“Information on quality of life feeds into the policy assessment of whether, and how, to intervene with respect to conditions such as asthma”(10)	
Deaths
(age 5 to 34 years)	“Informs re trends in burden and effectiveness of treatment”(2)  “simple non-controversial indicator, no need for verification”(2)  “Deaths among young people for whom the asthma diagnosis is reasonably clear is an even clearer measure of how well we're doing in a policy sense in addressing asthma”(2)  “Ultimate arbiter”(5)	“Mortality statistics very helpful in determining whether current asthma strategies are effective in keeping asthma under control”(6)	
General practice encounters	“Helps denote size of problem”(2)  “Burden on HRU”(4)	“Previously incomplete data. Would be invaluable to assess more comprehensively”(6)  “Information about the use of health care resources is an important measure of how well we're doing at addressing asthma in the community, and in assessing the overall cost of the condition”(8)  “Need baseline re GP care...could be used for planning and eventually linking with chronic disease prevalence and service delivery”(8)	
Preventer use	“Very insightful together with level of asthma control, key part of optimal management”(2)  “Index of control strategy effectiveness”(3)  “Quality of intervention”(3)	“Indicates self care and prospect of preventing attacks”(6)  “Will correlate with asthma control”(7)  “Some measure of treatment levels. Indirect indicator of doctor's assessment of patient's asthma severity”(9)
	
Urgent asthma visits	“Important outcome measure (exacerbations) generally used for studies”(2)  “Asthma is an acute and chronic disease, and this is a good marker of whether asthma is well-controlled among those who have it.”(3)  “May indicate either a) adequacy of control or b> adherence to action plan”(5)  “Some measure of control along with ED visits”(5)
“Captures community burden”(5)	“Useful re management.... and current local care”(7)	
Hospital re-admissions	“Anything to do with Hospitals =Costly & very inconvenient for patient”(1)  “Measure of quality of hospital and GP care”(5)	“Gives indication of potential for improvement in delivery of care and proxy for control”(6)  “Again determines effectiveness of current asthma strategies and the need for further improvements or resources”(10)
	
Indicator	                                           Ranking 1– 5	                                          Ranking 6 –10	
Occupational asthma	No comments	“There is a real need to better understand and prevent”(6)  “Highly preventable cause of asthma-also helps galvanise public action”(7)  “Need some measure of the impact of work-place related asthma in total burden of asthma”(8)  “If significant, should be a strategic focus area in terms of preventing asthma. Burden in terms of absenteeism, productivity also largely felt in workplace”(9)
“Potentially preventable”(10)	
Hospital days	“Good indicator of the severity of asthma & adequacy of control”(2)  “Gives sense of burden of disease and proxy for severity”(5)	“Useful in determining impact of asthma hospitalisations on health services including average length of stay in hospital”(8)
	
Airway Hyperresponsiveness	“Aid to differential diagnosis”(3) “Confirms diagnosis of asthma and is useful in estimating current resources required and those in the future”(4)	“Gold standard of asthma management & diagnosis”(7)	
Re-attendances	“Clear indicator that strategies not working, excepting those who have severe asthma”(5)	“A large public health problem, services and support available but not working”(7)
“Information about the use of health care resources is an important measure of how well we're doing at addressing asthma in the community, and in assessing the overall cost of the condition”(7)	
Current wheeze	“Simple - but error prone”(2)  “Needed for population prevalence”(3)  “Assists in estimating the number of people who report that they have a wheeze but who not been diagnosed with asthma. This may occur in Indigenous and ethnic communities”(3)	No comments	
Individual hospitalisations	“Number for ongoing comparison”(2)  “Can give reliable ecological estimates for costs by severity”(3)	“Determines effectiveness of asthma control and severity of asthma”(9)	
Spirometry	“Difficulty in PC re differential diagnosis”(4)	“Important for quantisation and good clinical practice”(6)  “Gold standard of asthma management & diagnosis”(7) 	
Cycle of Care uptake	“Indicator of quality and quantity of care” (3)	“Proxy for best community practice”(10)	
Smoking	“Can do something about this”(3)  “Control of smoking is the one public health factor proven to reduce the frequency & severity of asthma”(4)	No comments	
Children residing with smokers	“No child with asthma should be exposed to tobacco smoke”(5)	No comments	

RESULTS FROM ROUND 1-Q3. You were asked “Please list up to five indicators from the previous list of 24 that you believe could be excluded from the asthma data monitoring system.”  
The answers from 27 respondents are shown in Figure 3 and Table 2
Figure 3. Potential Asthma Indicators to be excluded. 
For each indicator, the bar shows the percentage of respondents who thought that the indicator should be excluded. The indicators are listed in order of the percentage of respondents recommending exclusion, i.e. the indicators at the top of the figure were the ones that most respondents considered should be excluded. Next to the bar is the median ranking that these respondents gave for exclusion of each indicator, as well as your own ranking (1= most strongly recommended for exclusion; 3=least strongly recommended for exclusion). 

Table 2. Reasons for excluding Asthma Indicators. 
Indicators are listed from the lowest to the highest median. At the end of each comment, the ranking is given in brackets.
Indicator	Ranking	
Airway hyperresponsi-veness	“This information is not available uniformly across all the populations and it is not easy to implement conducting BHR on all asthmatics at a population level”(1)  “Probably not feasible”(1)  “Difficult to measure and thus difficult to be representative of the population”(1)  “Rarely tested”(1)  “Does not inform significantly any of the 3 and rarely done overall”(1)  “Not convinced this is a policy relevant indicator”(1)  “Not useful for most of care providers and patients”(2)	
Asthma Cycle of Care uptake	“Not a great indicator of activity”(1)  “Don’t know about evidence for its use”(1)  “This appears to be complex and based around both physician and patient preferences and so not useful for either”(1)  “Not useful to measure”(1)  “There is a severe GP workforce crisis throughout most of Australia & GPs do no feel that they have time to perform non core activity”(2)  “Not sure that this captures anything useful”(3)	
Current wheeze	“Confusion re definitions”(1)  “Difficulties in collecting valid data from the general population”(1)  “Self reported, unreliable”(2)  “Redefine the key indicators current wheeze may not be a true indicator”(3)	
Hospital days	“Not sure what it would indicate”(2)  “Not sure of how this actually helps with long tern planning maybe one off re costs”(3)  “Length of stay is of less importance”(3)	
Occupational asthma	“Too specific”(2)  “Important but not a general indicator”(3)  “Subgroup of asthma”(3)	
Ever asthma	“Self reported, unreliable”(1)  “Because asthma is often transient and recall is unreliable this information is of little use”(1)  “This has very low specificity in relation what you are trying to achieve i.e. the above dot points”(2)  “Want to know now and then use this as the benchmark”(2)	
General practice encounters	“I would think would be difficult to attribute to effective management of asthma”(2)  “Provides the largest volume of care and determines the level of burden of asthma in the community. A decrease in GP visits may indicate improvements in asthma management in the community”(3)	
Smoking	“What has any one ever done with this information?”(1)  ”A separate issue”(2)  “Again of lesser importance in policy making given that all understand its importance”(3)	
Re-attendances	”The 28 day cut-off is arbitrary - a better marker would be number of attendances/year for individuals - can be done through data linkage” (2)
“May also determine whether investigation is required and level of resources to be made available”(2)	
Spirometry	“Not necessarily relevant”(1)  “How is it useful?”(2)  “Probably not feasible”(3)	
Deaths (age 5 to 34 years)	“Already included deaths”(2)  “Extremely small numbers result in the need to aggregate data from several years to get figures which can be released. not sensitive to measuring change in the population except on a very long term scale”(2)	
Asthma action plans	“Systematic Review Evidence of NO value in children”(1)  “I routinely provide written instructions & info on Asthma management but many of these patients cannot recall that they were given them!!! My experience is not uncommon”(3)  “Too difficult to verify and nothing has had much of an impact on uptake”(3)	
Asthma control	“Not convinced this is policy relevant”(2)  “Difficult to measure at times?”(2)	
Deaths (all ages)	“Contaminated data”(3)	
Quality of life	“Self reported, very subjective. A more objective measure is needed eg. Asthma Score, Juniper scale”(3)	
Children residing with smokers	“Only one of the important triggers”(2)	
Individual hospitalisations	“At policy level, not convinced need to know individual rates”(3)	
Hospital re-admissions	No comments	
ED attendances	“Determines level of asthma control in the community who is likely to use this facility and the severity of asthma exacerbations” (1)	
Preventer use	No comments	
Urgent asthma visits	No comments	
Costs of asthma	No comments	
Current asthma	No comments	
Hospitalisations	No comments	

RESULTS FROM ROUND 1-Q4. You were asked “If you have any further suggestions for indicators that you believe could be important in monitoring the status of asthma in Australia, please list below (optional)”
Please find below the comments some panellists made in regards to other potential additional asthma indicators. 

“Difficult, probably impossible to measure, but it would be nice to present the number of people exposed to various levels of air pollutants known to be associated with acute asthma exacerbations and/or development of asthma”
“In considering the various primary, secondary and tertiary prevention intervention options which would in 2008 -2012 make a real difference to burden, effectiveness and management change, it is hard to know whether self management or even health knowledge/literacy indicators would be useful. I presume that for the foreseeable future it is what the person at risk or with the disorder does that will make a difference rather than what the provider does” 
“Perceived control of asthma by the patient? Asthma knowledge?”
 “One of the already available indicators that could be collated may be the ambulance management and transport time policy and to see whether there is consistency nationally i.e. what % of the states meet certain key criteria such as asthma being priority 1 rating and a transport time of say < 10mins t hospital for severe asthma” 
“Ethnicity and Aboriginal sub classification”
“Prevalence of atopy”
“It is crucial that ACAM define in the next period what are useful indicators for quality and safety measurement at a State and regional level. It is important that AHIW and Qual and Safety Commission also agree, they should also be able to be used across all sectors including public and private”
“Frequency of use of relievers”

Appendix 1. Asthma Indicators: Name and description

	ASTHMA INDICATORS	
1	Ever asthma: Reporting ever having doctor-diagnosed asthma	
2	Current asthma: Reporting doctor-diagnosed asthma plus symptoms of or treatment for asthma in the last 12 months	
3	Current wheeze: Reporting wheeze in the preceding 12 months	
4	Airway hyperresponsiveness: Proportion of the population who are diagnosed with airway hyperressponsiveness	
5	Deaths (all ages): Deaths due to asthma in the population	
6	Deaths (age 5 to 34 years): Deaths due to asthma among people aged 5 to 34 years	
7	Hospitalisations: Episodes of hospitalisation for asthma	
8	Hospital days: Patient days (or “bed days”) in hospital for asthma 	
9	Individual hospitalisations: People hospitalised for asthma 	
10	Hospital re-admissions: re-admissions to hospital for asthma within 28 days of a previous admission for asthma 	
11	Emergency department attendances: attendances at emergency departments in the population for asthma each year	
12	Re-attendances: Re-attendance at either hospital or an emergency department for asthma within 28 days of a previous attendance for asthma 	
13	General practice encounters: General Practice encounters for asthma	
14	Urgent asthma visits: Total healthcare visits (hospital, emergency department and general practice) for asthma exacerbations or worsening asthma	
15	Cycle of Care uptake: Asthma Cycle of Care (formerly Asthma 3+ Visit Plan) Practitioner Incentive Program payments	
16	Asthma action plans: People with asthma who have a written asthma action plan	
17	Preventer use: People with asthma who use preventers (inhaled corticosteroids, leukotriene receptor antagonists or similar drugs) regularly	
18	Quality of life: People with asthma who report poor health-related quality of life 	
19	Smoking: Current smoking among people with asthma	
20	Children residing with smokers: Smoking in households where children with asthma reside	
21	Spirometry: People with asthma who have had spirometry within the last 12 months	
22	Asthma control: A composite indicator developed from measures of symptoms and medication use to impute the proportion of people with asthma who have poor clinical control	
23	Occupational asthma: Asthma caused by occupational exposure	
24	Costs of asthma: An index derived from expenditure and burden of disease data to examine the costs of asthma to individuals	

Appendix 2. Log-in instructions ACAM Delphi Second Survey
Log-in instructions
Link to Delphi survey
Click here or paste the following link into your browser:

http://129.78.134.50/delphi/applications/woolcock/delphi_2/index.htm

Two steps for accessing the survey
1.	After following the link, enter the secure site using:
Username: delphi
Password: survey

Please note, username and password are case sensitive 

2.	Log into the survey, using your registered username and password:
Username: joebloggs@hotmail.com
Password: health
These same details will also be used in subsequent rounds of the survey

Abbreviations
ACAM		Australian Centre for Asthma Monitoring
AIHW		Australian Institute of Health and Welfare
ASMA		Australian System for Monitoring Asthma
CRC		Cooperative Research Centres
ED		Emergency Department
GP		General Practitioner
SEIFA		Socio-Economic Indexes for Areas

References
ACAM (Australian Centre for Asthma Monitoring) 2007. Australian asthma indicators: Five-year review of asthma monitoring in Australia. Cat. no. ACM 12. Canberra: AIHW.
AIHW (Australian Institute of Health and Welfare) 2000. National health priority area indicators for monitoring asthma. Report of a consultation workshop. Canberra: AIHW.
Baker DF, Marks GB, Poulos LM & Williamson M 2004. Review of proposed National Health Priority Area asthma indicators and data sources. AIHW cat. no. ACM 2. Available at <www.asthmamonitoring.org>. Canberra: Australian Institute of Health and Welfare.
Canadian Institute for Health Information (CIHI) 1999. National consensus conference on population health indicators. Final report. Ottawa: Canadian Institute for Health Information.
CIHI 2005. The Health Indicators Project: the next 5 years. Report from the second consensus conference on population health indicators. Ottawa: Canadian Institute for Health Information.
Hader M & Hader S 1995. cited in the European Commission for Learn Guide: Delphi Studies. At <http://forlearn.jrc.ec.europa.eu/guide/2_scoping/meth_delphi.htm> viewed April 2008.
Hasson F, Keeny S & McKenna H 2000. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing 32:1008–15.
IOM (Institute of Medicine) 1999. Leading health indicators for healthy people 2010: second interim report. Washington, DC: National Academy Press.
Keeny S, Hasson F & McKenna H 2001. A critical review of the Delphi technique as a methodology for nursing. International Journal of Nursing Studies 38:195–200.
Keeny S, Hasson F & McKenna H 2006. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of Advanced Nursing 53:204–12.
NHS Health Scotland: Parkinson J 2007. Establishing a core set of national, sustainable mental health indicators for adults in Scotland: rationale paper. Updated October 2007. Glasgow: NHS Health Scotland, Scottish Government.
Parkinson J 2006. Establishing national mental health and well-being indicators for Scotland. Journal of Public Mental Health 5(1):42–8.
Powell C 2003. The Delphi technique: myths and realities. Journal of Advanced Nursing 41:376–82.

List of tables and figures
Table 1: 	Current national asthma indicators
Table 2: 	Correlation matrix among the six asthma indicators

Figure 1: 	Rating of asthma indicators in terms of their value in providing information to policy 
makers about the status of asthma in Australia
Figure 2: 	Asthma indicators recommended for retention and exclusion
Figure 3: 	Rating of additional asthma indicators proposed by panellists
Figure 4: 	Correlation between hospital separations and hospital patient days by age group