Methodology
People facing housing insecurity, that is, those experiencing homelessness or at risk of homelessness, can access support from Specialist Homelessness Services (SHS) across Australia. The Specialist Homelessness Services Collection comprises data that are collected as part of agency’s interactions with clients or potential clients. As such, the data are 'found', rather than 'made', meaning that the data were not specifically designed or collected for detailed research into the life experiences of people facing housing insecurity, instead they are a by-product of service provision. Such data sets are commonly known as administrative datasets. As is the case with many administrative datasets, this means that the SHS data have limitations and may not be suitable for testing specific hypotheses (Connelly et al. 2016).
Furthermore, although the SHS collection is relatively rich in the depth and variety of variables collected about clients and their needs, the structure of the data is not intrinsically suited to longitudinal analyses. Significant reworking is required to transform administrative data from a series of point-in-time snapshots (for example, at the time a client starts a support period and monthly time points thereafter) to a longitudinal product capable of providing insights into the 'prolonged situation of life' (Pattaro et al. 2020) that is associated with social disadvantage.
The need for significant reworking is typical of administrative data sets – the benefits and limitations of which have been reviewed in numerous studies (Productivity Commission 2013, Hurren et al. 2017, Spallek et al. 2020), though each administrative data set is nuanced enough warrant its own evaluation in the context of longitudinal work. This section therefore describes the SHSC data in terms of its potential for longitudinal analyses – primarily in terms of scope and structure.
Each longitudinal study undertaken with the SHS data will have its own derivation method and, consequently, the potential and limitations of the data will vary for each study cohort. The broad principles described in this section will, however, generally apply to all analyses using the SHS longitudinal data set.
A key feature of administrative data like the SHS collection is that the fundamental unit of collection is not individuals but is instead the services provided to individuals. This means that an individual only features in the data when they receive (or, in the case of the SHS, requests) a service. This is perhaps the biggest limitation of administrative data sets, including the SHS collection, as it limits the coverage of the data and introduces significant potential bias. It also is a key determinant of how the longitudinal data are structured for analysis.
Coverage
The SHS longitudinal data comprises information about clients that received services. This includes people that are already homeless as well as those at risk of becoming homeless. As such, the SHS longitudinal data cannot provide insights into the incidence or prevalence of homelessness in Australia; the best estimate for that remains the Australian Census, which in 2016 found that there were over 116,000 homeless people (including over 51,000 people in severely overcrowded dwellings) on census night, which featured nearly 19,000 people aged 55 and over (ABS, 2016).
At any given time, there is a group of people in Australia that are homeless or at risk of becoming homeless. The size of this group is unknown, but a subset seek support from SHS agencies, and a further subset receive services and are subsequently included in the SHS longitudinal population (Figure Methods.1).
Some of those people experiencing homelessness, or at risk, will approach SHS agencies and request services. The number of potential clients that do not approach SHS agencies is unknown. Of those that do approach agencies, some do not receive any service and are not part of the SHS longitudinal dataset; a limited amount of information is available on these interactions (see Unassisted requests). Those people who receive any service will have administrative data collected and form the basis of the SHS longitudinal data.
The absence of these otherwise-eligible clients means that, if the intent is to study the phenomenon of homelessness, the SHS collection only covers a subset of the population of interest. It also means that there is potential for significant bias in the characteristics of the clients captured in the SHS longitudinal data.
Bias
At each step in Figure 1, there is significant potential for bias to affect which clients are represented in the SHS longitudinal data.
In the first step, only a subset of people who are homeless or at risk of homelessness approach services; most people who experience homelessness do not approach SHS agencies. Insights into this bias are available from the General Social Survey, which in 2014 sampled nearly 13,000 dwellings in Australia (ABS, 2015). Of the subset of people that experienced homelessness in the previous 10 years, only one third sought assistance from a service organisation for their most recent episode of homelessness, most likely because they stayed with relatives or friends instead (ABS, 2015). The tendency to seek support from agencies varies with age, ranging from little over one-quarter for those 15 to 34 years old age to over 40% for those 35 to 64 and over 43% for those aged 65 and over (ABS, 2015).
It is not only social capital or alternative sources of support that precludes people from approaching SHS for assistance. Some will be prevented from approaching agencies because of various obstacles, foremost of which is a lack of local services – especially in regional and rural areas (ACOSS, 2019). Cultural barriers may also be relevant, as is a tendency for some client groups (particularly older clients) to not approach services because of a sense of shame or because they do not know about the availability of services (Thredgold et al. 2019). Additionally, states and territories have different service models and may fund the same services through non-SHS funded agencies, in which case a client that only receives that service through non-SHS funded agencies will not appear in the SHS data. Finally, there are a number of services provided by non-government agencies that are not funded by governments; these philanthropic supports are not captured in the SHS data.
Figure Methods.1: Coverage of the SHS longitudinal data, relative to potential coverage
Notes
- Sizes of boxes depicting subpopulations do not reflect the actual size of each subpopulation.
- While the number of unmet requests for services is known, the number of people who are unassisted is unknown because the SLK validity rates for unmet requests are low (52% in 2020–21).
Visibility
Longitudinal capabilities are a particular strength of administrative people-centred data, particularly since there is comparatively little loss to follow up compared with surveys. However, a client is only visible in the data for the time during which they receive services. Insight into client characteristics may therefore be sporadic in an incomplete picture of their life journey.
For example, long-term outcomes for a client cannot be measured reliably for SHS clients because the reasons clients no longer access services is not absolute. For example, a client may no longer need services, the client may be getting services elsewhere, or the client may request but not receive services from SHS agencies.
The structure of the SHSC also means that there is no visibility of client characteristics between support periods. For example, there is no way of knowing if a client that presents needing financial assistance is also struggling financially between support periods or if they only request for services when they have financial difficulties.
This is ameliorated, to some extent, in the SHS collection because agencies collect some client characteristics based on recall. For example, agencies ask clients if they have been in a variety of facilities in the previous 12 months, whether they have ever been diagnosed with a mental issue and if they have received services or assistance for those issues or whether the client has been homeless in the previous month or previous 12 months.
Sporadic visibility is a fundamental concern when using the SHS data for longitudinal analyses and is a key determinant of how the SHS data are structured for longitudinal analyses.
SHS agencies collect a minimum amount of data as described by the Specialist Homelessness Services National Minimum Data Set, depicted in Figure Methods.2. It demonstrates the sporadic nature of the client visibility in the dataset. Each time a client begins a period of support (i.e. when a client first receives a direct service from a SHS agency) the agency will undertake an assessment of the client’s needs and collect various client characteristics at the time of presentation and, for some variables, at various times before presentation (see the SHS Collection Manual). For each subsequent month, the agency will collect the variables on the ‘ongoing client form’. The last month in which the client received services will include information from the ongoing client form that could not be completed while services were ongoing (for example, the reason why support ended; see the SHS Collection Manual).
The data from the initial client form, the ongoing monthly client forms, and the final ongoing client form are aggregated to create support-period level data for each client. The support-period data form the foundation of the SHS longitudinal data.
Each client can have multiple support periods created through interactions with one or more agencies; these support periods can overlap and, in some cases, multiple support periods can start at the same time with different agencies. With the exception of accommodation services, clients can receive the same or different services from agencies at the same time; accommodation can only be received from one agency at a time. The variables listed in Figure Methods.2, which are collected for each support period, can therefore be inconsistent for a client over a given time interval, depending on how many support periods they have.
This is important for assessing which variables can be aggregated across study periods in the SHS longitudinal data and, subsequently, how these should be interpreted.
Figure Methods.2: Variables collected in the SHSC, by timing relative to a support period
The basic premise of the longitudinal analysis of SHSC data is that, within the constraints of the support-period data structure, there are meaningful variables that can be aggregated (summarised) for a client in their past (retrospective), present (defining) and future (prospective) periods. Furthermore, that these clients can be meaningfully studied as diverse cohorts that are defined based on client characteristics available in the SHSC data.
The size of the defining study period is the same for each client. This ensures that aggregations (for example, whether the client was ever homeless, or whether the client needed accommodation) can be compared across clients because they are measured over the same length of time.
This means that the defining study period, which is defined by the client meeting a particular condition in a particular time interval, will have different start and end dates for each client, based on when they first met the condition. For example, for older clients the condition might be that the defining study period commences from the start of their first support periods in which they are aged 55 or over within a given time period (such as 2015–16).
The shifting start and end dates for the defining study period means that the retrospective and prospective study periods will also shift, as shown in Figure Methods.3.
The figure demonstrates that to allow for a consistent sampling time for each client their start and end periods need to be allowed to shift based on the timing of the start of their cohort conditions being true; variables from any support periods that start within these intervals will be aggregated (summarised) into those intervals.
Figure Methods.3: Representation of offset study periods for two example clients
When appropriate values are aggregated for each study period, each client can be characterised by aspects of their past SHS experiences (retrospective period), their SHS experiences during the defining study period – which is typically central to the definition of that cohort – and their SHS experiences after that time (prospective study period). For example, a cohort of women experiencing family and domestic violence might analyse:
- their conditions during the defining study period, such as living and residential arrangements, presenting unit circumstances, or employment conditions, as well as their service need and provision
- their conditions before that time – some will have had a history of SHS usage, others will not. This in itself is an interesting variable for study, as is the services needed and provided
- their conditions after the defining study period, which can also be characterised by whether or not they return to SHS and, if they do, what type of services they required or received.
The longitudinal analyses characterise client’s pathways and outcomes by aggregating some client characteristics or experiences within each study period.
Some client characteristics are fixed and do not vary over time. The client’s state or territory (which can vary but is kept fixed in the longitudinal analyses), sex and age are all recorded as at the start of the first support period in the defining period. Variables such as Indigenous status and whether a client was born overseas are fixed for clients across all support periods. In both cases, if the variable is “true” in any support period (that is, Indigenous versus non-Indigenous or not-stated, overseas-born versus Australian-born or not-stated), then the client is marked as such in all their support periods.
Other characteristics are aggregated within study periods. For example, within each of the three study periods (retrospective, defining, prospective) a client can have any number of support periods. Within each support period, client circumstances are recorded at three times – the week before support commenced, at the start of support (“at presentation”) and when support ended.
Variables are aggregated in a binary manner by examining whether a particular event or situation occurred at any of the three times for any support periods that occur within each study period.
The longitudinal study variables are aggregations within study periods, and therefore differ significantly from similar variables in annual SHS reporting. For example, a homeless client in the SHS annual report will have been homeless when they started their first support period in the reporting year. In the longitudinal data, however, the homeless variable is a binary marker that records whether the client experienced homelessness at any of the three time points in any support period in each study period.
The longitudinal variables essentially answer the question, “Was the client ever known to be [homeless/not employed/a couch surfer/and so on]?”. For example, if a client was homeless in any time period (week before, at presentation, at the end) of any support period then the longitudinal variable homeless will be true. Conversely, if they did not report being homeless at any time they will not be recorded as homeless; this applies even if homeless status is unknown in some or in all time periods. This is a significant difference to the approach used in the annual reporting of the SHS data, for which clients with unknown status are treated differently (for example, excluded from proportions).
Regardless of when a support period ends, the client characteristics in each support period are aggregated into the study period in which that support period commenced. In a small number of cases a support period may cross study periods. For example, if a client started a support period close to the end of their retrospective study period, this support period might end during the defining study period. In such a case, the client characteristics would be aggregated into the retrospective period.
Detailed information on how data items in the SHSC are derived can be found in the Technical information page within the Specialist homelessness services annual report.
Client vulnerabilities
For research and analysis purposes, SHS clients were assessed for the presence of vulnerabilities including mental health issues, drug and/or alcohol problems, and experience of family and domestic violence (FDV) issues, all of which are known to be risk factors in housing instability. The derivation of these vulnerabilities is outlined below.
Clients with a current mental health issue
For each support period, a client is considered to have a current mental health issue if they were 10 years or older at the start of the support period and the client had any of the following:
- reported ‘mental health issues’ as a reason for seeking assistance at the beginning of the support period
- was formally referred to the specialist homelessness agency by a mental health service
- reported at the beginning of the support period that they had been in a psychiatric hospital or unit in the last 12 months
- reported at the beginning of the support period that they were receiving services or assistance for their mental health issues or had in the last 12 months
- had psychiatric hospital or unit as their dwelling type either a week before presenting to an agency or when presenting to an agency
- reported psychiatric hospital or unit as their dwelling type during that support period
- required psychological services, psychiatric services, or mental health services during that support period.
Clients with problematic drug and/or alcohol use
For each support period, a client is considered to have problematic drug and/or alcohol use if they were 10 years or older at the start of the support period and the client had any of the following:
- reported 'problematic drug or substance use' or 'problematic alcohol use' as a reason for seeking assistance at the beginning of the support period
- was formally referred to the specialist homelessness agency from a drug and alcohol service
- reported at the beginning of the support period that they had been in a rehabilitation facility in the last 12 months
- had rehabilitation facility as their dwelling type either a week before presenting to an agency or when presenting to an agency
- reported rehabilitation facility as their dwelling type during that support period
- required drug or alcohol counselling services during that support period.
Clients who have experienced family and domestic violence (FDV)
Before 2019–20, a client was considered to have experienced family and domestic violence if 'domestic and family violence' was reported as a reason for seeking assistance (only reported at the beginning of the support period) or they required family/domestic violence services during that support period.
From 2019–20 onwards, a client is considered to have experienced family and domestic violence (which includes both victim-survivors and perpetrators) if:
- domestic and family violence' was reported as a reason for seeking assistance (only reported at the beginning of the support period)
- they were formally referred from a non-SHS family and domestic violence service provider (only reported at the beginning of the support period)
- they required family/domestic violence services during that support period.
Before 2019–20, the SHS data did not distinguish between victim-survivors and perpetrators among FDV clients. Therefore, a small proportion of the clients included in FDV study cohorts will be perpetrators, some of which may also be victim-survivors.
Young people presenting alone
Young people are defined as clients aged 15–24 who presented alone at the start of the support period.
Clients leaving care
Clients are counted as transitioning from care arrangements if:
- the dwelling type was: hospital (excluding psychiatric), psychiatric hospital or unit, disability support, rehabilitation or aged care facility
- their reason for seeking assistance was transition from foster care/child safety residential placements or transition from other care arrangements.
Clients who were exiting custodial arrangements
Clients are counted as leaving a custodial setting if:
- their dwelling type was: adult correctional facility, youth or juvenile justice detention centre or immigration detention centre or
- their reason for seeking assistance was: transition from custodial arrangements or
- their source of formal referral to the agency was: youth or juvenile justice detention centre, or adult correctional facility.
Some of these clients were still in custody at the time they began receiving support.
Older clients
Older clients are defined as those who were aged 55 and over at the start of any of their support periods.
Children on care and protection orders
Children are counted within this cohort if:
They were under 18 and were on a care and protection order (CPO) either the week before presenting for a service or on presentation and had the following care arrangements:
- residential care
- family group home
- relatives/kin/friends who are reimbursed
- foster care
- other home-based care (reimbursed)
- relatives/kin/friends who are not reimbursed
- independent living
- other living arrangements
- parents.
The cohort also includes clients aged under 18 who reported ‘transition from foster care and child safety residential placements’ as their reason for seeking support from SHS.
Indigenous clients
Clients are counted as Indigenous if they were recorded as identifying as Indigenous during any of their support periods, regardless of timing, and regardless of whether they were recorded as non-Indigenous in any or all other support periods.
Longitudinal variables
The longitudinal analyses characterise client’s pathways and outcomes by aggregating some client characteristics or experiences within each study period.
Some client characteristics are fixed and do not vary over time. The client’s state or territory (which can vary but is kept fixed in the longitudinal analyses), sex and age are all recorded as at the start of the first support period in the defining period. Variables such as Indigenous status and whether a client was born overseas are fixed for clients across all support periods. In both cases, if the variable is “true” in any support period (that is, Indigenous versus non-Indigenous or not-stated, overseas-born versus Australian-born or not-stated), then the client is marked as such in all their support periods.
Other characteristics are aggregated within study periods. For example, within each of the three study periods (retrospective, defining, prospective) a client can have any number of support periods. Within each support period, client circumstances are recorded at three times – the week before support commenced, at the start of support (“at presentation”) and when support ended.
Variables are aggregated in a binary manner by examining whether a particular event or situation occurred at any of the three times for any support periods that occur within each study period.
The longitudinal study variables are aggregations within study periods, and therefore differ significantly from similar variables in annual SHS reporting. For example, a homeless client in the SHS annual report will have been homeless when they started their first support period in the reporting year. In the longitudinal data, however, the homeless variable is a binary marker that records whether the client experienced homelessness at any of the three time points in any support period in each study period.
The longitudinal variables essentially answer the question, “Was the client ever known to be [homeless/not employed/a couch surfer/and so on]?”. For example, if a client was homeless in any time period (week before, at presentation, at the end) of any support period then the longitudinal variable homeless will be true. Conversely, if they did not report being homeless at any time they will not be recorded as homeless; this applies even if homeless status is unknown in some or in all time periods. This is a significant difference to the approach used in the annual reporting of the SHS data, for which clients with unknown status are treated differently (for example, excluded from proportions).
Regardless of when a support period ends, the client characteristics in each support period are aggregated into the study period in which that support period commenced. In a small number of cases a support period may cross study periods. For example, if a client started a support period close to the end of their retrospective study period, this support period might end during the defining study period. In such a case, the client characteristics would be aggregated into the retrospective period.
Homeless
This variable refers to whether the client ever reported being homeless during the study period and is examined at all 3 time points of each support period in each study period (week before, at presentation, at the end).
Couch surfer
This variable refers to whether the client ever reported their housing situation as 'couch surfer' (specifically, living in a house, townhouse or flat but without tenure). It is examined at all 3 time points of each support period (week before, at presentation, at the end).
Presented for support alone
Presenting unit refers to the composition of the group when presenting to SHS agencies for support. This variable refers to whether a client was ever recorded as having presented for support (that is, started a support period) alone. Unlike many other variables, this is only recorded in the SHS data at the start of support periods. Counts of clients with values of No include cases where the variable is not stated or unknown.
Children presenting alone:
Children may be reported as presenting alone to an SHS agency for several reasons:
- It is possible that a child physically presented with an adult to an agency, but only the child required and received SHSC services. In this case, the child is reported as 'presenting alone' as the accompanying adult does not have an SHSC support period that can be linked to the child client.
- Alternatively, a child may have presented with an adult to an SHS agency and both received services, but the agency worker may not have properly linked the child to the accompanying parent/guardian when opening a support period for the child; hence the child is reported as presenting alone.
- Service was sought by and provided to the child only (without an accompanying adult) and therefore the child is the only client and is reported as presenting alone.
In addition, South Australia has a comparatively high number of children reported as presenting alone. This may be due to a difference in how presenting units are recorded in South Australia's client management system.
Presented for support with child(ren)
Presenting unit refers to the composition of the group when presenting to SHS agencies for support. This variable indicates whether the client presented for support (that is, started a support period) as part of a group which contained one or more children. Unlike many other variables, this is only recorded in the SHS data at the start of support periods. Counts of clients with values of No include cases where the variable is not stated or unknown.
Experienced FDV
This variable refers to whether the client had experienced FDV in any support period in the study period. Specifically, those clients where:
- domestic and family violence was reported as a reason for seeking assistance (only reported at the beginning of the support period)
- they required family/domestic violence services during that month.
Had mental health issues
This variable refers to whether the client identified as having had a mental health issue in any support period in the study period. Specifically:
- they indicated that at the beginning of a support period they were receiving services or assistance for their mental health issues or had in the previous 12 months
- their formal referral source to the specialist homelessness agency was a mental health service
- they reported mental health issues as a reason for seeking assistance
- their dwelling type was as a psychiatric hospital or unit either a week before presenting to an agency, or when presenting to an agency
- they had been in a psychiatric hospital or unit in the previous 12 months
- at some stage during their support period, a need was identified for psychological services, psychiatric services, or mental health services.
Had problematic drug or alcohol issues
This variable refers to whether the client had problematic drug or alcohol issues in any support period in the study period. Specifically, they:
- recorded their dwelling type as rehabilitation facility
- required drug or alcohol counselling
- were formally referred to the SHS from an alcohol and drug treatment service
- had been in a rehabilitation facility or institution during the past 12 months
- reported problematic drug, substance or alcohol use as a reason for seeking assistance or the main reason for seeking assistance.
- The identification of clients with problematic drug and/or alcohol use may be current or recent; referring to issues at presentation, just prior to receiving support or at least once in the 12 months prior to support.
Never employed in study period
This variable refers to whether the client did not have employment at any time in the study period. That is, the client was either unemployed or not in the labour force at all time points of all support periods in the study period (the 3 time points are, week before, at presentation, at the end of the support period).
It is intended to reflect the vulnerability of clients that have no independent means of financial support.
Not employed at least once in study period (Not Employed).
This variable refers to whether the client had periods of not being employed. It is a marker of whether the client was either unemployed or not in the labour force, in at least one time point during the study period. The client may have been employed at some time during the study period.
It is intended to reflect the vulnerability of clients that have no independent means of financial support. It is examined at all 3 time points of each support period (week before, at presentation, at the end).
Owned home
This variable refers to whether the client had tenure in a home as the owner at any time in the study period. Ownership can be outright (fully owner), mortgaged or shared equity or part of a rent/buy scheme.
Accommodated
This variable refers to whether the client was provided any form of accommodation in the study period and includes short term or emergency accommodation, medium term or transitional housing, or long-term housing.
Ended support period in public or community housing, started elsewhere
This variable refers to whether the client ever reported their housing situation as 'Public or community housing - renter or rent free' at the end of a support during the study period and they did not report 'Public or community housing - renter or rent free' at the start of the support period.
Started in public or community housing, ended elsewhere
This variable refers to whether the client ever reported their housing situation as 'Public or community housing - renter or rent free' at the start of a support during the study period and they did not report 'Public or community housing - renter or rent free' at the end of the support period.
For many cohorts, the longitudinal analyses also include a comparison cohort, that is, a set of clients that lack the distinctive feature(s) used to define the study cohort. For example, the FDV cohort may be defined as clients that experienced FDV anytime during the defining study period (for example, 2015–16). Importantly, data for the 12 months following the first support period in the defining study period is aggregated into the defining study period for that individual client (Figure Methods.4). The comparison cohort would then be defined as clients that did not experience FDV in the 12 months from their first support period in the defining study period.
These definitions mean that there are also clients who will fit neither category and are subsequently excluded from the analysis. For example, a client who did not experience FDV in 2015–16, but did experience FDV after 2015–16 but within their 12-month defining period.
This is demonstrated in Figure Methods.4 below.
Figure Methods.4: Selection of study and comparison cohorts
ABS (Australian Bureau of Statistics) (2015) General social survey: Summary results, Australia, 2014. ABS Cat. no. 4159.0. Canberra: ABS.
ABS (2016) Census of population and housing: Estimating homelessness, 2016. ABS Cat. no. 2049.0. Canberra: ABS.
ACOSS (Australian Council of Social Service) (2019) Demand for community services snapshot December 2019. Sydney: ACOSS.
AIHW (2019) Specialist homelessness services annual report 2018–19. AIHW Cat. no. HOU 318. Canberra: Australian Institute of Health and Welfare.
Connelly R, Playford CJ, Gayle V, Dibben C (2016) The role of administrative data in the big data revolution in social science research. Social Science Research, vol. 59, pp 1-12.
Hurren E, Stewart A, Dennison S (2017) New methods to address old challenges: The use of administrative data for longitudinal replication studies of child maltreatment. International Journal of Environmental Research and Public Health, vol. 14, issue 9, 12 pages.
Pattaro S, Bailey N & Dibben C (2020) Using linked longitudinal administrative data to identify social disadvantage. Social Indicators Research, vol. 147, pp. 865-895
Spallek M, Haynes, M, Baxter J & Kapelle N (2020) The value of administrative data for longitudinal social research: a case study investigating income support receipt and relationship separation in Australia. International Journal of Social Research Methodology, vol. 23, issue 5, pp 467-481.