Showing posts with label #CENSUSfail. Show all posts
Showing posts with label #CENSUSfail. Show all posts

Saturday 24 December 2016

Australian Bureau of Statistics under Kalisch continues to prove that Census 2016 was expensive as well as a statistical and public relations disaster



Information coming out of the once-proud Australian Bureau of Statistics (ABS) again proves that it approached the most radical change to the national census of population and housing with an almost complete lack of understanding of the mood of the populace1.

ABC News, 23 December 2016:

Taxpayers spent close to $200,000 to turn the Sydney Opera House green to promote the 2016 census, without any clear reference to the national survey.

The seven sails of the national landmark were lit up for two nights but did not include any information about the census, the website, a hashtag or branding.

Internal documents show it cost taxpayers $192,000 for setup, equipment hire, management and support.

The Australian Bureau of Statistics (ABS) chief statistician David Kalisch described it as a "major public relations opportunity" and said it was likely to attract "social media influencers".

"This will maximise awareness and engagement with the census, and help create a national conversation," Mr Kalisch wrote in the document.

The Opera House turned green for census night and the night before but the social media conversation was dominated by the website's failure.

There was a 40-hour outage caused by four Distributed Denial of Service (DDoS) attacks that had been the subject of a blame game between the Australian Bureau of Statistics (ABS) and contractors for months.

The Opera House was part of a national campaign to light up landmarks with the colour green.

The Melbourne Arts Centre, Canberra's Telstra Tower, Brisbane's City Hall and the Darwin Convention Centre were some of the 20 sites to "go green" for the census…..

The total census campaign media budget was $12 million.



Currently ABS alleges that the national census response rate exceeds 96 per cent - comprising over 4.9 million online forms and over 3.5 million paper forms representing 8.4 million households/dwellings.

A rather strange statement by the Bureau, given it previously stated in the lead up to the census that it expected to survey close to 10 million dwellings and afterwards that there were exactly 9.8 million dwellings within the survey pool.

The failure to genuinely meet response rate requirements being papered over by the many personal forms in addition to the household ones [Senate Economics References Committee, 24 November 2016, inquiry report, 2016 Census: issues of trust, p.80].  Presumably these personal forms were official census forms which stated the person was in transit (travellers, homeless, & hospital patients) – all est.1.2 million of them if the deliberately vague assertion of the Bureau is to be believed.

The next public confidence hurdle for the failed 9 August 2016 Census comes when preliminary population and dwelling counts are released in April 2017 – given that so many people are aware of friends or acquaintances who deliberately refused to supply name and/or address or filled in their census forms with inaccurate or misleading information in an attempt to avoid having their genuine personal information retained by the federal government indefinitely in a national database for as yet unstated or unexplained purposes.

NOTES:

1. Previous North Coast Voices posts on 2016 Census here.
    Bill McLennan, 2016, Privacy and the 2016 Census.

Sunday 27 November 2016

Australian #CensusFail 2016 reports by both the Senate and Cybersecurity Special Advisor have been released


Following #CensusFail 2016 Prime Minister Malcolm Turnbull ordered a review of events by Cybersecurity Special Advisor, Alastair MacGibbon.

In what the media is characterising as an excoriating report, a mirror was held up to the Australian Bureau of Statistics – sadly one that it is unlikely to avail itself of given its current leadership.

DPM&C, Office of the Cybersecurity Special Advisor, Review of the Events Surrounding the 2016 eCensus (October 2016):

Not just communications, but engagement…

In most respects, the ABS had a well formed and prepared communications strategy and awareness raising campaign; but it was focussed on the wrong things. The communications problem they needed to address was not a low level of awareness of the Census, but rather, the introduction of a ‘digital first’ approach and the associated barriers to participation – concerns over security and
privacy.

The ABS failed to adapt its media and communications in response to the public relations storm that built up in the weeks prior to the Census regarding privacy and security in both mainstream and social media. Instead, ABS rigidly stuck to its plans, forgoing crucial opportunities to influence and drive the conversation around the Census. Processes for approval of campaigns, and changes to them, may need to be changed to promote agility.

On Census night, the ABS severely underutilised social media as a communications tool to keep the public up to date and informed of the incident. The ABS’s lack of timely and transparent
communications lost it trust because it opened the door to speculation. The continued slow updates and virtual absence from the media meant that ABS struggled to win back the trust of the public in the following days. Ministers must also be supported with clear and accurate advice, and senior executives must be equipped to understand and talk about cyber security as a matter of business
risk……

Reacting to public sentiment

…..The ABS announced public consultation regarding privacy via a media release issued on its website, with a submission period of only four weeks – 11 November to 2 December 2015 (just before Christmas).

The ABS received only three public submissions. Not only should this low response rate have indicated to the ABS that its public engagement on the key issue of privacy was inadequate, it also left a huge vacuum with regard to capturing public concerns. So the ABS missed an opportunity to identify how to evolve its communication plans developed following qualitative research in 2014 to address more up to date concerns.

As a result, the ABS was ill-equipped to manage the impact changes in the Census would have on a small but important segment of the population and their willingness to complete the eCensus online.

In January 2016, seven months out from the Census, the first articles raising concerns about privacy and security of data appeared in the media. More substantial rumblings began in March, with two main themes emerging:

• That the Census was intrusive and no longer anonymous
• The Census was vulnerable to hackers.

The ABS prides itself on the constant measuring of public sentiment and awareness using traditional survey techniques (see Figure 4, page 53). The Review concludes that these surveys contributed to a false sense of security and failure – still at time of writing – to grasp the significance and power of social media groundswells.

Major shifts in public statements regarding the security of the Census began the week prior to Census night, culminating in Senator Nick Xenophon and several other parliamentarians issuing warnings about security and privacy concerns and apparent implementation problems leading to a ‘debacle.’

Prior to the closure of the eCensus form, over 11,000 individual mentions (social and mainstream media) were published voicing concerns about the privacy and security of the eCensus. The closure of the eCensus resulted in 17,730 privacy related mentions, far outweighing mentions (1,200 total) of the technical issues experienced – i.e. what happened (see Figure 4).
This coverage created overwhelming ‘noise’ making it difficult for the ABS to remain on message.

The ABS’s planned communications were being drowned out. But rather than trying to adapt its approach to limit the impact the reporting had on the public sentiment toward the Census, the ABS stuck to planned messaging ignoring the public relations storm brewing around them.

The failings of the ABS to address issues of concern in the media extend to its use of social media. Analysis conducted on ABS Twitter and Facebook accounts shows that at no point did the ABS significantly change its planned posting schedule or content as a result of critical media reporting (shown in Figure 5, page 54) and of considerable online chatter around privacy (Figure 4). The ABS did change its social media advertising as well as engage posters directly on social media. But this was not enough.

The ABS’s virtual absence from the privacy and security debate is reflected in its social media crisis escalation matrix – the process designed to monitor, escalate  and handle social conversations. The matrix had two main flaws:

1. The ABS’s ‘qualifiers’ (thresholds that had to be met to raise concern) were too high. A ‘red level scenario,’ the highest categorisation for negative conversation, was enacted only if someone had 10,000 plus followers or a post had over 30 engagements.
2. The ABS’s response/action for a ‘red scenario’ was to hold all social media communications.

The ABS’s social media strategy was too restrictive and didn’t allow enough flexibility to respond to changing trends in media and social media. As a result, the ABS missed crucial opportunities to inform the conversation around privacy and security and the benefits of the digital first approach.

When public discourse was rising on the issues, the ABS should have been on the front foot addressing these concerns. Key spokespeople should have been conducting interviews, issuing media releases and engaging on social media to drive the conversation and shape the debate.

While the ABS did eventually start engaging in the mainstream media, it was too little, too late. And on the whole the ABS steadfastly stuck to its communications plans, allowing the media, and subsequently the public, to take the lead role. The ABS failed to insert itself in the conversation and underutilised mainstream and social media as a vehicle to shape the debate around the benefits of a digital first approach.

Recommendations for the Australian Bureau of Statistics

• The ABS should engage an independent security consultant for a wide-ranging examination of all aspects of their information collection and storage relating to Census data – from web application through to infrastructure and policies and procedures.

• The ABS should ensure future significant changes to personal information handling practices are subject to an independently-conducted privacy impact assessment and are supported by broad ranging consultation.

•  The ABS should adopt a privacy management plan to enhance its capability to identify and manage new privacy issues.

• The ABS should assess and enhance existing ABS privacy training for staff.

• The ABS should develop a specific strategy to remove the current state of vendor lock-in.

• The ABS should strengthen its approach to outsourced ICT supplier performance management to ensure greater oversight and accountability.

• The ABS should draw upon the lessons it takes from the Census experience to help to guide and to advocate for the cultural change path it is following.

• The ABS’s decision in August to assemble an independent panel to provide assurance and transparency of Census quality is supported and the resulting report should be made public.

•The ABS should implement a targeted communication strategy to address public perceptions about Census data quality.

The ABS should report monthly to their Minister outlining progress against the above recommendations

Also following #CensusFail the Australian Senate Standing Committees on Economics conducted an inquiry into the preparation, administration and management of the 2016 Census by the Australian Bureau of Statistics.

Its final report 2016 Census: issues of trust made these recommendations:

Recommendation 1
4.81 The committee recommends that all future Privacy Impact Assessments
relating to the census, are conducted externally with the final report published on
the ABS website 12 months in advance of the census to which it relates.

4.82 Following the release of a PIA recommending changes to future censuses,
consultation across the Australian community should be undertaken by the ABS
with the outcomes clearly documented on the ABS website no less than six
months before a future census.

Recommendation 2
4.83 The committee recommends that the ABS update its internal guidelines to
make clear that consultation requires active engagement with the nongovernment
and private sector.

Recommendation 3
5.46 The committee recommends that the ABS publicly commit to reporting
any breach of census related data to the Office of the Australian Information
Commissioner within one week of becoming aware of the breach.

Recommendation 4
6.89 The committee recommends that the Australian Government commit the
necessary funding for the 2021 census in the 2017–18 Budget.

Recommendation 5
6.90 The committee recommends that the ABS conduct open tendering
processes for future census solutions requiring the participation of the private
sector.

Recommendation 6
6.91 The committee recommends that the ABS give greater attention to
intellectual property provisions in contracts that include licensing and royalty
arrangements.

Recommendation 7
6.92 The committee recommends that the 2021 eCensus application be subject
to an Information Security Registered Assessors Program Assessment.

Recommendation 8
6.93 The committee recommends that the ABS take a more proactive role in
validating the resilience of the eCensus application for the 2021 census.

Recommendation 9
6.94 The committee recommends that the Department of Finance review its
ICT Investment Approval Process to ensure that projects such as the
2016 Census are covered by the cabinet two-pass process.

Recommendation 10
6.95 The committee recommends that the Australian Government provide
portfolio stability for the ABS.

Recommendation 11
6.96 The committee recommends responsible ministers seek six-monthly
briefings on the progress of census preparations. These briefings should cover
issues including, but not limited to, cyber security, system redundancy,
procurement processes and the capacity of the ABS to manage risks associated
with the census.

Recommendation 12
6.106 The committee recommends that the ABS consider establishing a
dedicated telephone assistance line for people who require special assistance in
completing the census.

Recommendation 13
7.28 The committee recommends that the maximum value of fines and any
other penalties relating to the census be explicitly stated.

Recommendation 14
7.29 The committee recommends that the Australian Bureau of Statistics
develop a clear communications strategy outlining the outcomes for
non-compliance with the census, including resolution processes and the value of
possible penalties.

Recommendation 15
7.57 The committee recommends that the Australian Government provide
sufficient funding for the ABS to undertake its legislated functions to a continued
high standard.

Recommendation 16
7.58 The committee recommends that the responsible minister act as a matter
of urgency to assist the ABS in filling senior positions left vacant for greater than
6 months.

Tuesday 27 September 2016

#CENSUSfail: so there was this little survey....


On 21 September 2016 The Sydney Morning Herald reported:

Almost 95 per cent of households have completed a census form despite an embarrassing website outage on census night and lingering political controversy over the national headcount.

The Bureau of Statistics says it already has sufficient data for a "high quality" census, ahead of the deadline for forms on Friday.

The Australian Bureau of Statistics (ABS) also tries to pretend that refusal to complete the census form is the only civil disobedience it has to contend with when collating household responses.

However a little survey which was included in one submission to the Senate Economics References Committee Inquiry into the 2016 Census indicates that the ABS may have other problems with reliability of the data it can subtract from some Census questions.

The possibility that false information has become a significant factor in Census data sets is buttressed by previous findings in the Office of the Australian Information Commissioner (OAIC) Community Attitudes to Privacy Survey Research Report 2013.

Responses of the 1,000 participants in the OAIC combination fixed line/mobile 'phone privacy survey resulted these percentages:

More concerned about providing personal information electronically or online than were 5 years ago – 67%
Concerned about possibility of becoming victim of identity theft or fraud in the next year – 69%
Provided false personal details when completing online forms – 31%
Provided false name when completing online forms – 30%
Refused to deal with government agency/public sector organisation due privacy concerns - 23% 
At some time have refused to supply personal information – 90%.

Excerpt from that little submission to the current Senate inquiry:

Q4 If you did fill out the 2016 Census, did you include your real name and address?

This question was required and had 3 answer options from which respondents could choose only one: Yes, No and other.


A substantial majority (74%) of respondents either did not fill out the census form at all or responded in ways intended to frustrate efforts to share their information in ways that could identify them as individuals, match their census data to other data or track their census data from one census to the next. The commonly found terms below are not mutually exclusive of one another.
This high number of people who have chosen not to complete the census or refused specific questions/ removed identifying information is directly relevant to the Inquiry Terms of Reference seeking feedback on impacts on data quality for the 2016 census.
Click on the commonly used terms to explore responses or click here to display all responses alphabetically

54 comments provided under 'other'

9/1/2016 8:06 PM : address only.
8/30/2016 8:17 PM : Blank address- name The Householder
9/3/2016 2:05 PM : census not done. Not going to be done either
8/30/2016 11:44 PM : completed address- left names blank
9/2/2016 6:49 AM : created a new persona for street address
9/1/2016 5:19 PM : Current address was already on the form- provided suburb and postcode for other addresses.
9/1/2016 7:10 PM : False name
9/4/2016 11:04 AM : False name- address- and some personal details. Only what I consider relevant to stastical analysis was completed approximately accurately.
9/1/2016 9:25 PM : false name- correct address
9/1/2016 5:38 PM : Filled in address but gave name as UNDISCLOSED
9/1/2016 6:53 PM : First name only. Make the computer work that little bit harder.
8/30/2016 6:35 PM : Gave correct address- no name
8/30/2016 7:08 PM : Gave first name- not last
9/3/2016 5:04 PM : I cut out NAD and identifying bar codres before returning the form
9/3/2016 10:59 AM : I did- but am unhappy about having to do so under threat of a fine
9/2/2016 5:48 PM : I didn't put my real name
8/30/2016 8:17 PM : I do not intend to use my name or address
9/3/2016 7:07 PM : I included a false name for privacy and security reasons.
9/1/2016 8:42 PM : I included it in the palest of blue coloured pencil- so it could not be scanned but required dedicated effort AND inckuded cover letter saying it was complted under duress asnd in great anger at them compromising such an important process.
9/1/2016 6:25 PM : I intend to leave name and address blank
8/30/2016 10:03 PM : I intend to leave name- all addresses (both current and past)- and age blank- to frustrate creation of an SLK or any possible link to past census data
9/3/2016 2:04 PM : I left name blank but the address was already printed on the form
9/1/2016 5:49 PM : I made a statement of objection but I gave Postcode.
9/7/2016 9:09 AM : I made my address suburb only with previous addresses
8/30/2016 7:33 PM : I provided a false name but address and other details were true
8/30/2016 5:05 PM : I put in my postcode and suburb only.
9/2/2016 11:50 AM : I put not necessary for purpose of Census data
9/3/2016 1:54 AM : I redacted identifying information and competed the data section truthfully
9/1/2016 9:15 PM : I used a false name but real address. [This should be a category]
9/6/2016 9:55 PM : i will be using a false name
9/8/2016 5:08 PM : I will do it but am unhappy about it and have no wish to take it seriously again. I think the info will be used for any purpose the govt wants and fear lack of security.
9/1/2016 4:07 PM : I will fill in the Census form when asked to in writing by the Chief Statistician. When I fill it in- I will omit my name and address details.
9/2/2016 11:35 AM : I will leave blank space for my name on the paper form.
9/3/2016 1:12 PM : I will not fill in the form if this is required.
8/30/2016 8:02 PM : I would never provide my name. The ABS is only authorised to hold statistical dat!
9/3/2016 3:45 PM : I'll use false name if/when I do it
8/30/2016 7:08 PM : If I complete the paper form- I will be leaving off my name and address. Still debating whether to do this or boycott.
8/30/2016 7:49 PM : if instructed by chief ABS to complete form- I will not include my name and add
9/1/2016 6:04 PM : Just address as it was printed on the form won't get names though as I feel names make the census a data trawling tool.
9/3/2016 7:27 AM : Misrepresentation of the depth of data linkage-cross referencing and retention
8/30/2016 6:56 PM : No name. Correct address
9/1/2016 6:29 PM : Not in Aus
9/2/2016 8:11 AM : Put initials
8/30/2016 8:20 PM : Real address but a blank name
9/1/2016 5:18 PM : Real address- as it was printed on the paper- fake names.
8/30/2016 6:05 PM : Removed all tracking items on form
9/1/2016 5:30 PM : silent voter ...omitted name
9/1/2016 5:57 PM : Tossed up + in the end did a variation
9/7/2016 5:32 PM : Used a very runny ink from a fountain pen which might 'accidently' smear
9/1/2016 9:21 PM : used married name- which I don't use in real life
9/1/2016 6:21 PM : Yes I did- but reluctantly!!
8/30/2016 6:58 PM : yes- but i didn't really want to
8/31/2016 10:36 PM: Names and DOB blank
9/10/2016 2:44 PM : Stated suburb-postcode-age and gender.