Sunday 11 September 2016

#CensusFail 2016 continues to erode public trust



Finally it appears that the Australian Bureau of Statistics (ABS) is admitting to being in a crisis situation with regards to the 2016 national census.

A census for which it had anticipated requiring around 39,000 Australians to help us run the Census. [ABS, Forward Work Program 2015-16 to 2018-19, p19]

One month after Census Night on 9 August and the ABS is now advertising for additional Census Field Officers - presumably to chase up all those households who are reluctant to participate in a national survey which retains names and addresses to assist with future data matching for unspecified purposes or who are now worried about the integrity of data security supposedly safeguarding their personal information.

Successive funding cuts under the last two federal governments may go some way to explaining the workplace situation in Belconnen, as does the leadership role of Australian Statistician David Kalisch.

However it is interesting to note that serious problems in the ABS first made an appearance in the wake of the Abbott Government coming to power and intensified after Prime Minister Turnbull and the Dept. of Prime Minister and Cabinet became involved.

Given the nature of politicians and bureaucrats (and their inability to fully accept responsibility for the consequences of their own actions) I expect that many ordinary Australians will find ABS letters threatening legal action amongst their Christmas mail this year.

All in the possibly vain hope that more than 95.1 per cent of an est. 9,268,673 Australian households [ABS, (March 2015) Household and Family Projections, Australia, 2011 to 2036will have been successfully coerced into completing the census form, thereby keeping the statistical margin of error low enough to render analysis reliable across sub-populations/populations and save the 2016 census from being ignominiously binned.

Right now I'm betting that the ABS is somewhere between 464,000 and 500,000 completed census forms short of meeting that target.

BACKGROUND

Excerpt from Australian Bureau of Statistics letter to James Smith, 6 September 2016:

The ABS does not profile people who are fined, and as such a single document explaining this does not exist. Rather, detailed case files are kept for each person.

Crikey.com.au, 8 September 2016:

A "crisis" in the rate of returns of censuses has forced the Australian Bureau of Statistics to increase its level of harassment of Australians -- even of people who've already completed and lodged their forms.

Senior Australian Bureau of Statistics officials have warned staff of a “crisis” over the rate of return of censuses and demanded collectors increase the level of harassment of Australians, including those who can furnish evidence they have already lodged it.

Sources within the bureau have told of an extraordinary change in policy yesterday by ABS executives. On Monday, census field officers were told in writing that they did not need to return to a dwelling that had advised that the census has been completed. This included anyone who had produced a receipt number for their online census submission or who said they had done so, or who said they had posted the census form. However, citing a “crisis” in the rate of returns, ABS management yesterday told staff that they were now required to make a further visit. Householders who asked why were to be told that “it is still being investigated and [they are] still on their list requiring a visit”.

That’s despite multiple instances of people complaining on social media that collectors had continued to pursue them despite their having submitted, and even of collectors complaining that their data on who had submitted forms was out of date.

The internal admission of a “crisis” within the ABS about the census contradicts its public insistence that all is well with the census and that, if anything, return rates are ahead of where they were in previous censuses.

Under the Census and Statistics Act, census collectors have extraordinary powers to enter any building except immediate residential premises without a warrant. While there is no legislative power to physically enter residential premises, people have reported on social media collectors trying to open locked doors and, in at least one case, entering a rural residence while the resident was absent……..

Senator Scott Ludlam, on 1 September concerning a matter he will put before the Senate on 12 September 2016:

NOTICE OF MOTION 1 September 2016

I give notice that on the next day of sitting I shall move that -
The Senate -                                                     
1.       Notes
a.       the technical failures of the ABS website on census night - August 9th 2016 - prevented thousands of people completing the census;
b.      subsequent attempts to address the technical failures further added to confusion and impacted public confidence in the census process;
c.       the census website was offline for several days after census night, and sporadically offline in the following weeks;
d.      thousands of Australians were unable to complete the census, and thousands of Australians were reluctant to do so due to privacy concerns.
2.       Calls on the government to direct the Australian Bureau of Statistics to issue a statement declaring that no Australians will be fined for failing to complete the census.



Australian Bureau of Statisticsmedia release, 5 June 2014:

ABS announces planned changes to future work program

The acting Australian Statistician, Jonathan Palmer, today announced planned reductions to the Australian Bureau of Statistics (ABS) work program.

The ABS must reduce expenditure by about $50m over three years. While the ABS has been able to implement efficiencies in its operations, these are insufficient to meet the expenditure target. As a result, the statistical work program will be reduced from 2014-15. 

Mr Palmer said the revised work program, developed after consultation with key Australian Government agencies, will continue to meet Australia’s core statistical needs.
“Our highest priority was to maintain activities that are critical to effective government decision making and deliver the most public benefit.

“While the revised forward work program retains core statistical elements and outputs, we have had to discontinue or reduce outputs in areas that are valued by the users of those statistics. If funding is provided for the work we are ceasing, we will reinstate it.

“The quality, integrity and relevance of our statistics are critical to informing effective decision making and we must not lose sight of that as we plan for the future,” Mr Palmer said.

The work program changes, which will be implemented from 1 July 2014, are:

Discontinue
Environment collections from Australian Households
Waste Account
Measures of Australia's Progress
Australian Social Trends
Survey of Tourist Accommodation
ABS funded component of Culture, Sport and Recreation statistics 

Reduce
Industry statistics research, development and reporting in selected areas
Social conditions statistics research, development and reporting in selected areas
State and territories statistical services engagement and analysis activities
Regional statistics analysis and development
Macroeconomic research and development engagement in international activities
National information and referral services response times
External statistical education development programs

Review
Review the House Price Index, with the view to discontinuing it pending identification of alternative sources to meet the Australian National Accounts and other requirements

Further details of the work program changes will be advised to affected users in due course.
As Australia’s national statistical agency, the ABS provides official statistics on a wide range of topics relevant to government, business, and the Australian population.

Dept. of Prime Minister & Cabinet, PUBLIC SECTOR DATA MANAGEMENT IMPLEMENTATION REPORT, July 2016:

Overview
On 3 December 2015, the Department of the Prime Minister and Cabinet (PM&C) released the Public Sector Data Management Report.
Since then, significant progress has been made to implement the recommendations of the report. This Implementation Report outlines the outputs and initiatives that have been undertaken to date to meet the recommendations of the report and further the public data agenda.
The Australian Government remains committed to effectively managing, sharing and publishing public data as a national resource for the benefit of the Australian people.
As the lead for public data policy, PM&C continues to work closely with government agencies, states and territories, research organisations, industry and the public to maximise the use and reuse of public data across the whole economy……

Recommendation 9
Establish a linked Commonwealth dataset with a domain for secure access by government and researchers, providing:
* a collection of linked administrative and survey data that links across time and across sectors;
* data linkage conducted by accredited integration authorities (ABS); and
* streamlined access to aggregate and individual data, while maintaining appropriate protection of individuals’ records……

Multi-Agency Data Integration Project (MADIP)
The Multi-Agency Data Integration Project (MADIP) is a collaborative cross-portfolio partnership to improve accessibility to, and maximise use of, public data. Commonwealth data custodians are working together to establish processes for sharing and safe dissemination of linked data.
The MADIP partner agencies include the Department of Social Services, Department of Health, Department of Human Services, Australian Taxation Office and the Australian Bureau of Statistics (ABS).
A 2011 snapshot dataset has been created and is currently being analysed and evaluated by nine partner agencies in-posted to the ABS.

Trusted Access Model
One component of the MADIP is the Trusted Access Model for microdata access.
This model seeks to enable users outside the ABS, such as researchers, to use the data while maintaining the privacy of individuals represented in the data.
A ‘privacy-by-design’ access approach is used, which aims to build trust in the use of public sector data and balance the need for information with the need to maintain the privacy of individuals.

Data integration and linkage projects
The ABS is working on a number of data linkage projects to produce valuable linked datasets.
These projects include the Expanded Analytical Business Longitudinal Database and the Linked Employer-Employee Database.
These enduring, linked datasets combine high value data sources to better support policy development through research and informed discussion.

The ABS has commenced trials of a virtual DataLab with a number of agencies to provide trusted access to microdata. The DataLab is a platform from which to fully implement trusted access in 2016/17……

Little Bird Network, 14 August 2016:

This powerpoint presentation is published by the Public Data Branch of the Department of Prime Minister & Cabinet under the CC by 4.0 licence.

This presentation outlines in detail the plans of the Department of Prime Minister and Cabinet to use de-anonymised Census data to integrate with administrative datasets, including that it has already been doing this for some years with a sample of Census data (Appendix 5). This plan was published prior to the ABS public ‘consultation’ (carried out between November and late December 2015) required to determine whether or not Australians agree with de-anonymisation of the Census for the purpose of these data linkage projects which suggests that the decision had already been made by the Department of Prime Minister & Cabinet without public consultation to authorise it.

The public consultation that was carried out by the ABS over 5 weeks in late December was not advertised or reported in any media. The presentation also indicates that while the ABS is the integrating authority (the agency with the legislative authority to create data linkage projects), DPM&C appears to be the lead agency. Commentary regarding who is responsible for recent and significant changes to the Australian Census should be considered in light of the information in this presentation.

Power point presentation here.


Computer World, 8 September 2016:

The government is preparing for a major overhaul of its open data portal, Data.gov.au.

The current Data.gov.au site is based on the open source CKAN data portal software. 
The site launched in 2013.

The Data Infrastructure and Government Engagement (DIGE) team within the Department of the Prime Minister and Cabinet’s (DPMC) Public Data Branch has partnered with the CSIRO’s Data61 for the overhaul. The Public Data Branch is a product of a 2015 fusion of the Department of Finance’s open data team, the Data Policy Branch and the Department of Communication’s Digital Government Strategy.

Data61 is planning to release a first prototype of the new portal later this year. The initial prototype is expected to offer improved searching across multiple data repositories. Further prototypes will be released in 2017 “focussed on data publishing, data quality, better spatial publishing, and integration” according to a blog entry by the DPMC’s Toby Bellwood.

Under the government’s National Innovation and Science Agenda Data61 has been tasked with employing “data analytics to connect disparate government datasets and publicly release them on open data platforms”.


The organisations is currently the subject of a Productivity Commission inquiry. A draft report of the inquiry is expected in November, ahead of a final report in March 2017.

Australian Bureau of Statistics, job advertisements, September 2016:


Canberra Office, Belconnen, ACT
Executive Level 1 ($94,933 - $109,741)
Ongoing Position

The Australian Bureau of Statistics (ABS) is looking to engage an innovative Media Manager who will lead, in partnership with the Director, the Corporate Communications team in promoting the ABS as a world leading national statistical office, and support the use of statistics for informed decision making by governments and the community.

As Australia's national statistical agency, the ABS produces official statistics that inform important decisions made by government, business and the community.

The Corporate Communications Section is responsible for ABS’ enterprise communication internally and externally. The Section manages key internal and external channels, associated content and contributes to strategic external partnerships such as the media and ministerial liaisons.

About the Position:
Working in partnership with the Corporate Communications Director, the successful applicant will manage a team of diverse communication professionals to promote the ABS as a world leading national statistical office and support the use of statistics for informed decision making by governments and the community.

The Media Manager will be accountable, under broad direction, for the ABS’ media activities, including the documentation of communication plans, drafting media releases, producing social media and managing media relations.

The role involves working with both internal and external partners to help achieve the corporate goals of the ABS, and assist in the development of strategies and procedures to ensure best practice communication. This role also involves working with the ABS senior executive on internal and external communication.

Qualifications and Experience:
*Experience in managing media and communication operations and teams
* Highly developed communication, influencing and interpersonal skills
* Ability to prepare high-quality and innovative communication materials
* Strategic stakeholder management and engagement
* Flexibility, initiative and organisational skills to work effectively in a small team to tight deadlines

Relevant tertiary qualifications or suitable experience and skills are essential, including specialist fields such as Journalism, Social Media, Communications, Marketing and or Public Relations.

Application process
Applications open Thursday 8 September 2016, and may be submitted by Wednesday 22 September 2016. For further information please refer to the Applicant Information Kit.

Help and Contact Information
If you require technical help submitting your application, please contact ABS Recruitment on 1800 249 583.

For any questions related to this role, please contact Tasma Vyver on (02) 6252 7968 or email tasma.vyver@abs.gov.au

The ABS encourages and values a diverse workforce. Aboriginal and Torres Strait Islander people and people with disability are encouraged to apply. The ABS is committed to improving the gender diversity of its leadership.

2016 Census Field Officers closes 11 September 2016 11:30 PM (AEST)
*Various non-ongoing positions available Australia wide
*Salary: $21.61 per hour (inclusive of 25% loading)
*Opportunity to work in your local community

Twitterverse copies of documents coming out of FOI requests :
Image:@ellipsii
Image: 

Image: @unfiler & @seanparnell
Image: @Asher_Wolf

File this one under it had to happen:




Saturday 10 September 2016

NSW Electoral Commission: Local Government Elections 2016 - vote tally live on the night from 6.30pm


NSW Electoral Commission, Results, 10 September 2016:

Virtual Tally Room - link active from 6.30pm

Results will be available as they are counted once polls close at 6pm on Election Day, 10 September.

The first results are estimated to be available about 6:30pm and most election night results by 10pm.

Results by NSW Local Government Area here.

Mayoral Results

Councillor Results

Referendum and Poll Results

Counting Schedule


NSW Northern Rivers History - Banyam/Baigham, Lismore


Banyam/Baigham, or North Lismore, is an important place for local Aboriginal people, the Bundjalung Nation and tribes from other countries......For thousands of years the showground site has been a significant gathering place, and the landscape around the Showground is rich with stories from the Dreaming through to the present day. [History Councils NSW]

Friday 9 September 2016

Australian Bureau of Statistics: the pot of gold that is big data


The privacy concerns arising in respect of big data tend to have two foci. First, there are ethical questions about how private information is captured and subsequently used without the subject’s knowledge or consent. Second, there are concerns that the way governments and corporations store and secure this data fails to reach an appropriate standard, leaving the door open for private individual data to be accessed by unauthorised persons, or otherwise released. [NSW LC, Standing Committee on Law and Justice, Remedies for the serious invasion of Privacy in New South Wales, 3 March 2016]

The Australian Bureau of Statistics (ABS) has shown an interest in accessing the gold mine that is retail scanner data.


Big data refers to the large volume of structured or unstructured data that organisations generate and store. It is characterised as data that generally contain high volume, high velocity and/or high variety information and demands cost-effective, innovative ways of processing for enhanced insight and decision making .

The opportunity that big data presents to statistical agencies is the potential to produce more relevant and timely statistics than traditional data sources such as sample surveys. As an input into official statistics, either for use on its own, or combined with more traditional data sources, Big data could help position National Statistical Offices (NSOs) to improve the accuracy of their measures or the quality of the statistics produced. It can also help improve the comprehensiveness of official statistics by addressing existing data gaps.

An example of Big data is transactions data from major retailers obtained from the electronic capture of product information at the point of sale. Transactions data contain detailed information about the business name and location of the transaction, date and time, quantities, product descriptions, values of products sold as well as their prices.


ABS innovations will meet new and emerging data needs. For example, the ABS is developing a prototype known as the Graphically Linked Information Discovery Environment (GLIDE), which is a suite of tools using Semantic Web methods to help analysts explore and visualise linked data. GLIDE has linked personal income tax data with business tax data to explore new methods to manage, link and analyse cross-sectional and longitudinal data.

A pilot project to inform policy development through the combination of Census and social security information was established between the ABS and the Department of Social Services.

The ABS is already a user of big data - with considerable potential to use much more - as effective use of this government data reduces our need to collect information separately and directly from households and businesses.

ABS is moving beyond the public data environment to draw insights from retail scanner data, to explore options with other data sources such as investigating the use of satellite imagery to measure agriculture crop yields and new methodological approaches to using telecommunication location information.

The spatial opportunities of big data approaches are considerable and have the potential to fundamentally change how we produce population information - especially the extent to which we can measure temporal dynamics which have generally been beyond the reach of traditional approaches.

And this online article gives a strong clue as to why the ABS would like to link national census data on individuals and households to retail scanner data – it will increase the commercial value of the statistical data products it offers for sale.

Ad News, 5 September 2013:

One of the country's biggest advertisers, Woolworths, said it doesn't need big splashy ad campaigns to launch its insurance offering. Because its database tells it the people it needs to target directly.

Woolworths Limited director of group retail services Penny Winn said the company has been deliberately shying away from traditional mass advertising for its new insurance business.

Woolworths' combined insurance statistics database and frequent shopper database found those who buy milk and red meat are better insurance risks than those who have pasta, rice and liquor in their shopping baskets. As a result, Woolworths are able to target those good insurance risk customers directly with better insurance offers.

“What we've been able to do is take our insurer's car crash database and overlay it with our Woolworth's Rewards database. I rarely see actuaries get excited but they were very excited about what we found because it was so statically significant,” said Winn.

“Because you see, customers who drink lots of milk and eat lots of red meat are very, very, very good car insurance risks versus those who eat lots of pasta and rice, fills up their petrol at night, and drink spirits. What that means is we're able to tailor an insurance offer that targets those really good insurance risk customers and give them a good deal via direct channels instead of above-the-line [advertising]. And it helps to avoid the bad insurance risks.”


It seems that along with an individual’s name, address, marital status, income range, education level, ancestry, personal hygiene regimen, criminal or traffic infringement record, taxation liability and/or welfare payment history, medication and health status, the ABS would also like to have the option of assessing the individual’s alcohol consumption and insurance risk level.

What a treasure trove for those with a malicious heart and refined hacking skills or overly inquisitive police and national security agencies.