Big data and its implications for the statistics profession and statistics education

The convergence of computer and communications technology, the processing speed of the computers, and the widespread use of the internet around the world have led us to the age of Big Data. Massive volumes of data have been and are being collected and this far outstrips the capacity to analyse the data and convert it into useful and usable information. The need for people who can analyse the data and obtain useful and usable information from it has led to a new career path: the “data scientist”. The data scientist is a kind of hybrid of a statistician and a computer scientist / IT professional. Big Data and its analysis is an area in which industry is leading and academia seems to be playing catch up. Universities are responding to the changing needs of industry and government by introducing new degree programs and units of study. In this thesis, we investigate the perceptions of graduates working in the analysis of Big Data and the perceptions of academics about the types of expertise and the types of software skills required for working in this new field via online surveys. This facilitates comparison between the perceptions of statisticians and of computer scientists about what expertise and software skills are required, and provides information useful for the purpose of developing the curriculum for new degrees in data science and statistics which is urgently needed.