Currently Viewing: Big Data

How many times have you heard that big data analytics is set to revolutionize the world? If you read IT blogs, the technology press or attend industry expos and conferences, chances are the answer is countless times.

In March 2016, AlphaGo – an artificial intelligence (AI) program developed by Google DeepMind – beat the world Go champion Lee Sedol 4-1 in a five-match challenge. The news sparked the same flurry of public fear and fascination that previously met IBM’s AI system Watson’s triumph in the US TV quiz show Jeopardy in 2011, as well as its supercomputer Deep Blue’s victory over chess champion Garry Kasparov back in 1997.

We often hear stories about how big data analytics is going to help companies accrue untold wealth. This month, however, the headlines have been dominated by revelations about how the rich and powerful attempt to hide their wealth from the world’s tax collectors.


We’re used to the idea that every two years computer processors double in speed and fall in cost – a prediction first made by Gordon Moore, then head of Intel, in 1965. For 40 years, what’s become universally known as ‘Moore’s Law’ has held true, fueling the phenomenal growth in IT that has transformed our world. But Moore’s observation was not the eternal, immutable ‘law’ that many assumed it to be. There’s a physical limit to how many transistors you can squeeze onto a wafer of silicon before they start behaving unpredictably.

Most coverage of the Internet of Things (IoT) centers around the proliferation of connected consumer gadgets – personal and home appliances such as security cameras, thermostats, smart lighting systems, health and fitness monitoring devices. But one area that’s set to feel the biggest impact of the technology is manufacturing.

Exponentially Growing Big Data

The Met Office is considered to be the most comprehensive weather and climate service in the world. It processes around a petabyte – that’s one million gigabytes – of data every day and has a data store that already stands at over 60 petabytes, and is growing exponentially. Data doesn’t get much bigger than this today – and CIO Charles Ewen wants even more of it to ensure the organization can meet its stated mission of working at the forefront of weather and climate science for the protection, prosperity and wellbeing of the environment, economy and citizens of the UK.

Big data and privacy seem to be at odds. Providing data to companies creates modern day conveniences, but it also allows businesses to see too much into our everyday consumer lives.

Imagine a world where civic life is transformed by a perfect storm of converging technologies – most notably ubiquitous high-speed cloud and smartphones – a slew of Internet-connected sensors, controllers and other devices (the Internet of Things), plus intelligent tools for aggregating and analyzing the huge volumes of big data that these devices will generate, often in real time. This is the idea behind smart cities.