With the installation of Apache Hadoop, a framework for storage and large scale processing of data sets on clusters, we are now able to provide academics and businesses of all sizes with a powerful analytic tool to help them interpret large and complex datasets, rapidly turning large volumes of structured and unstructured data into meaningful information.
It was estimated by the International Data Corporation (IDC) that 2.7 billion terabytes of data were created worldwide in 2012, a figure that is growing rapidly year on year.
With Big Data ranging across sectors from a few dozen terabytes to multiple petabytes – which are thousands of terabytes - and beyond, businesses and academics can now take advantage of this wealth of data with the new powerful analytics tool.
The Big Data solution can be used for anything from predictive analytics, social media analytics and text analytics to disease detection, prevention and treatment; financial modeling and smart energy metering.
With 17,000 computer cores and a peak processing performance of almost 320 TFlops, our supercomputing network is capable of running 320 trillion operations per second. This processing power combined with access to sophisticated software packages, like Hadoop and R analytics, mean businesses and academics can now harness the potential of supercomputing for the analysis of Big Data.
Professor Sian Hope, Interim CEO of HPC Wales, said:
“In just one minute of online activity, there are two million Google searches, 685,000 Facebook updates and over 200 million sent emails.
“Companies such as Amazon and Tesco have been harnessing Big Data for commercial gain for some time now. These firms gather tonnes of data on customers, from what they’ve purchased to what websites they visit, where they live, when they’ve contacted customer services, and if they interact with their brands on social media.
“With the launch of our new Big Data solution , businesses now have access to state-of-the-art technology to help them interpret this sort of data, providing valuable insights , helping them to boost their competitiveness in global markets.
A company in Wales already benefiting from the Big Data solution is Caerphilly-based Butterfly Projects, which provides Big Data and predictive analytics services to the likes of Lloyds Banking Group and Zurich Insurance.
Sara Boltman, Founding Director at Butterfly Projects, said:
“We are using the software package Hadoop for big data analysis and R for predictive analytics. In the past, we were only able to process up to 120 million records and to achieve that our statistical model building machines would have to run over night - for at least 12 hours.
“With access to the HPC Wales system, we no longer need to limit ourselves and can increase the complexity and volume of our workload, competing for larger and more competitive contracts.”