Kevin Cunnington, Digital Transformation Director General at the Department for Work and Pensions, shares his observations from a recent trip to the U.S.
I’ve just returned from California, where I spent my time visiting Silicon Valley tech companies, looking for trends and practical examples of technology that could be exploited by the Department for Work and Pensions.
Big data
While I was there I found that one item dominated the agenda: big data. Apache’s open source Hadoop project lets you store and process massive data sets on clusters of commodity hardware. Springing up all over the Valley are companies providing tools that build upon the strong foundations laid by the community involved in the Apache project.
Hadoop itself is made from a number of components, all built with the fundamental assumption that hardware will fail, but that the overall system should keep on functioning.
Companies like Cloudera, Pivotal and HortonWorks are all building upon Hadoop to supply impressive and innovative new products. Their presentations have given me much food for thought as to how we could re-purpose our existing DWP data warehouses away from Oracle, and toward a more open and distributed analytics architecture.
Analytics
Analytics are also, naturally, a hot topic. We spent some time with Revolution Analytics, whose founder left his job at Accenture when he saw an opportunity to transform the use of data to provide more comprehensive analytical capabilities. Like many new companies springing up in Silicon Valley, they’re adding value on top of existing open source technologies to provide something innovative and new. I can easily imagine using some of the techniques I saw to provide support to our policy research and fraud detection teams.
The second analytics company we met was Platfora. Platfora provide an end user-focused tool that allows users to access data held in Hadoop using a visual interface.
Cyber Security
We also met some specialist cyber security companies. I was impressed when we visited the UK company Bromium, who have an office in the Valley. They make a piece of hardware called vSentry that isolates applications into their own VMs to ensure security between them.
Adallom is a cloud security provider whose founders were previously with the Israeli Intelligence Corp. The product is primarily used for securing employee use of cloud-based software services which, in concept, is like our own DWP digital security service. Whilst DWP's security service is directly integrated into the customer experience, Adallom requires the customer to be redirected through the Adallom service prior to accessing the cloud-based software service.
Finally we met with Palantir Technologies who have been supported by the US security agencies and build a suite of products that have been broadly deployed in the intelligence, defence, law enforcement and financial sectors. I've invited them to meet with Cath Hamp, Director of DWP's departmental Security Design team.
Next week, I'll write all about our discussions with the Google X team who have been developing driverless cars and aerial wind turbines. I'll also share the latest trends in the Silicon Valley transport systems (and boy, do they need better transport systems!).