The idea of ‘Big Data’ is much discussed within the financial markets, as it is elsewhere; particularly concerning how it could and should be used. Yet there remains much confusion and not a great deal of consensus over what “Big Data” truly means, and more importantly what it means for businesses. In the trading world – and specifically within the trading communications space - such data can be incredibly valuable. But as is the norm when interrogating large datasets, it is vital that the right questions are asked in the first place.
When is “Big Data” being captured?
The simple answer is ‘all the time’. There are huge amounts of data from multiple channels of communication that can be captured, archived and analysed. Within the trading environment, this includes actual trade data, blotters & history – everything that takes place around a trade, as well as wider trading floor communications. MiFID II will require all communications that are intended to result in a trade to be recorded and stored. So when we talk about ‘trading floor communications’, this means fixed line and mobile calls, emails, and instant messengers – from Bloomberg and Reuters to MSN or Skype.
Capturing the vast array of data passing through an organisation is a tough technological challenge, but doing so is increasingly offering firms competitive advantages. Focusing exclusively on the financial markets, the ability to apply analytics to anything from trade executions and overall trader performance, to finding inefficiencies in a trading workflow, can offer firms the kind of governance and oversight which was simply not possible in the past.
Where are you storing it?
Requirements for data storage are now significant, and maintaining such storage is a job in itself. Storage needs to be resilient and secure, with well-maintained back-ups and the possible use of WORM-compliant storage solutions (WORM: ‘Write Once, Read Many’ - a protocol which guarantees that the data is not modified in any way after it is first written to the storage medium in question). In many cases, a lot of big data is currently stored on storage media that, if not already obsolete, is coming to the end of life – or as in the case of tapes, is a medium which is physically degrading and for which no like for like replacement parts are available.
There are particular risks associated with tapes, and in parallel, increasing pressure from regulators and recognition by banks that those need to be taken care of and pushed to network attached storage. Once this migration has been completed, one of the major benefits – as well as no longer being dependent on old tape-based storage media – is that you can apply new technologies to the historical data. This means that if a regulator makes demands for specific data from five (or more) years ago, the data can be searched for, accessed and swiftly presented in a way which was simply impractical, if not impossible, beforehand.
As a result of the demands upon financial institutions for secure and resilient data storage, the risks and costs associated with storing data on their premises are high. Increasingly therefore, hosted storage is being considered as an alternative model. For years, some data – such as emails – have been archived in the cloud. Nonetheless there has been resistance to storing other data, such as voice calls, there too. However, we are now at a time when many other things – ultimately all services - could and should be stored in the cloud. There are clear benefits of making the move: less risk, flexibility and scalability of storage, as well as cutting the cost of investing in large amounts of new infrastructure.
Are you using big data effectivity to your advantage?
For compliance, being able to use the available data in the right way when needed is crucial. Demands from regulators can be challenging: firms are expected to be able to respond to any request to produce the right extracts of the vast amount of data that forms massive storage archives. At the moment this can be costly, time-consuming and inaccurate; manually listening for sensitive phrases in phone calls for example, means that the possibility of uncovering anything from breaches of policy all the way through to market abuse is not achievable on a systematic basis.
However, there are now various technologies that offer the possibility of filtering through the hundreds of thousands of recorded communications using advanced phonetic searches and Compliance Officers can now proactively set up alerts linked to particular scenarios. This means that when certain terms or phrases are mentioned, a workflow is provided that allows you to identify, flag and mark for review any targeted communications. Firms should utilise such technology to test their data capture against specific regulatory scenarios that relate to their own compliance culture, to ensure they can meet regulators’ demands if required to.
Beyond the Compliance department, utilising technology to mine the vast amount of data being captured can be a significant benefit to your wider business strategy. From analysing sales if products are being mis-sold or spotting opportunities to cross and up-sell, to monitoring service quality and examining whether staff are following procedure and standards on client calls; the insights gained from Big Data can direct and drive improvements across business functions and give you a competitive advantage.
Looking forward to 2016 and the implementation of MiFID II, firms will need to become more proactive in the way they manage and monitor the many channels of communication used every day. Big Data is a key tool in doing that, once you think beyond simply capturing it, to how you can use it and what it can bring to your organisation.