By Israel Hersh, SVP Business Development, tekVizion

It’s the weekend. The markets are closed and the trading floor has fallen silent. The usual cacophony of voices negotiating complex FICC transactions has moved on, making its way to a nearby bar where they can recount the week’s highs and lows.

Only it’s not quite that simple. The trading floor may be quiet, but it’s not deserted. Markets may be closed, but trading systems are not lying idle – for it is at the weekend when the Ops team is able to get to work.

The reality for banks’ Operations teams is that the weekend is the only time when they can get access to the trading floor and carry out the all-important upgrades and testing of the business-critical voice trading system.

This traditionally manual task quite literally requires them to ‘walk the floor’, testing each and every turret to ensure that essential security patches and feature upgrades have been installed correctly and that the system is working properly ready for the markets to open on Monday morning.

But why should we care if these fixes and tests are being done over the weekend?  So long as they’re done, right?

Well no. Not only is it a significant labour cost for the business having to pay a premium for weekend hours, it’s also a significant investment of time by the Ops team which means they are having to work anti-social hours at a time when recruitment is already a challenge across the financial services sector.

So what’s the reality of this reliance by the Ops teams on having to physically ‘walk the floor’?

Take an average sized trading floor; that’s typically about 1,000 turrets that need to be manually tested over a weekend. That translates into a team of 20-25 people having to come into the office on a Saturday and each spending 4-5 hours simply checking everything is still working.

And this assumes everything IS working. If they find something wrong, then the impact on both people’s time and labour costs soon start to snowball.

If the onsite team can’t identify the problem, they have to bring in on-call engineers (so their weekends and family time are also never their own even if things are working). If these engineers can’t explain the reason for a failure and correct it, they have no choice other than to roll-back the software and schedule a repeat of walk the floor on Sunday with the previously working version. If that isn’t the definition of wasted time, effort and money, it’s hard to know what is!

Banks of course try to keep to their own Ops teams as small as possible and augment them with specialist on-call engineering support from their vendors and SIs. However, with security patches going live every month, break/fix patches quarterly and feature releases twice a year, maintaining the voice trading system is a costly business – and this is if everything is working as it should.  But if it isn’t, if issues are found and roll-backs are required and the update rescheduled, these costs are quadrupled. Quite simply the bank is left with no alternative but to pay for this engineering resource at premium weekend rates.

But is there another way? Is it really necessary for engineers to be surrendering their weekends to conduct manual testing of security patches and bug fixes? Is this really the sort of work they want to be doing? Is this the sort of mundane work that the bank wants to be spending its finite Ops budget on when they could have an Ops team working family friendly hours and doing work which is innovating and adding value to the business?

The answer lies in automation.  Not only does automation mean that testing can be done in a fraction of the time and at a fraction of the cost, it can also be done far more proactively.

This means the Ops team can now provide both a larger test coverage and greater predictability by always running the same test and same duration. This isn’t a matter of simple convenience, it delivers a real business benefit – a more reliable trading system with reduced downtime. And when downtime can cost a bank over $9 million every hour the system is offline, it can materially impact the bottom line.

Where manual testing is about making sure the lights haven’t gone off, automated proactive testing is about making the sure the lights stay on.

By integrating test plans into areas such as audio recording compliance, banks can take proactive steps to ensure that the all-important recording systems are always online and fix issues before they become problems.  Put simply, if a trade needs explaining it needs recording – and if you can’t record, you can’t trade.

And here lies another fundamental problem with the reliance on the weekend testing regimen. How can you test recordings if the trading floor is closed and there is therefore nothing to record?  Automated testing solutions such as tekVision Automation mean you can now test media and check the recording. In one step, you’ve taken a significant leap towards greater compliance.

By augmenting manual testing with proactive automated testing, banks can reduce downtime, free up Ops time for innovation and value creation, and make a substantive improvement to the quality of life of their Ops teams.

For more information on how automated testing can help keep a trading floor online and compliant, go to https://www.ipc.com/resource-center/ipc-and-tekvizion/

 © 2019 IPC Systems, Inc. All Rights Reserved. The contents of this publication are intended for general information purposes only and should not be construed as legal or regulatory advice.