Enhancing Critical Data for A Major Australian Financial Institution
In early 2022, Kasna brought together disparate data sources from legacy systems to democratise critical intelligence and innovate toward digital transformation for this major Australian bank.
Bottlenecks and opportunities identified to simplify complex business processes, achieving increased interoperability.
Fully automated big data processing ETL pipelines to ingest batch and streaming data securely from several sources in real-time.
Securely integrated engine for customer insights to make critical banking business intelligence intuitively available.
This major Australian Bank offers a myriad of banking products and services to a large number of customers globally. A key facility that their customers enjoy is top-tier wealth management and advisory which is renowned in the banking industry worldwide for its performance and risk appetite.
In order to provide quality wealth management suggestions to clients, this bank’s wealth managers and advisors rely on technology to collect massive amounts of customer banking data and crunch large numbers to draw portfolio performance insights and trends. Kasna identified the following in its key examinations:
- The ageing technologies that traditionally powered their wealth management have begun to show difficulty keeping up with the massive surge in the amount of data that has been generated over time and continues to accelerate at an exponential rate as the days go by. Of late, the reliability of legacy systems has dropped so low that rerunning failed batch jobs have begun to overshoot daily SLAs.
- These problems have led wealth management business applications to gradually become slower over time. The freshness of critical banking data necessary for wealth management advisory is dropping at a fast rate. Lack of fresh data availability from disparate systems in a fast, event-driven way is increasingly turning into a major bottleneck, preventing wealth advisors from being able to provide a customised and timely wealth management advisory to their clients. Of late, data freshness has dropped so low that data can often be up to a day old, and the lack of an up-to-date daily transactions ledger has restricted wealth management advisors to a snapshot of data that can be up to 2 days old.
- Clients are missing out on great opportunities to make use of wealth management products and services in order to time their investment moves to attain profitable positions quickly. The inability to provide customised and timely wealth management services to clients is affecting overall customer experience as well as advisor satisfaction, thereby leading to a drop in customer conversion and retention.
This institution needs to quickly solve the problems of interoperability between disparate banking systems and data standardisation. They also need to elevate data freshness by leveraging the power of streaming real-time data and event-driven processing. These critical enhancements are necessary in order to enable wealth advisors to obtain data updates within seconds of changing, in order to provide accurate and timely wealth management advice to their clients in an ever-volatile and fast-paced investment banking environment.
The following key challenges were identified:
- Banking solutions built with the cutting-edge technologies of yesteryear are suffering from capacity and performance bottlenecks today. Bulky, monolithic software running on-premises on virtual machines and dedicated hardware is getting harder and more expensive to upgrade and maintain at peak performance. It is getting harder to find, hire and retain technical resources that can maintain critical banking software built with legacy technologies, which is a major risk. The inability to scale ageing banking systems is taking a toll on customer experience.
- Over time, various banking departments have developed custom solutions to handle their own data in bespoke ways, thereby creating inconsistencies, duplication and barriers to interoperability between systems. Non-alignment with modern open-sourced and global data conventions and standards is holding back the innovation and modernisation of systems. This inability to modernise is adversely affecting the quality and accuracy of wealth management advice that advisors can give their clients.
- The decline in data freshness has been attributed to the numerous steps and hoops of a growing number of slow, intermittent batch processes that have been historically put as tactical stop-gap solutions over the past few years. The inability to acquire and process data within seconds of creation/change is reducing the freshness and timeliness of critical insights available at the advisor’s disposal to make profitable wealth management advice in a timely manner.
This bank set out to reimagine and rebuild an engine for customer insights on the Google Cloud platform to power the future of banking and wealth management.
To accomplish this, bulk and streaming banking big data siloed within dozens of disjointed applications will be securely ingested and processed at scale and in real-time. The insightful business intelligence metrics that the online portal for advisors and other consumers need to empower wealth managers and clients, will be modelled, computed and unlocked within seconds of a change in data.
Fully managed (serverless) products and services on Google Cloud will be leveraged to scale out/in per demand/volume of data. Elastic autoscaling will be used to optimise operational costs, thereby eliminating capital expenditure otherwise spent on the maintenance of physical hardware used in on-premises data centres.
- Working closely with stakeholders, Kasna identified major bottlenecks and opportunities to simplify complex business processes. Complicated workflows that had become convoluted over years of repetitive tactical fixes were streamlined. This allowed us to gain access to critical banking data raw (facts) as soon as data was created/updated. We successfully standardised disparate data into uniform, globally renowned data formats and conventions for increased interoperability.
- Working closely with Google, Kasna architected fully automated big data processing ETL pipelines to ingest batch and streaming data securely from several sources in real-time. We wrangled and modelled the data at scale to aggregate and standardise data from a number of incompatible data silos. By leveraging powerful big data processing products on GCP such as PubSub and Cloud Spanner, we unlocked critical business insights with the optimal schema and performance.
- We securely integrated an engine for customer insights with an online web application to make critical banking business intelligence intuitively available to wealth managers, advisors and customers through an online portal for advisors.
Key Products/Services Used
Google Cloud Storage
Google Kubernetes Engine
- Google Observability Suite
Release-1 of the engine for customer insights was built, tested, secured and deployed to production in early 2022. The Enterprise scale Customer Data Platform securely ingests several hundred Gigabytes of batch and streaming big data reliably, with a resolution of seconds as opposed to hours. This was achieved by building efficient, modern big data processing pipelines with Dataflow that brought down the processing time from ~18 hours to under 8 minutes. Armed with such fresh data, wealth management advisors are now able to obtain an up-to-the-minute ledger of customer transactions, trends and insights. This has enabled their clients to benefit from the most profitable wealth management advice available within minutes as opposed to hours or days.
The reliability of data processing improved massively, thanks to the fully managed cloud-native products and services such as Dataflow and Cloud Spanner. This has enabled wealth management advisors to request/rerun data processing jobs on demand, in order to customise and personalise their product recommendations for clients on demand, within minutes or hours as opposed to days.
In a first-ever, the Cloud Spanner-based operational data store blends and models banking data from legacy sources for consumption through an online portal to democratise BI capability for wealth managers, advisors and clients. Low latency queries and distributed real-time transactions have allowed clients, advisors and wealth managers to work on the same dataset so that everyone sees the single source of truth. This has eliminated any data inconsistencies arising due to old/duplicate data, thereby elevating the accuracy of personalised wealth management recommendations massively.
‘By bringing together critical banking big data from several disparate and legacy sources together, the engine for customer insights has for the first time unlocked actionable business intelligence out of data that was otherwise hidden and unreachable. This is a major leap towards Digital Banking Transformation through data driven enablement of customers via self service. Secure and automated SRE practices are banking-first’.
– Customer Quote