Data transformation and automation for a well-known financial services organization
A popular financials solutions provider for Small and Medium Enterprises (SMEs), this Non-Banking Financial Company (NBFC) offers a wide range of loan, financing, and funding solutions.
Finance
Published Date : 02-May-2023
Challenge/Need: The NBFC wanted to design and develop a Data transformation engine that would help generate aggregated data. The transformed data would then serve as a data service layer for systemized reports, visualization dashboards and data science uses in future. The client had finance and customer related information which were not consolidated, and pulling the data for specific needs was time consuming. They ran the data every day and sometimes during month closing, or they held and ran data for the previous 4-5 days together. They did not have the flexibility to run data for different date ranges. They wanted to consolidate their critical data for ease of visualization and insights.
Umbrella Infocare solution: Umbrella Infocare assessed their existing data setup and needs and helped them leverage AWS Glue for migrating their data processing to a Spark based framework. Our team created the framework to automate data processes that helped them achieve their goals. We also set up alerts and notifications for detecting job failures.
Benefits: With a framework for automating and integrating data, the financial services company gains from quick data visualization, fast alerts on job failures and have more flexibility with filtering data for various purposes. AWS Glue helped them run all scripts and jobs smoothly to ensure efficient data querying, running queries based on conditional requirements and monitoring and logging.
Finance
Big Data and Analytics
Dr Lal Pathlabs
Big Data and Analytics
