Big Data on Cloud for Popular Loyalty Card Program
As India’s largest multi-brand loyalty program, this leading player in the Finance industry has more than 100 partners and a growing base of nearly 100 million members. The loyalty program is also widely accepted across 12000 touch points in more than 100 cities. Customers can add points to their membership card and redeem the points at different stores.
The Big Data Cloud Journey
The well-known loyalty program service originally had all their systems running on premise. Their workloads were on the Statistical Analysis System (SAS) application, which is a highly standardized and reliable solution for data mining, processing complex raw data and generating meaningful insights. However, the customer wanted a Big Data solution on Cloud that would meet the following goals:
- To cut down on massive licensing costs, they wanted to migrate all their SAS applications to AWS
- They wanted better scalability, speed and agility for data consolidation, reports and visualization
- They wanted to optimize data and predictive analytics for planning their marketing campaigns
Umbrella’s Solutions for Leveraging Data on Cloud
Umbrella’s Data Lake Framework and Data Warehouse helped move all the functionalities performed by the SAS application to AWS and ensured cost and data optimization:
- Data is first consolidated from On prem Oracle Warehouse and ingested into the AWS S3 data lake using AWS Glue
- The raw data is processed, and a single repository is established to handle redundancy and disparate formats
- Multiple layers of security ensure data protection and access
- The data is cleaned and stored for user accessing with AWS Glue
- The consolidated data is then uploaded to AWS Redshift for analysis
Reports Transformation Architecture
Umbrella Value
- 100 tables ingest data from Oracle Data Warehouse to the Umbrella data lake everyday
- 90+ reports including revenue and settlement reports were rewritten using Opensource technologies such as Spark SQL running over AWS Glue
- Two data science models based on the Amazon Opensource software called R on EC2 help predict the type of campaigns to be launched in the market
Business Impact
- Saved vastly on costs for SAS licenses with Amazon Opensource
- 4 hours in existing Oracle/SAS based data processing system reduced to 30-40 minutes with Apache Spark on AWS Glue
- Reports that could be run only twice or thrice a week are running everyday
- Future scope of the project to leverage Amazon QuickSight for quick and smooth visualization
Umbrella helped the financial services provider to leverage Big Data and AWS for cost effective data consolidation that enables effective campaign planning.
Technology
- AWS S3
- AWS Glue
- Amazon Redshift
- Identity and Access Management (IAM)
- SNS for alerts
- CloudWatch
MyTeam11
Big Data and Analytics
