upscale.ai aimed to develop the industry’s first end-to-end AI/ML-driven marketing platform, enabling DTC and e-commerce brands to run performance campaigns on Streaming TV. Their vision included leveraging generative AI to create dynamic TV creatives using brand, product, and social assets while applying advanced ML to optimize campaign outcomes. To achieve this, they needed a robust MVP for an ETL pipeline to extract customer data from third-party APIs and load it into a data warehouse for analytics. With no internal developers available at the project’s inception, they required a partner who could not only build the foundational data infrastructure but also do so in a time-effective manner.
To accelerate development, we conducted in-depth research on third-party integration solutions, evaluating a range of NoCode, LowCode, SaaS, and self-hostable tools. This approach enabled us to reduce development time significantly—achieving in five weeks what would have otherwise taken three months. We then built integrations to extract bulk and incremental data from Shopify and Facebook via REST and GraphQL APIs. Our event-driven ETL pipeline, leveraging GCP Cloud Functions, facilitated seamless data ingestion into BigQuery. Beyond the initial MVP, we scaled and optimized the pipeline, ensuring reliability, maintainability, and higher data throughput to support larger customers. Additionally, we implemented Terraform-based multi-project configurations and introduced a Google Drive integration, enhancing the platform’s extensibility.
Our involvement transformed a proof-of-concept into a production-ready system, capable of handling millions of records while maintaining timely data processing. By migrating from Cloud Functions to Cloud Run, optimizing database settings, and refining query structures, we dramatically improved data throughput—from 18k to over 2M records—while preserving scalability and performance. upscale.ai successfully navigated their alpha stage, handled peak end-of-year marketing demands, and entered beta with a stable, scalable infrastructure. Our contributions not only accelerated the platform’s go-to-market timeline but also directly enabled upscale.ai to attract and onboard larger customers, supporting their business growth.
