“From Data to Intelligence: Real-World AI Systems on the Lakehouse” will be held

Speakers

• 

  Software Engineer at PayPay

  Boyang Yue

  Title: Building a Recommender on the Lakehouse: An End-to-End MLOps Architecture
  Abstract: Delivering personalized recommendations to over 70 million users requires more than just powerful algorithms; it demands a robust, scalable, and automated data intelligence platform. As the largest mobile payment provider in Japan, PayPay is committed to continuously improving the user experience for every customer. This presentation will outline how we build the end-to-end architecture to power our core recommendation system: PayPay Brain. We will walk through the entire machine learning lifecycle, from architecting data ingestion and preparation, to developing and training models, and finally to deployment and serving. Beyond the technical implementation, the talk will distill our lessons learned in building a sophisticated ML system within a regulated fintech environment. We will discuss key architectural decisions for breaking down data silos between teams, implementing robust governance and security without sacrificing agility, and maintaining operational efficiency.
  ​Bio: Boyang Yue is a software engineer with a master's degree in computer science. He joined PayPay in 2023. His interests include data engineering, recommender systems, high-performance computing, and functional programming. When he's not at his keyboard, you can find him exploring new hiking trails or learning new skills.
  Title: Building a Recommender on the Lakehouse: An End-to-End MLOps Architecture
  Abstract: Delivering personalized recommendations to over 70 million users requires more than just powerful algorithms; it demands a robust, scalable, and automated data intelligence platform. As the largest mobile payment provider in Japan, PayPay is committed to continuously improving the user experience for every customer. This presentation will outline how we build the end-to-end architecture to power our core recommendation system: PayPay Brain. We will walk through the entire machine learning lifecycle, from architecting data ingestion and preparation, to developing and training models, and finally to deployment and serving. Beyond the technical implementation, the talk will distill our lessons learned in building a sophisticated ML system within a regulated fintech environment. We will discuss key architectural decisions for breaking down data silos between teams, implementing robust governance and security without sacrificing agility, and maintaining operational efficiency.
  ​Bio: Boyang Yue is a software engineer with a master's degree in computer science. He joined PayPay in 2023. His interests include data engineering, recommender systems, high-performance computing, and functional programming. When he's not at his keyboard, you can find him exploring new hiking trails or learning new skills.

• 

  Senior Data Scientist at PayPay Corporation

  Waylon Chen

  Title: Taking Machine Learning models from research to deployment
  ​Abstract: We will discuss how we streamline the process of taking ML models and their dataflows from research to production with Databricks at PayPay. From a DS/MLE perspective, we will walk through how we develop our ML models at scale with PySpark and Jobs/Workflows within Databricks' ecosystem, focusing on the pain points abstracted away by Databricks. We will also share the simplified process to seamlessly automate deploying an ML model into the production environment with Databricks Asset Bundle.
  ​Bio: Waylon is a Senior Data Scientist at PayPay Corporation, leading credit modeling development works. Since joining PayPay in May 2023, he has built various models for PayPay’s lending business. He has been pioneering model development and leading the implementation of end-to-end ML workflows with Databricks at PayPay.
  Title: Taking Machine Learning models from research to deployment
  ​Abstract: We will discuss how we streamline the process of taking ML models and their dataflows from research to production with Databricks at PayPay. From a DS/MLE perspective, we will walk through how we develop our ML models at scale with PySpark and Jobs/Workflows within Databricks' ecosystem, focusing on the pain points abstracted away by Databricks. We will also share the simplified process to seamlessly automate deploying an ML model into the production environment with Databricks Asset Bundle.
  ​Bio: Waylon is a Senior Data Scientist at PayPay Corporation, leading credit modeling development works. Since joining PayPay in May 2023, he has built various models for PayPay’s lending business. He has been pioneering model development and leading the implementation of end-to-end ML workflows with Databricks at PayPay.

• 

  Solutions Architect at Databricks

  Shuji Oya

  Title：Building and Bridging AI Agents with Databricks
  ​Abstract: This presentation will cover how to use AI agents with Databricks, focusing on two main approaches: Building and Bridging. First, we will explain how to build AI agents directly on the Databricks platform. We'll show how you can use your own data to create these agents, with options for no-code, low-code, or fully code-based development.
  Next, we will discuss bridging, which involves connecting external AI agents to data managed within Databricks. This part of the talk will highlight the open nature of the platform. By covering both methods, we will demonstrate how Databricks is designed to help you avoid vendor lock-in, giving you the flexibility to choose the best tools for your data.
  ​Bio: Shuji Oya is a Solutions Architect at Databricks, focused on the digital native industry. With his prior experience at several companies in this space, such as Yahoo and Google, Shuji is passionate about understanding core client challenges and delivering impactful business solutions.
  Title：Building and Bridging AI Agents with Databricks
  ​Abstract: This presentation will cover how to use AI agents with Databricks, focusing on two main approaches: Building and Bridging. First, we will explain how to build AI agents directly on the Databricks platform. We'll show how you can use your own data to create these agents, with options for no-code, low-code, or fully code-based development.
  Next, we will discuss bridging, which involves connecting external AI agents to data managed within Databricks. This part of the talk will highlight the open nature of the platform. By covering both methods, we will demonstrate how Databricks is designed to help you avoid vendor lock-in, giving you the flexibility to choose the best tools for your data.
  ​Bio: Shuji Oya is a Solutions Architect at Databricks, focused on the digital native industry. With his prior experience at several companies in this space, such as Yahoo and Google, Shuji is passionate about understanding core client challenges and delivering impactful business solutions.