November 16, 2021 by Manpreet Singh

Talk Summary: Since we started our first successful execution of “hello world” during our first programming assignment to maintain or roll out complex product and infrastructure changes, debugging has become an integral part of an engineer’s journey. Lack of sufficient logging, mysterious behaviours of systems, the inability to add a live-debugger, and the complexity of components involved in a single change can all make debugging a frustrating, challenging, time-consuming, and at times a discouraging and non-rewarding process. However, knowing some common debugging principles can help you identify root-causes in a more structured, timely, and playful way.

In this presentation, I will illustrate some common debugging principles using various examples, such as from mysterious issues we faced using Spark on large scale data engineering problems at Yelp. I will share how this journey can help design test set-ups for faster debugging iterations. I will also demonstrate how these mysterious situations and debugging principles can help propel the learning process for you and your teams, as these principles can be extrapolated across most complex industry-wide systems.

Speaker: Manpreet Singh

Manpreet Singh is a software engineer at Yelp who has led multiple streaming and batch infrastructure projects. His team is responsible for building and maintaining Yelp’s machine learning infrastructure and data mining tools. The components owned are powered by Spark, Kubernetes, and JupyterHub, an interactive notebook search and archiving tool. These components collectively handle the building of hundreds of ML models and the publishing and processing of billions of messages per day. In his time at Yelp, he has spearheaded the efforts on multiple Kafka cluster upgrades, such as re-architecturing Kafka clusters to enable on-demand Spark cluster for ML and data mining applications, upgrading Spark to the latest versions, maintaining Spark infrastructure platform, and replacing the Spark job scheduler from Mesos to native Kubernetes. He has also mentored many engineers.

Moderator: Cecilia Ng

Cecilia Ng is a technical recruiting coordinator who joined Yelp 5 months ago. While she usually communicates with candidates to arrange interviews and chats with fellow Yelp employees, she also helps onboard new employees. She is currently based in Toronto, Canada. Prior to Yelp, she worked as a Learning & Development coordinator in the Civil Engineering industry. Outside of work, she loves to listen to all genres of music, play video games and take walks with her corgi.

About Yelping From Home:

Yelp’s mission is to connect people with great local businesses, and two of the teams working behind the scenes to make that happen are our five star engineering and product teams.

Playing an active role in our community is a value that we have strongly prioritized over the years. Typically we would invite you to come to Yelp’s HQ to network, grab a slice of ‘za, and learn about the technology and culture behind Yelp. With stay-at-home orders across the globe, Yelp (like many of you) has had no choice but to adapt to these changes. We understand that this is a strange and challenging time for many, so we’d like to invite you to join us as we navigate this new normal for the tech community together.

To stay connected, create space for folks to collaborate and share some of the things Yelp has been working on, we are launching our very own virtual tech talk series. This series of talks will feature Yelpers from engineering and product and will focus on themes such as career development, technical skills, workplace challenges and Yelp’s response to COVID-19.