April 17, 2024 — Panel Discussion

Trust & Safety Industry Trends at Yelp

At Yelp, consumer trust is one of our top priorities, which is why we implement extensive trust and safety measures. Fresh off the release of our 2023 Trust & Safety Report, this Yelping From Home: Tech Talk series featured a panel discussion with Yelp experts who shared their insights on trust and safety and their experiences helping Yelp remain a reliable and safe platform. They discussed the emerging industry trends and the value of building evolving protections for users. The event provided valuable information on how we protected the integrity and quality of content on Yelp to better connect people with great local businesses.

Tell us about your role/team at Yelp and how it contributes to maintaining trust and safety on our platform.

Marcello: I am currently the Tech Lead of the Content Safety team at Yelp, and our team is responsible for engineering most of the systems that provide real time moderation of user generated content and help classify vast amounts of text and image/videos to protect Yelp from spam and inappropriate content or fraud. My role is to provide the technical roadmap and vision for the Content Safety team such that we maintain an edge over bad actors in this very challenging and adversarial domain.
Jiachen: I am the product manager for Content Safety and Traffic Quality team at Yelp. My main responsibilities are finding important current and future threats for Yelp users' trust & safety, and collaborating with different stakeholders to make sure our engineering teams are set up for success to solve those problems. I work on addressing inappropriate content or fraud like Marcello mentioned, and also things related to malicious bot activities.
Sepideh: I am a machine learning engineer in the Spam team. My team is mainly focused on surfacing reliable and trustworthy reviews on Yelp. At the core of our work is Yelp’s automated recommendation software, which is composed of ML models that evaluate all reviews on the platform in terms of quality, reliability and trustworthiness. As a Machine Learning Engineer, I am actively involved in every stage of our ML projects’ lifecycle. This includes everything from initial ideation and analysis, to feature engineering, training models, and ultimately deploying and monitoring them.

What’s an interesting aspect about Yelp’s Trust & Safety policies that people may not know about?

Jiachen: I think one thing that sets Yelp apart is that we prioritize content quality over quantity. For example, when a user posts a review on Yelp they must include review text with a star rating. Some other review platforms don’t require this, but we believe this information helps consumers make more informed decisions, and it helps businesses improve their customer experience.
Sepideh: I want to give another example of how we prioritize quality over quantity. We're also aggressive about identifying and weeding out unhelpful and unreliable reviews. For instance, our automated recommendation software looks for reviews that may be unfairly biased — such as reviews people may write about their competitors or businesses they’re affiliated with. Or as another example the software looks for reviews that may have been solicited. As of December 2023, about 18% of all reviews contributed to Yelp are not recommended by the software.
Marcello: I am a machine learning engineer in the Spam team. My team is mainly focused on surfacing reliable and trustworthy reviews on Yelp. At the core of our work is Yelp’s automated recommendation software, which is composed of ML models that evaluate all reviews on the platform in terms of quality, reliability and trustworthiness. As a Machine Learning Engineer, I am actively involved in every stage of our ML projects’ lifecycle. This includes everything from initial ideation and analysis, to feature engineering, training models, and ultimately deploying and monitoring them.

Shifting gears a bit, what are some of the ways Yelp maintains transparency, while also fostering a safe platform?

Jiachen: Every year, we collaborate with different teams at Yelp to publish the Trust & Safety Report. We publicly share data about our automated recommendation software, content moderation efforts, and detection of other deceptive behaviors. We also use this medium to share insights on how we’re using new technologies to make Yelp more helpful and reliable. We also work with other product teams at Yelp to help them think about how to keep the Yelp community safe when we release new features.
Sepideh: Other than the Trust & Safety report that Jiachen mentioned, we offer clear Content Guidelines and policies that outline the dos and don'ts for contributing various types of content, such as reviews, videos, and photos. As an example, these guidelines explicitly state that business owners and employees should not write reviews for their own businesses or their competitors. In fact, our recommendation software is engineered to detect and unrecommend reviews that may be a conflict of interest.
Additionally, to enhance transparency, reviews that are not currently recommended by Yelp’s automated recommendation software are moved to a separate section, hyperlinked at the bottom of the business page. This allows consumers to still access and view them. The software is continuously learning more and evaluating reviews, which means that reviews can shift between recommended and not recommended over time.

Now that we have explored Yelp's trust and safety policies, how have the latest advancements in tech shaped the work you do when it comes to content moderation and keeping users safe

Sepideh: In the last few years, we've seen some incredible advancements in AI that have been pretty exciting. One of the biggest game-changers are LLMs. They're like the superheroes of the AI world, transforming how industries operate. We also leverage this technology right here in Trust & Safety! As an example, they help us dig deeper into reviews, understand them better, and classify them more effectively.
Marcello: Recent advancements in image and text processing have allowed Yelp to provide an even better user experience. In the Trust & Safety space, AI, ML, Deep Learning (DL), and especially LLMs, can be leveraged to improve the efficacy and efficiency of content moderation. LLMs are especially powerful in this context since they can directly interpret existing content policies and don’t need large labeled datasets for training and evaluation.
Jiachen: Building on what Marcello mentioned, last year we started using LLMs to better detect reviews that may contain egregious instances of hate speech or vulgar language before they are ever published to Yelp. This new system flags potentially inappropriate reviews to our human moderators who evaluate the content, which helps make a determination to further inform the LLM.

What’s the most rewarding part about working in trust and safety at Yelp for you?

Marcello: Being a Yelp user myself it feels especially good to know how my work improves the quality and trust of the product. It also gives me visibility on the dark side of the internet and the type of complex issues that can arise; it requires switching your mindset to understand the dynamics and incentives driving bad actors.
Sepideh: The most rewarding part for me is definitely the significant impact our efforts in Trust and Safety have on gaining user trust in Yelp. Our efforts make a big difference for both businesses and consumers. It’s great for helping businesses get noticed and gain customers. And we also help consumers discover the businesses they’re looking for by pointing them to relevant and reliable information.

Panelist:

Jiachen joined yelp in April 2022. She is currently a Product Manager in the Trust & Safety team. She spends her time identifying and evaluating opportunities to make Yelp a safe platform for both users and business owners. Before Yelp, Jiachen worked in finance and insurance.

Marcello joined Yelp in November 2019 as a Machine Learning engineer for the Traffic Quality team. He spearheaded efforts and vision that eventually resulted in the creation of the Content Safety team of which is currently the tech lead. Before Yelp, he worked at Veritas Technologies as Sr. Data Scientist working on time series forecasting and graduated from the Florida Institute of Technology with a Ph.D in Computer Science researching human mobility.

Sepideh is a Machine Learning Engineer at Yelp. She is part of the Spam team, dedicated to enhancing and maintaining the Yelp review recommendation system. Before Yelp, she was a Software Scientist in the energy industry and conducted her PhD research in Software Engineering at the University of Calgary, exploring machine learning in safety applications for intelligent and connected vehicles. Originally from Iran and now based in Calgary, Alberta, Sepideh loves to travel and engage in outdoor activities like hiking and skiing, taking pleasure in the natural beauty around her.

Moderator

Emily is a Technical Recruiter at Yelp, having transitioned her career from the field of teaching to the realm of recruitment. With approximately seven years of experience in technical recruitment, she has gained valuable insights while working within the ecommerce and data security domains. Emily holds degrees in Psychology and Human Resources from Concordia and McGill University, respectively.

Her passion lies in the field of technical recruitment, where she strives to incorporate an inclusive and diverse mindset into every aspect of her work. Emily recognizes the profound importance of human connections and places great emphasis on creating exceptional candidate experiences. Her genuine intention is to make candidates feel comfortable and empowered to be their authentic selves throughout the recruitment process.

About Yelping From Home: