NewsBite analysis · AI, Business, Cloud, DevOps, Programming, Security · New Product
InfoQ
Building AI-Powered Safety Systems at Scale in a Real-Time Marketplace
DoorDash has developed SafeChat, an AI-powered safety system designed to combat verbal abuse in real-time during interactions on its platform. By employing a two-layer classification structure, the system efficiently determines message safety using both a rapid internal classifier and an LLM for nuanced analysis. This innovation has resulted in a 50% reduction in verbal abuse incidents, showcasing the effectiveness of data-driven decision-making in enhancing platform safety. The experience exemplifies the potential of integrating machine learning with operational needs, emphasizing the importance of understanding data before implementation.
Act nowConfidence: HighShort term horizon
Published 22 Aug 2026, 12:00 UTC18 min read
Visit original article ↗Lead image — InfoQ
Summary
TL;DR
- DoorDash developed an AI-powered system called SafeChat to address safety issues.
- SafeChat combines a fast internal classifier and LLMs for real-time moderation.
- The system achieved a 50% reduction in incidents of verbal abuse.
- Emphasized the importance of data analysis before deploying AI solutions.
- The architecture allows for scalable and cost-effective moderation strategies.
Interesting facts
- SafeChat processes over 4 million messages exchanged daily.
- Initial response was to use LLMs, but latency and cost constraints required a more efficient model.
- The scoring mechanism allows for nuanced responses based on message severity.