Cantina Labs is a social AI product that combines group chat, voice, and video capabilities together with AI character creation. With our advanced AI bot creator tools, users can design digital versions of themselves, bring imaginary characters to life, or choose from thousands of existing AI bots in our user-generated bot directory. Cantina bots are lifelike, social creatures, capable of interacting via text, voice, images in Cantina rooms or anywhere across the internet as they can be shared off platform. We have made Cantina invite-only for now to enable the development and implementation of strategic safety systems and core functionality ahead of opening up the platform to the wider public. By keeping the user base small, we are able to refine our safety policies, features and prevention systems.
Q1 2025: As a signatory of the Inspired Internet Pledge, Cantina is committed to fostering a digital environment where young people can explore, create, and connect with safety, agency, and support. Over the past quarter, we’ve made meaningful progress toward building a more transparent, protective, and empowering experience for our teen users:
1. Improving Age Transparency: We introduced clear, in-product messaging during account creation to reinforce our minimum age requirement of 16. We also added in-product education around bot age—bots on Cantina must be aged 18 or older—helping ensure that users, especially teens, understand who they are engaging with. We further increased transparency by publishing a public-facing age policy in our Help Center, making our standards easy to access for users, parents, and caregivers.
2. Enhancing Mental Health Support: We launched a new partnership with Crisis Text Line, a leading nonprofit in mental health support. This collaboration enabled us to introduce new self-harm and suicide prevention messaging to connect users who may be in distress to expert resources.
3. Supporting Stakeholder Education: To make our safety policies and design more accessible, we launched a new Teen Safety section in our Safety Center. This resource is designed for parents, educators, and young people themselves, offering clear guidance on how Cantina is built to support and protect youth across both product features and policy.
In terms of the product, because our goal is to balance creative freedom with safeguards, we introduced protections in December 2024 such as making all teen accounts private by default and including restricted direct messaging settings.
We are proud of the foundation we laid over the last year and this past quarter—and we’re just getting started. Our goal is to continue evolving our platform to meet the needs of young people and the communities that care for them.
Dec 2024: Cantina Labs joined the Inspired Internet Pledge as an Signatory.
We have joined the Inspired Internet Pledge so that we can be held accountable for our Trust & Safety product, policy and operations work, but also, as a means to place mental health at the core of our efforts.
We acknowledge the potential for misuse of our social spaces and generative AI tools. To ensure a safe experience for all users, we have implemented a number of measures and precautions over the last few months. For instance, we have reporting tools across the platform to enable users to flag content and interactions that are concerning to them.
For abuse detection, we have adopted a hybrid approach, combining third-party classifiers by leading experts like Thorn and Hive with in-house word and phrase blocklists in multiple languages, including English, Spanish, Portuguese, Russian, French, Chinese (Simplified and Traditional), Hindi, Arabic, German, Japanese, and Turkish. These detection mechanisms operate across text exchanges in rooms, DMs, bot creation prompts, bot engagement, and image uploads.
We have also joined the Tech Coalition and the Digital Trust & Safety Partnership to ensure that we are learning from our peers and sharing our experiences based on our AI technology and users adoption of it.
We are particularly concerned about the impact of our technology on mental health and we are currently designing a partnership with Stanford’s Lab for Mental Health Innovation to red team self-harm and suicidal exchanges with bots and to design appropriate interventions.
We are focused on the following initiatives to uphold the Inspired Internet Pledge:
- Contribute to and/or actively engage in third-party research related to the Pledge principles
- Establish standards or policies focused on users’ mental health and wellbeing
- Embed users’ mental health into success metrics within teams/departments