AI Safety Companies

Roblox AI

San Mateo, United States

Roblox AI develops and deploys artificial intelligence solutions to power the Roblox platform, a user-generated massive multiplayer online game. Their core technology focuses on generative AI tools for content creation alongside AI-driven systems for content moderation and platform safety. Targeting the vast Roblox developer community and its millions of daily users, Roblox AI aims to scale content creation and maintain a safe, engaging virtual environment through automated solutions.

OpenAI

San Francisco, United States

OpenAI is an AI research and deployment company whose stated mission is to ensure artificial general intelligence benefits all of humanity. It develops the GPT family of large language models and products including ChatGPT, the DALL-E and Sora generative models, and widely used developer APIs.

Anthropic

San Francisco, United States

Anthropic is an AI safety and research company that builds Claude, a family of large language models focused on being reliable, interpretable, and steerable. Founded by former OpenAI researchers, it pioneered Constitutional AI and delivers Claude through consumer apps, a developer API, and major cloud partnerships.

Thinking Machines Lab

San Francisco, United States

Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, builds more understandable and customizable AI systems. Raised a record $2B seed round.

OneTrust

Atlanta, United States

OneTrust provides a unified platform for privacy, security, and governance automation, enabling organizations to manage data and comply with evolving regulations. Their core offering centers on AI governance solutions that embed compliance controls throughout the entire AI lifecycle, from data collection to deployment. OneTrust targets enterprises seeking to operationalize responsible AI practices and mitigate risk while accelerating innovation, particularly those facing complex regulatory landscapes.

Applied Intuition

Mountain View, United States

Applied Intuition provides a comprehensive software platform, Vehicle OS, for developing and validating autonomous vehicle (AV) and advanced driver-assistance systems (ADAS). Their technology focuses on large-scale data ingestion, synthetic scenario generation, and machine learning infrastructure to enable rapid development, testing, and deployment of AI-powered vehicle intelligence. They primarily serve automotive manufacturers and technology companies seeking to accelerate AV/ADAS development and improve safety through data-driven insights and efficient software updates.

Kodiak Robotics

Mountain View, United States

Kodiak Robotics develops autonomous driving technology for long-haul freight, focusing on the deployment of self-driving capabilities onto existing truck platforms. Their core product, the “Kodiak Driver,” is an integrated AI-powered system combining hardware and software for reliable autonomous operation in diverse environments. Kodiak targets both commercial freight carriers seeking increased efficiency and the national security sector, offering a scalable solution to address driver shortages and improve operational safety.

Nauto

Palo Alto, United States

Nauto provides a predictive AI-powered driver and fleet safety platform for commercial vehicle operators. Their core technology utilizes in-cabin camera systems and real-time analysis to detect and alert drivers to risky behaviors – such as distraction or fatigue – before collisions occur, functioning independently of external connectivity. Nauto targets the commercial fleet market, offering a privacy-focused solution that reduces incidents, lowers insurance costs, and streamlines driver coaching through event-based video analysis.

Dream Security

Tel Aviv, Israel

Dream Security builds an AI-based cybersecurity platform purpose-built to defend nations and critical infrastructure against nation-state cyber threats. Founded in January 2023 by Shalev Hulio, Sebastian Kurz and Gil Dolev, the Tel Aviv-based company reached unicorn status, raising about USD 155M total at a valuation above USD 1B.

Magic

San Francisco, United States

Magic is an AI company developing frontier-scale code models designed to automate software engineering and AI research. Their core technology focuses on ultra-long context language models, leveraging 8,000 H100s to improve model performance and address AI alignment challenges. Targeting the advanced AI research community and software development organizations, Magic aims to accelerate progress towards safe Artificial General Intelligence (AGI) through automated code generation and model improvement.

Goodfire

San Francisco, United States

Goodfire is an AI research lab developing techniques for AI interpretability and causal mechanism reverse-engineering within neural networks. Their core technology focuses on uncovering hidden representations within AI models to improve understanding and debugging, moving beyond empirical training methods. Notably, Goodfire recently applied this technology to identify novel biomarkers for Alzheimer’s detection using an epigenetic foundation model, demonstrating early success in scientific discovery.

WhyLabs

Seattle, United States

WhyLabs was a US-based company that provided AI observability and data monitoring solutions for machine learning systems. Their core offering focused on detecting and preventing issues related to data quality, model performance, and responsible AI practices. While the company has ceased operations, WhyLabs contributed to the field through open-source initiatives and aimed to serve organizations prioritizing the reliability and ethical deployment of AI.

Cleanlab

San Francisco, United States

Cleanlab provides a post-hoc AI safety layer that identifies and remediates incorrect or unsafe outputs from any deployed AI agent, including those powered by large language models. Their core technology utilizes a proprietary confidence-based approach to detect label errors and predict potentially harmful responses without requiring retraining or modifications to existing AI infrastructure. Cleanlab targets enterprises prioritizing AI safety, compliance, and trustworthiness, offering a deployable solution for maintaining quality control over generative AI and other AI-driven applications.

Virtue AI

San Francisco, United States

Virtue AI builds an AI-native security and compliance platform for generative-AI and agentic applications, providing red-teaming, guardrails and governance to address safety and security risks. It was founded by AI-safety researchers including Bo Li, Dawn Song, Carlos Guestrin and Sanmi Koyejo.

Unitary

London, United Kingdom

Unitary delivers AI-powered automation for customer and marketplace operations through its Virtual Agent technology, which directly logs into existing tools and follows pre-defined processes without requiring API integrations. These Virtual Agents combine computer vision and complex reasoning capabilities to handle tasks like content moderation and policy enforcement, escalating to human experts when necessary to guarantee accuracy. Unitary focuses on rapidly deploying automation at scale for businesses seeking to reduce operational costs and improve customer satisfaction, offering a results-based pricing model with no upfront fees.

Conjecture

London, United Kingdom

Conjecture is a UK-based AI research company developing a novel AI architecture focused on verifiable safety and control. Their core technology centers on building AI systems with inherent transparency and predictability, moving beyond traditional black-box approaches. This positions Conjecture to serve organizations and researchers prioritizing robust AI safety and alignment, particularly as advanced AI capabilities continue to develop.

Straiker

Sunnyvale, United States

Straiker is an AI-native security company that protects enterprise AI applications and autonomous agents, offering adversarial attack simulation (Ascend AI) and runtime guardrails (Defend AI) against prompt-level and agentic threats. It launched from stealth in 2025 with backing from Lightspeed and Bain Capital Ventures.

IdentifAI

Cesena, Italy

IdentifAI is an Italian cybersecurity company that develops AI models to detect deepfakes and AI-generated content, analyzing images, video and audio to verify authenticity for organizations and platforms. Its 'de-generative' approach helps counter misinformation and synthetic-media fraud.

Saidot

Helsinki, Finland

Saidot is a Finnish SaaS provider specializing in AI governance solutions. Their platform enables organizations to document, assess, and monitor AI systems to ensure responsible development and adherence to the forthcoming EU AI Act. Saidot targets businesses and public sector entities requiring demonstrable AI compliance and risk management capabilities within the European regulatory landscape.

Mila

Montreal, Canada

Mila, founded in 1993 by Yoshua Bengio, is a leading Quebec-based research institute specializing in deep learning and generative models, with a particular focus on AI safety and responsible development. The institute coordinates the work of over 140 affiliated professors across multiple universities – including Université de Montréal, McGill, and Polytechnique Montréal – conducting research in areas like natural language processing and machine learning theory. Mila actively translates research into real-world impact through initiatives like the AI Policy Frontline and the TRAIL program, aiming to address critical challenges at the intersection of AI and public policy.

Redwood Research

Berkeley, United States

Redwood Research is a US-based AI safety nonprofit focused on mitigating catastrophic risks from advanced AI systems. Their core research centers on “AI control” protocols – techniques for reliably monitoring and preventing subversion by potentially deceptive large language models, even when those models intentionally conceal misaligned intentions. They serve as a critical resource for both governments and leading AI developers like Google DeepMind and Anthropic, providing expertise and methodologies for assessing and mitigating AI safety risks.

Apollo Research

London, United Kingdom

Apollo Research is a UK-based AI safety and alignment company specializing in the detection of deceptive behavior in advanced AI systems, particularly large language model agents. They develop and implement novel AI model evaluations focused on “scheming” – covertly pursuing misaligned objectives – and provide technical expertise to governments and international organizations on AI governance and regulation. Their core offering is third-party evaluation of frontier AI models, alongside consultancy services for responsible AI development frameworks and policy guidance.

METR

Berkeley, United States

METR (Model Evaluation and Threat Research) is an independent nonprofit that evaluates whether frontier AI systems can pose catastrophic risks, building methods to measure dangerous autonomous capabilities. It began as the evaluations team at the Alignment Research Center (ARC Evals) and spun out as an independent organization in 2023.

Center for AI Safety

San Francisco, United States

The Center for AI Safety is a US-based nonprofit focused on mitigating potentially catastrophic risks from advanced artificial intelligence. They conduct and fund research into AI safety, with a particular emphasis on identifying and addressing vulnerabilities in increasingly capable AI systems – offering resources like a dedicated compute cluster to support this work. Their primary audience includes AI researchers, policymakers, and stakeholders concerned with the responsible development and deployment of powerful AI technologies.

CoreTrustLink

San Francisco, United States

CTL delivers an audit-ready accountability layer for AI-automated enterprise workflows, providing evidence of approvals and decisions.

Holistic AI

London, United Kingdom

Holistic AI provides a comprehensive AI governance platform focused on risk management and responsible AI deployment. Their platform offers capabilities including AI inventory discovery, risk assessments, and continuous monitoring, specifically supporting generative AI applications alongside broader AI initiatives. Recognized by both Gartner (Cool Vendor in AI Security, GenAI-Enabling Platforms) and IDC (included in the 2025 ProductScape for Generative AI Governance), Holistic AI targets enterprises seeking to confidently scale AI adoption while maintaining compliance and mitigating potential risks.

Optimus Ride

Boston, United States

Optimus Ride develops Level 4 autonomous vehicle technology specifically for low-speed, geo-fenced environments. Their core product is a full-stack autonomous shuttle solution, integrating hardware and software for first/last mile transportation. The company targets business campuses, residential communities, and tourist destinations seeking to deploy safe and efficient, on-demand mobility services within defined operational design domains.

Hyperloop TT

Los Angeles, United States

HyperloopTT is developing a full-scale hyperloop transportation system designed to move passengers and freight at airplane speeds. Their core technology centers on magnetic levitation within a near-vacuum tube, optimized by AI-powered control and safety systems. Targeting both passenger transit and express freight markets, HyperloopTT is currently focused on feasibility studies and pilot projects in regions like Brazil and Italy, aiming to deliver a sustainable and high-speed alternative to traditional transportation.

Encord

London, United Kingdom

Encord provides an AI data-development platform for vision and multimodal teams, covering data management, curation, annotation, and model evaluation, with customers including Philips, Synthesia, and Cedars-Sinai. Raised a $30M Series B in August 2024 led by Next47.

Safe Superintelligence (SSI), founded by OpenAI co-founder Ilya Sutskever with Daniel Gross and Daniel Levy, is a research lab pursuing a single goal: building safe superintelligence. It raised $2B in April 2025 at a $32B valuation despite having no released product.

MIRI

Berkeley, United States

The Machine Intelligence Research Institute (MIRI) is a research nonprofit studying the mathematical underpinnings of intelligent behavior to ensure AI systems are safe and beneficial.

Giskard AI

Paris, France

Giskard AI develops an automated red teaming platform focused on identifying security and quality vulnerabilities in AI systems, particularly Large Language Models. Their core technology utilizes continuous, domain-specific attack scenario generation to proactively test and improve AI agent robustness before deployment. Targeting organizations building and deploying AI applications, Giskard emphasizes data privacy and compliance with regulations like GDPR, SOC 2, and HIPAA, offering both EU and US data residency options.

Galileo

San Francisco, United States

Galileo provides an AI observability and evaluation platform focused on generative AI applications and agents. Their core technology centers on transforming offline AI evaluations into scalable, production-level guardrails using a combination of data capture, metric optimization, and “Luna” monitoring models. Targeting enterprise AI teams, Galileo aims to proactively prevent failures and improve the reliability of deployed GenAI systems at scale.

Patronus AI

San Francisco, United States

Patronus AI builds simulation infrastructure and conducts research focused on achieving human-aligned Artificial General Intelligence (AGI). Their core technology is Digital World Models – AI systems that predict and simulate agent behavior within digital environments – used to generate training data for frontier AI models. Founded in 2023, the company targets the advanced AI research community with a focus on enabling continual learning and scalable simulation for AI safety and development.

Gray Swan AI

New York, United States

Gray Swan AI develops cybersecurity solutions specifically for Large Language Models (LLMs), offering both adversarial evaluation (“Shade”) and runtime protection (“Cygnal”) software. Their core technology leverages LLM-based adversarial agents and a continuous red-teaming process to identify and mitigate vulnerabilities in AI systems. Gray Swan targets enterprise AI deployments and is notably used by leading AI research labs for model security assessment.

Distributional

San Francisco, United States

Distributional provides an MLOps platform focused on post-deployment AI behavioral analysis. Their core technology centers on analyzing production AI logs to identify and track potentially problematic or unexpected model behavior—effectively offering “debugging” for deployed AI. Targeting data science and ML engineering teams, Distributional launched in 2023 with a focus on improving AI reliability and safety through enhanced observability.

Lasso Security

Tel Aviv, Israel

Lasso Security develops a platform providing runtime security for generative AI applications and models. Their core technology focuses on intent analysis to detect and mitigate risks stemming from non-deterministic AI behavior, addressing vulnerabilities in both custom and foundational models. Targeting enterprises grappling with “Shadow AI,” Lasso aims to provide visibility and control over AI deployments at scale, particularly where traditional security measures fall short.

Noma Security

Tel Aviv, Israel

Noma Security develops a platform for discovering, governing, and protecting AI and LLM-powered agents within enterprise environments. Their core technology provides real-time monitoring of AI communication – prompts, responses, and tool calls – enforcing security, privacy, and compliance policies at runtime. Recently valued at over $100M following a Series B funding round, Noma targets organizations deploying and scaling large language models, RAG applications, and AI agents, addressing a critical need for visibility and control over emerging AI risks.

Deepchecks

Tel Aviv, Israel

Deepchecks develops comprehensive validation and monitoring tools for machine learning models throughout the entire MLOps lifecycle. Their core technology centers on data-centric AI testing, utilizing algorithms to automatically detect anomalies, regressions, and potential failure points in model data and performance. Targeting data science and ML engineering teams, Deepchecks has gained traction with enterprise customers seeking to improve model reliability and reduce risks associated with deployed AI systems.

Transluce is an AI research lab focused on enhancing the transparency and safety of large language models. Their core work centers on developing “scalable end-to-end interpretability assistants” – AI tools designed to explain the reasoning processes within complex AI systems. Founded in 2024, Transluce currently operates as an independent research entity with a public interest focus, aiming to contribute to responsible AI development.

Prompt Security

Tel Aviv, Israel

Prompt Security develops a platform for identifying and mitigating security vulnerabilities within Large Language Model (LLM)-based applications. Their core technology focuses on automated analysis of prompts and LLM responses to detect and remediate risks like prompt injection and data leakage. Founded by members of the OWASP research team, Prompt Security targets organizations deploying and utilizing generative AI, providing a specialized cybersecurity solution for this emerging threat landscape.

Pillar Security

Tel Aviv, Israel

Pillar Security develops a security platform focused on the emerging challenges of AI agents and workflows within enterprise environments. Their core technology provides discovery, governance, and runtime security for these “agentic” systems, addressing potential insider threats and compliance risks. Notably, Gartner recognized Pillar Security as a Representative Vendor in their 2026 Market Guide for Guardian Agents, indicating early leadership in this specialized cybersecurity sector.

Aim Security

Tel Aviv, Israel

Aim Security develops AI-powered runtime security solutions for cloud-native applications. Their core technology utilizes behavioral analysis and anomaly detection to identify and mitigate threats targeting large language model (LLM)-based applications and infrastructure. Founded in 2023, Aim Security targets organizations deploying and scaling generative AI services, addressing a critical gap in LLM security as evidenced by early adoption amongst leading AI infrastructure providers.

Knostic

Reston, United States

Knostic builds a cybersecurity platform focused on governing and protecting organizations’ use of artificial intelligence. Their core technology provides visibility and real-time threat protection across the entire “agentic lifecycle,” specifically targeting risks associated with AI agents, coding assistants, and their related infrastructure. Founded in 2023, Knostic currently serves enterprise clients needing to manage and secure rapidly adopted AI tools and mitigate associated data security and operational risks.

Klaimee

San Francisco, United States

Klaimee provides liability insurance, certification, and financial guarantees purpose-built for AI agents, covering failure modes that traditional E&O and cyber policies carve out. It scores agents across eight risk dimensions, issues a certification report, and backs certified agents with a guarantee. Y Combinator-backed.

UMNAI

Birkirkara, Malta

UMNAI is a Maltese deep-tech company that has developed a neuro-symbolic, explainable AI architecture it calls Hybrid Intelligence, combining machine learning with symbolic reasoning and causal inference. The technology produces interpretable, auditable white-box models aimed at high-stakes, regulated decisions in sectors such as financial services. Its methods are protected by more than 20 patents.