About the Challenge We're Tackling: As enterprises integrate LLMs into their existing applications, traditional observability tools fall short in addressing the unique safety and operational risks posed by LLM interactions. These tools are adept at monitoring conventional metrics like rate limits, latency, and cost breakdowns but lack the capacity to assess the stochastic risks inherent in LLM inputs, outputs, and inter-LLM communications. This gap represents the primary barrier to confidently deploying LLMs in enterprise settings. At ThirdLaw, we empower IT and Security teams with the tools to answer the foundational question; "Is this OK?" and take decisive action when it isn't. We provide the next-generation monitoring solutions necessary to evaluate, investigate, and mitigate the unique risks associated with LLM deployments. About the role: AI is reshaping software development, enterprise knowledge management, and the way work gets done. By giving IT and Security professionals the tools to make sure AI is doing everything it should, and nothing it shouldn’t, you’ll be enabling the safest path to a wave of incredible AI-powered innovation. In this role, you will develop software that supports continuous LLM evaluation and control. You will develop infrastructure needed to serve and scale AI-powered applications, emphasizing the integration of AI and ML algorithms into real-time products. What you’ll be doing: Architecting a system that enables low-latency evaluation of AI model and agent inputs and responses, seeking to measure specific risks. Expand and improve our existing data collection and integration frameworks Building novel query and investigation experiences within the ThirdLaw platform. Skills and Qualities you’ll need to bring Experience building data pipelines or distributed systems on top of multiple storage tiers Expertise in Python and/or Golang. Comfortable with Linux and Docker, serverless architectures. Interest and willingness to learn concepts in artificial intelligence, machine learning, and deep neural networks. You are excited about the possibilities of LLMs . Good interpersonal communication skills, both verbal and written Clear ability to own features and products from start to finish Nice-to-have: Ideally, you live in the bay area or want to be here enough to collaborate in person sometimes, but we are able to work with anyone in the continental United States. Join us as we pursue our mission to unlock the boundless possibilities of generative AI by ensuring AI trust and safety. We're looking for people who bring thoughtful ideas and aren't afraid to challenge the norm. Our team is small and focused, valuing autonomy and real impact over titles and management. We need strong technical skills, a proactive mindset, and clear written communication, as much of our work is asynchronous. Our product is new and operates in a rapidly changing ecosystem of generative AI; we are builders with the ability to dispatch ambiguity to solve customer pain. If you're organized, take initiative, and want to work closely with customers to shape our products, you'll fit in well here.