Sherlocks.ai is your SRE teammate, handling alerts, conducting RCAs, and planning long-term stability projects. We aim to resolve incidents faster and prevent future outages. Our founding team has deep expertise in scaling startups and AI-driven ventures.
Benchmarking existing agents: Compare Sherlocks’ current agents with other available models, including open-source alternatives, to understand their strengths and gaps.
Diagnosing issues: Identify which agents lead to suboptimal responses, analyze why those failures happen, and form hypotheses about patterns and root causes.
Running quick experiments: Test fixes or improvements on sample cases to validate hypotheses and enhance agent performance.
Extending agent capabilities: Assist in implementing new or improved agents aimed at boosting the accuracy and efficiency of Sherlocks’ incident management.
Exploring new tools and solutions: Evaluate emerging tools and platforms, and run small proofs of concept to gradually offload or enhance in-house systems with scalable and enterprise-grade alternatives
Python
LangChain or LLM-Ops experience
Prompt and eval design