Helicone's core technology is a proxy that routes all your OpenAI requests through our edge-deployed Cloudflare Workers. These workers are incredibly reliable and cause no discernible latency impact in production environments. As a proxy, we offer more than just observability: we provide caching and prompt formatting, and we'll soon add user rate limiting and model provider back off to make sure your app is still up when OpenAI is down.
Our web application then provides insights into key metrics, such as which users are disproportionately driving costs and what is the token usage broken down by prompts. You can filter this data based on custom logic and export it to other destinations.
Getting started with Helicone is quick and easy, regardless of the OpenAI SDK you use. Our proxy-based solution does not require a third party package—simply change your request's base URL from https://api.openai.com/v1 to https://oai.hconeai.com/v1. Helicone can be integrated with LangChain, LLama Index, and all other OpenAI native libraries. (https://docs.helicone.ai/quickstart/integrate-in-one-line-of...)
We have exciting new features coming up, one of which is an API to log user feedback. For instance, if you're developing a tool like GitHub Copilot, you can log when a user accepted or rejected a suggestion. Helicone will then aggregate your result quality into metrics and make finetuning suggestions for when you can save costs or improve performance.
Before launching Helicone, we developed several projects with GPT-3, including airapbattle.com, tabletalk.ai, and dreamsubmarine.com. For each project, we used a beta version of Helicone which gave us instant visibility into user engagement and result quality issues. As we talked to more builders and companies, we realized they were spending too much time building in-house solutions like this and that existing analytics products were not tailored to inference endpoints like GPT-3.
Helicone is developed under the Common Clause V1.0 w/ Apache 2.0 license so that you can use Helicone within your own infrastructure. If you do not want to self-host, we provide a hosted solution with 1k requests free per month to try our product. If you exceed that we offer a paid subscription as well, and you can view our pricing at https://www.helicone.ai/pricing.
We're thrilled to introduce Helicone to the HackerNews community and would love to hear your thoughts, ideas, and experiences related to LLM logging and analytics. We're eager to engage in meaningful discussions, so please don't hesitate to share your insights and feedback with us!
by Andrey
Azimov