Hydra (YC W22) – Query any database via Postgres

Hi HN, we’re Joe and JD from Hydra (https://hydras.io/). Hydra is a Postgres extension that intelligently routes queries through Postgres to other databases. Engineers query regular Postgres, and Hydra extends a Postgres-compliant SQL layer to non-relational, columnar, and graph DBs. It currently works with Postgres and Snowflake, and we have a roadmap to support MongoDB, Google BigQuery, and ClickHouse.

Different databases are good at different things. For example, Postgres is good at low-latency transactional workloads, but slow when running analytical queries. For the latter, you're better off with a columnar database like Snowflake. The problem is that for each new database added to a system, application complexity increases quickly.

Working at Microsoft Azure, I saw many companies juggle database trade-offs in complex architectures. When organizations adopted new databases, engineers were forced to rewrite application code to support the new database or use multiple apps to offset database performance tradeoffs. All this is expensive busy work that frustrates engineers. Adopting new databases is hard and expensive.

Hydra automatically picks the right DB for the right task and pushes down computation, meaning each query will get routed to where it can be executed the fastest. We’ve seen results return 100X faster when executing to the right database.

We've chosen to integrate with Snowflake first so that developers can easily gain the analytical performance of Snowflake through a simple Postgres interface. To an application, Hydra looks like a single database that can handle both transactions and analytics. As soon as transactions are committed in Postgres, they are accessible for analytics in real-time. Combining the strengths of Postgres and Snowflake in this way results in what is sometimes called HTAP: Hybrid Transactional-Analytical Processing (https://en.wikipedia.org/wiki/Hybrid_transactional/analytica...), which is the convergence of OLTP and OLAP.

Existing solutions are manual and require communicating with each datastore separately. The common alternative is trying to combine all of your data together into a data warehouse via ETL. That works well for analysts and data scientists, but isn't transactional and can't be used to power responsive applications. With Hydra engineers can write unified applications to cover workloads that had to be separate before.

Hydra runs as a Postgres extension, which gives it the ability to use Postgres internals and modify execution of queries. Hydra intercepts queries in real-time and routes queries based on query type, user settings, and Postgres' cost analysis. Writes and operational reads go to Postgres, analytical workloads go to Snowflake.

Recently committed transactions are moved from Postgres to Snowflake in near real-time using Hydra Bridge, our built-in data pipeline that links databases from within Postgres. The bridge is an important part of what we do. Without Hydra, workloads are typically isolated between different databases, requiring engineers to implement slow and costly ETL processes. Complex analytics are often run on older data, updated monthly or weekly. The Hydra bridge allows for real-time data movement, enabling analytics to be run on fresh data.

We make money by charging for Hydra Postgres, which is a Postgres managed service, and Hydra Instance, which attaches Hydra to your existing Postgres database. Pricing is listed on the product pages: https://hydras.io/products/postgres and https://hydras.io/products/instance.

A little about our backgrounds: Joseph Sciarrino - Former PM @ MSFT Azure Open-Source Databases team. Heroku (W08) and Citus Data (S11) alum. Jonathan Dance - Director @ Heroku (2011-2021)

Using Hydra you can create a database cluster of your own design. We’d love to know what Hydra clusters you’d be interested in creating. For example, Elasticsearch + Postgres, BigQuery + SingleStore + Postgres, etc. Remember - You can experiment different combinations without rewriting queries, since Hydra extends Postgres over these other databases. When you think about databases like interoperable parts you can get super creative!

Get Top 5 Posts of the Week

best of all time best of today best of yesterday best of this week best of this month best of last month best of this year best of 2023 best of 2022 yc w24 yc s23 yc w23 yc s22 yc w22 yc s21 yc w21 yc s20 yc w20 yc s19 yc w19 yc s18 yc w18 yc all-time 3d algorithms animation android [ai] artificial-intelligence api augmented-reality big data bitcoin blockchain book bootstrap bot css c chart chess chrome extension cli command line compiler crypto covid-19 cryptography data deep learning elexir ether excel framework game git go html ios iphone java js javascript jobs kubernetes learn linux lisp mac machine-learning most successful neural net nft node optimisation parser performance privacy python raspberry pi react retro review my ruby rust saas scraper security sql tensor flow terminal travel virtual reality visualisation vue windows web3 young talents

andrey azimov by Andrey Azimov