When I (Shimon) was the manager of a 400-developer company's infrastructure engineering team, we had an issue where a developer committed AWS secret keys into a public GitHub repo. We were very, very lucky that the bad actors who quickly got ahold of the keys "only" spun up compute instances to mine bitcoin.
Mistakes happen and they happen to the best of us. No developer wants to make mistakes, especially ones impacting production. Those mistakes can be not only costly to the business, but emotionally painful for the developer.
After finding out about the issue, I had to search for any other leaked secret in our repositories to make sure we were no longer exposed. The next thing that I had to do was to take steps to help folks avoid making this mistake again.
It's easy to create a policy that says "do not commit secrets to GitHub" (which was what I did) but in reality, this is much harder to implement. I would do things like sending a mass email to all of Engineering and having code reviewers check for it manually during code reviews. Problem is, these approaches don't work consistently—if at all.
The bigger the engineering team—and the faster it ships software—the bigger this problem becomes. Also, developers today operate more independently and have broader responsibilities; they are responsible for not just writing code, but also testing, and deployment to production. You might expect that developers would follow best practices, standards, and policies, but of course, in practice, these things fall through the cracks. That's why we built Datree.
What we built is a rules engine, which is essentially a server-side git-hook platform. We connect it to the organization’s source control, scan the layout of the repository, parse all structured files like YAML / JSON / XML / Dockerfile, and build a catalog with the organization’s metadata—such as packages used, container images, and all the properties in the structured files.
The engine performs an automatic check each time code is committed to GitHub. This happens before the code can be merged to master. It runs just like your CI tests. It checks if the rules you've set are followed—and tells the developer when they aren't and how to fix it, but not like your CI configuration, Datree is running on the org level so you can apply any rule on all of your repositories in just one click.
You may be asking “is this another static code analysis tool?” We see Datree as completing or complementing those tools, not competing with them. We’re seeing our customers create a rule with Datree to check and verify that static code analysis step is integrated and executed as part of their CI flow, instead of going over each CI config file in their repositories and updating it manually.
Rules could be anything: development best practices, lessons learned from post-mortems, security policies, or compliance standards. For example, a very popular rule is to prevent secrets from being merged into the master branch. Leaking secrets to source control is a common and potentially costly mistake (see https://news.ycombinator.com/item?id=19825202).
Often people ask us, “what rules should we adopt?” Because of this, we started curating industry best practices and turning them into rules they can simply enable when they use our product. Datree now comes with more than dozens of rules packs for all kinds of popular technologies (like Docker and serverless), languages and frameworks, tools (like GitHub and Travis CI), and even use cases (like SOC 2 compliance). Of course, you are free to create your own custom rules.
To date, Datree has run 100,000+ checks for Engineering teams large and small, including Microsoft, Globalgiving, Cybereason, and Gigster (YC S15, 400+ engineers).
We’re sure many HN members will have encountered similar problems and/or have expertise in this area. We’d love to hear from you: How do you ensure the adoption of development best practices for your team? What works and doesn’t? Thank you!
by Andrey
Azimov