People use security cameras (aka IP cams) for various purposes—to monitor their properties, their kids, their pets, for elderly care, as doorbells, etc. But a shortcoming these cameras have is they rely mainly on motion detection to alert users and that leads to too many false alarms.
What led me to work on this problem initially was my personal experience with the IP cameras which I used to watch my dog and also as a doorbell at my house which I rented out on Airbnb sometimes. After trying some of these cameras (Ring, Nest and Wyze), I realized motion alerts are pretty much useless and person detection that some like Nest offer is not broadly useful. For example, for my dog, I only cared to know if/when the dog walker picked her up or if she was doing something bad, like getting into my clothes, chewing my shoes/TV remote, getting on the bed, etc. The motion alerts were completely useless as she is moving all the time obviously--person detection was also not useful for any of these events. For my Airbnb rental use case (doorbell/outdoor cams), the main things I cared to know about were if the guests parked their cars in the wrong location which pissed off my neighbors, or if the garage door was left open, or if there were a lot more people staying at the house than allowed. Again, motion alerts or person detection were not useful at all.
Having a background in machine learning & computer vision, I felt this is a problem that is just starting to become solvable thanks to the powerful deep learning techniques developed in the the last 3-4 years.
Over the last 6 months, we have been building a cloud-based solution addressing this shortcoming for any IP camera without any dependency on the hardware. Our software allows users to create custom alerts for things that matter to them, like their dog chewing on shoes, their kid playing with the stove or their packages being stolen by porch pirates. It also allows them to search for past events after the fact instantly.
Currently, we support four categories of events: - A specific object appeared / disappeared, e.g. dog appeared, bicycle disappeared, package disappeared (coming soon.) - A specific object in a specific location, e.g. a car parked in front of the driveway, elderly person taking medications, dog in the (neighbor’s) lawn, person getting into the garage. - Two objects interacting, e.g. dog getting on the couch, kid playing with the stove, dog chewing on a shoe. - Facial recognition based events, e.g. new person detected, a specific person appeared, max occupancy violated.
Users can create a new event in any of the above categories by providing a few simple inputs, e.g. pick the objects involved and the interaction between them, or specify a zone. Once the event is created, our software can immediately recognize that event with good accuracy. The users can also give a thumbs up/down when they get an alert and their feedback is incorporated back into the models to improve their accuracy over time. Users can adjust the sensitivity for each event (precision and recall trade-off) based on their use case.
In addition to the smart alerts described above, we also index the footage in real time to allow users to query for past events after the fact and get the results instantly instead of having to go through all the past footage to find something they care about. For example: users can query the clips of when a laptop disappeared or a truck appeared.
Our solution can also alleviate privacy concerns since we only store short video clips on the cloud for alerts corresponding to user’s events of interest instead of for every motion detected.
We currently support Nest Cams and also offer our own cameras (same as the cameras sold by Wyze) with indoor and outdoor options.
I would love to hear any feedback/thoughts you have. We are exploring different niche use cases to focus on initially and would appreciate any thoughts you may have based on your personal experience or any insights you may have. Feel free to comment here or shoot me an email at [email protected]
by Andrey
Azimov