Would it be possible to make an object detector that works on any object?

It seems reasonable that an intelligent device should be able to automatically recognize the presence of an object. Perhaps not immediately knowing what the object is, but if it can instantly recognize a new object, the task of training becomes much faster. Well, thanks to the Open Source community, you can do just that! Check out the simple process here or just have a look at it running on an OAK camera below:

Detect objects never seen before

Normally, you train on specific object classes: bananas, apples, raccoons, etc. But what if you don't care what an object is, just that it's an object? When we put out that we were investigating the idea, the community responded when PINTO0309 and ibaiGorordo shared their solutions just a few days after our initial inquiry. 

What’s even better is that they used an image of an OAK-D-Lite as a test sample! That’s right, they took an OAK-D-Lite, created a class-agnostic object detector, and it successfully identified an OAK-D-Lite in an image as an object even though it had never seen one before. They were even kind enough to send us a screenshot of the object detection. Check it out! You can also check out the impressive number of models that PINTO0309 has found and done the hard work of converting here.

And it works on all sorts of things

Also, a big thanks to IbaiGorordo for sharing his experiments with this model, here.  We’ve picked out a few examples below so you can see just how diverse the applications can be.

A cool stop motion series! (Original video by Animist: https://youtu.be/uKyoV0uG9rQ)

A capybara! (Original image: https://commons.wikimedia.org/wiki/File:Capybara_portrait.jpg)

Coins! (Original image: https://commons.wikimedia.org/wiki/File:Japanese_Coins.jpg)

Shoes! (Original image: https://commons.wikimedia.org/wiki/File:Japanese_Coins.jpg)

SpaceX Crew Dragon! (Original image: https://en.wikipedia.org/wiki/Spacecraft#/media/File:SpaceX_Crew_Dragon_(More_cropped).jpg)

A window! (Original image: https://commons.wikimedia.org/wiki/File:Window_-_Paddington_-_London.jpg)

Why Is class agnostic object detection useful? Model re-training.

Class agnostic object detection solves one of the classic chicken-and-egg problems of deploying custom-trained models in the wild: how does my model know what to recognize if it’s never seen it before? Or, from a slightly different perspective: What if my model isn't picking up objects that it should?

This is a painful problem, particularly if you are deploying on the edge, where you probably don't want to be sending back tons of data to the cloud to check. Having a generic object detector is a hugely beneficial piece to solving it! It makes model training easy.

Here's the process:

1. Run the generic model in parallel with your custom model (remember, on OAK cameras you can do arbitrary series/parallel combinations of neural models).
2. Monitor on-device when the generic model detects things your custom model doesn't.
3. If/when this happens, save/send the cropped results from the generic detector back for analysis to re-train your custom model.

And best of all, since OAK is plug & play with Roboflow, you can automate all of this with their Upload API. This allows you to get feedback on all the objects your model isn't detecting, have them uploaded automatically so you can label the ones that should be detected, and retrain your model over time to continuously get smarter. Brilliant.