Concept #1 : Kitchen Plague Detector

Plague classification Neural Network running on stick GPU getting frames from camera in real time

One thing I know for sure… in my next life I’m going to be either a mouse, a rat or a roach. The reason I know that for sure is because I’ve killed so many since I’ve moved to Brooklyn that I have perfected my technique in ways I almost want to write a handbook about: Mice have really good hearing but they don’t have good eyes. If you stay still long enough they’ll start coming out from the corners. Or Rats, they are just messy and loud. They’ll just chomp through a box instead of trying to open it. Rats are zero stealth but they are damn fast. They can jump to places you can’t imagine (like to the top of a table) and eat through a wall. Roaches on the other hand, they basically hate humans, they go out when you are not there and pretend they are dead when you look at them.

So, what we know is that plagues: a. follow certain patterns , b. You’ll never see them enjoying themselves because they are always running from you. For every time you don’t see them , they are 100x on your food, on your clothes, on your dishes, etc. Maybe you just don’t want to know and pretend they don’t exist… or, you can use technology to reveal what really is going on =).

Plan of Action

We are going to design an hypothetical gadget that will capture images of the mice, rats or roaches while they are roaming around looking for food or just hanging out in an open space. This gadget would be a home version of the movement triggered cameras that nature photographers use to capture wildlife.

  1. Gadget is placed on floor level pointing to a corner or hole were the animal or insect is suspected to come from. Optionally, it could be pointing to a bait that we have setup for this purpose.
  2. Camera sends frames at a constant rate to the Stick GPU running the program
  3. Frame is input to the Movement detection neural network
  4. If Movement is detected, Frame is input to the Plague detection neural network
  5. If a rat, mouse or roach is detected, image is stored in Flash Drive.
  6. Once every hour, images are posted to IFTTT to a rule that send them to your email.

Further improvements

  • Have the program remember if is there have been recent events and have it in state of alert
  • Infer the presence of the Rat,Mouse or Roach from multiple frames (instead of one frame).
  • Use a microphone to cross reference sounds with images.

Design Notes

+ Even though a GPU Stick is the best option, a Raspberry Pi 3 could be capable to do the job. I’ve successfully run a face detection NN, a face recognition NN and a QR Code reader NN on a Raspberry Pi 3 in other projects.

+ No NN training is assumed.

+ I’ve never seen a Plague detection NN out there, it is just hypothetical. There is a big possibility that it is not even possible to build because of light conditions and size and speed of mice and rats.

+ The Internet Connection to IFTT is entirely optional. The user could retrieve the images from the SD Card directly.


This idea follows the Zombie Home Manifest

An hypothetical component library for Smart Homes as a vocabulary of complex ideas.

There is a very curious thing that happens when you are trying to learn a new (spoken) language, you run out of words pretty fast. When I was just learning english I wanted to use the work “good” for practically everything. Food was “good”, a movie was “good”, my seat was “good” and generally speaking life was “good”. People would definitely understand me but I felt kind of trapped in this one dimensional linguistic space. Problem is, that ideas are everything but one dimension.

Transplanting that concept to Smart Home, I was originally planning to write this blog under the constraints that every proposed idea was to be built with existing plug and play components available in the market. However… I found myself trying to work around the limited catalogue of existing components and basically coming up with one dimensional ideas. On the other hand, I could put my hat of electronic engineer, grab my multimeter and my soldering iron and build them myself with resistors, micro controllers, etc, however that would be like cheating, and I would be violating the principle of “The skill to setup a zombie home should be equivalent to the skills to connect any other home appliance.” .

For that reason, I’m going to propose here a list of elements that will fill an hypothetical box of components that I’ll use in every project.

Assume that each one of these components can just talk to each other. A way to visualize their communication is to see them as actors in a play and they are following a script. Let’s say I want to create a gadget that identifies bad smells in the kitchen.

The play would go like this:

Every five minutes, Sarah “The Controller” listens to Joe “The Air Quality sensor” yelling out loud his readings. When Sarah hears that the air quality is not good, she suggests Bill “The Fan” to start and run until further notice. In the meantime, Sarah suggests Mary “The color cube” to turn red. In addition to that, she writes in a notebook (a flash memory) what just happened and goes to sleep. Five minutes after, Sarah wakes up and pays attention to what Joe (the Air Quality guy) is saying again and figures out if something has changed. It the readings look better now, she suggests Bill (the fan) to stop and Mary (the color cube) to turn green. Sarah goes to sleep again. She repeats this action every 5 minutes.

You might be thinking this is stupid and childish… maybe =), but I’ve designed quite a couple software components in my life and realized early on that when you turn all the components into people, your social skills kick in and you start seeing what everybody is going and not going to do, who has enough information, who are you using a lot, who is being ignored, etc. The result is a much more well though and rounded up piece of software. Just for the sake of simplicity, I’m going to use this paradigm to explain the algorithms that run each one of the gadgets. You just need to assume that such algorithms (or plays) are already loaded in the gadget or in the controller so you don’t need to program them.

Conclusion: I just established a series of hypothetical components that by act of magic just talk to each other as if they were just people. Instead of software we’ll use some sort of play script that describes their interaction.

We can do better at Smart Home…

Today’s (2019) Smart Home offering is very politically correct, every gadget needs to look as if it was “Designed in California”. It needs to blend perfectly with your furniture and it must offer “analytics” not to mention an App in you smartphone.

I get it’s super exciting to see the temperature sway in your living room in real time while you are bored at the office and that the reported savings your smart appliance provided with its smart recommendations (which are partially true) will be a great conversation point, I know you’ll feel good about that light automatically changing its color when you come home after spending more on that bulb than you’d like to admit but… seriously, we can do better than that.

The current Problem : The common assumption is that throwing some sensors and actuators plus a couple of “ideas” to users, we’ll come up with awesome use cases and by doing so the revolution of smart homes will be bootstrapped.

In theory it is not a bad idea, I’m sure other industries have pulled a similar trick, the issue is that it is marketed as a plug and play solution. A real plug and play looks like a Gaming console that you connect and start playing with immediately after. I know a lot of people that have invested in smart gateways/lights or sensors just to open the box once and then abandon it because there is really no path towards solving a particular problem. To put it in perspective, imagine that IKEA sold exclusively a box with all the joints and screws and legs to assemble “something” for your house. I guess you could build the chair of your dreams but that is far from a straight path to solve a problem that you might have today and right now.

My proposal is to focus on problems or at least interesting projects first, technology second. As an engineer this is kind of saying that I’m going to the backseat. As a product designer I say: let’s come up with one hundred awesome ideas that demonstrate that you can have awesome home gadgets (or I’ll eat my hat). And just then, let’s start talking about implementation. If future doesn’t come, let’s bring it in =).

Zombie Home Manifest (and why I’m writing this blog)

Post apocalyptic movies depict a world were there is no internet, no electric grid, water is not always safe to drink and even more trivial aspects like lighting or communication can never be taken for granted.

You don’t need to go to the future or wait for the zombies to come, just go to a rural area around where you live and you’ll realize we are designing smart home technology for a very specific set of assumptions: Permanent connection to the internet, a smartphone and un-interrupted electric service.

I’m not proposing to cut the cord from the internet, go back to punchcards or operate everything on solar (although that would be super cool). Instead, I want to turn those “Must haves” into “Nice to haves”. Instead of trying to connect that lamp to the internet, let’s focus on things that are more important like: Reusability, Stand Alone capabilities, Simplicity and Cost. The how to get there is the Mission of this Blog.

Zombie Home Manifest

  • A Zombie Home uses technology to solve real problems, automate boring tasks and provide useful insight but:
  • A Zombie Home should not depend on smart phones to be operated.
  • A Zombie Home should not depend on Cloud Services or permanent Internet connection to work.
  • A Zombie Home should not depend on an electric grid to work.
  • The technology that makes a Zombie Home gadget work should be 100% salvageable to be used in another gadget.
  • The skill to setup a zombie home should be equivalent to the skills to connect any other home appliance.
  • Zombie gadgets should take advantage of old and latest technology without committing to any in particular.
  • But more important than anything: It should be affordable. Not only affluent people have problems that can be solved in a smart way with technology.