Irma > Prototype Zero

Irma is a Battery Powered, NFC provisioned, wirelessly charged BLE Mesh Network with I2C peripherals.

The idea behind Irma is to lay a low power Mesh Network capable of communicating peripherals wirelessly with a central (but not necessarily unique) main function via sub/pub. All the Irma blueprints are designed to run on the Irma Architecture.

Dog Feeder revisited

Dr. Brown from Back to the Future was one of the reasons I decided to be an engineer. Not because he invented a time machine (pfff) but because of the machine he created to feed his dog (Einstein). If you haven’t seen that movie, stop reading this and go watch it.

Funny thing is that while some of my other friends in Engineering school wanted to become engineers because they wanted to go to the moon or something as far fetched as building a sentient cyborg, I just wanted to build cool gadgets for myself.

In this Blog Post I present a quick draft of how could we build a dog feeder using technology I’m familiar with: Microcontrollers, sensors, a mesh network and off the shelf mechanical components.

The basic idea is to have a mechanical timer/relay like those used in gardens advertise if it is time to do something or not. The way I would implement this is by reading whether current is passing or not (sensor N). Yes, I would have to translate from 220V to 5V but that’s not a big deal. I could just use a 220v to 5V transformer ( also known as a phone charger) to generate an input signal.

After that, I would check whether there is food in the main container. I was thinking about a sonar (sensor B) , I’ve worked with sonar sensors in the past and they are pretty reliable once you calibrate them. Here, it would know if there’s food based on how long it takes the ping to come back. A value of 2 is equivalent to 20% of the container being empty. Sonars don’t rely on light so I assume it will work with some food dust around.

In addition to checking if there’s food I need to verify whether there is food in the plate or not (sensor I) . I would place a small light sensor on the bottom of the plate. If light is detected, that means there is no food on the plate.

If all looks good I would start the motor that drives a dispenser screw (actuator C) for it to drop some food to the pipes that are already pressurized by the air pump (actuator D). Similar to when you vacuum your own home, it will drive the food kernels through the pressurized pipes all the way to the element F.

A vibration sensor (sensor E) detects whether food is hitting the walls of the pipes. If no vibration is detected that means no food is in the pipes and something must be wrong: Either no pressure or no food. We output an error message to the message board (actuator M) and exit the program. Sorry dogs, no robot serviced food for you today.

Once food makes it to the intersection F, it will go either left or right. Actuator F is just a ramp that moves to one side or the other letting the food fall in Chi Chi’s plate or in Hugo’s plate.

Maybe this is an overkill, but there’s yet another sensor in the pipe that detects whether food is passing through all the way to the plate. This is to confirm that F did its job properly. I could just instead check sensor I to figure out if the plate is filled in.

Because dogs also need water to live, specially after eating that processed food, we check the level of the water using a floating magnet. A magnetic field magnet checks the intensity of the field, infers how close the magnet is (sensor K) and determines the level of the water. If water is needed, the water valve Q is opened for a certain amount of seconds to fill the bottle.

And all this so we can focus on the Flux Capacitor…

Concept #2 : Dog Gamification + Smart Home is

The idea in a nutshell: To install a series of levers, buttons, lights and sounds actuators around the house or apartment for dogs to be entertained while owners are not around. Ideally, the smart home would randomly generate new ways to call the dog’s attention and learn from what is effective or not.

Why would we want this? Let me use my own experience as an example : My dear dog “Chicharron” has destroyed about 15 books, 8 DVDs, a passport, a couple hats , a couch and even a snake plant with his teeth. At first I thought he was just a bad dog, then I got the feeling that he was just hungry so I started feeding him much more. He still chewed on random things all the time (I get he is young and his teeth are growing). We gave him a massive amount of chewable toys, all colors and sized but nothing really changed. I’d like to believe he is just bored. Chewing things seem to be the way he copes with boredom because he never does it when we are around.

Doing some research I found a Swedish Dog games designer called Nina Ottosson, in her website she mentions the following:

 “All dogs need to use their head sometimes in order to feel good, and some dogs have a greater need than others. If they don’t get to channel their energy into an organized activity, they can sometimes create their own “fun”, which is not always appreciated by the owners, such as chewing on things, or just get hyperactive”.

Worth mentioning: Nina Ottosson has a full line of Dog Activity Games that look great. I’m ordering a couple today for Chicharron. However what I’m proposing is a little bit more on the crazy Emmet “Doc” Brown side from “Back to the Future” with electronics, lights and mechanical components. If you remember the opening scene of the first movie, there is a full contraption that feeds “Einsteing” (his dog). I like to imagine that in this alternate and fictional movie universe, the “Doc” would have also invented and developed a way to stimulate “Einstein’s” brain (pun intended).

First of all we need to think about the Dog Interface/Experience (DI/DX):

  • The levers and buttons needs to be rugged enough to withstand chewing.
  • Dog should be able to interact with its Paws, Tongue, Teeth or Snout.
  • Light actuators should be within the dog’s line of sight.
  • Reward dispensers should make it easy for the dog to retrieve its payoff (and communicate explicitly that a reward was dispensed).
  • Sensors should not only include levers and buttons but also microphones (to pick up dog sounds like whining, crying or barking) and cameras for computer vision to be able to recognize bad habits (like taking things from the top of the table or opening the Trash can).
  • The feedback shouldn’t only come from the light actuators but also from any other smart home device (like living room lights, colors, blinds, etc)

Plan of Action

We are going to design an hypothetical system that will give cues to the dog(s) to take an action. The dogs will have to be trained to respond to the cues via the input gadgets (rope pulls, cranks, buttons) for them to acquire a reward (could be a treat or a sound). The system will get feedback (how and when the dog reacted to the cues) via the Microphone, Computer Vision enabled camera and the Movement sensors. A “Dog Actions Classifier” will interpret the feedback inputs and recommend the next cue.

Design Notes

  • The system should NOT be connected to the internet and that is OK. This is not intended for dog owners to play remotely with their dogs… This is literally, the Smart Home playing with the dog and learning from it.
  • Maybe it is not explicitly depicted in the diagram but all the components should communicate wirelessly. This would avoid the problem of dogs chewing on the cables.
  • The only components that need to be dog-ruggedized are the input gadgets : the cord pull, crank buttons, etc
  • Also, not explicitly depicted in the diagram but there is a micro controller that orchestrates all the gadget in an asynchronous way. Every gadget should still work even if that controller is down.
  • The most basic Classifier should be pre-trained, however it would be fascinating to have it train against every dog’s particularities. Also, to randomly come up with new cue-action-reward combos to discover new ways to entertain the dog.
  • The dog will reinforce the system the system will reinforce the dog.

Further Improvements

+ Connect the system to the internet for owners to play remotely with its dog. Although I suspect owners will rapidly loose interest after doing so a couple of times. Hence the importance of having the smart home interact with the dog directly

+ Maybe implement something similar for cats? Do they care?

Top of mind conclusions

The chances of somebody investing more than a $100 in a toy or a system to entertain a dog are practically zero (I usually wouldn’t). Nevertheless, as I’m writing this post, Chicharron just destroyed today the living room rug that costs much more than a $100. Could the case be made that the ROI for such implementation is positive? How do you know what your dog is going to destroy in advance? Also, is this a more humane way to keep dogs from destroying things? (opposed to keep them outside or in a cage). Or, are we creating obsessive compulsive animals that require constant feedback? Are dogs even interested in an interactive experience?

I’ll put this concept (Dog Gamification) in consideration for a prototype build. I might be able to assemble a very primitive version of it with some Arduinos and Raspberry Pis and some cheap sensors that I find online. That will be part 2 : Stay tuned.

This idea follows the Zombie Home Manifest

The lost and failed ancestor of your idea.

Let’s say you have a brand new idea. Let’s be honest, maybe you should just forget about it… Why? Because by definition you are absolutely biased by your own background, assumptions and context. You are to such idea what a mom is to her baby. You won’t see anything wrong with it, you’ll love it and you’ll do anything it takes to demonstrate the world that you are right (even if you aren’t).

As I was doing the laundry last Sunday, I realized that not because I do it every week, the clothes are getting cleaner, they are just the same every week. Same as the Jackpot player that pulls the lever once or a thousand times, the probabilities don’t really increase as it resets every time however subconsciously there is indeed a sense of progress. How many hours have you spent thinking about that awesome idea you had a year ago? How do you know that every day you don’t just reset your vision and replay the same scenarios on how that idea will succeed?

In order to break that mental ping pong you need to run a very simple mental exercise:

“Assume that exactly the same idea was already imagined by somebody smarter that you and with more funding than you… but it failed.”

By doing so, the following happens :

  • You stop day-dreaming that you have the killer, world changing idea that will be sold out the first day and focus in what is important which is your product.
  • It brings you closer to the hard reality that you might fail as well but it also gives you the chance to demonstrate that your execution and tactics are much better.
  • When somebody gives you feedback, you’ll actually listen instead of trying to “correct” them as they could be giving you valuable information that the other guys might have ignored.
  • You become more cold blooded with trivial features as making a series of wrong choices could have been the reason of the other guy’s demise .
  • You stop pretending you can be everything for everybody. You want to live to see the next day even if that involves only serving one segment at a time.

As the saying goes: “It is not the idea, it is the execution what matters”. Execution is the art of smuggling an idea from the Imaginary world to the Real world. The crystallization of something ethereal. We should focus on being the Marco Polo of ideas, you and only you know your silk road.

Every time I meet a film director I ask them the same question: If I gave the same script and budget to 5 different directors… would I get the same movie? The answer is unequivocally “no”.

So, why ideas would be different? Ideas being the equivalent of scripts. I would bet you would not be able to replicate Facebook even if you were sent back in time to that dorm at that moment in history same as you wouldn’t be able to replicate Clockwork Orange by Stanley Kubrick even if you had the exact same crew and script.

That is actually good news. Because you could be the “One” that executes right, that non original idea you just got. : )

Concept #1 : Kitchen Plague Detector

Plague classification Neural Network running on stick GPU getting frames from camera in real time

One thing I know for sure… in my next life I’m going to be either a mouse, a rat or a roach. The reason I know that for sure is because I’ve killed so many since I’ve moved to Brooklyn that I have perfected my technique in ways I almost want to write a handbook about: Mice have really good hearing but they don’t have good eyes. If you stay still long enough they’ll start coming out from the corners. Or Rats, they are just messy and loud. They’ll just chomp through a box instead of trying to open it. Rats are zero stealth but they are damn fast. They can jump to places you can’t imagine (like to the top of a table) and eat through a wall. Roaches on the other hand, they basically hate humans, they go out when you are not there and pretend they are dead when you look at them.

So, what we know is that plagues: a. follow certain patterns , b. You’ll never see them enjoying themselves because they are always running from you. For every time you don’t see them , they are 100x on your food, on your clothes, on your dishes, etc. Maybe you just don’t want to know and pretend they don’t exist… or, you can use technology to reveal what really is going on =).

Plan of Action

We are going to design an hypothetical gadget that will capture images of the mice, rats or roaches while they are roaming around looking for food or just hanging out in an open space. This gadget would be a home version of the movement triggered cameras that nature photographers use to capture wildlife.

  1. Gadget is placed on floor level pointing to a corner or hole were the animal or insect is suspected to come from. Optionally, it could be pointing to a bait that we have setup for this purpose.
  2. Camera sends frames at a constant rate to the Stick GPU running the program
  3. Frame is input to the Movement detection neural network
  4. If Movement is detected, Frame is input to the Plague detection neural network
  5. If a rat, mouse or roach is detected, image is stored in Flash Drive.
  6. Once every hour, images are posted to IFTTT to a rule that send them to your email.

Further improvements

  • Have the program remember if is there have been recent events and have it in state of alert
  • Infer the presence of the Rat,Mouse or Roach from multiple frames (instead of one frame).
  • Use a microphone to cross reference sounds with images.

Design Notes

+ Even though a GPU Stick is the best option, a Raspberry Pi 3 could be capable to do the job. I’ve successfully run a face detection NN, a face recognition NN and a QR Code reader NN on a Raspberry Pi 3 in other projects.

+ No NN training is assumed.

+ I’ve never seen a Plague detection NN out there, it is just hypothetical. There is a big possibility that it is not even possible to build because of light conditions and size and speed of mice and rats.

+ The Internet Connection to IFTT is entirely optional. The user could retrieve the images from the SD Card directly.

.

This idea follows the Zombie Home Manifest

An hypothetical component library for Smart Homes as a vocabulary of complex ideas.

There is a very curious thing that happens when you are trying to learn a new (spoken) language, you run out of words pretty fast. When I was just learning english I wanted to use the work “good” for practically everything. Food was “good”, a movie was “good”, my seat was “good” and generally speaking life was “good”. People would definitely understand me but I felt kind of trapped in this one dimensional linguistic space. Problem is, that ideas are everything but one dimension.

Transplanting that concept to Smart Home, I was originally planning to write this blog under the constraints that every proposed idea was to be built with existing plug and play components available in the market. However… I found myself trying to work around the limited catalogue of existing components and basically coming up with one dimensional ideas. On the other hand, I could put my hat of electronic engineer, grab my multimeter and my soldering iron and build them myself with resistors, micro controllers, etc, however that would be like cheating, and I would be violating the principle of “The skill to setup a zombie home should be equivalent to the skills to connect any other home appliance.” .

For that reason, I’m going to propose here a list of elements that will fill an hypothetical box of components that I’ll use in every project.

Assume that each one of these components can just talk to each other. A way to visualize their communication is to see them as actors in a play and they are following a script. Let’s say I want to create a gadget that identifies bad smells in the kitchen.

The play would go like this:

Every five minutes, Sarah “The Controller” listens to Joe “The Air Quality sensor” yelling out loud his readings. When Sarah hears that the air quality is not good, she suggests Bill “The Fan” to start and run until further notice. In the meantime, Sarah suggests Mary “The color cube” to turn red. In addition to that, she writes in a notebook (a flash memory) what just happened and goes to sleep. Five minutes after, Sarah wakes up and pays attention to what Joe (the Air Quality guy) is saying again and figures out if something has changed. It the readings look better now, she suggests Bill (the fan) to stop and Mary (the color cube) to turn green. Sarah goes to sleep again. She repeats this action every 5 minutes.

You might be thinking this is stupid and childish… maybe =), but I’ve designed quite a couple software components in my life and realized early on that when you turn all the components into people, your social skills kick in and you start seeing what everybody is going and not going to do, who has enough information, who are you using a lot, who is being ignored, etc. The result is a much more well though and rounded up piece of software. Just for the sake of simplicity, I’m going to use this paradigm to explain the algorithms that run each one of the gadgets. You just need to assume that such algorithms (or plays) are already loaded in the gadget or in the controller so you don’t need to program them.

Conclusion: I just established a series of hypothetical components that by act of magic just talk to each other as if they were just people. Instead of software we’ll use some sort of play script that describes their interaction.

We can do better at Smart Home…

Today’s (2019) Smart Home offering is very politically correct, every gadget needs to look as if it was “Designed in California”. It needs to blend perfectly with your furniture and it must offer “analytics” not to mention an App in you smartphone.

I get it’s super exciting to see the temperature sway in your living room in real time while you are bored at the office and that the reported savings your smart appliance provided with its smart recommendations (which are partially true) will be a great conversation point, I know you’ll feel good about that light automatically changing its color when you come home after spending more on that bulb than you’d like to admit but… seriously, we can do better than that.

The current Problem : The common assumption is that throwing some sensors and actuators plus a couple of “ideas” to users, we’ll come up with awesome use cases and by doing so the revolution of smart homes will be bootstrapped.

In theory it is not a bad idea, I’m sure other industries have pulled a similar trick, the issue is that it is marketed as a plug and play solution. A real plug and play looks like a Gaming console that you connect and start playing with immediately after. I know a lot of people that have invested in smart gateways/lights or sensors just to open the box once and then abandon it because there is really no path towards solving a particular problem. To put it in perspective, imagine that IKEA sold exclusively a box with all the joints and screws and legs to assemble “something” for your house. I guess you could build the chair of your dreams but that is far from a straight path to solve a problem that you might have today and right now.

My proposal is to focus on problems or at least interesting projects first, technology second. As an engineer this is kind of saying that I’m going to the backseat. As a product designer I say: let’s come up with one hundred awesome ideas that demonstrate that you can have awesome home gadgets (or I’ll eat my hat). And just then, let’s start talking about implementation. If future doesn’t come, let’s bring it in =).

Zombie Home Manifest (and why I’m writing this blog)

Post apocalyptic movies depict a world were there is no internet, no electric grid, water is not always safe to drink and even more trivial aspects like lighting or communication can never be taken for granted.

You don’t need to go to the future or wait for the zombies to come, just go to a rural area around where you live and you’ll realize we are designing smart home technology for a very specific set of assumptions: Permanent connection to the internet, a smartphone and un-interrupted electric service.

I’m not proposing to cut the cord from the internet, go back to punchcards or operate everything on solar (although that would be super cool). Instead, I want to turn those “Must haves” into “Nice to haves”. Instead of trying to connect that lamp to the internet, let’s focus on things that are more important like: Reusability, Stand Alone capabilities, Simplicity and Cost. The how to get there is the Mission of this Blog.

Zombie Home Manifest

  • A Zombie Home uses technology to solve real problems, automate boring tasks and provide useful insight but:
  • A Zombie Home should not depend on smart phones to be operated.
  • A Zombie Home should not depend on Cloud Services or permanent Internet connection to work.
  • A Zombie Home should not depend on an electric grid to work.
  • The technology that makes a Zombie Home gadget work should be 100% salvageable to be used in another gadget.
  • The skill to setup a zombie home should be equivalent to the skills to connect any other home appliance.
  • Zombie gadgets should take advantage of old and latest technology without committing to any in particular.
  • But more important than anything: It should be affordable. Not only affluent people have problems that can be solved in a smart way with technology.