1Click飲みRomoCartTempescope色色[:iroiro]Other Projects

2014年1月29日水曜日

Myra, an intelligent robotic lighting system for your living room

Myra is a robotic spotlight that adjusts autonomously to optimize the lighting condition in your home.
By sensing what you're doing (e.g reading, sleeping, eating, etc) using depth sensors, it changes the orientations of the lights to ensure you always have the perfect lighting, no fumbling around with switches needed.
I started this project in my spare time in December 2013.  I intend to opensource everything when I find the time.


See Myra in action:



The Objective
The motivation behind this project is simple: to free people's lives from the constraints of the "fixed" lighting system.

Conventional room lighting is static (or only mildly flexible).
- For example, many rooms (especially in Europe) are designed with a dark ambient, with some spots of light. 
This is aesthetically fine, but what if you decide that you want to read a book in the middle of the room? You can't.
With conventional lights, actions of people (e.g reading) are constrained to particular locations (e.g next to reading lamps).
- Another example is if a single room holds multiple functions, like a living room and dining room (as is often the case in small countries like Japan.)
It would make sense to be able to change the lighting based on how you're currently using the room;
when you're eating, you want to light up the table and hide everything else,
but perhaps if you're watching TV, you just want some light on the walls.
Again, the conventional static lighting system wouldn't allow this (very easily), so when a room is multi-functional, people have to compromise and live with suboptimal, inaesthetic lighting.

Such problems could be solved if we had a lighting system that:
(a) can orient light and shadow in any configuration
(b) does (a) with minimal user input (preferably completely autonomously)

This is the idea behind building Myra, a robotic spotlight array that will configure itself automatically by recognizing the residents' activities.

The Prototype

The Myra system consists of three components:
(1) The Myra Light: this is a robotic arm with an LED.  An array of these are placed around the room, and are controlled by a central PC.
(2) An RGBD sensor (e.g Microsoft Kinect, or Asus Xtion): this allows the PC to "see" where the residents are, and what they're doing.
(3) A PC: the PC analyses the readings from the sensor to classify user activities, decides how to light the room, and sends commands to individual Myra lights.


Components of the Myra system; an array of Myra lights (left) and an RGBD sensor (right)

(1) Building the Myra Light
○Hardware
The hardware is very simple. (schematics etc will be uploaded somewhere at some point)
It is an Arduino controlled wireless robot arm with a high-powered LED.
The LED is fitted with a lens, which gives a fairly strong light beam of 15° arc.
A Myra Light on my curtain rail

BOM (for a single Myra light):
The entire setup (for a single light) should be less than $60.
I have attached all of my lights to the curtain rail, using an attachment that I made using plywood.
You also need a 315MHz transceiver to wirelessly control the light from your PC.

○Software: 
The software that goes on the Myra light is a very simple Arduino sketch;
receive orientation (θ,φ) and the intensity (k) from the 315MHz receiver with VirtualWire.h, and then set the orientations of the servo (and the intensity of the LED) accordingly.

(2) Setting up the RGBD sensor
Any off-the-shelf RGBD sensor will do.
I used an Xtion Pro Live, because it requires no external power and so is easier to handle.

(3) Developing the controller PC
All of the intelligent decisions take place in a central PC that reads input from the sensor, and sends commands to the Myra lights.

The current system is written in Java, using OpenNI and NiTE for processing the RGBD sensor feed.
Visualizations are done using JMonkeyEngine.

The figure below shows the control pipeline for the Myra system:
Control pipeline for Myra

I won't go into the specifics of each step (I'll do that in a future post), but here is what each step does.
1) You live your life. Myra requires no user input.

2) The RGBD sensor collects a point cloud of your room at 60Hz.

3) Myra uses NiTE to extract people from the point cloud. NiTE tags people, and their features (e.g head, shoulders, hands, etc) automatically.  

4) This is where much of the hard work happens.  Myra first classifies the state of each person into 5 different activities: reading, watching TV, standing, walking, sleeping.
Then, "lighting targets" are set according to the person't activity, and their postures.
For example, if a person is "reading", then a target is set on the person's arm closest to their head.
 -> this allows Myra to continually adjust the light so that your book is never in shadow. 
If the person is "walking", multiple targets are set: one at the predicted destination, and others at various features in the path up to the destination.
 -> this is useful in cases like, when you are watching a film and you need to go to the toilet, or you get up in the middle of the night to get a glass of water.  There's no need to find the light switch, because Myra automatically shows the way to the door, without disturbing anyone else.

5) After lighting targets have been found, Myra assigns a robot light to point at one (or more) of the lighting targets.  The video above shows a scene where Myra picks the best light to shine on a book I am holding.

6) Finally, the orientations of each of the lights are sent wirelessly to each robot light.

Steps 1~6 are repeated about 3 times a second. 

(4) Calibration
When the system is first setup, Myra doesn't know where each of the lights are with respect to the sensor.
This is essential information, because it needs to be able to know which values of (θ,φ) would shine light on a particular position in the sensor's coordinate system.


Myra calibrating itself
I'm not going to go into details here, but I have developed a fully automatic calibration system (as shown in the video); I will explain how this works in a future post.
Essentially, the system points the light in random directions, and sees where the center of the lights lye.
It then uses simulated annealing to find the best-fit estimation of the light orientation.

So what can it do?
Here are some of the things that I have got Myra to do so far:

- automatically generate "skins" for different functions of your living room
The same room looks entirely different just from the lighting.

- point of interest tracking: ensure that whatever posture you're in, your book will always be lit
Myra follows you, however absurd your favorite posture may be.
- create a "path of light" to wherever you're going, so you can always find your way anywhere safely and without stress
You're never in the dark with Myra.
- automatically turn off when you fall asleep

- a "targeted alarm clock" that wakes a person up by shining targeted light at the target's face, without disturbing anyone else

Finally...
Myra is a project still very much in development.
I don't think the use cases I've described above fully exploit the potential of the very new concept of an autonomous robotic light array.
When I find the time, I will release all schematics and code in open source.
I will appreciate any comments or suggestions.
Thanks for reading.

3 件のコメント:

  1. Very nice. I thought about doing something like this on wire cabling lighting systems (i.e.: http://designsenselighting.com/the-special-cable-track-lighting/) and a movable/motorized light on the cables. Adding your pan/tilt along with RGDB would be a great combination.

    I look forward to reviewing schematics and code when you release them.

    返信削除
    返信
    1. Thanks for your feedback. Implementing a moving cabling lighting system sounds like a very interesting project. If you try to add pan/tilt to a cable system though, there may be some difficulty in stopping the lights from swinging. (I've never actually played with cable lights, so I may be wrong).
      I'd be very interested to know if it works. Please keep in touch!

      削除
  2. I would like to know what happens if you have two or more people in the same room. Would it scan everyone and still work. If no what would it need to make it work?

    返信削除