I challenge you to look at the financial world and not come away depressed. Or take a look at the happenings in the political arena and tell me if it doesn’t leave a bitter taste in your mouth. And then I invite you to turn your attention to what is happening in the technological and scientific world. I bet it will turn even the most despondent among us a little bit optimistic.
Then there are those times when you encounter something which just puts a smile on your face. Witnessing IBM Watson’s virtuoso performance was one such moment. Today, there is this news on TechCrunch of a new way of interacting with computers that a group of researchers from Microsoft and Carnegie Mellon University have come up with. Take a look:
Here is how Computing Now describes it:
A wearable projection system that Microsoft Research and Carnegie Mellon University (CMU) developed lets users create graphical-input interfaces on any surface. OmniTouch has a pico- projector that can display images of keyboards, keypads, or other traditional input devices onto any surface, even a human hand. It uses a depth-sensing camera—like that used in Microsoft’s Kinect motion-sensing input device for the Xbox 360 video game console—to track a user’s finger movements on any surface. The system is mounted on a user’s shoulder, but the researchers say it eventually could be reduced to the size of a deck of cards. Chris Harrison, Microsoft Research PhD Fellow and CMU doctoral student, said OmniTouch lets users have a wide range of input because it is optically-based. OmniTouch does not require calibration. According to the researchers, a user can begin utilizing the device without having to calibrate it.. Work on the project will be presented 19 October at the Association for Computing Machinery Symposium on User Interface Software and Technology in Santa Barbara, California. (PhysOrg.com)(Chris Harrison website)(Carnegie Mellon University)
Combine this with Watson like intelligence (or even something like Siri) and you have a powerful system. Shrink it down to a head-mountable size, small enough to fit on the front of a baseball cap, improve the computer vision algorithms it uses and you have a technology that is, in terms of features, already ahead of HAL 9000 or the on-board computer of the Enterprise of Star Trek fame. I believe this could be made available in roughly this configuration in 2 to 3 years.
Now, why do I believe this has the potential of revolutionizing healthcare? It addresses several prickly challenges peculiar to the doctors’ needs.
- The clinicians almost always have to use both their hands (and sometimes their minds) for the procedure they are performing (measuring BP, performing a clinical examination, surgical operation etc.). Leaving the patient to access a computer (or even a tablet) is not convenient. Using a tablet like device brings up the issue of sterilization and the ability of such devices to tolerate the sterilization procedures.
- Tablet computers show their outputs only in a limited area – a tiny screen bounded by its bezel. A representation of the real world has to be recreated within these confines (something like augmented reality). With the OmniTouch approach, any surface, be it the wrist of the surgeon, abdomen of the patient, or her pelvic cavity, not just becomes the input device; it also is transformed into a screen. Never before has computing been this close to the real world.
- Most computers require data to be entered explicitly. However, is it possible that we go about our business, and computers do the data capturing, without intruding? I think we are on the cusp of a technology convergence where this would not seem so far fetched. OmniTouch (or more specifically Kinect) technology will have to be combined with some nifty activity recognition capabilities to achieve just that. Imagine, a nurse administers an antibiotic injection while the head-mounted device, recognizes the drug, the patient, the nurse and the act of administering the drug, using a combination of bar-code/QR code recognition, facial recognition and activity recognition. All data recorded, no keys pressed, no notes dictated!
Many possible uses come to mind.
Just imagine, a newbie surgeon getting the guidance from this device. I can visualize him taking a peek at the text book, for the next step in surgery, projected next to the incision site on the sterile drapes covering the patient. Combined with a Siri like interaction capability, you could very well imagine a scenario like this:
(Let us call the smart tool made by melding these cool technologies, Annika*, and let the rookie surgeon’s appellation be Dr. Greenstick, for our little fantasy’s purposes)
Dr. Greenstick (muttering to self): This looks like the internal iliac artery.
Annika: Dr. Greenstick, I think it is the ureter. I would think twice before ligating it. Why don’t you clear away some of the fascia so that it is more visible.
(Dr. Greenstick teases off some of the fascia in the pelvic fossa)
Annika: I can see it is the ureter. Do you want me to point it out for you?
Dr. Greenstick: Yes, please.
(Annika projects a fluorescent green line, curving along the course of the ureter in the pelvis making it obvious)
– – – – end scene – – – –
As the nurse adjusts the Oxytocin infusion pump for a patient in labor with tardy uterine contractions, Annika projects the recommended infusion rate right on the surface of the infusion pump console, calculated based upon an assessment of contractions and the fetal heart rate.
The neonatologist examining a baby indicates to Annika, by placing both his index fingers on the opposite sides of the baby’s head, the level he wants the head-circumference to be measured. Annika obliges by displaying the circumference (and a graph to show if the circumference deviates from the normal) right on the forehead of the baby.
A diabetologist is monitoring the progress of a slow healing foot ulcer of patient on his return visit. Annika quietly displays, next to the ulcer its image from the patient’s previous visit, to allow the diabetologist to compare it with its present state.
I could go on, but I am sure you get the picture.
Oh, OmniTouch, how you have spurred the imagination.
Somewhat incongruously, it reminds me of a couplet from a ghazal (a form of music and poetry popular in the Indian subcontinent). It goes like this:
Agar sharar hain to bhadkein, jo phool hain to khilein
Tarah-tarah ki talab tere rang-e-lab se hai
My rough translation (with apologies to the great poet, Faiz):
If they be embers let them burst into flames, if they be flowers let them blossom
So many be the desires that the color of your lips inspires
Yet another secret desire – IBM Watson and OmniTouch teams please get together to bring these fantasies to life, for the larger good, eh? And come on now, what’s with the name OmniTouch!? Can you not think up of a name that befits such cool piece of technology.
*Annika – a human female who is ‘assimilated’ by the Borg, rendered into one of their own, enhanced by the technology and knowledge of all the civilizations previously assimilated, and given the designation, Seven of Nine. Later she was rescued by the Voyager team and inducted as a Starship Voyager’s staff – ruthlessly efficient, emotionally distant and yet very sexy. She goes through an agonizing process of rediscovering her humanity but the vestiges of the Borg still remain a part of her.