Wearables

Typing on Smart Glasses

A while back we talked about how we were able to help people type up to 30 wpm on their smart watch. Now we've updated the research to apply to an even smaller typing surface on the side of a pair of smart glasses.

Autodesk Research Smart Glasses

The challenge in adapting this from a smart watch is twofold:

  1. You can't see the surface you're typing on - most smart glasses use voice commands
  2. The surface you're typing on is long and narrow - the diagonal swipes used on a smart watch don't work as well

Adapting the SwipeBoard technology to the smart glasses with a heads up display gets around the first problem. The second problem is addressed by dividing the long strip into three, using a piece of tape for tactile feedback on the zones and eliminating the diagonal gestures with vertical gestures in each zone.

Autodesk Research Smart Glasses SwipeGlass

This is the first known work in this area and there are lots of possibilities for future study. Have a look at the video below for more details. 

 


Humans and Robots Collaborating at Autodesk University

Visitors to Autodesk University 2015 are invited to work with a robot through wearables and internet of things technologies to help build a 12' high architectural pavilion using bamboo.

Autodesk Research HiveThe hive will be created from 224 tensegrity units. Each tensegrity unity is composed of three bamboo rods held together with string that is wound by robotic arms. The magical part is that the bamboo rods are not touching each other. Each tensegrity unit is unique due to the bamboo rods having differences in length and diameter. They are connected together with special LED units that help the builders place the pieces and will create a light show. 

Autodesk Research Hive Bamboo Tensegrity Unit

The hive is coordinated and tracked by a system called the foreman engine. In it one can see the project status and contributions by all workers. 

Autodesk Research Hive Foreman System

You can see the full details in the movie below and follow the AU Hive on twitter

 

 

 


Research at Autodesk University 2015

For those attending Autodesk University this year in Las Vegas, Autodesk Research will have a booth in the “Central Park” section of the Exhibit Hall where we’ll be showcasing a number of exciting projects.  

Autodesk University 2015

The projects represented at this year’s conference will include:

Autodesk Research Autodesk University 2015

 

The Bio/Nano Research group will be showing the current status of their research on how to fold DNA to create functional nanostructures as well as how to grow artificial bones.

 

Autodesk Research Autodesk University 2015

 

Project Dreamcatcher will be exhibiting a number of generatively designed artifacts, such as the Optima Bow made by Pier 9 Artist-in-Residence John Briscella of Aminimal Studio, bicycles and motorcycle swingarms.

Autodesk Research Autodesk University 2015

 

 

The Design and Fabricaction group will be showing Meshmixer and how it is used to 3D print functional parts and body-fitting prosthetics. 

Autodesk Research Autodesk University 2015

 


Autodesk Within Medical, which allows implant designers to create porous coatings to aid bone and implant fusion (ie. osseointergration), will be displaying a number of their 3D printed medical components and explaining how their technology works. 

 

 

AU 2015 Map to Autodesk Researc
When you enter Sands Hall B & C, just walk to the Central Park and Autodesk Research will be on the right!

In addition to the booth, look for the Hive Project near the Exhibit Hall where Autodesk University attendees will build an architectural scale pavilion guided by human/robot interaction.

Autodesk Research Autodesk University 2015

A number of team members will be giving talks at AU:

 

Composite Materials and Manufacturing Processes for Automotive Applications

Tuesday, Dec 1, 3:00 PM - 4:00 PM, Location: Zeno 4701, Level 4

Massimiliano Moruzzi discusses composite applications in the automotive marketplace.

 

Automated Composite Manufacturing

Tuesday, Dec 1, 5:00 PM - 6:00 PM, Location: Marcello 4401a, Level 4

Massimiliano Moruzzi presents an end-to-end solution for the automated composite manufacturing process. This class will cover advanced lay-up design strategies such as fiber placement, tape layering, and robotics lay-up which are utilized when programming automatic material layup equipment. High composite production rates will be covered through automated robotic material nesting and taping.

 

Cultivating Innovation and Developing Intrapreneurs

Wednesday, Dec 2, 10:00 AM - 11:30 AM, Location: Zeno 4704, Level 4

Cory Mogk will be doing a talk on Cultivating Innovation and Developing Intrapreneurs that uses the tools from the Innovation Workshop. This class will talk about how Autodesk is helping intrapreneurs develop their ideas and we’ll provide tools and guidance that attendees can use on their own or in their organizations. 

 

Composite Manufacturing Solution for Optimum Material Nesting and Ply Layup

Thursday, Dec 3, 2:45 PM - 4:00 PM, Location: San Polo 3405, Level 3

Massimiliano Moruzzi will lead this two-part class where attendees will utilize Autodesk TruNest Composites to show the complete process from import to nesting to NC part cutting of ply materials. Special focus will be given to optimal nesting for efficient material usage. During the second half, we will utilize Autodesk TruLaser to perform laser projection for showing composite ply lay-up.

 

Once again, the Design Research team will be conducting user research sessions. This year’s focus will be on collecting feedback for Within and Dreamcatcher.  Look for the OCTO Airstream in the AU registration area.

  Autodesk Research Astronaut

We hope you’ll make some time to come by and meet some of the team. 

 

 


Magic at CHI 2015 (or how to hide your mobile device addiction)

Many people are addicted to their mobile devices and the constant flow of information. In social settings, such as work meetings, people know it's wrong and try to hide their device checking in many ways, including:

  • going to the bathroom
  • faking migraine headaches
  • hiding the device under a table or their clothing

Autodesk Research Deceptive Devices

The User Interface group at Autodesk Research conducted a survey of more than 200 people and 94% reported getting caught using a mobile device. Helping people to sneak a peak more easily seemed like a good challenge and the team looked towards magicians to see if they could learn things that could be applied to software and device design.

The team came up with some pretty cool gadgets including:

  • spyglasses
  • secret recorders that could play back the last few seconds of a meeting through a small earpiece to cover up that you weren't paying attention
  • a sensor for knowing when people are behind you
  • information embedded in audio tones that could be perceived as meeting reminders, email notifications or a ringing phone

The Phoney Phone

The Phoney Phone is an app that makes ones' phone look likes it's sleeping while letting the user see the results of their tapping on an alternate screen that could be hidden in the bottom of a coffee cup. To an observer, they may just look like they are fidgeting or contemplating the last sip of a drink.

Autodesk Research Phoney Phone

The Magput

The Magput hides sensors in a pencil and a notebook. What may appear to be random tapping or doodling could actually be be some serious work.

Autodesk Research Magput

You can see these gadgets in action and test how easy they are to tell when someone is using them in the following short video clip.

What does a Designer of Deceptive Devices Need to Know?

When designing for subtle interactions, designers should consider many of the same things magicians do:

  • User Customization: allow the user to customize their device. If they use a device that does not fit their environment or personality it could give them away
  • Modularity: allow the user to work with the system in pieces. Could a component of the system change location so that the user is not seen doing repetitive tasks?
  • Simulation and Dissimulation: Take advantage of existing devices that people obviously use. We know how most people type on a phone so if you can hide the interaction it, or make it appear inactive to observers, they will be less likely to suspect activity.
  • Separating Cause and Effect: Magicians introduce delays to misdirect the audience. This is counter-intuitive to traditional UI design so it requires special consideration.
  • User Training: Magicians practice and so should your users - so make it easy for them.

To take this magic further, Tovi covering for Fraser who was getting married at the time (congrats, Fraser!), added a magician to the presentation of this research at CHI 2015. The show is below.

Additional Uses for Subtle Interactions

Beyond helping people to sneak a peak at their devices, these techniques could be used to:

  • enhance presentations by giving presenters extra techniques to share their information in engaging ways
  • help with wearable device design and interactions where users cannot use a device in a traditional manner

For more details on the research and how some of these devices were made and controlled, please refer to the publication entitled Supporting Subtlety with Deceptive Devices and Illusory Interactions.


It is Indeed Possible to Type 30 Words Per Minute on a Smart Watch

Ironically, as cell phones are getting bigger, we see increasing popularity in ultra small screen devices such as smart watches. With these smaller screens we need to find ways to work more efficiently with them or risk these new devices being regarded as novelty items. The same old interfaces don't work.

Autodesk Research Swipeboard Smart Watch Text Entry
What time is it? It's time for Swipeboard!

One of the most common things to do on a mobile device is to enter text. We've learned to enter text with our thumbs so we can continue to learn. The problem with a smart watch is that it's a one handed device and the size of the screen really only works for a single finger without obscuring too much of the screen. Not content to revert to hunt-and-peck typing 101 the Autodesk Research User Interface group set out to find a solution.

Enter Swipeboard

Autodesk Research Swipeboard Title

Swipeboard takes inspiration from Morse code and gestural input for an easy to master text entry paradigm that sees users entering more than 30 words per minute (wpm).

Morse Code
The fastest recorded Morse code entry is 140wpm.

Swipeboard uses a QWERTY keyboard broken up into segments of 3 or 4 characters. The user simply taps in the region of the character block and then swipes to identify the character. Some users have achieved a level of comfort with the system that allows them to enter text without looking at the screen.

Autodesk Research Smart Watch Text Entry Swipeboard
First a QWERTY style keyboard is shown for selecting the character region
Autodesk Research Smart Watch Text Entry Swipeboard
After a tap, the keyboard zooms in to prompt for a gesture to define the specific character


Hard to believe? Watch the video of Swipeboard in action below. Note that the video is not sped up - you're seeing it work in real time.


 

What's next for Swipeboard?

Well, we'll be talking about it at UIST 2014, the User rInterface Software and Technology Symposium, for starters.

Autodesk Research Swipeboard Glasses
Swipeboard could be applied to other wearable devices such as glasses

For future work, this could be interesting to explore on other wearable devices like glasses and rings. It could also be interesting to see Swipeboard expanded from characters to complete words. What do you think?

If you liked this post, you might also like to read about Duet, a research project that looks at making a smart watch and smart phone work well together. Duet shows that 1 + 1 can equal more than 2.


How would Autodesk make the Apple Watch work with the iPhone?

For the impatient, the answer is we would make both devices active participants for display and input. Like two mathematical musicians playing a duet, the beautiful music they create can equal more than two (perhaps it can go as high as eleven).

Antique Clock Phone
Composite of images by Tim G. Photography and Garry Knight under the Creative Commons License

OK. Confession time: we do not have an Apple Watch or a new iPhone and we did this research before they were announced. We used devices that are publically available. But, that should not make this research any less interesting.

With the premise that two tools working together can create greater values, the User Interface group at Autodesk Research, with partners at the University of Toronto and Carnegie Mellon University, started to explore these possibilities:

  • new input methods 
  • new security possibilities
  • new operational abilities 

New Input Methods

The watch has an acceleronmeter in it so it can provide intital information about how the hand it is attached to is working as an input device. Specifically, what is the orientation of the hand relative to the phone. Knowing the orientation of the hand means a person is not limited to the traditional finger tip press. People could now also enter data with:

  • the side of the finger
  • the back of the finger, also known as the knuckle
Autodesk Research Knuckle Input with Smartphone and Smartwatch
A person may use their knuckle as an alternative to their fingertip or as an additional tool

What could this do for reading email? You could have one finger touch point for navigation (move through message, go to next message, etc.), one finger touch point for email management (archive, delete, etc.) and one finger touch point for things like cut, copy and paste.  

Of course, a person is not limited to entering data on the phone. Wearing the watch on the inside of the wrist, so that the watch screen is oriented in the same manner as the phone screen in the hand, a person could be gesturing across devices:

  • swipe from phone to watch (to mute the phone and set the watch to buzz mode)
  • swipe from watch to phone (to unmute the phone and turn the buzz off on the watch)
  • close pinch across devices (to mute both devices)
  • open pinch across devices (to unmute both devices)

  Autodesk Research Multi Device Gestures on Smartphone and Smartwatch

And of course you could tap the phone and watch together or tap and flip devices to initiate additional commands.

Autodesk Research Smartphone and Smartwatch Double Bump

New Security Possibilities

Knowing a phone and watch are paired, could improve security - you can only use the phone if your watch matches. Plus, a new gesture could be made for unlocking the phone.

Autodesk Research Smartphone and Smartwatch Security

New Operational Abilities

This is where it really gets cool!

What if you used the watch as a tool to zoom in on a map without losing your position?

Autodesk Research Smartwatch to zoom in on Smartphone Display

What if you used the watch as a tool palette for the phone?

Autodesk Research Smartwatch Smartphone Tool Palette
New gestures and connections open up a lot of possibilities. You can read more about this research in the Autodesk Research paper entitled Duet: Exploring Joint Interactions on a Smart Phone and a Smart Watch as well as watching the movie below. What kind of things could you imagine doing with these abilities?