Cool Things from Autodesk Research at UIST 2015

Autodesk Research will be presenting five papers at the 28th ACM UIST User Interface Software and Technology Symposium in Charlotte, NC, from November 8-11. UIST is the premier forum for innovations in human-computer interfaces. UIST brings together researchers and practitioners from diverse areas including graphical & web user interfaces, tangible & ubiquitous computing, virtual & augmented reality, multimedia, new input & output devices, fabrication, wearable computing and CSCW.

UIST 2015

This year there has been an explosion in research related to the areas of digital fabrication and fabricating electronics. You may browse the full program and see Autodesk's contributions below.

NanoStylus: Enhancing Input on Ultra-Small Displays


Candid Interaction: Revealing Hidden Mobile and Wearable Computing


MoveableMaker: Facilitating the Design, Generation, and Assembly of Moveable Papercraft


Smart Makerspace: An Immersive Instructional Space for Physical Tasks

Autodesk has contributed more to UIST 2015 than just papers. We're a platinum sponsor, Tovi Grossman has been serving as the Program Committee Co-Chair and Justin Matejka has been serving as the Video Previews Co-Chair.

Here's a preview of the rest of the show.



Autodesk Research at SIGGRAPH 2015

If you're attending SIGGRAPH 2015 in Los Angeles, watch out for these presentations from members of Autodesk Research and say hi to the presenters!:

Kitty: Sketching Dynamic and Interactive Illustrations

Monday, 10 August 3:45 pm - 5:15 pm, Los Angeles Convention Center, Room 402AB

Rubaiat Habib, Fanny Chevalier, Tovi Grossman and George Fitzmaurice

Kitty will be part of the UIST Reprise and you can read more about Kitty in this blog post.

Printing Elastics

Thursday, 13 August 10:45 AM - 12:15 PM, Los Angeles Convention Center, Room 150/151

Session Chair: Nobuyuki Umetani

Convolutional Wasserstein Distances: Efficient Optimal Transportation on Geometric Domains

Tuesday, 11 August 10:45 AM - 12:15 PM, Los Angeles Convention Center, Room 152

Justin Solomon, Fernando de Goes, Gabriel Peyré, Marco Cuturi, Adrian Butscher, Andy Nguyen, Tao Du, Leonidas Guibas

This paper introduces a new class of algorithms for optimization problems involving optimal transportation over geometric domains. The main contribution is to show that optimal transportation can be made tractable over large graphics domains, such as images and triangle meshes, improving performance by orders of magnitude compared to previous work.

OmniAD: Data-Driven Omni-Directional Aerodynamics

Wednesday, 12 August 2:00 PM - 3:30 PM, Los Angeles Convention Center, Room 153A-C

Tobias Martin, Nobuyuki Umetani, Bernd Bickel

Introducing OmniAD, a novel data-driven pipeline to model and acquire the aerodynamics of three-dimensional rigid objects simply by capturing their falling motion using a single camera. OmniAD enables realistic real-time simulation of rigid bodies and interactively designed three-dimensional kites that actually fly.

Computational Tools for 3D Printing

Thursday, 13 August 2:00 pm - 5:15 pm, Los Angeles Convention Center, Room 403AB

Nobuyuki Umetani, Bernd Bickel, Wojciech Matusik

This course reviews current 3D printing hardware and software pipelines, and analyzes their potential and shortcomings. Then it focuses on computational specification for fabrication methods, which allow designing or computing an object's shape and material composition from a functional description.

3D-Printed Prosthetics for the Developing World

Thursday, 13 August, 9-10:30 am,SIGGRAPH Studio

Ryan Schmidt, Ginger Coons, Vincent Chen, Timotheus Gmeiner, Matt Ratto

You can read more about this project here.

Autodesk Exhibitor Sessions

Members of your favourite Autodesk product teams will be showing off cool, new things throughout the week.

The Art of Fluid Animation

And you may see Jos Stam wearing a shirt like this - ask him about it :)

Autodesk Research The Art of Fluid Animation


Data Visualization is Beautiful as a Wall Decoration

Citeology is an interactive tool for visualizing relationships across research papers created by Justin Matejka, Tovi Grossman and George Fitzmaurice of the UI Group. Selecting any one of the 11,000 plus publications from CHI and UIST will show you its geneology; its parents (papers that it cites) and its children (papers that site it).

Beyond being helpful to the user interface community these graphs are beautiful. We have a wall size version of one graph in the Toronto Autodesk office.

Autodesk Research Toronto Citeology

The layout of the information is simple and effective. Across the horizontal axis is a listing of all the papers by year. As time progresses more papers have been published, much like our growing human population. Parent, or past papers are connected by blue lines while children, or future papers, are connected by red lines.

Autodesk Research Citeology

The lines drawn between papers are semi-transparent add build up to show multiple connections.

Autodesk Research Citeology

Similar to a word cloud, all the titles are displayed with the connected papers being shown in darker colors to stand out.

Autodesk Research Citeology

The complete tool shows some additional information and controls for refining your search results including:

  • shortest path between papers
  • number of children and parents to show
  • details about the active paper

Autodesk Research Citeology UI

Citeology uses research papers and it's interesting to think about what other kinds of relationships a tool like this could help to visualize:

  • Building on geneology, things like family trees, band memberships, and sports teams are likely candidates
  • Historical figures and events along with their triggers
  • Connections and dependencies between things in the Internet of Things

Autodesk Research Citeology

What would you use it for? Try Citeology and let us know what you think!


Hacking a Multi-touch Tabletop for Design Collaboration

Multi-touch tabletop computers are useful tools and the User Interface group at Autodesk Research has explored ideas on ways to make them even better with a system called Medusa. Imagine a world where the tabletop can recognize multiple users, differentiate between right and left hands and support non-touch, virtual reality type gestures like in the sc-fi movie Minority Report.

Medusa Autodesk Research Multi-touch user interface

It all starts by hacking a Microsoft Surface with 138 proximity sensors and Phidget Interface Kits. These sensors extend the touch capabilities of the computer surface to determine user proximity and the location of their hands. The sensors ane not only inexpensive but they remove complications like setting up cameras or requiring users to wear gloves or tracking markers. In a future incarnation, these sensors could be built into the table for a better aesthetic and to prevent the users from needing to worry about them.

Medusa Autodesk Research Microsoft Surface Tabletop Multitouch
Medusa'a sensors are arranged in three rings. An outward-facing ring of 34 sensors is mounted beneath the lip. Two upward facing rings atop the table are made-up of 46 sensors on the outer ring and 58 sensors on the inner ring.

All of this adds up to allow Medusa to support the following user interactions:

  • User Position Tracking
  • User Differentiation
  • Independent Left and Right Hand Tracking 
  • Hand Gestures (Pre-Touch Functionality) 
  • Touch + Depth Gestures

This was tested with a prototype UI creation application called Proxi-Sketch. Proxi-sketch allows users to collaboratively develop new graphical user interfaces. You can see it all in action in the following video. If you want to know more about building the system or how parts of it worked, please refer to the Medusa publication.  

Kitty Wins a Best Talk Award at UIST 2014

Great news! Autodesk Research was awarded a Best Talk Award at the 2014 User Interface Software and Technology Syposium (UIST) for Kitty, a tool that makes it possible to draw interactive experiences.

The team was very happy to present their research throughout the week, as you can in the photos below:

Baseball Video Lens Demonstration

PipeDream Demonstration

Kitty Demonstration

Here's Rubaiat accepting the award

Congratulations to the Kitty Team!

Afterwards, the team celebrates with an awesome display of their fire dancing skills (as a researcher it is important to have many varied interests and abilities)

Some members of the team enjoy simpler pursuits 

Beyond the awards and activities, there was food. Team members who didn't get to attend UIST were sad to miss the fancy breakfasts this year.

At least one UIST attendee found a new use for Oculus - making food better :-)


Four Cool Talks from Autodesk Research at UIST 2014

The ACM Symposium on User Interface Software and Technology (UIST) is just around the corner. This year Autodesk Research is a platinum sponsor and has four cool papers to present covering a diverse range of topics from 3D printing interactive objects to a new text entry method for smart watches to big data analytics with baseball and a new paradigm for drawing interactive content.


A Series of Tubes: Adding Interactivity to 3D Prints Using Internal Pipes 
Valkyrie Savage, Ryan Schmidt, Tovi Grossman, George Fitzmaurice, Bjoern Hartmann
10:30am Monday, October 6
Autodesk Research Pipedream
PipeDream can help makers create interactive 3D prints.
Kitty: Sketching Dynamic and Interactive Illustrations 
Rubaiat Habib Kazi, Fanny Chevalier, Tovi Grossman, George Fitzmaurice
12:30pm Tuesday, October 7
Kitty builds on the Kinetic Texturing work seen previously in Draco and allows the author to make it interactive.
Swipeboard: A Text Entry Technique for Ultra-Small Interfaces That Supports Novice to Expert Transitions 
Xiang 'Anthony' Chen, Tovi Grossman, George Fitzmaurice
11:00 Wednesday, October 8
Swipeboard takes inspiration from Morse code to present a new and fast method of gestural input for smart watches.
Video Lens: Rapid Playback and Exploration of Large Video Collections and Associated Metadata 
Justin Matejka, Tovi Grossman, George Fitzmaurice
11:00am Wednesday, October 8

Autodesk Research Baseball Video Lens

Video Lens is a great tool for exploring massive amounts of data and is built around baseball videos to show a practical application. 
Demos of all of these will be available on Monday night so stop by and say hello to the team!


Tubes Can Help You Make Interactive 3D Printed Objects

When you are making, do you 3D print sculptures or 3D print equipment that will need wires and sensors and lights? If it's the latter, you'll want to read about this new work from the Autodesk Research team called PipeDream that helps you easily put tubes into your models and extend the potential of your creations.

Autodesk Research 3D Printing Pipes
A 3D Printed Radio with Tubes for Controls

In the above example, you can see a 3D printed radio with tubes in it for the speaker, volume, power and tuning controls. Below is another example with tubes added into a desktop pen holder. Smaller tubes at the bottom have spaces for sensors to determine which pens are present. This kind of idea could be expanded to a work shop for tracking tools.

Autodesk Research 3D Printing Pipes
A 3D Printed Pen Sensing Container

PipeDream is some new research that has been prototyped within Autodesk Meshmixer.

Autodesk Research Meshmixer Tubes Pipedream
Looking at tubes in Meshmixer

When creating pipes in your models, you are presented with a number of possibilities:

  • Would you like to specify surface points where the tubes should start and end (like in our examples above)?
  • Woud you rather specify a specific path through the object to make neon lights or a marble maze?
  • What is the radius of the tube and does it very over the length?
  • Does your tube connect two points or does it radiate like branches on a tree?
  • Would you like your tubes to be capped so that you have a cavity in your object?

Capped pipes to make a cavity in an object?

Here's an example of a 3D printed bunny. This bunny is printed in a soft, pliable material. The cavity works as an air bladder so that the bunny can breathe with the help of an air pump. The air coming from the bunny could be used as a feedback mechanism in a toy or a teaching tool.

Autodesk Research 3D Printed Pipes Bunny
This bunny is connected to an air pump to simulate breathing

The cavity in a soft printed object could be fitted with other feedback devices like a noisemaker, buttons for lights, haptic buzzers, accelerometers and sensors to determine if an object has been touched.

Another great example is creating pathways for lights and wires. In the example below created for the 2014 UIST (User Interface Software and Technology) Symposium, the letters are connected to make a continuous path. With this continuous path, a neon sign can be created.

Autodesk Research 3D Printing Pipes UIST


Adding tubes to models that will be 3D printed opens up a lot of possibilities. One other interesting thing we tried was to fill the tubes with conductive paint instead of pushing wires through the tubes. This allowed for easily powering LED's in our models.

To see more of this technology, you may watch the video below or refer to the PipeDream research paper on AutodeskResearch.com.

If you are at UIST, please stop by to talk with the team. If you would like to share your thoughts on this technology or have questions about it, feel free to let us know here on the Autodesk Research blog.


It is Indeed Possible to Type 30 Words Per Minute on a Smart Watch

Ironically, as cell phones are getting bigger, we see increasing popularity in ultra small screen devices such as smart watches. With these smaller screens we need to find ways to work more efficiently with them or risk these new devices being regarded as novelty items. The same old interfaces don't work.

Autodesk Research Swipeboard Smart Watch Text Entry
What time is it? It's time for Swipeboard!

One of the most common things to do on a mobile device is to enter text. We've learned to enter text with our thumbs so we can continue to learn. The problem with a smart watch is that it's a one handed device and the size of the screen really only works for a single finger without obscuring too much of the screen. Not content to revert to hunt-and-peck typing 101 the Autodesk Research User Interface group set out to find a solution.

Enter Swipeboard

Autodesk Research Swipeboard Title

Swipeboard takes inspiration from Morse code and gestural input for an easy to master text entry paradigm that sees users entering more than 30 words per minute (wpm).

Morse Code
The fastest recorded Morse code entry is 140wpm.

Swipeboard uses a QWERTY keyboard broken up into segments of 3 or 4 characters. The user simply taps in the region of the character block and then swipes to identify the character. Some users have achieved a level of comfort with the system that allows them to enter text without looking at the screen.

Autodesk Research Smart Watch Text Entry Swipeboard
First a QWERTY style keyboard is shown for selecting the character region
Autodesk Research Smart Watch Text Entry Swipeboard
After a tap, the keyboard zooms in to prompt for a gesture to define the specific character

Hard to believe? Watch the video of Swipeboard in action below. Note that the video is not sped up - you're seeing it work in real time.


What's next for Swipeboard?

Well, we'll be talking about it at UIST 2014, the User rInterface Software and Technology Symposium, for starters.

Autodesk Research Swipeboard Glasses
Swipeboard could be applied to other wearable devices such as glasses

For future work, this could be interesting to explore on other wearable devices like glasses and rings. It could also be interesting to see Swipeboard expanded from characters to complete words. What do you think?

If you liked this post, you might also like to read about Duet, a research project that looks at making a smart watch and smart phone work well together. Duet shows that 1 + 1 can equal more than 2.

Kitty is a Drawing Tool for Interaction Authoring

Hopefully you're familiar with Project Draco, our answer to the question:

Can animation be made as easy as drawing?


We've discussed Draco here on the blog and have a video overview of what we were showing at this year's SIGGRAPH conference in Vancouver to catch you up.

Kitty builds on Draco and looks into the animation question and asks:

Can we make Draco interactive?


In the image above you'll see two interactions happening:

  • the user can move the dragon's head into the frame
  • the user can move the baby dragon into the pot

With the egg going into the pot, you'll notice that the monster's eyesfollow the egg and that the egg causes a particle splash as it enters the pot.

This opens up a lot of possibilities for iteractive storytelling. 

  • How would children like this for an ebook on a tablet?
  • Does it make web content more dynamic?
  • Could it be useful for game authoring?
  • Is it useful for training and instructions?

Kitty builds on Draco but how does it work?

We've introduced a simple node network to define the relationships between objects. Let's look at the picture below of a different egg going into a different pot - yes we like cooking here at Autodesk Research.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

We've set up the scene as you would in Draco with steam and splashing particles coming from the pot. In the following image you can see that we have a simple node graph that gets overlaid on the picture. This helps reduce UI while keeping the events and relationships in context.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

You can see the path the egg takes to get into the pot as well as two blue circles representing the particle events. The user is making a connection from the egg to the circle on the right to tell the splash to only happen when the egg is close.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

When the connection is made between the nodes, the egg path and the splash, the user can then choose how to link the events. In this case the movement of the egg is connected to the emission of the particles. The inlaid square defines the timing of the event. 

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

The curve can be redrawn to control what happens. The horizontal axis represents the object that triggers the event (the egg). The vertical axis represents the object that is being driven (the particle splash). When the line is flat, there are no particles being emitted.

In this image below we explore using Kitty to explain how an electric doorbell works.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

You can learn more about Kitty and see how easy it is to author these kinds of behaviours in the video below. More information is available on the Draco project page.

We'll be presenting this latest research at this year's UIST, the ACM User Interface Software and Technology Symposium, in Hawaii in October. If you are there, stop by to see the demo or attend the talk. 

Whether you are at UIST or not, please let us know what you think about these tools and the possibilities that they open up for you. 

One Awesome Tool for User Experience Designers and Researchers

Here's a great tool for those of you exploring the User Experience - the Paper Forager!

Autodesk Research Paper Forager for User Experience Design

There is lots of good material nowadays for exploring user experience. Perhaps so much that it makes it hard to get started or find what you need. And that's where the Paper Forager steps in to help! For those with acess to the ACM Digital Library, the Paper Forager let's you explore more than 5000 research papers from ACM CHI and UIST

The Paper Forager is easy to use and provides things like:

  • searching
  • filtering by date
  • most popular authors
  • thumbnail views
  • paper overviews

All without having to download the paper and open it in a viewer. This can really speed up your research.


To make things even faster, when browsing papers, the Paper Forager preloads adjacent papers ro quickly move forwards and backwards through your search results.

Please enjoy this video overview of the Paper Forager, complete with a toe-tapping, finger-snapping beat. If you're at UIST 2014, you can talk to members from the Autodesk Research team and complement them on their musical tastes :)