Dreamcatcher: The End of Design As We Know It?

Noted designer Dan Saffer wrote a good piece about the future of design and what changes could be coming called The End of Design As We Know It. Dan says:

"These changes will be drastic and design will never be the same afterwards. The canary in the coal mine is Autodesk’s Project Dreamcatcher...To many people this is a bleak, grim, oh-shit-there-goes-my-job, future." 

Autodesk Dreamcatcher End Of Design As We Know It

It's nice to being working on a project that captures attention and generates strong discussion. If you haven't heard of Project Dreamcatcher before, the high-level pitch is that you can supply the computer with a bunch of design goals and have the computer develop a bunch of options that meet your criteria for further refinement.

Dan goes on to make some good points on how this could work to benefit the designer as well as things that the designer will still be good at. A while back Erin, Francesco and Mark from the Dreamcatcher team did some research on how designers might interact with such a system and documented it in a paper called Parameters Tell the Design Story: Ideation and Abstraction in Design Optimization. The main finding from this research is summed up as:

"We found that the computed optimum was often used as the starting point for design exploration, not the end product."

In other words, designers would use such a system to generate and explore solutions, possibly finding unexpected solutions. Some call this design optioneering. In this sense we can redefine the meaning of CAD. With typical workflows, most of the design is still happening in the designer's head and the computer is helping to document those ideas - CAD could mean Computer Aided Documentation. Now with the computer actually doing design work, we can truly realize Computer Aided Design.

Autodesk Research Project Dreamcatcher

In common workflows, every design that must be delivered to a client takes time, effort and money so clients usually only get a couple of options. 

How does Design Optioneering Work?

Let's look at building design. There are a number of constraints to be considered, including:

  • site utilization
  • structural design
  • building form
  • energy use

  • buildability

  • operating costs

This many requirements can be very complex to design for and creates a huge number of possible outcomes. So huge actually that we can't even imagine a small fraction of the possibilities due to cognitive limitations. Using high performance computing and big data analysis techniques, many more design alternatives can be explored for a problem space.

To show how this might work, we can look at an architectural project researchers from the University of Southern California (USC) did at the Autodesk IdeaStudio with similar constraints. They developed a model and automated the process of exploring the design space. In their work, the USC researchers analysed the options to show how the different results met the criteria with both visual and quantitative results.

Design Optioneering from University of Southern California Researchers at Autodesk IdeaStudio

We could see a place in the future where designers could easily modify the results for further exploration. Instead of choosing between options A, B and C, designers could further explore the design space between options A and C by changing some design parameters. Of course, designers could modify the resulting CAD file to further customize and finalize the design. 

What do designers think?

In looking at how designers would work with Project Dreamcatcher, what kind of feedback did the team get?

“instead of starting with nothing, you start with something…your optimum gives you a starting hunch.”

“slight variations to form, to a designer’s eye, are either elegant or fat.”

“Good design has inspiration to it...if you have that vision you can encode it and parameterize it and explore it further. Now we have a rich flora of options.”

This indicates that such tools are beneficial and could help to improve the design process. Sounds great but what are the challenges?

Dan pointed out that these kinds of workflows would require some different thinking:

"In order to provide the algorithm the right information to suggest solutions, you have to be certain you’re solving the right problem."

Working differently through Design Optimization

One of the ways the Dreamcatcher team thinks about this is with design optimization. Design optimization puts the emphasis on defining the design problem. Design optimization is important to sustainability. Users of design optimization are making buildings and products more structurally sound with less building materials.

In a typical CAD workflow, one follows these steps:

  • design
  • evaluate
  • select or redesign

With Design Optimization, the steps are:

  • define the problem
  • generate and explore
  • select or redefine 

One participant in the study compared these two processes:

“The typical design workflow is to design then throw to the analyst. Redesign. And then keep playing catch. It’s inefficient. [Design optimization] captures the criteria that are important to you then [you] have the cloud process all the permutations.”

Uses of Design Design Optimization Autodesk Research
Uses of Design Optimization

Thinking differently will not be without challenges. This could require new skills and language. One study participant said:

“1200 variations, you’re not being an architect any more. You are a computer programmer."

Another stressed that the controls need to be easy to understand, control and help you produce good results:

“parameters need to tell the design story.”

This is great feedback and guidance. This is one of the main reasons we do research. Autodesk Research and the Dreamcatcher team are up for the challenge. As Dan said in his blog post:

"Lastly, the way all progress moves forward is by someone thinking differently, trying something that no one else has tried, breaking the established traditions and rules. This is the same in design as in any other field."

Project Dreamcatcher could be the end of design as we know it and like when we transitioned from paper documentation to computer documentation of designs, we may wonder how we ever got anything done before.

Four Cool Talks from Autodesk Research at UIST 2014

The ACM Symposium on User Interface Software and Technology (UIST) is just around the corner. This year Autodesk Research is a platinum sponsor and has four cool papers to present covering a diverse range of topics from 3D printing interactive objects to a new text entry method for smart watches to big data analytics with baseball and a new paradigm for drawing interactive content.


A Series of Tubes: Adding Interactivity to 3D Prints Using Internal Pipes 
Valkyrie Savage, Ryan Schmidt, Tovi Grossman, George Fitzmaurice, Bjoern Hartmann
10:30am Monday, October 6
Autodesk Research Pipedream
PipeDream can help makers create interactive 3D prints.
Kitty: Sketching Dynamic and Interactive Illustrations 
Rubaiat Habib Kazi, Fanny Chevalier, Tovi Grossman, George Fitzmaurice
12:30pm Tuesday, October 7
Kitty builds on the Kinetic Texturing work seen previously in Draco and allows the author to make it interactive.
Swipeboard: A Text Entry Technique for Ultra-Small Interfaces That Supports Novice to Expert Transitions 
Xiang 'Anthony' Chen, Tovi Grossman, George Fitzmaurice
11:00 Wednesday, October 8
Swipeboard takes inspiration from Morse code to present a new and fast method of gestural input for smart watches.
Video Lens: Rapid Playback and Exploration of Large Video Collections and Associated Metadata 
Justin Matejka, Tovi Grossman, George Fitzmaurice
11:00am Wednesday, October 8

Autodesk Research Baseball Video Lens

Video Lens is a great tool for exploring massive amounts of data and is built around baseball videos to show a practical application. 
Demos of all of these will be available on Monday night so stop by and say hello to the team!


It is Indeed Possible to Type 30 Words Per Minute on a Smart Watch

Ironically, as cell phones are getting bigger, we see increasing popularity in ultra small screen devices such as smart watches. With these smaller screens we need to find ways to work more efficiently with them or risk these new devices being regarded as novelty items. The same old interfaces don't work.

Autodesk Research Swipeboard Smart Watch Text Entry
What time is it? It's time for Swipeboard!

One of the most common things to do on a mobile device is to enter text. We've learned to enter text with our thumbs so we can continue to learn. The problem with a smart watch is that it's a one handed device and the size of the screen really only works for a single finger without obscuring too much of the screen. Not content to revert to hunt-and-peck typing 101 the Autodesk Research User Interface group set out to find a solution.

Enter Swipeboard

Autodesk Research Swipeboard Title

Swipeboard takes inspiration from Morse code and gestural input for an easy to master text entry paradigm that sees users entering more than 30 words per minute (wpm).

Morse Code
The fastest recorded Morse code entry is 140wpm.

Swipeboard uses a QWERTY keyboard broken up into segments of 3 or 4 characters. The user simply taps in the region of the character block and then swipes to identify the character. Some users have achieved a level of comfort with the system that allows them to enter text without looking at the screen.

Autodesk Research Smart Watch Text Entry Swipeboard
First a QWERTY style keyboard is shown for selecting the character region
Autodesk Research Smart Watch Text Entry Swipeboard
After a tap, the keyboard zooms in to prompt for a gesture to define the specific character

Hard to believe? Watch the video of Swipeboard in action below. Note that the video is not sped up - you're seeing it work in real time.


What's next for Swipeboard?

Well, we'll be talking about it at UIST 2014, the User rInterface Software and Technology Symposium, for starters.

Autodesk Research Swipeboard Glasses
Swipeboard could be applied to other wearable devices such as glasses

For future work, this could be interesting to explore on other wearable devices like glasses and rings. It could also be interesting to see Swipeboard expanded from characters to complete words. What do you think?

If you liked this post, you might also like to read about Duet, a research project that looks at making a smart watch and smart phone work well together. Duet shows that 1 + 1 can equal more than 2.

Kitty is a Drawing Tool for Interaction Authoring

Hopefully you're familiar with Project Draco, our answer to the question:

Can animation be made as easy as drawing?


We've discussed Draco here on the blog and have a video overview of what we were showing at this year's SIGGRAPH conference in Vancouver to catch you up.

Kitty builds on Draco and looks into the animation question and asks:

Can we make Draco interactive?


In the image above you'll see two interactions happening:

  • the user can move the dragon's head into the frame
  • the user can move the baby dragon into the pot

With the egg going into the pot, you'll notice that the monster's eyesfollow the egg and that the egg causes a particle splash as it enters the pot.

This opens up a lot of possibilities for iteractive storytelling. 

  • How would children like this for an ebook on a tablet?
  • Does it make web content more dynamic?
  • Could it be useful for game authoring?
  • Is it useful for training and instructions?

Kitty builds on Draco but how does it work?

We've introduced a simple node network to define the relationships between objects. Let's look at the picture below of a different egg going into a different pot - yes we like cooking here at Autodesk Research.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

We've set up the scene as you would in Draco with steam and splashing particles coming from the pot. In the following image you can see that we have a simple node graph that gets overlaid on the picture. This helps reduce UI while keeping the events and relationships in context.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

You can see the path the egg takes to get into the pot as well as two blue circles representing the particle events. The user is making a connection from the egg to the circle on the right to tell the splash to only happen when the egg is close.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

When the connection is made between the nodes, the egg path and the splash, the user can then choose how to link the events. In this case the movement of the egg is connected to the emission of the particles. The inlaid square defines the timing of the event. 

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

The curve can be redrawn to control what happens. The horizontal axis represents the object that triggers the event (the egg). The vertical axis represents the object that is being driven (the particle splash). When the line is flat, there are no particles being emitted.

In this image below we explore using Kitty to explain how an electric doorbell works.

Autodesk Research Kitty Interaction Authoring Dynamic Drawings

You can learn more about Kitty and see how easy it is to author these kinds of behaviours in the video below. More information is available on the Draco project page.

We'll be presenting this latest research at this year's UIST, the ACM User Interface Software and Technology Symposium, in Hawaii in October. If you are there, stop by to see the demo or attend the talk. 

Whether you are at UIST or not, please let us know what you think about these tools and the possibilities that they open up for you. 

One Awesome Tool for User Experience Designers and Researchers

Here's a great tool for those of you exploring the User Experience - the Paper Forager!

Autodesk Research Paper Forager for User Experience Design

There is lots of good material nowadays for exploring user experience. Perhaps so much that it makes it hard to get started or find what you need. And that's where the Paper Forager steps in to help! For those with acess to the ACM Digital Library, the Paper Forager let's you explore more than 5000 research papers from ACM CHI and UIST

The Paper Forager is easy to use and provides things like:

  • searching
  • filtering by date
  • most popular authors
  • thumbnail views
  • paper overviews

All without having to download the paper and open it in a viewer. This can really speed up your research.


To make things even faster, when browsing papers, the Paper Forager preloads adjacent papers ro quickly move forwards and backwards through your search results.

Please enjoy this video overview of the Paper Forager, complete with a toe-tapping, finger-snapping beat. If you're at UIST 2014, you can talk to members from the Autodesk Research team and complement them on their musical tastes :) 

How would Autodesk make the Apple Watch work with the iPhone?

For the impatient, the answer is we would make both devices active participants for display and input. Like two mathematical musicians playing a duet, the beautiful music they create can equal more than two (perhaps it can go as high as eleven).

Antique Clock Phone
Composite of images by Tim G. Photography and Garry Knight under the Creative Commons License

OK. Confession time: we do not have an Apple Watch or a new iPhone and we did this research before they were announced. We used devices that are publically available. But, that should not make this research any less interesting.

With the premise that two tools working together can create greater values, the User Interface group at Autodesk Research, with partners at the University of Toronto and Carnegie Mellon University, started to explore these possibilities:

  • new input methods 
  • new security possibilities
  • new operational abilities 

New Input Methods

The watch has an acceleronmeter in it so it can provide intital information about how the hand it is attached to is working as an input device. Specifically, what is the orientation of the hand relative to the phone. Knowing the orientation of the hand means a person is not limited to the traditional finger tip press. People could now also enter data with:

  • the side of the finger
  • the back of the finger, also known as the knuckle
Autodesk Research Knuckle Input with Smartphone and Smartwatch
A person may use their knuckle as an alternative to their fingertip or as an additional tool

What could this do for reading email? You could have one finger touch point for navigation (move through message, go to next message, etc.), one finger touch point for email management (archive, delete, etc.) and one finger touch point for things like cut, copy and paste.  

Of course, a person is not limited to entering data on the phone. Wearing the watch on the inside of the wrist, so that the watch screen is oriented in the same manner as the phone screen in the hand, a person could be gesturing across devices:

  • swipe from phone to watch (to mute the phone and set the watch to buzz mode)
  • swipe from watch to phone (to unmute the phone and turn the buzz off on the watch)
  • close pinch across devices (to mute both devices)
  • open pinch across devices (to unmute both devices)

  Autodesk Research Multi Device Gestures on Smartphone and Smartwatch

And of course you could tap the phone and watch together or tap and flip devices to initiate additional commands.

Autodesk Research Smartphone and Smartwatch Double Bump

New Security Possibilities

Knowing a phone and watch are paired, could improve security - you can only use the phone if your watch matches. Plus, a new gesture could be made for unlocking the phone.

Autodesk Research Smartphone and Smartwatch Security

New Operational Abilities

This is where it really gets cool!

What if you used the watch as a tool to zoom in on a map without losing your position?

Autodesk Research Smartwatch to zoom in on Smartphone Display

What if you used the watch as a tool palette for the phone?

Autodesk Research Smartwatch Smartphone Tool Palette
New gestures and connections open up a lot of possibilities. You can read more about this research in the Autodesk Research paper entitled Duet: Exploring Joint Interactions on a Smart Phone and a Smart Watch as well as watching the movie below. What kind of things could you imagine doing with these abilities?

What is the latest in state of the art baseball analytics?

What is the latest in state of the art baseball analytics?

Who said Autodesk was only about designing stuff?

How do you visualize and analyze mass amounts of video data?

Read on for answers to these pressing questions and more.

Video Lens is one of the freshest pieces of work from the User Interface group at Autodesk Research. Something really interesting is that this uses baseball as the foundation for this exploration into big data and analytics.

Autodesk Research Video Lens User Interface Overview
The Video Lens User Interface is Easy to Use and Provides Real Time Feedback

As we know from stories like Moneyball, sports analytics is a serious business and lots of data is captured. The team worked with about 8000 video clips and the metadata captured by the PITCHf/x toolset to quickly visualize what happens in the game of baseball and analyze specific plays. Video Lens had four design goals:

  1. Maximize Information Desnity: Reveal as much as possible about the underllying data
  2. Treat Events as the Primary Entity: Events are the important parts in the data; each data source (video clip) may have multiple events
  3. Support Rapid Playback of Video Results: If you have to wait for things to happen the system is of lower value
  4. Encourage Exploration of the Data: trying things out are low cost

Imagine you want to see all the knuckleball pitches thrown by a southpaw that were high and to the right. You would circle the target in the pitch zone UI, mark the pitch and pitcher type. Three quick steps in this interactive tool and all the appropriate video clips play back.

Autodesk Research Video Lens Strikezone User Interface
The Video Lens Strikezone User Interface Allows the User to Quickly Isolate Specific Pitches

Want to narrow that down to the team you're facing next week? 

Autodesk Research Video Lens Teams User Interface
Autodesk Research The Video Lens Teams User Interface

What if you were a sports commentator and had to fill in one of those awkward moments where a streaker was being escorted off the field? You could go to the video Lens and talk about potential strategies with real time feedback to illustrate. Or maybe you just want to see how many balls are hit to the area of the field your streaker was spotted in and talk about the chances of him being hit by a live ball

Autodesk Research Video Lens Field User Interface
The Video Lens Field User Interface

Intrigued? Here's a video showing the Video Lens system in action:

Taking this back to Autodesk and the possibilities for use in other tools, here are some possibilities:

  • TV broadcasters looking for B roll footage in their database of a rainy day at a famous landmark
  • Film editor sorting through possibilities in their storytelling
  • Advanced learning possibilities for tools like Autodesk Screencast
  • Connecting security camera data to your BIM
  • Connecting video recordings of your real work product stress testing (eg crash test dummies, force testing chairs, etc.) to your digital product design

We think Video Lens is a great toolset for so many things. Let us know what you think here on the blog or on our Video Lens Research page where you can also find more info and contact information.

By the way, if you happen to be in Hawaii around October 8th, please stop by the UIST Symposium where we'll be talking about Video Lens in person.

Play ball!

Come try Project Draco - our drawing/animation/effects innovation mash-up - at the SIGGRAPH 2014 Studio

We'll be showing Project Draco at the SIGGRAPH 2014 Studio in Vancouver and you can get your hands on it! 

SIGGRAPH 2014 Studio 

Here's how SIGGRAPH describes the SIGGRAPH 2014 Studio:

The SIGGRAPH 2014 Studio is a collaborative working environment where the latest technologies and brightest minds come together to learn, experiment, and create. Attendees of the SIGGRAPH Studio explore wide range of new techniques and media with help from experienced hands. Attendees experience the latest in 3D printing, modeling, and animation software. The SIGGRAPH Studio is made for anyone interested in cutting-edge DIY gadgets, quasi-objects, Rube Goldberg machines, user-generated content projects, software, hardware, processes, workflows, technologies, etc.

Like any good SIGGRAPH experience there are moving pixels to describe the space and event. We're very happy to see Project Draco as the lead piece in this movie!

We'll be in the Studio Sunday through Thursday and have put our offer to you in the form of a short poem:

please stop by

give Draco a try

help us clarify

how we may supply

this technology

to girl and guy

In other words, we are looking for feedback on the technology and seeking input and ideas from people on what to do next. Letting you get your hands on it should help with that.

Project Draco

Project Draco comes from the User Interface Group here at Autodesk Research. Draco mashes up illustration, animation and effects in an easy to use interface. Fast Company says "Autodesk's Draco Lets You Animate An Illustration In Seconds". Take a look at the following illustration. There are a lot of simple elements in a picture that can be made to move and create an even richer experience for the viewer.

Autodesk Research Project Draco Animated Elements

Autodesk is well known for tools like Maya, 3ds Max and Flame that let people create similar things. Project Draco takes some of these concepts and makes them available to just about anyone. The learning curve is very flat and the immediate feedback is incredibly gratifying. Have a look at the intuitive sketch-based workflow:

Autodesk Research Project Draco Workflow Diagram

 More examples of the workflows and things you can create are available in this short movie:

We hope to see you at SIGGRAPH. If you can't be there in person, take a couple moments to let us know if you're interested in testing Draco in the future.

Autodesk Screencast: From Idea to UI Research to Project Chronicle to You

ScreencastLogoHopefully you've heard that Autodesk Screencast is a new tool that lets you capture your workflows, to easily create powerful and engaging learning materials. What you may not know is the history of how this tool came to be. 

Way back in 2010, Tovi Grossman, Justin Matejka and George Fitzmaurice from the Autodesk Research User Interface Group published a paper entitled Chronicle: Capture, Exploration, and Playback of Document Workflow Histories

Chronicle started with the idea that the majority of tools today support undo functionality. The undo queue has a list of the commands that have been executed by the users and is therefore something that could be utilized to playback what the user did for others to learn from.

From that idea, there was exploration around how to improve the video playback experience. As video is a visual experience, it was important to give the user more insight into various parts of the video, as you can see below in the Chronicle prototype built into Paint.NET, with images showing what happens at various stages and a rich timeline referencing different events. Having a such a prototype allowed the group to test the concepts with users, measure the success of the tools and refine the workflows.

Autodesk Research Publication Project Chronicle User Interface

In reviewing the Chronicle functionality with the test users, the feedback was very positive and suggested for:

  • Team Support: review how a colleague carried out tasks to understand the current state of a document (e.g. for trouble-shooting)
  • Implicit Learning Aid: when working with publically shared documents, the user could review the associated tools and workflows (e.g. comparing software versions) 
  • New Tutorial Format: this is a much easier way to create tutorials
  • Self-Retrospect: help a user to remember how they did something or what their tool settings were 

With this in mind, the Autodesk Research Transfer group was engaged to help bring Chronicle to a wider audience. Project Chronicle was released to Autodesk Labs, our place to share innovative new technolgies in a way that we can collaborate with our users, for more people to try in the context of AutoCAD, Inventor and Revit.

Autodesk Research Project Chronicle Banner from Autodesk Labs

During this time, the toolset and interface went through some refinements (you can see a little of that in the above image). The user feedback continued to be positive and the Autodesk Knowledge Network stepped forward to make Project Chonicle into an official tool and rebranded it as Autodesk Screencast. Here is a nice overview:

The journey from the initial spark of an idea to finished tool can take patience and many hands. With Autodesk Screencast, we hope you'll agree that it's worth it. Download Screencast now for Windows or Mac and give it a try!