Finally the final part of the 3d blog. As I mentioned in the previous posts on 3d, there are very poor tools for creating great looking 3d user experiences. This article will not concern itself with the aesthetics of 3d or the usefulness, utility or desirability of 3d UI’s, for discussions on that see here.
First off I’ll do a quick overview of the basic tools for creating 3d assets before diving into the tools for creating interactive 3d for different kinds of user experiences. This is a high level guide to the topic and in no way attempts to be exhaustive, just hopefully helpful.
Tools for digital illustration - Just creating great looking 3d.
The biggest tool provider for anykind of 3d anything is Autodesk. They are the million pound gorilla when it comes to 3d tools. Over the last few years they have swallowed up what used to be the top 3 tools in the 3d world, Autodesk SoftImage (formally XSI), Autodesk 3ds MAX and Autodesk MAYA - all previously rivals (stunning the DOJ where asleep at the wheel for those purchases) . These are considered the mega premium tools and have almost religious followings in the 3d community from game makers to film makers but to be honest from my perspective as a 15,000 foot generalist they are more or less identical in terms of capabilities. They are also ‘huge’ from a time investment perspective as they combine multiple sub tools, for animation, modeling and rigging, physics, creating shaders, lighting and rendering. Whilst they are widely used and regarded by moviemakers and game-makers there are plenty of much less expensive alternatives that can do 90% of the same basic things.
Sooner or later you will want to visit Turbosquid - it’s a huge 3d web marketplace for models, shaders, plug in tools and tutorials some of which are free.
COLLADA™ Is an open source intermediate file format that defines an XML-based schema to make it easy to transport 3D assets between applications - something that you will quickly discover is a god send. This includes geometry, materials, textures, lights, camera, animations, instantiations, and scenes. COLLADA also includes skinning and morphing for character animation. Shader effects for several shader languages as well as some physics support.
Once you’ve created or bought your 3d models, products like Autodesk Showcase include a fancy real-time 3d engine which allows you to move around high end renderings of industrial design models in real time, complete with soft shadows and high end shaders. (Car interior image below is showcase)
This tool is primarily intended for doing ‘pre-visualization’ and design review of industrial design forms using the power of the high end video cards and is a glimpse at what’s possible with the latest rendering engines. They also include the ability to play with libraries of pre made common materials, and environment lighting and immediately see the result, which has historically been a massive time waste with traditional renderer’s.
Hypershot Bunkspeed, is a quasi-competitor to Showcase allowing you to do semi-real time high dynamic range lighting, (HDRI). HDRI is a technique for doing image based lighting, which is how you get something to look photo real. The product is mesmerizing, super easy to use, and really useful for illustration. Simply drag the high end shaders onto the model, pick your image based lighting and (image below) is what you get. The soft shadows and ray tracing capabilities make the output look spectacular.
There is also a plugin for Google’s Sketchup tool that uses Bunkspeed. Sketchup is the world’s easiest 3d box-modeling tool, ideal for quick 3d sketches of physical relationships; bunkspeed kind of makes anything look hot with the right HDRI image choice.
RTT is a technology company making another one of these real time 3d rendering Showcase like tools. I find this type of product interesting because of how they clearly demonstrate what is possible with real time, high-end rendering. RTT also offers sever based rendering solutions which can deliver rendered frames to less capable hardware in semi-real time. This allows car configuration UI’s to be done remote from the client. (TT image below is real time)
3d Real time UI Rendering API’s
OpenGL® ES is a low-level API for advanced graphics using well-defined subset of OpenGL. Open GL has largely become the standard API for doing high end 3d outside of the PC and XBOX platforms which are dominated by Microsoft’s competing Direct X technology. GL-ES provides a low-level applications programming interface (API) between software applications and hardware or software graphics engines. Apple developers have the option to use open GL ES on the I-phone and now the I-PAD for example.
The wizzy bookshelf spin into the bookstore effect from the IPAD launch video, another example of 3d metaphors successfully working to add some bling to a product announcement.
These API’s make it easier for developers (mostly games developers) to write code that is slightly abstracted from the exact hardware it will run on, making it more compatible with a wider audience.
WEBGL - WebGL is a cross-platform, royalty-free, open web standard for a low-level 3D graphics API based on OpenGL ES 2.0, exposed through the HTML5 Canvas element. The basic premise is that most browser makers (except IE perhaps) are working together to enable basic support for this as part of the HTML 5 specification, which is still a year or two away from prime time. These open standards would go along way to making 3d UI not require a specialized engine plugin (like Flash or Anark); many people are hyped up about the potential for this to finally let the 3d cat out of the proverbial bag.
Nvidia’s TEGRA 3d UI
This 3d UI (carousel image shown in the article header) was shown in working from June 2008, and generated significant excitement that a 3d touch based UI that looked reasonably hot would actually become a commercial reality. Turns out this UI will not apparently see the light of day according to this report. I believe the UI below was authored using the now extinct tool ANARK game face which was bought from Anark after having been a tool for implementing 3d UI’s in games for a several years.
Tools for building 3d UX Today.
Today as a designer wanting to create a real time 3d interactive experience your choice is pretty much limited to Adobe Flash with one of the various plug-ins, like Papervision or Away 3d. Inherently Flash is not a great 3d platform basically because the first thing you should know about 3D and Flash is that Flash doesn’t really support 3d. What Flash does know is how to display vector shapes on the screen and how to calculate math expressions. With that, 3D can be faked by doing really sophisticated mathematical transforms. Generally this approach is ok for minimal 3d transitions and simple geometric models but as soon as the going gets rough (say you need high polygon models or shaders) designers resort to non real time 3d and interactive pre-rendered 3d Flash movies. Still, Flash 10 or one of the plugins mentioned above is really still the de facto option for developing interactive 3d sites today albeit they are often less than spectacular and don’t really take advantage of any of your specific GPU or video card hardware (features like anti-aliasing and texture smoothing are missing). One of the worst qualities of flash based 3d is the menacing ‘jaggies’ of aliased fonts and geometry, and the ‘sparkly’ effect that is created with non smoothed textures. The creative industry is obviously heavily invested in Flash, so if Adobe keeps plugging away it’s likely 3d will become easier and better through their classic and venerable authoring suite.
Electric Rain are a small software company who make Swift 3d which is a well respected modeling and animation program for Flash. It takes 3d rendered geometry and does something like the ‘live trace’ feature in illustrator, converting it into less memory consuming 2d vectors. For canned 3d effects it’s an easy and well travelled solution for designers.
Scaleform are a company plugged into the Games community with a tool that enables them to support Flash movies inside proprietary games engines, a great deal of modern games use this technology which gives them the, coding, production and animation benefits of Flash inside their rendering engines, thus making the production of sophisticated menu systems, Hud’s, (text, maps, equipment status, health etc) or the configuration UI’s much easier.
Expression Blend is a UI building tool produced by Microsoft which can create 2d or 3d interfaces for Microsoft’s client application engine WPF, or its online cousin Silverlight. Expression Blend is ‘good’ for building 2d UI, however when it comes to 3d we are back in the dark ages. It can display and manipulate 3d models with simple mapped textures but every time we’ve tried projects utilizing 3d - we have quickly run into performance challenges (even on high end hardware). There’s no support for any sophisticated effects, shaders, anything more than the most fundamental lighting, no decent animation support. There are some 3rd party tools to make life easy on WPF. Another Electric Rain product (I’ve not used) ZAM 3D is a 3D XAML Tool for Microsoft Windows Application Development. It provides developers and designers with a quick and easy solution for creating 3D elements for Microsoft Expression Blend and Visual Studio projects. It also acts as a 3ds or dxf to XAML converter.
It seems increasingly clear to me that Microsoft has de-invested in WPF, and I’ve given up hope that it would become a decent way to create a 3d UI. In all the hype about WPF at its launch, there was some good demos and excitement, yet here we are 4 years on and really there are no decent product examples of WPF applications, and very few announcements from Microsoft about its roadmap.
There are a few other choices out there like Anark, which is a long in the tooth tool from yesteryear which looked very promising as a way to author 3d user experiences about 10 years ago, but today seems to have been abandoned by its parent company as a super niche tool. We’ve built some incredible 3d demo’s in Anark but the tool seems to have not received a significant update in a long while and it’s expensive.
ARToolKit is a Cross-platform Library for the creation of augmented reality applications, developed by Hirokazu Kato in 1999 and was released by the University of Washington HIT Lab. Currently it is maintained as an open source project hosted on Source Forgewith commercial licenses available from ARToolWorks
ATOMIC Authoring Tool is a Cross-platform Authoring Tool software, for Augmented Reality Applications, which is a Front end for the ARToolKit library. It was developed for non-programmers, to create small and simple, Augmented Reality applications, and released under the GNU GPL License. I’ve not used it and can’t speak to its usefulness.
The Astonishing Tribe, also known as TAT have a 3d development environment I’m curious about, its called Kastor, and I think works with TAT Cascades which is a UI framework. Some of the demo’s mentioned in my previous post’s on 3d were probably developed using this tool set. The Red-dish demo (below) they did last year utilized OPEN GL ES 2.0 shaders and anti aliasing functionality.
Kanzi is a suite of tools and technology for 3d UX development. The Kanzi SDK tool is designed to be used with OPEN GL ES on mobile devices. Its Rendering engine Kanzi Engine is supported by a range of silicon manufacturers. I’ve not used this tool in a professional context yet, but the examples on the site suggest a mature product with many of the capabilities for building good looking, hi quality 3d. At the time of writing this, I just recieved an email from a Finnish company called Rightware that has just acquired Kanzi from Futuremark.
Images above from Kanzi demos above.
I’ve purposefully left out discussing the merits game engine technologies like Unreal, ID- Tech and Valve’s Source engine, Microsoft’s XNA and many others which are less ‘tools’ and more like ‘starter kits’ for the development of 3d games, they handle low level features but they are really only for those conversant in C# or lower level languages and not really optimized for building user experiences or more general products.
I’m very interested if you know of other commercial tools for building UX and will update this blog with suggestions and comments when I receive them.
Tools for building 3d UX tomorrow.
At heart the people using UX building tools and contemplating 3d UI concepts are desperately in need of tools that allow them to ‘play’ in high performance real-time 3d quickly and easily. The lack of these tools, I’d argue is the main adoption blocker for seeing more high quality 3d user experiences. Occasionally I see something really impressive in 3d that makes it more clear what we are striving towards in terms of output from such tools. At the Intel booth at CES 2010 was a very impressive dual touch wall (Image below). This was not created (to my knowledge) with a cutting edge 3d tool but it demonstrates the kind of output UX designers like ourselves at Artefact are striving/desiring to create.
Check out a video of the animation on Youtube. This would be impossible to build in any tool that I’m currently aware of and had to be coded from scratch.
Watch this video of someone creating a “little big planet” level. This is a tool that comes with the game for players to create additional content and share with the community. You use a PS3 controller and a character to create levels within the game. Note in the video that there is no difference between building the level and being in it which makes the authoring process very experimental. Objects are ‘physically’ enabled by default. There is minimal difference between the “authoring process” and the “experiencing it” process. This tool is actually very simple to learn, super complex to master. It’s also a modern work of art.
This tool represents a kind of design target for me, today it feels especially for UX like we are down where C++ developers where years ago, down in the weeds creating everything from scratch, specifying every possible variable and property. The desire is to have a palette of starter objects, 3d controls, shaders, lights, behaviors and effects that are a starting point to start sketching great user experiences. When my sketch work is complete, allow me to dive in and get busy scripting custom behaviors, defining custom shader’s and geometry.
The complexity of 3d tools makes production a very deliberate, detailed and labor intensive process. As a result designers are limited in their ability to experiment and iterate in building 3d user experiences. So my ideal tool combines lots of ‘starter objects’ with the ‘stateless’ UI demonstrated by Little Big Planet and Autodesk’s Showcase application. With such a tool I think we’d be pretty close to the kind of tool that would take the world by storm. (Quietly I’m hoping someone sends me a link) Contact me if you want Artefact to design such a tool, right now that would be a dream project.
One of the more obvious trends at this year’s CES was 3d TV, all the display manufacturers had major installations showing off their 3d HDTV offerings. The CE industry seems to have collectively decided that 3d is the next big thing and that consumers will go for it in the home. The mood at CES from the attendees I encountered however seemed overwhelmingly skeptical about what seems to be a supply side (industry) push of a new technology that is just not ready for primetime. There is an obvious need for the CE industry to repeat the success its had in recent years getting consumers to buy into big flat screen HDTV’s and they are obviously trying to get us to all upgrade again in the next few.
The skeptical arguments I heard landed in four camps, the effectiveness of the technology, (is the effect cool enough?) the content pipeline (Is there enough 3d content? ) , the cost of new glasses and displays?, and finally the glasses make me look like a ‘dork’ factor. (I must look like a right plonker wearing the oversized polarizing glasses over top of my existing glasses).
Almost all the demo’s I saw currently utilize the approach that requires polarized glasses to see the intended 3d effect. Let me tell you if you’ve not seen it, the effect of the 3d on these big displays with glasses is pretty compelling - not as dramatic as the first time you saw full HD, or say the transition from Color to black and white. But with good authoring the effect can be pretty cool and it’s not hard to see its significantly better than HD as long as you are wearing the glasses.
The Demo’s I saw of non glasses 3d relies on lenticular lenses overlays and multiple image perspectives being rendered at the same time for you to see the desired effect. They have a ‘sweet spot’ of convergence where your eyes are far enough apart to see 2 slightly different perspective images and the effect is cool if you are in the ‘zone’. If you are not in the zone you need to move your head from side to side to see the different perspectives. One technical problem with this technique is the need to render several different perspectives of the same image. This is fine for CG 3d game content, but 3d cameras with more than 2 lenses don’t exist to my knowledge, not to mention there are no standards or formats to support transmission of multi-perspective video content.
The lack of 3d content problem is non-trivial issue but something the industry from broadcasters to camera makers and TV manufacturers are probably prepared to invest several years to fix, (the way HD content eventually became commonplace).
Toshiba’s alternative approach is to convert 2d content into 3d on the TV in real time utilizing its just announced ‘Cell TV’. The Cell TV has ten times the processing power of most personal computers. All the computing power is used to convert 2d HD video into 3d on the fly which is a pretty impressive feat but starts to raise the obvious question of cost. As an aside the Toshiba Cell demo had so much computing power that they also used a magic wand controller (like the wiimote) for selection and manipulation of their 3d UI. The UI (below) was a vision of 3d UI hell, a nested sphere of spheres with no order, rhyme or reason. (See my 3d UI blogs)
Sony was demoing some 3d Playstation games with a PS3 firmware update they said would be available soon, games are an obvious way in for the likes of Sony to create a compelling amount of content with relative ease. The driving game I played was pretty cool the 3d seemed to work really well with a 3rd person camera perspective as good 3d requires, decent foreground, mid and long distance content to ‘pop’.
As far as cost goes, on top of the $2-3000 or so you may spend on a 3-D set, you’ll have to pay extra to buy or rent 3-D movies. Apparently Sony plans to start selling sets labeled “3-D ready,” meaning they can be upgraded to 3-D later with some additional costs and accessories. I don’t know about you but I’m happy to have just fairly recently upgraded to a nice HDTV so I’m going to be in the market for a newer better one for five or so years. I also have not heard anyone rule out the possibility of having the 3d effect simply be a standard HD video signal from an external box which could obviously reduce the cost of the upgrade to 3d significantly.
According to the latest data, around 126 million citizens of the United States of America currently wear eyeglasses. Since there are around 300 million people living in the United States, that’s almost half of the population. Looking like a Dork with two pairs of glasses on aside, customers of 3d who currently wear glasses will need to make a big behavioral shift (swapping glasses or adding an extra set) which I believe they simply not do - the effect is neat but not that good. Maybe optically correct 3d polarized glasses will become the norm at the eye doctor?,( let me hear a collective sigh of skepticism. ). Also for some of my colleagues at the show, the glasses seemed like a source of much frustration claiming they were causing strain and would lead to exhaustion. I think an additional negative of the glasses is the scenario of gathering a group of folks around to watch the ball game, with everyone having to wear the glasses or else what they see is a blurry unwatchable nightmare. Talk about a nerd fest.
I for one don’t think glasses based 3d HD TV’s will become broadly adopted. These devices will become an option for the high end technophiles who have to have the latest thing and perhaps loner gaming enthusiasts but broad based adoption will take many years as current flat screens are too good to warrant the upgrade for most people.
I think there’s enough technical and infrastructure challenges that perhaps 3-5 years from now we may start to see some major commercial adoption of non glasses 3d HD. Eventually I expect that 3d cameras like Microsoft’s Natal will track our gaze with sufficient resolution that a more convincing 3d can be created ‘in software’ with no glasses or requirement for new display devices. Until then 3d displays are and glasses are not for me.
This is the third installment of my blog on 3d. The first part outlined the gist of the story, The second part was about the aesthetics of 3d. This installment deals with the use of 3d to convey a ‘conceptual model’ for a user interface. In this article I’ll identify different approaches to building 3d user interfaces and ask the question are they useful?, usable?, and/or desirable?. I’ve provided tons of examples partly to make the point there is a ton of activity in this space but not a great deal of discussion around it and of course a terrible lack of tools to create it. But of course the real reason I’m writing this is ‘Just saying ‘Z-space’ raises your IQ 10 points.’
When talking about 3d conceptual models of different user experiences the litmus test of ‘usefulness’ is that the third dimension the ‘Z’ is conveying some additional data or meaning that a 2d representation of the same thing would not have. Presumably this additional information is of significant value, and relevant in helping people understand it.
When it comes to ‘usability’ of 3d UX, it’s a big topic. When using 2d input devices, the design choice of free-form cameras v constrained cameras can have a big effect on the usability of 3d user interfaces. By free-form, I refer to giving the user the controls to point the camera in any direction. This is useful in 3d environments (interiors), but comes with the baggage that the user must now construct a mental map of where everything in the space is relative to any other point in space.
Constrained looking of the camera on the other hand limits the ability for the user to look in certain directions. This ‘Cone’ of visibility usually has the advantage of keeping the user focused where they need to be and stops them becoming disoriented. This may not sound significant but the difference between these approaches makes a massive difference to ‘usability’ of 3d interfaces.
Similarly enabling the user control of free or constrained movement of the camera in space has a big impact on usability too. Here’s some useful advice:
If you give the user free form ability to point the camera in any direction – think twice. If you decide its for the best then constrain their movement of the camera through space, the easiest constraint is just giving them X and Y movement which is familiar to all users. If you give them the ability to move the camera through Z space then constrain where they can point the camera.
One more: If you’re building an interactive 3d thing don’t choose ‘both constraints’, (i.e.: no ability to move the camera or move the viewpoint) this defeats the entire point and btw you should be in the movie industry.
If you are working with exteriors looking at objects from the outside, its sometimes useful to constrain the cameras target to the center of the object and then giving the user the freedom to ‘orbit’ the object as they see fit. This is often used in car configuration UI’s. Being able to zoom in (rather than moving the camera) gives the user the ability to see more detail without becoming disoriented. Other common approaches here are to allow the camera to move in and out of close ups, and when in the close-up providing a button to escape from this mode.
This is called putting the camera on ‘rails’ for transitions from one place in space to another allows the user to specify a different item to orbit around, perhaps a different car in the showroom. Even if you have rails, and constraints in full effect, giving the users camera ‘a few degrees’ of freedom is a nice touch often this is accomplished by mapping the position of the cursor in the scene to move the camera a few degrees as it moves around the screen.
This portfolio site shows good use of animation around a simple hierarchy. The camera utilizes limited degrees of freedom for camera movement, but keeps the impression of control over the user. 2d text instructional cues on the 3d objects also help the usability.
Full six degrees of freedom, (up, down, left, right, Forward, backwards, pitch left/right, yaw left right,, rotate left, rotate right) is rarely useful, turning off as many of these as possible generally improves usability.
That’s not to disregard ‘desirability’, I said about aesthetics in the previous article, sometimes just because it looks great is enough to make a design sufficiently novel, different, interesting, at least in the short term. Desirability in a crowded market can be the stand out best place to innovate.
2d Cards in 3d space
Take an inventory of all the ‘digital’ things we have in our lives and the list can get pretty long, for example; Songs, documents, spreadsheets, messages, photos, movies, contacts, conversations, bookmarked locations on the Internet, blogs, appointments, applications, places of interest, lists of things to do. Before our digital lifestyles these things had physical precedents, type is 2d, illustrations and photos are 2d, audio and video is represented by 2d box covers, calendars, books, business cards essentially are just collections of 2d sheets.
No surprise then that our first category of 3d UI’s simply put these 2d digital things onto ‘cards’ and then put the cards into some kind of metaphorical, logical or ‘real’ spatial environment. The cards themselves add no dimensional data but presumably their position in three-dimensional space conveys some additional meaning.
I’m going to call this 2d ‘Cards’ UI and there is allot of it out there. Cards don’t literally have to look like cards they simply are using 2d text or images to convey the content.
I’ve broken it down into four reasonably distinct uses of cards, although this taxonomy is just for discussion rather than an attempt to be authoritative on the topic:
Cards in metaphorical space
Cards in logical space
Cards on geometric space
Cards in ‘real’ space
The second category of three-dimensional UI I’ve seen uses data that is naturally 3d, for example; models of physical objects, buildings, maps, bodies, space, environments are of course the simplest to understand 3d data types. They are relatively abundant on the web and already part of our mainstream digital lives. Second life, Virtual and Google Earth, Google Street view, and almost all video games use 3d effectively in this capacity.
Here the creative discussion is about creating immersive environments that need to be explored by the user because the data is naturally spatial and best understood from multiple perspectives and vantage points, or the creative team want ‘exploration’ to be part of the core reason for the user experience.
I’ll call this 3d ‘objects’ UI and like above I think there’s at least three important variants of it.
3d objects in Metaphorical or virtual space
3d objects in logical space
3d objects in real space (Augmented reality)
2d Cards
1. 2d Cards in metaphorical space
The basic idea in this example from Microsoft Research (1997) was testing people’s ability to organize and remember activities when mapped to a metaphorical physical environment. The idea was to tap into our spatial memory and perception as the ‘added value of 3d’. Skewing the windows enabled you to see more windows at once, but this two can be done effectively in 2d without the ugly skewing distortion that’s occurring below.
Variants of this approach have continued for years substituting walls and corridors for cities, art galleries, hillsides, abstract environments of all kinds you name it. As the years have gone by the windows look better, the environments are cooler, but fundamentally this isn’t going anywhere.
Put yourself in the user shoes for a moment and consider about the amount of ‘organization’ it takes to create a custom environment where activities or windows or collections of digital things are positioned in an environment. Then think about the extra time, (and controls) it takes for me to navigate around in this space and basically the UI is starting to obstruct the user’s primary task, its getting in the way of whatever I was trying to do in the first place.
This organizational burden, place on the user demands they pay attention to the positioning of various things in space. Some research studies have proven that user recall has improved when things where spatially presented in environments like the one above, however perhaps they ignore the added time it takes for people to position, navigate and conceive the appropriate organization of these things.
The logical conclusion of this approach 3dna’s Desktop application for Windows gives you a complete spatial environment filled with lots of 2d pictures as links to various windows you have open, organized into metaphorical objects and posters.
The marketing banter for this company talks about the ‘Fun’ of it all, which I suppose if you like this kind of thing is about the only thing you can say about it.
Useful – So as far as ‘usefulness’ this approach is fraught with the organizational overhead management, and raises questions about multi-tasking and task switching that are irreconcilable with physical space. (Alt tab, or Expose are fine ways to move from task to task, walking across a room gets kind of old once the metaphor is established.). Other questions mostly around the scalability of solutions like this (how many windows can it handle before the metaphor breaks down).
Usable – Usability here is a mixed bag, full camera movement systems, and manipulating things around in space obviously put a burden on the usability of these systems, but at least the metaphors are ‘understandable’ which gives them familiarity to less experience users.
Desirable – Best case the desirability of this technique is limited to niche appeal, technology history is littered with companies big and small (Microsoft Bob, General Magic) who attempted to extend the computing metaphor into a more literal spatial environment. If you follow this approach no doubt a small community of hardcore fans will love you playing house, but mainstream consumers don’t’ really need this.
2. 2d Cards in logical space
Three dimensionally sorted stacks of 2d cards have quickly become commonplace in adding 3d to our user experiences, on desktop, mobile and console devices. Can you count the number of carousels, walls, curved walls, piles, boxes that have shown up in our UI’s recently?
A toolkit above from Fotoviewer provides a variety of different card arrangements for images. All these owe a great deal to ‘Coverflow’ ,which was originally developed by an artist, Andrew Coulter Enright and was purchased by apple in 2006.
Bing’s new visual search using shelves
Clever use of slight overlap and perspective convergence gives us the layout benefits of getting more items on the screen, than a standard 2d Grid. This is straight forward basic use of 3d, no conceptual headaches, simple to understand, takes advantage of 3d layout, and is visually ‘novel’ something Microsoft needs to help it convince users its different/better than Google. Also in the same vein is Search me. And Safari’s 4 Top site view. Also worth a mention is Cool Iris; and Sun’s dead Looking Glass project if your interested in the history.
AT&T’s Pogo Browser
2d Cards, showing search history. Safari 4 uses Coverflow for the same functionality.
Useful – So is this Useful? They can be more efficient in showing more content than 2d views if careful attention is paid to layout. The third dimension for these cards is used to convey no additional information than a 2d sorted list, but distance away from the camera typically conveys some kind of sort order. On balance somewhat useful for interaction designers where visual searching through limited numbers of things is required.
Usable – From a usable perspective, these systems typically don’t rely on any camera movement beyond what we are already familiar with in 2d UI, (Zooming, X and Y translation.). So there’s no inherent complexity being added to these kinds of user experiences.
Desirable – From a desirability standpoint, the swift proliferation and success of Coverflow, carousels and systems like Cool iris is hard to deny in making our commonplace UI mechanisms slightly slicker.
There’s a community of information designers and developers out there working in scientific fields and on complex data visualization. They are building tools that enable very large data sets to be processed and visualized in flexible ways.
Some of the work of this group is fascinating; ground breaking stuff, and visually amazing. Every once in a while someone dabbles with 3d to ‘help’ visualize the complexity. The results are almost always spectacular failure of clarity. The raises an important issue about 3d UX of all kinds and that is, static representations of these networks are truly an unfair way to judge them. Unlike static 2d posters, which are designed, for comprehension from a static point in space and time, 3d becomes infinitely more valuable when it uses time (animation) and interactive movement around the 3d space to understand its topology.
Useful – Generally, Usefulness, I’d argue that these super complex visualizations of nodes, networks, social hierarchies and systems are ‘best case’ marginally valuable to a small group of IT professionals or social scientists.
Usable – From the perspective of Usability, they are disasters for all the reasons I outlined in the camera notes at the beginning – they literally require full freedom camera control. (Ed. common I’m being provocative here… someone send me a good example please).
Desirable – There is some aesthetic desirability to this approach for designers. I can’t help seeing these kinds of things and like a moth to flame, drawn in by the promise of ‘understanding’ data in a more visceral and immersive level. I think however on balance most consumers would look at these kinds of experiences and blanche.
3. 2d Cards on geometric space
Somewhere between, cards in metaphorical space and logical space is putting cards on geometric primitives which are not really 3d objects because there’s no added information to the additional dimensions this brings. Personally I hope to never see another 3d browser application that maps webpage’s or content onto cubes.
I’ve seen so many examples of this recently, There’s really nothing to say here but ‘stop it’. Why are you doing this???
There’s always some exception’s however. Here’s Absolut world a Paper vision extravaganza this physics powered UX, shows a dynamic site that has great music/art direction. Opening the cube’s is a really nice touch revealing nice shiny horizon/dioramas. Because the geometric cubes are objects not environments this approach to 3d UX is easy to understand and lots of fun to play with. I could even see this inspiring more productivity related scenarios.
Apple filed a bunch of patents with these schematics showing a 3d desktop last year. This patent is a bit mysterious but you can see again the idea that you are looking at the inside of a box with a floor, walls, ceiling and background.. 3d geometric primitives are shown on the tray at the bottom, utilizing ‘stacks’ of 2d content. Apple has been aggressive adopter of 3d with Time Machine, various dock visuals, Cover Flow and even specific spinner controls in the iPhone widget library. I doubt this design will see the light of day as described.
Bump top - 3d desktop
Really how much time do you spend organizing files on your desktop? The answer: lots more if you have something fun like Bump top to waste your time. The genius work here, is in the gestures, this product should have remained the cool demo it was four years ago, or it should have been a product 10 years ago, back when the desktop really was still a relevant place for innovation. Physics integration is nice but re-creating the imperfections of the real world (messy collisions) I can do without. Apple invented Piles, back in the early 90’s. Which is the coolest aspect of this whole product.
Useful – Is 2d content in Geometric space a useful approach? The ability to contain things and organize groups of things into relatively abstract containers like cubes is somewhat useful although frankly arbitrary. (The Absolut example is perhaps the best example here). Angled panes on walls reduce the real estate required for items but can create ugly skewing artifacts.
Usable – If the Camera remains static relative to the objects this approach is a simple extension of 2d techniques. The moment the camera starts navigating around or inside the geometric primitive all bets are off for people maintaining a useful mental model of the space.
Desirable – I’m not sure about the Desirable? Personally I don’t like the aesthetic of boxy environments too much. Time will tell if Bump top reaches any kind of commercial acceptance similar to the response it garnered from the audience at TED.
4. 2d Cards in ‘real’ space
Potentially one of the least explored ideas is putting 2d cards into real environments. ‘Acrossair’ has a nice demonstration of a way finding application for the New York Subway system that overlays cards (and arrows) onto the iPhone camera output. It uses compass and GPS information to determine where the camera is looking.
This is likely to be most explosive area of growth especially given the excitement around augmented reality applications. Take the amount of data that has real world data and its not hard to see applications of this everywhere from advertising, shopping, way finding, social networking to real estate.
One additional mention for Photosynth which takes 2d photos of things and creates a navigable 3d space. I have to say like everyone else when I first saw this I was amazed more by the ‘deep zoom’ than the stitching element between the photographs which sort of ‘recreates’ a three dimensional browsable space.
3d objects
1. 3d objects in metaphorical or virtual space
Obviously this is the realm of maps and games but its worth calling out a few of the many examples that continue to shape our digital experience.
Mapping applications like Google’s streetview or Bing maps are well on the way to providing a photo realistic version of our world that even 10 years ago would have been considered science fiction. The latest engines are doing automatic placement of 3d vegetation based on infra-red satellite images, actual weather information in the sky and perspective correction on the sides of buildings. This opens up the opportunity for virtual tourism. Almost full camera control is offered to the user that makes the systems clumsy to use from a humans eye line perspective. Today the content in the mapping application is mostly 3d buildings and vegetation but expect 2d content (call-outs, advertising, etc) to show up in the near future.
There have been and continue to be lots of attempts to do virtual shopping malls. Reading the blurb on these company’s sites is a little depressing. The unbridled marketing fueled enthusiasm they have for what they think is a new breakthrough in shopping experiences. In essence these sights are just fancy link catalogs.
This represents the next generation of retail, through a virtual environment; it’s original, it’s exciting and it has amazing potential, … The 3D visualization technology means consumers can enjoy the experience of shopping at any time of day or night.
Another attempt to make the web 3d, this one goes slightly beyond shopping and at least has a slightly distinct visual style. Really though how does this scale? I cant really imagine a mall that’s got hundreds of stores without them requiring, maps, directories and search which of course begs the question why bother with the whole spatial metaphor of a mall in the first place.
The R.U.S.E demo shown at E3 is a nice demonstration of multi-touch and 3d gaming, seriously watch the interactions in the demo and you’ll see an expert at work. This demo jockey is using a palette of learned gestures, from two-finger drag to two-finger spread to one finger arcing to change the camera perspective. Selection of objects is done with a two-finger poke drawing a selection rectangle. I’ve yet to play a RTS game on a multi-touch surface, its sure to be tiring, but this feels like the perfect use of all those gestures we’ve been seeing on surface tables for a couple of years now. Games are obviously fine with pushing the ‘usability’ of these systems, as learning them is part of the conquering the challenge of the mission.
Above, T.A.T’s recent ‘conceptual’ exploration of a 3d mobile phone UI. The UI is made to look physical (80’s plastic toy buttons and carousels). The camera moves around a ‘scene’ of widgets and various pivots are shown working in a kind of inspector carousel with filters. These guys have built their own rendering engine and tools pipeline to be able to do this all in real time and the output looks great especially for a mobile platform. The novelty of this makes the reaction of most technology people ‘wow’ but this is a deeply impractical UI.
When you watch this movie notice how the demo jockey has curved his finger to move the carousel… (cheater). This illustrates one of the fundamental issues with all these dimensional presentations of content and that is our input systems are still 2d. Fingers and mice don’t really map that well to manipulating 3d objects in space and until someone invests some kind of magic haptic feedback its never going to be easy.
Useful – Perhaps more than other examples this is highly dependent on the users, tasks and context. Generally 3d maps with 3d content (arrows, points of interest, landmarks) are obvious cases where this works perfectly.
I’d don’t think pursuing 3d objects as UI controls is a very good way to go (e.g. the plastic carousel in the TAT image above). The physical world is such a constraining starting point, but in limited contexts, (like the I-phone picker control that uses a 3d curved appearance whilst actually rendering all the text in 2d.) it might work as an affordance rather than a metaphor.
Usable – Again highly dependent on the model used, but this style suggests camera movement is a must.
Desirable – There are plenty of contexts where building a 3d world with 3d objects makes perfect sense, and has some aesthetic appeal in making it ‘beautiful’ and idealized. The dioramas style from the aesthetics post is an ideal style for this kind of thing.
3d objects in logical space
Typically the best applications of this are seen in the scientific visualization community where 3d objects are taken out of their context to aid in understanding them. This example from the cover of Tufte’s book, Envisioning information is a great example of providing just enough extra information in the abstract to aid in understanding the phenomena being described.
The same egregious use of overly complex data visualization from the 2d cards example also continues in 3d space. This tool isn’t exactly built for mainstream users but all the same super complex systems visualization continues to befuddle comprehension. (How many variables can a human process simultaneously?).
Useful – Useful, this can be the only way to convey the underlying structure of complex machinery, internal systems, biological systems, and hidden natural systems.) So yes its very useful for the scientific, engineering, educational and medical markets.
Check out systems like Catia for the complex engineering 3d software.
3d objects in real environments
Camera driven Augmented reality, the Wii, to Sony’s Eye toy, to Microsoft’s project Natal), Apple’s Iphone, Google’s G1 are using mostly camera’s to enable 3d input for the first time. Real 3d inputs involve moving a device like the Wii-mote or you’re phone or your arm around in X,Y and Z space and mapping real physical spatial inputs directly to outputs. Some aspect of this is Natural user interaction is destined to become part of our daily interactions with technology over the next few short years.
Total immersion did a demo 4 years ago that still is very impressive and blew my socks off at the time. Its taken a few years but we are now starting to see an explosion of toolkits that enable live video feed image processing to do motion tracking, even on relatively low powered devices. The result is 3d objects bursting out of video screens all over the web.
Sony at this years E3 showed Eye Pet which shows a 3d critter working with a PS3 and a Sony eye toy (web cam). There’s tons of interesting aspects to this its almost unbelievable. The camera is able to sense the ground plane and actually map the little pet character correctly into the scene.
This great project from some students showing off a pretty compelling game using a TEGRA development kit (more on this in the next article.). This is the first game I’ve seen using augmented reality through a mobile device, obviously it’s just the beginning.
Useful - There are so many potential applications of this from serious everyday productivity to entertainment to social that its hard to not see this as the most compelling use of 3d to date.
Usable - The obvious benefit of mobile systems for AR, is the camera being on the back of the device rendering the screen as a direct looking glass onto the real world, sort of ‘direct viewport manipulation’. Laptop or web cam form factors put the cameras facing the viewer, which inverts your inputs. Note the kid’s eye line is off center in the Eye toy video meaning they are looking at a TV to see the image we are seeing. This presents some input problems. No tactile feedback is another problem in this space that makes gesturing with a 3d device hard to judge.
Desirable – This is sufficiently magical that it stands as one of the longest standing most over-hyped technologies out there (along with VR). With 3d input devices and 3d outputs possible with head tracking there’s going to be some incredible things enabled that will challenge all of what we think we understand about U.X design. Who needs the full emersion geek headgear for that?
Final thoughts and tips
Some arguments can be made for 3d around efficiency of layout, overlapping content can take up less real estate, but that only works in the most camera constrained cases.
Research suggests that people can utilize spatial memory in the recall of objects placed in spaces but this can be true in 2d too, and the extra burden of users have to learn the physical layout of a space and manipulating objects in that space takes away more than it adds.
The screen and mice are both 2d devices. Multi-touch gives us a few extra degrees of control but its still best as a 2d input system.
3d works best when combined with animation and camera movement. This typically makes it ‘4d’ because the expectation is that greater comprehension of the UX comes through exposure and exploration of it over time.
A general rule of good information design is ‘reduction’ removing the un-necessary in favor of enhancing the important. 3d by nature can make that reduction task more complex and thus 3d designs tend to be more ‘complex’ visually. (Especially if they utilize the effects that make 3d worthwhile in the first place, lighting, shadows, material qualities etc).
3d organizational systems and user interfaces work well when they map to ‘simple’ metaphors but don’t take the metaphors too literally. Coverflow has reached widespread adoption because the underlying idea is incredibly simple ‘a stack of albums’. I don’t need camera movement or 3d input technologies to go next/back. Carousel’s borrow from the physical idea of a lazy-Susan device and similarly don’t need much in the way of explanation or flashy camera movements to work.
If you are using 2d inputs, consider not giving the user any real freedom to move the camera, or the target of the camera, other than pan, track and zoom.
Summary
I think there’s a palpable hunger for the next novelty in the consumer technology space. 3d graphics and effects are for the time being a source of that visual novelty, and will help new products differentiate themselves in very competitive markets. This more than any other factor is the central reason to use it. But do so with care, so many bad experiences have been created that a backlash against 3d is likely if we continue to abuse it.
Mainstream adoption of 3d inputs sensor technologies, (including augmented reality) and eventually 3d output technologies will make a huge difference to adoption of 3d as a useful and practical interface experience. But don’t ask me to put on special glasses or headgear.
I’m still hopeful that someone will come up with a holistic consumer friendly 3d user experience that actually works, and is more useful, usable and desirable than any 2d counterpart across a wide range of activities. It’s conceptual model will be instantly familiar, natural but not kitsch or cheesy with overuse of tired metaphors. It’s clever integration of spatial information will give me new and important insights into practical tasks and activities and its use of 3d aesthetics (lighting, physics, subtle animation and materials) will engage me emotionally the way great design always has. Here’s hoping.
The next and final part of this blog series is on the tools and technology part of the story, what technologies exist, do they scale to different platforms, what tools make it easy for creative’s of all kinds to push the status quo.
In this first installment I’m going to address the ‘where are we’ question, but mostly from the perspective of visual language, rather than interaction design, I’ll deal with that in the next installment.
When interaction and visual designers take the time to understand and master the new visual techniques at their disposal with 3d UI, (cameras, lighting and shadows, materials and physics.) really exciting and delightful things can emerge. We’ve been collecting a tiny fraction of the great work out there and see various trends emerging with designers getting creative with the extra dimension. What’s important about these examples is that it becomes clear that 3d has broken through as a viable graphic styling language and is being adopted widely in marketing, promotion and cutting edge user experience design. We shamelessly often look to emerging visual trends for ways to make stimulate UX thinking beyond standard flat 2d GUI controls when its appropriate.
I trained in Graphic Information design before my Interaction design training so there’s a part of me, (the Tufte indoctrinated part) that wants to boil everything down to its clearest, purist form to make it more understandable, communicative and effective. However when creating ‘experiences’ for consumer software applications we are often tasked with more than just simply communication. We are asked to move the needle on ‘excitement’, ‘animation’, branding language and many other variables. 3d is certainly not going to be a panacea for interaction designers or visual designers but when done right some beautiful and uniquely digital styles can be very effective.
I’ve identified three distinct visual trends we’ve noticed with 3d, I call them “Chunky Fonts”, “Shiny horizons” and “Diorama’s”.
Visual Trend #1. “Chunky fonts”
Chunky fonts is a style that uses 3d typography as the primary visual ‘content’. I’ve seen some great examples of this all over the place, ( here and here and here are some great inspiration). In making the fonts work visually, it helps to chamfer the edges of the extruded font to get a nice edge light of the shapes themselves. Lighting is key to making the font work in 3d, so designers resort to something called Ambient Occlusion rendering which gathers shadows in every nook and cranny of the font. (more on this later). High end shaders (reflections and sub surface scattering) and 3 point lighting helps the front face of the font the visually dominant component. Making things appear to be physical and touchable objects, is part of the appeal of 3d and gaining in importance with the rise of more natural user interfaces.
Zune Marketing getting in on the action.
Visual Trend #2. “Shiny Horizons”
This second style has been in vogue in the software industry for a little too long in my personal opinion. Perhaps it started with Coverflow or the Matrix scene where they order ‘guns’ in the preloader environment.
You know it when you see it, a see of white or dark on an infinite horizon plane with objects on it that has a reflection directly beneath the objects. This has been used ‘everywhere’ from Mac OS taskbar where everything is mega glossy, to the Xbox 360 NDE.
It has gotten so commonplace that PowerPoint 12 included the ‘automatic’ floating reflections as a canned styling effect a couple of years ago. Still whilst it’s a bit passé its still works and is (to these eyes) undeniably attractive. I think the reason it works so well visually is that with the ground plane establishes immediately a sense of physicality and dimension which makes content even just 2d cards in space, look ‘special’, touchable.
Visual Trend #3. “Dioramas”
Dioramas are like highly crafted little ‘sets’ of models staged to look hyper-real or cute and cuddly, physical, dimensional and realistic right down to the tiniest details. I’ve seen this style become a fixture in marketing materials recently, especially motion graphics. One common aesthetic technique is to use a collage style of as part of this, found objects, paper stencils, even hand-drawn assets, this nicely takes the geeky edge off the sometimes excessively garish shiny metallic computer graphics.
Still one of the most impressive things I’ve seen accomplished in real time, Ecozoo (Papervision 3d.) The reason it works so well, (apart from amazing art direction) is nice ‘baked’ lighting in the textures, excellent camera work which explains the concept, great depth established with blurred foreground and background elements. I know enough about 3d to be blown away that this ever got done, I’d have to imagine the designer was a person who’d worked in the games industry, it goes to show that very little geometry (Even 50 polygons) can look awesome with great (albeit not real time) lighting.
Well if you think, the EcoZoo is cool, Little big planet takes it up about a dozen notches in terms of complexity and execution. Little big planet is just a visual feast of a game for the PS3. The little ‘sack’ guy plays in a wonderland of found objects, which behave with amazing physics. This game has won every art award the game industry has to throw at it.
Lots of little physical objects in little dioramas, in Sprint’s – ‘what’s happening now’ campaign.
GE’s eco-magination website using amazing 3d rendered illustrations to convey some nice information graphics. These photo-real little sets are lovingly animated and super cute.
These trends are important, regardless of the fact that the rendering for many of them is not realtime.
If you want to start building beautiful real-time 3d what are the areas of knowledge you are going to have to invest in and get really good at.?
Cameras
So it’s almost so obvious that its easy to overlook one of the things about 3d vs 2d that’s completely different is the ‘camera’. There’s obviously an ability in 2d design to move the objects in the view around, but only in 3d can you also move around the objects. This small detail conceptually makes a big difference to interactive 3d design and can potentially get us into a world of trouble with producing a simple to understand, easy to use interactive 3d navigational model. (More on this in Blog 4).
Top tips, First if you want to get really good at this, you should invest in a cinematography book (which will help with the lighting topic too) Film and Photography craft skills are incredibly relevant to many aspects of 3d CG. Some simple interactive techniques that we’ve seen work well are really subtle camera movement ‘breathing’ and ‘panning/tracking’ are two simple techniques to make a static screen state become more interesting. Breathing is simply moving the camera, the camera’s target, or animating the field of view of the camera in a simple loop. This only needs to be a tiny amount to make things appear more dynamic and interesting visually. It can have secondary benefits, for keeping the screen from ‘burn in’ and inviting the possibility of interaction. Panning and tracking just a little bit the camera in response to finger or mouse or sensor inputs also succeeds in quickly creating a delightful sense of direct manipulation.
Generally speaking in interactive work ‘on rails’ camera movements (key framed movements of the camera to absolute coordinates) are a mixed blessing. They can ‘simplify’ 3d interactions considerably but at the cost of creating something that can feel less dynamic and interactive as a whole.
Lighting and Shadows
Of all the tricks to master in designing for 3d, this is the one topic you have to really understand, and it’s a ‘deep’ topic. (invest in a good CG lighting book). Even simple cubes in 3d can become really amazing when you have really good lighting and shadows. Essentially, most existing real time 3d technologies can’t do anything like the above image in real-time, the image above probably took about a gazillion hours to render. But ‘wait’ you say –“I’ve played Unreal tournament, that’s real-time and they have amazing lighting”. They do indeed, but they have a ‘lighting engine’ that renders the lighting information back onto the textures during the art production process (this is called ‘baking’ the textures), so the lighting component is not done live and no performance penalty is paid. It’s very effective to bake the lighting as long as you don’t move the objects or change the lights, everything is fine. So I could get close to the above image in real-time with baked textures on a bunch of cubes and a ground plane, but I couldn’t move any of the objects or the lights.
This ‘baking-the-lights cheat’ is actually just one of about a 100 cheats like this that real time 3d demands you learn to achieve a good-looking result. Game engines, (typically the most advanced real-time 3d technology out there) use hundreds of different cheats like this, carefully chosen to suit the needs of the content. Big outdoor games have very different rendering engines to driving games which are quite different to fighting games, which are completely different to first person corridor shooting games all have tons of tradeoff’s made to enhance quality and boost performance.
Some tips on lighting… I briefly mentioned in the ‘Chunky Fonts’ trend, 3-point lighting for maximizing the dimensionality of geometry in your environment. Tutorials abound on 3-point lighting because conceptually it’s the same as lighting for normal photography and film so if you understand the concepts of Key, Fill and edge lighting you are set. With real time lighting today however there’s no real notion of lights being anything more than color being added in variable amounts to the shading of a surface, real time lights in most real time 3d engines don’t cast shadows - Its widely considered to computationally intense, but ‘there are cheats’ for this that are highly effective.
Materials and Shaders
Next to lighting, and intimately tied to lighting is working with ‘materials’. What you need to understand is that real time materials are more complex than simple textures. Textures are images that are mapped onto geometry. In some real-time engines out there you can add multiple kinds of textures to the same geometry and tell the rendering engine to use the texture image to do special things, make it bumpy, shiny, transparent etc. A simple multi-texture example would be to make a alpha blended mask which when applied as a ‘specular’ map would make some areas of the object look more shiny than others. Its super time consuming to create good looking multi-textured geometry and for real time its more complicated in that there are few standards I’m aware of for exporting all that nice setup work you do in the 3d program into a real time engine.
Real time shaders (and there are many different kinds of shaders, vertex, geometry and pixel are the most common) do funky math at render time to make stuff render differently. A complex shader could for example make 3d model render in what appears to be in a 2d illustration (Toon shaders). Shaders are a deep science but in real time computer graphics engines (again mostly games) there are almost always custom shaders written to render aspects of a scene. From a Art direction perspective scene shaders are how you can tweak the overall aesthetic of your CG, There’s a ton of subtle shaders at work in this image below including the way sunlight bouncing of some materials tends to ‘bloom’ and blow out the contrast in an photographic image.
If you are doing really high end looking 3d work you going to need to hire someone from the games industry (a shader artist, or technical artist) who understands how to make great shaders. The good/bad news is that to my knowledge there are no tools or engines commercially available to do this kind of work for real time UI contexts. I’ll take up this topic in the final part of this blog, because there needs to be!
Effects
Aesthetic effects like depth of field, blur, bloom glow are all examples of sophisticated shaders, most of these are the subtle secret sauce that will make 3d come alive in real time. Depth of filed effects in an interactive context can provide users with a sense for what literally has ‘focus’. Expect to see lots of moderately excessive use of these effects in the coming years.
Physics
- A good example of the ‘chunky fonts’ trend, with baked lighting on parts of the geometry, with Physics applied in real time.
Physics engines simulate what happens when multiple objects hit each other are tied together, spring loaded, attached to one-another etc. They simulate mass, velocity, friction, inertia, gravity and other forces so that dynamic interactions between objects and environments can be created. This dynamism is really well suited to some kinds of interactive applications (certainly entertainment) but can be also applied to more subtle elements of the user experience especially with regards to animation. Take the nice example of Apple’s iphone UI where you drag a list beyond the final entry in the list and you reveal darkness, when you release the list it springs back into the correct position. This creates a nice ‘secondary’ animation effect that reinforces in the users mind that you’ve reached the end of a list. In 3d this can be extended so that objects that are attached to other objects (say Place of interest icons on a map) can jiggle, (as if attached to a springy suction cup) when the map is manipulated this secondary motion perhaps would add a sense of playfulness to a more natural direct manipulation experience.
Watch this to see the full range of state of the art real-time physics effects (cloth, springs, collisions, explosions etc). There are lots of real time open source physics engines again not much in the way of a great commercial available solution.
Anti-Aliasing
So aliasing is the little jagged lines that appear to stair step on diagonal edges of polygons, anti-aliasing is the smoothing of the Jaggies. Of all the factors about real-time 3d, this one can be the most distasteful aesthetically. Most modern video cards can overcome this but normally the anti-aliasing (smoothing) has to be manually turned on by the user (you need the correct driver installed and go into the control panel and futz with settings to make this happen- frankly it may as well not exist at all). One advantage of high dpi screens (especially mobile devices) is that the pixel dots are so close together that optically you’re eye doesn’t notice un-anti-aliased edges as much.
Production Pipeline
Working in real time 3d is a tools workflow headache. Geometry, Lighting, Shaders, animation, physics can bend your mind around a whole set of new production challenges that are immensely time consuming. Most of the 3d tools I know are squirrelly feature-bloated monsters… wildly esoteric from a UI perspective and designed for people who seem understand what the correct value to type into the ‘light bounce co-efficient’ dialog. These ‘fictional’ users also seem to have infinite amounts of time to spend playing with the tools.
Best advice if you are embarking on a ‘real-time’ 3d interactive extravaganza, hire someone who’s set up complex production pipelines, these people are called Technical artists in the games industry and movie special effects studios. If you don’t have that luxury give yourself ‘triple’ the amount of time you’d think it would normally take you to accomplish something in 2d. The mantra should be realtime 3d stands for 3 x duration and 3x difficulty and 3x demanding.
Summary
So the ‘technology’ to do high quality real-time 3d with all sorts of incredible lighting and shading effects, does exist (we see it daily when we look at games) but today there isn’t a good ‘engine’ and perhaps more importantly ‘toolset’ for doing this for building UI. So even if you understand and have mastered all the topics above, you are basically SOL when it comes to making it dynamic and interactive. Today’s hardware on most PC’s is so powerful that ‘anything’ is possible from a creative perspective, any of the images in the blog could be real time rendered if there was a general purpose UI engine available to do that. That is part of the reason I’m writing this slightly bloated blog article. In my 3final blog post in this series I’ll explain a little bit of what I want someone to build to make it easy for an army of interactive and interaction designers to get busy defining some amazing next generation user experiences.
The next blog post is about why some conceptual, organizational and navigational models work and some don’t and the rise of augmented Reality. The final part of this series will be more about tools and technology engines available today with a call to action about what would be the ideal UI rendering engine?
3D capabilities are obviously not new to UX, I’ve been building real-time 3D UI’s for 10 years, the academic community has been exploring the use of the third dimension for over 30 years (without much fruit I might add). Hollywood continues to fantasize and provoke us UX designers to imagine more dimensional user experiences. Perhaps for the first time however we are witnessing a bit of a critical mass for real working 3D UI’s. I think there are a number of factors influencing this trend from the change of perception of console gaming as a popular consumer activity vs. niche hardcore activity.
There are many different ways to break down this huge topic, we could talk about 3d natural input systems (from Nintendo’s Wii to Sony’s Eye toy, to Microsoft’s project Natal), and this defiantly worthy of later blog post. We could talk to the many examples recently we’ve seen augmented reality demos take hold and stir the imagination of digital designers everywhere. Similarly the use of 3D presentation techniques continues to stir the imagination as well as innovations around the use of eye and head tracking to generate the impression of 3D in your mind. I’ll ignore the stupid ‘3D headsets’ sub-story here, personally I hope to never don 3D glasses or a headest in order to use a piece of software, outside of gaming. So for this blog post series I wanted to concern myself with onscreen real-time 3d graphics used in traditional GUI and UX design.
This post sets up the series, and there are three distinct subtopics I want to address at a pretty high level. My second post is going to be about ‘3D Aesthetics’, the third post is ‘3D UI systems’, the fourth is ‘3D tools’. The 3D Aesthetics topic looks at the visual appeal of 3D graphics, the styles possible the new techniques a visual designer must become familiar with. The 3D UI systems topic addresses some simple truths about what seems to work, what doesn’t work and why, and what seems to put people off. The 3D tools topic explores the state of the tools, engines and how as a designer can you pull this off today.
3D is here, but there’s still a lot to learn before creating something compelling
At Artefact over the last year we’ve noticed a visual trend towards more use of 3D graphics techniques in all forms of digital design. The consumer electronics industry and motion picture industry are really pushing 3D technologies hard. As a company focused on building next generation user experiences we’ve consciously and deliberately been exploring the use of 3D graphics techniques in the design of new platform user experiences and rich internet applications. A cursory look at the last 10 projects we’ve worked several have explored 3D UX and visual aesthetics as part of the proposed final product experience.
A couple of personal beliefs I have about screen based 3D. Firstly, I don’t buy into the idea that 3D UI is inherently better in some way than 2D UI. The reality is that 3D UX is way more difficult to get right and can have some inherent drawbacks in terms of fundamental visual communication principles (usually excessive clutter). The simple reality is that most UX content, (Video, Photos and text) are inherently ‘2D’ digital objects so how can there really be any benefit to presenting them in a 3d space?. Free form 6 axis navigation without full emersion virtual reality is a terrible choice for almost all applications outside of simulating exploration of simulated virtual worlds. There’s a ton of research about the added conceptual burden of navigating through fully 3D UI’s, but at the same time the obvious fact remains that we live and operate successfully in the real 3D world.
On the positive side of this argument, I hope to demonstrate that there is something ‘viscerally’ compelling about 3D movement and 3D visual style, which personally and I think commercially is undeniably successful. When designers take the time to understand and master the new visual levers at their disposal, (depth, lighting, animation, cameras, physics and materials.) really exciting and delightful experiences can be created that give user experience designers new ways to engage and delight users as they interact with digital products of all kinds. In addition I’ll argue that various technologies are going to bring 3D UX to the mainstream in the very near future so we’d better start some kind of discourse about them.
When UX designers don’t understand 3D UI very well, we unsurprisingly end up with crap.
The desktop metaphor has long been understood and used for decades. We’ve touched it, we’ve clicked it, we’ve even made it into a panorama! Well Shock Desktop 3D takes us one step closer to this model of computing with their version of what some of you may be familiar with, BumpTop.
It basically turns your desktop [Windows only folks] into a 3D environment where you can bump your files around in space like physical objects. It’s a fun idea that doesn’t require heavy amounts of computing resources to run.
Contact me if you want Artefact to design such a tool, right now that would be a dream project.
Source - 
by 
