[tweetmeme]Finally the final part of the 3d blog. As I mentioned in the previous posts on 3d, there are very poor tools for creating great looking 3d user experiences. This article will not concern itself with the aesthetics of 3d or the usefulness, utility or desirability of 3d UI’s, for discussions on that see here.

First off I’ll do a quick overview of the basic tools for creating 3d assets before diving into the tools for creating interactive 3d for different kinds of user experiences. This is a high level guide to the topic and in no way attempts to be exhaustive, just hopefully helpful.

Tools for digital illustration – Just creating great looking 3d.

The biggest tool provider for anykind of 3d anything is Autodesk. They are the million pound gorilla when it comes to 3d tools. Over the last few years they have swallowed up what used to be the top 3 tools in the 3d world, Autodesk SoftImage (formally XSI), Autodesk 3ds MAX and Autodesk MAYA – all previously rivals (stunning the DOJ where asleep at the wheel for those purchases) . These are considered the mega premium tools and have almost religious followings in the 3d community from game makers to film makers but to be honest from my perspective as a 15,000 foot generalist they are more or less identical in terms of capabilities. They are also ‘huge’ from a time investment perspective as they combine multiple sub tools, for animation, modeling and rigging, physics, creating shaders, lighting and rendering. Whilst they are widely used and regarded by moviemakers and game-makers there are plenty of much less expensive alternatives that can do 90% of the same basic things.

These tools are so big they have become platforms for thousands of vertical solution providers, a few of my favorites for 3ds Max specifically include Splutterfish Brazil, a plug-in renderer, Dreamscape (rendering landscapes, seascapes), V- Ray, (a plug-in renderer for architecture).

Other long time contenders and less expensive choices for 3d creation include Caligari Truespace, Eon Vue, (Landscapes and terrain), Rhinoceros 3D (rhino) for industrial designers, Cinema 4d (popular with film-makers and motion graphics folks), there is a great Wikipedia article on all the different tools here.

Sooner or later you will want to visit Turbosquid – it’s a huge 3d web marketplace for models, shaders, plug in tools and tutorials some of which are free.

COLLADA™ Is an open source intermediate file format that defines an XML-based schema to make it easy to transport 3D assets between applications – something that you will quickly discover is a god send. This includes geometry, materials, textures, lights, camera, animations, instantiations, and scenes. COLLADA also includes skinning and morphing for character animation. Shader effects for several shader languages as well as some physics support.

Once you’ve created or bought your 3d models, products like Autodesk Showcase include a fancy real-time 3d engine which allows you to move around high end renderings of industrial design models in real time, complete with soft shadows and high end shaders. (Car interior image below is showcase)

This tool is primarily intended for doing ‘pre-visualization’ and design review of industrial design forms using the power of the high end video cards and is a glimpse at what’s possible with the latest rendering engines. They also include the ability to play with libraries of pre made common materials, and environment lighting and immediately see the result, which has historically been a massive time waste with traditional renderer’s.

Hypershot Bunkspeed, is a quasi-competitor to Showcase allowing you to do semi-real time high dynamic range lighting, (HDRI). HDRI is a technique for doing image based lighting, which is how you get something to look photo real. The product is mesmerizing, super easy to use, and really useful for illustration. Simply drag the high end shaders onto the model, pick your image based lighting and (image below) is what you get. The soft shadows and ray tracing capabilities make the output look spectacular.

There is also a plugin for Google’s Sketchup tool that uses Bunkspeed. Sketchup is the world’s easiest 3d box-modeling tool, ideal for quick 3d sketches of physical relationships; bunkspeed kind of makes anything look hot with the right HDRI image choice.

RTT is a technology company making another one of these real time 3d rendering Showcase like tools. I find this type of product interesting because of how they clearly demonstrate what is possible with real time, high-end rendering. RTT also offers sever based rendering solutions which can deliver rendered frames to less capable hardware in semi-real time. This allows car configuration UI’s to be done remote from the client. (TT image below is real time)

3d Real time UI Rendering API’s

OpenGL® ES is a low-level API for advanced graphics using well-defined subset of OpenGL. Open GL has largely become the standard API for doing high end 3d outside of the PC and XBOX platforms which are dominated by Microsoft’s competing Direct X technology. GL-ES provides a low-level applications programming interface (API) between software applications and hardware or software graphics engines. Apple developers have the option to use open GL ES on the I-phone and now the I-PAD for example.

The wizzy bookshelf spin into the bookstore effect from the IPAD launch video, another example of 3d metaphors successfully working to add some bling to a product announcement.

These API’s make it easier for developers (mostly games developers) to write code that is slightly abstracted from the exact hardware it will run on, making it more compatible with a wider audience.

WEBGL – WebGL is a cross-platform, royalty-free, open web standard for a low-level 3D graphics API based on OpenGL ES 2.0, exposed through the HTML5 Canvas element. The basic premise is that most browser makers (except IE perhaps) are working together to enable basic support for this as part of the HTML 5 specification, which is still a year or two away from prime time. These open standards would go along way to making 3d UI not require a specialized engine plugin (like Flash or Anark); many people are hyped up about the potential for this to finally let the 3d cat out of the proverbial bag.

Nvidia’s TEGRA 3d UI

This 3d UI (carousel image shown in the article header) was shown in working from June 2008, and generated significant excitement that a 3d touch based UI that looked reasonably hot would actually become a commercial reality. Turns out this UI will not apparently see the light of day according to this report. I believe the UI below was authored using the now extinct tool ANARK game face which was bought from Anark after having been a tool for implementing 3d UI’s in games for a several years.

Tools for building 3d UX Today.

Today as a designer wanting to create a real time 3d interactive experience your choice is pretty much limited to Adobe Flash with one of the various plug-ins, like Papervision or Away 3d. Inherently Flash is not a great 3d platform basically because the first thing you should know about 3D and Flash is that Flash doesn’t really support 3d. What Flash does know is how to display vector shapes on the screen and how to calculate math expressions. With that, 3D can be faked by doing really sophisticated mathematical transforms. Generally this approach is ok for minimal 3d transitions and simple geometric models but as soon as the going gets rough (say you need high polygon models or shaders) designers resort to non real time 3d and interactive pre-rendered 3d Flash movies. Still, Flash 10 or one of the plugins mentioned above is really still the de facto option for developing interactive 3d sites today albeit they are often less than spectacular and don’t really take advantage of any of your specific GPU or video card hardware (features like anti-aliasing and texture smoothing are missing). One of the worst qualities of flash based 3d is the menacing ‘jaggies’ of aliased fonts and geometry, and the ‘sparkly’ effect that is created with non smoothed textures. The creative industry is obviously heavily invested in Flash, so if Adobe keeps plugging away it’s likely 3d will become easier and better through their classic and venerable authoring suite.

Electric Rain are a small software company who make Swift 3d which is a well respected modeling and animation program for Flash. It takes 3d rendered geometry and does something like the ‘live trace’ feature in illustrator, converting it into less memory consuming 2d vectors. For canned 3d effects it’s an easy and well travelled solution for designers.

Scaleform are a company plugged into the Games community with a tool that enables them to support Flash movies inside proprietary games engines, a great deal of modern games use this technology which gives them the, coding, production and animation benefits of Flash inside their rendering engines, thus making the production of sophisticated menu systems, Hud’s, (text, maps, equipment status, health etc) or the configuration UI’s much easier.

Expression Blend is a UI building tool produced by Microsoft which can create 2d or 3d interfaces for Microsoft’s client application engine WPF, or its online cousin Silverlight. Expression Blend is ‘good’ for building 2d UI, however when it comes to 3d we are back in the dark ages. It can display and manipulate 3d models with simple mapped textures but every time we’ve tried projects utilizing 3d – we have quickly run into performance challenges (even on high end hardware). There’s no support for any sophisticated effects, shaders, anything more than the most fundamental lighting, no decent animation support. There are some 3rd party tools to make life easy on WPF. Another Electric Rain product (I’ve not used) ZAM 3D is a 3D XAML Tool for Microsoft Windows Application Development. It provides developers and designers with a quick and easy solution for creating 3D elements for Microsoft Expression Blend and Visual Studio projects. It also acts as a 3ds or dxf to XAML converter.

It seems increasingly clear to me that Microsoft has de-invested in WPF, and I’ve given up hope that it would become a decent way to create a 3d UI. In all the hype about WPF at its launch, there was some good demos and excitement, yet here we are 4 years on and really there are no decent product examples of WPF applications, and very few announcements from Microsoft about its roadmap.

There are a few other choices out there like Anark, which is a long in the tooth tool from yesteryear which looked very promising as a way to author 3d user experiences about 10 years ago, but today seems to have been abandoned by its parent company as a super niche tool. We’ve built some incredible 3d demo’s in Anark but the tool seems to have not received a significant update in a long while and it’s expensive.

ARToolKit is a Cross-platform Library for the creation of augmented reality applications, developed by Hirokazu Kato in 1999 and was released by the University of Washington HIT Lab. Currently it is maintained as an open source project hosted on Source Forge with commercial licenses available from ARToolWorks

ATOMIC Authoring Tool is a Cross-platform Authoring Tool software, for Augmented Reality Applications, which is a Front end for the ARToolKit library. It was developed for non-programmers, to create small and simple, Augmented Reality applications, and released under the GNU GPL License. I’ve not used it and can’t speak to its usefulness.

The Astonishing Tribe, also known as TAT have a 3d development environment I’m curious about, its called Kastor, and I think works with TAT Cascades which is a UI framework. Some of the demo’s mentioned in my previous post’s on 3d were probably developed using this tool set. The Red-dish demo (below) they did last year utilized OPEN GL ES 2.0 shaders and anti aliasing functionality.

Kanzi is a suite of tools and technology for 3d UX development. The Kanzi SDK tool is designed to be used with OPEN GL ES on mobile devices. Its Rendering engine Kanzi Engine is supported by a range of silicon manufacturers. I’ve not used this tool in a professional context yet, but the examples on the site suggest a mature product with many of the capabilities for building good looking, hi quality 3d. At the time of writing this, I just recieved an email from a Finnish company called Rightware that has just acquired Kanzi from Futuremark.

Images above from Kanzi demos above.

I’ve purposefully left out discussing the merits game engine technologies like Unreal, ID- Tech and Valve’s Source engine, Microsoft’s XNA and many others which are less ‘tools’ and more like ‘starter kits’ for the development of 3d games, they handle low level features but they are really only for those conversant in C# or lower level languages and not really optimized for building user experiences or more general products.

I’m very interested if you know of other commercial tools for building UX and will update this blog with suggestions and comments when I receive them.

Tools for building 3d UX tomorrow.

At heart the people using UX building tools and contemplating 3d UI concepts are desperately in need of tools that allow them to ‘play’ in high performance real-time 3d quickly and easily. The lack of these tools, I’d argue is the main adoption blocker for seeing more high quality 3d user experiences. Occasionally I see something really impressive in 3d that makes it more clear what we are striving towards in terms of output from such tools. At the Intel booth at CES 2010 was a very impressive dual touch wall (Image below). This was not created (to my knowledge) with a cutting edge 3d tool but it demonstrates the kind of output UX designers like ourselves at Artefact are striving/desiring to create.

Check out a video of the animation on Youtube. This would be impossible to build in any tool that I’m currently aware of and had to be coded from scratch.

Watch this video of someone creating a “little big planet” level. This is a tool that comes with the game for players to create additional content and share with the community. You use a PS3 controller and a character to create levels within the game. Note in the video that there is no difference between building the level and being in it which makes the authoring process very experimental. Objects are ‘physically’ enabled by default. There is minimal difference between the “authoring process” and the “experiencing it” process. This tool is actually very simple to learn, super complex to master. It’s also a modern work of art.

This tool represents a kind of design target for me, today it feels especially for UX like we are down where C++ developers where years ago, down in the weeds creating everything from scratch, specifying every possible variable and property. The desire is to have a palette of starter objects, 3d controls, shaders, lights, behaviors and effects that are a starting point to start sketching great user experiences. When my sketch work is complete, allow me to dive in and get busy scripting custom behaviors, defining custom shader’s and geometry.

The complexity of 3d tools makes production a very deliberate, detailed and labor intensive process. As a result designers are limited in their ability to experiment and iterate in building 3d user experiences. So my ideal tool combines lots of ‘starter objects’ with the ‘stateless’ UI demonstrated by Little Big Planet and Autodesk’s Showcase application. With such a tool I think we’d be pretty close to the kind of tool that would take the world by storm. (Quietly I’m hoping someone sends me a link) Contact me if you want Artefact to design such a tool, right now that would be a dream project.