[tweetmeme]The beautifully designed http://www.paintwithben.com A great example of the ‘Chunky fonts’ trend.

In this first installment I’m going to address the ‘where are we’ question, but mostly from the perspective of visual language, rather than interaction design, I’ll deal with that in the next installment.

When interaction and visual designers take the time to understand and master the new visual techniques at their disposal with 3d UI, (cameras, lighting and shadows, materials and physics.) really exciting and delightful things can emerge. We’ve been collecting a tiny fraction of the great work out there and see various trends emerging with designers getting creative with the extra dimension. What’s important about these examples is that it becomes clear that 3d has broken through as a viable graphic styling language and is being adopted widely in marketing, promotion and cutting edge user experience design. We shamelessly often look to emerging visual trends for ways to make stimulate UX thinking beyond standard flat 2d GUI controls when its appropriate.

I trained in Graphic Information design before my Interaction design training so there’s a part of me, (the Tufte indoctrinated part) that wants to boil everything down to its clearest, purist form to make it more understandable, communicative and effective. However when creating ‘experiences’ for consumer software applications we are often tasked with more than just simply communication. We are asked to move the needle on ‘excitement’, ‘animation’, branding language and many other variables. 3d is certainly not going to be a panacea for interaction designers or visual designers but when done right some beautiful and uniquely digital styles can be very effective.

I’ve identified three distinct visual trends we’ve noticed with 3d, I call them “Chunky Fonts”, “Shiny horizons” and “Diorama’s”.

Visual Trend #1. “Chunky fonts”

Chunky fonts is a style that uses 3d typography as the primary visual ‘content’. I’ve seen some great examples of this all over the place, ( here and here and here are some great inspiration). In making the fonts work visually, it helps to chamfer the edges of the extruded font to get a nice edge light of the shapes themselves. Lighting is key to making the font work in 3d, so designers resort to something called Ambient Occlusion rendering which gathers shadows in every nook and cranny of the font. (more on this later). High end shaders (reflections and sub surface scattering) and 3 point lighting helps the front face of the font the visually dominant component. Making things appear to be physical and touchable objects, is part of the appeal of 3d and gaining in importance with the rise of more natural user interfaces.

Zune Marketing getting in on the action.

Visual Trend #2. “Shiny Horizons”

This second style has been in vogue in the software industry for a little too long in my personal opinion. Perhaps it started with Coverflow or the Matrix scene where they order ‘guns’ in the preloader environment.

You know it when you see it, a see of white or dark on an infinite horizon plane with objects on it that has a reflection directly beneath the objects. This has been used ‘everywhere’ from Mac OS taskbar where everything is mega glossy, to the Xbox 360 NDE.

It has gotten so commonplace that PowerPoint 12 included the ‘automatic’ floating reflections as a canned styling effect a couple of years ago. Still whilst it’s a bit passé its still works and is (to these eyes) undeniably attractive. I think the reason it works so well visually is that with the ground plane establishes immediately a sense of physicality and dimension which makes content even just 2d cards in space, look ‘special’, touchable.

Visual Trend #3. “Dioramas”

Dioramas are like highly crafted little ‘sets’ of models staged to look hyper-real or cute and cuddly, physical, dimensional and realistic right down to the tiniest details. I’ve seen this style become a fixture in marketing materials recently, especially motion graphics. One common aesthetic technique is to use a collage style of as part of this, found objects, paper stencils, even hand-drawn assets, this nicely takes the geeky edge off the sometimes excessively garish shiny metallic computer graphics.

Still one of the most impressive things I’ve seen accomplished in real time, Ecozoo (Papervision 3d.) The reason it works so well, (apart from amazing art direction) is nice ‘baked’ lighting in the textures, excellent camera work which explains the concept, great depth established with blurred foreground and background elements. I know enough about 3d to be blown away that this ever got done, I’d have to imagine the designer was a person who’d worked in the games industry, it goes to show that very little geometry (Even 50 polygons) can look awesome with great (albeit not real time) lighting.

Well if you think, the EcoZoo is cool, Little big planet takes it up about a dozen notches in terms of complexity and execution. Little big planet is just a visual feast of a game for the PS3. The little ‘sack’ guy plays in a wonderland of found objects, which behave with amazing physics. This game has won every art award the game industry has to throw at it.

Lots of little physical objects in little dioramas, in Sprint’s – ‘what’s happening now’ campaign.

GE’s eco-magination website using amazing 3d rendered illustrations to convey some nice information graphics. These photo-real little sets are lovingly animated and super cute.

These trends are important, regardless of the fact that the rendering for many of them is not realtime.

If you want to start building beautiful real-time 3d what are the areas of knowledge you are going to have to invest in and get really good at.?

Cameras

So it’s almost so obvious that its easy to overlook one of the things about 3d vs 2d that’s completely different is the ‘camera’. There’s obviously an ability in 2d design to move the objects in the view around, but only in 3d can you also move around the objects. This small detail conceptually makes a big difference to interactive 3d design and can potentially get us into a world of trouble with producing a simple to understand, easy to use interactive 3d navigational model. (More on this in Blog 4).

Top tips, First if you want to get really good at this, you should invest in a cinematography book (which will help with the lighting topic too) Film and Photography craft skills are incredibly relevant to many aspects of 3d CG. Some simple interactive techniques that we’ve seen work well are really subtle camera movement ‘breathing’ and ‘panning/tracking’ are two simple techniques to make a static screen state become more interesting. Breathing is simply moving the camera, the camera’s target, or animating the field of view of the camera in a simple loop. This only needs to be a tiny amount to make things appear more dynamic and interesting visually. It can have secondary benefits, for keeping the screen from ‘burn in’ and inviting the possibility of interaction. Panning and tracking just a little bit the camera in response to finger or mouse or sensor inputs also succeeds in quickly creating a delightful sense of direct manipulation.

Generally speaking in interactive work ‘on rails’ camera movements (key framed movements of the camera to absolute coordinates) are a mixed blessing. They can ‘simplify’ 3d interactions considerably but at the cost of creating something that can feel less dynamic and interactive as a whole.

Lighting and Shadows

Of all the tricks to master in designing for 3d, this is the one topic you have to really understand, and it’s a ‘deep’ topic. (invest in a good CG lighting book). Even simple cubes in 3d can become really amazing when you have really good lighting and shadows. Essentially, most existing real time 3d technologies can’t do anything like the above image in real-time, the image above probably took about a gazillion hours to render. But ‘wait’ you say –“I’ve played Unreal tournament, that’s real-time and they have amazing lighting”. They do indeed, but they have a ‘lighting engine’ that renders the lighting information back onto the textures during the art production process (this is called ‘baking’ the textures), so the lighting component is not done live and no performance penalty is paid. It’s very effective to bake the lighting as long as you don’t move the objects or change the lights, everything is fine. So I could get close to the above image in real-time with baked textures on a bunch of cubes and a ground plane, but I couldn’t move any of the objects or the lights.

This ‘baking-the-lights cheat’ is actually just one of about a 100 cheats like this that real time 3d demands you learn to achieve a good-looking result. Game engines, (typically the most advanced real-time 3d technology out there) use hundreds of different cheats like this, carefully chosen to suit the needs of the content. Big outdoor games have very different rendering engines to driving games which are quite different to fighting games, which are completely different to first person corridor shooting games all have tons of tradeoff’s made to enhance quality and boost performance.

Some tips on lighting… I briefly mentioned in the ‘Chunky Fonts’ trend, 3-point lighting for maximizing the dimensionality of geometry in your environment. Tutorials abound on 3-point lighting because conceptually it’s the same as lighting for normal photography and film so if you understand the concepts of Key, Fill and edge lighting you are set. With real time lighting today however there’s no real notion of lights being anything more than color being added in variable amounts to the shading of a surface, real time lights in most real time 3d engines don’t cast shadows – Its widely considered to computationally intense, but ‘there are cheats’ for this that are highly effective.

Materials and Shaders

Next to lighting, and intimately tied to lighting is working with ‘materials’. What you need to understand is that real time materials are more complex than simple textures. Textures are images that are mapped onto geometry. In some real-time engines out there you can add multiple kinds of textures to the same geometry and tell the rendering engine to use the texture image to do special things, make it bumpy, shiny, transparent etc. A simple multi-texture example would be to make a alpha blended mask which when applied as a ‘specular’ map would make some areas of the object look more shiny than others. Its super time consuming to create good looking multi-textured geometry and for real time its more complicated in that there are few standards I’m aware of for exporting all that nice setup work you do in the 3d program into a real time engine.

Real time shaders (and there are many different kinds of shaders, vertex, geometry and pixel are the most common) do funky math at render time to make stuff render differently. A complex shader could for example make 3d model render in what appears to be in a 2d illustration (Toon shaders). Shaders are a deep science but in real time computer graphics engines (again mostly games) there are almost always custom shaders written to render aspects of a scene. From a Art direction perspective scene shaders are how you can tweak the overall aesthetic of your CG, There’s a ton of subtle shaders at work in this image below including the way sunlight bouncing of some materials tends to ‘bloom’ and blow out the contrast in an photographic image.

If you are doing really high end looking 3d work you going to need to hire someone from the games industry (a shader artist, or technical artist) who understands how to make great shaders. The good/bad news is that to my knowledge there are no tools or engines commercially available to do this kind of work for real time UI contexts. I’ll take up this topic in the final part of this blog, because there needs to be!

Effects

Aesthetic effects like depth of field, blur, bloom glow are all examples of sophisticated shaders, most of these are the subtle secret sauce that will make 3d come alive in real time. Depth of filed effects in an interactive context can provide users with a sense for what literally has ‘focus’. Expect to see lots of moderately excessive use of these effects in the coming years.

Physics

- A good example of the ‘chunky fonts’ trend, with baked lighting on parts of the geometry, with Physics applied in real time.

Physics engines simulate what happens when multiple objects hit each other are tied together, spring loaded, attached to one-another etc. They simulate mass, velocity, friction, inertia, gravity and other forces so that dynamic interactions between objects and environments can be created. This dynamism is really well suited to some kinds of interactive applications (certainly entertainment) but can be also applied to more subtle elements of the user experience especially with regards to animation. Take the nice example of Apple’s iphone UI where you drag a list beyond the final entry in the list and you reveal darkness, when you release the list it springs back into the correct position. This creates a nice ‘secondary’ animation effect that reinforces in the users mind that you’ve reached the end of a list. In 3d this can be extended so that objects that are attached to other objects (say Place of interest icons on a map) can jiggle, (as if attached to a springy suction cup) when the map is manipulated this secondary motion perhaps would add a sense of playfulness to a more natural direct manipulation experience.

Watch this to see the full range of state of the art real-time physics effects (cloth, springs, collisions, explosions etc). There are lots of real time open source physics engines again not much in the way of a great commercial available solution.

Anti-Aliasing

So aliasing is the little jagged lines that appear to stair step on diagonal edges of polygons, anti-aliasing is the smoothing of the Jaggies. Of all the factors about real-time 3d, this one can be the most distasteful aesthetically. Most modern video cards can overcome this but normally the anti-aliasing (smoothing) has to be manually turned on by the user (you need the correct driver installed and go into the control panel and futz with settings to make this happen- frankly it may as well not exist at all). One advantage of high dpi screens (especially mobile devices) is that the pixel dots are so close together that optically you’re eye doesn’t notice un-anti-aliased edges as much.

Production Pipeline

Working in real time 3d is a tools workflow headache. Geometry, Lighting, Shaders, animation, physics can bend your mind around a whole set of new production challenges that are immensely time consuming. Most of the 3d tools I know are squirrelly feature-bloated monsters… wildly esoteric from a UI perspective and designed for people who seem understand what the correct value to type into the ‘light bounce co-efficient’ dialog. These ‘fictional’ users also seem to have infinite amounts of time to spend playing with the tools.

Best advice if you are embarking on a ‘real-time’ 3d interactive extravaganza, hire someone who’s set up complex production pipelines, these people are called Technical artists in the games industry and movie special effects studios. If you don’t have that luxury give yourself ‘triple’ the amount of time you’d think it would normally take you to accomplish something in 2d. The mantra should be realtime 3d stands for 3 x duration and 3x difficulty and 3x demanding.

Summary

So the ‘technology’ to do high quality real-time 3d with all sorts of incredible lighting and shading effects, does exist (we see it daily when we look at games) but today there isn’t a good ‘engine’ and perhaps more importantly ‘toolset’ for doing this for building UI. So even if you understand and have mastered all the topics above, you are basically SOL when it comes to making it dynamic and interactive. Today’s hardware on most PC’s is so powerful that ‘anything’ is possible from a creative perspective, any of the images in the blog could be real time rendered if there was a general purpose UI engine available to do that. That is part of the reason I’m writing this slightly bloated blog article. In my 3final blog post in this series I’ll explain a little bit of what I want someone to build to make it easy for an army of interactive and interaction designers to get busy defining some amazing next generation user experiences.

The next blog post is about why some conceptual, organizational and navigational models work and some don’t and the rise of augmented Reality. The final part of this series will be more about tools and technology engines available today with a call to action about what would be the ideal UI rendering engine?