Embedded Device Videos for McAfee

We wrote and developed these videos for McAfee:

We created relatively simple 2D animations, and moved them in space to tell the story more effectively, and to give a sense of space and depth. 


This allowed us to keep close to the McAfee brand guide for illustrations, and give a fresher, more interesting feel to each piece.

The Truc Designs Holiday Card

The HTML 5 Holiday Card:

Notes: 

• jQuery is the framework
• All of the elements are divs, animated with a spritesheet in a background image 
• Loading image sequences was very cumbersome, with too many browser requests, especailly on mobile. There may be a way to request multiple images simultaneously.
• Animations were done in Lightwave > Swift3D > Flash. Spritesheets were created with SpritesheetSWF by Keith Peters:  http://www.bit-101.com/blog/?p=2939
• Positioning each element absolutely by manually writing CSS is tedious - (for example, placing and aligning the shadow of each element)  -  could something WYSIWYG like Adobe Edge or Dreamweaver be worth the effort here?
• Multiple-frame svgs or svgz's seem like a possibility once Firefox and IE reliably support them
• The string between the lights is a canvas element
• Would all elements animate more smoothly if they were rendered directly on canvas instead of within divs?
• An animated intro starts jQuery animations at various frame numbers
• The mobile version removes the plane and robot elements to improve performance
• Elements are made draggable depending on whether the broswer is mobile or not
• Each elements position, velocity, frame number, and z-index is modified every frame
• Collisions are checked between all elements
• This seems especially processor-intensive on mobile
• A background gradient provided visual depth 
• Performance was by far the best on Chrome. This may carry over to Android once Chrome becomes the default Android browser.

On the fall of Flash and the rise of HTML5

As a Flash developer and animator, I can say that after 4 months of playing with “HTML5″, (more specifically Canvas, CSS3, jQuery, and the video tag) I have some thoughts.

On HTML5:

• HTML5 Video is surely the most over-hyped technology of the 21st century. Embedding will require at least 3 different encoded video files. Three. Plus a Flash fallback wrapper. Not good. Plus some dimensions and encoding specs won’t work on some devices and browsers. And iOS renders mp4s with a nasty gamma shift (at last mp4s not made with Final Cut Pro or Sorenson Squeeze). It’s a nightmare – something has to give here.
• Touch events beyond simple clicking (muti-touch, gesture) are NOT handled well at all by HTML5, despite what the dead guy in the turtleneck said. You need three different codes to prevent default zooming, and even then the animation freezes on dragging, etc.
• I’ve yet to see any HTML5 animation/ interaction that looks great on mobile. Desktop/ laptop, yes. Desktop WebGL, hell yes. (Check out http://www.chromeexperiments.com/) Mobile, no. Performance of HTML5 on mobile does not (will never?) match that of a native app. Hardware acceleration, anyone?
• I much prefer the syntax of JavaScript to AS3 – it’s essentially the same as AS2.
• Style sheets, in general, stink. They often require lots of forensic digging to find out why a div doesn’t look like it should. Chrome’s “Inspect element” helps.
• Doing things immediately inside the browser with no compiling or publishing is kind of exciting (though you could consider jQuery a plugin).
• SVGs (scalable vector graphics) are invisible to many browsers, mobile and otherwise. Remember that if you try out Adobe Wallaby.
• Debugging multiple browsers is a HUGE obstacle, to coin a phrase. jQuery helps a bit.

On Flash:

• On newer Android devices, most Flash content looks great and performs beautifully. If Adobe had continued to work with Google, it could have been a major selling point for their devices. In fact, Adobe should sell the platform to them – though it may be too late now.
• Most large business clients use plain old IE. They can’t see much HTML5 for the most part. They can all see Flash.
• The “motion editor” introduced in CS4, to me, was a flop – very non intuitive, even more so than After Effects – too much of a learning curve. As was the endlessly, needlessly configurable interface. Adobe Edge looks a bit better.
• Flash got a bad rap as adware. It was too easy to make ugly moving banners – so people blamed Flash.
• I don’t think I can convey the essential power of this strongly enough: Flash content looks the same everywhere. Building something once meant that it was built, period.

HTML5: Getting dragging to work on Mobile

This is as close as I can get to mobile/non-mobile "draggability" this week:

Major Steps:

• Dragging in both touch and mouse interactions CAN work from a single codebase.
• On a touchscreen, performance hiccups can occur when an object is toggled between animated via code versus dragging. 
• Android tablet (XOOM Honeycomb) performs fairly well - Android mobile (HTC Shift - Android 2.2) performs somewhat worse.
• Screen scrolling (and rotating) are very annoying (an 800 x 500 screen improves matters somewhat, as it takes up all the real estate on a smartphone)
• Performance is too slow on an iPhone 3Gs from 2008. (It's doubtful that any HTML5 animations can really work on that vintage of iOS device)
• Accessing child objects of a given class is possible with efficient jQuery code - this is useful for providing dynamic captions, etc. For example:

('#blobby').children('.caption').text('Hello World,) 

accesses the 'caption' class under the blobby div (in this case it's a <span> tag).

Outstanding Questions/ Issues:

• Exactly how big is the standard Andoid/ iOS device screen in pixels?
• The game cries out for collision detection and reaction between sticks. We have rudimentary detection - but need more. Seth Ladd probably has some good answers: http://blog.sethladd.com/
• Other resources:
• http://io-2011-html5-games-hr.appspot.com/#83
• http://codeflow.org/entries/2010/nov/29/verlet-collision-with-impulse-preservation/
• Can we eliminate the dragging hiccups?

Other interesting things:

In looking at upcoming conferences on Adobe I came across open source 3D engines for Flash AS 3:
http://away3d.com/
http://www.stasha.info/index.php?id=25
http://www.nissan-stagejuk3d.com/

I guess this is where VRML-like interactions are today. The Nissan Juke ad takes forever to load and lacks anti-aliasing, but has reflections, particles, and "airflow" effects that are either particles or polygonal with transparency maps. Overall very impressive.

Blender and 3DS Max was mentioned as a possible building tool -- how well can Lightwave work with these applications?

HTML5: An early app

"Blobby." Drag and throw the orange creature:



http://trucdesignstesting.com/HTML5/sprite_anim/blobby_v008.html

Notes:

• Dragging is possible with touchscreen devices, but we're not capturing the dragging "start" event yet. 
• jQuery has a plugin that does not seem to work on Android: http://plugins.jquery.com/project/mobiledraganddrop
• This one has more promise: http://www.jqtouch.com/
• In either case, we need to detect the mobile browser and set a flag
  
• The character is a <div> using jquery.jmovie to control a sequence of png images exported from Flash. 
• Svgs are not used, to allow display in Android

• jmovie is not showing any animation, since it is called on each frame - flags are needed to show when the "turnLeft"/"turnRight" animations are completed. Some elegant logic is called for here.
  
  
• The VerletSystem from bit-101.com is used to create the structures in a Canvas tag, 
• Performance on mobile is very choppy
• I modified the js to allow "sticks" to be removed dynamically -- this creates a lot of possibilities for collapsing structures
• Further mods would allow sticks to collide and break
• A "_dragging" flag on the character div is crucial - when it is released, it is moved by a point in the VerletSystem, when dragging, the character div moves that point.
  
• Canvas aligns well with other divs, raising possibilities for integrating Canvas-based animations with images, text, etc.
• An Illustrator plugin allows export to Canvas code
  
• A script is required to make Cnavas work on IE8 - even then it's chopppy
• Early iterations used a canvas based "trail" of bubbles, and a fade-in of successive backgrounds - good effects, but probably not realistic on mobile.
  
• An actual game would need collision detection and (probably) an objective.

Our major issues and possible solutions:

Performance is choppy on mobile

• Make a smaller version (300 x 300 or similar)
• Smaller pngs (50 x 50 max) - use a few stills only - no jquery.jmovie
• Make the canvas elements smaller 
• Research canvas redraw optimization -- make sure canvas is cleared each frame

Drag and drop won't work on mobile

• Check out and test http://www.jqtouch.com/

Objective: Interactive applications across most platforms

We want a single platform for web-based applications that can reach all major browsers, iOS, and Android devices. By "web-based," we mean that they are viewable immediately in a browser, without downloading or installing any additional software. 

What are the options?


Flash:

• Is restricted from iOS 
• Requires a plugin that many Android devices do not have

HTML5:

• Generally relies on some combination of JavaScript, jQuery, CSS, and the Canvas tag
• Is viewable (to some extent) on all devices with Webkit browsers
• Will (supposedly) replace Flash (and Java, and other browser plugins)

Thus, the rationale for creating HTML5 apps. What are the challenges?

1) There is no complete WYSIWYG solution for developing HTML5 pages with animation. There are:

• Wallaby, a tool for converting Flash to HTML, jQuery, and CSS. HTML elements are visible and editable.
• The timeline seems to be placed into a CSS doucument
• jQuery animations seem to work well on Wallaby-generated HTML elements
• Exports svgs, which Android currently can't see
• Edge,  a timeline-based tool for creating output in jQuery, and CSS. 
• HTML is dynamically generated by Edge Javascript. 
• The "timeline" appears in the Edge Javascript, with "tweens" in a text format
• Exports svgs, which Android currently can't see
• Komodo Edit (and many similar code/output editors)
• Swiffy, a Flash 6 swf to Javascript converter
• The Javascript generated is highly condensed and unreadable - thus un-editable, and not useful if modifications are needed, or communication outside the app is needed
• Very interesting that it generates an entire application with no CSS or multiple HTML elements

2) jQuery and Canvas performance are currently VERY choppy on mobile (on three iOS and Android devices tested). In fact, Flash performance on Android devices seems considerably smoother.

3) There is minimal support for touch and gesture interactions in jQuery, and likely in Javascript as well. This is disappointing, considering the attention and hype Apple in particular has given HTML5 as a Flash replacement. Touch-based drag-and-drop seems particularly challenging. jQuery mobile does include a slider element, which is also very choppy on android.

4) Annoying highlights appear when interactive elements are clicked. This happens on most browsers -- mobile as well. CSS may provide a solution.

5) Video appears to require 3 different encoding formats: mp4 (x264), webm, and ogg/theora PLUS a Flash fallback (mobile like Flowplayer). 

• Certain sizes of video appear incompatible with some mobile devices. 
• Is a batch multi-encode possible (Firefogg? Handbrake?)
• More on this later

What are the next steps?

1) Create an HTML5- based app that performs across Webkit browsers. Make use of:

• Dynamic text
• Real-time interactions
• Dragging and Dropping
• XML parsing
• Vector-based assets
• Images
• Physics engine (can be crude)
• Sound

2) Answer these questions:

• Is real-time game-like interaction possible on mobile browsers? (See existing HTML5 games as well)
• What are the impediments to performance?
• Is it realistic to pursue a single codebase?
• How can sound be integrated?