Archive for the 'JavaScript' Category

Social button BFFs

Tuesday, September 27th, 2011

TL;DR: Loading JavaScript asynchronously is critical for the performance of your web app. Below is an idea how to do it for the most common social buttons out there so you can make sure these don't interfere with the loading of the rest of your content. After all people need to see your content first, then decide if it's share-worthy.

Facebook now offers a new asynchronous snippet to load the JavaScript SDK, which lets you load social plugins (e.g. Like button) among doing other more powerful things.

It has always been possible to load the JS SDK asynchronously but since recently it's the default. The code looks pretty damn nice (I know, right!), here's how it looks like (taken from here):

(function(d, s, id) {
  var js, fjs = d.getElementsByTagName(s)[0];
  if (d.getElementById(id)) {return;}
  js = d.createElement(s); js.id = id;
  js.src = "//connect.facebook.net/en_US/all.js#xfbml=1";
  fjs.parentNode.insertBefore(js, fjs);
}(document, 'script', 'facebook-jssdk'));

Some nice steal-me JS patterns here:

  • immediate (self-invoking) function so not to bleed vars into global namespace
  • pass oft-used objects (document) and strings ("script", "facebook-jssdk") to the immediate function. Sort of rudimentary manual minification, while keeping the code readable
  • append script node by using the first available script element. That's 99.99% guaranteed to work unless all your code is in body onload="..." or img onload or something similar (insanity, I know, but let's allow generous 0.01% for it)
  • assign an ID to the node you append so you don't append it twice by mistake (e.g. like button in the header, footer and article)

All buttons' JS files

Other buttons exist, most notably the Twitter and Google+1 buttons. Both of these can be loaded with async JavaScript whether or not this is the default in their respective configurators.

So why not make them all get along and shelter them under the same facebook immediate function? We'll save some bytes and extra script tags in the HTML. For G+/T buttons all we need is a new script node. Google+'s snippet has some additional attribs such as type and async, but these are not really needed. Because type is always text/javascript and async is always true. Plus we kinda take care of the async part anyways.

The end result:

  <div id="fb-root"></div>
  <script>(function(d, s, id) {
    // fb + common
    var js, fjs = d.getElementsByTagName(s)[0];
    if (d.getElementById(id)) {return;}
    js = d.createElement(s); js.id = id;
    js.src = "//connect.facebook.net/en_US/all.js#xfbml=1";
    fjs.parentNode.insertBefore(js, fjs);
    // +1
    js = d.createElement(s);
    js.src = 'https://apis.google.com/js/plusone.js';
    fjs.parentNode.insertBefore(js, fjs);
    // tweet
    js = d.createElement(s);
    js.src = '//platform.twitter.com/widgets.js';
    fjs.parentNode.insertBefore(js, fjs);
  }(document, 'script', 'facebook-jssdk'));</script>

So this thing loads all three JS files required by the three buttons/plugins.

Additionally we can wrap the node creation/appending part into a function. So all the code is tighter. Here's the final snippet:

<div id="fb-root"></div><!-- fb needs this -->
<script>(function(d, s) {
  var js, fjs = d.getElementsByTagName(s)[0], load = function(url, id) {
    if (d.getElementById(id)) {return;}
    js = d.createElement(s); js.src = url; js.id = id;
    fjs.parentNode.insertBefore(js, fjs);
  };
  load('//connect.facebook.net/en_US/all.js#xfbml=1', 'fbjssdk');
  load('https://apis.google.com/js/plusone.js', 'gplus1js');
  load('//platform.twitter.com/widgets.js', 'tweetjs');
}(document, 'script'));</script>

All buttons' markup

Next is actually advising the scripts where the widgets should be rendered. Facebook offers XFBML syntax, with tags such as <fb:like>, but it also offers pure HTML(5) with data-* attributes. Luckily, so do all others.

Here's an example:

<!-- facebook like -->
<div class="fb-like" data-send="false" data-width="280"></div>
<!-- twitter -->
<a class="twitter-share-button" data-count="horizontal">Tweet</a>
<!-- g+ -->
<div class="g-plusone" data-size="medium"></div>

G+ requires a div element (with g-plusone class name), Twitter requires an a (with a twitter-share-button class name). Facebook will take any element you like with a fb-like class name (or fb-comments or fb-recommendations or any other social plugin you may need)

Also very important to note that you can (and should) load the JS files once and then render as many different buttons as you need. In Facebook's case these can be any type of plugin, not just like buttons. Economy of scale - on JS file, many plugins.

All together now

So here's the overall strategy for loading all those buttons.

  1. Copy the JS above at the bottom of the page right before /body just to be safe (G+ failed to load when the markup is after the JS). This will also help you make sure there should be only one place to load the JS files, although the snippet takes cares of dedupe-ing.
  2. sprinkle plugins and buttons any way you like anywhere on your pages using the appropriate configurator to help you deal with the data-* attributes (FB, G+, Tw)
  3. Enjoy all the social traffic you deserve!

To see it all in action - go to my abandoned phonydev.com blog. Yep, those buttons play nice in mobile too.

 

Lazy HTML evaluation

Wednesday, June 8th, 2011

#7 This post is part of the Velocity countdown series. Stay tuned for the articles to come.

Some time ago Google talked about using a sort of lazy JavaScript evaluation which especially helps mobile devices. The idea was to comment out a chunk of JavaScript you don't need right away and serve it this way. Later, when you need it, you get the content of the commented code and eval() it. More here and here.

At the last Fronteers conference I had the pleasure of chatting with Sergey Chikuyonok, who is so great and (among other things) is responsible for coming up with zen coding and writing a bunch of deep articles on image optimization for Smashing Magazine. So he told me he experimented with similar lazy HTML evaluation and it proved to be incredibly helpful for mobile devices. Not only the overall experience is faster but the initial rendering happens sooner and we all know how important that is.

Sergey is a busy person and chances of him writing about his experiment in English seemed pretty low at the time, so I decided to do an experiment on my own and see what happens. Meanwhile he did write about it so I forgot all about my findings, but here they are now.

Long document

I took one big HTML document - The adventures of Sherlock Holmes, which is half a megabyte or about 200K gzipped. Page A is the document as-is, plus some JS for measurements.

Page B (lazy) is the same page but with about 95% of its content commented out. The remaining 5% is a whole chapter so there's plenty of time to deal with the rest while the user is reading. After onload and a 0-timeout I take the commented markup (conveniently placed in <div id="lazy-daze">) and strip the comments. Then take the "unwrapped" time after another 0-timeout to let the browser repaint the DOM and regain control.

The overall skeleton of the lazy page is like so:

<!doctype html>

<html>
<body>
  <h1>THE ADVENTURES OF<br/>
  SHERLOCK HOLMES</h1>
  ...
  ... to chat this little matter over with you.</p>

  <div id="lazy-daze">
  <!--
    <p>II.</p>
    <p>
    At three o’clock precisely ...
    ... she has met with considerable success.</p>
  -->
  </div>

<script>


window.onload = function () {

    setTimeout(function(){

        var daze = document.getElementById('lazy-daze'),
            inner = daze.innerHTML;

        daze.innerHTML = inner.substring(4, inner.length - 4);

        setTimeout(function(){
            // take end time... 
        }, 0);

    }, 0);
};

</script>
</body></html>

Experiment

All the files are here:
http://www.phpied.com/files/lazyhtml/

We have the plain normal document - http://www.phpied.com/files/lazyhtml/sherlock-plain.html
And the lazy one - http://www.phpied.com/files/lazyhtml/sherlock-lazy.html

In order to run the experiment you just go to
http://www.phpied.com/files/lazyhtml/start.html
And click "Go nuts". This will load each of the two documents 20 times and take a few time measurements. "Go nuts" again and you'll get 20 more data points.

The time measurements I take are:

  • "plain" - unload to onload of the base version
  • "lazy" - unload to onload of the lazy version NOT including unwrapping it. This should be quicker than the plain version
  • "unwrapped" - unload to onload plus time to unwrap and rerender - this is expected to be bigger than "plain" because the browser has to render twice and is therefore doing more work
  • DOM loaded "plain" - unload to DOMContentLoaded instead of onload
  • DOM loaded "lazy"

Then I take the same 5 measurements but instead of starting at unload of the previous page, it starts at the top of the documents, as soon as a timestamp can be taken with JavaScript. This will exclude DNS, establishing connection, time to first byte...

Results

Here are the results from back when I did the experiment originally last year, using iPhone 2 (with iOS 3.2 or thereabouts)

I ran this experiment over Wifi and again over 3G.

First striking thing - it takes the about the same time to load the plain old page over Wifi and over 3G. For the smaller, "lazy" document, there is a difference, but there's virtually none for the plain base page. The guess here is that the rendering and its cost in terms of memory and CPU is far greater than the actual download time. In other words it takes longer to render than it does to download an HTML. At least in this class of phones. This guess is confirmed when you look at the time from the top of the documents, when the request overhead is removed:

With or without the request time - it's all pretty much the same.

The next striking thing - and how about that lazy document! It renders 3-4 times faster than the whole plain document. Not bad.

And one more surprise - lazy+unwrap time is less than the plain old document. Now that's interesting. It appears faster to split the task into two and do the whole double-rendering, which should've been slower because it's extra work. I guess that poor phone really chokes on the long document.

The same I found is true in Firefox, but almost the difference is negligible.

iPhone 4

I repeated the experiment tonight on iPhone 4 and wifi. And boy, is there a difference. What used to take 13 seconds is now under 3s.

The lazy + unwrap time is more than the plain time, which was to be expected.

Rendering that initial lazy document is still 2-3 times faster that waiting for the whole document.

The numbers:

  • 2765 plain (2014 DOM)
  • 1268 lazy
  • 2995 lazy+unwrap

Ignoring the request overhead:

  • 2200 plain (1421 DOM)
  • 715 lazy
  • 2423 lazy+unwrap

And one last run/observation - on the 3G and iPhone 4 there isn't much benefit of lazy-evaluation and empty cache. The request seems much more expensive. unload to onload 4.9s where document top to onload is 2.5. When the request overhead is out of the picture than lazy eval wins again - 1.7s compared to 2.5s

Parting words

  • Lazy HTML FTW?
  • Who the heck loads an entire book in a page?! Well it may happen. It may not be a whole book, but just a lot of markup. The entire book gzipped was 219K. A hefty document, but have you seen some of those news sites?
  • Possible use case - blog comments. Lots and lots of blog comments. Or posts.
  • If you're going to lazy-load something and get it with an ajax request, why not save yourself the request and ship with another chunk of html
  • This was a simple layout task. Just a bunch of text. I'm guessing there could be much more complicated pages and layouts to render. And rendering is what takes the time it seems.
  • Drawbacks a plenty because of the hidden content - accessibility, SEO.

Thoughts? Anyone want to run the test on Android or any other phone/device/tab/pad/whathaveyou? The guess is that the newer/powerful the device the smaller the difference. But it will be nice to know.

 

HTTPWatch automation with JavaScript

Sunday, April 10th, 2011

Background

I gave this short presentation at the recent Yahoo FE summit's open mic, here are the slides and some notes.

Slides and screencast vid

Screencast to see the thing in motion:

Notes

Here's the transcript of the slides as produced by slideshare.net. I've added some notes here and there to make it more readable when the presenter is missing.

JavaScript shell scripting - Presentation Transcript

  1. JavaScript is everywhere #42:
          C:> WSH
  2. Stoyan

    I do programming.

  3. Programming

    There are many options to choose from when you decide top practice the
    art and craft of programming.

  4. JavaScript

    ... is a very good one. Simply because JavaScript...

  5. ... is everywhere
  6. On the server

    node.js, asp.net

  7. Mobile

    html5, phonegap, titanium

  8. Desktop

    XULRunner lets you create cross-OS GUI apps

  9. Browser extensions

    FF, Chrome, bookmarklets...

  10. Photoshop

    yep, that too
    Several Adobe products actually let you script common/uncommon/programmable tasks

  11. Form validation too!

    (this was supposed to be funny)

  12. Shell scripting

    let's talk about shell scripting with JavaScript

  13. In Windows
  14. WSH: Windows Scripting Host

    All reasonable Windows machines (at least as old as Win2000) have this Windows Scripting Host in there.
    You can write VBScript or JavaScript (OK, JScript) to ... well, script.
    How does it work?

  15. C:>edit hello.js

    You create a file.

  16. var a = "Hello",
        b = " WSH!",
        c = a + b;
    
    WScript.Echo(c);

    Put any old JavaScript in there and print out some results

  17. C:>cscript hello.js
    Hello WSH!

    And this is how you run it.

    Or this:

    C:>wscript hello.js
  18. Open apps

    In addition to regular sysadmin tasks (copy, write files, move) you can open and script applications too.

  19. var ie = new ActiveXObject("InternetExplorer.Application");
    ie.Visible = true;
    ie.navigate(yahoo.com);

    This is how you open IE and point it to a page.

    Notice something familiar? ActiveXObject - the thing we used in IE before it got native XMLHttpRequest

  20. Firefox?

    Can you also open FF?

    Not really, as it doesn't have COM interface (whatever that is).

    But there's an easy solution

  21. HTTPWatch

    Finally we come to the topic of the talk.

  22. Speed

    Performance is critical for the success of any web app.

    Really, it is.

    When talking about improving speed there are two main steps:

  23. 1. Fix with YSlow

    Take a slow page, run YSlow, do what it says.

    Voila - a fast(er) page.

  24. 2. No regressions

    The second step is to not allow regressions.

    Whatever you fix in step 1 will be slow in the next few months.

    Even less than months the bigger the team or the rate of changes.

    So to prevent regressions, you need to constantly...

  25. Monitor
  26. Set limits

    The simplest way to prevent regressions is to set some limits.

    If you go over the limits, an email is sent, an alarm sounds, panic instills and you've got to fix whatever cause it was.

  27. e.g.
    max 2 scripts
    max 2 styles
    max 9 images
    max 0 redirects
  28. Scripting HTTPWatch

    Watching for violations of the limits manually every day is not a job anyone would want.

    So automating it will help a great way towards employee satisfaction :)

  29. var http = new ActiveXObject("HTTPWatch.Controller"),
        ie = http.IE.New(),
        ff = http.Firefox.New();

    This is how you open IE and FF with HTTPWatch's help.

    FF - yey!

  30. // browser cache
    ie.clearCache();
    
    // show HTTPWatch
    ie.OpenWindow(false);

    Examples of stuff you can do with the HTTPWatch API.

    You can for example hit the page with empty cache and then again with full cache.

    Best of all - these are the real browsers with their sometimes kinky behaviors.
    Actually if you setup several machines for the monitoring (or somehow do multiple IEs)
    you can test with different versions of the browsers. Nothing synthetic!

  31. ie.Record();
    ie.GotoUrl("yahoo.com");
    http.Wait(ie, -1);
    ie.Stop();

    Start monitoring, go to a page, stop monitoring after the page "settles" meaning some time after onload.

    ie.CloseBrowser();
  32. new HTTPWatch()
          http://github.com/stoyan/etc/

    I did this JavaScript thingie to make everything a little easier.

  33. var http = new HTTPWatch(ff);
    http.go(search.yahoo.com);
    http.done();

    Example usage.

  34. [video]
  35. var har = http.toHAR();
    har = eval(( + har + ));
    
    print(har.log.browser.name);
    print(har.log.browser.version);
    print(# requests: );
    print(har.log.entries.length);

    Opening and navigating browsers is cool. But we need some data back.

    HTTPWatch can export a HAR (HTTP Archive) file. I have this toHAR() method.
    It writes the file, than reads and returns it.
    You can than eval() it because it's just a JSON string.
    And you get the data back in convenient JS objects and arrays.

  36. Internet Explorer 6.0.29...
    # requests: 10
    
    Firefox 3.5.6
    # requests: 15

    Result of running the above.

  37. [video]
  38. var comps = http.getComponentsByType();
    
    for (var i in comps) {
      print(i);
      print(comps[i].length);
    }

    Another method I thought would be useful is getComponentsByType()

  39. redirect: 1
    text/html: 3
    image/gif: 4
    image/png: 3
    text/javascript: 1

    Results of the code above.

  40. But wait...

    There's more :)

  41. What about DOM?

    So far we only talked about HTTP traffic inspection - headers and such.

    Good news is that you can also inspect the DOM (in IE only) for any potential red flags.

    For example having the number of DOM elements sharply increase.

  42. [video]
  43. var http = new HTTPWatch();
    http.go(search.yahoo.com);
    
    var d = http.watch.container.document;
    
    print(d.getElementsByTagName(*).length);
    print(d.documentElement.innerHTML);

    That works!

    All your DOM voodoo skillz are suddenly reusable.

  44. require(statz.js);
    
    var doc = http.watch.container.document;
    var html = http.har.log.entries[0].response.content.text;
    
    var out = statz(document, html);
    print(out.join("\n"));

    This is me repurposing two old bookmarklets that gather some interesting stats (one of them was even featured on Ajaxian, remember?).

    It was pretty easy to repurpose the bookmarklets, because it's just JavaScript.

    The stats thingie can inspect both raw HTML that went over the wire, as well as innerHTML that was the result of any additional DOM manipulations.

  45. JS attributes (e.g. onclick): 1207 bytes
    CSS style attributes: 883
    Inline JS: 5243
    Inline CSS: 5015
    All innerHTML: 17283
    # DOM elements: 134
    
    Total size: 14124 bytes
    Content size: 401 bytes
    Content-to-markup ratio: 0.03

    Sample results.

  46. To summarize...
  47. JavaScript
    WSH
    HTTPWatch
    Monitor
    DOM and HTTP
    IE and Firefox
  48. Thanks
    
    phpied.com
 

JavaScript-style object literals in PHP

Sunday, March 20th, 2011

The object literal notation in JavaScript looks like:

var fido = {name: "Fido", barks: true};

or

var fido = {};
fido.name = "Fido";
fido.barks = true;

From assoc arrays to objects

In PHP you would call that an associative array.

$fido = array(
  'name' => "Fido",
  'barks' => true
);

And you can easily make it an object too:

$fido = (object)$fido;
echo gettype($fido); // "object"

Or if you want to start with a blank object and add stuff to it:

$fido = (object)array();

or

$fido = new StdClass();

and then

$fido->name = "Fido";
$fido->barks = true;

A little explanation maybe: objects in JavaScript are hashes, maps, whatever you decide to call them. Objects in PHP were an afterthought in the language and (at least initially) were not much more than "fancy arrays". Fancy associative arrays (hashes, maps, whatever you call them).

Objects in PHP need a class, but the new stdClass() lets you start quickly without the class {...} jazz. Same for casting an array (upgrading it in its fanciness) to an object with (object)array().

So far - so good. What about methods?

Methods anyone?

JavaScript doesn't care about properties versus methods. It's all members of an object (like elements of an assoc array). Only if a member happens to be a function, it's invokable.

fido.say = function () {
  if (this.barks) {
    return "Woof!";
  }
};

fido.say(); // "Woof!"

Turns out, since PHP 5.3 there are closures in PHP too. So you can do:

$fido->say = function() {
  if ($this->barks) {
    return "Woof";
  }
};

The difference is that $fido->say() won't work. Two reasons for that:

  1. say is not a method. It's a property. For PHP it matters. You can however assign the property say to a new variable $callme. This variable is now a closure object. As such you can invoke it:
    $callme = $fido->say;
    echo $callme();

    Note the $ in $callme().

  2. the above will also fail because $this is an weird context and doesn't point to the object $fido. But you can use $self and point it to the global object $fido.

So that's a little .... unpretty, but it works:

$fido = (object)array();
$fido->name = "Fido";
$fido->barks = true;

$fido->say = function() {
  $self =& $GLOBALS['fido'];
  if ($self->barks) {
    return "Woof";
  }
};

$callme = $fido->say;
echo $callme(); // "Woff!"

And a sprinkle of magic

We can make this prettier with the help of a little PHP magic. PHP has some magic methods going on and one of these is the __call() method. If you implement it in a class, then it will be invoked whenever someone tries to call a method that doesn't exist.

In our case $fido->say is not a method. So __call can intercept $fido->say() calls and invoke the $fido->say property as a closure object. Closures are callable and call_user_func() and call_user_func_array() work fine with them. So all in all we should make this work:

$fido = new JSObject();
$fido->name = "Fido";
$fido->barks = true;

$fido->say = function($self) {
  if ($self->barks) {
    return "Woof";
  }
};

echo $fido->say();

As you can see, very JavaScript-esque. Except that $this is $self and will always be the first argument passed to every method. The secret sauce to make this happen is the JSObject() class.

class JSObject {
  function __call($name, $args) {
    if (is_callable($this->$name)) {
      array_unshift($args, $this);
      return call_user_func_array($this->$name, $args);
    }
  }
}

Nice and easy. Namely:

  1. __call takes the name of the missing method and any arguments.
  2. It checks whether there's a callable property with the same name (a closure object property).
  3. It adds $this to the arguments list and calls the closure.

Yupee! Now you can haz moar class-less JS-like PHP objects :)

(Note that $this->$name is not a typo and should not be $this->name because it's a dynamic property name.)

And one more thing

If we add a constructor to JSObject, it can accept any properties at creation time. So you can be even closer to JavaScript and allow both creating an "empty" object and adding to it later, or creating an object and adding properties simultaneously.

The slightly modified JSObject:

class JSObject {
  function __construct($members = array()) {
    foreach ($members as $name => $value) {
      $this->$name = $value;
    }
  }
  function __call($name, $args) {
    if (is_callable($this->$name)) {
      array_unshift($args, $this);
      return call_user_func_array($this->$name, $args);
    }
  }
}

And example use:

$fido = new JSObject(array(
  'name' => "Fido",
  'barks'=> true,
  'say'  => function($self) {
    if ($self->barks) {
      return "Woof";
    }
  }
));

echo $fido->say(); // "Woff"

This is pretty close to what you can have in JavaScript (adding $ and ' even though we can do without them), only changing a few things like -> to . and => to :

$fido = {
  'name' : "Fido",
  'barks': true,
  'say'  : function() {
    if (this.barks) {
      return "Woof";
    }
  }
};
$fido.say(); // Woof

JS and PHP look like twins now don't they.

JS for PHP devs at confoo.ca

This was extracted from a talk I gave at the confoo.ca conference a week or so ago. Below are the slides:

JavaScript for PHP developers
View more presentations from Stoyan Stefanov

 

cssmin.js in windows shell

Friday, October 29th, 2010

JavaScript can run virtually anywhere, including as a windows exe and the windows command line.

Say you have a JavaScript function foo()

function foo(input) {
  var output = input;
  // .. unicorns
  return output;
}

In order to make this a windows shell script you add at the and a way to read standard input and then write to the standard output:

(function () {

    var input  = WScript.StdIn.ReadAll(),
        output = foo(input);

    WScript.StdOut.Write(output);

}());

Then you run this script, say foo.js, like:

$ cscript foo.js < input.txt

And it prints the output to the console.

If you want to read and print the code of foo.js itself you go:

$ cscript foo.js < foo.js
Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

function foo(input) {
  var output = input;
...

You can remove this "Microsoft (R) Windows..." stuff with //NoLogo parameter:

$ cscript //NoLogo foo.js < foo.js
function foo(input) {
  var output = input;
...

CSSMin.JS

Alrighty, going back to the title of the post.

CSSMin.js is a port of YUICompressor's CSS minifier (source, hosted tool). Now adding a few lines at the end makes a windows shell script:

(function () {

    var src = WScript.StdIn.ReadAll();

    if (!src) {
        // help!
        WScript.StdOut.WriteLine("cscript //NoLogo cssmin.js < in.css > out.css");
        return;
    }

    WScript.StdOut.Write(YAHOO.compressor.cssmin(src));

}());

Download it here.

Use it like:

$ cscript //NoLogo cssmin.js < in.css > out.css

Don't forget the //NoLogo or you'll end up with "Microsoft..." in your minified files

Random observation: "dude"[0] === "d" in most JS environments but is undefined in WSH (Windows Scripting Host). So "dude".substring(0, 1)

 

Preload, then execute

Saturday, October 23rd, 2010

I talked before about using an object element to load scripts and styles without executing/applying them to the current document. And this is nice for preloading assets - faster and less error-prone than simple inclusion if the document or iframe.

But what about preloading assets (scripts) and then executing them on the current page? Turns out it works just fine, like so:

  • create an object element and point its data to the JavaScript file to be loaded
  • listen to the load event of the object
  • once loaded, create a script element and point its src to the file, which would be already cached
  • optionaly, execute a callback function once the script element is properly inserted into the DOM

Something like this:

function yield(file, callback) {
    var o = document.createElement('object');
    o.data = file;

    o.width  = 0;
    o.height = 0;

    o.onload = function() {
        var s = document.createElement('script');
        s.src = file;
        s.onload = function() {
            callback(o, file);
        };
        document.getElementsByTagName('head')[0].appendChild(s);

    };

    document.body.appendChild(o);
}

But, but.. why?

Well, first off it's yet another way to load JavaScript files asynchronously, in a non-blocking fashion. And that's a good thing. Async = win. This method sure beats others such as document.write (eek) or iframe. It has advantage over XHR because there's no same-domain restriction. DOM include is probably the only simpler and more robust method, but there are cases where the object element might be preferable.

DOM-included scripts in FF and Opera always execute in order. That's cool, but sometimes you may prefer out of order async execution. Whenever a script is ready - go! Or sometimes you may prefer if the script is not executed at all! That may sound odd (why would I load it if I don't want to execute it?) but I have a use case - auto-complete. When your data source is on another domain and you can't use XHR, then JSON-P would be the way to go. But the network is a jungle and anything can happen.

Say you make requests for auto-complete suggestions with queries "h", "he", "hel", "hell", "hello". For some reason "hel" is really slow and you already have the results for "hell" and even "hello". What you want to do is just kill and cancel the "hel" response, who cares - it's outdated already. But if you use JSONP in FF and Opera "hel" will block the others. If order is not preserved it might be even weirder - you update the page with the "hello" suggestion and then "hel" finally arrives and updates the suggestion once again. Fail.

Using the object element approach you can keep a list of requests that went out and simply choose to ignore previous requests if newer responses are already there.

I have a test case even.

Small print

This technique assumes that the scripts you load don't have any headers preventing caching, otherwise they'll be requested twice.

My example code above ignored IE completely, but you can easily make it work, either using new Image() instead of object or tweaking the object with height and width of 1 as shown in the old article. Or you can use the trick that LABjs uses (I think) which is to DOM-insert a script element with invalid type such as cache/javascript. Also you need to listen to onreadystatechange instead of load event when inserting a DOM element.

 

The European Tour 2010

Saturday, October 2nd, 2010

So it's been pretty quiet around here. I'm still alive and very much so. Just awfully busy.

Bulgaria

I took a long trip in Bulgaria. About 1 month and 20 days. Including the traditional around-Bulgaria let's-get-drunk-in-different-cities-every-day tour de force with the gang. With unusual amount of guitar playing this time.

Not all fun and roses, though, I had to work most of the time for a superimportant gamechanging ultrasecret project for Yahoo! Search. And work on the finishing touches for a book.

France, Italy, UK

While spending time in Bulgaria I did some quick jumps to other European countries - France, Italy, UK. Everything is so close in Europe, it's a shame not to wander around once you cross the big pond.

In Paris I had the pleasure to practice some French, but most importantly to meet with the local speed freaks at the Web Perf Meetup organized by Éric Daspet. Great experience to interrupt sightseeing and soufflé-eating just to be among fellow geeks in a small and cozy group.

web perf meetup paris

photo: @tbassetto

Second tour leg

Back in California to the normal life and kids' school year start end of August. But a second European tour leg was just around the corner.

JSConfEU in Berlin was last week, this is hands-down the best conference. I cannot speak high enough about it, the organizers, the people, speakers, everyone, the parties (ouch, the parties). I met so many people I have been in contact with (or wanted to be) for quite a while, including the new book's tech reviewer Andrea aka @WebReflection. It's surprising, for the regular introvert geek I am, the pleasure of meeting and talking to people. Oh, the miracles of the little, or not so little, quantities of Berliner Pilsner.

Back from Berlin, a copy of the new book was waiting, how adorable! (I really need to put some marketing effort into it... at least put up a page what's it about, since people are asking). It's been rating very well in Amazon's "JavaScript" category. Saw it at #3 a few times, watch out "good parts"!

Next week it's Fronteers conference in Amsterdam. Really looking forward to this one. After all, the one and only PPK started it all. I'll be speaking about progressive downloads and progressive rendering. Progress is important, progress is critical! And I've never been to Amsterdam.

Then Moscow and highload++ conference. How cool that would be. I always wanted to see Moscow or any part of Russia really.

So... busy, busy, busy. And another book baby on the road. And one last conference but here in LA's "backyard" Las Vegas this time.

 

JavaScript JS Array Documentation (#jsonf, #promotejs)

Sunday, September 26th, 2010

JavaScript JS Documentation: JS Array unshift, JavaScript Array unshift, JS Array .unshift, JavaScript Array .unshift

Join me in the initiative to promote proper JavaScript documentation from MDC. It's embarrassing the quality of links that pop up in search engines when searching JavaScript general questions. Let's help connect newcomers to JavaScript with better reading. PromoteJS.com will give you a unique code.

 

Preload CSS/JavaScript without execution

Wednesday, April 21st, 2010

Preloading components in advance is good for performance. There are several ways to do it. But even the cleanest solution (open up an iframe and go crazy there) comes at a price - the price of the iframe and the price of parsing and executing the preloaded CSS and JavaScript. There's also a relatively high risk of potential JavaScript errors if the script you preload assumes it's loaded in a page different than the one that preloads.

After a bit of trial and lot of error I think I came up with something that could work cross-browser:

  • in IE use new Image().src to preload all component types
  • in all other browsers use a dynamic <object> tag

Code and demo

Here's the final solution, below are some details.

In this example I assume the page prefetches after onload some components that will be needed by the next page. The components are a CSS, a JS and a PNG (sprite).

window.onload = function () {

    var i = 0,
        max = 0,
        o = null,

        // list of stuff to preload
        preload = [
            'http://tools.w3clubs.com/pagr2/<?php echo $id; ?>.sleep.expires.png',
            'http://tools.w3clubs.com/pagr2/<?php echo $id; ?>.sleep.expires.js',
            'http://tools.w3clubs.com/pagr2/<?php echo $id; ?>.sleep.expires.css'
        ],
        isIE = navigator.appName.indexOf('Microsoft') === 0;

    for (i = 0, max = preload.length; i < max; i += 1) {

        if (isIE) {
            new Image().src = preload[i];
            continue;
        }
        o = document.createElement('object');
        o.data = preload[i];

        // IE stuff, otherwise 0x0 is OK
        //o.width = 1;
        //o.height = 1;
        //o.style.visibility = "hidden";
        //o.type = "text/plain"; // IE 
        o.width  = 0;
        o.height = 0;

        // only FF appends to the head
        // all others require body
        document.body.appendChild(o);
    }

};

A demo is here:
http://phpied.com/files/object-prefetch/page1.php?id=1
In the demo the components are delayed with 1 second each and sent with Expries header. Feel free to increment the ID for a new test with uncached components.

Tested in FF3.6, O10, Safari 4, Chrome 5, IE 6,7,8.

Comments

  • new Image().src doesn't do the job in FF because it has a separate cache for images. Didn't seem to work in Safari either where CSS and JS were requested on the second page where they sould've been cached
  • the dynamic object element has to be outside the head in most browsers in order to fire off the downloads
  • dynamic object works also in IE7,8 with a few tweaks (commented out in the code above) but not in IE6. In a separate tests I've also found the object element to be expensive in IE in general.

That's about it. Below are some unsuccessful attempts I tried which failed for various reasons in different browsers.

Other unsuccessful attempts

1.
I was actually inspired by this post by Ben Cherry where he loads CSS and JS in a print stylesheet. Clever hack, unfortunately didn't work in Chrome which caches the JS but doesn't execute it on the next page.

2.
One of the comments on Ben's post suggested (Philip and Dejan said the same) using invalid type attribute to prevent execution, e.g. text/cache.

var s = document.createElement('script');
s.src = preload[1];
s.type = "text/cache";
document.getElementsByTagName('head')[0].appendChild(s);

That worked for the most parts but not in FF3.6 where the JavaScript was never requested.

3.
A dynamic link prefetch didn't do anything, not even in FF which is probably the only browser that supports this.

for (i = 0, max = preload.length; i < max; i += 1) {
    var link = document.createElement('link');
    link.href = preload[i];
    link.rel = "prefetch";
    document.getElementsByTagName('head')[0].appendChild(link);
}

Then it took a bit of trial/error to make IE7,8 work with an object tag, before I stumbled into IE6 and gave up in favor of image src.

In conclusion

I believe this is a solution I could be comfortable with, although it involves user agent sniffing. It certainly looks less hacky than loading JS as CSS anyways. And object elements are meant to load any type of component so no semantic conflict here I don't believe. Feel free to test and report any edge cases or browser/OS combos. (JS errors in IE on the second page are ok, because I'm using console.log in the preloaded javascript)

Thanks for reading!

 

Publishing 5 books this year

Thursday, April 1st, 2010

So I'll be publishing 5 books this year. Isn't that incredible? Is it even possible? And good quality books at that? It's a nice challenge (my last year's challenge failed, I didn't even bother to count how bad it failed). I think it's possible, especially if you bend a little bit the meaning of "5", "year", "publishing" and "me" :)

Book #1 - High-Performance JavaScript

hpjs

Let's start bending - this is a book where I wrote just one chapter. It's a book by Nicholas Zakas with contributions from:

And I wrote my chapter mainly the last year. My chapter is about the DOM. But the book became available just now, few days ago, so it's published this year (bending, bending...)

Book #2 - JavaScript Patterns

I am hard at work on this one currently (explains the low activity on this blog). I started last year but only finished two chapters in '09. The bending part here is that I've already given presentations on the topic and have been writing a "patterns" column for JSMag for a while, so I can recycle quite a bit of content.

You can see the tentative cover, I hope it stays tentative and we can replace the hen with a nice cute little zebra (a.k.a. donkey with patterns). Between you and me, I think there's a new designer in O'Reilly with a bird fetish.

I expect the first draft for this one to complete within weeks. And no, it's not about implementing the Gang of Four patterns in JavaScript (has been done already by Ross, see above), although there's one chapter on a selected few - Singleton, Factory, Observer, Proxy, Decorator...

Book #3 - Speed Matters

I've contracted with Peachpit Press to write a book about performance targeted mainly at designers. It will be about the business (why go fast), technology (how) and psychology (perception of speed) of web performance. I'm excited about this one for a number of reasons:

  • there's a lot of misconceptions being spread around in designer blogs and books, especially sad when one of the books in question is a sort of a bible for web designers. I mean things like PNG vs. GIF, gzipping and others. I hope I can present a readable, concise and, above all, technically correct text for designers who may find Steve Souders' HPWS, a.k.a. "The Bible" a little too dry because it's from O'Reilly and has no colors
  • the publisher is considering a sort of novel approach to writing the book, fingers crossed, because I believe it's the right way to write technical books.
  • at the very least, the book will be available as early drafts while it's being written, which is new to me, but always wanted to do.
  • the book will be full color - again, new experience to me

The bending here comes from the fact that I'll try to reuse from the perf advent calendar if I can. So some content may be pre-written.

Book #4 - Object-Oriented JavaScript (2nd edition)

The bending here is obvious - it's just a second edition, not a completely new book from scratch. My goal here is:

  • address errata
  • address some excellent critiques (of this otherwise bestselling book!), such as this one by @kangax, which is the article that actually prompted me to pitch a second edition to the publisher. So many thanks to Yuri! Also thanks to Asen who's been sending me invaluable and detailed feedback on the first edition. And now thanks to Asen and Kangax (and also Dmitry) I'm spending some time lurking on comp.lang.javascript mailing list, which is full of great discussions.
  • ECMAScript5 update
  • some concepts such as hoisting, NFE, property attributes, etc
  • one completely new chapter on testing and docs
  • answers to the end-of-chapter exercises - an often-requested update

Hoping this title will not take a lot of time.

And since these 4 books should be finished by the end of August or thereabouts, this will give me whole 4 months (1/3 of an year) to dive into something I've been thinking about, two things actually - CSS and self-publishing.

Book #5 - CSS for web devs

CSS is widely misunderstood by many people, me including. I'm convinced we only use a portion of all that CSS is, and use it badly. I'm not saying it will be CSS: The Good Parts, but I plan to address what I consider bad habits in CSS (mis)use and write a book as a learning experience. This is the best way to learn IMO. It will be self-published and probably available online for free too. And by self-publish I don't mean lulu.com or some of the other resellers, but working with the printer and distributor directly.

Too ambitious? April Fool's?

Probably, but with all the pre-written stuff and other cheating, it may very well be doable. Then I guess I'll take a 5 year break :)

 

YUICompressor’s CSSMin

Wednesday, March 10th, 2010

Honored to be a part of the YUI project, I am now helping with the maintenance of the CSSMin part of the YUICompressor. My changes are now part of the trunk on github, so I'm official. Next on the agenda is documenting the thing, so that's what I'll try to do here, maybe in a few posts. You know, divide and conquer.

PHP, Java and a JavaScript port

Originally written in PHP by Isaac Schlueter and ported to Java by Julien Lecomte, CSSMin got a JavaScript port by yours truly some time ago. Because, after all, JavaScript is the language of the web, isn't it?

You can play with the latest git version of the JS port online here.

I'm also happy to report that the JS port is now used in PageSpeed and YSlow (as you probably know Firefox extensions are written in JavaScript)

Page Speed

YSlow

Building

If you want to play on your own with the source version of YUICompressor without waiting for the next release, you can build it like so:

  1. Checkout or download the code from http://github.com/yui/yuicompressor/
  2. Navigate to the root yuicompressor/ directory
  3. Type ant and hit enter

In order for this to work you need a somewhat recent Java SDK installed and also Ant running. (On the Mac, just do port install apache-ant to get Ant)

This is for the Java version, the JS version needs no building, of course.

Tests

There's a bunch of new tests now (and if you want to contribute to the project, you can always write more tests and test cases for any bugs), you can run them with the suite script that Isaac wrote:

  1. cd tests/
  2. ./suite.sh

One thing I added (and loved it) is to run the tests using the JS port as well. Since the JS min part is using Mozilla's Rhino (slightly modified), Rhino is part of the code. So I'm using this already available JavaScript interpreter to run the JS port. Convenient.

The procedure to write new tests is simple:

  1. Create source CSS file in the tests/ directory, e.g. new-test.css
  2. Create a new file with the expected result and name it with a .min extension, e.g. new-test.css.min

You can use the handy-dandy online version to help with the tests creation.

Next time

With those details out of the way, the next time I'll talk more about the different things that CSSMin does to your CSS code. Thanks for reading!

 

One-click Minifier Gadget (OMG) – initial checkin

Sunday, January 31st, 2010

So I've been thinking and talking to folks about this idea of having one-stop shop for all your minification needs. Minification of JS and CSS as well as image optimization helps site performance by reducing download sizes. This is good. But not a lot of people do it.

People don't do it, because it's a PITA :) It's simple enough, but with deadlines upon you and all that, you don't want to do an extra step. That's why having a build process helps, by automating this. But setting up a build process is yet another PITA. So it goes.

So my idea was to help busy designers and developers, that wouldn't invest their time researching which minifiers are good, downloading setting up, learning about the 10+ PNG optimization tools... That's how the the idea for the one-click OMG tool came about. (One-drag is more appropriate, come to think of it...) One tool that runs on all operating systems - Win, Mac, Linux - and delivers all minification and optimization tools you need as one package.

Running

Running the tool is as simple as drag/dropping a bunch of files and directories. Here I've dropped "wordpress" directory. The tool recursively looks into the dropped files for things it can optimize. More information here.

OMG screenshot

Download

Version 0.0.1 is here. It doesn't do image optimization, only JS and CSS minification, but please feel free to download and give it a shot. Unzip the package for your OS and run omg.exe (Windows), OMG.app (Mac), or the omg binary (Linux)

Open source

The code is on GitHub. Fork and enjoy.

The developer's notes are there too - how to setup, run, package. Also a list of todos if you want to help.

Next?

This is just a preliminary version. Feel free to join, comment, suggest. Hate the name? Say so :)

Personally, looks like my plate is very full for the next moth or two, so I probably won't be actively working on the tool. I hope though the foundation is good enough and relatively documented, should be easy to pick up if anyone's interested in contributing.

Built with XUL

This has been a learning experience for me with XULRunner. I loved it. I love the idea of being able to create cross-OS desktop apps with JavaScript alone.

Behind the scenes, I'm using my JavaScript port of YUICompressor's CSSmin and Doug Crockford's JSMin. JSMin should be replaced with YUICompressor (or Google closure compiler) in the next release.

 

Extreme JavaScript optimization

Sunday, December 20th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 20 This article is part of the 2009 performance advent calendar experiment. Today's article is a second contribution from Ara Pehlivanian (here's the first).

There's a Belorussian translation provided by Patricia. Thanks!

Ara PehlivanianAra Pehlivanian has been working on the Web since 1997. He's been a freelancer, a webmaster, and most recently, a Front End Engineer at Yahoo! Ara's experience comes from having worked on every aspect of web development throughout his career, but he's now following his passion for web standards-based front-end development. When he isn't speaking and writing about best practices or coding professionally, he's either tweeting as @ara_p or maintaining his personal site at http://arapehlivanian.com/.

There's an odd phenomenon underway in the JavaScript world today. Though the language has remained relatively unchanged for the past decade, there's an evolution afoot among its programmers. They're using the same language that brought us scrolling status bar text to write some pretty heavy duty client-side applications. Though this may seem like we're entering a Lada in an F1 race, in reality we've spent the last ten years driving an F1 race car back and forth in the driveway. We were never using the language at its full potential. It took the discovery of Ajax to launch us out of the driveway and onto the race track. But now that we're on the track, there's a lot of redlining and grinding of gears going on. Not very many people it seems, know how to drive an F1 race car. At least not at 250 mph.

The thing of it is, it's pretty easy to put your foot to the floor and get up to 60 mph. But very soon you'll have to shift gears if you want to avoid grinding to a halt. It's the same thing with writing large client-side applications in JavaScript. Fast processors give us the impression that we can do anything and get away with it. And for small programs it's true. But writing lots of bad JavaScript can very quickly get into situations where your code begins to crawl. So just like an average driver needs training to drive a race car, we need to master the ins and outs of this language if we're to keep it running smoothly in large scale applications.

Variables

Let's take a look at one of the staples of programming, the variable.Some languages require you to declare your variables before using them, JavaScript doesn't. But just because it isn't required doesn't mean you shouldn't do it. That's because in JavaScript if a variable isn't explicitly declared using the 'var' keyword, it's considered to be a global, and globals are slow. Why? Because the interpreter needs to figure out if and where the variable in question was originally declared, so it goes searching for it. Take the following example.

function doSomething(val) {
    count += val;
};

Does count have a value assigned to it outside the scope of doSomething? Or is it just not being declared correctly? Also, in a large program, having such generic global variable names makes it difficult to keep collisions from happening.

Loops

Searching the scope chain for where count is declared in the example above isn't such a big deal if it happens once. But in large-scale web applications, not very much just happens once. Especially when loops are concerned. The first thing to remember about loops, and this isn't just for JavaScript, is to do as much work outside the loop as possible. The less you do in the loop, the faster your loop will be. That being said, let's take a look at the most common practice in JavaScript loops that can be avoided. Take a look at the following example and see if you can spot it:

for (var i = 0; i < arr.length; i++) {
    // some code here
}

Did you see it? The length of the array arr is recalculated every time the loop iterates. A simple fix for this is to cache the length of the array like so:

for (var i = 0, len = arr.length; i < len; i++) {
    // some code here
}

This way, the length of the array is calculated just once and the loop refers to the cached value every time it iterates.

So what else can we do to improve our loop's performance? Well, what other work is being done on every iteration? Well, we're evaluating whether the value of i is less than the value of len and we're also increasing i by one. Can we reduce the number of operations here? We can if the order in which our loop is executed doesn't matter.

for (var i = 100; i--; ) {
    // some code here
}

This loop will execute 50% faster than the one above because on every iteration it simply subtracts a value from i, and since that value is not "falsy," in other words it isn't 0, then the loop goes on. The moment the value hits 0, the loop stops.

You can do this with other kinds of loops as well:

while (i--) {
    // some code here
}

Again, because the evaluation and the operation of subtracting 1 from i is being done at the same time, all the while loop needs is for i to be falsy, or 0, and the loop will exit.

Caching

I touched briefly on caching above when we cached the array length in a variable. The same principle can be applied in many different places in JavaScript code. Essentially, what we want to avoid doing is sending the interpreter out to do unnecessary work once it's already done it once. So for example, when it comes to crawling the scope chain to find a global variable for us, caching it the reference locally will save the interpreter from fetching it every time. Here, let me illustrate:

var aGlobalVar = 1;

function doSomething(val) {
    var i = 1000, agv = aGlobalVar;
    while (i--) {
        agv += val;
    };
    aGlobalVar = agv;
};

doSomething(10);

In this example, aGlobalVar is only fetched twice, not over a thousand times. We fetch it once to get its value, then we go to it again to set its new value. If we had used it inside the while loop, the interpreter would have gone out to fetch that variable a thousand times. In fact, the loop above takes about 3ms to run whereas if avg += val; were replaced with aGlobalVar += val; then the loop would take about 10ms to run.

Property Depth

Nesting objects in order to use dot notation is a great way to namespace and organize your code. Unforutnately, when it comes to performance, this can be a bit of a problem. Every time a value is accessed in this sort of scenario, the interpreter has to traverse the objects you've nested in order to get to that value. The deeper the value, the more traversal, the longer the wait. So even though namespacing is a great organizational tool, keeping things as shallow as possible is your best bet at faster performance. The latest incarnation of the YUI Library evolved to eliminate a whole layer of nesting from its namespacing. So for example, YAHOO.util.Anim is now Y.Anim.

Summary

These are just a few examples of how to improve your code's performance by paying attention to how the JavaScript interpreter does its work. Keep in mind though that browsers are continually evolving, even if the language isn't. So for example, today's browsers are introducing JIT compilers to speed up performance. But that doesn't mean we should be any less vigilant in our practices. Because in the end, when your web app is a huge success and the world is watching, every millisecond counts.

 

The new game show: “Will it reflow?”

Saturday, December 19th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 19 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the articles to come.

Intrigued by Luke Smith's comment and also Alois Reitbauer's comment on the previous post about rendering I did some more testing with dynaTrace and SpeedTracer. Also prompted by this tweet, I wanted to provide an example of avoiding reflows by using document fragments as well as hiding elements with display: none. (btw, sorry that I'm slow to respond to tweets and blog comments, just too much writing lately with the crazy schedule, but I do appreciate every tweet and comment!)

So welcome to the new game show: "Will it reflow?" where we'll look into a few cases where it's not so clear if the browser will do a reflow or just a repaint. The test page is here.

Changing classnames

The first test is fairly straightforward - we only want to check what happens when you change the class name of an element. So using "on" and "off" class names and changing them on mouse over.

.on {background: yellow; border: 1px solid red;}
.off {background: white; border: 1px dashed green;}

Those CSS rules shouldn't trigger a reflow, because no geometry is being changed. Although the test is pushing it a bit by changing borders, which may affect geometry, but not in this case.

The test code:

// test #1 - class name change - will it reflow?
var onoff = document.getElementById('on-off');
onoff.onmouseover = function(){
  onoff.className = 'on' ;
};
onoff.onmouseout = function(){
  onoff.className = 'off';
};

Sooo.. will it reflow?

In Chrome - no! In IE - yes.

In IE, even changing the class name declarations to only change color, which is sure not to cause reflow, still caused a reflow. Looks like in IE, any type of className change causes a reflow.

cssText updates

The recommended way to update multiple styles in one shot (less DOM access, less reflows) is to update the element's style.cssText property. But.. will it reflow when the style changes do not affect geometry?

So let's have an element with a style attribute:

...style="border: 1px solid red; background: white"...

The JavaScript to update the cssText:

// test #2 - cssText change - will it reflow?
var csstext = document.getElementById('css-text');
csstext.onmouseover = function(){
  csstext.style.cssText += '; background: yellow; border: 1px dashed green;';
};
csstext.onmouseout = function(){
  csstext.style.cssText += '; background: white; border: 1px solid red;';
};

Will it reflow?

In Chrome - no! In IE - yes.

Even having cssText (and the initial style) only play with color, there's still a reflow. Even trying to just write the cssText property (as opposed to read/write with += ) still causes a reflow. The fact that cssText property is being updated causes IE to reflow. So there might be cases where setting individual properties separately (like style.color, style.backgroundColor and so on) which don't affect geometry, might be preferable to touching the cssText.

Next contestant in the game show is...

addRule

Will the browser reflow when you update stylesheet collections programatically? Here's the test case using addRule and removeRule (which in Firefox are insertRule/deleteRule):

// test #3 - addRule - will it reflow?
var ss = document.styleSheets[0];
var ruletext = document.getElementById('ruletext');
ruletext.onmouseover = function(){
  ss.addRule('.bbody', 'color: red');
};
ruletext.onmouseout = function(){
  ss.removeRule(ss.rules.length - 1);
};

Will it? Will it?

In Chrome - yes. The fact that style rules in the already loaded stylesheet have been touched, causes Chrome to reflow and repaint. Even though class .bbody is never used. Same when creating a new rule with selector body {...} - reflow, repaint.

In IE there's a repaint definitely, and there's also a kind of reflow. Looks like dynaTrace shows two kinds of rendering calculation indicators: "Calculating generic layout" and "Calculating flow layout". Not sure what is the difference (web searches disappointingly find nada/zero/rien for the first string and my previous blog post for the second). Hopefully "generic" would be less expensive than "flow".

display: none

In my previous post I boldly claimed that elements with display: none will not have anything to do with the render tree. IE begs to differ (thanks to dynaTrace folks for pointing that out).

A good way to minimize reflows is to update the DOM tree "offline" out of the live document. One way to do so is to hide the element while updates are taking place and then show it again.

Here's a test case where rendering and geometry are affected by simply adding more text content to an element by creating new text nodes.

// test #4 - display: none - will it reflow
var computed, tmp;
var dnonehref = document.getElementById('display-none');
var dnone = document.getElementById('bye');
if (document.body.currentStyle) {
  computed = dnone.currentStyle;
} else {
  computed = document.defaultView.getComputedStyle(dnone, '');
}

dnonehref.onmouseover = function() {
  dnone.style.display = 'none';
  tmp = computed.backgroundColor;
  dnone.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  dnone.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  dnone.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
}
dnonehref.onmouseout = function() {
  dnone.style.display = 'inline';
}

Will it reflow?

In Chrome - no. Although it does do "restyle" (calculating non-geometric styles) every time a text node is added. Not sure why this restyling is needed.

In IE - yes. Unfortunatelly display: none seems to have no effect on rendering in IE, it still does reflows. I tried with removing the show/hide code and having the element hidden from the very beginning (with an inline style attribute). Same thing - reflow.

document fragment

Another way to preform updates off-DOM is to create a document fragment and once ready, shove the fragment into the DOM. The beauty is that the children of the fragment get copied, not the fragment itself, which makes this method pretty convenient.

Here's the test/example. And will it reflow?

// test #5 - fragment - will it reflow
var fraghref = document.getElementById('fragment');
var fragment = document.createDocumentFragment();
fraghref.onmouseover = function() {
  fragment.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  fragment.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  fragment.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
}
fraghref.onmouseout = function() {
  dnone.appendChild(fragment);
}

In Chrome - no! And no rendering activities take place until the fragment is added to the live DOM. Then, just like with display: none a restyle is being performed for every new text node inserted. And even though the behavior is the same for fragments as for updating hidden elements, fragments are still preferable, because you don't need to hide the element (which will cause another reflow) initially.

In IE - no reflow! Only when you add the final result to the live DOM.

Thanks!

Thank you for reading. Tomorrow if all goes well there should be a final post related to JavaScript and then moving on to ... other topics ;)

 

DOM access optimization

Friday, December 18th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 18 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the articles to come.

This blog series has sailed from the shores of networking, passed down waterfalls and reflows, and arrived in ECMAScriptland. Now, turns out there's one bridge to cross to get to DOMlandia.

(OK, I need to get some sleep, evidently. Anyway.) Ara Pehlivanian talked about strategies for loading JavaScript code. Yesterday's post was about rendering and how you can prevent making things worse in JavaScript. Today's post will be about DOM access optimizations and, if all is good, tomorrow's post will round up the JavaScript discussion with some techniques for extreme optimization.

What's with the DOM

Document Object Model (DOM) is a language-independent API for accessing and working with a document. Could be an HTML document, or XML, SVG and so on. DOM is not ECMAScript. ECMAScript is just one way to work with the DOM API. They both started in the web browser but now things are different. ECMAscript has many other uses and so has the DOM. You can generate a page server side, using the DOM is you like. Or script Photoshop with ECMAScript.

All that goes to show that ECMAScript and DOM are now separate, they make sense on their own, they don't need each other. And they are kept separate by the browsers.

For example WebCore is the layout, rendering and DOM library used by WebKit, while JavaScriptCore (most recently rewritten as SquirrelFish) is the implementation of ECMAScript. In IE - Trident (DOM) and JScript. In Firefox - Gecko (DOM) and SpiderMonkey (ECMAScript).

The toll bridge

An excellent analogy I heard in this video from John Hrvatin of MSIE is that we can think of the DOM as a piece of land and JavaScript/ECMAScript as another piece of land. Both connected via a toll bridge. I tried to illustrate this analogy here.

domland and ecmaland connected with a bridge

All your JavaScript code that doesn't require a page - code such as loops, ifs, variables and a handful of built-in functions and objects - lives in ECMALand. Anything that starts with document.* lives in DOMLand. When your JavaScript needs to access the DOM, you need to cross that bridge to DOMlandia. And the bad part is that it's a toll bridge and you have to pay a fee every time you cross. So, the more you cross that bridge, the more you pay your performance toll.

How bad?

So, how serious is that performance penalty? Pretty serious actually. DOM access and manipulations is probably the most expensive activity you do in your JavaScript, followed by layouting (reflowing and painting activities). When you look for problems in your JavaScript (you use a profile instead of shooting in the dark, of course, but still) most likely it's the DOM that's slowing you down.

As an illustration, consider this bad, bad code:

// bad
for (var count = 0; count < 15000; count++) {
    document.getElementById('here').innerHTML += 'a';
}

This code is bad because it touches the DOM twice on every loop tick. It doesn't cache the reference to the DOM element, it looks for that element every time. Then this code also updates the live DOM which means it causes a reflow and a repaint (which are probably buffered by the browsers and executed in batches, but still bad).

Compare with the following code:

// better
var content = '';
for (var count = 0; count < 15000; count++) {
    content += 'a';
}
document.getElementById('here').innerHTML += content;

Here's we only touch the DOM twice at the end. The whole time otherwise we work in ECMAland with a local variable.

And how bad is the bad example? It's over 100 times worse in IE6,7 and Safari, over 200 times worse in FF3.5 and IE8 and about 50 times worse in Chrome. We're not talking percentages here - we talk 100 times worse.

Now obviously this is a bad and made up example, but it does show the magnitude of the problem with DOM access.

Mitigating the problem - don't touch the DOM

How to speed up DOM access? Simply do less of it. If you have a lot of work to do with the DOM, cache references to DOM elements so you don't have to query the DOM tree every time to find them. Cache the values of the DOM properties if you'll do a chunk of of work with them. And by cache I mean simply assign them to local variables. Use selectors API where available instead of crawling the DOM yourself (upgrade your JavaScript library if it's not taking advantage of the selectors API). Be careful with HTML Collections.

// bad
document.getElementById('my').style.top = "10px";
document.getElementById('my').style.left = "10px";
document.getElementById('my').style.color = "#dad";

// better
var mysty = document.getElementById('my').style;
mysty.top = "10px";
mysty.left = "20px";
mysty.color = "#dad";

// better
var csstext = "; top: 10px; left: 10px; color: #dad;";
document.getElementById('my').style.cssText += csstext

Basically, every time you find you're accessing some property or object repeatedly, assign it to a local variable and work with that local variable.

HTMLCollections

HTMLCollections are objects returned by calls to document.getElementsByTagName(), document.getElementsByClassName() and others, also by accessing the old-style collections document.links, document.images and the like. These HTMLCollection objects are array-like, list-like objects that contain pointers to DOM elements.

The special thing about them is that they are live queries against the underlying document. And they get re-run a lot, for example when you loop though the collection and access its length. The fact that you touch the length requires re-querying of the document so that the most up-to-date information is returned to you.

Here's an example:

// slow
var coll = document.getElementsByTagName('div');
for (var count = 0; count < coll.length; count++) {
    /* do stuff */
}

// faster
var coll = document.getElementsByTagName('div'),
    len = coll.length;
for (var count = 0; count < len; count++) {
    /* do stuff */
}

The slower version requeries the document, the faster doesn't because we use the local value for the length. How slower is the slower? Depends on the document and how many divs in it, but in my tests anywhere between 2 times slower (Safari) to 200 times slower (IE7)

Another thing you can do (especially if you'll loop the collection a few times) is to copy the collection into an array beforehand. Accessing the array elements will be significantly faster than accessing the DOM elements in the collection, again 2 to 200 times faster.

Here's an example function that turns the collection into an array:

function toArray(coll) {
    for (var i = 0, a = [], len = coll.length; i < len; i++) {
        a[i] = coll[i];
    }
    return a;
}

If you do that you also need to account for the one-off cost of copying that collection to an array.

Using event delegation

Event delegation is when you attach event listener to a parent element and it handles all the events for the children because of the so-called event bubbling It's a graceful way to relieve the browser from a lot of extra work. The benefits:

  • You need to write less event-attaching code.
  • You will usually use fewer functions to handle the events because you're attaching one function to handle parent events, not individual function for each child element. This means less functions to store in memory and keep track of.
  • Less events the browser needs to monitor
  • Easier to detach event handlers when an element is removed and therefore easier to prevenk IE memory leaks. Sometimes you don't even need to detach the event handler if children change, but the event-handling parent stays the same.

Thanks for reading!

  • Don't touch the DOM when you can avoid it, cache DOM access to local references
  • Cache length of HTMLCollections to a local variable while looping (good practice for any collections or arrays looping anyway). Copy the collection to an array if you'll be looping several times.
  • Use event delegation

Links

 

Rendering: repaint, reflow/relayout, restyle

Thursday, December 17th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 17 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the articles to come.

Nice 5 "R" words in the title, eh? Let's talk about rendering - a phase that comes in the Life of Page 2.0 after, and sometimes during, the waterfall of downloading components.

So how does the browser go about displaying your page on the screen, given a chunk of HTML, CSS and possibly JavaScript.

The rendering process

Different browsers work differently, but the following diagram gives a general idea of what happens, more or less consistently across browsers, once they've downloaded the code for your page.

Rendering process in the browser

  • The browser parses out the HTML source code (tag soup) and constructs a DOM tree - a data representation where every HTML tag has a corresponding node in the tree and the text chunks between tags get a text node representation too. The root node in the DOM tree is the documentElement (the <html> tag)
  • The browser parses the CSS code, makes sense of it given the bunch of hacks that could be there and the number of -moz, -webkit and other extensions it doesn't understand and will bravely ignore. The styling information cascades: the basic rules are in the User Agent stylesheets (the browser defaults), then there could be user stylesheets, author (as in author of the page) stylesheets - external, imported, inline, and finally styles that are coded into the style attributes of the HTML tags
  • Then comes the interesting part - constructing a render tree. The render tree is sort of like the DOM tree, but doesn't match it exactly. The render tree knows about styles, so if you're hiding a div with display: none, it won't be represented in the render tree. Same for the other invisible elements, like head and everything in it. On the other hand, there might be DOM elements that are represented with more than one node in the render tree - like text nodes for example where every line in a <p> needs a render node. A node in the render tree is called a frame, or a box (as in a CSS box, according to the box model). Each of these nodes has the CSS box properties - width, height, border, margin, etc
  • Once the render tree is constructed, the browser can paint (draw) the render tree nodes on the screen

The forest and the trees

Let's take an example.

HTML source:

<html>
<head>
  <title>Beautiful page</title>
</head>
<body>

  <p>
    Once upon a time there was
    a looong paragraph...
  </p>

  <div style="display: none">
    Secret message
  </div>

  <div><img src="..." /></div>
  ...

</body>
</html>

The DOM tree that represents this HTML document basically has one node for each tag and one text node for each piece of text between nodes (for simplicity let's ignore the fact that whitespace is text nodes too) :

documentElement (html)
    head
        title
    body
        p
            [text node]

        div
            [text node]

        div
            img

        ...

The render tree would be the visual part of the DOM tree. It is missing some stuff - the head and the hidden div, but it has additional nodes (aka frames, aka boxes) for the lines of text.

root (RenderView)
    body
        p
            line 1
	    line 2
	    line 3
	    ...

	div
	    img

	...

The root node of the render tree is the frame (the box) that contains all other elements. You can think of it as being the inner part of the browser window, as this is the restricted area where the page could spread. Technically WebKit calls the root node RenderView and it corresponds to the CSS initial containing block, which is basically the viewport rectangle from the top of the page (0, 0) to (window.innerWidth, window.innerHeight)

Figuring out what and how exactly to display on the screen involves a recursive walk down (a flow) through the render tree.

Repaints and reflows

There's always at least one initial page layout together with a paint (unless, of course you prefer your pages blank :) ). After that, changing the input information which was used to construct the render tree may result in one or both of these:

  1. parts of the render tree (or the whole tree) will need to be revalidated and the node dimensions recalculated. This is called a reflow, or layout, or layouting. (or "relayout" which I made up so I have more "R"s in the title, sorry, my bad). Note that there's at least one reflow - the initial layout of the page
  2. parts of the screen will need to be updated, either because of changes in geometric properties of a node or because of stylistic change, such as changing the background color. This screen update is called a repaint, or a redraw.

Repaints and reflows can be expensive, they can hurt the user experience, and make the UI appear sluggish.

What triggers a reflow or a repaint

Anything that changes input information used to construct the rendering tree can cause a repaint or a reflow, for example:

  • Adding, removing, updating DOM nodes
  • Hiding a DOM node with display: none (reflow and repaint) or visibility: hidden (repaint only, because no geometry changes)
  • Moving, animating a DOM node on the page
  • Adding a stylesheet, tweaking style properties
  • User action such as resizing the window, changing the font size, or (oh, OMG, no!) scrolling

Let's see a few examples:

var bstyle = document.body.style; // cache

bstyle.padding = "20px"; // reflow, repaint
bstyle.border = "10px solid red"; // another reflow and a repaint

bstyle.color = "blue"; // repaint only, no dimensions changed
bstyle.backgroundColor = "#fad"; // repaint

bstyle.fontSize = "2em"; // reflow, repaint

// new DOM element - reflow, repaint
document.body.appendChild(document.createTextNode('dude!'));

Some reflows may be more expensive than others. Think of the render tree - if you fiddle with a node way down the tree that is a direct descendant of the body, then you're probably not invalidating a lot of other nodes. But what about when you animate and expand a div at the top of the page which then pushes down the rest of the page - that sounds expensive.

Browsers are smart

Since the reflows and repaints associated with render tree changes are expensive, the browsers aim at reducing the negative effects. One strategy is to simply not do the work. Or not right now, at least. The browser will setup a queue of the changes your scripts require and perform them in batches. This way several changes that each require a reflow will be combined and only one reflow will be computed. Browsers can add to the queued changes and then flush the queue once a certain amount of time passes or a certain number of changes is reached.

But sometimes the script may prevent the browser from optimizing the reflows, and cause it to flush the queue and perform all batched changes. This happens when you request style information, such as

  1. offsetTop, offsetLeft, offsetWidth, offsetHeight
  2. scrollTop/Left/Width/Height
  3. clientTop/Left/Width/Height
  4. getComputedStyle(), or currentStyle in IE

All of these above are essentially requesting style information about a node, and any time you do it, the browser has to give you the most up-to-date value. In order to do so, it needs to apply all scheduled changes, flush the queue, bite the bullet and do the reflow.

For example, it's a bad idea to set and get styles in a quick succession (in a loop), like:

// no-no!
el.style.left = el.offsetLeft + 10 + "px";

Minimizing repaints and reflows

The strategy to reduce the negative effects of reflows/repaints on the user experience is to simply have fewer reflows and repaints and fewer requests for style information, so the browser can optimize reflows. How to go about that?

  • Don't change individual styles, one by one. Best for sanity and maintainability is to change the class names not the styles. But that assumes static styles. If the styles are dynamic, edit the cssText property as opposed to touching the element and its style property for every little change.
    // bad
    var left = 10,
        top = 10;
    el.style.left = left + "px";
    el.style.top  = top  + "px";
    
    // better 
    el.className += " theclassname";
    
    // or when top and left are calculated dynamically...
    
    // better
    el.style.cssText += "; left: " + left + "px; top: " + top + "px;";
  • Batch DOM changes and perform them "offline". Offline means not in the live DOM tree. You can:
    • use a documentFragment to hold temp changes,
    • clone the node you're about to update, work on the copy, then swap the original with the updated clone
    • hide the element with display: none (1 reflow, repaint), add 100 changes, restore the display (another reflow, repaint). This way you trade 2 reflows for potentially a hundred
  • Don't ask for computed styles excessively. If you need to work with a computed value, take it once, cache to a local var and work with the local copy. Revisiting the no-no example from above:
    // no-no!
    for(big; loop; here) {
        el.style.left = el.offsetLeft + 10 + "px";
        el.style.top  = el.offsetTop  + 10 + "px";
    }
    
    // better
    var left = el.offsetLeft,
        top  = el.offsetTop
        esty = el.style;
    for(big; loop; here) {
        left += 10;
        top  += 10;
        esty.left = left + "px";
        esty.top  = top  + "px";
    }
  • In general, think about the render tree and how much of it will need revalidation after your change. For example using absolute positioning makes that element a child of the body in the render tree, so it won't affect too many other nodes when you animate it for example. Some of the other nodes may be in the area that needs repainting when you place your element on top of them, but they will not require reflow.

Tools

Only about a year ago, there was nothing that can provide any visibility into what's going on in the browser in terms of painting and rendering (not that I am aware of, it's of course absolutely possible that MS had a wicked dev tool no one knew about, buried somewhere in MSDN :P ). Now things are different and this is very, very cool.

First, MozAfterPaint event landed in Firefox nightlies, so things like this extension by Kyle Scholz showed up. mozAfterPaint is cool, but only tells you about repaints.

DynaTrace Ajax and most recently Google's SpeedTracer (notice two "trace"s :) ) are just excellent tools for digging into reflows and repaints - the first is for IE, the second for WebKit.

Some time last year Douglas Crockford mentioned that we're probably doing some really stupid things in CSS we don't know about. And I can definitely relate to that. I was involved in a project for a bit where increasing the browser font size (in IE6) was causing the CPU go up to 100% and stay like this for 10-15 minutes before finally repainting the page.

Well, the tools are now here, we don't have excuses any more for doing silly things in CSS.

Except, maybe, speaking of tools..., wouldn't it be cool if the Firebug-like tools showed the render tree in addition to the DOM tree?

A final example

Let's just take a quick look at the tools and demonstrate the difference between restyle (render tree change that doesn't affect the geometry) and reflow (which affects the layout), together with a repaint.

Let's compare two ways of doing the same thing. First, we change some styles (not touching layout) and after every change, we check for a style property, totally unrelated to the one just changed.

bodystyle.color = 'red';
tmp = computed.backgroundColor;
bodystyle.color = 'white';
tmp = computed.backgroundImage;
bodystyle.color = 'green';
tmp = computed.backgroundAttachment;

Then the same thing, but we're touching style properties for information only after all the changes:

bodystyle.color = 'yellow';
bodystyle.color = 'pink';
bodystyle.color = 'blue';

tmp = computed.backgroundColor;
tmp = computed.backgroundImage;
tmp = computed.backgroundAttachment;

In both cases, these are the definitions of the variables used:

var bodystyle = document.body.style;
var computed;
if (document.body.currentStyle) {
  computed = document.body.currentStyle;
} else {
  computed = document.defaultView.getComputedStyle(document.body, '');
}

Now, the two example style changes will be executed on click on the document. The test page is actually here - restyle.html (click "dude"). Let's call this restyle test.

The second test is just like the first, but this time we'll also change layout information:

// touch styles every time
bodystyle.color = 'red';
bodystyle.padding = '1px';
tmp = computed.backgroundColor;
bodystyle.color = 'white';
bodystyle.padding = '2px';
tmp = computed.backgroundImage;
bodystyle.color = 'green';
bodystyle.padding = '3px';
tmp = computed.backgroundAttachment;

// touch at the end
bodystyle.color = 'yellow';
bodystyle.padding = '4px';
bodystyle.color = 'pink';
bodystyle.padding = '5px';
bodystyle.color = 'blue';
bodystyle.padding = '6px';
tmp = computed.backgroundColor;
tmp = computed.backgroundImage;
tmp = computed.backgroundAttachment;

This test changes the layout so let's called it "relayout test", the source is here.

Here's what type of visualization you get in DynaTrace for the restyle test.

DynaTrace

Basically the page loaded, then I clicked once to execute the first scenario (requests for style info every time, at about 2sec), then clicked again to execute the second scenario (requests for styles delayed till the end, at about 4sec)

The tool shows how the page loaded and the IE logo shows onload. Then the mouse cursor is over the rendering activity following the click. Zooming into the interesting area (how cool is that!) there's a more detailed view:

dynatrace

You can clearly see the blue bar of JavaScript activity and the following green bar of rendering activity. Now, this is a simple example, but still notice the length of the bars - how much more time is spent rendering than executing JavaScript. Often in Ajax/Rich apps, JavaScript is not the bottleneck, it's the DOM access and manipulation and the rendering part.

OK, now running the "relayout test", the one that changes the geometry of the body. This time check out this "PurePaths" view. It's a timeline plus more information about each item in the timeline. I've highlighted the first click, which is a JavaScript activity producing a scheduled layout task.

dynatrace

Again, zooming into the interesting part, you can see how now in addition to the "drawing" bar, there's a new one before that - the "calculating flow layout", because in this test we had a reflow in addition to the repaint.

dynatrace

Now let's test the same page in Chrome and look at the SpeedTracer results.

This is the first "restyle" test zoomed into the interesting part (heck, I think I can definitely get cused to all that zooming :) ) and this is an overview of what happened.

speedtracer

Overall there's a click and there's a paint. But in the first click, there's also 50% time spent recalculating styles. Why is that? Well, this is because we asked for style information with every change.

Expanding the events and showing hidden lines (the gray lines were hidden by Speedtracer because they are not slow) we can see exactly what happened - after the first click, styles were calculated three times. After the second - only once.

speedtracer

Now let's run the "relayout test". The overall list of events looks the same:

speedtracer

But the detailed view shows how the first click caused three reflows (because it asked for computed style info) and the second click caused only one reflow. This is just excellent visibility into what's going on.

speedtracer

A few minor differences in the tools - SpeedTracer didn't show when the layout task was scheduled and added to the queue, DynaTrace did. And then DynaTrace didn't show the details of the difference between "restyle" and "reflow/layout", like SpeedTracer did. Maybe simply IE doesn't make a difference between the two? DynaTrace also didn't show three reflows instead of one in the different change-end-touch vs. change-then-touch tests, maybe that's how IE works?

Running these simple examples hundreds of times also confirms that for IE it doesn't matter if you request style information as you change it.

Here's some more data points after running the tests with enough repetitions:

  • In Chrome not touching computed styles while modifying styles is 2.5 times faster when you change styles (restyle test) and 4.42 times faster when you change styles and layout (relayout test)
  • In Firefox - 1.87 times faster to refrain from asking computed styles in restyle test and 1.64 times faster in the relayout test
  • In IE6 and IE8, it doesn't matter

Across all browsers though changing styles only takes half the time it takes to change styles and layout. (Now that I wrote it, I should've compared changing styles only vs. changing layout only). Except in IE6 where changing layout is 4 times more expensive then changing only styles.

Parting words

Thanks very much for working through this long post. Have fun with the tracers and watch out for those reflows! In summary, let me go over the different terminology once again.

  • render tree - the visual part of the DOM tree
  • nodes in the render tree are called frames or boxes
  • recalculating parts of the render tree is called reflow (in Mozilla), and called layout in every other browser, it seems
  • Updating the screen with the results of the recalculated render tree is called repaint, or redraw (in IE/DynaTrace)
  • SpeedTracer introduces the notion of "style recalculation" (styles without geometry changes) vs. "layout"

And some more reading if you find this topic fascinating. Note that these reads, especially the first three, are more in depth, closer to the browser, as opposed to closer to the developer which I tried to do here.

 

JavaScript loading strategies

Tuesday, December 15th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 15 This article is part of the 2009 performance advent calendar experiment. Today's article is a contribution from Ara Pehlivanian, author of two JavaScript books. Please welcome Ara and stay tuned for the articles to come.

Ara PehlivanianAra Pehlivanian has been working on the Web since 1997. He's been a freelancer, a webmaster, and most recently, a Front End Engineer at Yahoo! Ara's experience comes from having worked on every aspect of web development throughout his career, but he's now following his passion for web standards-based front-end development. When he isn't speaking and writing about best practices or coding professionally, he's either tweeting as @ara_p or maintaining his personal site at http://arapehlivanian.com/.

JavaScript has a dark side to it that not many people are aware of. It causes the browser to stop everything that it's doing until the script has been downloaded, parsed and executed. This is in sharp contrast to the other dependencies which get loaded in parallel--limited only by the number of connections the browser and server are able to create. So why is this a problem?

Good question! Before I can answer that, I need to explain how the browser goes about building a page. The first thing a it's does once it receives an HTML document from the server is to build the DOM--an object representation of the document in memory. As the browser goes about converting HTML into the DOM, it invariably encounters references to external dependencies such as CSS documents and images. Every time it does so, it fires off a request to the server for that dependency. It doesn't need to wait for one to be loaded before requesting another, it makes as many requests as it's capable of. This way, the page gets built one node at a time and as the dependencies come in, they're put in their correct placeholders. What gums up the works though, is when a JavaScript dependency is encountered. When this happens, the browser stops building the DOM and waits for that file to arrive. Once it receives the file, it parses and executes it. Only once all of that's done does the browser continue building the DOM. I suspect this has to do with wanting to provide as stable a DOM to the script as possible. If things were in flux while the script attempted to access or even modify a DOM node, things could get dicey. Either way, the time it takes before the browser can continue depends entirely on the size and complexity of the script file that's being loaded.

Now imagine loading a 200k JavaScript file right in the <head> of a document. Say it's a JavaScript file that's not only heavy but also does some fairly complex computing that takes half a second to complete. Imagine now what would happen if that file took a second to transfer. Did you guess? Yup, the page would be blank until that transfer and the computation were complete. A second and a half of a blank page that the visitor has to endure. Given that most people don't spend more than a few seconds on the average web page, that's an eternity of staring at a blank page.

Reduce

So how can this problem be overcome? Well, the first thing that should be done, is to reduce as much as possible, the amount of data that's being sent over the pipe. The smaller the JavaScript file, the less waiting the visitor has to do. So what can be done to reduce file size? JavaScript files can be run through a minifier such as YUI Compressor (which removes unnecessary white space and formatting, as well as comments, and is proven to reduce file size by 40-60%). Also, if at all possible, servers should be set up to gzip files before they're sent. This can drastically reduce the number of bytes that get transferred since JavaScript is plain text, and plain text compresses really well.

Defer

So, once you've made sure your file is as small as possible, what next? Well, the first thing is to make sure the visitor has something to look at while the script is loading. Instead of loading JavaScript files in the document's <head>, put your <script> tags immediately before your page's closing </body> tag. That way, the browser will have built the DOM and begun inserting images and applying CSS long before it encounters your script tags. This also means that your code will execute faster because it won't need to wait for the page's onload event--which only fires once all the page's dependencies are done loading.

So with the script tags placed at the end of the document, when the browser does encounter them, will still halt operations for however long it needs to, but at this point the visitor is reading your page and unaware of what's going on behind the scenes. You've just bought yourself the time to surreptitiously load your script files.

Go Async

There is another way to load JavaScript files which won't block your browser, and that's to insert the script tags into your page using JavaScript. Dynamically including a script tag into the DOM causes it to be loaded asynchronously. The only trouble with that is that you can't rely on the code within the script file to be available immediately after you've included it. What you'll need is a callback function that is executed once your script is done loading. There are several ways of doing this. A lot of libraries have built in async script loading functionality, so you're likely better off using that. But if you want to do it yourself, be ready to deal with the idiosyncrasies of different browsers. For example, where one browser will fire off an onload event for the script, another will not.

Be Lazy

So now that we know how to load scripts behind the scenes, is there anything more we can do to improve performance? Of course.

Say for example your page loads up a large script that gives your site a fancy navigation menu. What if the user never uses the navigation menu? What if they only navigate your site through links in your content? Did you really need to load that script in the first place? What if you could load the necessary code only when it was needed? You can. It's a technique called lazy loading. The principle is simple, instead of binding your fancy navigation script to the menu in your page, you'd bind a simple loader script instead. It would detect an onmouseover event for example, and then insert a script tag with the fancy nav code into the page. Once the tag is done loading, a callback function wires up all the necessary events and presto bingo, your nav menu starts working. This way, your site doesn't have to needlessly bog visitors down with code they'll never use.

Bite Size

In keeping with lazy loading, try to also load only the core components that are needed to make your page work. This is especially the case when it comes to libraries. A lot of the time a library will force you to load up a huge amount of code when all you want to do is add an event handler, or modify class names. If the library doesn't let you pull down only what you need, try ripping out what you want and only load that instead. There's no point in forcing visitors to download 60k of code when all you need is 4k of it.

Do You Need It?

Finally, the best way to speed up JavaScript load times is to not include any JavaScript at all. A lot of times people go nuts for the latest fad and include it in their site without even asking themselves if they really need it. Does this fancy accordion thing actually help my visitors get to my content easier? Does fading everything in and out and bouncing things all over the place actually improve my site's usability? So the next time you feel like adding a three dimensional spinning rainbow tag cloud to your site, ask yourself, "do I really need this?"

Note from Stoyan:

I'd like to thank Ara for the great article, it's pleasure for me to be the blog host!

Also wanted to offer some additional links for your reading pleasure:

Please comment if you can think of more good resources on the topic.

 

Reducing the payload: compression, minification, 204s

Friday, December 11th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 11 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the next articles.

After removing all the extra HTTP requests you possibly can from your waterfall, it's time to make sure that those that are left are as small as they can be. Not only this makes your pages load faster, but it also helps you save on the bandwidth bill. Your weapons for fighting overweight component include: compression and minification of text-based files such as scripts and styles, recompression of some downloadable files, and zero-body components. (A follow-up post will talk about optimizing images.)

Gzipping plain text components

Hands down the easiest and at the same time quite effective optimization - turning on gzipping for all plain text components. It's almost a crime if you don't do it. Doesn't "cost" any development time, just a simple flip of a switch in Apache configuration. And the results could be surprisingly pleasant.

When Bill Scott joined Netflix, he noticed that gzip is not on. So they turned it on. And here's the result - the day they enabled it, the outbound traffic pretty much dropped in half (slides)

Netflix traffic after turning on gzipping

Gzip FAQ

How much improvement can you expect from gzip?
On average - 70% reduction of the file size!
Any drawbacks?
Well, there's a certain cost associated with the server compressing the response and the browser uncompressing it, but it's negligible compared to the benefits you get
Any browser quirks?
Sure, IE6, of course. But only in IE6 service pack 1 and fixed for after that. You can boldly ignore this edge case, but if you're extra paranoid you can disable gzip for this user agent
How to tell if it's on?
Run YSlow/PageSpeed and they'll will warn you if it's not on. If you don't have any of those tools just look at the HTTP headers with any other tool, e.g. Firebug, webpagetest.org. You should see the header:

Content-Encoding: gzip

provided, of course, that your browser claimed it supports compression by sending the header:

Accept-Encoding: gzip, deflate
What types of components should you gzip?
All text components:

  • javascripts
  • css
  • plain text
  • html, xml, including any other XML-based format such as SVG, also IE's .htc
  • JSON responses from web service calls
  • anything that's not a binary file...

You should also gzip @font-files like EOT, TTF, OTF, with the exception of WOFF. Average about 40% to be won there with font files.

How-to turn on gzipping

Ideally you need control over the Apache configuration. If not full control, at least most hosting providers will offer you ability to tweak configuration via .htaccess. If your host doesn't, well, change the host.

So just add this to .htaccess:

AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/javascript application/json

If you're on Apache before version 2 or your unfriendly host don't allow any access to configuration, not all is lost. You can make PHP do the gzipping for you. It's not ideal but the gzip benefits are so pronounced that it's worth the try. This article describes a number of different options for gzipping when dealing with uncooperative hosts.

Rezipping

As Billy Hoffman discovered, there's potential for file size reduction with common downloadable files, which are actually zip files in disguise. Such files include:

  • Newer MS Office documents - DOCX, XLSX, PPTX
  • Open Office documents - ODT, ODP, ODS
  • JARs (Java Applets, anyone?)
  • XPI Firefox extensions
  • XAP - Silverlight applications

These ZIP files in disguise are usually not compressed with the maximum compression. If you allow such downloads from your website, consider recompressing them beforehand with maximum compression.

There could be anywhere from 1 to 30% size reduction to be won, definitely worth the try, especially since you can do it all on the command line, as part of the build process, etc. (re)Compress once, save bandwidth and offer faster downloads every time ;)

15% uncompressed traffic

Tony Gentilcore of Google reported his findings that a significant chunk of their traffic is still sent uncompressed. Digging into it he realized there's a number of anti-virus software and firewalls that will mingle with the browser's Accept-Encoding header changing into the likes of:

Accept-Encoding: xxxx, deflxxx
Accept-Enxoding: gzip, deflate

Since this is an invalid header, the server will decide that the browser doesn't support gzip and send uncompressed response. And why would the retarded anti-virus program do it? Because it doesn't want to deal with decompression in order to examine the content. Probably not to slow down the experience? In doing so it actually hurts the user to a greater extend.

So compression is important, but unfortunately it's not always present. That's why minification helps - not only because compressing minified responses is even smaller, but because sometimes there is no compression despite your best efforts.

Minification

Minification means striping extra code from your programs that is not essential for execution. The code in question is comments, whitespace, etc from styles and scripts, but also renaming variables with shorter names, and various other optimizations.

This is best done with a tool, of course, and luckily there a number of tools to help.

Minifying JavaScript

Some of the tools to minify JavaScript include:

How much size reduction can you expect from minification? To answer that I ran jQuery 1.3.2. through all the tools mentioned above (using hosted versions) and compared the sizes before/after and with/without gzipping the result of minification.

The table below lists the results. All the % figures are % of the original, so smaller is better. 29% means the file was reduced to 29% of its original version, or a saving of 71%

File original size size, gzipped % of original gzip, % of original
original 120619 35088 100.00% 29.09%
closure-advanced 49638 17583 41.15% 14.58%
closure 55320 18657 45.86% 15.47%
jsmin 73690 21198 61.09% 17.57%
packer 39246 18659 32.54% 15.47%
shrinksafe 69516 22105 57.63% 18.33%
yui 57256 19677 47.47% 16.31%

As you can see gzipping alone gives you about 70% savings, minification alone cuts script sizes with more than half and both combined (minifying then gzipping) can make your scripts 85% leaner. Verdict: do it. The concrete tool you use probably doesn't really matter all that much, pick anything you're comfortable with to run before deployment (or best, automatically during a build process)

Minifying CSS

In addition to the usual stripping of comments and whitespaces, more advanced CSS minification could include for example:

// before
#mhm {padding: 0px 0px 0px 0px;}
// after
#mhm{padding:0}

// before
#ha{background: #ff00ff;}
// after
#ha{background:#f0f}
//...

A CSS minifier is much less powerful than a JS minifier, it cannot rename properties or reorganize them, because the order matters and for example text-decoration:underline cannot get any shorter than that.

There's not a lot of CSS minifiers, but here's a few I tested:

  • YUI compressor - yes, the same YUI compressor that does JavaScript minification. I've actually ported the CSS minification part of it to JavaScript (it's in Java otherwise) some time ago. There's even an online form you can paste into to test. The CSS minifier is regular expression based
  • Minify is a PHP based JS/CSS minification utility started by Ryan Grove. The CSS minifier part is also with regular expressions, I have the feeling it's also based on YUICompressor, at least initially
  • CSSTidy - a parser and an optimizer written in PHP, but also with C version for desktop executable. There's also a hosted version. It's probably the most advanced optimizer in the list, being a parser it has a deeper understanding of the structure of the styleshets
  • HTML_CSS from PEAR - not exactly an optimizer but more of a general purpose library for creating and updating stylesheets server-side in PHP. It can be used as a minifier, by simply reading, then printing the parsed structure, which strips spaces and comments as a side effect.

Trying to get an average figure of the potential benefits, I ran these tools on all stylesheets from csszengarden.com, collected simply like:

<?php
$urlt = "http://csszengarden.com/%s/%s.css";
for ($i = 1; $i < 214; $i++) {
  $id = str_pad($i, 3, "0", STR_PAD_LEFT);
  $url = sprintf($urlt, $id, $id);
  file_put_contents("$id.css", file_get_contents($url));
}
?>

3 files gave a 404, so I ran the tools above on the rest 210 files. CSSTidy ran twice - once with its safest settings (which even keep comments in) and then with the most aggressive. The "safe" way to use CSSTidy is like so:

<?php
// dependencies, instance
include 'class.csstidy.php';
$css = new csstidy();

// options
$css->set_cfg('preserve_css',true);
$css->load_template('high_compression');

// parse
$css->parse($source_css_code);

// result
$min = $css->print->plain();
?>

The aggressive minification is the same only without setting the preserve_css option.

Running Minify is simple:

<?php
// dependencies, instance
require 'CSS.php';
$minifier = new Minify_CSS();

// minify in one shot
$min = $minifier->minify($source_css_string_or_url);

As for PEAR::HTML_CSS, since it's not a minifier, you only need to parse the input and print the output.

<?php
require 'HTML/CSS.php';

$options = array(
    'xhtml' => false,
    'tab' => 0,
    'oneline' => true,
    'groupsfirst' => false,
    'allowduplicates' => true,
);

$css = new HTML_CSS($options);
$css->parseFile($input_filename);
$css->toFile($output_filename);
// ... or alternatively if you want the result as a string
// $minified = $css->toString();

So I ran those tools on the CSSZenGarden 200+ files and the full table of results is here, below are just the averages:

  Original YUI Minify CSSTidy-safe CSSTidy-small PEAR
raw 100% 68.18% 68.66% 84.44% 63.29% 74.60%
gzipped 30.36% 19.89% 20.74% 28.36% 19.44% 20.20%

Again, the numbers are percentage of the original, so smaller is better. As you can see, on average gzip alone gives you 70% size reduction. The minification is not so successful as with JavaScript. Here even the best tool cannot reach 40% reduction (for JS it was usually over 50%). But nevertheless, gzip+minification on average gives you a reduction of 80% or more. Verdict: do it!

An important note here is that in CSS we deal with a lot of hacks. Since the browsers have parsing issues (which is what hacks often exploit), what about a poor minifier? How safe are the minifiers? Well, that's a subject for a separate study, but I know I can at least trust the YUICompressor, after all it's used by hundreds of Yahoo! developers daily and probably thousands non-Yahoos around the world. PEAR's HTML_CSS library also looks pretty safe because it has a simple parser that seems to tolerate all kinds of hacks. CSSTidy also claims to tolerate a lot of hacks, but given that the last version is two years old (maybe new hacks have surfaced meanwhile) and the fact that it's the most intelligent optimizer (knows about values, colors and so on) it should be approached with care.

204

Let's wrap up this lengthy posting with an honorable mention of the 204 No Content response (blogged before). It's the world's smallest componet, the one that has no body and a Content-Length of 0.

Often people use 1x1 GIFs for logging and tracking purposes and other types of requests that don't need a response. If you do this, you can return a 204 status code and no response body, only headers. Look no further that Google search results with your HTTP sniffer ON to see examples of 204 responses.

The way to send a 204 response from PHP is simply:

header("HTTP/1.0 204 No Content");

A 204 response saves just a little bit but, hey, every little bit helps.

And remember the mantra: every extra bit is a disservice to the user :)

Thank you for reading!

Stay tuned for the next article continuing the topic of reducing the component sizes as much as possible.

 

Duplicates and near-duplicates

Wednesday, December 9th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 9 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the next articles.

One of Yahoo!'s first batch of performance best practices has always been "Avoid duplicate scripts" (check Steve Souders' post). Later we added "... and styles". This is a pretty obvious, kind of a "Duh!" type of recommendation, it's like saying "Avoid sleep() in your server-side scripts". But it didn't come up out of thin air, duplicates were noticed on some quite high-profile sites.

Duplicates are easy to spot (and YSlow will warn you), but let's talk a bit about another concept - let's call it near-duplicates - when two components are similar, almost the same, but not quite.

Duplicate scripts and styles

As a refresher and a quick illustration of the effects of duplicate scripts and styles, fire off your HTTP sniffer and hit this test page.

(btw, this is a simple page I put up to test different YSlow scenarios, you can actually use it as a web service of sorts to create any type of components with different options)

Firefox 2 downloading both duplicate styles and scripts:

FF2 duplicate scripts and styles

IE6 and duplicate scripts:

IE duplicate scripts

Exact details of when/which browsers chose to download duplicates are not that interesting, it's obviously bad to waste time downloading the same resource. Even if no repeating download happens, the browser still has to parse through and execute the script/style for a second time.

Even if you have iframes you don't need to repeat the same JS/CSS in each frame, you can "borrow" them from the parent page, here's an example.

Near-duplicates

Near duplicates can be:

  • components with the exact same response bodies but different URLs causing the browser to do double work
  • components (images) that are too close to each other - in terms of looks or purpose. Only one component should be selected in this case.

Same component, different URLs

This could happen especially when you have user-generated content such as image uploads for profile photos and avatars in social sites, forums, images people put in comments on MySpace and so on.

Also images of stuff for sale (Craigslist, eBay). Often different sellers offering the same item would take the same photo from the manufacturer's site and upload it over and over again.

Luckily, PageSpeed warns about components with identical content, so those can be identified:

In the screenshot above, you see one image (2.3K) repeated 3 times, another (the iPhone, 1.7K) is repeated 4 times, and yet another one (2.8K) repeated 2 times.

It's not exactly trivial to avoid this type of duplications with user-generated content (for example, the first poster may delete the photo, in which case the second poster's photo will need to "shine through"). But it's not impossible, using for example a hash of the component's content as an identifier.

Loading...

Ajax loading indicators are a great idea to give feedback to the user that something is happening. They come in all shapes and sizes... sometimes on the same page, unfortunately. And again, sometimes it's the same stock image but used at different stages of gradual "ajaxification" of the page and with different URLs.

As we're moving more and more towards modular pages and client-side logic, often different modules on the same page are coded by different teams at different times, independently, without being aware of each other's assets. This way of building pages has it's challenges and one is that common components, such as Ajax loading indicators, should be shared.

Too similar modules

Along the same lines - different modules are sometimes created by different designers at different times. The result - one rounded corner box with 1px shadow and one with 2px shadow, both on the same page. Or two different shades of the same gray color, which no one can tell apart. That's just a waste. (See Nicole Sullivan's presentation for illustration, e.g. slides 44, 45)

Below is an example, can you tell that these 5 rounded corner boxes are all different - slightly different shadow, color or radius? How many different boxes does this page need?

Different sizes of the same image

It's highly recommended to not scale images in HTML (or CSS). If you need a 100x100 image you don't use a 400x400 one with <img width="100" height="100" ... />. That's a good rule of thumb... to break sometimes ;)

In cases where the same image is used with different sizes and likely even on the same page, it may be beneficial to reuse the same bigger image and scale it down, because this could be saving extra HTTP requests of downloading the same (but slightly smaller) image.

Facebook is an example, the same hairy guy on the screenshot has two images with different sizes. It's actually the same image but resized in CSS.

The relevant CSS which shows the profile image in LARGE and SMALL (and looks like there's even a TINY view, although I couldn't find an example on this page)

.UIProfileImage_LARGE{width:50px;height:50px}
.UIProfileImage_SMALL{width:32px;height:32px}
.UIProfileImage_TINY{width:25px;height:25px}

Thank you!

Thanks for reading! Reducing HTTP requests is critical for page performance. You've merged your scripts and styles as much as reasonable, you've crafted CSS sprites and inlined images with data URIs. Now it's time to look at what's left - are there components that are way too similar, are there any near-duplicates or even exact duplicates? Same image on different backgrounds? Ever-so-subtle gradients and shadows? Time to pick up the old axe and cut.

 

Reducing the number of page components

Saturday, December 5th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 5 This is the fifth in the series of performance articles as part of my 2009 performance advent calendar experiment. Stay tuned for the next articles.

Let's talk a but about waterfall optimization - the first thing that happens in Mr.Page's life. The best way to optimize and speed up the waterfall is to have less stuff in it. The fewer page components, the faster the page - simple as that.

Fewer components vs. component weight

The size of the page components, meaning their size in kB, is important. It makes sense - smaller pages will load faster, 100K JavaScript will load faster than 150K. It's important to keep sizes low, but it should be clear that the number of components is even more important than their filesize.

Why? Because every HTTP request has overhead.

OK, but how bad can it be, someone might ask. If you look at an HTTP request - it has a header and a body. A 100K body will greatly outweigh the size of the headers, no matter how bloated they are.

Here's the headers for a request to Yahoo! Search:

Host: search.yahoo.com
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5;) Firefox/3.5.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

To that request the server responds with the body (content) of the response prepended with some headers like:

HTTP/1.1 200 OK
Date: Sat, 05 Dec 2009 07:36:25 GMT
P3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR... blah, blah"
Set-Cookie: sSN=nTMt3Lo2...crazy stuff...nLvwVxUU; path=/;domain=.search.yahoo.com
Cache-Control: private
Connection: close
Content-Type: text/html; charset=ISO-8859-1

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">

<html lang="en"><head><meta...

This is 352 bytes request header and 495 bytes response header. Not so bad, eh? No matter how hard you try to make your cookies monster-size, the response body (9k gzipped in this case) will always be significantly bigger. So what's the problem with the overhead of the HTTP requests then?

The size of the headers is a problem when you make requests for small components - say requests for small icons - for example 1K or under. In this case you exchange 1K headers to get 1K of useful data to present to the user. Clearly a waste. Additionally this 1K of headers can grow once you start writing more cookies. It may very well happen that the HTTP headers size is bigger than the actual icon you need. And even if the headers are not bigger that the component, they are still big when you think percent-wise. 1K of 10K is 10%.

But the HTTP headers size is only one (and the smaller) of the problems.

The bigger problem is the HTTP connection overhead.

HTTP connection overhead

What happens (at a high level) when you type a URL and hit Enter? The browser sends a request to the server. Which server? The browser needs to know the IP address of the server, so if it doesn't have it in the cache, it makes a DNS lookup. Than the browser establishes a connection to the server. Then it waits for the first byte of the response from the server. Then it receives the full response (payload).

Here's how this looks like graphically represented by webpagetest.org

And the color legend:

  1. DNS lookup
  2. Initial connection
  3. TTFB (Time to first byte)
  4. Payload

So what do we have here - looks like in this particular case the browser is downloading content about 40% of the time. The rest of the time it's... well, not downloading content. How's that for an overhead. And the part of not downloading can be even greater, this above was just one example.

Now how about this - a bird-eye view of the 4 last pages in webpagetest.org's test history - just some random pages people have tested.

Do you see a lot of blue (time spent downloading content). Not as much as you would've hoped. There's some DNS lookups, some orange... and OMG, talk about going green! :)

In fact you may notice that the smaller the component, the smaller is the blue part.

What does all this tell us?

  1. A significant chunk of time is spent in activities other than downloading.
  2. Smaller components still incur HTTP overhead and for them the relative penalty (relative to their size) is atrocious.

So what's a performance optimizer to do? Reduce the number of components and so pay fewer penalties.

Just remove stuff

Truth is - a lot of stuff on the pages today is not needed. Features no one likes or uses clutter the page and make it heavier. Well, what can you do, the boss/client/marketing guy wants that feature there. What you can do is at least try. You can introduce some science into the marketing activities - measure how much a specific feature is used. Or if you already have the data - look at it. Decide what can a page go without.

It's going to be tough convincing people to remove stuff. After all you spend time developing it. Someone dreamt up that feature initially. Someone (not the users) loves it. People hate to let go. But still, it's worth to try.

Combine components

Now that the phase of convincing people to remove stuff is done, what's left needs to be combined. How do you combine components? Simple - all JavaScripts go into a single file, all CSS into a single file. All decoration images go into a sprite.

JavaScript example (from a page to remain anonymous)

Before:

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js"></script>
<script src="/javascripts/application.js?1258423604"></script>
<script src="/javascripts/ui/minified/jquery.ui.all.min.js?1258423604"></script>
<script src="/javascripts/ui-ext/ui.bgiframe.min.js?1258423604"></script>
<script src="/javascripts/ui-ext/ui.stars.pack.js?1258423604"></script>
<script src="/javascripts/ui-ext/ui.dimensions.js?1258423604"></script>
<script src="/javascripts/ext/jquery.form.min.js?1258423604"></script>

After:

<script src="/javascripts/all.js"></script>

Sizes, gzipped: 70029 bytes before, 65194 bytes after. Simply merging files and there's even 6.9% saving!
And the more important saving: 6 less HTTP requests

Repeat for CSS. Before:

/stylesheets/general.css?1258423604
/stylesheets/global.css
/stylesheets/ui.stars.css
/stylesheets/themes/enation/enation.all.css
/public/template/css/132/1245869225

After:

<link type="text/css" rel="stylesheet" href="/stylesheets/all.css" />

Sizes, gzipped: before 14781 bytes, after 13352 bytes, saving 9.6%.
But the bigger saving: 4 less HTTP requests.

If you wonder how come the sizes before and after are different since we merely concatenate the contents of the files, well, the savings come from gzip compression. When you have more characters in the file, there's more chance that some will repeat which means they will compress better. That's one. And then, the compression itself has an overhead which you incur once for the whole bundle of files, as opposed to for every file.

Now - let's make decoration images into sprites. Before:

... 15 image requests, 6.8K

After: (1 sprited image)

Result size: 1.4K, 7 times smaller!

Here the savings are so dramatic partially because the source files are GIFs and the result is a PNG8 but that's a whole other post.

So in conclusion: file concatenation is just awesome. You save both: bytes to download and, much more importantly, HTTP requests. Less of the green stuff in the waterfall!

x-type component concatenation

So far we combined .js with .js, css with css and images with images. How about a cross-component type concatenation?

You can inline images inside HTML and CSS (and why not JS if you want) using data URIs (another post coming).

And you can inline CSS and JS inside HTML too.

This means you can have your whole application inside of just one HTML file if you wish. Inside of the HTML you have inline styles, scripts and images.

Combining CSS with JS

Now how about mixing CSS and JS into one component. You can do that, and it's especially suitable for lazy-loaded, widget-type functionality.

Say you've loaded the page, then the user clicks some rarely used button. You haven't downloaded that content that is supposed to wow the user upon click of the button. So you send a request to grab it. The new content may come in the form of a JSON string. And what if the new content requires some stylesheet that was not part of the base page? You'll have to make another request to download that stylesheet too.

Or, you can download both content and styles in the same JSON response. You simply inline the style information as a string in the JSON. So:

1. uses clicks, you request feature.js which goes like:

{"content":"<p class=\"wow\">I'm a feature</p>", "style": "wow{font-size: 60px}"}

2. You process the JSON and shove the content into the page

var o = JSON.parse(xhr.responseText);
$('result').innerHTML = o.content;

3. You add the styles to the head:

var wow = document.createElement('style');
wow.type = "text/css";
if (wow.textContent) { // FF, Safari
    wow.textContent = o.style;
} else {
    wow.styleSheet.cssText = o.style; // FF, IE
}
document.documentElement.firstChild.appendChild(wow);

Nice and simple. Makes the features (that progressively enhance the page) atomic and self-contained.

More to reducing components?

For more and creative ways to reduce HTTP components you can take a look at MXHR and Comet

Another thing to check is the Keep-Alive setting on your server. Remember how there were 4 steps in the component download. When you request a second component you can have the connection open so that you don't need to re-establish it (skipping step 2). And since the DNS lookup was already made you get rid of step 1. Skipping 2 out of 4 is not bad at all.

Summary

Reducing the number of page of components is the top priority of any web performance optimization effort. HTTP requests are costly. In part because of the headers size overhead, but mostly because of the connection overhead. The browser spends a disturbing amount of time not downloading stuff, and we can't allow this!

 

Statsy – more data points for markup quality

Monday, November 30th, 2009

In the spirit of the content-to-markup ratio bookmarklet, here's another one that gives you some more data points to help you judge the quality of a page's markup and help answer the old question - where does all this page weight go.

Install the statsy bookmarklet

Drag this link to your bookmarks:

statsy

the results

Once you run the bookmarklet it alerts these stats points:

  • JS attributes (e.g. onclick) - this is the sum of all onclick, onmouseover and so on including the attribute names. So for example <a onclick="#"> is 11 characters (bytes) of JavaScript attributes code
  • CSS style attributes - the sum of all style="..."
  • Inline JS - the sum of all the contents of all script tags (excluding the tag itself)
  • Inline CSS - sum of all <style> tag contents
  • All innerHTML - this is document.documentElement.innerHTML.length, it should be close to the ungzipped size of a page, provided the page is not doing a lot of DOM manipulation
  • # DOM elements - the total number of elements on the page is counted simply using document.getElementsByTagName('*').length

Here's example output:
statsy bookmarklet output

The code

The code is here for your tweaking pleasure

Thanks!

Hope you'll find this bookmarklet useful when looking at a page as a companion to YSlow/PageSpeed.

What else should I add to this bookmarklet? # of font tags, # of table tags...?

 

Browser’s implied globals

Tuesday, October 20th, 2009

Like it's not bad enough that JavaScript has implied globals (forget var and you create a global), but the browsers have decided it's a good idea to add more pollution to the global namespace.

This has been a source of frustration before with IE, it's really hard to understand the logic behind it, but it's also happening in other browsers.

Consider this:

<meta name="description" content="test me" />

A normal META tag, right? But in IE this will create in a global variable called "description" pointing to that DOM node. Yep.

alert(description.content); // "test me"

That's pretty annoying. Even more annoying is that getElementById('description') will also point to the DOM node, although it doesn't even have an ID.

A test is born

Anyway, I wanted to test the effect of other name and id attributes in different tags and different browsers. With the exception of Firefox which doesn't create any globals, all other did to some degree. Rather disappointing. I tested IE6, 8 (plus compat view), FF 3.5, Safari 4 and Opera 10.

Here's the test page

And below are the results. The yellow x means that testing for the presence of this global returned "undefined", the white o means that the global variable points to an object. So for example continuing with the meta example above, typeof window.description will return undefined in FF (yellow x) and object in IE (white o).

global description IE FF Saf O
description <meta name="description"... o x x o
robots <meta name="robots"... o x x o
paragraph-id <p id="paragraph-id"... o x o o
paragraph-name <p name="paragraph-name"... x x x o
form-name <form name="form-name"... o x o o
form-id <form id="form-id"... o x o o
input-name <input name="input-name"... x x x x
input-id <input id="input-id"... x x o x
link-name <a name="link-name"... o x x o
link-id <a id="link-id"... o x o o
div-name <div name="div-name"... x x x o
div-id <div id="div-id"... o x o o

So...?

So this is a useless feature if you ask me. Not reliable, not cross-browser, maybe considered convenient back when rollover buttons and animated gifs were all the rage (and animated window.status, remember?), but today can only cause troubles where you least expect it. Should be removed in future browser versions.

For the time being we just have to remember to always declare and initialize our local variables because it looks like someone else might also decide to do so for us. Which can lead to errors if we assume too much.

 

cssmin.js

Wednesday, September 23rd, 2009

cssmin.js is a JavaScript port of YUICompressor's CSS minifier.

The motivation

Minifying CSS helps reduce file sizes and makes your pages faster and your users happier. YUICompressor is cool but is written in Java, which can be a blocker for some folks - you know JVM, command line, classpaths... No more excuses, now you have a simple light JavaScript version. And as you know, JavaScript is everywhere, so you can run it however you want, integrate with your editor and so on.

The links

The integration

If you want to integrate the library into your environment, it's really easy. It's just one file with one function in it. So, just a simple function call:

var result = YAHOO.compressor.cssmin(input_css_code);

The credits

Julien Lecomte - creator of YUICompressor
Isaac Schlueter - he maintains the YUICompressor and is the author of the original cssmin utility which was ported to Java by Julien.

Ha, what about a quiz? Guess the language of Isaac's original cssmin and I'll send you a free copy of Even Faster WebSites and I'll sign my chapter. Seriously.

UPDATE: For Ruby folks, there's a Ruby port from the Ryan Grove.

 

JavaScript shell scripting

Thursday, September 3rd, 2009

As you probably know, JavaScript is not limited to the browser. There's server-side JavaScript, JS for various extensions, you can script Photoshop operations with JavaScript if you feel like it. Or compile Windows executables. You see where I'm going with this. JavaScript is everywhere :)

And yes, you can do shell scripting in JavaScript. On any platform you can use Rhino to run your scripts. On Windows there's this WSH, Windows Scripting Host you can benefit from, built right into the OS so you don't have to install anything. You can run your JavaScript shell scripts with the cscript utility like:

C:\> cscript jslint.js

And on the Mac there's JavaScriptCore by WebKit. WebKit is not limited to Safari, it's used all over the place on the Mac. So there's a utility called jsc which can run your scripts.

JSC test run

JSC (JavaScriptCore) is well hidden in

/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc

Check it out, it should be there. And if it is, why not use it via a "shortcut"? So step 1:

$ sudo ln /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc /bin/jsc

Step 2... no, there's no step 2, just give it a shot:

$ jsc -h
Usage: jsc [options] [files] [-- arguments]
  -d         Dumps bytecode (debug builds only)
  -e         Evaluate argument as script code
  -f         Specifies a source file (deprecated)
  -h|--help  Prints this help message
  -i         Enables interactive mode (default if no files are specified)
  -s         Installs signal handlers that exit on a crash (Unix platforms only)

So you can just run any JavaScript code on the command line. You can also use jsc as a JavaScript console to type away stuff. Probably not that useful, because you have Firebug console to type into. But still, an option.

$ jsc
> var a = 1;
undefined
> a++
1
> a
2
>

Shell version of an online tool

Here's an example. Last night I worked on a quick hack to experiment with minifying CSS. There's an online tool over here as a result. Can I run this tool on the command line? Sure.

So imagine you have a JavaScript that parses CSS that looks like this. It defines an object called cssparse. You can use it in a browser-based tool, but also in the command-line version, without any changes. All you need is to create a new file that will be the shell version of the tool, say csspsh.js. In it put the something like:

if (!arguments[0]) {
    print('usage:\n $ jsc csspsh.js "`cat parseme.css`"');
    quit();
}

load('cssp.js');

print(cssparse.parse(arguments[0]));

You can probably guess but:

  • arguments[] array-like object contains command-line arguments
  • print() prints to the console
  • quit() exits JSC
  • load() loads and executes an external file

How do you pass arguments to your shell script? After a -- delimiter, like so:

$ jsc csspsh.js -- one two three

And since this particular script works with contents of files, I can use cat to read the file and pass it to the script.

$ jsc csspsh.js -- "`cat my.css`"

Shell-script away!

So there you have it. Shell scripting with JavaScript is at your fingertips, either Mac or Windows, or anywhere with Rhino. If you have some cool scripts that you want to run the on the command line, like cron jobs or some automated process, there has never been a better time :)

 

The art and craft of postload preloads

Thursday, August 27th, 2009

What can your tired old page, once loaded and used and read, can do for your user? It can preload components needed by the next page, so when the users visit the next page, they have the new scripts, styles and images already in the cache. Next page loads faster and the user's happy. On its death bed you tired old page has left a good inheritance for future generations. Good old page.

How can you go about preloading for the next page? By waiting for onload of the current page and requesting the new components. Here are 4 ways to do so, all using a timeout of 1 second after page load so that prefetching doesn't interfere with the user experience on the page.

One way... (DOM)

Using DOM you can create a new LINK element and a new SCRIPT element and append them to the HEAD. For images - it's a one-liner new Image.src="..."

The drawback of this method is that your CSS is executed against the current page and might affect display. Same for JavaScript - it's executed. The image is simply requested and never displayed.

window.onload = function() {
    setTimeout(function(){

        // reference to <head>
        var head = document.getElementsByTagName('head')[0];

        // a new CSS
        var css = document.createElement('link');
        css.type = "text/css";
        css.rel = "stylesheet";
        css.href = "new.css";

        // a new JS
        var js = document.createElement("script");
        js.type = "text/javascript";
        js.src = "new.js";

        // preload JS and CSS
        head.appendChild(css);
        head.appendChild(js);

        // preload image
        new Image().src = "new.png";

    }, 1000);
};

DOM test page

For this way of doing it you can use any JavaScript library's helper methods to load stuff on demand. Good examples - YUI Get and LazyLoad

... or another ... (using iframe)

Another option is to create an iframe and append your components to its head. Using an iframe you can avoid the CSS potentially affecting the current page. JavaScript will still be executed.

window.onload = function() {
    setTimeout(function(){

        // create new iframe
        var iframe = document.createElement('iframe');
        iframe.setAttribute("width", "0");
        iframe.setAttribute("height", "0");
        iframe.setAttribute("frameborder", "0");
        iframe.setAttribute("name", "preload");
        iframe.id = "preload";
        iframe.src = "about:blank";
        document.body.appendChild(iframe);

        // gymnastics to get reference to the iframe document
        iframe = document.all ? document.all.preload.contentWindow : window.frames.preload;
        var doc = iframe.document;
        doc.open(); doc.writeln("<html><body></body></html>"); doc.close(); 

        // create CSS
        var css = doc.createElement('link');
        css.type = "text/css";
        css.rel = "stylesheet";
        css.href = "new.css";

        // create JS
        var js = doc.createElement("script");
        js.type = "text/javascript";
        js.src = "new.js";

        // preload CSS and JS
        doc.body.appendChild(css);
        doc.body.appendChild(js);

        // preload IMG
        new Image().src = "new.png";

    }, 1000);
};

IFRAME test page

... I'm gonna find ya ... (static page in iframe)

If your components are static you can create a page that has them all and load that page into the dynamic iframe. Static means knowing them in advance, not relying on page's JavaScript to figure them out on the fly. As you can see, much simpler than the previous code.

window.onload = function() {
    setTimeout(function(){

        // create a new frame and point to the URL of the static
        // page that has all components to preload
        var iframe = document.createElement('iframe');
        iframe.setAttribute("width", "0");
        iframe.setAttribute("height", "0");
        iframe.setAttribute("frameborder", "0");
        iframe.src = "preloader.html";
        document.body.appendChild(iframe);

    }, 1000);
};

IFRAME static page test

... I'm gonna GETcha, GETcha, GETcha! (with Ajax)

Finally - Ajax. Scripts are not executed, CSS not used for rendering. Nice and clean. For simplicty the code has no support for browsers that don't know what XMLHttpRequest is.

window.onload = function() {
    setTimeout(function(){

        // XHR to request a JS and a CSS
        var xhr = new XMLHttpRequest();
        xhr.open('GET', 'new.js');
        xhr.send('');
        xhr = new XMLHttpRequest();
        xhr.open('GET', 'new.css');
        xhr.send('');

        // preload image
        new Image().src = "new.png";

    }, 1000);
};

Ajax test page

Thanks!

Any other ways you can think of?