About Me
Feed
More Haml Benchmark Issues
This morning I got a bit of an unpleasant surprise in my feed reader. had pointed out an interesting line in the longstanding ERB file that we use as a baseline against which to test Haml’s performance. The two files are supposed to be as similar as the difference in language permits, but Magnus pointed out a serious discrepancy.
The relevant line in the ERB file was <%= line %>,
but the one in the Haml file was - line.
That is, the ERB variable was being concatenated to the document,
but the Haml variable was not.
Which means the ERB benchmark was doing a lot more work
than the Haml benchmark.
The upshot of all this is that all previous benchmark results are invalidated;
in particular, Haml is significantly slower than I had thought and indicated.
The previous benchmarks indicated that Haml was slightly faster than ERB for similar documents.
Now according to my benchmarks Haml 2.0 is 3.24 times slower than ERB,
or 2.42 times slower with the :ugly option enabled.
The Haml master branch, which will become Haml 2.2 not too long from now, fares a little better.
According to my benchmarks, master is 2.54 times slower than ERB by default,
but is in fact 2% faster than ERB when the :ugly option is enabled.
This isn’t quite as good as I had thought, but it’s something.
In light of this, :ugly will probably be enabled by default for production in 2.2.
I’m as unhappy as anyone about this inaccuracy. I’ve gone through the benchmark to make sure nothing else is slanting it unfairly. If nothing else, this certainly provides incentive to keep speeding up Haml however possible.
Ada Lovelace Day: Gayle Laakmann
I’m very sorry to hear about this, hopefully you’ll be able to make Haml as fast as Erubis again.
While the benchmark tests a lot of weird cases, it’s still not a very realistic template. I think we should find a completely new template, which focuses more on escaping (for user-data), looping (lists) and anything else that’s used frequently. What’s your opinion?
Pretty much all of the weird features used in the template don’t actually show up in the compiled code, in either ERB or Haml. The only things that have a real effect are tags and plain text, which is slightly faster in Haml, and dynamic code, which is slightly faster in ERB (although identical in Haml with
:ugly).That said, I’d love to bring in some (potentially more expansive) real-world templates to replace the benchmark.
I can confirm that HAML is 3 times slower than ERB. I am developing Sinatra based application and I was considering HAML 2.0 for templates, but using ‘ab’ benchmark with HAML it was 9ms / req, with ERB ~2.5ms / req which makes a lot of difference.
It just seems that code laying at @precompiled variable is just slower than the code generated by ERB. However I believe there’s still room for improvement there especially looking at the 2.2 results (:ugly option)
Also it would be nice if someone of you updated the haml page, coz: “The speed increase for 2.0 is nowhere near as impressive as 1.8, but it’s still interesting. According to our benchmarks1, Haml is now slightly faster than ERB, even when rendering via ActionView.” is simply untrue and may ruing the trust in Haml (almost ruined mine ;P until I reached your blog). It would be better to put some good news about upcoming 2.2.
Keeping my fingers crossed!
Sinatra is not the best place to test Haml speed, because (at least last time I checked) it doesn’t do any caching of the Haml templates. Haml takes rather a while to compile the template to Ruby code; much more than ERB, for obvious reasons. If you’re interested in efficiency, this Ruby code should be eval’ed into a method that’s then re-used. That’s how the Haml benchmarks are run.
You’re right that the homepage should be updated; I forgot that it referred to the speed.