Tuesday, June 21, 2011

5.0 and 4.0.3 finals

With no major problems found in 5.0, the RC is now the final release. 4.0.3 is also released as a security and stability update only, with issue 63 also repaired, which marks the end of TenFourFox 4.

The next "beta train" for Firefox 6 leaves in two weeks. I'm in the process of moving to a new house, so this gives me a bit of a break before having to work on it. A "release pack" will be available with 5.0 changesets to apply to mozilla-release as soon as that repo stabilizes.

Unfortunately, Tobias' testing showed that tree vectorization didn't help much with gcc 4, which is still our baseline compiler. We'll look at this again when we consider jumping to 4.2 or 4.6. There are some other smaller things I'm considering for 6.0 that I can complete in the timeframe available. More on that later.

Thursday, June 16, 2011

5.0 RC released, 4.0.3 plans, and onward to 10.4Fx 6

For 5.0 we actually have a true RC, which I have released as (surprise surprise) version 5.0. If there are no major problems, then it will simply "become" the release version. The RC fixes issue 68, which turned out to be a JavaScript error Mozilla introduced that only affected us because we are pure tracejit (methodjit and tracejit together don't have the problem), and was fixed in bug 663789. It just so happened that pages that dynamically instantiated plugin elements were the most badly affected by it, but other pages could trigger it, and this doesn't change my intention to disable plugins by default in 6.0. However, no major issues have occurred with plugins otherwise, so 5.0 will be their last stand.

On or around June 21st, the following will happen:
  • The RC will become the final release, assuming a respin is not required. (If it is, you will be prompted to "upgrade to 5.0." Just do it, trademark of Nike, Inc.) General users will then be offered it.
  • 4.0 users will be also offered 4.0.3. This is a security update for 4.0 and will be the last of the 4.0.x TenFourFox releases. It will also have the fix for issue 63. They can choose to stay with 4.0 if they have extensions that require it, or jump to 5. Note that we will be somewhat forcing people to stay in update order; i.e., if you skip 5, you will be still offered the update to 5, which you should do; then update to 6. The reason is to allow the browser to rebuild certain databases that may change from version to version.
  • Builders: new patches against mozilla-release instead of mozilla-beta will be available at that time as well. I expect little or no changes, but you should use those patch sets and that repo. If Mozilla offers source for Firefox 5, then we'll work up a wiki document about applying your changes to that, because mozilla-release will be unmaintained, used only for chemspills, and will be overwritten by the next version. If they do not keep a source tarball around -- which admittedly would not be SOP for them -- then we will have to and our tarball will have the patches already applied.
With regard to the security aspect, the rather spectacular Firefox WebGL video memory access exploit that has been making the rounds thanks to an (IMHO) rather impolitic and indiscreet disclosure does not affect us, because we disable it at the code level.

Onward to 6. Unlike the transition from 4 to 5, there is a lot more in the transition from 5 to 6 due to a full refresh cycle. However, we are less likely to have the initial panic I had trying to get 5 together because we are now compatible with the current Mozilla build system using Tobias' ported linker and I am not aware of any large changes in that, in IPC or in NSPR, so the odds of jumping to 6, while never guaranteed, look pretty good.

For 6, Tobias has investigated accelerating JavaScript by compiling it with vectorization. My initial attempts to vectorize the entire browser were shockingly unstable, but his initial tests with just NSPR and JS in issue 52 look promising and yielded around 10% improvement in benchmarks. We'll need to make sure that the same thing applies for stock gcc 4.0, however, and unfortunately our G3 friends won't get this benefit since it requires AltiVec. We will also want to bang on it thoroughly for the kinds of subtle errors such compilation can introduce. Once this stabilizes, and assuming it works, it's time to start thinking about porting methodjit to PPC and taking advantage of even more power, since IonMonkey is still a good couple of iterations away.

Plugins will be pref-ed off in 6 and there will be more switches for that so that you can at least still use them in 6, or use them in less unsafe situations, albeit without a safety net. By 7, however, I have a pretty strong premonition that they will break in major ways. We'll see. Even though we "have" IPC in 5.0, it is a gutted and almost completely non-functional IPC library in 10.4, present merely to make the build system happy and give it something to link against, and so plugins remain in-process.

Towards the end of a plugin-free existence is this remarkable proof of concept from the Mozilla team on implementing PDF viewing in JavaScript. Not merely a simple renderer, the code actually turns the PDF into JavaScript and executes it -- a genius idea, because the JS generated will also be accelerated, making PDF rendering remarkably fast. Although his demo has some obvious graphical glitches still, it is amazing how well it works even in this embryonic state. Look for it as an add-on coming soon.

Speaking of, a number of you had voiced concern about Mozilla moving too quickly with version numbers and leaving extensions behind (part of why I'm hedging our bets a little with 4.0.3). It does certainly seem like this is on Mozilla's mind as well, which is the apparent reason behind a meeting on risk mitigation they have scheduled before the formal Firefox 5 release. However, it appears clear that right or wrong, good or bad, rapid release is here to stay. Let's all curse Google Chrome together.

Release notes and builds for 5 RC:

Thursday, June 9, 2011

TenFourFox 5 beta 2 released

Beta 2 is released. This fixes the issue with widgets and bookmarks on Leopard (confirmed this time :), and has the video repaint fix I was waiting for. Sadly, this fix reduces speed a bit, but the display glitches the bug was causing were really quite unacceptable. People who also want to use the Ghostery add-on should see if the tweak in issue 67 fixed things for you.

Beta 2 also adds more AltiVec-accelerated scaling and YCbCr conversion routines to make three of four scalers converted, which reduce CPU usage while playing back video by about 33%. For example, my 1.33MHz iBook G4 can now play Mozilla's brand video with only occasional video freezes, and the 450MHz Sawtooth can now finally play it at all. In fact, VP8 (WebM) video due to our juicing and custom code now performs better than VP3 (Theora). The fourth scaler is a complex edge case and I'm trying to weigh how to best do it. I should also note that these AltiVec accelerated modules are actually only about half as effective as they could be -- since they are more or less simple-minded ports of the Intel MMX versions, they only use 64 bits of the 128 bit AltiVec vector registers because the MMX registers are only 64 bits wide. Once our versions are well-proven, I will try to refactor them to handle more data at a time; it's just easier to debug the algorithms if they are similar.

That's that for feature development in TenFourFox 5. There is one new bug that came out late while I was building beta 2 and this one worries me a bit, namely, issue 68. This has to do with plugins being dynamically instantiated by JavaScript, and it fails (and, possibly, crashes) on TenFourFox 5 for reasons I am so far unable to fully determine. I can wallpaper the issue, and wallpapering the issue will at least let us ship a somewhat "safely broken" plugin host in 5, but really, even if I do solve the actual underlying failure this absolutely must be the last version for plugins. There are too many ugly plugin-related bugs already with non-obvious causes and I can't wait to see how Fx6 and Fx7 will make them worse. The wallpaper fix is not in beta 2, but it will be in the next one if I don't figure out what caused it.

Read the release notes, then get:

Tuesday, June 7, 2011

5b2 delayed, sorry

5b2 is being delayed a bit, for two reasons: 1) I want to get the complete AltiVec-enabled YCbCr-to-RGB conversion suite into it for testing (one part of four is in 5b1; I've finished a second part and am working on parts 3 and 4) and 2) bug 662450, which yours truly reported and is a Mozilla bug, but disproportionately affects us as we are unaccelerated. Mozilla is currently treating it as a blocker, so there should be some resolution to it soon.

I also landed issue 67, an issue with recursion in the JavaScript regular expression parser, which I'm a little concerned about functionally and don't want to land on 4.0.3 in case it has unusual side effects. Those of you who reported issues with Ghostery compatibility with TenFourFox 4 might want to try it with 5b2 when it comes out and see if that repairs your issues (Wolfgang O comes to mind, if you're reading).

As soon as the two issues are completed, I will run off new builds, which also fix issue 65 on Leopard. We will probably be able to get 5 final out on time with Firefox 5, and I plan to release 4.0.3 at the same time as the final release in the 10.4Fx 4 series. I have not seen any new landings on the Mozilla beta tree that would obviously break plugins, so assuming they still work, they will still stay. However, a big change landed on the tree for Firefox 7 that could potentially break them, so I still plan to pref them off in TenFourFox 6.

Thursday, June 2, 2011

Beta Five is alive!

Thanks to Tobias' linker and work on the build system, I am now pleased to unveil TenFourFox 5 beta 1 (roughly Firefox 5 beta 3, although it is hard to assign numbers due to Mozilla's new release strategy, so I'm arbitrarily calling ours "1"). And now you will get to play with it.

Firefox 5 is pretty much the 3.6 to Firefox 4's 3.5 (speaking of, 3.5 is toast, so if you don't want to use TenFourFox at least use 3.6 or Camino), which is to say, it is the minor feature and bug fix update. Actually, there are at least a few marquee features worth pointing out despite the brief gestation, and the big one is CSS animation, which is something WebKit has supported for awhile and now Mozilla's getting into the act. This is the technology that I personally hope will nuke Flash for good. You can see some demonstrations of it, including this simple animated text page and, impressively, the Mad Men title sequence rendered in CSS3 (high end G4 strongly advised). Most of the rest is ticky stuff and spec compliance. The return of WebSockets, a custom interface for permissions, and other bits of challenging electronics will not appear until Firefox 6, which will hit beta in about three weeks.

We haven't sat on our butt for this one, however. Besides the work expended on porting the Firefox/Chromium IPC code to 10.4 (mostly by gutting it) and hacking on the build system (more on that in a moment), we have some new features of our own. First, JavaScript acceleration is now enabled for browser chrome. This means that the browser itself, a fair bit of which is written in JavaScript too, now gets fed into our exclusive PowerPC JIT tracer. The improvement is not nearly as dramatic because a lot of features the browser handles do not benefit from it as much as web pages do, but it still makes the browser quite a bit smoother.

Second, JavaScript typed arrays are now enabled in the accelerator. These are a recent addition to Firefox allowing fast native storage of data, and are an important part of the audio API and certain high-performance applications (and WebGL, even though we don't support it). Previously TenFourFox could not use the JIT for this type of code, which meant such applications trying to do for speed actually ended up running slower. Now they don't. This means smoother audio and faster overall performance for these specific kinds of applications (note that this doesn't affect benchmarks much since SunSpider and Dromaeo don't really test this).

Third, we are starting to add more AltiVec acceleration to the content chain, as promised. First out is AltiVec-powered scaling, which uses a SIMD routine to speed up filtering and scaling of images and video. This is in this build already. The next step is to accelerate YCbCr ("YUV") colour conversion, and then specific optimizations for libjpeg-turbo (we are using the unaccelerated version in TenFourFox 5, but this is already quite a bit quicker than the old library, so it's already a win). This might slip to "TenFourFox 6," but hopefully some of it will appear in 5.

Other changes specific to TenFourFox include expanding the stack even further (now Kraken can run, which crashes 4.0.2; this fix will also be in 4.0.3) and dialing back the loop filter for WebM videos a bit, which gains some marginal improvement. Unfortunately, HTML5 video is still mostly the domain of the G5 right now.

Oh, yes, plugins: the "P" word. Surprisingly, they still appear to function, but I haven't exhaustively tested them. Since we have some higher priority work to do rather than expend time and effort into something I already know will be unbelievably unpopular, I'll let you play with it and if their support and operation seems no worse, I will leave them alone for 5. However, some changes are in the pipeline for 6, so expect it then. Please note that they do not run out-of-process; the IPC code is gutted and they can't even if they were capable of it. Also do note that by the time Firefox 6 (and hopefully TenFourFox 6) come out around August, almost a year of no updates to PPC Flash will have elapsed. Imagine how full of security holes and crash bugs it oughta be by then. Just saying.

Let's also talk about what had to be done to get it to compile (builders pay attention). Tobias' ported linker from Xcode 3 is now required to function and is available for download from the Google Code project. Although it appears to be perfectly fine to replace your Xcode 2.5 linker with (and supports 32 and 64 bit in one executable), I prefer to keep my old linker(s) around, so there is a shim that comes with the changesets to install in place of /usr/bin/ld which lets you dynamically change which linker you use (handy for making libtool use a linker other than the default). For details on that, see README.ld for how to install the shim and the Xcode 3 linker, and how to get the build system to use it. I am rewriting the build notes and those should be ready in the wiki already. You do not need to install the linker just to run TenFourFox, by the way, only to compile it from source.

The reason the custom linker is needed is because Mozilla requires everything linked together into a big orca-fat superlibrary called XUL. This library roughly consumes around 750MB of space, and the regular Tiger linker can't cope even in 64 bit mode. In debug mode it is not stripped and remains this size, but is runnable in Tiger. In the optimized builds, it is stripped down to the typical ~25-30MB size.

Part of XUL (a/k/a libxul) is the IPC system, derived from Chromium (yes, that Chromium). Most of it will not build on 10.4, but we are fortunate in that Mozilla doesn't use most of it -- they use NSPR. Therefore, getting it to build was simply stubbing out the functions that could not be converted until it finally compiled. The only part Mozilla seems to use is the IPC portion for out of process plugins, which we don't support anyway, but more to the point can't support: it requires posix_spawn, which does not exist in 10.4 and would be very hard to emulate. Fortunately Mozilla offers a way to blacklist plugins to always run in-process, and all plugins are blacklisted to do so in TenFourFox, so we don't ever run that code. We will have to revisit this problem when Mozilla goes to one-tab-per-process, however, because this code will be needed for it.

Things that need to be tested: I believe that our old update notification system should still work, but please let me know if you don't get update announcements. There will be one more beta prior to release, so you should get flagged when beta 2 arrives (don't be quick to update; wait and see if you get prompted). We will not support the auto-update strategy because frankly if you're using a Power Mac, I know you appreciate control over your setup and not being forcibly updated unless you explicitly ask. Also, like any beta, there may be Firefox-general bugs and you should be prepared for those, although I have been using TenFourFox 5 myself for the last week and it seems really quite solid.

Well, read the release notes, then go get it and tell me what you think: