The Windows Mobile Image Update System - Updating your ROM without losing data!

***THIS POST IS NOT COMPLETE, I WILL UPDATE MORE LATER***



First, an introduction:



The Image Update system allows the OEM (us! ) to issue updates to a "Live" filesystem - without disrupting user data. This allows, for example, a buggy driver to be updated after the phone has been shipped, or a software package to be updated to the latest version, with minimal knowledge on the user's part. The system validates all updates against an internal list of certificates, and refuses the update if a match is not obtained. This system can also be used to deploy entirely new software to the device (such as support for another locale, input method editor, application support for a new feature the carrier is rolling out, etc.)



Potential usage scenarios for this system



A central server could be maintained for all SYS/OEM updates - each ROM Chef would need to maintain a list of original packages, any updated package(s), and download URL's for each updated package. The user would then receive these updates through the built-in AutoUpdate facility in Windows Mobile, which can check periodically, or on-demand. Each Chef could maintain seperate download servers from the update server to minimize server load.



Alternatively, a chef could provide .cab.pkg updates in his or her ROM thread, on their own web site, etc., and the user could download these and install them at will. These packages can optionally be authenticated to be coming from the Chef, if the Chef wants to ensure updates are coming from him only. A public certificate could also be used to allow users to issue updates as well.



The more technical Summary



Image Update allows an OEM to issue updates to the OEM's, XIP, SYS, (possibly) Radio, or any combination of these. The update can be pushed to the user via a specially formatted SMS or by manual execution. There are at least 2 levels of certificate checking involved in the process, I believe against \SYS\Metadata\DefaultCerts.dat. The system reboots into the ULDR to apply the update, because the filesystem cannot be modified while actively mounted. The ULDR provides a minimal operating enviornment to facilitate this.



How does a Chef need to prepare a ROM for Image Updating?



The Chef would need to use a ROM Kitchen that leaves the .dsm and .rgu file structure intact (i.e. an "unprotected" ROM) - All .dsm's in this ROM would need to be properly formatted with Package Name, versioning info, etc. during the cooking process, in order to facilitate version checking, etc. Each .dsm would need to be signed with a certificate the Chef would use to validate the update as coming from him or her. (Alternatively a public certificate could be used like SDKCerts if the Chef chooses not to maintain direct control over updates, and allow other users to create updates as well)



The Chef also needs to ensure not to Reduce the ULDR partition or remove it; this would cripple the Update Loader and prevent the Image Update system from functioning.



The .cab.pkg format



At the root of a package, the .dsm defines the Package structure (all files, registry entries, etc.) It contains version info, certificates, and other data. A ROM consists of a number of these packages, in an area of flash memory that is not user-writable. When someone wants to issue an update using the ImageUpdate system, they create a matching .dsm, same guid, with a newer version number, same internal package name, same processor ID, os version, etc., there is also a flag that can be marked as an update package or a new package - in this .dsm they define the files that will make up the new, updated package. Here they can add or remove files. One of the files defined by the package is optionally an .rgu, and this defines the registry entries associated with the package. Also optionally included is a provxml to facilitate file operations (copy/replace/delete/rename/etc.) and further registry or metabase operations. This collection of files is rolled up into a ".cab.pkg" archive by a program like cabarc. Once in a pkg.cab format, the package is signed.



A .cab.pkg is referred to as a "Canonical Package" and multiple Canonical Packages can be rolled up into a single "Super Package" to facilitate updating multiple Packages at the same time.



Inside this .cab.pkg is where the "MNGE" file format comes in to play. Essentially, this format is Microsoft's way of storing an XIP Module in the filesystem. (Their equivalent of our imageinfo.bin + s000, s001, etc.) - The "MNGE" format is simply a 4-byte MNGE header, followed by the imageinfo.bin, followed by the s00x sections attached to the end. When the Image Update system processes a file in this MNGE format, it is added to the IMGFS as an XIP Module.



Deploying a .cab.pkg to a device



Now there are several ways to deploy this package to the device, the primary way is an OMA-DM SMS message, which triggers the Image Update system to initiate a download from a server defined in the SMS message. The server can be either a normal HTTP server or a secure HTTPS server. The Image Update system will prompt the user to plug in the usb cable to download over the users home connection if the user has not already authorized the Auto Update system to utilize their GPRS data connection.



Another way for a .cab.pkg update to be pushed to the system is simply through a file copy operation, be it ActiveSync, SD Card, Bluetooth, or otherwise. Once on the device, the .cab.pkg is executed by the user the same way a .cab would be.



The Update Agent



Initiated by either a completed download from push SMS, or user-executed, the "Update Agent" program (which is part of the \SYS\FWUPDATE Package) attempts to validate the Certificates, Package dependancies, and other info contained in the .cab.pkg and .dsm. Once validated, the "Update Agent" sets a flag that the bootloader reads, the flag is a boolean, off = boot into normal OS, on = boot into ULDR - so then the system reboots, the flag is read, and you load into...



The Update Loader



The "Update Loader" or "ULDR" which is a minimal kernel configuration, that provides just enough driver support to display info on screen, respond to user input, and read/write from the internal flash (NAND or NOR)



From here the ULDR does further validation on the .cab.pkg, and applies it to the filesystem. If there are any modules in the package it dynamically relocates the memory map to make sure there are no overlaps. This is why it's important that reloc's not be removed from your ROM - ULDR will fail in this case.



The End Result



Once the ULDR has completed updating, the device is again rebooted, back into the full system, where the now-updated packages are now a part of the IMGFS, any updated files are processed (.rgu, .provxml, etc.) - The package is now a full part of the ROM.



The new .dsm replaces the old .dsm (along with the other files in the package) and now a future update will be checked against this new package.



If the update was pushed via OMA-DM SMS, or AutoUpdate, the device Pushes a notification to the OMA-DM server notifying it of the update status.



What's missing right now to implent the ImageUpdate system?



We need a Kitchen that's properly configured to allow us to create versioning info, proper package names, and insert this along with a certificate (or multiple certificates) into the .dsm's.

We also need the Kitchen to be able to modify \SYS\Metadata\DefaultCerts.dat with the certificates used, so that it passes authentication. Alternatively the authentication checking could be patched out. (this one is easily doable at build-time)

We need a program that can convert from a standard file to an MNGE format, so we can implement modules in our .cab.pkg's. (done it seems, thanks ervius!)

We (optionally) need a properly configured web server that supports HTTP/HTTPS, can communicate the proper xml configuration data, and can be updated with new packages by Chefs. (this one's a ways off)

We (optionally) need a program to convert from MNGE format to a standard file to facilitate extracting modules from .cab.pkg's. (working hard on that)

Read more...

How Do I Stop My Browser from Slowing to a Crawl?

Dear Lifehacker,

Everyone bickers about which browser is faster, but in my experience all of them eventually slow down after I've opened a few tabs and used them for a while. Whether I use Chrome, Firefox, or IE, my CPU spikes, the browser eats my memory, and it slows to a crawl. How can I stop this?

Please Help,

Need for Speed

Photo by William Warby.

Dear Need for Speed,

That's the trouble with browsers these days-everyone argues over a few seconds of startup and rendering time, but eventually everyone winds up complaining about how slow they get with regular use. Everyone has their favorite, but since we're not talking about any specific browser, here are a few suggestions to speed up any browser that's gotten bogged down over time.

Uninstall Unnecessary Extensions

Regardless of the browser you use, there are likely some extensions running in the background that you've kept as the browser has updated that you no longer need. The folks at the Mozilla Foundation are so convinced that old extensions are the reason behind poor experiences with Firefox that they've dedicated an entire page to calling out browsers that haven't updated their code for new versions. For example, FastestFox, a Firefox extension meant to speed up Firefox, is actually one of the browsers top-five slow performing add-ons. Chrome Extensions are no exception to this rule either. Even if they're helpful, they can often be the culprit when it comes to a slow browsing experience.

One of the best ways to speed up a bogged down browser is to take a hard look at the extensions you have installed and uninstall the ones that you don't need anymore. Also, check to see if newer extensions exist that perform the same functions as the older ones that, while not broken, also haven't been updated for a long time. For example, when I discovered that the author of one of my favorite mouse gestures extensions for Firefox, Mozgest, flat-out refused to support or update for any version higher than 3.6, the community stepped in and updated the code for him to support new versions. Unfortunately, the extension got slower and slower, and even though it was useful, there are other extensions and developers willing to keep their code up to date. Look around, you might be surprised.

Uninstall/Reinstall Flash and Java

For many people, especially those with iOS devices, living without Adobe Flash isn't such a far-fetched idea. You still need it for a number of sites, specifically sites that don't have mobile equivalents and require you navigate the site using Flash, but if you can get away with uninstalling it entirely and browsing without it, your browsing experience will be smoother. Google Chrome has Flash under the hood without you having to install a plug-in, which helps with stability, but you can still get a speedier browsing experience by disabling the Flash plugin. If you don't want to live in a Flash-less world, consider at least uninstalling Adobe Flash entirely and then re-installing the most recent and up-to-date version. You'll get the benefit of having the latest version, as well as the one with all the most recent security updates.

Java is the same way – unfortunately it's not as obvious when you need it and when you don't, so it's harder to enable when you want it and disable any other time, but Java is well known for leaving copies of itself lying around your Mac or Windows PC, updating frequently without removing old versions or files, and slowing your system to a crawl when in heavy use. At the very least, consider using a program like previously mentioned JavaRa to remove previous versions and clean up the mess it leaves behind.

Delete Browsing History/Caches/Personal Data

This is an old tip, but a good one-there's no real replacement for just cleaning out your browser's cache and all of the data that it's been keeping about you to speed it up. Plus, if you've never cleaned that data up, now is a good time to do it anyway.

In Firefox, you can clean up your personal data in the Firefox menu, under Options (Windows) or Preferences (Mac OS.) Click the Privacy tab to clear your recent history, cookies, or other browsing data. In Google Chrome, click the wrench in the toolbar and open the Options menu (Windows) or Preferences menu (Mac OS,) then click "Under the Hood" to see the buttons to "Clear Browsing Data."

Once it's all clean, you might consider reducing the duration between automatic cleanings by telling Firefox for example to only accept third-party cookies until you close the browser, or only save your history for a day instead of a week or month.

Change Your DNS Servers

DNS is like the phone book of the Internet, translating the web addresses and URLs that you type into the menu bar into the IP addresses of the servers where the sites you want or information you need resides. If you're like most people, the DNS servers your computer speaks to every time you type in a Web address or click on a link belong to your ISP, and are set up automatically when you connect to the Internet. However, those servers may not be the fastest, especially when compared with other DNS services like Google's free DNS and OpenDNS.

Namebench is a utility we've mentioned before that lets you compare how each of those DNS services, including your ISP's DNS, perform from your computer. Just install the app, and run it from your computer. You'll see the DNS servers you're connected to, and which ones are most responsive from your location. The service will even suggest how you should configure them on your system, in order of overall speed. Your ISP may win and you won't get a benefit from switching, but if you're like most people who are already displeased with the speed their ISP offers, you may benefit from a change.

Work with Multiple Browsers

If all else fails, consider using different browsers for different tasks. If you love your Firefox extensions and want to continue using them, but Flash video stutters and stops frequently in Firefox, consider using Chrome for your Flash-related needs and Firefox for other tasks. If you love Chrome's speed but find your app tabs crash all of the time, consider using Firefox just for those webapps and Chrome for your other browsing.

Go out on a limb and install Opera or Safari and give them a try for a little variety, and more importantly to keep some of your most resource intensive tasks isolated from the other things you want to do in your browser. That way your web surfing won't get bogged down because Flash is slowly crashing in a tab you left open a while ago, or because your app tab isn't refreshing as cleanly as it should.

Hopefully those are a few ideas that will help you speed up any browser, regardless of the one you choose to use. Plus, none of our suggestions involve installing something like an extension or plug-in that promises to give you speed back in exchange for the resources it uses. Remember, if all else fails, it might just be time to back up your data or profile and reinstall your browser to make it feel like new. Good luck!

Love,

Lifehacker

PS – What are some of your favorite ways to speed up your browser, whatever browser you may use? Share your tips in the comments below.

Read more...