Universal Debugger Project

This is a reference and general documentation for a project I (Parasyte) have been interested in pursuing for several years. This article will discuss the project's history, current developments, and future.

Project history

I began my research into debuggers sometime in 1999. Things were different back then; computers were just barely able to emulate SNES, even with emulators of the time taking numerous shortcuts to help speed up emulation. At the time, I only know of one SNES emulator pre-compiled with any kind of debugging functionality, and that was a build of SNES9x (then a chromeless application which was horribly difficult to use) by LordTech. It allowed the user to enable and disable assembly tracing while the game ran. This was a good start; you could see what the game was doing, and gain a lot of information from it. But it was still clunky.

To find the assembly which read or wrote to a specific address, you had to search through (often many megabytes) of text for those addresses. It gave you a good piece of the puzzle, but it did not give you the whole picture. Often many instructions were branched over, and were entirely missing from the log.

A few years later, I had some C programming experience of my own under my belt, and I had also found a new NES emulator called NESten. It wasn't a great NES emulator, but it was decent; it ran most of the NES games I was interested in, and it also had a fairly nice debugger built in! It was during this time that I got my first real taste of debugging. (Previously I experimented with UltraHLE's debugging features, but I don't recall them being much use; it used another "look but don't touch" passive take on debugging.)

NESten's debugger (and the emulator itself) did have some annoying flaws. And after getting sick of it, I decided (like many pioneering hackers before me) to take a decent open source emulator and extend it with some decent debugging capabilities. I chose FCEU (FamiCom Emulator Ultra) by Xodnizel as my base. (Note: FCEU was previously based on FCE, by Bero.) FCEU was a good candidate because it had decent emulation capabilities, and also because it was quite fast; you really need a fast base if you're just going to slow it down with powerful features. And I'll get to that point later.

FCEUd

And with that, FCEUd (FCEU-debug) was born. It did not take long to get a stable debugger integrated into the CPU core and hooked up with a little chrome. Seems like it was only 3 or 4 months before it was usable, and a total of 6 months or so development time before I was happy with it. This debugger took all of the major ideas from NESten (including its disassembler syntax for displaying current work addresses) and added some crucial elements that NESten lacked (write breakpoints triggered by INC/DEC instructions, which increment and decrement bytes in memory). I also used ideas from PSX debuggers, such as "step out" which mutually comples the "step in" and "step over" process. And I'll get to that point, again, later.

Since then, FCEUd has been further extended by other members of the emulation/ROM hacking scenes. To this day, FCE has been the most widely forked emulator, in part, because of the work we did extending it from an NES player to an NES development tool. But as you will see, this will bring me to another point later: The work done on FCE-derived forks is very widespread without much communication between parties. Ultimately, it's a mish-mash mix of talent tackling the same ideas from different vantage points without direct collaboration. And I think it's a shame.

GCNrd

The main development cycle on FCEUd was during 2002. I learned quite a bit from this project, and it got me interested in debugging other architectures. The next architecture I took on was Nintendo GameCube, officially abbreviated "GCN".

Sometime in 2003, hackers discovered a flaw in the GCN game Phantasy Star Online, which was compatible with the GCN BroadBand Adapter (BBA). This quickly became known as the PSO exploit, and allowed running "homebrew" code on a consumer GCN.

The PSO exploit worked something like this: Hacker Joe sets up his own DNS server, and tells PSO to use that server IP address as its default gateway. This way, PSO asks Hacker Joe's DNS server where it can find Sega's PSO server. Hacker Joe's DNS server tells PSO that Sega's server is located on Hacker Joe's PSO server. PSO happily connects to Hacker Joe's PSO server, which then tells PSO that there is an update available. PSO happily downloads the "PSO loader stub" which is posing as an official update to PSO. PSO happily launches the PSO loader stub which takes over the GCN and begins downloading a larger executable (of Hacker Joe's choice) from Hacker Joe's PSO server. The PSO loader stub then launches Hacker Joe's executable, and Hacker Joe now has total control over his GCN. In short, PSO is tricked by Hacker Joe into running code that it never should have in the first place.

This was a huge breakthrough for GCN hacking. I was in a very poorly state at the time, with no real job and no real income. I was doing work for a certain company selling GBA flash carts. It got me by, but it was a far cry from decent living. I mention this because it means I couldn't afford PSO or a BBA; the two things I needed to begin work on GCN debugging. I had the skill to accomplish it certainly, so I took up donations from members in the community. Only a few responded, but that's all I needed. About $100 USD was enough to pick up the two items I needed, and I was on my way to fame...

History of GCNrd

06-22-03: First GCN AR codes decrypted.
06-23-03: GCNcrypt v1.0 released.
06-25-03: First unofficial GCN AR codes released: Sonic Adventure DX (U) codes,
          by Sappharad.
06-27-03: GCNcrypt v1.1 released.
07-03-03: Planning for GCNrd begins.
07-07-03: GCNrd development begins.
08-27-03: Development on GCN network library begins.
08-29-03: First data received from GCN over network, using homebrew network
          library.
09-01-03: First data received by GCN over network, using homebrew network
          library.
09-13-03: Harddrive crash causes problems. Interest in GCNrd is lost.
12-31-03: After 3 months without working on GCNrd, aside from the occasional
          bugfix in the libs, GCNrd development finally continues.
12-31-03: Milestone #1. First successful RAM dump grabbed from a GCN game using
          GCNrd. It takes 35.735 seconds to complete.
12-31-03: BBA is started in 100mbit full duplex mode. Full RAM dump completes
          in 14.687 seconds.
01-03-04: Work on screenshot support begins. First successful screenshots are
          taken from Rayman 3 and Rogue Leader.
01-12-04: Mario Kart: Double Dash!! (U) is supported by GCNrd. This was the
          first successful "BBA Reset" patch for a LAN\network-enabled game.
01-15-04: Milestone #2. GCNrd rightly claims better compatability than Datel's
          FreeLoader v1.06b.
01-17-04: All tested games boot! 64 of 64.
01-19-04: Kenobi begins work on a GUI for GCNrd, making the program a bit
          easier to use, and adding code search features.
01-23-04: Milestone #3. First GCN AR code created using only GCNrd and Kenobi's
          GCNrd GUI. Wave Race: Blue Storm (U), 99 Points, by Knux0rz.
01-25-04: Started adding breakpoint support. Breaks work for Read, Write, and
          Read+Write. Execution breakpoints will be next. BPR\W gave me a bit
          of trouble. The biggest problem I had was getting around the 8-byte
          boundary which PowerPC uses for data breakpoints. This was completely
          unacceptable. As an example, if you set a breakpoint on address
          805E4FF5, a break would be reported every time an address between
          805E4FF0 - 805E4FF8 was accessed. Even though only one address within
          that range had a breakpoint set. My current solution to this problem
          is pretty nasty, but it works. In fact, it only reports a hit if the
          break addresses match perfectly. Maybe a little inconvenient, but
          hey!
03-11-04: GCN CodeType Helper v1.0 released.
03-11-04: GCNcrypt v1.2 released.
03-16-04: Finished up breakpoint support today, which should now be 100% fixed.
          The debugger now waits until just before returning to the game
          before enabling any r\w breakpoint. This will help bypass any
          misleading break hits that are caused by the debugger itself. Such as
          when accessing the stack and etc.
          New plan in the works to work-around memory constraints. May not work
          on all games, but should solve all problems with PacMan World 2.
          Which currently suffers from a nasty crash when accessing the memory
          card. The crash appears to be caused by how much memory I have
          allocated for the debugger.
03-17-04: Work begins on patching AR's code engine. With the patch in place,
          the code engine will run GCNrd's memory-resident debugger, allowing
          users to hack games with multiple executables, such as demo discs and
          the James Bond 007 games.
03-18-04: Kenobi adds real-time AR code list handling to the GUI. Supported AR
          versions are v1.06 and v1.08. Support for additional versions will be
          made available as soon as we receive AR RAM dumps from the other
          versions.
03-19-04: Win9x compatibility issues are worked on. GCNrd console now accepts
          commands passed on the command line. With commands entered on the
          command line, the console program will exit immediately after
          completing the requested command. Support for Win9x compatibility in
          the GUI is limited, but progressing rapidly.
03-20-04: Support for AR v1.11 completed.
03-31-04: First public release! GCNrd v1.00b is made available to hackers
          everywhere.

GCNrd GUI

The GUI for GCNrd was written entirely separately from my own GCNrd development. It was called simply GCNrdGUI and was written by a French hacker, Kenobi, in Delphi. This split up the workload considerably, and also added a new level of modularity to the project overall. The overall GCNrd project consisted of four components:

• GCNrd Loader: The UI that the user is presented with on the GCN itself. The loader is responsible for GCN-side configuration, initializing the network hardware, launching DVDs, and setting up the debugger/hooking the executable.
• GCNrd debugger: This is a piece that the user never sees, but directly interacts with. It's a very small program (32KB total program code and memory usage) that runs "behind" the game, listening for instructions coming over the network, and acting upon them.
• GCNrd client: A command-line application run on the user's PC which can be used to send and receive commands and data to/from the debugger over the network.
• GCNrdGUI: Kenobi's GUI for interacting with the GCNrd client.

It was not necessary for the GUI to interact with the debugger through my CLI client, but that was the option chosen for GCNrdGUI. Later on another hacker, Sappharad, wrote his own GUI in Java which communicated with the debugger directly, bypassing the need for GCNrd client; the modularity of the project comes full circle.

GCNrd v1.10b and beyond

GCNrd v1.10b was released July 28th, 2005. It contained a number of bug fixes and new features. Some new features were only shiney on the surface (background images and UI colors in the loader, date/time display...), but some were actually quite important. The MMU handling, for example, allowed hacking games like Star Wars: Rogue Leader, Star Wars: Rebel Strike, and Metal Gear Solid: The Twin Snakes, which used the MMU extensively. It was a slightly rushed released, even though it did show some promise.

This was the last official release I made, along with a private release (only available to close friends) which included some special features like DVD dumping and loading/debugging DVD images over the BBA. This "DVD simulation" was quite advanced for its time, and allowed me (and a few others) to hack downloaded/dumped games. There are often times that the only way to get a copy of a game is downloading it. Either because it was never officially released in your region, or because it hasn't been officially released at all. This private "v1.10b+" version was never made public due to piracy concerns.

Those piracy concerns were actually forfeited after other hackers disabled the DVD booting code in v1.10b to allow debugging DVD-Rs booted with a "Cobra-like" DVD BIOS.

I had plans at one time to rewrite GCNrd so that it could be relocatable within GCN memory. This would solve problems with memory allocation on games that don't play nice. Resident Evil 4 is a good example of such games; it cannot be booted at all by any version of GCNrd. The rewrite would also make the source code legible enough that I could release it under the GPL. Granted, legibility is not necessary for source code release, but it was important for reasons of maintainability.

GCNrd source code used a wild mix of C and assembly in its main debug core. Escaping the low levels of assembly was impossible, but it could be structured nicely anyway. This wasn't the only problem, of course. Both the loader and debugger had to have their own separate build of the network library. It was too costly (memory-wise) to put the full library into the debugger, even though this would allow the loader to use the debugger for all of its networking tasks.

Instead, the debugger had a very minimal network library; I cut out all of the initialization sequences, all of the CRC code for packet data integrity checking, and a whole lot of other not-completely-necessary packet handling code. It was important to keep the debugger as small as possible so it would not interfere with the game's use of memory.

That gave me two entirely different network stacks to maintain; one was already quite a bit of work, as it was. There was also the problem of compiling the debugger separately from the loader, and then compiling the debugger binary directly into the loader itself.

There are a number of ways these issues could have been addressed, and I'm not going to cover them all. But I will get back to the early point about modularity. As you will see, I am hoping to solve a great deal of these problems using that frame of mind. Needless to say, I never did complete the rewrite of GCNrd, which would have become "GCNrd v2". But the problems displayed by it were the cornerstones of what was to become the Universal Debugger Project.

PCS and Mupen 64

The next year, in 2006, I went back to hacking N64. This was a time when some very interesting N64 hacks were made [1]. By this time, we were all very familiar with Nemu and its debugging/cheating capabilities. The debugger was not bad, but its emulation left quite a bit to be desired. Once again, unhappy with the tools available, I set out to tackle the problem and create my own tools.

The first was Parasytic Cheat Search, a video plugin for PJ64 (and other emulators compatible with its plugin spec) that allowed cheat searching through the emulator's memory. It contained many bugs and architectural flaws (being a video plugin which passed all video-related messages to a real video plugin) but it provided a start for me to get into N64 emulator development.

My next N64 project was tackling the debugger problem. Unlike a simple passive cheat searching program, a debugger has to be hooked directly to the CPU to work consistently. Like the FCEUd project years earlier, I wanted to start with a decent open source emulator and extend its capabilities to include decent debugging features.

I chose Mupen 64 as my base, this time. It had relatively high compatibility, stability, and speed. It also contained a debugger for its GTK+ build, which I knew I could use to my advantage. I was still a Win32 programmer at the time; I did not have much experience outside of Windows. Naturally, I focused on Mupen's Windows code, which is where my debugger would live.

After patching Mupen's CPU core (all three of them!) I started implementing the debugger window and all of its fancy widgets. This is when a very bad case of Deja Vu hit me. I had done all of this work before; I was reinventing the wheel. Not good. So I left Mupen 64 with a bare-bones cheat feature (including search) and debugger. This is when I began formulating the idea that debugger interfaces should be modular, just like GCNrd. Then came an avalanche of questions and issues. More on that later.

NTRrd and Kwurdi

In 2007, we had Nintendo DS (officially abbreviated NTR, unofficial NDS), and we had a means to hack it thanks to the efforts of the PassMe crew. I did not jump into DS dev quite as enthusiastically as I had GCN, but I eventually got into it and started playing around with ideas to debug the little machine. There were several problems, initially: The first was a communications port. I chose to use the GBA X-port, which was a rather expensive device (especially for video game hackers, who are typically in their teens, and without jobs), but it was attractive to me because of its FPGA.

The X-port comes by default with a simple LPT serial port that can be used as a communications link. But it is generally slow and unreliable for large data transfers. On the other hand, the FPGA (and a LOT of IO) can be leveraged to implement a much faster and more stable communications link. And that's exactly what I did, writing custom FPGA logic for a simple 8-bit ECP-mode LPT port. The result was a much faster (~4MB/s ... still much slower than ethernet, WiFi, or USB would have been) and much more reliable (~0.0001% error rate) communications link that I could use to dump NDS memory to perform cheat searching.

The initial result of this work was "NTRrd" which never saw any kind of release. That's when the ideas from past projects began to meld into an idea well ahead of its time. The idea of a universal debugger. Or at least, a universal debugger interface. So I came up with a silly acronym: KWURDI, for KodeWerx Universal Remote Debugger Interface. And I wanted NTRrd to be its first "client".

The idea was that NTRrd would be a three-piece project: NTRrd loader, NTRrd debugger, NTRrd client. And Kwurdi would be its GUI. The loader would load the debugger, the debugger would talk to the client, and the client would talk to the GUI. That's a lot of levels of abstraction. As it turns out, though, all of those levels are quite important. In fact, it's this level of abstraction and modularity which make this idea of a universal debugger important.

The universal debugger

The original article for the Universal Debugger Project can be found in the Debugging Modern Computer Architectures article. While it covers a good general scope of the idea itself, it does not lay down any foundation other than a very generic description of the roles that each piece of the project might play. To highlight the main points of the project:

• The universal debugger interface itself is a user interface to allow interaction with a low-level debugger.
• The low-level debugger may or may not live on the same machine as the universal debugger interface. The universal debugger interface communicates with the low-level debugger through a standard communications link and protocol.
• The communications link should be an existing standard. For example, UNIX domain sockets or DBUS for a local low-level debugger, and TCP/IP for a remote low-level debugger.
• The protocol should also be an existing standard, but to this date, there does not appear to be anything which fits the goals of a universal debugger interface. One possibility is RFC-909, Loader Debugger Protocol.

Project goals

The long term goals of the Universal Debugger Project include:

• Modularity
• Standardization

These items are to be taken as mutually inclusive. For modularity to work, there must be a set of standards in place. For a set of standards to be successful, the modularity granted by the standards must be exploited.

For our purposes, I wish to implement a set of standards for a "universal debugger" with the help of the community, and implement a debugger interface on top of those standards as an example of modularity. The implementation has the following goals:

1. It will be Free Software, released under a free software license compatible with the GNU General Public License.
2. It will be a community-oriented project; accepting ideas, patches, code, images, and documentation, among other items, from the community which supports it.
3. It will be designed as a cross-platform application, able to be used on multiple operating systems.
4. It will be built with internationalization and localization in mind; capable of being adapted to different languages.
5. It will communicate with and support any low-level debugger which supports the standards defined by the Universal Debugger Project.
6. It will be configurable, extensible, and modular by nature, to support any conceivable current or future architecture.
7. It will support highly advanced debugging techniques and bleeding-edge technologies to get the most out of the user's debugging experience.
8. It will support and encourage community activity through collaborative debugging.

I will cover these points one at a time in more detail:

Free software

Free Software released under the GNU General Public License or a compatible license is meant to be community-driven. This means, anyone who wants to participate in the development of the project is free to participate. You don't become a community member by being one of a few elite deemed "worthy"; you become a member simply by joining an active discussion or contributing in some other way.

The source code will be made available for everyone to look at or extend, which benefits from many eyes keeping the code safe and secure, and also from many hands working in parallel to quickly produce a stable product that is free for anyone to use in any way they see fit. These are two of the basic freedoms granted by a Free Software license: Freedom to look at and extent the product, and freedom to use the product in any way. There are many more freedoms granted by Free Software, and if interested you should consult the GPL documentation or other authoritative sources.

Community-oriented, community-driven

A community-oriented and community-driven software project benefits from its main source of use: its own users. Such projects become self-sustaining, as their users take the secondary roles of developers. I will not cover many of the details relating to community-oriented or community-driven software, but there are plenty of great resources available for further reading. Some suggested reading includes The Cathedral and the Bazaar [2] by Eric S. Raymond, the About Mozilla pages [3] [4] [5] [6], and the online book Open Sources: Voices from the Open Source Revolution [7].

As a community-oriented, community-driven project, I believe there is no end to the ways in which we, as a community of programmers and hackers, will be able to debug. This community would ideally involve the active roles of everyone from video game and ROM hackers, application and operating system developers, and people in the field of academic research of computer architectures. It doesn't have to be the biggest and the best of the debugging communities (and it probably won't!) but it does have to be an accepting community, willing to allow contributions from anyone and everyone who wishes to contribute.

Cross-platform design

Among the design goals of the project, the universal debugger implementation must be able to run on multiple computer platforms and operating systems. This is a difficult undertaking, especially for a program focusing on a graphical user environment. Several bases for cross-platform development already exist, so dealing with intricacies of operating systems can be cut down to a minimum, while retaining a high level of overall product quality.

I have personally researched several offerings for cross-platform frameworks and libraries, and I believe the best choice is the Mozilla platform, XULRunner. XULRunner is still in its infancy, but is already very capable; it's the powerhouse behind the Firefox web browser, which runs on a large percentage of popular operating systems.

Here is a short list of advantages to using XULRunner over just any cross-platform framework:

• XULRunner is Free Software, and its development is community-oriented.
• Unlike Java and .NET, XULRunner applications are built with native code and have a native look-and-feel ("native" is in reference to the host operating system).
• Also unlike Java and .NET, XULRunner applications are written in C++, a highly portable and widely used programming language.
• XULRunner provides many features that Firefox users are already familiar with: scripting with Javascript, application extensibility through "extensions", networking and popular web protocols support, HTML processing (and a LOT more) with the Gecko rendering engine.
• User interfaces in XULRunner are written in XUL (an XML dialect), styled with CSS, and setup to "do things" with Javascript -- all current web standards. The use of CSS allows the user interface to be "skinned" to a user's personal tastes.
• UI functionality is mostly implemented with Javascript, but backbone and critical code can be written in C++ and accessed from the Javascript environment.

For a very informative read on the Mozilla platform, I recommend the online book Creating Applications with Mozilla.

Internationalized and localized

Because community strengths are so important to a project of this size and magnitude, the global community becomes an invaluable asset. And as a diverse global community, not everyone will speak or understand a single common language. In this case, I am directly referring to the English language, which is personally my primary language, and the only language I am entirely fluent. Programmers and hackers in a similar situation, though with a different primary language, will benefit from a universal debugger. We should not neglect any user in the community. Making the universal debugger accessible and usable for everybody must be a top priority of the project.

Choosing the Mozilla platform as the application's foundation will play a crucial role in the internationalization of the product. Because the user interface elements are written in an XML dialect, all text can be referenced with DTD entities, just like you might write > in XHTML to print the greater-than sign (>). Likewise, you might write

<caption label="&hello;">

in XUL which would display a label with "Hello" or "こんにちは" depending on the locale set on the user's machine when the XUL application is started.

DTD entities are not magic translators; the text must be translated by a human who is fluent in both (source and destination) languages. However, using DTD entities is a great way to write user interfaces (and other things, like configuration files, for example) without locking the program to a specific language. It also makes the translator's work much easier, since all text strings which require translation will be neatly stored in a single location.

Having the capability to bring a universal debugger to anyone, no matter their ethnic background, will help to expand the debugger's user base and ultimately its supporting community of developers. Designed from the beginning with breaking language barriers in mind, there should be few stumbling points along the road to the success of reaching this goal.

Standards compliant

Standards make the technological world work. And standards compliance makes compatibility issues entirely negligible. This is perhaps the most important point I will make within this documentation: defining a series of open standards to encompass all foreseeable debugging functionality is a necessity.

The major reason for adopting a standard or series of standards is for compatibility purposes. Compatibility is critical for modular designs. And modularity is an important part of the overall user-configuration and extensibility of an application or sets of applications. Modularity also has the benefit of splitting workloads, as we did previously with GCNrd. Splitting workloads decreases development times, improves communication, and in some cases even improves product quality.

In simpler terms, starting with splitting developer workloads, we all know that it's easier to cover more ground by splitting up. So long as you have a central location to meet up later, this tactic works wonderfully for rapid application development. In the case of developing a debugger, you might typically have one team focusing on the low-level handling of the CPU and hardware, another team tackling the user interface to the low-level debugger, and perhaps a third team covering the protocol between the two.

I realize that the "third team" in this example is most likely set aside as another task for the first team in the real world. But if your project does require some sort of communication layer and you pull that out of the first team's hands, you can effectively pass that responsibility to a sort of "standards body" who will oversee the development and standardization of that communication layer. The job of such a body would be to ensure the standard is usable by anyone and for any purpose, generic enough to support all current and foreseeable architectures, and extensible enough to support all unforeseeable future architectures.

The extensibility of the standard would be the most difficult to maintain. While a protocol or communication layer of this sort should be extensible for the reasons listed, it should only be extensible to the extent that any new features added would not break previous features or introduce new compatibility issues with previous designs and implementations of the protocol or communication layer. This means, generally, that the extensibility must remain open and optional; no extended feature should ever be mandatory for any user to implement.

There will undoubtedly be important new technologies which require mandatory extensions to the protocol during the life cycle and evolution of debugging technology. Under these circumstances, the standards body would be responsible for designing and publishing a new major version of the protocol with the new mandatory technologies. Implementers of the protocol, should they choose to support newer major versions, would then be required to implement all mandatory features in order to claim standards compliance of their implementation.

What we need is a standards body; a group of individuals, mostly volunteers or "elected" members, to define and publish a series of standards for debugging modern computer architectures. They will be responsible for taking everyone's concerns from the community into account, and act on those concerns within their design and definition. They will be responsible for knowing that if they fail to do a good job, the community will stop listening to them and they will easily be replaced by anyone who steps up with a better idea.

The goals of the standards body will be defining:

• The wire; how we get data from point A (low-level debugger) to point B (user interface). Ethernet? RS232? ...
• The language; how our data is interpreted as points A and B speak. XML? JSON? Pure binary packets? ...
• The expansion; how we can introduce new features without creating vendor locks or introducing incompatibilities. Header/Info/About/Options commands which specify the optional/extended features we support? ...

The goals for us developers will be inheriting some of these responsibilities as we implement these standards, to create a flourishing environment for debugger developers to contribute with innovation and healthy competition.

Configurable, extensible, and modular

Debuggers are all about user experience. I say this with honest integrity because debugging, by its very nature, is a difficult task. Making a debugger very simple to use is a bullet point; you don't want to overwhelm a user (who is already working on a very difficult problem) with bad user experience. To improve the user experience, it is important to allow a level of customization among the user interface. Also important is extensibility and modularity, to make it easy for the user to expand the interface to fit her needs, and easy to replace one component with another that she prefers.

Some of the items of the user interface which should be configurable include the overall look-and-feel of the interface (skin), the organization of windows and widgets (tabs, information panes, etc.), and the styles of text, fonts, and [image-]backgrounds used throughout.

The look of the application, as well as the functionality, should be extensible using styling and scripting languages. The Mozilla platform once again becomes an asset as it allows simple integration of CSS and Javascript for the purposes of extending applications based on the platform. Extensions can be written and managed by the user community, much the same way that Firefox extensions are written and distributed. A central management location for extensions would be a great thing to have, but is not necessary to the success of a thriving extension-based sub-community.

Including certain feature sets in the application's base install as pre-installed extensions means that users may uninstall those extensions at will to remove undesired features, or replace those extensions with other extensions that implement comparable (and often better) features. This modular design will help to increase the user experience as well as the user's personal productivity with the application. This, too, can be supported by a thriving extension-based sub-community, and the community at large.

Advanced debugging technologies

[Work in progress]

Encouraging community activity

[Work in progress]

See also

• Debugging Modern Computer Architectures