Debugging Modern Computer Architectures
How does one solve the problems with current debuggers? First, by identifying those problems. Next by addressing them. Finally, in implementation.
Problems with integrated debuggers
So what are the problems with debuggers integrated in today's emulators? Well, for one thing, they are integrated. This can cause portability problems, in many cases (I am ashamed to admit my guilt in perpetuating this problem, by writing debuggers that vendor lock users to the Windows operating system). It can also cause undue stress for debugger developers. We are a lazy species, and we do not like rewriting the same debugger multiple times, attempting to port our work to a newer, better emulator, or porting it to a completely new emulated architecture. And then there is the problem of features, or lack thereof. Some hackers and homebrewers need specialized features in their debuggers.
Modularity is one possible solution to these problems.
The first thing to do is segregate the low-level debug primitives (functions and whatnot) from the user interface; make the interface modular, interchangeable with any interface. Then you define how the debug primitives interact with the interface via a communications link; make the communications link modular, able to establish communication using any number of interchangeable modules for TCP/IP sockets, operating system pipes, RS232, USB, etc. Next, you define the protocol; make the protocol modular, a 'universal language' that describes generic debug primitives, and allow it to be extensible as necessary. Finally, you define those debug primitives and provide a base implementation that can be expanded if required. However, a well-defined set of primitives is unlikely to need expansion for anything but the most exotic architecture configurations.
What does all of this mean? Where does it leave us, the debugger developers? And where does it place the users, the hackers, and the homebrew developers?
It means that the debugger developers can implement an accepted standard (accepted being the keyword) for debugger support within not only emulators, but any kind of virtual machine or interpreted byte code in any kind of program. It could be a simple set of debug primitives (in a static or linked library, for example) added by an emulator author (or emulator extender) that connects to a debugger interface of the user's choice. The interface might be highly specialized for a particular architecture, or it might be very complex and advanced with universal support for many architectures.
This would put a large number of options into the hands of users.
Now let me try to get a more solid description of this idea out there. The number one underlying technology to be assessed to make any of this work is simply the protocol. That means, a formal description of how a target (an emulator, or other program wishing to use debugger functionality) talks to an interface (a separate program designed to give the user direct access to the debug primitives and link them together in ways that provide many very advanced features ... such as stepping backwards in architecture-time).
This would probably be a command reference which supplies things like:
- A description of the architecture (the emulated system, like NES). This description would include the number of CPUs available, the type of the CPUs, endianness, memory maps as accessible by the CPU, memory maps not accessible to the CPU, etc. Basically a complete virtual model of the architecture.
- Debug primitives: breakpoints and stepping functionality; read/write access to the memory maps, cpu registers and statuses, and access to internal hardware registers; interrupt and exception handling; scripted macros with callback functions; essentially all of the basic functions which the interface can use to procedurally create high-level features.
- Extensibility; able to provide expansions to architecture descriptions, debug primitives, and other specialty features.
With such a protocol in place, the interface can do the rest of the high-level work; disassembling, video memory viewing and modification, hex editing, cheat searching and management, etc.
There is currently an ongoing effort to research the possibilities defined in this article. You can read more about the current progress of this project, along with related details, on the Universal Debugger Project page.
I'm hoping this has been verbose enough that you all understand where I am coming from, but not too verbose that I've created confusion or completely went the wrong direction in the discussion.
Bottom line is, I think we only need to agree on one thing: the protocol. If you refuse to believe that, and only want to do your own thing with your own emulator, that's quite alright. But if you want to reap the benefits of interchangeable debugger interfaces [pick your favorite, or just choose the right one for the job at hand] that are platform-independent [can run on any host operating system, even a completely different machine from the target emulator; not at all bound to the target emulator] and potentially architecture-independent [capable of debugging NES, Genesis, PS2, Wii, Java, brainf**k, the custom scripting language in your new game, you name it!] then I say let's work some crazy Voodoo and invent ourselves a standard for modern debugging!
That said, RFC-909, Loader Debugger Protocol looks like a good place to start.