Kodewerx

Our culture has advanced beyond all that you could possibly comprehend with one hundred percent of your brain.
It is currently Thu Mar 28, 2024 1:50 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 16 posts ] 
Author Message
PostPosted: Sat Dec 19, 2009 9:52 pm 
Offline
Kommunist
Kommunist

Joined: Sat Dec 19, 2009 2:10 pm
Posts: 7
I have implemented RC4 encryption and LZO compression in Zelda 64 based games (Ocarina of Time and Majora's Mask). At the moment, I only have it fully working in the Master Quest debug build (ZELOOTMA), but the code is easily adapted to another game. The hack works by replacing the function call to the compressed file loader with its own routine. I won't go into detail about the underlying method used to patch my code into the ROM, but you can read more about it here.

I'm not sure if LZO is quicker than Nintendo's Yaz0 (especially with RC4 lumped in there), but it certainly provides better compression. Here are some examples.

Standard Ocarina of Time (US, v1.0):
Code:
$ z64repack -rnT -I OOT.z64 -O OOT-repacked.z64 -t 0x00007430 -l 24160 -k 'Kodewerx'
[##################################################] 1510/1510 100.00%
Old size: 31.48M
New size: 30.20M
Saved:    1.28M
Percent:  95.93%
Time:     5.94s


And with Majora's Mask (US):
Code:
$ z64repack -rnT -I MM.z64 -O MM-repacked.z64 -t 0x0001A500 -l 24832 -k 'Kodewerx'
[##################################################] 1552/1552 100.00%
Old size: 30.63M
New size: 29.16M
Saved:    1.47M
Percent:  95.19%
Time:     5.51s


The debug build of Master Quest also compresses quite nicely. However, since it is not compressed at all to begin with, we have to explicitly blacklist some files from compression, as such:
Code:
$ z64repack -nT -I ZELOOTMA.n64 -O ZELOOTMA-repacked.n64 -t 0x00012F70 -l 24512 -k 'Kodewerx' -E 0-27,942,944,946,948,950,952,954,956,958,960,962,964,966,968,970,972,974,976,978,980,982,984,986,988,990,992,994,996,998,1000,1002,1004
[##################################################] 1532/1532 100.00%
Old size: 51.74M
New size: 30.80M
Saved:    20.94M
Percent:  59.52%
Time:     5.44s

Most importantly, LZO compression is faster than the Yaz0 compressors by orders of magnitude.


How to use
"How to use" is a bit misleading. What I'm going to write about here is how to apply the patch I've included. The gameplay is, of course, unaltered, so if you wish to observe the changes I suggest you fire up Nemu's debugger and set a breakpoint on 0x807b80e8. As mentioned earlier, the only game with full support is the Master Quest debug ROM. Acquire this, then download and compile "z64repack".
Code:
$ wget -q -O - 'http://hg.64.vg/files/z64repack-1.0.0.tar.gz' | tar -xzf -
$ cd z64repack-1.0.0
$ mkdir obj; cd obj
$ ../configure --prefix=$HOME/.local
$ make install CFLAGS='-O3'

If you are a Windows user, you will probably have to go into Cygwin or something similar (Interix?).
Use the following command can be used to compress/encrypt the ROM. Do not change the encryption key as the binary contained within the patch is hard-coded to use "Kodewerx" as the key.
Code:
$ z64repack -nT -I ZELOOTMA.n64 -O ZELOOTMA-repacked.n64 -t 0x00012F70 -l 24512 -k 'Kodewerx' -E 0-27,942,944,946,948,950,952,954,956,958,960,962,964,966,968,970,972,974,976,978,980,982,984,986,988,990,992,994,996,998,1000,1002,1004

Finally, apply the PPF patch down at the bottom (z64ce), and everything should be ready to go.

I have not been bothered to test this out on real hardware yet, but both Nemu and Project 64 (if you set the CPU core to interpreter) seem to be able to handle it.


How to build
Uhh. Compile binutils, gcc, and newlib (in that order) with mips as the target. You can find some scripts here. You'll also need some other tools that I've written, most of which you can find in my mercurial repository or files directory. I'll update this post with more info later if there's any interest.


Downloads
  • z64repack
    Source code: hg | tgz
  • z64ce - This build ONLY supports the debug build of Master Quest.
    Binary: ppf | elf
    Source code: hg | tgz


I apologize if this post is badly structured. I'm very tired.


Last edited by Marshall on Tue Feb 23, 2010 6:49 am, edited 1 time in total.

Top
 Profile  
Reply with quote  
PostPosted: Mon Dec 21, 2009 6:31 am 
Offline
Krew (Admin)
Krew (Admin)
User avatar

Joined: Sun Oct 01, 2006 9:26 pm
Posts: 3768
Title: All in a day's work.
Interesting project!

Forgive me for the stupid question, but what overall purpose does RC4 serve? Intuition would tell me that combining encryption with compression would make the compression algorithm worse. Or, if the RC4 implementation is used only for protecting parts of the game, I don't know why you would want to do that.

And if it is some kind of protection, I gently remind you that it's been done before; even with industry standard encryption algorithms, like DES. But what do I know? ;)

Also, a better alternative to Cygwin is Mingw/MSYS. Mingw provides the compiler and binutils (among other things) and MSYS provides a UNIX-like shell/build environment.

Thanks for sharing!

_________________
I have to return some video tapes.

Feed me a stray cat.


Top
 Profile  
Reply with quote  
PostPosted: Mon Dec 21, 2009 10:57 am 
Offline
Kommunist
Kommunist

Joined: Sat Dec 19, 2009 2:10 pm
Posts: 7
Parasyte wrote:
Forgive me for the stupid question, but what overall purpose does RC4 serve? Intuition would tell me that combining encryption with compression would make the compression algorithm worse. Or, if the RC4 implementation is used only for protecting parts of the game, I don't know why you would want to do that.

Yeah, the goal of encryption would be to prevent clueless plagiarizers from stealing your work. Even so, you're right - it would be a good idea to make encryption optional. The files are compressed before encryption, so there is no effect on the yield.

Parasyte wrote:
And if it is some kind of protection, I gently remind you that it's been done before; even with industry standard encryption algorithms, like DES. But what do I know? ;)

I chose RC4 for its simplicity, given that I have zero cryptography experience. Its small size and fast operation are perfect for the N64, where both speed and memory come at a premium.

Parasyte wrote:
Also, a better alternative to Cygwin is Mingw/MSYS. Mingw provides the compiler and binutils (among other things) and MSYS provides a UNIX-like shell/build environment.

I've tried building under MinGW, but there are a few functions missing in Windows (truncate(2), to name one). Windows compatibility is not one of my priorities. The only people who benefit from a Windows port are the YouTube 12-year-olds.

Parasyte wrote:
Thanks for sharing!

My pleasure.


Top
 Profile  
Reply with quote  
PostPosted: Thu Dec 24, 2009 10:58 am 
Offline
Kommunist
Kommunist

Joined: Thu Jul 03, 2008 8:27 pm
Posts: 19
I would definitely vote for Martial as president of Brazil.


Top
 Profile  
Reply with quote  
PostPosted: Fri Dec 25, 2009 11:24 pm 
Offline
Komrade
Komrade
User avatar

Joined: Tue Mar 27, 2007 6:23 pm
Posts: 1354
Location: Mario Raceway, 1509.831, 217.198, -564.429
Title: Mario Kart 64 Hacker
Marshall wrote:
Yeah, the goal of encryption would be to prevent clueless plagiarizers from stealing your work.
*sigh*

What the fuck is wrong with everyone involved with this game? I wasn't surprised to see such foolishness from the n00b-flooded SMW community a few years back, but people capable of hacking N64 should be smart enough to know better. (Incidentally, the SMW community is a great example of the points I'm about to make. Look at the amount of, and reaction to, plagiarism before LM had encryption, after it was introduced, and after RLM was released.)

Let me explain a few things.
  1. The data has to be decrypted to be used. That means you are giving out both the lock and the key. Especially today, when anyone can load up mupen64plus-debug or Nemu and have total control over the system, it takes little more than basic RAM searching skills to extract resources from any game. Sony paid a bunch of professionals ridiculous amounts of money to try to "protect" the video content on Blu-ray, and look how well that's worked - what makes you think one or two hobbyist hackers modifying a game for a well-known, well-emulated system can do better? You cannot give someone content and not let them have it. In other words, you can't have your cake and eat it too.
  2. If your work is good enough to be plagiarized, people are going to recognize it as yours. Even if they don't, simply pointing out where your hack released before used the very same thing is pretty damn solid evidence. It's also hideously easy to hide your name in it somewhere, even just in plain sight late enough in the game that some idiot ripping it off won't ever play through enough to see it.
  3. People who plagiarize are too stupid to make anything worth playing anyway. As a general rule, the more effort you put into DRM (especially when it comes to emulation and ROM hacking), the less you put into actually making the product any good. Oh noes, they copied my graphics (which, in all likelihood, you copied from another game anyway) into their retarded hack consisting of nothing but a bunch of edits and patched-together stolen material! NOBODY FUCKING CARES.
  4. Others will see your work, like it, and be intelligent enough to make something good using it. Those people will in almost every case also be intelligent enough to ask permission and/or give credit. (If not, see point 2.) This benefits everyone: you get more recognition for your work, they don't have to try to hack together a crude replacement for their own version (delaying their release, and likely suffering in quality if e.g. they aren't good at making their own or yours fits just so perfect that any replacement just won't be the same), and players benefit from having more great hacks out and more variety. This is the mentality behind open source, creative commons, etc. Learn it, love it.

Tell you what: send me any N64 hack in which the creators have somehow "encrypted" its resources, and when I have the time, I will develop and release a decryptor/extractor, in source and binary form, along with all of the extracted, decrypted resources, and if I'm in a particularly good mood, a patch to make the hack use unencrypted resources.
(I can't guarantee I'll do it in a timely fashion, or that I'll ever get around to setting up cross-compiling for Windows or x64. x86 Linux binaries at least will be provided. This is just for fun and to mock the geniuses who did it in the first place; I recommend simply avoiding/shunning them rather than even bothering with their likely sub-par work.)

Open-source creations cannot be plagiarized, pirated, or stolen. Instead of worrying about the one or two morons who will try (and fail) to take credit for someone else's work, encourage the people who want to enhance it. Compare: IE vs Firefox, Windows vs Linux, RealPlayer vs mplayer, etc.

To everyone involved in OoT hacking: GROW THE FUCK UP, YOU RETARDS.

_________________
Image 143
HyperNova Software is now live (but may take a few tries to load) currently down; check out my PSP/DS/Game Boy/Windows/Linux homebrew, ROM hacks, and Gameshark codes!


Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: Sat Dec 26, 2009 2:40 am 
Offline
Komrade
Komrade

Joined: Tue Mar 27, 2007 10:18 am
Posts: 1328
Once I realized the futility and non-necessity of keeping Runetard from spreading something I didn't want him to spread for purposes of not ruining my name or the game that the thing supposedly "broke", I also began to realize that knowledge should be free and abundant, and spread willingly at all times.

And so now when I develop something that could potentially ruin an online game, I release it anyway for documentation purposes (though I generally avoid hacking online games now).

In short, HyperHacker is right in every way and there's not really any reason ever to keep information to yourself, although it is good to at least attach your name to it to guide people who need help using the information or utility that implements it to someone who knows what they're talking about.

_________________
Image


Top
 Profile  
Reply with quote  
PostPosted: Sat Dec 26, 2009 10:04 am 
Offline
Krew (Admin)
Krew (Admin)
User avatar

Joined: Sun Oct 01, 2006 9:26 pm
Posts: 3768
Title: All in a day's work.
HyperHacker wrote:
*snip*

Open-source creations cannot be plagiarized, pirated, or stolen.

I was planning to respond to your post with something along the lines of "The open source model is an effective solution to every one of your concerns." But you've already covered that.

Also, notice that both RLM and GCNcrypt (linked above as examples of how to "fix" similar protection schemes) are both released under the terms of the GPL.

_________________
I have to return some video tapes.

Feed me a stray cat.


Top
 Profile  
Reply with quote  
PostPosted: Sat Dec 26, 2009 11:09 am 
Offline
Kommunist
Kommunist

Joined: Sat Dec 19, 2009 2:10 pm
Posts: 7
HyperHacker wrote:
Marshall wrote:
Yeah, the goal of encryption would be to prevent clueless plagiarizers from stealing your work.
*sigh*

What the fuck is wrong with everyone involved with this game? I wasn't surprised to see such foolishness from the n00b-flooded SMW community a few years back, but people capable of hacking N64 should be smart enough to know better. (Incidentally, the SMW community is a great example of the points I'm about to make. Look at the amount of, and reaction to, plagiarism before LM had encryption, after it was introduced, and after RLM was released.)

Let me explain a few things.
  1. The data has to be decrypted to be used. That means you are giving out both the lock and the key. Especially today, when anyone can load up mupen64plus-debug or Nemu and have total control over the system, it takes little more than basic RAM searching skills to extract resources from any game. Sony paid a bunch of professionals ridiculous amounts of money to try to "protect" the video content on Blu-ray, and look how well that's worked - what makes you think one or two hobbyist hackers modifying a game for a well-known, well-emulated system can do better? You cannot give someone content and not let them have it. In other words, you can't have your cake and eat it too.
  2. If your work is good enough to be plagiarized, people are going to recognize it as yours. Even if they don't, simply pointing out where your hack released before used the very same thing is pretty damn solid evidence. It's also hideously easy to hide your name in it somewhere, even just in plain sight late enough in the game that some idiot ripping it off won't ever play through enough to see it.
  3. People who plagiarize are too stupid to make anything worth playing anyway. As a general rule, the more effort you put into DRM (especially when it comes to emulation and ROM hacking), the less you put into actually making the product any good. Oh noes, they copied my graphics (which, in all likelihood, you copied from another game anyway) into their retarded hack consisting of nothing but a bunch of edits and patched-together stolen material! NOBODY FUCKING CARES.
  4. Others will see your work, like it, and be intelligent enough to make something good using it. Those people will in almost every case also be intelligent enough to ask permission and/or give credit. (If not, see point 2.) This benefits everyone: you get more recognition for your work, they don't have to try to hack together a crude replacement for their own version (delaying their release, and likely suffering in quality if e.g. they aren't good at making their own or yours fits just so perfect that any replacement just won't be the same), and players benefit from having more great hacks out and more variety. This is the mentality behind open source, creative commons, etc. Learn it, love it.

Tell you what: send me any N64 hack in which the creators have somehow "encrypted" its resources, and when I have the time, I will develop and release a decryptor/extractor, in source and binary form, along with all of the extracted, decrypted resources, and if I'm in a particularly good mood, a patch to make the hack use unencrypted resources.
(I can't guarantee I'll do it in a timely fashion, or that I'll ever get around to setting up cross-compiling for Windows or x64. x86 Linux binaries at least will be provided. This is just for fun and to mock the geniuses who did it in the first place; I recommend simply avoiding/shunning them rather than even bothering with their likely sub-par work.)

Open-source creations cannot be plagiarized, pirated, or stolen. Instead of worrying about the one or two morons who will try (and fail) to take credit for someone else's work, encourage the people who want to enhance it. Compare: IE vs Firefox, Windows vs Linux, RealPlayer vs mplayer, etc.

To everyone involved in OoT hacking: GROW THE FUCK UP, YOU RETARDS.

Cool story bro. For the record, content producers in OoT hacking are grown. It's just the consumers that are - and will always be - 12 year olds who post our work on YouTube and get 100K views (without giving credit). Don't take this so seriously - it's just a tech demo.


Top
 Profile  
Reply with quote  
PostPosted: Tue Dec 29, 2009 2:41 am 
Offline
Komrade
Komrade
User avatar

Joined: Tue Mar 27, 2007 6:23 pm
Posts: 1354
Location: Mario Raceway, 1509.831, 217.198, -564.429
Title: Mario Kart 64 Hacker
Can't be arsed to read a few paragraphs? Fine, here's a summary: encryption in a hack is pointless and only fools use it.

The improved compression is certainly useful, and it's interesting from a technical standpoint that you managed to implement this, but there is no good reason to ever use it.
Marshall wrote:
For the record, content producers in OoT hacking are grown. It's just the consumers that are - and will always be - 12 year olds who post our work on YouTube and get 100K views (without giving credit).
This smells of contradiction. In any case, all I've seen out of OoT hacking is drama, drama, drama. On the off chance any useful information is posted, the poster is immediately accused of stealing it and a flamewar ensues. Competition, fake screenshots, and accusations far outnumber anything useful.

While I can see the appeal of implementing an encryption algorithm in a hack for fun, actually using it is just ridiculous, let alone creating a tool to apply it automatically.

Who cares about some 'tard on Youtube? Hell maybe you can report them for it. Youtube just loves deleting things. What's encryption going to do to stop someone posting a video of your hack anyway?

On a more positive note, I'm curious why you needed to hook into the boot process at all, and how you get Gnu AS not to botch the assembly when targeting MIPS.

_________________
Image 143
HyperNova Software is now live (but may take a few tries to load) currently down; check out my PSP/DS/Game Boy/Windows/Linux homebrew, ROM hacks, and Gameshark codes!


Top
 Profile  
Reply with quote  
PostPosted: Tue Dec 29, 2009 5:23 am 
Offline
Kommunist
Kommunist

Joined: Sat Dec 19, 2009 2:10 pm
Posts: 7
HyperHacker wrote:
Can't be arsed to read a few paragraphs?

Correct.

HyperHacker wrote:
Fine, here's a summary: encryption in a hack is pointless and only fools use it.

I concede, but I will not remove this utility or the source code to the hack.

HyperHacker wrote:
The improved compression is certainly useful, and it's interesting from a technical standpoint that you managed to implement this,

Yes.

HyperHacker wrote:
but there is no good reason to ever use it.

Well, fucking around with zeldafags who have wet dreams about prototype ROMs is a good reason for anything.

HyperHacker wrote:
Marshall wrote:
For the record, content producers in OoT hacking are grown. It's just the consumers that are - and will always be - 12 year olds who post our work on YouTube and get 100K views (without giving credit).
This smells of contradiction. In any case, all I've seen out of OoT hacking is drama, drama, drama. On the off chance any useful information is posted, the poster is immediately accused of stealing it and a flamewar ensues. Competition, fake screenshots, and accusations far outnumber anything useful.

You have the 12 year olds to blame for all of those things.

HyperHacker wrote:
While I can see the appeal of implementing an encryption algorithm in a hack for fun, actually using it is just ridiculous, let alone creating a tool to apply it automatically.

That sounds like a personal problem to me.

HyperHacker wrote:
Who cares about some 'tard on Youtube? Hell maybe you can report them for it. Youtube just loves deleting things. What's encryption going to do to stop someone posting a video of your hack anyway?

Yes.

HyperHacker wrote:
On a more positive note, I'm curious why you needed to hook into the boot process at all,

Yeah, the hack is fully possible without hooking into the boot process. The reason I'm doing that is because it's the quickest way to get code running in the ROM. It's a part of my method for injecting code into N64 games.

HyperHacker wrote:
and how you get Gnu AS not to botch the assembly when targeting MIPS.

How can you not? Could you give me an example? One thing you have to keep in mind is that you should not pass assembly source files directly to GNU as, but rather through GCC. Also use the ".S" (capital 's') extension for assembly source files. The makefile takes care of everything anyway.

Pro-tip: Linux or BSD MIPS sources are handy references for coding clean assembly and utilizing the assembler to its full potential.


Top
 Profile  
Reply with quote  
PostPosted: Wed Feb 17, 2010 12:33 am 
Offline
Komrade
Komrade
User avatar

Joined: Tue Mar 27, 2007 6:23 pm
Posts: 1354
Location: Mario Raceway, 1509.831, 217.198, -564.429
Title: Mario Kart 64 Hacker
Marshall wrote:
HyperHacker wrote:
Who cares about some 'tard on Youtube? Hell maybe you can report them for it. Youtube just loves deleting things. What's encryption going to do to stop someone posting a video of your hack anyway?

Yes.
...that's not a yes/no question.

Anyway, example of GAS botching its input:
Code:
LUI $k1, 0x8034
LB $t0, 0xB21D($k1)

BNEL $t0, $zero, next
ADDIU $t0, $t0, 0x0001
BEQZ $a0, next
ADDIU $t0, $t0, 0xFFFF
LUI $t3, 0x5555

next: ADDIU $zero, $zero, 0x1234
becomes:
Code:
 0: 3c1b8034 lui k1,0x8034
4: 3c080001 lui t0,0x1
8: 011b4021 addu t0,t0,k1 ;<-- use of K1 has been changed, increasing number of instructions
c: 8108b21d lb t0,-19939(t0)
10: 55000006 bnezl t0,2c
14: 00000000 nop ;<-- NOP added to delay slot breaks code
18: 25080001 addiu t0,t0,1
1c: 10800003 beqz a0,2c
20: 00000000 nop ;<-- NOP added to delay slot breaks code
24: 2508ffff addiu t0,t0,-1
28: 3c0b5555 lui t3,0x5555

0000002c :
2c: 24001234 li zero,4660
Feeding it through GCC may generate code that doesn't get its delay slots broken, but it's not going to be as efficient. (Plus who wants to try to write entirely in assembly inlined in a C program?)
Command line: mips-unknown-linux-gnu-as -EB -mabi=32 -march=vr4300 -mtune=vr4300 -mno-shared -K --warn $1.asm -o $1.elf
"mips-unknown-linux" was the only one I could get working... don't remember the precise detail as that was a while ago.


BTW, how fast does your routine decompress compared to Nintendo's? The improved compression, I could probably use in my hacks, if it's not too slow. (Mario Kart has a LOT of compressed files per level, so any reduction in decompression speed is likely to be noticed.)

_________________
Image 143
HyperNova Software is now live (but may take a few tries to load) currently down; check out my PSP/DS/Game Boy/Windows/Linux homebrew, ROM hacks, and Gameshark codes!


Top
 Profile  
Reply with quote  
PostPosted: Mon Feb 22, 2010 7:44 am 
Offline
Kommunist
Kommunist

Joined: Sat Dec 19, 2009 2:10 pm
Posts: 7
Sure is extremely late replies in here (you - not me).

HyperHacker wrote:
Marshall wrote:
HyperHacker wrote:
Who cares about some 'tard on Youtube? Hell maybe you can report them for it. Youtube just loves deleting things. What's encryption going to do to stop someone posting a video of your hack anyway?

Yes.
...that's not a yes/no question.

Image

HyperHacker wrote:
Anyway, example of GAS botching its input:
Code:
LUI $k1, 0x8034
LB $t0, 0xB21D($k1)

BNEL $t0, $zero, next
ADDIU $t0, $t0, 0x0001
BEQZ $a0, next
ADDIU $t0, $t0, 0xFFFF
LUI $t3, 0x5555

next: ADDIU $zero, $zero, 0x1234
becomes:
Code:
 0: 3c1b8034 lui k1,0x8034
4: 3c080001 lui t0,0x1
8: 011b4021 addu t0,t0,k1 ;<-- use of K1 has been changed, increasing number of instructions
c: 8108b21d lb t0,-19939(t0)
10: 55000006 bnezl t0,2c
14: 00000000 nop ;<-- NOP added to delay slot breaks code
18: 25080001 addiu t0,t0,1
1c: 10800003 beqz a0,2c
20: 00000000 nop ;<-- NOP added to delay slot breaks code
24: 2508ffff addiu t0,t0,-1
28: 3c0b5555 lui t3,0x5555

0000002c :
2c: 24001234 li zero,4660
Feeding it through GCC may generate code that doesn't get its delay slots broken, but it's not going to be as efficient. (Plus who wants to try to write entirely in assembly inlined in a C program?)

Your assembly leaves much to be desired in the aesthetics department.
  • Don't use ALL CAPS. It's not 1950 anymore.
  • Don't use hex when it isn't necessary. When in the world is 0xFFFF more appropriate than -1?
  • Don't use k0/k1; if you want a globally unused register try $gp or the frame pointer ($fp/$s8)
  • ... or just follow the ABI and avoid getting a headache.
Furthermore, GNU as is most certainly not botching its input. It is not a verbatim assembler. Macros and reordering are on by default. You can disable them by using ".set noreorder". Here is an example of good assembly (or disassembly, in this case) with no reordering and no macros.

And just who says an empty delay slot is inefficient? You will waste 4 bytes, yes, but otherwise you may STALL THE PIPELINE!!!
Image
^ This is basically what happens except you are not Bill Kaulitz and instead of burning your tongue the pipeline stalls.

HyperHacker wrote:
Command line: mips-unknown-linux-gnu-as -EB -mabi=32 -march=vr4300 -mtune=vr4300 -mno-shared -K --warn $1.asm -o $1.elf
"mips-unknown-linux" was the only one I could get working... don't remember the precise detail as that was a while ago.

Try this (I've probably typo'd something but I'm sure you can figure it out):
Code:
wget -O - http://ftp.gnu.org/gnu/binutils/binutils-2.20.tar.gz | tar -xzf -
wget -O - http://ftp.gnu.org/gnu/gcc/gcc-4.4.3/gcc-core-4.4.3.tar.bz2 | tar -xjf -
mkdir binutils-obj gcc-obj
cd binutils-obj
../binutils-2.20/configure --prefix=/usr/mips --target=mips-elf
make
sudo make install
cd ..
cd gcc-obj
export PATH="$PATH:/usr/mips/bin"  # You should add this to your profile
# You really don't need the bulk of the below configure options, but whatever. Doesn't hurt!
../gcc-4.4.3/configure --prefix=/usr/mips --target=mips-elf --disable-werror --enable-languages=c --disable-threads --disable-multilib --disable-libssp --with-newlib --with-abi=32 --with-arch=vr4300 --with-tune=vr4300 --with-float=hard
make
sudo make install

It's also not a good idea to run GNU as DIRECTLY on the assembly files. Don't ask me why, I won't have an answer for you. Just use mips-elf-gcc to compile them (Makefiles do this automatically).
And don't forget to use a capital S as the file extension.
HyperHacker wrote:
BTW, how fast does your routine decompress compared to Nintendo's? The improved compression, I could probably use in my hacks, if it's not too slow. (Mario Kart has a LOT of compressed files per level, so any reduction in decompression speed is likely to be noticed.)

I have not done any benchmarks. I'm going to go ahead and assume that LZO is practically faster than whatever Mario Kart 64 uses if only because Nintendo's C compiler (mipsPRO, actually - you would think SGI could put together a better solution) was about as good an optimizer as a dildo. LZO has undergone extensive development for circa 14 years, though, so the algorithm may very well be faster.

Thank you.
Image


EDIT:
I just read through the thread again and realized that you disregarded a lot of what I wrote earlier (how to build GCC, how to compile assembly source files with it). What the fuck, man?


Top
 Profile  
Reply with quote  
PostPosted: Tue Mar 09, 2010 2:49 am 
Offline
Komrade
Komrade
User avatar

Joined: Tue Mar 27, 2007 6:23 pm
Posts: 1354
Location: Mario Raceway, 1509.831, 217.198, -564.429
Title: Mario Kart 64 Hacker
Marshall wrote:
Sure is extremely late replies in here (you - not me).
Yes, I don't read this forum often. Is there a point to this statement?
Quote:
  • Don't use ALL CAPS. It's not 1950 anymore.
  • Don't use hex when it isn't necessary. When in the world is 0xFFFF more appropriate than -1?
  • Don't use k0/k1; if you want a globally unused register try $gp or the frame pointer ($fp/$s8)
  • ... or just follow the ABI and avoid getting a headache.
[...]
And just who says an empty delay slot is inefficient? You will waste 4 bytes, yes, but otherwise you may [annoying format]STALL THE PIPELINE!!![/annoying format]
I guess you're new to N64 hacking? When I started quite a few years ago, it was a pretty small community of Windows users. There was no gcc, ABI, etc, just crude, limited assemblers people put together in languages like VB6 and Delphi. A lot of ROM hacking communities are still like this. (Look at the piece of shit assemblers SNES hackers are using! They barely even work!) These tools worked, but tended to have such limitations as not supporting decimal constants (you'd write LUI $A0, 8011 and it'd be read as 0x8011), and I believe some of them wouldn't accept lowercase either. Old habits die hard, and since I'm generally the only one reading this code and I have no trouble reading it, there's been no reason to change them.
Personally, I think MIPS/hex in lowercase look ugly, mostly due to the frequency of small letters next to large numbers and '$' characters. It's just a matter of preference.

As for ABI, k0, pipelining, etc, this is all irrelevant in most of what I'd be doing. Size is far more important than speed. These would be short hacks, maybe up to 50 lines, patched into a subroutine somewhere, distributed in Gameshark code format, that generally would run once every time a button is pressed or a certain event is triggered. Once per frame in the worst cases. Adding a few extra cycles to shave off a few bytes is basically always a good thing in this scenario.
I do distribute the source of my hacks, but the only people who are ever interested in it are people who do similar hacks and have no trouble reading a few lines of raw, capitalized, assembly with all-hex constants.
Using $k0 can lead to all sorts of problems in the right situation, sure. These are situations that will never arise in these simple cases. It's been one of the first pieces of advice given to new hackers in these communities for years: as far as you're concerned, $k0 is unused, so when you need a free register, it should be your first choice. If you ever dig deep enough to come across a situation where it's not safe to use, you'll probably know well enough what you're doing to figure it out.

It'd be different if I were writing a large, complex hack such as this. That would call for a real assembler, nicer formatting, concern about speed and ABI, and all that good stuff. For a small "if(x) y=1;" subroutine patched into Mario's "is he dead yet" conditional, all that matters is A) it works and B) it's short.
Quote:
It is not a verbatim assembler. Macros and reordering are on by default. You can disable them by using ".set noreorder".
Thank you.
Quote:
I just read through the thread again and realized that you disregarded a lot of what I wrote earlier (how to build GCC, how to compile assembly source files with it). What the fuck, man?
The only such instructions I can see are:
Quote:
Uhh. Compile binutils, gcc, and newlib (in that order) with mips as the target.
and some links to sources for your project. Was I supposed to dig through your repository for a script or something?

_________________
Image 143
HyperNova Software is now live (but may take a few tries to load) currently down; check out my PSP/DS/Game Boy/Windows/Linux homebrew, ROM hacks, and Gameshark codes!


Top
 Profile  
Reply with quote  
PostPosted: Wed Mar 10, 2010 3:17 am 
Offline
Kommunist
Kommunist

Joined: Sat Dec 19, 2009 2:10 pm
Posts: 7
HyperHacker wrote:
Marshall wrote:
Sure is extremely late replies in here (you - not me).
Yes, I don't read this forum often. Is there a point to this statement?
Quote:
  • Don't use ALL CAPS. It's not 1950 anymore.
  • Don't use hex when it isn't necessary. When in the world is 0xFFFF more appropriate than -1?
  • Don't use k0/k1; if you want a globally unused register try $gp or the frame pointer ($fp/$s8)
  • ... or just follow the ABI and avoid getting a headache.
[...]
And just who says an empty delay slot is inefficient? You will waste 4 bytes, yes, but otherwise you may [annoying format]STALL THE PIPELINE!!![/annoying format]
I guess you're new to N64 hacking? When I started quite a few years ago, it was a pretty small community of Windows users. There was no gcc, ABI, etc, just crude, limited assemblers people put together in languages like VB6 and Delphi.

All of the tools I utilize were around before these Windows-using, VB6-loving teenagers started reinventing the wheel. People just don't know how to use them. They not only understand little about software development in general, they also know nothing about software development on embedded platforms. It's a shame. Also the o32 ABI (which the N64 utilizes) has been around since the early 90's, and was a de-facto standard before then.

HyperHacker wrote:
Personally, I think MIPS/hex in lowercase look ugly, mostly due to the frequency of small letters next to large numbers and '$' characters. It's just a matter of preference.

Yes - your credibility as a good assembler coder disappears the minute you start using archaic conventions like this.

HyperHacker wrote:
As for ABI, k0, pipelining, etc, this is all irrelevant in most of what I'd be doing. Size is far more important than speed. These would be short hacks, maybe up to 50 lines, patched into a subroutine somewhere, distributed in Gameshark code format, that generally would run once every time a button is pressed or a certain event is triggered. Once per frame in the worst cases. Adding a few extra cycles to shave off a few bytes is basically always a good thing in this scenario.
. . .
Using $k0 can lead to all sorts of problems in the right situation, sure. These are situations that will never arise in these simple cases. It's been one of the first pieces of advice given to new hackers in these communities for years: as far as you're concerned, $k0 is unused, so when you need a free register, it should be your first choice. If you ever dig deep enough to come across a situation where it's not safe to use, you'll probably know well enough what you're doing to figure it out.

Pipelining - I guess, but k0/k1? Definitely not irrelevant. Every single time an interrupt is triggered k0 and k1 will change. You could always disable interrupts while your cute little hack runs, but you have size concerns so a few mfc0's, mtc0's and or's are probably not what you want. Hence, you should use the data pointer or the frame pointer - both of which are unused (well, if you happen to have a game linked with the debug N64 OS libraries the frame pointer will be in use; otherwise, no).

HyperHacker wrote:
It'd be different if I were writing a large, complex hack such as this. That would call for a real assembler, nicer formatting, concern about speed and ABI, and all that good stuff. For a small "if(x) y=1;" subroutine patched into Mario's "is he dead yet" conditional, all that matters is A) it works and B) it's short.

Yes, you are correct. While we're on the subject, a Gameshark is a fucking AWFUL way of modding N64 games - especially on emulators. Thankfully, the dark era of N64 emulation (LOL CLOSED SOURCE MICROSOFT VISUAL C DELPHI DEBUGGER DONATIONS PLEASE) is behind us and Mupen64plus is shaping up to be a great emulator. I haven't checked v2, but in v1.5 there was a debugger - a little shoddy, but it was on par with Nemu. Someone (me? :o) will eventually improve the debugger and add support for patching games in real time.

HyperHacker wrote:
Quote:
I just read through the thread again and realized that you disregarded a lot of what I wrote earlier (how to build GCC, how to compile assembly source files with it). What the fuck, man?
The only such instructions I can see are:
Quote:
Uhh. Compile binutils, gcc, and newlib (in that order) with mips as the target.
and some links to sources for your project. Was I supposed to dig through your repository for a script or something?

I linked to a directory where the build scripts were in my first post. Not going to link them again because I described the entire process in my previous post in the code tags.

The point of ABI conformance is so that you can cleanly modify and hook into the game's execution. It saves you from having to tediously go through and check which registers are unused in order to inject some obfuscated snippet.


Top
 Profile  
Reply with quote  
PostPosted: Tue Apr 06, 2010 1:10 pm 
Offline
Kommunist
Kommunist
User avatar

Joined: Fri Oct 27, 2006 8:58 am
Posts: 19
Location: The glorious fishbowl of souls
Marshall wrote:
All of the tools I utilize were around before these Windows-using, VB6-loving teenagers started reinventing the wheel.

The old tools are certainly the best tools, or at least the most useful tools, when you can find the blasted things. Everyone acts like it's something new, just because all the old sites are down and nothing is in one place any longer.

Marshall wrote:
(well, if you happen to have a game linked with the debug N64 OS libraries the frame pointer will be in use; otherwise, no).

Little known fact, but Rare got 50% off for keyboards without a delete key. NGEE still looks for, reads from, and writes to its Indy board if you ask it nicely. Bloody jerks.
Your best bet is to look at what is expendable, which is almost always AT,V0, or one of the TX's. If you capture a JAL, you can bet one or the other is expendable.

Marshall wrote:
...a Gameshark is a fucking AWFUL way of modding N64 games - especially on emulators

You'd have to be psychotic writing any sort of ASM hack with GS codes on an emu--of which I am the most guilty. The only real application would be for console, only because of availablity, and now that nobody has parallel ports that's out the window as well. You'd think by now somebody would have made a nice, cheap, USB + flashdrive device. No, they decide to charge out the nose for a bloody GBA cart-backed piece of crap. WTF?

Or for that matter, you'd think by now somebody would support hotpatching N64 files, concidering even the backup devices could manage that. Seriously, even as something as utterly useless as .ips would be okay.


Seriously though, neat idea to replace Yaz0, and not just with Yay0 like another project I saw someplace. I may just use the same concept to replace the LZH library in Ogre64 with anything that isn't that. Decompression times are evil.


Top
 Profile  
Reply with quote  
PostPosted: Tue Apr 06, 2010 4:48 pm 
Offline
Krew (Admin)
Krew (Admin)
User avatar

Joined: Sun Oct 01, 2006 9:26 pm
Posts: 3768
Title: All in a day's work.
Zoinkity wrote:
Marshall wrote:
(well, if you happen to have a game linked with the debug N64 OS libraries the frame pointer will be in use; otherwise, no).

Little known fact, but Rare got 50% off for keyboards without a delete key. NGEE still looks for, reads from, and writes to its Indy board if you ask it nicely. Bloody jerks.
Your best bet is to look at what is expendable, which is almost always AT,V0, or one of the TX's. If you capture a JAL, you can bet one or the other is expendable.

Actually, your best bet is following the MIPS ABI.

The following statements assume you've properly (safely) hooked the executable, by replacing an existing JAL/JALR, or JR $ra. (Note that breaking out in the middle of a subroutine[/function] is OK if you're very careful about how you use registers. In this case, the following list doesn't necessarily apply.)

Long story, short:
  • $a0-3, $t0-9, and $v0-1 registers are volatile; use them at will:
    • $a0-3 are used for function arguments; avoid these if you've hooked a JAL/JALR ... unless you're sure what arguments the original function expects
    • $t0-9 are used for temporary storage; use these! There are ten of them! Shibby!
    • $v0-1 are used for return values; might not want to mess with these if you've hooked a JR $ra ... unless you're sure about what that function returns
  • $s0-7, $sp, $gp, $ra, [and sometimes $fp] are non-volatile. These should always be pushed to the stack before use... Except for $sp of course.
  • $at is reserved for the assembler; you can almost use this for whatever you want ... it depends entirely on how the original assembler used it when the game was compiled
  • $k0-1 are reserved for operating system use; avoid these unless you're writing an exception handler

Zoinkity wrote:
You'd have to be psychotic writing any sort of ASM hack with GS codes on an emu--of which I am the most guilty. The only real application would be for console, only because of availablity, and now that nobody has parallel ports that's out the window as well. You'd think by now somebody would have made a nice, cheap, USB + flashdrive device. No, they decide to charge out the nose for a bloody GBA cart-backed piece of crap. WTF?

Chill out, some of us are actually working on it. [source]

Zoinkity wrote:
Or for that matter, you'd think by now somebody would support hotpatching N64 files, concidering even the backup devices could manage that. Seriously, even as something as utterly useless as .ips would be okay.

I kindly remind you that mupen64plus is open source... Have at it! :D

Marshall wrote:
Image

He has a pretty face.

_________________
I have to return some video tapes.

Feed me a stray cat.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 117 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group