This is a step by step guide to how I re-create the source code from bootable
diskettes. Since it's a tedious process, certainly some improvements can be
made. Please give me feedback or suggestions.
Tools I use:
dsk2img - reads a diskette and creates an image file of it. (part of the flopper project)
img2dsk - inverse of the above utility.
soft ice debugger - good for poking around and watching code execute.
sourcer disassembler - converts binary executables back to x86 assembly source code.
hex editor - handy for quick edits and extracting code/data sections from disk image.
code editor - DOS brief is all I ever needed.
masm/link - compiler to put it all back together.
The first task in converting a game that came on a floppy disk is to extract
the data off the diskette and get it to the hard drive. There are many
utilities to make uncompressed disk image files; I used the one from the
flopper project, dsk2img.
Once the entire disk contents were saved as a disk image file, it was then a
matter of figuring out what sections of the disk were loaded when the game
was booted. I then want to take those sections out of the disk image and
convert them back to source code.
The easiest way to figure out what code is loaded during the game's boot is to watch it happen with a debugger. I load up soft-ice in DOS and set a breakpoint on BIOS service INT 13 (diskette access) and "boot" the machine from s-ice, using an actual floppy disk with the game saved to it. Since my machine doesn't have a 5.25" drive anymore, I use the img2dsk program to re-write the disk image to a 3.5" diskette. Provided there is no copy protection in the game, writing a 360k diskette image to a 1.44MB diskette will work just fine. The game won't know the difference.
As the game boots under soft-ice, I can watch every INT 13 get called and
see what data is being loaded from the disk to memory. The boot loader routine is often jumped to immediately after the
first sector of the diskette is loaded and executed, so I don't have to trace too much code to get to the good stuff. Copy protection may be checked at this stage, so watch out.
I keep a fair amount of notes during this process so I know if I've gotten
all the main code off the diskette:
destination address # of sectors data loaded
50:0 8 sectors "c4 e6 10 23 10 bd a1 90 90 cd 19 57"
50:1000 8 sectors "89 16 48 59 29 de 92 7a 16 aa 97 34"
50:2000 8 sectors "74 27 95 66 18 48 7d 71 7a d7 e2 d0"
50:3000 8 sectors "56 90 16 47 7a 8c 6d dd 1f 6a 8e 1a"
From the INT 13 interface I can tell how many sectors are loaded, from what
section of the diskette and where the data is loaded to in memory.
After each INT 13 load sector is called, I go look at the destination
address and check what data was copied into memory.
Check with ralph brown if you're a little fuzzy on the INT 13 parameters.
After the main code section of the game is loaded, the loader will then jump to it in memory, often with a far jump. I make note of where in the code the 1st jump is; it's not always at the very beginning of the code. After watching the game load itself and jump to the first instruction, I now know what bits of code I need to extract from the disk image.
Once running, the game might load additional sections of data or code off the disk, but I can handle those at another time. Getting the 1st big chunk is a huge step.
Isolate the code from the disk image.
Now that I know how much code is loaded during the main boot load, I need to extract this data from the disk image file so I can convert it back to source code again.
With my notes in hand, I start up a hex editor with the capabilities of
cutting/pasting and copying data to a new file. I think most hex editors
nowadays have these features; I use my own homebrewed program.
I search the disk image file for the strings of data I have in my notes.
Since the game loaded 8 sectors of data each time it read from the disk,
and I know each sector is 512 bytes in length, I know I need to copy 4096
(or 1000h) bytes of data from the disk image and paste that data into a new file. If I'm lucky,
I might even find the next 3 chunks of data all located contiguously in the
disk image, so it's just a matter of copying 4 * 4096 bytes of data to a new
file. If I'm not so lucky, I might have to search the disk image to locate
all 4 data strings and copy the data and copy/paste them to 1 new file with all the data concatenated together.
It's often fairly obvious where the code/data starts and stops in a disk image file. Empty sectors are often
filled with the "F6" character. We don't want any of that data, since that was put there by DOS format.
This portion of the project is very much like figuring out a jigsaw puzzle.
At first glance, it looks like an impossible task, but you'll find a small
piece to work on and those small pieces help you figure out bigger sections,
and slowly things fall into place. Similarly, any subroutine that appears to be accessing memory in the 0B800h
segment is working with the screen. (at least in programs that use graphic
video modes) Any subroutine that accesses port 61h is likely to be a speaker
routine. Sourcer often gives hints in the comments as to what the code is
likely to be doing. As the smaller routines get labelled correctly, it
becomes easier to figure out what the bigger routines do, and eventually
it'll all make sense. Or at least we hope. Step 1: Original code.
Now that the data has been extracted into a new file, we essentially have an executable binary image of the game.
It won't run directly the way it is, but it can be disassembled. It's time for sourcer to work its magic.
┌──────────────────────────────────────────────────────────────────────────────┐
│ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀▀▀▀▀ αßΓπΣσ V COMMUNICATIONS, INC. │
│ ▀▀▀▀▀ ▀ ▀ ▀ ▀ ▀ ▀ ▀▀▀▀ ▀ ±≥≤⌠ µτΦΘΩδ∞φ⌡÷≈°∙¼√ⁿ²■ «» │
│ ▀▀▀▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ε∩ V3.09 │
│ │
│ F1 - help ┌ File format, list(.lst)/source(.asm) Analysis options menu │
│ ┌────────────┴───────────────────────────────────────────────────────────┐ │
│ ┌ Output filename ┌ Header title Xref (cross reference) OFF │
│ ┌┴───────────┐┌──────┴──────────────────────────┐ Press return when done │
│ bcw.asm 000000: 20 00 00 CF 20 00 CF F0 0 Page 3 │
│ xor dl,dl ; Zero register │
│ int 11h ; Put equip bits in ax │
│ mov cx,data_6 ; (6033:A1E4=3C1h) │
│ └┬───┘ └┬─────────────────└┬─────────┘ └──┬─────────────────┘ │
│ └ Segment display └ Word case style └ Label type └ Remarks - all │
│ on/off (decimal) │
│ Target Assembler: MASM-5.1 Press "Q" to Quit │
├───────────────────────────────┬──────────────┬───────────────────────────────┤
│ Input filename bcw.bin │ Label Counts │ Drive g: used for output │
│ Beginning addr 76A7:0000 │ Data 0 │ > 100 Megs available │
│ Ending address 76A7:1E00 │ Sub 0 │ 217K bytes free (using EMS) │
│ Math off uP 8086/8088 │ Loc 0 │ Code zero start Passes 5 │
├───────────────────────────────┴──────────────┴───────────────────────────────┤
│ Select highlighted options desired, and enter "G" to Go begin processing. │
└──────────────────────────────────────────────────────────────────────────────┘
I've not really found the optimal settings for sourcer yet. I do have it
attempt to build a MASM compatible .asm file.
The output from sourcer is often a big mess. It probably won't compile without
some major rework, and it certainly won't run correctly. It's time to start
digging through the code and figure out what's going on.
I like to start with the smallest and most obvious subroutines first. If I
find a subroutine that has INT 10h functions, it's going to be something
involving the video; either a print routine or a set video mode. I rename
sourcer's label of "sub_###" to something more human readable, such as
"setVideoMode" I then change all references of "sub_###" to setVideoMode.
Even if it turns out later that I named a subroutine incorrectly for what
its real purpose is, it's easy to change again later on, as the code evolves into
something more readable.
The next step is to convert all the variables that sourcer has placed
into the code with temporary variable names. When doing this, I like to
keep a 2nd computer running with the actual game loaded under soft-ice.
Until I figure out what a variable's true purpose is, I name them based on
the memory location of that variable in the real game. This way if I see
a variable named "v8022" in my code, I can always reference the physical address 8022
in the real game and see if my varaible has the same value in it. This also
makes for quick cross references between the sourcer code and the original game
code. Once the true purpose of the variable is found (if it's ever found)
I'll rename it to a more human readable name, although I'll keep a note of the
physical address in the comments in case I ever need to investigate its
function in the real game.
Here's a code snippet example of the evolution of a routine in 3 steps.
Icky. I can't make heads or tails of this.
loc_336::
mov dx,0Dh
call sub_39
mov bl,byte ptr ds:[4FAh]
xor bh,bh ; Zero register
add bx,60h
call sub_65
xor dx,dx ; Zero register
call sub_39
mov bx,5Eh
call sub_65
cmp byte ptr ds:[4FBh],0FFh
jne loc_337 ; Jump if not equal
mov bx,5Fh
call sub_65
loc_337::
mov dx,1700h
call sub_39
mov bx,6Ch
cmp byte ptr ds:data_9e,1
je loc_338 ; Jump if equal
inc bx
loc_338::
call sub_65
call sub_10
loc_339::
mov ah,1
int 16h ; Keyboard i/o ah=function 01h
; get status, if zf=0 al=char
jnz loc_340 ; Jump if not zero
call sub_13
jmp short loc_339
loc_340::
mov ah,0
int 16h ; Keyboard i/o ah=function 00h
; get keybd char in al, ah=scan
cmp al,0Dh
je loc_343 ; Jump if equal
cmp al,3
jne loc_341 ; Jump if not equal
xor byte ptr ds:data_9e,21h ; '!'
Start replacing the hard-coded memory references with variables named by their phyisical location in the original code. To do this, I cross reference the code snippet against the real code under soft-ice. Once I do a variable, I just search/replace the rest of the code, which makes this portion of the job a bit easier. This is probably the most frustrating and boring phase of the operation. Unfortunately, this is also the most bug-prone phase. It's easy to make mistakes, such as a typoed search/replace, which will introduce bugs and haunt the final game for many generations of future code.
Step 2: Variables fixed, some call routines figured out.
loc_336::
mov dx,0Dh
call locate
mov bl,byte ptr ds:[v4FA]
xor bh,bh ; Zero register
add bx,60h ; 96
call sub_49 ;
xor dx,dx ; Zero register
call locate
mov bx,5Eh
call sub_49 ;
cmp byte ptr ds:[v4FB],0FFh
jne loc_337 ; Jump if not equal
mov bx,5Fh
call sub_49 ;
loc_337::
mov dx,1700h
call locate
mov bx, 6Ch ;
cmp byte ptr ds:[v0c],1 ;data_9e,1
je loc_338 ; Jump if equal
inc bx
loc_338::
call sub_49
call getakey
loc_339::
mov ah,1
int 16h ; Keyboard i/o ah=function 01h
; get status, if zf=0 al=char
jnz loc_340 ; Jump if not zero
call sub_13
jmp short loc_339
loc_340::
mov ah,0
int 16h ; Keyboard i/o ah=function 00h
; get keybd char in al, ah=scan
cmp al, CR
je loc_343 ; Jump if equal
cmp al,3
jne loc_341 ; Jump if not equal
xor byte ptr ds:[v0c], 21h ;data_9e,21h ; '!'
After replacing the memory references with variables, some things are starting to shape up. I've figured out what one of the calls did (locate) since it was a tiny subroutine and its obvious use of INT 10 was a giveaway.
I can see there is some user input going on because of the INT 16. With the locate subroutine in place, it appears this code prints a lot of something to the screen at various positions, probably text.
Step 3: final code
loc_336:: mov dx,0Dh ; row, col call locate mov bl,byte ptr cs:[skillLevel] ;0-4 xor bh,bh ; Zero register add bx,60h ; 96 call printMsg ; input bx=message # to print xor dx,dx ; Zero register call locate mov bx,5Eh call printMsg ; "skill level..." cmp byte ptr cs:[resumeGame],0FFh jne loc_337 ; Jump if not equal mov bx, 5Fh call printMsg ; "D) resume game" loc_337:: mov dx,1700h call locate mov bx, 6Ch ; monitor type cmp byte ptr cs:[monitorType],1 ;data_9e,1 je loc_338 ; Jump if equal inc bx loc_338:: call printMsg call getakey loc_339:: mov ah,1 int 16h ; Keyboard i/o ah=function 01h ; get status, if zf=0 al=char jnz loc_340 ; Jump if not zero call getRand ; seed the rnd generator while we wait jmp short loc_339 loc_340:: mov ah,0 int 16h ; Keyboard i/o ah=function 00h ; get keybd char in al, ah=scan cmp al, CR je loc_343 ; Jump if equal cmp al, ESCAPE je exitProgram cmp al,3 jne loc_341 ; Jump if not equal xor byte ptr cs:[monitorType], 21h ;data_9e,21h ; '!'
Here, through a bit of guesswork and code deduction, the last of the variables have been renamed and the often-used "sub_49" turns out to simply be a print string function, used everywhere throughout the code.
This section of code prints the current skill level and monitor type to the screen during the main menu. It is also responsible for getting the user input. Pretty straightforward overall, except a neat surprise was that while the routine is waiting for user input, it is continually seeding the random number generator, giving a very random human input variable into the randomness of the game. Nice touch.
At this section in the code, I added a check for the Escape key to allow the user to exit back to DOS.
The code labels have not been changed from loc_xxx:: because the code is fairly readable at this state and retaining the original loc_xxx labels allows me to cross reference code against previous versions in case of bugs that may have crept in.
Now that the code is in the final stages of revision, it's time to make minor
enhancements and additions to the game. I first like to attempt to get full
backwards compatibility for the game; I want to make it run exactly the same
on an IBM XT as it does on a pentium IV 2.5GHz machine, thus the only changes
will be "behind the scenes" to retain the original look and feel. Essentially,
I am attempting to reproduce the game the way it originally should have been
released, had the original author been able to see 20 years or more into the
future of computer technology and adjusted for it.
Speed fixing and throttling:
The biggest issue in playing old games on new computers is speed.
Not only have CPU clock speeds increased, but today's CPUs run smarter and
more optimally than their predecessors. If you took a pentium based CPU and
throttled the clock speed back to the original IBM PC speed of 4.77MHz,
the pentium would still run too fast. Caching and branch prediction tricks
aside, pentium CPUs simply do things in less clock cycles than the early CPUs
did 20 years ago.
The trick is then providing the illusion that the game is running at the same
speed on any computer. Traditionaly, if you wanted to play an old game on a
fast computer, you'd reach for a slowdown program which interrupts the
computer at regular intervals (several times per second) and make it delay a
tiny amount, thus simulating a slower computer. As speeds have increased, the
amount of delay has to increase, which eventually leads to very jerky motion
and unresponsive gameplay, since the computer is doing more waiting than game
processing.
By restoring the source code to the game, strategicaly placed, real-time based delays can be added to restore a game to its original playability without sacrificing responsiveness or video quality. Using delays based in real time, rather than delay loops ensure that the game will run at the proper speed on any cpu, regardless of processor speed.
The most effective area to start throttling a game is the video. I'm currently
using a method that ensures that sprite movements and screen refreshes are
always done in a minimum of 1ms. I start a timer at the start of a video
routine, then at the end of that routine, I make sure at least 1ms has passed
before returning from the procedure.
Since video updates happen often in action games, a well placed delay can have an incredible impact on overall game speed without impacting other aspects of play. Using specific delays like this results in games visually operating at the same speed regardless of CPU speed. The only thing to be careful of is running the game on an original 4.77MHz machine. The added overhead of checking the time before and after a video update may very well take over 1ms in time, which can then result it a game running too slow! A reasonable fix for this is to detect the CPU type at startup, and if running on an 8088 or 8086 processor, simply skip all delays that were added to the original code. Those folks running on an overclocked 8088 or 10MHz 8086 can just use their "turbo" button. ;)
The next hurdle in converting this game was the voice samples played over the PC
speaker. The timing on the playback of these samples is extremely precise, or at least
as precise as one could be back in 1985. Machines running too fast played the samples too
quick; just a fast blurp of sound. If I put too much delay in, the samples draw out like
satan himself was speaking. Because of the precise timing requirements, it was difficult
to find a solution that worked on all speeds of machines, and in fact I'm not sure if I've
even found one yet. Windows complicated matters even further. With proper timings that
worked in a DOS environment, the game's sounds were simply noise under windows 2000 or XP.
I believe this is due to the overhead of the hardware abstraction layer. I'm not certain
if an all encompassing solution can be made for this issue.
A side-solution and game enhancement was to use the soundblaster to play samples from the original game. It was an odd
solution indeed; playing 1 bit samples from a PC speaker on a sound
card capabable of CD quality playback! Anything in the name of video game conservation I guess.
I suppose this is no different than playing a game based on IBM PC hardware on a new Xeon 2.5G.
Adding soundblaster support opens up many possibilities for enhancing the game play in
future revisions. Better voice samples, sound effects, mood music, and much more can
be added to bring gameplay to another level. Of course all of the enhanced features must always have the ability to be
turned off so that the game remains true to its origins.