UN-NEW: This Is Not New

Lately I've been playing with the inner workings of BASIC, and it struck me to try to revive a program I accidentally killed. This is certainly no news flash, and my solution is probably not the shortest, but hopefully it's interesting.

I came up with two BASIC one-liners that can get a lost program back in pristine condition, from immediate mode. Here I explain my solution and get into some of the details of how BASIC works, including some handy tricks.

I have been able to get "newed" (compare and contrast: "nude") programs back in the past, and I don't think I ever managed to have a 3rd-party solution at hand when I needed it.  It was a seldom enough occurrance that I never really mastered any technique, so it was always a 30-minute (or more) period of pouring over the memory map in my PRG and doing dozens of manual peeks until I managed to get it right or botch the whole thing in the process.  I had about 60% or better success rate in getting something back (a good share of those other times I just decided it wasn't worth the time and threw in the towel), but actually ending up with an undamaged version of what I lost was a rarer occasion.  I usually guessed at the program end, because going through all those links one-by-one was about as fun as paging through a textbook to erase all the pencil markings.  Often, before I started the recovery I managed to do something that wiped out part of the program, and I usually settled for something that would at least list, so I could get back recent code changes one screen at a time.  (Can you guess what technique I used?)

Well, lately I've been learning all the little nuances of BASIC program and variable storage as I've worked on various things.  So at some point recently I found myself with enough knowledge rattling around in my head that when I typed NEW by accident I didn't hesititate to jump in and get it back.  Turns out it was a very large program, and I decided to come up with some automated way to get any program back.  I had recently been introduced to someone's quest to write a Tetris program in BASIC that would fit inside one screen (it's awesome... see Really Tiny BASIC Tetris on Lemon64), so I wanted to come up with a one-liner solution.  Well, I did not succeed... (yet).  But I did come up with a one-liner that will get your program to list, and a second one-liner that finishes the job.  (I have yet to search for others' solutions... I will probably feel rather up-staged when I do....)

Alright, on to the meat of the topic already!  Here's the first "one-liner" (defined as... fits in 80 column virtual line limit).  Keyword shortcuts have to be used to make it fit.

Po46,159:Cl:A=2053:FoL=0TO0:B=Pe(A):A=A+
1:L=B>0:Ne:Po2049,AAND255:Po2050,A/256

First, what will this do?  This simply gets your program to list again, by restoring the first BASIC line link to the correct value.  All the background necessary to understand how this works is beyond what I want to cover in this post, but I'll go ahead and list the high-level steps in the un-new process.

1. Move variables to a safe location so they don't obliterate your precious code
2. Find the end of the first line in the lost program
3. Repair the first link in the program so that it points to the second line of code
4. Find the end of the program by traversing the line links
5. Set "start of variables" to 1 byte past the final link

Note that this first snippet only handles steps 1 thru 3.  The second one-liner below takes care of the rest.  I repeat the code here w/o the shortcuts, for easier reference.

POKE46,159:CLR:A=2053:FORL=0TO0:B=PEEK(A):A=A+
1:L=B>0:NEXT:POKE2049,AAND255:POKE2050,A/256: REM SAME CODE AS ABOVE--BUT WITH FULL KEYWORDS

Next it's helpful to consider all of the challenges you face in performing an un-new:

• It would be nice if you could write an un-new program in BASIC, and just load and run it.  But since the C64 is not designed to handle multiple programs at once, this is ruled out.  You could do some tricks to move BASIC start temporarily, but that starts getting a bit messy.
• Without the option of writing a BASIC program proper, you are stuck with using immediate mode commands.  To automate the process you need control statements and IFs, but your options are limited w/o numbered lines.
• Using variables in immediate mode can wipe out your code.  Harmless as little variables may seem, after performing a NEW, the start of variables is poised like an assassin with your program in the cross-hairs.
• We would like a solution that is as quick and painless as possible.

So, the stage is now set to explain some of the tricks I resorted to.  And the tricks were rather necessary, especially to keep it inside 2 80-char lines.  The first POKE moves the start of variables right up against the top of basic memory to protect the program.  The exact location is not important, so it is only necessary to change the high-byte of the address, and the CLR instruction sets all the other variable pointers for us.

Next (no pun intended...), we need a loop to find the first end-of-line marker in the program. Remember, we can't use GOTO and IF startments.  (Yes, technically you can use an if statement, but BASIC 2.0 has no else statement.... discuss...)  And we don't know yet where the loop should stop, or else we wouldn't need the loop now would we?  So I take advantage of  the fact that FOR loops intrinsically perform the function of IF and GOTO statements--you just have to play by its rules.  So I set up a "dummy" loop to do what in C would be a do {...} loop, or in Pascal, a REPEAT ... UNTIL structure.  This technique relies on very specific details of how Commodore BASIC 2.0 evaluates the counter variable.  In short, we can assert that the loop will not end until the variable L  has a value of 0 or more when the NEXT startement is encountered.  Here's where another trick comes in to play...  There are no restrictions on assigning to the loop index variable... so we can manipulate L so that its value stays below zero until the end-of-line marker (a 0 byte) is encountered.  The value assigned to L is a boolean expression that produces a true (numerical value -1) if the byte is greater than 0, and a false (numerical value 0) when we do see a 0 byte.  So, by assigning L a value just before the NEXT statement, once we find the 0-byte, execution will continue to the statement that comes after the NEXT.  But wait, we now need to point the first link to the address of the following link, and not to the 0-byte marker.  By incrementing A immediately after taking the PEEK(A), it is already conveniently pointing one byte past the 0-byte, which is what we need.  And the rest is just word / byte pointer math.

At this point, if you execute LIST, it should show you the whole program, perfectly intact.  However, if you saved it as is, you would have a very large program (152 blocks!) and you would run into problems when you tried to edit (or run) the program.  So the un-new is not complete without a bit more fix-up.  Here is part two (and you guessed it, there is more trickery afoot):

FoA=ATO0STe0:B=A+2:A=Pe(A)+Pe(A+1)*256:N
e:Po2,B/256:Po45,BAND255:Po46,Pe(2):CLR

Again, the more readable form:

FORA=ATO0STEP0:B=A+2:A=PEEK(A)+PEEK(A+1)*256:NE
XT:POKE2,B/256:PEEK45,BAND255:POKE46,PEEK(2):CLR

This time, the FOR loop almost looks like a normal, straight-forward loop.  The index variable, A, is actually being used like a real loop-control variable.  Hmmm... I guess that's about where normal ends, tho.  How is the loop abnormal?  Let me count the ways... 1) initial value;  2) end value;  3) step value.  Guess that covers it.  :-)  After executing part 1 of the un-new, variable A just happens to already be pointing at the 2nd link in the program, so I am able to use its current value as the starting point.  In this loop we want to hop from link to link, not go 1 byte at a time, so the index variable get's set manually rather than by the NEXT statement.  (More on the STEP value below...).  What's going on with the loop limit of 0??  Well, the way that we know we've actually reached the end of the program is that the link to the next line contains a value of 0 (i.e., there is no next line...).  By setting the loop limit to 0, the NEXT statement simply falls through once A=0, and no boolean trick is needed.

Now, about that STEP value.  Steppnig by 0 (normally, a prescription for infinite loops) performs 2 vital functions here.  The obvious one is that it prevents the NEXT statement from altering the value so tenderly assigned to A just moments before.  The more subtle (and actually, more important) benefit is that it allows the loop limit to be less than the initial value.  In this version of BASIC (as in most versions of BASIC Classic [TM] ),  all for loops are executed at least once, even if the range is Nil.  For example [really, pun not intentional, but emphasis was added intentionally after the fact...  hehe...] this simple loop-- FOR A=1 TO 0: PRINT A: NEXT -- will result in the value 1 being printed, even though A starts out greater than the limit of 0.  Being the interpreted language that it is, the matter of getting out of the loop is not even considered until the interpreter stumbles upon a NEXT statement.  At that point, the counter is incremented, and if the new value is less than or equal to the limit (in this case, 0), it will branch back to the first statement after the FOR.

But, of course, that's not the whole story.  FOR allows you to step through the loop range in increments other than the default of (positive) 1.  In fact, the STEP value can be anything, which could mean fractional (non-integer), negative (stepping backwards), and as you see here, even ZERO.  If the NEXT statement always worked exactly as I just described, then it would be impossible to make loops that count down, since the counter is intended to always be greater than the limit.  So the check performed by the NEXT statement must vary based on the sign of the step value.  For positive steps, the loop ends when the index (after incrementing) is greater than the limit.  For negative increments, the loop exits when the index becomes  less than the limit.  And, continuing that pattern, the exit criteria for STEP 0 is "equals the limit".  Fortunately, the designers of this BASIC decided to allow STEP 0 to be used, and chose the most logical behavior, in my opinion.  And what this means for the un-new loop is that the loop will always repeat until A is exactly 0 when NEXT executes.  (For fun, play with these two loop variations... FOR I=1 TO 5 STEP 0  vs.  FOR I=1 TO 1 STEP 0 )

Oh, but wait, there's still more... Without the B=A+2 statement, once the loop had located the end of the program, it would have, in the process of reading the special end-of-program link value (zero), forgotten where the end of the program actually was.  The "+2" reflects that the start-of-variables pointer (address 45/46) needs to point to the byte after the last link, not at the link itself.

Last, but not least, note that the high byte of the address value, that goes in address 46, is first stored at address 2.  This is necessary because once we change either byte of the start-of-variables pointer (note the name of the pointer...), any existing variables can no longer be located.  Kinda like sitting on the wrong side of the limb you're sawing off.