381 days later…

Spurred on by the recent one year anniversary of the release of v0.01 and the YouTube videos by Rod Hull (HE IS ROD HULL!) of TheFutureWas8Bit fame showcasing my long filename browser for esxDOS with his DivMMC Future device, I thought I’d post a small update.

v0.17 with custom colour scheme based on the standard esxDOS NMI browser

At the time of writing this, version 0.17 has just been released which contains a couple of important bug fixes to the code used by the NMI version of the browser. So, if you’re on an older version or running the No_MMC_Memory version on your divIDE device, I would strongly recommend updating to this version even if you’re just using it to launch your .tap and snapshot files and not using some of the more advanced functionality.

As always, you can get the latest version of the browser here and the forum thread at spectrumcomputing.co.uk is a good place to ask for advice, request features, report bugs and get the latest test versions of the browser before official releases.

If you’re a fan of videos with poor production quality, that use the Spectrum ROM font and feature shonky camera phone or video capture footage, then you may also be interested in my fledgling YouTube channel, where you can find videos highlighting new features in upcoming versions of the browser.

I’d like to say a big thank you to all the people who have reported bugs, made suggestions, given feedback and spread the word about my browser. Seeing where the browser is now, just a year on from the first release (which only worked with .tap files on FAT16 cards) is very satisfying!

The Long And The Short Of It

Like many people during the months of April and May, I found myself with a lot more free time on my hands after being put on furlough whilst COVID-19 caused things to grind to a halt across the world. I’d been spending some of it playing around with the ZX-UNO I purchased a while back. This is an FPGA recreation of the Sinclair Spectrum family of computers which also has support for the Russian ‘Pentagon 128’ variant – and the ability to simulate other 8 bit micros like the BBC, C64 and Amstrad.

Picture of my ZX-UNO.

It uses an SD card to provide an emulated DivIDE/DivMMC interface. These are hardware that let you access an IDE drive, Compact Flash or SD Cards on a real Spectrum. The ‘OS’ these devices use is called ESXDOS. This gives you a command driven interface for accessing files on the disk as well as providing a hardware NMI (Non Maskable Interrupt) button which launches a file browser. This works like a freezer cartridge letting you break in at any point to save a snapshot of the currently running program. It can navigate the contents of the disk as well as letting you automatically load .tap files (virtual Spectrum tape images), .sna, .z80 (memory snapshots) and .trd files (virtual Spectrum disk images normally found on Pentagon machines).

For reference, here’s the standard ESXDOS NMI browser:

The default esxdos NMI browser.
The standard ESXDOS NMI browser with a guest appearance from the ROM font.

You’ll notice it doesn’t handle long file names. ESXDOS supports FAT16 and FAT32 file systems but is limited to the old DOS 8.3 style filenames. The files are also listed in the order they appear on the disk – not in alphabetical order. After using this for a while and having forgotten the joys of dealing with an 8.3 filename world, I started to wonder whether it was possible to do a nicer file browser that could somehow support long filenames.

My first attempt was to write a Python script which took a folder on the SD card and read in all the long filenames and wrote them out to a binary file with a special filename in that folder. I then wrote a basic browser program in C that looked for this file and displayed the contents on the screen to show the long filenames. It then converted the long file name on the fly back to the 8.3 form and launched that through ESXDOS. This sort of worked but had several limitations.

Any time you deleted a file, created a folder, added a new file or renamed an existing one you needed to rebuild the special data file to keep everything in sync. I also ran into an issue during development (I’m primarily running Ubuntu these days. Life’s too short for Windows 10 hassle) where I couldn’t easily determine the 8.3 filename from the original long filename. Most of the time, this wasn’t an issue as the majority of the files all mapped to unique 8.3 filenames. The order of files I was getting in my Python script when enumerating a folder wasn’t quite the same as the order on the disk. So I couldn’t be a 100% sure when you had two filenames like ‘Jet Set Willy.tap’ and ‘Jet Set Willy 2.tap’ – which would appear in 8.3 format as JETSET~1.TAP and JETSET~2.TAP – which short filename related to which long filename.

I decided to start again, this time seeing whether I could access the information directly from the FAT file system. Using the supplied ESXDOS dot command .dskprobe (a low level disk utility) I saw it was possible to access the FAT boot sector which was the first step in getting the information I required. At this point, I switched to C and wrote a test console program that loaded in a disk image I had made of the 512MB FAT16 SD card I’d been using with the ZX-UNO.

After reading a few web pages on the FAT file system, a couple of trips to the hex editor to confer with my disk image and lots of trial and error I had located the root folder and could read the directory entries in it. Long filename support on FAT is rather convoluted due to maintaining backwards compatibility with existing software (it uses pseudo invalid 8.3 filename entries to store the additional information). After getting my head round that, I now had some test code which could list the files in a given directory and show long filenames where used.

The browser code I’d written during my first attempt was then modified to use a list of data structure supplied directly from the FAT rather than from the pre-populated binary file – so that wasn’t a complete waste. I had to do a spot of disassembly on the .dskprobe program as the parameters to the ESXDOS API call to access the disk were not particularly well documented. This wasn’t the only time I had to resort to disassembling ESXDOS components during development of the browser. Fortunately, z88dk has ESXDOS support so it’s includes and code filled in a lot of the blanks.

After a couple of days of hacking around with my FAT test program, I’d knocked up a prototype .dot command called .browse using z88dk. The C FAT code from my test program cross compiled without major issues in z88dk (most of the browser was written in C, except for some assembly graphics routines).

Here’s how the initial version 0.01 of .browse looked:

Version 0.01 of my browser.
Long filenames abound!

It supported long file names and also sorted the filenames – directories first, then files in A-Z order. The colours were hardcoded so folders appeared in blue and files appeared in black. Launching .tap files was supported – the other formats – .sna, .z80 and .trd weren’t. It was also limited to FAT16. There was no support for FAT32. At this stage it was a bit rough around the edges and lacked some key features but was a good enough proof of concept.

My first go at FAT32 support was added in version 0.02, alongside some fixes to the existing FAT code to handle folders that spanned multiple clusters. I also fixed an issue from the first version where on launching .browse, you were always dumped into the root folder regardless of your current directory before hand. I added some code that took the current directory and located it in the FAT.

Version 0.03 of the browser
Those extra 10 characters make all the difference.

On the Spectrum, if you use 8×8 characters in your font you can fit 32 characters on a line. There have been a number of ways to workaround this limitation. Tasword Two, a word processor that shipped with certain editions of the Spectrum 48k+ used a custom font of 4 x 8 to fit 64 characters on to the line. This works but the characters can look poorly defined due to the halved width. A good compromise is custom font of 6 x 8 which gives you 42 characters. I’d seen this used to good effect having played the Adventure International adventure games like Spiderman or Buckaroo Banzai. I decided to go with this to give the browser a bit more space to show filenames. I wrote the 42 column text routine myself in assembly which was a bit of challenge due to the memory layout of the screen on the Spectrum.

Alongside the more compact display, very long filenames were now truncated to the screen width with a custom ellipsis character. I also got .trd files to auto start from the browser after disassembling the .vdisk dot command and ESXDOS NMI browser to see how this was achieved. This was released as v0.03 and was the point when I noticed I was starting to get traction with the project. I’d started a topic on the Spectrum Computing website forum and was getting replies and feedback. From this, I discovered that my FAT code had a number of issues. On some configurations the browser just showed a blank screen and failed to start or listed the root folder correctly but showed empty sub folders or folders full of garbage characters.

v0.04 had a fix for the FAT16 problem where empty or garbage folders were being displayed rather than the actual contents. I also added the ability to load .z80 and .sna files using the ESXDOS .snapload dot command.

Picture of the configuration editor.
Green screens was my delight.

A user asked for the ability to change the colour scheme so I wrote a separate configuration editor program to adjust the colours. I also added the ability to specify the ESXDOS device identifier to use for directly accessing the disk in the browser for configurations with multiple disks or partitions. This was released in version 0.05 alongside some more FAT fixes – I wasn’t handling FAT32 clusters correctly which meant that files on larger drives with clusters greater than 65536 were returning empty folders or garbage folder listings. To my knowledge the FAT code is now working correctly (famous last words).

The hopeful finalisation of the FAT code seems like a good point to pause the story of developing the browser. There’s a lot of other things to go through – replacing the NMI, hitting the limit of .dot commands and accessing the ZX-UNO turbo modes – but that’s for another post.

The latest version of my browser can always be downloaded from here. There’s also the browser topic at the Spectrum Computing forum which contains ongoing information and discussions.

A hand-holding exercise – the Nintendo DS

I’ve always been a fan of Nintendo’s portable game devices, from the early Game and Watch units to the various editions of the Game Boy. Dr Mario on the original Game Boy is still one of my favourite games to this day. I’d go as far as saying it’s better than Tetris – I once played it for so long that when I went to sleep and closed my eyes I could still see small pills falling downwards.

I’d already helped Nintendo along by getting a Game Boy Color and Game Boy Advance (which I picked up on a holiday in Japan), so when the chunky metallic blue Nintendo DS came on sale I got one. The launch titles weren’t too bad and showed what the touch screen brought to the party. I initially thought the dual screens and stylus were seen just gimmicks but they did bring some genuine innovation to handheld games.

Gameboy Color, Gameboy Advance, Nintendo DS and Nintendo DS Lite.
A family reunion.

I’d obtained flash cartridges for the DS’s ancestors. These let you put multiple dumps of ROM images on one cartridge – so you only needed one cartridge and didn’t have to carry all your games around with you. Apart from software piracy, they unlocked the world of home brew on these devices. The Game Boy Color and Game Boy Advance had a surprising amount of home brew software- I had versions of Jet Set Willy and Jet Pac for the Game Boy Color and a full Spectrum emulator for the Game Boy Advance. Outside of doing proper development work for Nintendo and having access to the official developer kits and hardware, flashcards were the only way for home brew developers to get their code running on the Game Boy and Game Boy Advance.

Jet Set Willy on the Gameboy Color.
Who needs a Vega Plus?

The first DS flash card I got was called the GBA Movie Player. This fitted into the DS’s GBA cartridge slot (Slot 2). It had to go here as it was too large to fit inside the native DS cartridge slot (Slot 1). The card had a full sized SD card slot which you could copy files on to. Nintendo had also implemented some security on the DS where you could only run code from Slot 1 if the correct RSA security signature was present. All legitimate DS cartridges had this signature so they could run on the device.

To get round this, a second device called a Passkey had to be fitted into Slot 1, alongside with an actual DS game. The passkey then used the signature from the real game to get the code running from the Slot 2 device. While this worked, it wasn’t an ideal solution as the DS would no longer lay flat on a surface as the Passkey and donor game hung out of Slot 1. The situation would eventually be improved by the continual shrinking of flash memory technology. SD cards were superseded by Micro SD cards. The size of these cards easily fit inside the Slot 1 cartridge form factor.

One Slot 1 flashcard went on to become the standard, the google or photoshop if you will – the R4. I bought an original one from a computer fair for £30 pounds. As befitting for a device that enables piracy, the R4 was copied and cloned and various knock off versions flooded the market. The R4 removed the need for a Slot 2 device and also did away with the need for the Passkey.

Nintendo DS flashcards.
The inevitable march of technology.

The R4 flashcard made piracy a lot easier on the device. I remember the guy who sold it to me at the computer fair telling me I could fill the Micro SD card with games and his genuine surprise when I told him that you could also run home brew titles like Lemmings DS or emulators on it too. Subsequent versions of the DS hardware and some games included more checks to stop games running if they were being played from one of these devices. Unsurprisingly newer cards were produced and in game protections were removed by the groups releasing game dumps for these devices.

It is fair to say that the original DS is not a looker. The chunky casing and styling was a nod back to the original dual screen Game and Watch devices. Nintendo released the much slinkier looking DS Lite. This was basically the original Nintendo DS put on a severe diet and taking styling pointers from Apple. I held out for a while before purchasing this as I was slightly miffed at buying the first version at full price only for this much improved version to turn up not much later. There is a lesson here – never buy the first version of a Nintendo product.

Nintendo DS and a DS Lite.
Sibling rivalry.

As I now had a chunky blue paperweight in my possession, I decided to perform some light surgery on it. Some clever people had managed to crack the security on the DS firmware so you could run Slot-1 code without the security signature. You just needed to re-flash the firmware. This involved removing the battery cover and jamming a metal screw into an innocuous hole inside the casing to short a connection – what could possibly go wrong? After a nervy few minutes, I ended up with a modded DS and not a chunky bricked blue paperweight.

Home brew development on the Nintendo DS was made possible by libnds – a subset of the devkitpro home brew development kit (this also supported other devices like the GP32 and Game Boy Advance). Installing this gave you a gcc style environment to compile C / C++ into the .nds file format that ran on the Nintendo DS. As this development kit was not officially sanctioned by Nintendo, accessing the features of the device had to be done from scratch and not using any copyrighted material. In earlier versions of libnds, the routines to handle and read locations from the stylus and touchscreen were not as accurate compared to that in commercial versions. As the developers unlocked the secrets of the device, these problems went away.

The first thing I managed to get running with the libnds development kit on my DS was a hacked up port of Quirky’s Chaos Advance. This itself was a port of the seminal ZX Spectrum game Chaos for the Game Boy Advance. This utilised the larger screen of the DS so you no longer had to scroll the screen around to see the full game board – the Spectrum screen resolution of 256 x 192 pixels (minus the border) was the same as the DS. It also added it some basic touch screen input so you could enter player names with an on screen keyboard or select spells.

I did the bulk of the work using an emulator – Dualis originally, then no$gba once it added more Nintendo DS features – as the workflow was much faster than having to compile the code, take the SD card out of the flash card, write the new .nds file from the PC to the SD card, put it back into the flashcard, turn the DS on and then select the file from the loader menu. You couldn’t completely avoid using real hardware as the emulators were still quite early on in their development so you could end up in situations where the emulator crashed running your code, whereas the DS didn’t and vice versa.

ChaosDS screen shot
While it’s all kicking off below, the upper screen is not being used to it’s full potential.

When I did encounter crashes or bugs either in the emulator I then had the fun or trying to track the problem down without a debugger. I’d been developing on Windows with Visual C++ for some years and had been spoiled rotten by it’s lovely debugger. I had to switch back to printf-ing debug output onto one of the screens with messages to see how far the code had gotten when the crash occurred – Got here, Got here1, Got here2 and so on. I used this as a divide and conquer mechanism to narrow down the section of code that was causing the issue.

Quirky eventually updated his GBA port to run on the Nintendo DS, negating the need for my hacked up version. This was not a wasted endeavour as it gave me a good grounding in developing homebrew on the Nintendo DS with libnds. I was now on the look out for a new project.

To be furthered…

So, about last year…

Turns out that I’m better at writing code than writing about code. Though that’s not too surprising if I’m honest. Documentation is always a chore and my past low tech attempts at chronicling events can be found in various unfinished diaries.

So to start 2019 with slightly better intentions – and yes, I’m aware it’s already mid February – let’s begin with some details on my latest release for the venerable ZX Spectrum, Your crackers, m’lord!

Towards the end of October last year, R-Tape took his special ceremonial horn off the wall, put it to his lips and let out a clarion call for submissions to the second edition of the Woot! Christmas tape magazine . I had an idea for something to do but couldn’t get it together in time as I was struck down with a chronic bout of level designer’s block. With the December deadline rapidly approaching, I went back to the drawing board.

So what to do in a short amount of time? I still wanted to knock a simple game together. I tend to favour writing puzzle games as they can be easier to implement in a short amount of time – especially if the main mechanic isn’t too complicated. Then, it’s just a matter of going through and adding all the levels.

By chance, I stumbled across a Scratch remake of Kiragames’ ‘Unblock Me’ by griffpatch. The aim of the game is to slide the pieces of wood vertically or horizontally to let you move the red block across the screen to the hole on the right hand side. The red block can only slide horizontally and the others can only slide in the direction they are oriented. It had a small level set, was challenging and the cleanly designed graphics would work well on the Spectrum.

griffpatch’s Scratch remake

Over the first weekend in December, I fired up my trusty text editor alongside with the z88dk C compiler and started work on my conversion. By Sunday afternoon, I had this garish prototype:

Crackers on acid!

It had a frontend (recycled from Thoroughly Modern Willy), 4 levels and the code to display the board – but you couldn’t move any of the crackers around or finish a level. Over the subsequent days, common sense prevailed and I settled on a more sedate set of cracker graphics. I also added the code to select a cracker (using the Spectrum’s BRIGHT attribute as an easy way to implement the selection cursor), move the cracker around the board vertically or horizontally and to detect when the red cracker was at the gap in the board which meant the level was finished.

The finished product.

With the game code sorted I set about playing the levels on the original to obtain the level data for the 28 remaining levels. These were pretty straightforward until the last 2, which had been sourced from a harder level of play. After some head scratching, I had the full level set and the skills to play test all the levels. For each level, I took screenshots of the game’s web page on starting and finishing a level so I could check I’d entered the level data in correctly and knew where I needed the crackers to end up to complete the level.

At this point, the only reference to Christmas was the cracker graphics. When you pull a cracker at Christmas, aside from the coloured paper crown you usually end up with a gift of dubious quality – usually a tiny notepad, some impossible joined metal loop puzzle or a red plastic fish with ‘psychic’ powers – and a piece of paper with a poor quality Christmas joke printed on it. I decided to source 32 cracker jokes, one for each level to display on completing a level. Fortunately the internet had my back with various web sites compiling lists of the best worst Christmas cracker jokes.

The main constraint for using a joke was whether I could fit the question and answer parts of the joke on one line. This proved challenging with an 8 x 8 pixel font, so I used Einar Saukas’ fzx font system support in z88dk to use a variable width font which let me squeeze more characters on to the line.

Just one of the 32 pieces of comedy gold strewn between the levels.

The name ‘Your crackers, m’lord’ comes from a sketch by the Two Ronnies (a popular UK comedy double act from the 1970s and 80s whose Christmas show was a big telly event back then). The sketch is in a similar vane to the classic ‘Fork handles / Four candles’ where the property of certain English phrases sounding identical to each other but meaning vastly different things is used to comic effect. In this case, the butler whilst serving courses of a meal to an aristocratic couple is saying nice things to his Lordship’s wife “Your sweet m’lady.” but is insulting to his Lordship “Your nuts m’lord.”Your crackers m’lord”.

This sketch also gave me the bare bones of a story to hang the game off. The reason you’re having to go to all this trouble moving these other crackers around is to satisfy your aristocratic employer’s demand for a red cracker with his festive food stuffs. I’m sure this was one of the sub plots of a Christmas edition of Downton Abbey one year.

For a rush job I was happy with the overall result. I’d like to thank griffpatch for his original Scratch game which provided the inspiration and source material for my conversion and R-Tape for compiling another Woot! tape magazine.

Statement of intent

Hello. This is the first post on my site. If you have high expectations, allow me to dash them quickly. This site will showcase new and current projects as well as  some of the older things I’ve worked on.

I’m currently writing code for the ZX Spectrum (an 8 bit computer from the UK) but have dabbled with 68000 assembly on the Amiga, lua games programming (through love) and homebrew on the Nintendo DS using libnds and gcc.