Subdirectory support on Gotek

Post Reply
Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Subdirectory support on Gotek

Post by Bagpuss » Tue Mar 27, 2018 10:22 am

Hi There,

Just wondering if anyone can help me.

I've just purchased a Gotek, and have performed the OLED, rotary encoder and speaker mods.
Everything is working just fine on my BBC Micro. I'm running HxC version V3.1.21.2a.

I was originally running the FlashCopy firmware, but found that support for Acorn formats isn't currently ideal.
I have therefore moved across to HxC and things seems a lot better.

I have some .SSD images in the root directory of my FAT32 USB stick, and these can be mounted and used fine.
However, if I copy one of them to a subdirectory, then when I select the subdirectory, they cannot be found in the list of files.

Are there any specific rules for how to put .SSD files into subdirectories?
I don't think case is a factor, as filenames with mixed case work just fine in the root directory.

Any help would be much appreciated.

Andy.

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Tue Mar 27, 2018 10:32 am

I think I might have worked this out.

SSDs that show up in the menu are all 204.8Kb in size. Ones that don't are smaller than this.
I'm guessing that these smaller files (downloaded from the stardot forum) might need to be padded?

Alternatively, is there a firmware feature to either allow them as-is, or auto-pad them?

Andy.

Jeff
Site Admin
Posts: 6649
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Subdirectory support on Gotek

Post by Jeff » Tue Mar 27, 2018 11:21 am

Bagpuss wrote:
Tue Mar 27, 2018 10:32 am
I think I might have worked this out.

SSDs that show up in the menu are all 204.8Kb in size. Ones that don't are smaller than this.
I'm guessing that these smaller files (downloaded from the stardot forum) might need to be padded?

Alternatively, is there a firmware feature to either allow them as-is, or auto-pad them?

Andy.
"Valid" file image detection is currently based on file size.
Please share your SSD files and if you know it, the disk geometry used. I can add them into the firmware format database in 2 minutes :wink: .

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Tue Mar 27, 2018 11:35 am

Hi Jeff,

Thanks for the rapid response. Much appreciated.

I'm currently chatting with the creator of the SSD files on stardot. Just trying to establish how he created them.
The standard tools for the Acorn machines to manipulate SSD files wouldn't normally create files that are non-standard lengths.
Previously, I've been importing these files into an SD card based storage solution called MMFS. This doesn't appear to care about the length, and simply pads the SSD on import. It's why I never noticed before.

I'll upload an example and some details on it as soon as I can.

Andy.

Jeff
Site Admin
Posts: 6649
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Subdirectory support on Gotek

Post by Jeff » Tue Mar 27, 2018 11:40 am

They have maybe less tracks. Can you tell me the exact file sizes ? (an ls -lsa or dir output will be perfect)

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Tue Jun 26, 2018 1:24 pm

Hi Jeff,

Sorry for not getting back to you on this topic. I know we followed up a bit on the stardot forum.

I've found that the following can pad the undersized .SSD images so that they work:
# Pad the file out to 204800 bytes
dd if=/dev/zero of=zeros bs=1k count=200
cat original.ssd zeros | dd of=padded.ssd bs=1024 count=200

I've not typically seen an issue with .DSD images, but if they did need padding, they would be double the size of the .SSD images.

Interestingly, it now has an autopad facility which allows the undersized images to work without modification.

The undersized images are quite prevalent, as much of the new software being created is shipped in this way (it appears to be a symptom of using an emulator for development, rather than real hardware). It would be great if the HxC firmware could include this feature.

Thanks,

Andy.

Jeff
Site Admin
Posts: 6649
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Subdirectory support on Gotek

Post by Jeff » Tue Jun 26, 2018 11:16 pm

The "autopad" feature sound to me a bad idea and an half-working solution : This make the file format quite ambiguous and will bring issues when writing new file to the images. The image size need to be increased during writing and the usb stick fat table need to modified in realtime. I am sure that no firmware is currently supporting this. :wink:

anyway i can enable this hack if you want (but this really sucks...).

Could you please at least tell me which non-standard file size you have met ? (Just post here an ls -l / dir command result or send me some of them to hxc2001 at hxc2001.com )
I can instead change easily the internal table to match the number of tracks of these "special" images. That's why i asked you this some months ago :D. I just need to "see" the file images size, and then i will make the modification.

Apart from this, what is missing for the images you are using ?

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Wed Jun 27, 2018 12:06 pm

Hi Jeff,

Thanks for getting back to me so quickly.

ls output is as follows. I'll send some example images in e-mail too.

-rw-rw-r--. 1 bagpuss bagpuss 48640 Mar 11 2010 10xSID.ssd
-rw-rw-r--. 1 bagpuss bagpuss 81920 Aug 4 2012 beebsid10.ssd
-rw-rw-r--. 1 bagpuss bagpuss 89600 Feb 9 2013 beebsid11.ssd
-rw-rw-r--. 1 bagpuss bagpuss 43520 Apr 29 2013 beebsid12.ssd
-rw-rw-r--. 1 bagpuss bagpuss 94720 Feb 2 2014 beebsid13.ssd
-rw-rw-r--. 1 bagpuss bagpuss 53760 Jul 20 2014 beebsid14.ssd
-rw-rw-r--. 1 bagpuss bagpuss 74240 May 10 2015 beebsid15.ssd
-rw-rw-r--. 1 bagpuss bagpuss 104960 Dec 20 2015 beebsid16.ssd
-rw-rw-r--. 1 bagpuss bagpuss 58880 Dec 22 2016 beebsid17.ssd
-rw-rw-r--. 1 bagpuss bagpuss 107520 Nov 21 2017 beebsid18.ssd
-rw-rw-r--. 1 bagpuss bagpuss 107520 Jun 5 2010 beebsid3.ssd
-rw-rw-r--. 1 bagpuss bagpuss 94720 Jul 3 2010 beebsid4.ssd
-rw-rw-r--. 1 bagpuss bagpuss 110080 Aug 7 2010 beebsid5.ssd
-rw-rw-r--. 1 bagpuss bagpuss 58880 Oct 7 2010 beebsid6_bbc_1770.ssd
-rw-rw-r--. 1 bagpuss bagpuss 58880 Oct 7 2010 beebsid6.ssd
-rw-rw-r--. 1 bagpuss bagpuss 140800 Sep 24 2011 beebsid8.ssd
-rw-rw-r--. 1 bagpuss bagpuss 125440 Jan 4 2012 beebsid9.ssd
-rw-rw-r--. 1 bagpuss bagpuss 23040 Sep 4 2010 carousel-o-sid.ssd
-rw-rw-r--. 1 bagpuss bagpuss 6400 Feb 7 2010 Chordian.ssd
-rw-rw-r--. 1 bagpuss bagpuss 7680 Jun 4 2012 Crazy Comets.ssd
-rw-rw-r--. 1 bagpuss bagpuss 7680 Jan 30 2010 CybernoidII.ssd
-rw-rw-r--. 1 bagpuss bagpuss 10240 May 31 2012 MontyOnTheRun.ssd
-rw-rw-r--. 1 bagpuss bagpuss 15360 Apr 29 2012 nyansid.ssd
-rw-rw-r--. 1 bagpuss bagpuss 7680 Jun 4 2012 Ocean Loader Remix.ssd
-rw-rw-r--. 1 bagpuss bagpuss 69120 Feb 28 2010 pj_sid_01.ssd
-rw-rw-r--. 1 bagpuss bagpuss 43520 May 1 2010 pj_sid2.ssd
-rw-rw-r--. 1 bagpuss bagpuss 48640 May 24 2015 TurboOutrun.ssd
-rw-rw-r--. 1 bagpuss bagpuss 76800 Dec 2 2010 xmassid.ssd

These truncated images are very common in the Acorn community, so support for them would be very nice to have.
The root of the issue lies with the BBC emulators, which create these truncated images as standard. This then causes problems when you try to move to real hardware. Fixing the emulators is one option, but that won't help with the many SSD files that are already out there.

With regards to other features, then I'm not aware of anything else that's missing. I'll keep looking, though. ;-)

Regards,

Andy.

Jeff
Site Admin
Posts: 6649
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Subdirectory support on Gotek

Post by Jeff » Wed Jun 27, 2018 12:41 pm

I will not for sure not let the gotek directly extend the file images since this can lead to some catastrophic results if the fat table/partition is not in good condition, which is quite common with these removable medias or if an unexpected power loss happen.
(Plus this operation tend to fragment the files on the disks.)

Anyway here is a first attempt :
https://hxc2001.com/vrac/HxCUSBFE.UPD

Is only the *.ssd files having this issues or any acorn images (DD & SD) can have the issue ?
Is there a way to check the expected disk format in the boot sector ?

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Wed Jun 27, 2018 5:31 pm

Thanks for the rapid response. I'll try to give the firmware a try tonight when I get home from work.
The issue can happen with both DSD and SSD images, but it's far more common for SSD as this is the default format created by the emulators.

A .SSD image should be 204800 bytes and DSD 409600.
The first two disk sectors contain most of the catalogue information, and there's also a field which holds the number of sectors in the volume.
For a .SSD, there should be 800 sectors, and double that for a .DSD.

I can confirm that by looking at a full .SSD image, where I see 320 hex from byte 7 of sector 1.

Jeff
Site Admin
Posts: 6649
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Subdirectory support on Gotek

Post by Jeff » Wed Jun 27, 2018 10:58 pm

Bagpuss wrote:
Wed Jun 27, 2018 5:31 pm
Thanks for the rapid response. I'll try to give the firmware a try tonight when I get home from work.
The issue can happen with both DSD and SSD images, but it's far more common for SSD as this is the default format created by the emulators.

A .SSD image should be 204800 bytes and DSD 409600.
The first two disk sectors contain most of the catalogue information, and there's also a field which holds the number of sectors in the volume.
For a .SSD, there should be 800 sectors, and double that for a .DSD.

I can confirm that by looking at a full .SSD image, where I see 320 hex from byte 7 of sector 1.
Thanks for these informations, i will use them to implement an better ssd/dsd support :D.
Anyway let me know the tests result with firmware above.

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Thu Jun 28, 2018 11:36 am

Haven't had chance to test the firmware yet, but I will do it today.
Work is really busy at the moment, so my spare time is a bit limited.

On the disk format side of things, there a great deal of information here:
http://beebwiki.mdfs.net/Acorn_DFS_disc_format

Thanks again,

Andy.

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Mon Jul 02, 2018 2:40 pm

Hi Jeff,

Just tested the new firmware, and it appears to work just fine.
The truncated images mount, and show a smaller number of tracks that matches the image size.

For example, I have an SSD which is 7.7kb, which shows as total size of 3 tracks (7884 bytes / 2560 ) = 3.
2560 being the number of bytes in a track.

If I run the *FREE command, it reports as if the image was a full size 80 track disk.
I can also run *VERIFY, and this completes as if the disk was the full 80 tracks.
It is weird to see the track counter show 79 of 03 tracks.

I've also tried saving a new file to the image, and this also works, but the total track counter stays set at 3.
The interesting thing is that the image file on the USB stick hasn't increased in size when I go back to my PC.
However, when I remount the image on the Gotek, the file I added is there and the SSD appears to be bigger.
For example, *FREE shows a decreased amount of free space.

I'm just curious how that's working, and how I might get the Gotek to mount the image as it actually is, opposed to the version I've added files to.

Thanks again,

Andy.

Jeff
Site Admin
Posts: 6649
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Subdirectory support on Gotek

Post by Jeff » Mon Jul 02, 2018 3:30 pm

Write in this particular case is not supposed to work and i think that this new file is corrupted if it is stored on a new track since the data is dropped by the emulator. The fact that you can see the file in the catalog is normal because this one is stored on the first tracks.

Bagpuss
Posts: 12
Joined: Tue Mar 27, 2018 10:13 am

Re: Subdirectory support on Gotek

Post by Bagpuss » Mon Jul 02, 2018 3:38 pm

Thanks Jeff. Writes definitely not working. The file appears to be the correct size, and a *DUMP command results in a hex dump of the correct length.
However, the dump appears to consist of a repeated set of data which presumably managed to fit in the free space remaining in the truncated image.

Also, the firmware didn't show a truncated DSD file which is on the USB stick.
I've attached the file to this post.
Attachments
dsdexample.tar.gz
(97.56 KiB) Downloaded 82 times

Post Reply