Hard sector format

HxC Floppy emulator support for all others computers...
Post Reply
snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

I uploaded a tarball with these files:

Code: Select all

-rw-r--r-- hirsch/hirsch 179200 2005-03-31 19:44 Gbasic550.nsi
-rw-rw-r-- hirsch/hirsch 358400 2015-01-02 17:47 ns1423_hard_disk_startup.nsi
-rw-rw-r-- hirsch/hirsch 358400 2015-01-02 17:45 s231_hard_disk_suppl.nsi
The shorter one is single-sided, others double-sided.

Jeff
Site Admin
Posts: 8092
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Hard sector format

Post by Jeff »

snhirsch wrote:
Wed May 23, 2018 2:01 pm
Jeff wrote:
Wed May 23, 2018 8:49 am
Ok here all the tests images :

https://hxc2001.com/vrac/HS/tests/

Let me know what work and what doesn't.

About the sectors 342 & 600 i am not sure where they are located on the disk (at which track/side ?). It look like these sectors are "empty", but i am not sure which track/side to look.

Btw is there a some raw sectors images available ? I have created some ".NST" raw file from the kryoflux dump, but i want to make the hxc software input/output compatible with existing raw file for Northstar... (If this format is already existing...)
The disks are written with 10 sectors per track, 35 tracks per side. Sides are handled in a serpentine manner, tracks 0-34 on side 0, then 34-0 on side 1. Many older formats use this arrangement, since it saves a full disk seek on files that straddle sides. Sector numbering starts at 0. So, sector 342 is "track 34, sector 2 (side 0)". The first sector on side 1 is 350 (track 34, sector 0, side 1). So, sector 600 lies 250 sectors - or 25 tracks - out towards the edge of the diskette. 34 - 25 = 9, so it looks like sector 600 is physically "track 9, sector 0" on side 1. Hopefully this makes the layout clear.

There should be no "empty" sectors. Low level format writes every sector on the diskette. The surface scan tool (built into the Northstart 'copy' program) reads every sector sequentially. The original diskette reports NO bad sectors so they must all be readable by the diskette controller.

I will send some "official" NSI images to the ftp upload directory. As I mentioned in a note last week, these are simply sector data in a linear arrangement. You will need to keep the serpentine ordering (as describe above) in mind when organizing the double-sided NSI into HFE images.
Just to be sure about the NSI file format :
Is this the right layout for a double sided image ? :

SOF
-Side 0 Track 0 Sector 0-Side 0 Track 0 Sector 1-Side 0 Track 0 Sector 2- .....-Side 0 Track 0 Sector 8-Side 0 Track 0 Sector 9
-Side 0 Track 1 Sector 0-Side 0 Track 1 Sector 1-Side 0 Track 1 Sector 2- .....-Side 0 Track 1 Sector 8-Side 0 Track 1 Sector 9
-Side 0 Track 2 Sector 0-Side 0 Track 2 Sector 1-Side 0 Track 2 Sector 2- .....-Side 0 Track 0 Sector 8-Side 0 Track 2 Sector 9
-Side 0 Track 3 Sector 0-Side 0 Track 3 Sector 1-Side 0 Track 3 Sector 2- .....-Side 0 Track 0 Sector 8-Side 0 Track 3 Sector 9
...
...
-Side 0 Track 33 Sector 0-Side 0 Track 33 Sector 1-Side 0 Track 33 Sector 2- .....-Side 0 Track 33 Sector 8-Side 0 Track 33 Sector 9
-Side 0 Track 34 Sector 0-Side 0 Track 34 Sector 1-Side 0 Track 34 Sector 2- .....-Side 0 Track 34 Sector 8-Side 0 Track 34 Sector 9
-Side 1 Track 34 Sector 0-Side 1 Track 34 Sector 1-Side 1 Track 34 Sector 2- .....-Side 1 Track 34 Sector 8-Side 1 Track 34 Sector 9
-Side 1 Track 33 Sector 0-Side 1 Track 33 Sector 1-Side 1 Track 33 Sector 2- .....-Side 1 Track 33 Sector 8-Side 1 Track 33 Sector 9
-Side 1 Track 32 Sector 0-Side 1 Track 32 Sector 1-Side 1 Track 32 Sector 2- .....-Side 1 Track 32 Sector 8-Side 1 Track 32 Sector 9
...
...
-Side 1 Track 0 Sector 0-Side 1 Track 0 Sector 1-Side 1 Track 0 Sector 2- .....-Side 1 Track 0 Sector 8-Side 1 Track 0 Sector 9
EOF

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

That will work, but very slowly due to lack of physical interleave. A Northstar floppy is arranged this way:

Physical Order from rotational index: 0 2 4 6 8 1 3 5 7 9

If you arrange that interleave reads will be faster. Again, this is a physical (not logical) interleave. The computer won't care either way, but linear arrangement will waste time.

I have tried all the images you sent. The ones that come closest to working properly are the "norm_preamble" files. Both of those boot and - almost - work. There continue to be read errors on mass file copy and surface scan. Sector 600 almost always shows up as bad - others vary. All of these images are based on the Kryoflux captures, correct? I'll be very interested to see how an emulation file that's correct by construction (from NSI image) will work.

Jeff
Site Admin
Posts: 8092
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Hard sector format

Post by Jeff »

snhirsch wrote:
Thu May 24, 2018 1:00 am
That will work, but very slowly due to lack of physical interleave. A Northstar floppy is arranged this way:

Physical Order from rotational index: 0 2 4 6 8 1 3 5 7 9

If you arrange that interleave reads will be faster. Again, this is a physical (not logical) interleave. The computer won't care either way, but linear arrangement will waste time.

I have tried all the images you sent. The ones that come closest to working properly are the "norm_preamble" files. Both of those boot and - almost - work. There continue to be read errors on mass file copy and surface scan. Sector 600 almost always shows up as bad - others vary. All of these images are based on the Kryoflux captures, correct? I'll be very interested to see how an emulation file that's correct by construction (from NSI image) will work.
This is very odd : For sure the Kryoflux dump have no interleave at all and no "serpentine" ! I tend to believe the kryoflux dump, unless there is something special with the original disk creation ?
Another point : i think i have see NSI files with ~75KB file size. Is there an Northstar FM format ?

For the random read errors, i think that will have to have a look into the firmware, but before this i want to "finish"/release the hxc software with the Northstar & Heatkit support (for KF import/export and raw R/W files support).

Jeff
Site Admin
Posts: 8092
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Hard sector format

Post by Jeff »

I have updated the latest beta windows version (can be build on linux from sources) :

https://hxc2001.com/download/floppy_driv ... t_beta.zip

- Read/Write NSI support added.
- The KF raw jitter problem is now fixed, so you can now load the kf raw files and export to NSI or HFEv3.
(Note : Probably work without this step but from an KF raw import you may need to normalize/clean up the bitrate before exporting to the HFEv3 for the emulator. Use the edit tools and the "Set Disk bitrate" button).

The NSI loader have the "serpentine" side layout, but the sectors are not interleaved since i only see interleave 1 in the kf raw files.
Please try it, convert raw files, try the generated NSI files with some emulators. I want to be sure that the layout is the right one.
Don't hesitate to use the track viewer in track mode and disk mode to check the loaded disk layout !

Have fun ! :D

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

Jeff wrote:
Thu May 24, 2018 7:28 am
snhirsch wrote:
Thu May 24, 2018 1:00 am
That will work, but very slowly due to lack of physical interleave. A Northstar floppy is arranged this way:

Physical Order from rotational index: 0 2 4 6 8 1 3 5 7 9

If you arrange that interleave reads will be faster. Again, this is a physical (not logical) interleave. The computer won't care either way, but linear arrangement will waste time.

I have tried all the images you sent. The ones that come closest to working properly are the "norm_preamble" files. Both of those boot and - almost - work. There continue to be read errors on mass file copy and surface scan. Sector 600 almost always shows up as bad - others vary. All of these images are based on the Kryoflux captures, correct? I'll be very interested to see how an emulation file that's correct by construction (from NSI image) will work.
This is very odd : For sure the Kryoflux dump have no interleave at all and no "serpentine" ! I tend to believe the kryoflux dump, unless there is something special with the original disk creation ?
Another point : i think i have see NSI files with ~75KB file size. Is there an Northstar FM format ?

For the random read errors, i think that will have to have a look into the firmware, but before this i want to "finish"/release the hxc software with the Northstar & Heatkit support (for KF import/export and raw R/W files support).
I will try it, thanks. The Kryoflux simply reads inward from the edge with sides alternated. It couldn't know about serpentine data layout since it makes no attempt to interpret the data. Just to be clear: The sector and track identification (not data) is 0-to-34 on both sides of a Northstar diskette (I noted how the track identifier is formed in my original document I sent). The serpentine layout is how sequential logical sectors are assigned. It has nothing to do with low-level formatting and is only important when, e.g. converting from an NSI to a diskette image.

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

Jeff wrote:
Thu May 24, 2018 7:28 am
This is very odd : For sure the Kryoflux dump have no interleave at all and no "serpentine" ! I tend to believe the kryoflux dump, unless there is something special with the original disk creation ?

Another point : i think i have see NSI files with ~75KB file size. Is there an Northstar FM format ?
As I mentioned in a previous reply: The Kryoflux could NOT know about serpentine layout, since it's the order that logical sectors are used by the operating system. It is NOT how tracks and sectors are physically marked (by track/sector byte). It is relevant only when (a) interpreting a raw kryoflux dump as a CP/M volume or (b) creating an emulation file from an NSI file.

To your other question: Yes, there is a Northstar FM format. I have no hardware capable of either reading or writing it.

From the document I put together and sent you:

Code: Select all

NOTE:

The Northstar technical documentation has a discrepancy between
figures and text.  Figures 3-4 and 3-5 show the layout for DD MFM
diskettes with 512-byte sectors.  The text references an older SD FM
format with 256-byte sectors.

The document you are reading now is intended to cover the DD MFM
512-byte format.
So, if you read the text in the PDF extract I also sent, you'll see the description for the SD FM format. It's similar, but has a shorter preamble (since data cells are 2x longer) and FM encoding in the data field. I do not think there are any other differences. For testing you would need to find someone with an older controller.

Jeff
Site Admin
Posts: 8092
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Hard sector format

Post by Jeff »

Further progress : Heathkit sector decoding :
heathkit.jpg
heathkit.jpg (178.92 KiB) Viewed 4176 times
heathkit2.jpg
heathkit2.jpg (173.15 KiB) Viewed 4176 times
:D

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

That is great news - thanks!

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

Hi, Jeff. No luck so far with the HxC windows app. I used the "Load" button to read a Northstart NSI file (boot diskette) and the tool did seem to understand it. Then I exported as HFE and tried it. No luck. I cannot read the emulated image. Throws an immediate error on the first sector accessed. Is this currently supposed to work?

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

No luck with Kryoflux conversion either. I'm not completely sure how to load all 70 individual files, since the tool won't let me highlight more than one. So, I pointed to the first track and it seemed to read everything in. However, side 1 of the diskette image shows no data, which is not correct. When I produce an HFE from it, that does not read at all. Same as previous report. Immediate read error.

It's possible I'm just doing something wrong. Please advise?

Jeff
Site Admin
Posts: 8092
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Hard sector format

Post by Jeff »

Have you checked the track/disk viewer ? This window show you how is the currently loaded disk. (Set the Northstar mode.)
Also : You must use the HFE V3 for hard sectored disks.

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

Jeff wrote:
Fri May 25, 2018 8:40 pm
Have you checked the track/disk viewer ? This window show you how is the currently loaded disk. (Set the Northstar mode.)
Also : You must use the HFE V3 for hard sectored disks.
That works much better using HFE v3 - didn't know about that option. Both a KF import and NSI import nominally work, but both throw many read errors. Going by the sector numbers, the vast majority of these appear to be the first sector on the track. I suspect something going wrong immediately after step.

The KF import still appears to be missing all data on the second side of the diskette. I am determining that by looking at the track/disk viewer. There should be data on the inner 2-3 tracks. Instead, it shows all zeroes.

Jeff
Site Admin
Posts: 8092
Joined: Fri Oct 20, 2006 12:12 am
Location: Paris
Contact:

Re: Hard sector format

Post by Jeff »

snhirsch wrote:
Fri May 25, 2018 9:50 pm
That works much better using HFE v3 - didn't know about that option. Both a KF import and NSI import nominally work, but both throw many read errors. Going by the sector numbers, the vast majority of these appear to be the first sector on the track. I suspect something going wrong immediately after step.
Yes this is possible. Will have a look into the firmware right after the heath & northstar import/export validation.
snhirsch wrote:
Fri May 25, 2018 9:50 pm
The KF import still appears to be missing all data on the second side of the diskette. I am determining that by looking at the track/disk viewer. There should be data on the inner 2-3 tracks. Instead, it shows all zeroes.
The KF import is not lying : the kf ns_ds_kf side 1 sectors are all zero for sure...

snhirsch
Posts: 170
Joined: Tue Dec 08, 2015 1:42 am

Re: Hard sector format

Post by snhirsch »

Jeff wrote:
Sat May 26, 2018 12:49 am
snhirsch wrote:
Fri May 25, 2018 9:50 pm
That works much better using HFE v3 - didn't know about that option. Both a KF import and NSI import nominally work, but both throw many read errors. Going by the sector numbers, the vast majority of these appear to be the first sector on the track. I suspect something going wrong immediately after step.
Yes this is possible. Will have a look into the firmware right after the heath & northstar import/export validation.
snhirsch wrote:
Fri May 25, 2018 9:50 pm
The KF import still appears to be missing all data on the second side of the diskette. I am determining that by looking at the track/disk viewer. There should be data on the inner 2-3 tracks. Instead, it shows all zeroes.
The KF import is not lying : the kf ns_ds_kf side 1 sectors are all zero for sure...
The diskette I thought I imaged was > 50% full and should have had files on the other side. Maybe I messed up and imaged something else :-)

I"ll try the exported KF images with Heath and Northstar emulators in the next couple of days.

Post Reply