Stream file conversion with hxcfe

[Rouquemoute]

Hi!

I'm trying to read the content of old 8'' floppies (FM, single side, single density, 77 tracks, 26 sectors of 128 bytes each) on Windows. So far I've dumped the flux reversal signal into stream files using a Kryoflux, then converted the stream/raw files into full disk images with dtc.exe.

It works well for some of the floppy disks, but I've got "Bad sector" errors for some others. I've then tried converting those faulty raw files with hxcfe and surprisingly it worked flawlessly most of the time. The data produced this way has been validated through other means, so I guess hxcfe really can read those floppies while dtc can not. Hats off for that!

However I've still got a few concerns about a handful of floppies whose data won't validate. First of all whenever I select the -verbose option with hxcfe.exe I run into a runtime error. The exact same command without the -verbose option works well. Is it a bug or did I do something wrong? See the attached file exemple00.0.raw (first track of one of the faulty floppies).

Code: Select all

> hxcfe.exe -finput:exemple00.0.raw -infos -verbose
HxC Floppy Emulator : Floppy image file converter v2.0.3.1
Copyright (C) 2006-2019 Jean-Francois DEL NERO
This program comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions;

libhxcfe version : 2.10.2.2

verbose mode
Input file : exemple00.0.raw
---------------------------------------------------------------------------
-                        File informations                                -
---------------------------------------------------------------------------
File: exemple00.0.raw
Checking exemple00.0.raw
KryoFluxStream_libIsValidDiskFile
File loader found : KRYOFLUXSTREAM (KryoFlux Stream Loader)
Loading exemple00.0.raw
KryoFluxStream_libIsValidDiskFile
file loader found!
KryoFluxStream_libLoad_DiskFile
Can't open config.script !
1 track (0 - 0), 1 sides (0 - 0)
------------------------------------------------
Loading exemple00.0.raw...
---Index--- : 0 sp:32753
StreamPosition: 0x00003CB1 SysClk: 0xB9820681 Timer: 0x0000002D
---Index--- : 1 sp:98259
StreamPosition: 0x00014551 SysClk: 0xB989B743 Timer: 0x0000002E
---Index--- : 2 sp:163765
StreamPosition: 0x00024DF1 SysClk: 0xB9916703 Timer: 0x0000002D
---Index--- : 3 sp:229271
StreamPosition: 0x00035691 SysClk: 0xB9991828 Timer: 0x0000002E
---Index--- : 4 sp:294777
StreamPosition: 0x00045F31 SysClk: 0xB9A0C7A4 Timer: 0x0000002E
---Index--- : 5 sp:354230
StreamPosition: 0x000567D1 SysClk: 0xB9A878C6 Timer: 0x0000002E

runtime error R6002
- floating point not loaded

More to the point my main question is whether sector CRCs are checked accurately while converting. In the example below the track is converted without error into an image, but with a hexadecimal editor it still seems that some sectors are very suspicious, like the first one (0x00-0x7F). All data on this track should indeed be EBCDIC-encoded hexadecimal values. How is it possible to extract obviously invalid data from a sector without triggering a CRC error?

Code: Select all

> hxcfe.exe -finput:exemple00.0.raw -foutput:exemple.img -conv:RAW_LOADER
HxC Floppy Emulator : Floppy image file converter v2.0.3.1
Copyright (C) 2006-2019 Jean-Francois DEL NERO
This program comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions;

libhxcfe version : 2.10.2.2

Input file : exemple00.0.raw
Output file : exemple.img
Checking exemple00.0.raw
File loader found : KRYOFLUXSTREAM (KryoFlux Stream Loader)
Loading exemple00.0.raw
file loader found!
Can't open config.script !
hxcfe_FxStream_AddIndex : streamposition beyond of stream limit ! (354257 >= 354230)
track file successfully loaded and encoded!
Write RAW file exemple.img...
track:00:0 file offset:0x000000, sectors: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 ,128B/s
Stopping HxCFloppyEmulator...

Any help would be much appreciated. Thanks!

[Jeff]

I recommend you to use the latest version:

https://hxc2001.com/download/floppy_dri ... t_beta.zip

and also use the GUI :

Screenshot at 2020-10-15 22-26-43.png (43.47 KiB) Viewed 2666 times

The stream appears to be clean. I don't think that there is any physical damage.
Interesting point : The sectors with the bad CRC have the 0xF8 data mark which mean "deleted data mark".
So i think that this is not a bug but a feature : I suspect that the machine write a partial sector with the 0xF8 data mark to "delete" the sector.
(Note : a .img can't store the data mark and crc state. I recommend you to use another format : hfe, imd,...)

[Rouquemoute]

Thanks a lot Jeff! The beta version works much better indeed
And you're right, the track I uploaded probably just contains a lot of garbage data.

I do use the GUI sometimes and find it very useful for forensic purposes, but I have to deal with literally thousands of floppies so the process has to be automated to some extent.

By the way is there an option to have a summary of all CRC errors and odd data marks in the conversion log? The only way I found was to use the -verbose option and parse the output log, but the latter is pretty big (~35 Mo). If I understood correctly another way to get the info would be to convert to formats like hfe or imd instead of img and extract the sector status afterwards. Are there specifications or libraries to read those formats? Since I'm only interested in the data contained in the floppy and not actually using them for emulation it would be a bit of an overkill though.

Also there seem to be a problem with unicode support in the hxcfe.exe CLI. There is an error if I try to open a file with a non-ASCII character somewhere in its path. The weird part is that hxcfe first correctly accesses and identifies the input file format, then throws a load error when trying to actually read it. If I remove the non-ASCII character from the path everything goes fine.

Code: Select all

C:\HxCFloppyEmulator_soft_beta_v2.5.2.1>hxcfe.exe -infos -finput:testà\Disquette_0001_00.0.raw
HxC Floppy Emulator : Floppy image file converter v2.0.3.1
Copyright (C) 2006-2020 Jean-Francois DEL NERO
This program comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions;
 
libhxcfe version : 2.13.2.2
 
Input file : testα\Disquette_0001_00.0.raw
---------------------------------------------------------------------------
-                        File informations                                -
---------------------------------------------------------------------------
File: testα\Disquette_0001_00.0.raw
Checking testα\Disquette_0001_00.0.raw
File loader found : KRYOFLUXSTREAM (KryoFlux Stream Loader)
Loading testα\Disquette_0001_00.0.raw
file loader found!
Load error! error -2
Stopping HxCFloppyEmulator...

[Rouquemoute]

Hi again! Happy New Year

Last year I kept on waddling through the 8" floppies archive and I've now dumped more than a thousand of them. It's going great!

However I recently came across an odd problem. If I try loading a particular kryoflux stream disk image into HxC Floppy Emulator the latter hangs up. I've narrowed it down to 2 specific tracks (04 and 57, attached), the rest are loading just fine. I've also tried to convert those 2 tracks with hxfce to get a detailed log: everything seems to run fine, CRCs are OK, but before the output image can be written out hxfce freezes. I tried letting it run for 10 mn but no luck, it never finishes.

And it's just those 2 tracks, the other 80000 or so never made HxC freeze even though some of them had reading erros or worse. I tried with the stable HxC version 2.5.6.6 as well as the latest beta 2.5.7.2, same behavior. Here's the short output for track 04 with v2.5.6.6:

Code: Select all

HxC Floppy Emulator : Floppy image file converter v2.5.5.1
Copyright (C) 2006-2021 Jean-Francois DEL NERO
This program comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions;
 
libhxcfe version : 2.13.6.6
 
Input file : Floppy_04.0.raw
Output file : test
Checking Floppy_04.0.raw
File loader found : KRYOFLUXSTREAM (KryoFlux Stream Loader)
Loading Floppy_04.0.raw
file loader found!
Can't open config.script !
Revolution 0 track generation... First valid index position : 32979
Revolution 1 track generation... First valid index position : 85801
Revolution 2 track generation... First valid index position : 138625
Revolution 3 track generation... First valid index position : 191448
Revolution 4 track generation... First valid index position : 244272
<Freezes here>

Any idea as to what might be causing that?

[Jeff]

This is now fixed, thanks for the report :

https://hxc2001.com/download/floppy_dri ... t_beta.zip

[Rouquemoute]

Thanks! Works like a charm now.