An user was interested in opening a large multi-save magnetic tape image file
with jSAVF, so I have made a few changes to support that use-case. It also
enabled me to add limited support for optical disk image files (ISO9660 only
for now, not UDF) since these may also contain multiple save files.
The tape imaging tool he was using produces a tape image file in the AWSTAPE
format (see Description, Hercules-390 Emulator),
which inserts 6 byte block headers before each tape block.
This tape image format is not ideal given it describes the lengths of the
previous and current block with 16-bit words, when modern tapes appear to have
256KB blocks or more. This will lead to overflows, so for example a block size
described as 0xF000 in the block header can mean either 0x0F000, 0x1F000,
0x2F000 or 0x3F000 for a tape block size of 0x40000 (256KB) described in the
ANSI X3.27-1978 tape file header labels.
jSAVF works around the issue by looking at these possible offsets for an
AWSTAPE block header with a previous block length corresponding to the block
length of the current header modulo 65536, and checking that the header flags
look correct. If unlucky there may be data at one of these alternative offsets
which looks enough like the real block header, so in that case jSAVF will bail
with an error. If you're in that case I may improve upon the current
strategy to distinguish the real header from data (by looking a the next
headers), but since it seems to work well enough for now I'll leave it as
it is unless someone has a tape which can't be read this way.
As scanning all the alternative block headers of a multi-gigabyte tape image
file to find all the save files written on it can be a lengthy process, jSAVF
will generate a compressed tape file index next to it named
"xxx.jsavf_awstape_index.bin" where "xxx" is the original tape image file name
when that takes more than 3s. The index contains the tape labels for all files
on it, and offset tables to be able to quickly convert a save file offset to a
tape file image position. This lets jSAVF open the file nearly instantly the
next time, and doesn't take too much space (~1MB index for a ~30GB file).
When there are multiple saves in the file, jSAVF will now prompt the user to
select the one to open. For batch scripts this is handled using a new
jsavf:openMultiSaveFile(path, selector) API which calls a user-provided
selector function with the list of save files found in the tape or optical
image file and expects the selected one to be returned. This allows a script
to select it based on the save size, name, or order. The JEXL dependency was
updated to v3.3 to make this possible, so if you find incompatibilities in your
batch scripts please tell me about them so I can see if it's something I
can fix. I've exposed a few more classes to the batch API which help
converting text to int or long, and updated the batch API documentation
accordingly.
This version also fixes two bugs:
- Some extractions failed to determine the saved item structure because of an
error in the way jSAVF was computing the position of the first section when the
item header length was exactly 4096 bytes.
- Some large CSV extractions with CHAR VARYING or CLOB or BLOB fields
couldn't be performed when the total amount of data in these fields was
above some limit, which happened on tables with many records or with enough
data in them. jSAVF now correctly handles tables with many records, but may
still have issues with BLOB / CLOB whose values are very large. Don't
hesitate to reach out to me if you're confronted with such an issue.
The CSV extraction of BLOB fields was also modified to extract their values
as a hexadecimal string, because putting raw binary inside a UTF-8 CSV file is
not very useful.
I've also updated the jSAVF dependencies and the embedded JRE bundled
in the installable version for Windows. So jSAVF now needs a Java 21 version or
better.
If you encounter any issue with this version don't hesitate to reach
out to me.
Anerty's Lair - News
