Re: Miscellaneous topics, mainly floppy disk support

spotUP wrote:

there are plenty of sites that hosts games with permission from the authors... let's not go into this boring discussion shall we?...

Perhaps it is "boring" in your country, but the U.S. is a real cause of concern.  Copyright infringement cases against retro-computing companies is becoming more and more common.  Yes, there are certainly some authors who have released their works to public domain.  However, I can tell you with absolute certainty that others are actively protecting their copyright and have turned in many hosting sites.  Recently, there was an action filed against some eBay sellers for distributing their works as "public domain" without permission.  I have been trying to contact as many companies as possible to BUY their copyrights, and the answer I am always getting is "No, it's not for sale - and it's not public domain".

So - with that in mind, to stay 100% legal for a vast portion of the world, there really needs to be a way that FPGA Arcade can take a legal backup and use it.  Technically, the only method allowed under U.S. law is by using an exact copy, one that has the copy protection intact.  The only way this is possible is with a flux image.  You are welcome to look up the laws yourself.  You are fooling yourself if you don't believe that there are copyright holders who are not taking this seriously.

Thanks for chiming in, Mike.  D64 is good for data disks, as is ADF.  Neither will handle even basic protection checks but work great for data transfers.  Let me know if you need an Amiga sided driver for handling the hard drive emulation.  I wrote dozens of device drivers for the Amiga.  Since I write in assembly code, they are extremely fast and we can do things like invoke special traps if needed to get access to the FPGA side of things.  I did that quite a bit to bridge the Amiga and Mac emulation.


Re: Miscellaneous topics, mainly floppy disk support

JimDrew, I am talking of legal stuff. I'd say the format has to gain popularity, it is unrealistic to believe that everyone will generate the flux files by himself - especially for Freeware, public domain, demos and so on. Thus, this stuff is in PRG and D64 format (and maybe g64). Obviously one can ask the author(s) and provide a flux-based files or there are tools to generate them from existing formats? But advertising any file format is not really a topic of the Replay community...

For testing I prefer demos as these usually take most out of the HW. If you have already some scp files available for cbm/1541 disks, it would help as I can't generate them (and I would not be sure if they are correct). Maybe you can put them on the download section on cbmstuff and prepare a tool converting existing formats to this new format (especially to make the format more popular).

Back to the technical discussion, I think I need to explain why this topic is much more complicated that it seems to be:

G64 (and SCP) would be easier to support from HW side for the stream itself, but its overall format require a processor to parse it (especially for SCP). Thus it is good for SW-emulators (and obviously invented by a SW guy thinking in SW data structures), but really bad for a pure HW implementation (requiring simple, fixed structures):
* mixing 8bit/16bit/32bit word information (at least G64 use a consistent, strict 32bit word aligment)
* can not be easily read in a "generic" way using fixed block sizes at fixed starting points (needs LUTs to handle all this)
* format handling complexity (g64 use at least only two "static" table definitions followed by track data where a FSM could be still used to handle it, scp use luts pointing to other luts in tracks --> this is "killing" for a pure "on the fly" HW support)

D64 (and afaik ADF) is is set up strictly block-based, thus it is very easy to process in HW directly, but it needs slightly more work to generate the coded stream from binary data. --> The ARM just needs to send blocks from the sdcard to the FPGA, which can be done very generic as well. Thus it will be supported first.
Same is valid for PRG, BIN and CRT files - they are just binary streams, no problem for generic ARM support and handling in FPGA.

Now to the Replay setup:

Taking into account that the Replay board shall support multiple cores and has to support different formats (including HDD formats, by the way) from beginning (this is a different goal than all the other projects out there: primarily their intention was a setup supporting ONE core first and thus only formats related to it), I see three ways to support this approach general:

1) use the ARM on the Replay (must be done by core developer, nearly no generic Replay framework support)
Pro: toolchain and development setup already available
Con: would require new FW releases and ARM flashing for each and every format one would like to use (incl. bugfixes)
          core is always delivered with a specific ARM firmware, multiple ARM code setups will be the result
          in a w.c. a user has to flash the ARM each time he wants to test cores with different formats supported by the ARM
          --> maintenance nightmare sad

2) use a generic ARM setup for the file reader and use a soft core on FPGA (generic Replay framework setup + user coding)
Pro: updates based on bin files via sdcard together with the core
Con: wastes (in  my view) quite a lot FPGA ressources just to handle a file format
          complicated to handle the "generic" handshake to get data from sdcard to ARM (no fixed sizes of data junks)
          beside this additional complexity, it requires yet another SW development project (probably own toolchain,...)
          a core developer must use the xilinx stuff and the additional SW toolchain (and the ARM toolchain for bugfixes there)
          still a lot to do for maintenance of the framework

3) use a generic ARM setup for the file reader (handled by Replay framework) and some handler on FPGA (handled by core)
  Pro: works well for block based formats
          generic support possible, no change on ARM FW needed for different formats
          core developer only needs Xilinx stuff (and maybe ARM toolchain for bugfixes)
          easy to maintain the framework
  Con: some overhead to be set up on FPGA to generate the block number for ARM requests
           will work for D64, ADF, PRG, CRT, BIN only (and maybe G64 and similar complex ones)

Thus my favourite to support this files in future is to improve the Replay-App concept I started with (actually used only for sdcard-based ARM flashing) to run this format handlers in SRAM on the ARM, but this will definitely take some time - having a solid base setup is more important for now. --> this would correspond to solution 1) with some generic approach to avoid ARM flashing and keep a consistent FW setup. The HW would contain a FIFO filled by the ARM (and translated from the file if necessary and running just in RAM of the ARM) with stream data the core can process. The HW just inform the ARM about head movements. Sounds simple, but is a LOT of work to do...

For now we focus on 3) which is nearly no overhead for ADF and D64 and can be handled in a generic way on ARM. It might even work for G64 (when setting a larger block size and load the two LUTs for offsets and speed handling once at startup with a HW state machine) but as I saw up to now it won't work for SCP (as it needs a more complex handler). So this format needs to wait until we have time for a better framework setup.


Re: Miscellaneous topics, mainly floppy disk support

Track lengths vary with G64 files, just as they do with flux.  Flux data provides the exact bitcell time for the data separator.  Instead of shifting in the bits from the G64, you use the bitcell times... It's actually much simpler and automatically handles everything that G64 images don't provide (like multiple densities on a single track). Bitcells are currently fixed at 16 bits, and will probably remain that way.  I made a provision in the format to use any width, but I do not see a case where this will ever be necessary.  The flux data is the very same data that comes off the head.  If you want 100% compatibility, flux is the only way.