Short: Memory Efficient RAM Disk for BeOS.
Author: agmsmith@rogers.com (Alexander G. M. Smith)
Uploader: agmsmith@rogers.com (Alexander G. M. Smith)
Website: http://members.rogers.com/agmsmith/
Version: 20040403 (unfinished - but useable)
Type: System Files / File System Addons, Development / Example Code
Requires: BeOS 5.0+

AGMSRAMFileSystem stores files in RAM rather than on disk.  This makes it a bit
faster than a real disk, more memory efficient than a RAM disk that simulates
disk sectors (like AGMSRAMDiskDevice), but also means it forgets its contents
as soon as you turn off the computer or reboot.  There are also a few
experimental features you won't find in regular file systems.  It works, even
down to the complexity of live update notification and BeOS query parsing,
evaluation and optimisation, but is still unfinished due to a lack of time (or
money :-).

Installation and Uninstallation

To install it, move the ARFSv0 file to your
/boot/home/config/add-ons/kernel/file_systems directory (you can drag it to the
supplied symbolic link to do that).  To uninstall, delete that add-on from
there.  The only other program you need is AGMSFileSystemPacker, you can throw
away the rest of the files if you want to.  You can optionally run
AGMSFileSystemPacker and click the "Install MIME Types" button.  Maybe also
then add a related "operation completed" sound effect to the system sound
effects.

Using It and Unfinished Gaps

To manually create a RAM disk, use the AGMSFileSystemPacker program.  Select
the Create radio button and hit the "Create FS" button.  If you wish, you can
change the volume name and maximum size before hitting "Create FS".  Create
several volumes (use different names!) if you wish to.

For automatically creating a RAM disk when you boot up, first move
AGMSFileSystemPacker to the /boot/home/config/bin directory.  Run it once and
change the settings to use a top level directory of "/RAMDisk" with a maximum
size of 500000000 bytes (exiting will save the settings).  Put your initial RAM
disk contents in the RAMDisk.zip file and decide on which indices you need (the
ones here are for Pineapple News and Mozilla).  Then use StyledEdit to add this
single long line to the end of the /boot/home/config/boot/UserBootscript file:

(sleep 5 ; AGMSFileSystemPacker CreateFilesystem ; cd /tmp ; cd /RAMDisk ; mkindex "PINEAPPLE:msgid" ; mkindex "BEOS:APP_SIG" ; unzip -n /boot/home/RAMDisk.zip ) &

Internally, AGMSFileSystemPacker just uses the mount system call with extra
arguments that the file system needs (which the standard mount command doesn't
let you provide, even though the extra arguments are just a text string).  I
was thinking of writing a replacement mount command that would let you give it
the extra arguments string, but time doesn't permit...

To get rid of the RAM disk volume, unmount it (right click on the disk icon or
use the command line).  Or reboot.  Note that the desktop umount command will
fail if you have system debug logging turned on because printing out the log
messages slows it down and the desktop Tracker becomes impatient (use the
command line unmount instead).

If you turn on the system debug console (an option in the BeOS boot menu), it
will print out lots and lots of syslog debug messages, at 19200 baud.  This
will make it very slow.  It will still work, and you can examine the log by
either using the serial port or this Terminal command:
tail --retry --follow=name /var/log/syslog

Nifty Features

* Multithreaded design.  There is a multiple readers / single writer semaphore
  on each open file and no global semaphore.  That means that multiple programs
  can use the whole file system, and only ever wait if they conflict with a
  program that's trying to write to the same file or directory.  You can get
  deadlocks, but it has timers and retry attempts to try to get around them.
  So if you get an EDEADLK error code, you're really unlucky, or more likely,
  have found a bug.

* Has AGMS links.  These are kind of like hard links mixed with MacOS aliases
  and a twist of lemon.  An AGMS link allows you to put a file in two or more
  directories - so you can put a photo of your aunt's cottage in the Aunt
  folder and the Cottage folder.  It also supports putting directories inside
  multiple other directories, and even properly handles the case where a
  directory is (via a few other parent directories) inside itself by doing a
  graph traversal when it needs to rename or delete directories.  The extra
  parent directories show up as "..." for the first one, "...." for the second,
  and so on.  If you want to make an AGMS link then use a Terminal command like
  "ln /boot/home/Blah/OriginalFileName LinkName" to do it (you need the -d
  switch to hard link directories).  See the AGMSLinkExplanation.txt file or
  the comments at the top of the AGMSRAMFileSystem.c file for an explanation.

* Has live update notification.  Like BFS, whenever you change something an
  update message is sent to interested applications.  Unlike BFS, it sends them
  every second or two when a file is being written to so that you can see the
  file size change in Tracker.  It also sends notifications for changes to the
  index directories, which may overload Tracker (making it miss some updates
  since BeOS BLoopers by default can only have a backlog of 100 update
  messages).

* Has BeOS queries.  This means code to parse the queries into subexpressions,
  then more code to optimise the order of evaluation, then yet more code to
  test attribute changes against the queries and send live updates.  The parser
  language has been enhanced a bit, with <> and >< operators to look for files
  that have or do not have an attribute.  The full query language is now
  documented, see the comments inside AGMSRAMFileSystem.c or File System API
  Notes.txt.

* Has Indices visible as directories.  Now you can see what the indices are
  doing.  Look inside .Indices/name and you will see the names of all the files
  in the RAM disk.  Each name is shown as a symbolic link to the file (Tracker
  gets confused when direct links are used, so until it's rewritten...).  The
  hexadecimal number after the attribute value is there to make the names
  unique, in case you have several files with the same attribute value or have
  an empty string as the attribute value.

* Has a multiple keyword index.  If your attribute is "Make Money Fast", it
  will add the three words separately to the index, so you can search for
  "Make" or "Money" or "Fast" and it will find your file.

* Doesn't have queries as directories.  I wanted to add magic directories where
  their contents would be the files that matched a query, with live updates as
  it changed.  The query would be stored perhaps as the directory name, or
  maybe as an attribute on the directory.  But time ran out...

Glitches

Tracker doesn't recognise the file system so it doesn't know where to put the
trash.  Need to fix Tracker and the BeOS FindTrashDirectory(DiskVolume)
function, which means waiting for OpenBeOS I guess.

If you make the maximum size too big, you could add enough files to use up all
the memory.  It uses kernel space memory, of which about 500MB are free (if
your swap file is big enough or you have that much real memory).  You may also
run into the BeOS memory allocation bug, where allocating hundreds of megabytes
(of virtual pages) takes a while and causes other programs that are trying to
get memory to fail (guess their deadlock timer was too short).  That could
happen if you copy several hundred megabytes to the RAM disk in one operation
(unlikely, Tracker and "cp" break it up into smaller operations).  Most BeOS
software doesn't check for memory allocation failure, this includes the
application server, and the OS, so a system crash will likely happen soon after
the memory is used up or the big copy operation temporarily makes memory
allocation fail.

If you find any bugs in the RAM disk, please let me know, and I'll file them
away for when I have time to work on it.

Programs and What They Do

AGMSFileSystemPacker is unfinished.  It is a combination of an archiver (like
zip) and RAM disk manager.  I planed to have it recreate a RAM disk from an
archive (double click on a .pafs Packed Archived File System file to make the
disk appear).  But I didn't get that far.  Instead, it can only create an empty
RAM disk.  It can also compress a directory tree (a file system is also a
directory) into an archive file, optionally using zlib compression (same
compression technique as zip).  Unlike zip, there is no file size limit!  See
AGMSFileSystemPacker File Format.txt for a description of the file format,
which is streamed plain text (thus no size limit) if you don't use compression.
However, it can't fully restore an archive yet - doesn't restore links or
indices (just files, directories and attributes).  The unpack to RAM disk
choice is ignored.  Also I haven't copied the compress-a-directory-tree into
the file system code, so that unmounting a RAM disk doesn't yet automatically
save the contents into a .pafs file.

AGMSRAMFSMounter is a test program which compares the operation of different
file systems against a test suite.  If you run it from Terminal, you can save
the test results.  It was used for making sure that AGMSRAMFileSystem behaved
the same way as BFS, or better.  Pick one of the two first options to choose
the file system to test, or mount your own file system at /Test.  Transient
messages will flash for a few seconds, final results stay on the display.

mkindex is an enhancement of the old BeOS mkindex utility.  Put it in /bin if
you want to replace the old one.  It's mainly useful for making an index with
the multiple-keywords feature.  Give the "mkindex" command with no arguments if
you want to see a page full of its verbose help info.

AGMSRAMFileSystem is released under the GNU Lesser General Public License.  See
comments in AGMSRAMFileSystem.c for details.

- Alex (Ottawa, February 2004)