Re: Presentation on Xanalogical Data Structures

[Jeff Rush <jeff@xxxxxxxxxx>]

Greetings.  Depending upon whether you are trying to -use- Xanadu or
-understand- Xanadu, I may have some code for you.  I have Green running
under Linux (Fedora Core 1) but it's cobbled together and crashes
easily.

I distinguish use/understand because while I am able to run a Xanadu
server, and at a command-line add docs, link them and so forth, I don't
have a graphical front-end so that it is of any use to others.  The PyXI
front-end is a proof of concept, really.

What being able to run it does for me, is that I am able to dump the
internal data structures out, in both text and a graphical diagram (see
graphvis in google), in order to study the algorithms.  I can cross-link
a paragraph, tediously at the command-line, and then look to see how the
various enfilades are modified.

I then take what I learn and adjust my implementation in Python of the
data structures to match.  Believe me, the Python version is -much-
simpler than the C code, omitting garbage collection, object persistence
and disk blocking/caching, all of which I get in Python for free.

You see, while the C code may run, without a basic understanding of the
algorithms it is -very- hard to track down bugs or add enhancements.  In
the C code, the various aspects of the architecture cannot be
separated.  You can't touch enfilades w/o understanding tumblers, and
you can't change tumblers without understanding the disk blocking
system, and you can't tweak the disk blocking without understanding how
the magic constants sprinkled thru the code interact.  Tweak one think,
it segfaults in some piece of seemly unrelated code and you have no idea
why.  Sigh.

In my Python code, you can use tumblers to the exclusion of all else. 
And you can use a 1-d enfilade without tumblers; it works with integers,
currency, dates or whatever.  And you can use the Zope/ZODB object
persistence mechanism, or Python's pickles, or some other method.  Each
piece stands alone, is a Python module and comes with Python-style API
docs.

I also have an HTTP-accessible blob server; you stuff chunks of data in,
get assigned a unique URL ID and can retrieve it later.  The chunks are
internally appended to files that grow up to the size of a CDROM/DVD,
and then a new file is started.  And as you access chunks, they migrate
from old files to new ones, letting you retire content that hasn't been
used for some time without breaking the system.  You retire them by
burning them onto CD/DVD media and the server will return an HTTP code
equivalent to "migrated off".  The blob server also supports ethernet
multicasting, in that you can ask the server to begin streaming a blob
over the local LAN (or mbone).  I intend to store audio and video in my
Xanadu server.  All done via URI/URL syntax, conformant to the concepts
of REST (i.e. all objects addressable via URLs).

The blob server, while not part of the original Xanadu software, is my
vision of how Xanadu is really only intended to index information, not
store it.  I'll store the content elsewhere and let the enfilades
contain content URIs.

I also have a rough prototype HTTP proxy server, that lets you surf the
web -thru- it, and records everything it fetches into the blob server,
as a look-aside cache.  I haven't hooked in to the enfilade stuff yet,
but it will give me a way to keep track of what websites I visit and
maintain copies of those sites that change/remove content.  Subsequent
revisions of the same site would automatically be versioned using Xanadu
facilities and I'll be able to write documents that transclude to real
sources and retain references to them.

And for a Xanadu front-end, I plan to use the Mozilla platform.  It lets
you write non-webish applications using the XUL layout language and
JavaScript.  Using this approach, I can provide a platform-neutral,
click-to-install-Mozilla-plugin, Xanadu front-end that still lets you do
those non-simple-minded-browser things you want in a real hypertext
system, such as pages diffs, historical traces, transclusion leap-thrus
and so forth.

And I have no idea how yet, but I'm hoping to tie into some peer-to-peer
technology to produce the multi-server distributed aspect.  The
backend-to-backend aspects of Xanadu don't exist in either the Green C
code nor Gold Smalltalk code, nor are they explained in any detail in
the videotapes or documents I have here.  But hey, even a *personal*
Xanadu server would be awfully cool.

My goal is to leverage the existing web to leapfrog to a Xanadu-based
one.  The current web has too much inertia to just replace.  The design
won't be exactly like that in Nelson's __Literary Machines__ with Silver
Stands and such, but I am trying to remain as true to the book as
possible.

And I have to find a way to cram this into a 50-minute talk in Vancouver
for people many of whom have never heard of Xanadu.  Yeah, right. 
That's why I'm going to focus just on the algorithms of tumblers and
enfilades.

Oh, thanks for the ref to hyperworlds.  I'm working my way thru it right
now.  Lots of good content, although mostly at the high-level, which is
fun to discuss, and I'm slogging away at the low-level structures.  I
can't wait to get back to the high-level once I have a solid library of
building blocks.

-Jeff

On Mon, 2004-07-05 at 16:31, Jack Seay wrote:
> I have a big interest in understanding Xanadu related data structures. 
> My knowledge of Xanadu so far is fairly limited to high level concepts, 
> but I wish to also grasp the low level structures. Roger Gregory has 
> been logging is at the root level onto my computer, trying to get the 
> Green code running on my Mac OSX computer. I don't know how close he is 
> to getting it operational. Do you have any backend Python software that 
> might be easier to get running than the C code? After you publish your 
> code and notes, I will add links from the http://hyperworlds.org site. 
> If you would allow me to read any of it before publication, I would be 
> glad for the opportunity, and will give any feedback I am capable of.
> 
> On Saturday, July 3, 2004, at 10:11 PM, Jeff Rush wrote:
> 
> > Other than what's been on my site www.sunless-sea.net for some time,
> > no.  And my site is broken, due to some Zope infrastructure thing when 
> > I
> > moved it from one server to another.  Sigh.
> >
> > I'm pulling together all of my scattered notes and over a dozen partial
> > attempts to re-implement these data structures for this presentation.
> > I've learned a lot and just need to organize it.  After the workshop
> > I'll put up the result.  The code for what I create is all GPL.
> >
> > -Jeff Rush
> >
> > On Wed, 2004-06-30 at 08:12, Andrew Pam wrote:
> >> On Wed, Jun 30, 2004 at 06:09:54AM -0500, Jeff Rush wrote:
> >>> On August 2nd, I'll be giving a presentation at the Vancouver Python
> >>> Workshop on what I've learned about Xanadu/Udanax over the years.  
> >>> I'll
> >>> be discussing my implementations of tumblers and enfilades in Python,
> >>> and the general architecture of Udanax Green.  It's only a 50 minute
> >>> slot so it'll be an overview with a focus on how natually Python 
> >>> lends
> >>> itself to implementing xanalogical data structures.
> >>
> >> Excellent!  Do you have any documentation or code available online?
> >>
> >> Cheers,
> >> 	Andrew
> >
> >
> Jack Seay - Lubbock, Texas
> http://hyperworlds.org - Replacing the Web