A big suggestion

[Benjamin Fallenstein <b.fallenstein@xxxxxx>]

Hi everyone,

sorry for the delay -- it was harder to find a telephone line than I had
expected. Now, I still can't exactly ask questions because I have to get
familiar with the new code that has shown up in the meantime first.
Pondering about my programming system applitude, the formatted text
applitude and other applitudes, though, something else has formed in my
mind which, I think, deserves consideration on your part: a suggestion
for a system with the function of systems like OLE or OpenDoc, linking
different applitudes together. Besides the general description of the
suggestion, I've also included a detailed specification of a possible
set of interfaces inside this system. Probably I haven't thought of
everything, as this is a very first draft; but if the system itself
makes sense to you, tell me your problems with the spec, and I will try
to make a better one.

Hope this is of help,
- Benja 



ABSTRACT

A recommendation to applitude programmers to split up their applitude's
data into "things," as small as possible, which adhere to a common
interface (not unlike OLE, but better), so that they can be reused in
other applitudes.



DESCRIPTION

In the draft for Cybertext, you (Tuomas & Vesa?) say: "The interesting
thing about applitudes is that they can be designed to be connected to
each other in a more fine-grained fashion than current embedding
frameworks such as Bonobo, KParts, or OLE/COM." I think such connections
can happen on two levels: custom applitude connections, which are
designed to connect a special applitude A to a special applitude B; and
connections within a broader framework, where all applitudes have a
common interface through witch they can communicate with each other. Of
course you can always connect a cell which has a special meaning to
applitude A to a cell which has a special meaning to applitude B on a
dimension of your own, but these connections will show in neither
applitude's views; this is exactly what we need, though. For example, we
want to include a picture in a text; ZigZag should empower us to use ANY
kind of picture supported by ANY gZZ applitude, not only those the text
applitude was designed to support. This, after all, is what we mean by
breaking up monolithic applications, isn't it?

So, what we need for gZZ is something with the same function as KParts
or OLE/COM; not with the code overhead, as we're going to use the ZZ
model (that's gZZ's main advantage here), but with the function of
inserting one applitude's stuff into another applitude's stuff. We want
it to be more flexible than those examples, though, and we can have that.

My suggestion is to represent data structured into things -- that's a
bunch of cells refered to by a handle. This is an extension of the
"everything is a cell" concept, not something to complement it; the idea
is that the handle cell is the "outside" of the thing, whose "inside" is
the bunch of cells. The problem is that not everything can be a cell in
the current gZZ system; a bunch of cells can never be one cell, and each
nontrivial data structure needs a bunch of cells, not only one. A very
basic operation a handle should support, for example, is that marking
the handle marks all cells of the thing as well (or optionally does);
this way, a user can send the whole item by eMail easily. However, how
this exactly works is outside the scope of this suggestion draft.

By Ted's definition (cited from memory), a thing is an item that can be
used (accessed) by a ZZ programmer; and an item is anything that is of
interest for a ZZ user. Mine is just an operational definition, but I
MEAN the same.

I suggest that a text vstream, for example, be a thing. It can contain
other things, like pictures (this is an extension of the vstream
concept, but I think a good one). On the other hand, it can be contained
in other things, like a spreadsheet. It can be contained in *multiple*
other things, by cloning the handle. As the handle is the "outside" of
the thing, cloning it creates a second "outside," with the SAME
"inside." This is not exactly Euclidean, but that shouldn't be a
problem. :)

Things inside things don't need to be a different "kind" of thing than
the thing they're in. (Possible name for things inside things:
subthings.) A text vstream can contain another text vstream, and this is
my suggestion for your "structured text," Tuomas. You can make different
chapters (text vstream things) and put them into a book (another text
vstream thing). However, you can also clone them (their handles) into
your TODO list, or you can put them into weird constructs (yeah, you can
clone a chapter into itself, although it doesn't make much sense to ME).
You can as well use them as subthings of totally different things, and
this might be more exciting. Using them in an item cloud (more general
form of a text cloud) comes to mind.

Now, a thing knows different ways to "visualize" itself. (How exactly
this works is part of the specification, below.) This concept is broader
than it sounds: I consider a sound output "kind" of a visualization, for
example. That way, you can have a sound vstream inside another sound
vstream the same way as with text vstreams, and they will play right.
But the same sound vstream items can have a visualization on the screen;
this is a real visualization, then. Maybe the more general concept
should be called "accesibilisation," but this sounds to complex and
artificial for me.

A visualisation (my abbrevation: viz), thus, is a way to bring a thing,
as a whole, to the user. Screen visualizations will commonly be rasters
(although you could say that special-case gZZ flobs are part of the visualization).

An applitude will now define a number of "thing types." A text vstream
applitude will only define the text vstream thing type, but more complex
applitudes might define more complex thing types. All things of the same
thing type are connected to a list of possible visualizations for this
thing type. There's no formal definition; but it's implicit that a thing
connected to a visualization have to be readable by the visualization.
All the parameters the visualization needs have to be there, etc.

A visualization is only referenced from the structure; the actual
visualization is the Java class which does the layouting on the screen,
prepares the sound output, etc. These visualizations are grouped into
viz typees, which, unlike thing types, have to be well-defined in the
Java manner of an Interface (or an abstract class). Now, when the viz
code of a thing finds a subthing (e.g. a chapter vstream in a book
vstream), it calls gZZ to get a certain visualization of this subthing.
A text vstream requests a text layout viz, for example; this viz is
defined to be able to layout a text line-by-line (so that it can be
billowed). If there's no such viz available, it requests a viz layouting
the whole subthing into a rectangular area. This works with pictures; it
also works with virtually everything else, like sound things which are
visually represented, etc.

The box viz is also used to layout a thing into a cell raster. That's
true, a thing's outside (it's handle) isn't shown as a label, but as a
visualization of the thing. This seems most useful, as often you don't
want to assign a label (I don't think of my series of dog pics as dog
pic 1, dog pic 2, dog pic 3 -- that's a ideosyncracy invented by the
file system). If you want labels or icons anyway, no problem. There will
be a label applitude which defines a label thing type, visualized as a
label, but containing as a subthing the thing which is meant by the
label; pressing the right key will jump right into editing the subthing,
just like it would if you'd use the thing directly. Same goes for icons.

Each thing (more accurately: each thing type) has a list of possible
visualizations, ordered by preference. Also, there is a list of standard
"viz converters:" the most important example being the converter from a
text viz to a box viz. Such a converter is never invoked by the user;
the system uses them internally. The converter implements the interface
of the viz type they convert to, but the implementation internally calls
the viz converted from. The converter mentioned above is invoked all the
time, over and over, to convert from the standard text viz of non-handle
cells (representing the cell's contents) to the box viz used by the cell
rasters. (Although possibly this is to slow and there'll be an
efficiency hack which goes around visualization search in this special
case.) When an item doesn't define the viz type requested, gZZ searches
its converter list for a conversion to the desired viz type, and if it
finds it, it returns an instance of the converter, which "masks" the
"real" viz.

An interesting special-case viz is the HTML viz. This doesn't put
anything on the screen, but generates an HTML document representing the
thing. This can be used in conjunction with Ted's idea of http://
requests returning a HTML page, while xu:// requests return the cells
(or other Xanadu data). The HTML viz would be invoked to create the
http:// results.

Another interesting viz, which would be helpful for my applitude, would
be an "execution viz," which doesn't visualize something on the screen,
but instead executes the thing. If the interface for this is general
enough, it allows combination of different gZZ programming languages.

If no viz of the requested type and no conversion can be found, the
layouter requesting the viz must define what to do; it might inform the
user that there was an error, or it might just ignore the subthing.
Maybe it could render the handle cell's contents instead, if any.

Some UI considerations: A text thing might want to blend in other things
containing text seamlessly; that means that the insertion cursor, when
it it moved over the edge to another thing, is just passed on to that
thing. This is represented by moving the zz cursor so that it points
into the subthing. With pictures, on the other hand, it might be
desirable not to pass on the cursor, but to put the cursor onto the
whole thing; this is done by moving the zz cursor to the handle of the
subthing. Which of these behaviors is used is in the hands of the thing
which has the insertion cursor before it is moved.

Things are useful for drag&drop between applitudes. When you start to
drag, the gZZ flob reacting to the mouse event is called for a cell
which is dragged. This can be just a cell, or it can be a handle to a
thing. You can drop a thing virtually everywhere. The two places where
you can store things you don't need at the moment are clouds and the
cell rasters themselves. Clouds are useful when you don't need a special
structure, just want to store a group of things. If you want to order
things in arbitrary structures, you drag them on a cell raster. Here,
every cell has four sections (left, up, right, and down); they are
divided by two diagonal lines, from the upper left corner of the cell to
the lower right, and from the upper right to the lower left. If you drop
a thing on one of the sections, it is inserted in the associated
direction from the cell: dragging it in the down section inserts the
thing on Y+ from the cell (or rather, inserts the thing's handle). When
you shift-drag a thing, its handle is cloned. (The clone of a handle is
considered a handle of the same thing.) Note that you can drag&drop
non-handle cells the same way, including cloning.

Something that deserves some consideration are the editing modes. I
would like both in-place editing and own window editing, but in-place
editing should be the standard and own window editing should have to be
requested by the user. This, by the way, would be helpful for normal
cells as well, when you use a raster in which the text you need doesn't
fit: you'd press one key and a new window pops up in which you can enter
a word-wrapped text as long as you desire.

In-place editing poses a tricky problem, though: which things can
currently be edited and which can't? If I have a tree of text vstream
things, then I want to be able to edit the different parts of my text
seamlessly: moving from one text to another with the insertion cursor
should just do what I expect, go to that other text, and shouldn't mark
the other text first and require me to hit "edit" before I can change
it. On the other hand, if I include a small icon in my text, I might not
want that: I might want to treat the icon just like a character in my
text, instead of "jumping right into it." The thing is that from there,
hitting the Right cursor key might not get me out of the icon, but move
my cursor INSIDE the icon somehow -- not exactly what I desire.

First, I would say that when a thing is currently editable, then the
view's cursor accurses a cell inside that thing; if a thing is accursed,
but not editable, then the view's cursor accurses the handle of the
thing, not something inside the thing itself. This has complex
implications for cursor-centric views, though. It is probably necessary
to have a second cursor when in editing mode, an edit cursor. For
example, we might have a text vstream thing, and we might put a handle
to it somewhere. When we go to the handle cell (which shows the vstream
as a whole) and hit the edit key, the handle cell with the vstream gets
editable. We have an insertion cursor. But this insertion cursor needs
to be inside the vstream! Still, the view should be centered on the
handle cell. We want to see the outside of the vstream, not its inside.

So, during editing, I suggest we have a second cursor, the insertion
cursor or more generally edit cursor. This cursor is inside the thing
that is currently edited. This thing gets all key events -- actually,
the viz that has drawn the thing on the screen is called for a list of
key bindings, which are then interpreted by gZZ. There's one exception,
the ESC key (or something similar), which escapes the editing mode.
(Another key might be specified for this in the system configuration,
but it is important that this key overrides all thing-specific key
bindings.) When the edit cursor is moved inside the thing, now, and
"hits" a subthing, the thing decides if it is "given" to the subthing,
or if the thing wants to keep it. If the latter, hitting TAB or some
other key specified in the thing's key bindings will give the edit
cursor to the subthing, making it editable. (Hitting TAB when the edit
cursor is not on a subthing's handle should give the edit cursor back to
the next higher thing, positioning it on the handle for the current
thing, making the current thing non-editable.)

Something I would like are menus specific to the thing currently edited.
When I edit a text vstream, the menu should show a cut command; when I
edit a figure, the menu should show an insert circle command. The menu
should always be the context menu for the thing that has the edit
cursor. How the view presents the menu to the user is the business of
the view; which cell is the beginning of the menu is the business of the
thing's viz.

The beauty of the "everything is a thing which can be visualized" system
is that it unifies applitudes much more than OLE could ever dream of. It
also makes currently existing and planned applitudes easier to
implement, because it boosts reusability. And, to me, it seems to be
very consistent with the overall ZZ philosophy.

I'd greatly appreciate discussion! ;)



SPECIFICATION

SPEC (Visualization): A viz extends a viz type (abstract class which
implements ZZVisualization). It has a constructor which takes as an
argument the "thing home cell" (see below) it shall visualize as its
only parameter. 

SPEC (Visualization type): A viz type is an abstract class which
specifies an interface for visualizing a thing -- e.g., the box viz type
specifies a function which is called with a ZZGraphics object and the
boundaries in which to layout. The viz type class contains a static
final member of type Class, called viztype, which is set to the viz type
class itself; this way, the viz type of subclasses can be found out quickly.

SPEC (Visualization converter): Like a visualization, but takes a viz of
a different viz type as the parameter of the constructor. Masks the viz
it gets at initialisation. Has a static final member "fromviztype,"
which is like viztype, but specifies the viztype the constructor takes
as its argument.

SPEC (Naming convention): It is recommended that viz type class names
have the form ZZ<TypeName>VizType and individual vizzes have the form
ZZ<ThingName><TypeName>Viz, for example ZZCellBoxViz for the box viz of
a cell. Inner classes may leave the zz out, and, when it is clear to
which thing type they belong, even the thing name; a module only dealing
with text vstreams may just have a BoxViz inside, because its clear that
the box viz for a text vstream is meant.

SPEC ("Thing home cell"): This is the "first" cell of a thing. This is
given to a viz at instantiation, and it also is used to determine a
thing's possible visualizations.

RATIONALE: Why don't we use the handle? Because the handle is not the
thing itself; it is a cell that contains the whole thing.

SPEC (Handle): The original handle of a thing is connected to the "thing
home cell" on d.handle. The thing home cell is the negward neighbour of
the handle. The handle cell has no posward neighbours on this dimension.
All clones of this original handle are considered handles of the same thing.

SPEC (Visualizations for a thing): From the "thing home cell", go to the
posend on d.visualization. From here posward on d.visualization-set (or
d.2?) is the list of possible visualizations. Empty cells are ignored;
if a cell except the first one is connected along d.visualization, count
the visualization set at the posend of that rank as visualizations of
the thing, too. Each visualization is represented either in the form
"ClassName," where the class with this name resides in fi.iki.lukka, or
"ModuleName.InnerClass," where the module resides in fi.iki.lukka.module
and InnerClass is a static subclass.

RATIONALE: It is expected that applitudes will connect all things of the
same thing type, i.e. the same internal structure, on d.visualization.
If they want to group visualizations, they can use the recursive
structure which goes along d.visualization, again.

RATIONALE (Reference by string): This probably ought to change at some
point, but it's consistent with the rest of the system right now.

SPEC (Viz searching function): This function (residing SOMEWHERE, maybe
ZZUtils) finds the best viz of a given type for a given thing. It starts
with the posend of d.visualization from the "thing home cell", and
progresses on d.visualization-set (or d.2?). If it finds a cell with a
posward connection on d.visualization-set (d.2), it uses the same
algorithm recursively on the posend of that rank. The first viz whose
viztype member equals the requested viztype is returned. If no viz
matches, the viz convertion search is invoked (see below). If there
still isn't any match, null is returned.

SPEC (Viz conversion search): In the system configuration, there's a viz
conversion list listing viz convertion classes, which are referenced in
the same way as the viz classes for things. Some efficiency hack ought
to store this converters internally. Now the "shortest path conversion"
is searched: preferably only one conversion, if necessary two, if that
doesn't work three etc. I'm sure you can come up with a better internal
algorithm for this that I.

"SPEC" (Drag&Drop events): This should be consistent with the existing
flob code, which I'm not familiar with. Drag&Copy events (commonly
invoked by Shift-Drag) on a thing clone the handle of the thing. (The
dragItem is always the handle, the thing's "outside," not its "inside.")

SPEC (ZZBoxVizType, rectangular view viz type): This should have a
raster method taking ZZGraphics, x, y, prefwidth, prefheight, maxwidth,
maxheight parameters and returning the width and height actually used
(which may not be bigger than maxwidth and maxheight, even if the viz
has to be clipped.) The different width and height parameters can be set
to -1, which means "any width" and "any height;" but not both the
prefwidth and prefheight parameters may be set to -1. The max parameters
may be omitted (in which case the same function is called internally,
with maxwidth and maxheight set to prefwidth and prefheight). The pref
params mean "preferably not greater than."

RATIONALE: This is intentionally very open, as basically *everything*
should be optionally layoutable in a rect. The "not both prefs -1"
constraint is used for everything arbritarily scaleable, like vector
graphics. If you have a vector graphic with a width/height ratio of 2
(i.e., two times as wide as high,) and get a prefwidth or prefheight,
you can layout the stuff (and if the computed width or height is greater
than the maxwidth or maxheight, you layout it smaller). If you have *no*
values given (-1 -1 -1 -1), you can't layout the vector graphic, because
it has no intrinsic size. If your thing has a fixed width and height, on
the other hand (like bitmaps), you'll probably ignore prefwidth and
prefheight, and only scale or clip if maxwidth or maxheight are smaller
than your pic's w or h. -- You might also decide that it doesn't make
sense to layout this pic this small, in which case you return {0, 0}.
This is helpful when things are very "deep" inside the screen.

CONSIDERATION: An optional "lighten" parameter might be useful, which is
a fraction range 0 to 1; if it is one, an empty box is drawn. This would
be used for the vanishing rasters.

"SPEC" (ZZTextVizType): This should use the existing LineBreaker etc.
stuff, with witch I'm not yet familiar, as mutch as possible.

SPEC (Edit cursor): A view which is in editing mode has an edit cursor
which accurses a cell inside the thing that is currently edited. For
efficiency, edit cursors (and edit cursor stacks, see below) may be
implemented virtually; this needs some modifications, but these should
be straight-forward.

SPEC (Edit cursor stack): Every time the edit cursor is given to a
subthing, the old edit cursor is put on an edit cursor stack. Each view
in edit mode has an edit cursor stack (which might be empty), a common
list of cursor-cargo cells. When the edit cursor is given to a subthing,
the subthing viz is called for a new edit cursor, returning its cargo
cell; this is attatched in place of the old cargo cell, which is
inserted on top of the edit cursor stack. When the edit cursor is given
back to the next higher thing, the current edit cursor is destroyed and
the last edit cursor is popped from the edit cursor stack. If there is
no older edit cursor on the stack, the view quits editing mode.

RATIONALE: A thing can have many handles (when cloned), and more than
one of these handles might be placed on the same screen. For example, I
could clone a figure two times in a thing cloud. Now, when the edit
cursor is inside that thing, which of the clones on my screen do I edit?
Oh, possibly I could show the insertion cursor in each one of them, but
when I quit editing the figure, to which higher-level thing do I give
the edit cursor back? To the one from which I reached the thing, of
course, and keeping track of this is the responsibility of the edit
cursor stack.

SPEC (Requesting edit cursor): A viz may or may not accept the edit
cursor. If it does can be found out by calling its boolean
acceptEditCur() method. If it accepts the cursor, ZZCursor getEditCur()
can be called, which creates a new edit cursor inside the thing. A viz
type might specify other parameters to call getEditCur with; for
example, ZZTextVizType specifies getEditCur(int pos), so that the edit
cursor is positioned at the beginning when the insertion cursor comes
from the left, and at the end (-1) when the insertion cursor comes from
the right.

SPEC (Requesting menu cell): To get the start cell of the thing's menu,
the ZZCell getMenuCell() method is called. It may return null, in which
case the thing doesn't have a menu. Menu cells may be requested only if
a viz accepts the edit cursor.

SPEC (Requesting key binding lists): A viz must define a method ZZCell[]
getKeyBindingLists(). The cells returned by this method are the starting
cells of key binding lists (the events are found poswards on d.2, and
the corresponding actions are found poswards on d.1 from the events).
The key binding lists are used in order: if the event isn't found in the
first list, the second one is searched, then the third one etc. Key
bindings lists may be requested only if a viz accepts the edit cursor.

SPEC (Requesting sub viz): To request the visualization of a subthing
inside the thing of a visualization, the ZZVisualization
getSubViz(ZZCell handle) method is used. The handle given is the handle
for the subthing, which must be placed inside the thing.

RATIONALE: When the view wants to request the menu cell or key bindings
list of the thing its edit cursor is in, it needs to call that thing's
viz. Because the sub vizzes are generated by the higher-level vizzes to
visualize their thing's subthings, the higher level vizzes need to be
called for the lower-level vizzes.

SPEC (Finding the viz associated with the edit cursor): Start with the
highest level viz as the current viz. For every handle accursed by a
cursor on the edit cursor stack, starting with the first item put on the
stack: call the getSubViz method of the current viz with this handle.
Set current viz to mean the viz returned by getSubViz.