Simon Forman's stuff about xerblin is fascinating, and I'm excited about where it's heading, and how we can incorporate ideas from eToys into Python! I like the idea of having visual meta-languages that are compiled into Python, which avoids the problems of editing Python text or parse trees directly, and can support simplified "kindergarten" languages as well as more advanced forms.
The drag-and-drop stack and code outliner ideas work well with PostScript, which is a stack based but lispy code=data dynamic language that easily supports smalltalk-like object oriented programming via PostScript's "dictionary stack". Python + Cairo is also a great platform for implementing that kind of stuff, with dynamic layout of hybrid text and outline graphics, which scales and zooms and supports direct manipulation of data structures!
Here's a paper about PSIBER (PostScript Interactive Bug Eradication Routines), a visual interface to the PostScript interpreter in NeWS, and some links to video demos, too. Sorry about the flashing and poor compression -- they're recorded off a hires Sun monitor whose refresh rate was different than the camera, and I mercilessly compressed them a few years ago when the Internet was slower.
The Shape of PSIBER Space:
PostScript Interactive Bug Eradication Routines
PSIBER Demo: (9434433 bytes)
Demo of the NeWS PSIBER Space Deck. Research performed under the
direction of Mark Weiser and Ben Shneiderman. Developed and documented
thanks to the support of John Gilmore and Julia Menapace. Developed and
demonstrated by Don Hopkins.
One problem with PSIBER was that it was too easy to make a mistake dragging and dropping, and accidentally totally hose the internals of the window system, since you were editing shared structures in the NeWS server, like classes and canvases and event handler threads! It needed some kind of read-only safety shield or edit mode switch. Like Emacs, its main purpose in life was to develop and debug itself (and secondarily other NeWS applications like HyperTIES and NeMACS)!
A regular hierarchal outliner like most gui toolkits support might be too limiting for a visual programming language. Objects might have several ways to "open" them, and links coming in as well as going out. Any object might be at the intersection of several trees or sequences at once (like the class hierarchy, and the window hierarchy, the set of instances of the same class, and an ordered list of search results).
PSIBER supported "peripheral views" that let you attach embedded visual editors and open objects in different ways. Good XML editors support a branch for element attributes as well as a separate branch for sub-elements and text. Check out the way 3D Studio Max has outlines with two kinds of branching at each level of the 3d object tree (one branch for animatable object properties, and another branch for attached sub-objects), and the way it crosses a vertical outline with a horizontal timeline. It would be nice to be able to view an object in one or more hierarchies or sequences at once (like 3dsmax's property/sub-object outline + timeline), and easily pivot the editor between different hierarchies and sequences and alternative views (narrowing it to just a timeline, or just a sub-object outline, or a free-form graph view).
I can't remember what he called his system, but Steve Strassmann did some cool stuff on Mac Common Lisp or Dylan with "butterfly diagrams" that branched out in different directions, left for incoming links and right for outgoing links.
The closest thing I could google about Strassmann's butterfly diagrams
was his infamous "Is There Toscanini's Ice Cream in Heaven?" flowchart:
Marc H. Brown and Robert Sedgewick at Brown University developed a cool visual interface to Pascal called Balsa (named after a tree, of course), which supported multiple synchronized views of Pascal programs (lexical structure outline, Nassi-Shneiderman flowcharts, dynamic scope views, pascal syntax graphs, algorithm animation, etc). But it was pretty restrictive and ungainly about how you could input and edit a program (you could not do anything that wasn't syntactically correct, and I don't think it supported drag-and-drop), so you couldn't just type Pascal code into a text editor and watch the code views update in real time.
Here's a paper by Brad Myers that mentions Balsa and lots of other cool stuff like Henry Lieberman's "Tinker" Lisp programming by demonstration system:
Brad Myers: Taxonomies of Visual Programming and Program Visualization
Marc H. Brown, Robert Sedgewick: A system for algorithm animation
Henry Lieberman: Tinker: A Programming by Demonstration System for
Beginning Programmers, in Watch What I Do: Programming by Demonstration,
Allen Cypher, ed., MIT Press, 1993.
One problem with editing programs as text while trying to maintain a visual representation, is that typing in and editing a program as text puts the program through many syntactically incorrect states, before you've closed all your parens and balanced all your blocks, and you have a horrible correspondence problem mapping between changes in the text to changes in the structure. So it's hard to have your cake and eat it too. Even Emacs Electric-C Mode can get pretty annoying when it tries to close your parens and reindent your program for you while you're typing, if you're not trained to expect it. Of course it's much easier to attempt with languages like Lisp and Python that have simple clean syntax, rather than languages like Perl and C++ with complex byzantine syntax.
PS: Some weird videos:
Here's an incomprehensible video I recorded late at night, of the freaky
"PseudoScientific Visualizer" stuff:
Pseudo Scientific Visualizer Demo: (21431618 bytes)
Demo of the PseudoScientific Visualizer and NeWS PSIBER Space Deck.
Research performed under the direction of Mark Weiser and Ben
Shneiderman. Developed and documented thanks to the support of John
Gilmore and Julia Menapace. Developed and demonstrated by Don Hopkins.
HyperTIES Demo: (3562242 bytes)
University of Maryland Human Computer Interaction Lab HyperTIES Demo.
Research performed under the direction of Ben Shneiderman. HyperTIES
hypermedia browser developed by Ben Shneiderman, Bill Weiland, Catherine
Plaisant and Don Hopkins. Demonstrated by Don Hopkins.
NeMACS Demo: (3511315 bytes)
Demo of UniPress NeMACS running in the NeWS Window System. Emacs
development performed under the direction of Mike Gallaher. NeWS user
interface developed and demonstrated by Don Hopkins.
HyperLook SimCity Demo: (49816346 bytes)
Demonstration of SimCity running under the HyperLook user interface
development system, based on NeWS PostScript. Includes a demonstration
of editing HyperLook graphics and user interfaces, the HyperLook
Cellular Automata Machine, and the HyperLook Happy Tool. Also shows The
NeWS Toolkit applications PizzaTool and RasterRap. HyperLook developed
by Arthur van Hoff and Don Hopkins at the Turing Institute. SimCity
ported to Unix and HyperLook by Don Hopkins. HyperLook Cellular Automata
Machine, Happy Tool, The NeWS Toolkit, PizzaTool and Raster Rap
developed by Don Hopkins. Demonstration, transcript and close captioning
by Don Hopkins. Camera and interview by Abbe Don. Taped at the San
Even more weird videos: http://www.donhopkins.com/home/movies/
Alan Kay wrote:
It's hard to point to any programming language for beginners that has a really great form. One thing that has consistently worked is "close to natural language but clearly not natural language". That is, it really helps if the gist-view of a program is a kind of metaphor for what it does, even if one has to think harder about the detailed meaning. For children, Hypercard was OK in many respects for the gist-view, but was too like English for both deep understanding and for programming (many children had a hard time getting past the idea that Hypercard couldn't understand and do any reasonable English sentence). This was debated endlessly in Logo circles, and Logo wound up going from a much more English-like syntax to one much more like Lisp (this was a big mistake in my view). Finding the balance between these is critical, because it governs how much brain is left to the learner to think about content rather than form. And for most learners, it is the initial experiences that make the difference for whether they want to dive in or try to avoid future encounters.
This is an essay written a while ago (1986 or so) by Richard M Stallman (RMS), about his experiences at the MIT AI Lab, and the story of the Lisp Machine Wars.
Machine Room Folk Dance, Thursday at 8pm.
Come Celebrate the Joy of Programming,
with the World's Most Unbureaucratic Computers.
(There were only five of us dancing, but we had a good time.)
My first experience with computers was with manuals for various languages that I borrowed from counselors at camp. I would write programs on paper just because of the fascination of the concept of programming. I had to strain to think of what the programs should do, because I had nothing to supply me with a goal except that I wanted to program. I wrote programs to add up the cubes of a table of numbers in several assembler languages at various times.
The first actual computers I met were IBM 360's, at the IBM New York Scientific Center, when I was a student in high school. There I quickly developed interest in language design, operating systems and text editors. Hired for the summer to write a boring numerical analysis program in Fortran, I surprised my boss by finishing it after a couple of weeks and spent the rest of the summer writing a text editor in APL.
I also quickly manifested a lack of proper reverence for authority. The whole center had been denied access to the IBM computer in the building, and we had to use slow telephone connections to the Cambridge Scientific Center. One day an IBM executive came to tell us about the work various IBM scientific centers were doing, and finished with, "Of course you all know the important work being done here." I asked him, "If our work is so important, why can't we use the computer in this building any more?" After the meeting, my friends told me they had wanted to say such a thing but were afraid of reprisals! Why? Certainly nothing happened to me as a result. They seem to have learned the habit of cowering before authority even when not actually threatened. How very nice for authority. I decided not to learn this particular lesson.
This is a message I sent to Paul Haberli a several years ago, brainstorming some ideas for an XML based scripting language. More recently, after discovering OpenLaszlo, I ran across this email, and it made me realize why I was so happy to find Laszlo, which embodies many of these ideas!
From: Hopkins, Don [mailto:Hopkins, Don]
Sent: Sunday, November 08, 1998 3:06 AM
Subject: RE: http://reality.sgi.com/grafica/motion/
I think something cool to do, would be to apply XML to describing user interface layouts and behavior and constraints.
It would have to describe the interfaces and connections between the components, as well as the graphical layout and properties.
It should have a general purpose message passing model that is independant of any scripting language, and you could also embed scripts in the xml in different language like is currently done with html.
But you should be able to do a lot without even resorting to scripts, just hooking components together.
And then if course I want to design a visual programming language in XML.
;;; -*- Mode: LISP -*- ;;; ;;; Pushy bounce window mixin ;;; Implemented for the Lisp Machine with Flavors ;;; By Don Hopkins (defflavor pushy-bounce-window-mixin (x-vel y-vel gravity friction proc delay) () :gettable-instance-variables :settable-instance-variables :initable-instance-variables (:required-flavors tv:window)) (defflavor pushy-bounce-lisp-listener () (pushy-bounce-window-mixin tv:lisp-listener))
Garnet is an advanced user interface development environment written in Common Lisp, developed by Brad Meyers (the author of the article). I worked for Brad on the Garnet project at the CMU CS department back in 1992-3.
One thing I like about Brad Meyers is that he's a strong programmer, as well as an excellent researcher, so he had a first-hand understanding of the real-world issues involved in programming languages and user interface architecture, unlike many academics who talk a lot of theory but never get their hands dirty. Brad Meyers understands where the rubber hits the road, and how important it is to have good tires.
At the time I worked on it, Garnet didn't have pretty graphics like Flash, but the underlying programming system had some advanced features that are sorely lacking from most modern user interface development environments.
Laszlo is an modern open source GUI programming system, with many of Garnet's advanced "natural programming" features like prototypes and constraints. Laszlo currently uses Flash as its virtual machine, but it's a much higher level way to program dynamic interactive web based applications, without using the proprietary Flash authoring tool.
Stanislaw Lem writes wonderful satirical introductions and reviews of imaginary books in his real book Imaginary Magnitude. Here's an actual review of a fictional introduction of an imaginary book that I'd really love to read, A History of Bitic Literature:
The introduction to A History of Bitic Literature brims over with startling ideas. The work introduced is a multi-volume survey of literature written by artificial intelligences, such as an extrapolated work of Dostoevsky's that Dostoevsky never dared to write himself, revolutionary books on physics (in this case the content is, I am afraid, rather less shocking than Lem intended it to be--I've read weirder things in orthodox textbooks--the last chapter of Misner, Thorne, and Wheeler's Gravitation comes to mind), and a mathematical work revealing that "the concept of a natural number is internally contradictory." Mentioned in passing is a procedure that can transform great philosophical systems into graphical representations that ultimately end up sold as mass-produced knickknacks.
The best two pieces, though, are the last, "Non Serviam", and "The New Cosmogony". "Non Serviam" was reprinted in Hofstadter and Dennett's book "The Mind's I". It is supposed to be a paper by a researcher into "personetics", the science of creating artificial personalities inside worlds inside the computer. The researcher has absolute power over his creations; he can bring them into existence, destroy them, and change their world at will. He is to these creatures as God would be to us. His main interest in them, therefore, is having them argue theology. Most of the paper is a debate among the personoids on what should be their proper attitude towards their creator. Their conclusion: "we shall not serve".
At the time, I was just making fun of VRML, and the people who push and hype useless standards for questionable political reasons instead of practical technical reasons. But as I read through the contraversy surrounding RSS, RDF, Atom and other syndication formats, somehow I'm reminded of AIML and BSML...
AIML: Artificial Intelligence Marketing Language