Markup

OLPC Visual Programming Languages for Education

Simon Forman's stuff about xerblin is fascinating, and I'm excited about where it's heading, and how we can incorporate ideas from eToys into Python! I like the idea of having visual meta-languages that are compiled into Python, which avoids the problems of editing Python text or parse trees directly, and can support simplified "kindergarten" languages as well as more advanced forms.

The drag-and-drop stack and code outliner ideas work well with PostScript, which is a stack based but lispy code=data dynamic language that easily supports smalltalk-like object oriented programming via PostScript's "dictionary stack". Python + Cairo is also a great platform for implementing that kind of stuff, with dynamic layout of hybrid text and outline graphics, which scales and zooms and supports direct manipulation of data structures!

Here's a paper about PSIBER (PostScript Interactive Bug Eradication Routines), a visual interface to the PostScript interpreter in NeWS, and some links to video demos, too. Sorry about the flashing and poor compression -- they're recorded off a hires Sun monitor whose refresh rate was different than the camera, and I mercilessly compressed them a few years ago when the Internet was slower.

The Shape of PSIBER Space: PostScript Interactive Bug Eradication Routines
http://www.donhopkins.com/drupal/node/97

PSIBER Demo: (9434433 bytes) Demo of the NeWS PSIBER Space Deck. Research performed under the direction of Mark Weiser and Ben Shneiderman. Developed and documented thanks to the support of John Gilmore and Julia Menapace. Developed and demonstrated by Don Hopkins.
http://www.donhopkins.com/home/movies/PSIBERDemo.mov

One problem with PSIBER was that it was too easy to make a mistake dragging and dropping, and accidentally totally hose the internals of the window system, since you were editing shared structures in the NeWS server, like classes and canvases and event handler threads! It needed some kind of read-only safety shield or edit mode switch. Like Emacs, its main purpose in life was to develop and debug itself (and secondarily other NeWS applications like HyperTIES and NeMACS)!

A regular hierarchal outliner like most gui toolkits support might be too limiting for a visual programming language. Objects might have several ways to "open" them, and links coming in as well as going out. Any object might be at the intersection of several trees or sequences at once (like the class hierarchy, and the window hierarchy, the set of instances of the same class, and an ordered list of search results).

PSIBER supported "peripheral views" that let you attach embedded visual editors and open objects in different ways. Good XML editors support a branch for element attributes as well as a separate branch for sub-elements and text. Check out the way 3D Studio Max has outlines with two kinds of branching at each level of the 3d object tree (one branch for animatable object properties, and another branch for attached sub-objects), and the way it crosses a vertical outline with a horizontal timeline. It would be nice to be able to view an object in one or more hierarchies or sequences at once (like 3dsmax's property/sub-object outline + timeline), and easily pivot the editor between different hierarchies and sequences and alternative views (narrowing it to just a timeline, or just a sub-object outline, or a free-form graph view).

I can't remember what he called his system, but Steve Strassmann did some cool stuff on Mac Common Lisp or Dylan with "butterfly diagrams" that branched out in different directions, left for incoming links and right for outgoing links.

The closest thing I could google about Strassmann's butterfly diagrams was his infamous "Is There Toscanini's Ice Cream in Heaven?" flowchart:
http://www.wunderland.com/WTS/Ginohn/cetera/heaven/heaven.gif
http://www.wunderland.com/WTS/Ginohn/cetera/heaven/index.html

Marc H. Brown and Robert Sedgewick at Brown University developed a cool visual interface to Pascal called Balsa (named after a tree, of course), which supported multiple synchronized views of Pascal programs (lexical structure outline, Nassi-Shneiderman flowcharts, dynamic scope views, pascal syntax graphs, algorithm animation, etc). But it was pretty restrictive and ungainly about how you could input and edit a program (you could not do anything that wasn't syntactically correct, and I don't think it supported drag-and-drop), so you couldn't just type Pascal code into a text editor and watch the code views update in real time.

Here's a paper by Brad Myers that mentions Balsa and lots of other cool stuff like Henry Lieberman's "Tinker" Lisp programming by demonstration system:

Brad Myers: Taxonomies of Visual Programming and Program Visualization
http://www.cs.cmu.edu/~bam/papers/vltax2.pdf

Marc H. Brown, Robert Sedgewick: A system for algorithm animation
http://portal.acm.org/citation.cfm?id=808596&coll=portal&dl=ACM

Henry Lieberman: Tinker: A Programming by Demonstration System for Beginning Programmers, in Watch What I Do: Programming by Demonstration, Allen Cypher, ed., MIT Press, 1993.
http://web.media.mit.edu/~lieber/Lieberary/Tinker/Tinker.html

One problem with editing programs as text while trying to maintain a visual representation, is that typing in and editing a program as text puts the program through many syntactically incorrect states, before you've closed all your parens and balanced all your blocks, and you have a horrible correspondence problem mapping between changes in the text to changes in the structure. So it's hard to have your cake and eat it too. Even Emacs Electric-C Mode can get pretty annoying when it tries to close your parens and reindent your program for you while you're typing, if you're not trained to expect it. Of course it's much easier to attempt with languages like Lisp and Python that have simple clean syntax, rather than languages like Perl and C++ with complex byzantine syntax.

-Don

PS: Some weird videos:

Here's an incomprehensible video I recorded late at night, of the freaky "PseudoScientific Visualizer" stuff: Pseudo Scientific Visualizer Demo: (21431618 bytes) Demo of the PseudoScientific Visualizer and NeWS PSIBER Space Deck. Research performed under the direction of Mark Weiser and Ben Shneiderman. Developed and documented thanks to the support of John Gilmore and Julia Menapace. Developed and demonstrated by Don Hopkins.
http://www.donhopkins.com/home/movies/PseudoScientific.mov

HyperTIES Demo: (3562242 bytes) University of Maryland Human Computer Interaction Lab HyperTIES Demo. Research performed under the direction of Ben Shneiderman. HyperTIES hypermedia browser developed by Ben Shneiderman, Bill Weiland, Catherine Plaisant and Don Hopkins. Demonstrated by Don Hopkins.
http://www.donhopkins.com/home/movies/HyperTIESDemo.mov

NeMACS Demo: (3511315 bytes) Demo of UniPress NeMACS running in the NeWS Window System. Emacs development performed under the direction of Mike Gallaher. NeWS user interface developed and demonstrated by Don Hopkins.
http://www.donhopkins.com/home/movies/NeMACSDemo.mov

HyperLook SimCity Demo: (49816346 bytes) Demonstration of SimCity running under the HyperLook user interface development system, based on NeWS PostScript. Includes a demonstration of editing HyperLook graphics and user interfaces, the HyperLook Cellular Automata Machine, and the HyperLook Happy Tool. Also shows The NeWS Toolkit applications PizzaTool and RasterRap. HyperLook developed by Arthur van Hoff and Don Hopkins at the Turing Institute. SimCity ported to Unix and HyperLook by Don Hopkins. HyperLook Cellular Automata Machine, Happy Tool, The NeWS Toolkit, PizzaTool and Raster Rap developed by Don Hopkins. Demonstration, transcript and close captioning by Don Hopkins. Camera and interview by Abbe Don. Taped at the San Francisco Exploratorium.
http://www.donhopkins.com/home/movies/HyperLookDemo.mov

Even more weird videos: http://www.donhopkins.com/home/movies/

FauxCast on Freedom: SimFaux as Open Source Software

Somebody asked me about SimFaux:

This is pretty funny as it stands, but can it parody things Libs don't like to talk about like Dem politicos that flipflop every other speech, or Lib talking heads that forge news on the fly?

Of course SimFaux can parody liberals as well as conservatives, but the fact of the matter is that it's the conservatives who are fucking up the country right now and sending kids to die. The liberals are powerless and aren't the ones to blame for the horrible calamity that Bush has caused. So it's important to call the people who are causing the death and suffering on their lies.

The conservatives had their chance with Clinton's blow job, and they totally blew it with their self rightious indignation over harmless consentual sex. Talk about flip-flops: Where is the furious anger and relentless calls for impeachment now?

It's high time to impeach Bush for going to war based on lies and deception, sending thousands of Americans to their death, killing hundreds of thousands of Iraqis, leaking classified information about a CIA employee out of revenge for her husband telling the truth, breaking the law and subverting the constitution by spying on American citizens, his miserable failure with Hurricaine Katrina, his deep shameless corruption with Haliburton and Abramoff, etc. If you think none of that is any worse than a blow job, then you're a part of the problem.

I just started programming SimFaux at the beginning of this month, and the HuffingtonPost Contagious Festival has a 15 meg limit on size. But after the festival concludes, I'll host it on my own web site without any size limitations, put up a whole lot more content like characters, sound bites and videos, and release the source code as Open Source Software so you can modify it and put your own stuff in. But right now I have to cut it to the bone to stay within the 15 meg limit.

"FauxCasting" is a cinematic leap beyond podcasting and blogging, because it integrates keyword tagged text, video, sound bites, simulated characters and interactive widgets together in a way that the viewer has control over what they see and hear. You can experienced the joy of telling Bill O'Reilly to Shut Up and Go Away! The keyword tagging enables the characters and videos to interact with each other, and results in surprizing juxtapositionings, like Frank Zappa and Dick Cheney finding common ground by using the same cuss word that begins with F.

I'm inspired by ideas like Will Wright's "Designing for Emergent Behavior" and Chris Trottier's "Tuned Emergence" and "Design by Accretion", where the more content you put in, and the better you tune it, the closer you get to a nuclear reaction.

Sims Designer Chris Trottier on Tuned Emergence and Design by Accretion

"If a game isn't tuned, it's a drag, and you can't stand to play it for an hour. The Sims and SimCity were "designed by accretion": incrementally assembled together out of "a mass of separate components", like a planet forming out of a cloud of dust orbiting around star. They had to reach critical mass first, before they could even start down the road towards "Tuned Emergence", like life finally taking hold on the planet surface. Even then, they weren't fun until they were carefully tuned just before they shipped, like the renaissance of civilization suddenly developing science and technology. Before it was properly tuned, The Sims was called "the toilet game", for the obvious reason that there wasn't much else to do!"

I will soon release the source code and content of SimFaux as Open Source Software, so you can see how it works, create your own characters, sound bites, video clips, talking points, surveys, games, etc. If you want to do that yourself, you should become familiar with XML, and you should start learning OpenLaszlo, which is the free Open Source XML/JavaScript based language it's written in.

http://www.OpenLaszlo.org

Here's another more down-to-earth example of what you can do with OpenLaszlo:

http://www.LaszloMail.com

Have fun!

Maximizing Composability and Relax NG Trivia

Here's some interesting stuff about the design and development of Relax NG:

James Clark wrote about maximizing composability:

First, a little digression. In general, I have made it a design principle in TREX to maximize "composability". It's a little bit hard to describe. The idea is that a language provides a number of different kinds of atomic thing, and a number different ways to compose new things out of other things. Maximizing composability means minimizing restrictions on which ways to compose things can be applied to which kinds of thing. Maximizing composability tends to improve the ratio between functionality on the one hand and simplicity/ease of use/ease of learning on the other.

Clark describes the derivative algorithm's lazy approach to automaton construction:

I don't agree that <interleave> makes automation-based implementations impossible; it just means you have to construct automatons lazily. (In fact, you can view the "derivative"-based approach in JTREX as lazily constructing a kind of automaton where states are represented by a canonical representative of the patterns that match the remaining input.)

The Relax NG derivative algorithm is implemented in a few hundred elegent declarative functional lines of Haskel, and also in tens of thousands of lines and hundreds of classes of highly abstract complex Java code.

Clark's Java implementation of Relax NG is called "jing", which is a Thai word meaning truthful, real, serious, no-nonsense, and ending with "ng".

Comparing the Java and Haskell implementations of Relax NG illustrates what a wicked cool and powerful language Haskell really is. The Java code must explicitly model and simulate many Haskel features like first order functions, memoization, pattern matching, partial evaluation, lazy evaluation, declarative programming, and functional programming. That requires many abstract interfaces,, concrete classes and brittle lines of code.

While the Java code is quite brittle and verbose, the Haskell code is extremely flexible and concise. Haskell is an excellent design language, a vehicle for exploring complex problem spaces, designing and testing ingenious solutions, performing practical experiments, weighing trade-offs, and writing succinct, elegant, mathematically rigorous specifications that actually work. Haskell code is useful as a blueprint for implementations in less luxurious languages like Java.

Relax NG: Design-by-Inspired-Individuals vs. Design-by-Committee

In The State of XML, Edd Dumbill explains the secret behind the success of Relax NG:

Incidentally the RELAX NG success can equally well be framed as a case of design-by-inspired-individuals vs. design-by-committee as much as it can be seen as a OASIS vs. W3C thing.

Relax NG Compact Syntax: no to operator precedence, yes to annotations!

James Clark is a fucking genius! He’s the guy who wrote the Expat XML parser, works on Relax NG, and does tons of other important stuff. Relax NG is an ingeniously designed, elegant XML schema language based on regular expressions, which also has a compact, convenient non-xml syntax.

I totally respect the way he throws down the gauntlet on operator precedence (take that you Perl and C++ weenies!):

There is no notion of operator precedence. It is an error for patterns to combine the |, &, , and - operators without using parentheses to make the grouping explicit. For example, foo | bar, baz is not allowed; instead, either (foo | bar), baz or foo | (bar, baz) must be used. A similar restriction applies to name classes and the use of the | and - operators. These restrictions are not expressed in the above EBNF but they are made explicit in the BNF in Section 1.

You can translate back and forth between Relax NG's XML and compact syntaxes with full fidelity, without losing any important information. Relax NG supports annotating the grammar with standard and custom namespaces, so you can add standard extensions and extra user defined meta-data to the grammar. That's useful for many applications like user interface generators, programming tools, editors, compilers, data binding, serialization, documentation, etc.

Here's an interesting example of a complex Relax NG application: OpenLaszlo is an XML/JavaScript based programming language, which the Laszlo compiler translates into SWF files for the Flash player. The Laszlo compiler and programming tools use this lzx.rnc Relax NG schema for the OpenLaszlo XML language. This schema contains annotations used by the Laslzo compiler to define the syntax and semantics of the XML based programming language.

The schema starts out by defining a few namespaces:

default namespace = "http://www.laszlosystems.com/2003/05/lzx"
namespace rng = "http://relaxng.org/ns/structure/1.0"
namespace a = "http://relaxng.org/ns/compatibility/annotations/1.0"
datatypes xsd = "http://www.w3.org/2001/XMLSchema-datatypes"
namespace lza = "http://www.laszlosystems.com/annotations/1.0"

The a: namespace defines some standard annotations like a:defaultValue, and the lza: namespace defines some custom annotations private to the Laszlo compiler like lza:visibility and lza:modifiers. Thanks to the ability to annotate the grammar, much of the syntax and semantics of the Laszlo programming language are defined directly in the Relax NG schema in the compact syntax, so any other tool can read the exact same definition the compiler is using!

To show how truly simple and elegant it is, here is the snake eating its tail: The Relax NG XML syntax, written in the Relax NG compact syntax:

Laszlo OPML Drupal Taxonomy Site Map Browser

Here's a simple Laszlo application, based on Max Carlson's tree control from myLaszlo. It's an OPML Browser that lets you browse the site map generated by my OPML Site Map Based on Drupal Taxonomy. It's an easy to understand example of OpenLaszlo programming -- here's the source code.

OPML Site Map Based on Drupal Taxonomy

I've just implemented a Drupal module that renders an OPML site map, based on the sitemenu taxonomy tree.

Each taxonomy term has an outline node, which contains child nodes. The first child shows the term's description, and has a link to the term's web page. Subsequent children are links to articles directly related to that term, followed by recursive outlines of sub-terms.

Dave Winer put my OPML site map feed into his OPML World Outline, where you can browse it as html, and OPML Search has automatically indexed the content on my site! That's quite cool, and it's just the tip of the iceberg -- many interesting applications are possible!

Applying XML to Describing User Interface Layouts and Behavior and Constraints.

This is a message I sent to Paul Haberli a several years ago, brainstorming some ideas for an XML based scripting language. More recently, after discovering OpenLaszlo, I ran across this email, and it made me realize why I was so happy to find Laszlo, which embodies many of these ideas!

From: Hopkins, Don [mailto:Hopkins, Don]
Sent: Sunday, November 08, 1998 3:06 AM
To: paul@isdn-balla.corp.sgi.com
Cc: dhopkins@maxis.com
Subject: RE: http://reality.sgi.com/grafica/motion/

I think something cool to do, would be to apply XML to describing user interface layouts and behavior and constraints.

It would have to describe the interfaces and connections between the components, as well as the graphical layout and properties.

It should have a general purpose message passing model that is independant of any scripting language, and you could also embed scripts in the xml in different language like is currently done with html.

But you should be able to do a lot without even resorting to scripts, just hooking components together.

And then if course I want to design a visual programming language in XML.

Designing to Facilitate Browsing: A Look Back at the Hyperties Workstation Browser

Designing to Facilitate Browsing: A Look Back at the Hyperties Workstation Browser

By Ben Shneiderman, Catherine Plaisant, Rodrigo Botafogo, Don Hopkins, William Weiland.

Human-Computer Interaction Laboratory
A.V. Williams Bldg., University of Maryland
College Park MD 20742, U.S.A.

Abstract

Since browsing hypertext can present a formidable cognitive challenge, user interface design plays a major role in determining acceptability. In the Unix workstation version of Hyperties, a research-oriented prototype, we focussed on design features that facilitate browsing. We first give a general overview of Hyperties and its markup language. Customizable documents can be generated by the conditional text feature that enables dynamic and selective display of text and graphics. In addition we present:

  • an innovative solution to link identification: pop-out graphical buttons of arbitrary shape.
  • application of pie menus to permit low cognitive load actions that reduce the distraction of common actions, such as page turning or window selection.
  • multiple window selection strategies that reduce clutter and housekeeping effort. We preferred piles-of-tiles, in which standard-sized windows were arranged in a consistent pattern on the display and actions could be done rapidly, allowing users to concentrate on the contents.

HyperTIES Hypermedia Browser and Emacs Authoring Tool for NeWS

HyperTIES is an early hypermedia browser developed under the direction of Dr. Ben Shneiderman at the University of Maryland Human Computer Interaction Lab.


HyperTIES Browser (right) and UniPress Emacs Multi Window Text Editor Authoring Tool (left), tab windows and pie menus, running under the NeWS Window System.


HyperTIES Browser NeWS Client/Server Software Architecture.

Syndicate content