Heard In The Halls

Just a link this time...in case people aren't finding it via Google (or your favorite search engine)

Heard In The Halls is a Markup Conference in August in Montreal tradition that I find particularly fun. It shines a light on one of the interesting points of Balisage.

(no, not the stupid one-liners, you cynic)

A blank couple sheets of paper where people write down things they overhear while attending the conference. It always ends up with strange statements that can be taken WAAAAY out of context. In fact, one of the heards this year was, "You can't say anything without it being taken out of context.".

Take a look at the link above and perhaps get a giggle or two. It is a VERY in-crowd kind of thing. But even some of them make sense to the mildly technologically literate.

But Wait There's More...

C. M. Sperberg-McQueen - Closing speech

XML 10 is a good time to have a look back and where we've been, where we wanted to go, where we actually went, and where we want to go from here.


There are two kinds of projects in the world

1. Barn raisings - gather materials and people, do the thing, and then everybody goes home.
2. community farming - gather materials and people, work together, but no one EVER goes home.

Standards development is more of the later than the former. Revolutions are almost never barn raisings...success means that you are in charge...and then the work is never done.

The biggest problem is Semantics.
It is all those things that we don't know how to do really well. Goal should be to isolate substructures within Semantics to fix them and thus make them no longer part of Semantics.

"the road to hell is paved with compact syntax"

Next Year in Montreal.

Text retrieval of XML-encoded corpora: A lexical approach

Liam Quinn - W3C - Paper

Text retrieval - building a persistent index that makes finding documents for things they contain more simple.

lq-test design- a text retrieval system he developed a long time ago and is seeing if it will work in the XML world.

He's doing a little demonstration of the index using "75 or 300 MB of data, I don't really recall".

I hope his paper has some of this code in it...might be interesting to examine...it appears he's making his own xml-database that is accessed without the need of XQuery...but only because the needs are so basic.

It is useful? For him it is...

Is it as useful as XQuery - XQ can't do match highlighting. XQ can't mix with broken HTML and text. But overall XQuery is more useful...but perhaps not for concordincies like he is using here.

How much to update to XML? Probably better to rebuild than retrofit.

Practical question - won't run under windows (pipelines are implemented by running a program competly to a temporary file and then giving that to another program....in Unix the programs alternate)

Freedom to constrain: Where does attribute constraint come from, Mommy?

Syd Bauman - Brown University, Women Writers Project - Paper

"This paper does not represent an enormous amount of original research." Syd in a rare self-deprecating moment. (I kid, I had a great dinner with Syd last night...I would love to chat with him for hours, but David Durbin and the German guys(TM) had some great games to play instead)

EOA - Explanation of Acronyms

Freedom to Constrain:
Whose freedom am I talking about? The typical humanities project doing text encoding.

• subject matter expert
• XML expert
• encoders (newkeying, post OCR, vendor)
• proofreaders, web designers, managers, Research Assistants

Constrain what?
For this talk he is narrowing down to an enumerated list of possible vales of an attribute.

Syd is an interesting speaker. He drives directly through topics and talks in a way that makes them accessible to me, which means they are likely easier for the rest of the room to understand. But then he throws in strange things like referencing a his asthma (due to cats) and then throws a cat image on the screen. Which both works and doesn't work at the same time. For me it is a good presentation style...it gives my brain time to catch up with the topic while he branches off to some strange digression.

Schema languages are used for constraint....duh? Oh he was comparing it to spam filters....I think I get it.

Literate programming....one source file contains the program code and the end-user documentation. Single file advantages include that you are forced to think about documentation as you write the code...that is a GREAT idea...

TEI uses this concept in the ODD file, which uses the concept of declarative constraint with formal documentation language.

Comments/Questions:
Matt Johnson - Lexus-Nexis:
How do we get this information exposed to the user without having to have an XML expert insert this into the file? Basically a nice user interface is required.

Translation between RDF and Topic Maps: Divide and Translate

Christo Dichev - Winston-Salem State University - Paper

Creation of a tool to reuse and integrate between existing Topic Maps and RDFs.

This is a good discussion of of translation between different XML tagsets. What do you drop? What matches? What is duplication in the original/combination?

Peter Brown - Pensive - Paper
He has been speaking very interestingly on the situations and contexts of context. But my system would not find a wireless connection...Hotel Europa really needs to invest in 802.11g routers/access points...so I don't have much in the way of notes. But read his paper because it made some interesting points.

And because I have enjoyed listening to Peter talk during much of the week.

Slides will be available at www.pensive.eu/uid/0203

Parser possibilities: Why write a markup parser?

Norman Smith - Science Applications International Corporation - Paper

Markup Parser: A parser that handles SGML, ML, WordML, etc transparently.

It is a non-validating parser

Reasons not to write a parser: it is a lot of work, validating parsers are even more work.

Reasons to write one: use multiple meta-languages. Learning experience both of the markup language and maybe a programming language.

Structural metadata and the social limitation of interoperability: A sociotechnical view of XML and digital library standards development

Jerome McDonough - University of Illinois at Urbana-Champaign - Paper

"What I'm really going to try and convince you of that we have a tremendous problem in the XML world...we simply do not have enough standards in XML community."

There is a considerable number of different tagsets that need to be interconnected...but there is no standard of content (dates are the common example and the one he used here).

Socio-technical system theory - States that technology and social context are not separable.

This is kind of like a porthole view into the way librarians handle (and have handled) markup technologies.

Why is XML like a rope? - give them enough flexibility and they'll hang themselves.

Solutions?
Argues for treating translations as standards in the same way we treat tagsets as standards.

Also that education has to look into communication vs control.

Wordle Image

Just a little graphical display of the past few posts. I noticed the NCBI folks messing with this website between sessions and thought I'd do the same.

Kind of cool.

An Onion of Documents and Metadata

D. Matthew Kelleher - Y-12 - Paper

Must create an electronic inspection form using XML....But it must be usable by floor shop managers, look the same as the current forms, and be able to run out to HTML, back into XML, and then out again to PDF.

But it is not just the XML file, but it is also things like image, HTML, transform files, etc. So all this had to be packaged together.

They used a Arbortext due to the interface ease of use for non-XML users.

Workflow- Online Form collects data into a XML data file which is merged into an XML file and then output to the final document.

Next Steps: need to handle form management and automatic generation of HTML. Data validation and entry need to be improved. And of course, train the users....which is ALWAYS a challenge.

Linking Page Images To Transcriptions With SVG

Hugh A. Cayless - Carolina Digital Library and Archives - Paper

Preface: "There is very little practice or theory in this talk."

There is a lot of discussion in certain areas of the blogsphere regarding the problems of text-image linking. Hugh has an upcoming huge manuscript digitization projects and might be able to use this theory/practice.

Goals:

• Create an SVG overlay of the manuscript page image
• Analyse the structure of the SVG document to detect lines, etc.
• Link the groups so produced to structures in a TEI transcription
• Display the results in a usable GUI

Inkscape (the SVG drawing tool) has a tracing tool...trace the bmp (or jpg, etc) and then plug the resulting SVG output into your XML application (freaking cool...didin't know Inkscape could do that).

He gathered tools from the open source commnity rather than writing his own monolithic thing.

potrace - takes bitmap and converts to a vector graphic image - the image has to be bitmap (using imageMagick) The output from potrace was not what he wanted exactly so he used Inkscape from the command line to convert to absolute coordinates. Then he used XSLT to do a little cleanup (adding specifically named IDs)

He then used lxml - ElementTree and numpy - the script reads in the SVG produced by potrace and filtered through Inkscape, does some filtering,detects the lines, then serializes the results back to SVG and Javascript.

Problems:

• How much can be automated? Now not much is.
• How deeply can this be analysed?
• What is the best testing mechanism?

How to tell potrace the black/white cutoff is the major sticking point right now. Also image pre-processing isn't clearly defined and automation of linking/path disposal is pretty important.

His process is very preliminary but really sounds fascinating to me. I have tons of documents from m grandmothers' homes that I am in fear of losing....to either the elements or the trashcan. Not everyone in the family finds them useful. But if I could digitize them, at least I could maintain the content, if not the source material. I will have to look into aspects of this process.

Reconsidering Conventional Markup for Knowledge Representation

David Dubin - University of Illinois Graduate School of Library and Information Science in Champaign, Illinois - Paper

Looks like a theoretical paper....but it's a Dubin presentation so who knows?

Ok, he is discussing finding the youngest (closest to the leaves in the tree) ancestor of node X and node Y using a type of formal logic programming.

He's using formal logic statements to setup a more difficult finding experience. Unfortunately for you, dear reader, I have not used logic operators in a VERY long time, so I won't be able to provide anything beyond his paper.

If you are interested in using formal logic in your Prolog demo, then you know more about this than I ever will and should just go read the paper. ;-)

Dirty Laundry : Comittee Disasters, What Happened,What We Learned

Cavet: I am typing this as quickly as I can...I am probably getting things wrong (names, dates, concepts, whatever). If it doesn't have quotes around it is it completely a paraphrase and not their actual words. Think of this as just getting the atmosphere of the session, NOT some kind of historical record.

David Orchard - Current Available for Employment:
One of the creators of Xinclude Was the BEA standards lead.
Dirty Laundry - URIs and non-use of URIs in web services

His story was about how the SOAP group would not embrace URIs. The lesson learned is that if a group doesn't buy into the feedback then there really isn't any point to give the feedback at all. Sounds to me that you can't force anything on a Working Group if they don't want it...that doesn't sound like a great way to get solid specifications. Maybe I don't want to know how the sausage is made.

James David Mason - Y-12 National Security Complex
In standards since 1981 when he attended the organizational meeting of V1(?). Charles Goldfarb was the chief evangelist behind SGML...and we wouldn't have SGML/XML without him...but...

James tells of seeing Goldfarb standing at a blackboard lecturing while two people were arguing loudly about something else, and someone else is yelling at Goldfarb. The only person with the correct response in the room was the person curled in a fetal position in the corner. Steve Newcomb is sitting next to me and states "have you ever been to a standards meeting?" me: "no" Steve: "He isn't exageratting."

What did he learn? Committee chairmen have to be stuborrn as a mule. Editors are not allowed to lock up a text and show it to no one else...in electronic form.

Lauren Wood -
Chair DOM working group
Her husband is Tim Bray, who was editor of the XML spec, Charles Goldfarb tried to get him to do something by working on her....obviously misunderstanding the professionalism of Lauren Wood.

The hardest thing to deal with is when you can't get companies to care about the topic of the committee. But sometimes when companies care too much it becomes a backroom kind of political games thing. Where people would call around to committee members and their bossses to put pressure for their particular spec change or desire.

Microsoft wanted Tim Bray to drop as XML spec Editor because he was consulting with Netscape. But Softquad, where Lauren Wood was working, took the stand that their suggestion for a replacement Eve Mahler who worked for Arbortext. What is comes down to is that people can do good technical work even if they are working for your competitor.

Mavis Cournane

In 2002 a European standards committee through Oasis where the after-market participants were non-technical and the automotive participants had no interest changing their processes. So the process went nowhere and the process died. No spec was created, it was a failed committee spec. So in 2007 a European Union legislative requirement came through that states automotive manufactures have to hold to the Oasis spec (that doesn't exist and no one agreed to at the time). So this is as a mess.

TO fix the mess...Oasis created a new type called an Oasis format...and the auto makers are basically screwed.

What she learned: Curb the powers of the EU. A sucessful standard will have to have a win-win for the key implementors of the standard. There should be some way to stop failed standards from becoming law.

Jon Bosak -
Started with formal standards on DSSL.

XML namespaces is his committe disaster. Namespaces were in play from almost the beginning...but not the forefront like we have now. Until the WWW6 conference in Santa Clara. Tim Burners Lee wanted to have a way to combine multiple repositories (?) in order to support something he called "the semantic web".

By 1997 the group tried to put the whole namespace thing behind them...but Tim wouldn't let it. By July of 1998 they had something but nothing that they were happy about. It arrived at W3C but Tim wouldn't approve it. Because they were using public identifiers for namespace implementation. In the end they were given one week to base namespaces on attributes based on an existing project of Microsoft. That's what we have here.

Jon doesn't blame Tim for this...although he wasn't happy with him at the time. The way W3C was setup at the time made everything roll through Tim which caused these problems. He also said that there is no way of knowing if the public identifiers would have worked better. But pushing it through too quickly is a real problem.

And governance matters. We need to setup committees and then wait awhile to see how things work out. Pushing things isn't working.

Also be concerned about the transparancy of the process. A lot of the problems came from hidding things.

Patrick Durusau -

A group of vendors wrote part of a copright protection standard that they wanted to push through Oasis. It was a DRM standard...a horrible piece of work. But the opposition wasn't uniform. There were people who wanted to just say "hell no" and hold the process hostage. It eventually goes on until there is a motion to disolve the TC. One of the very few TCs that just died. So the original group took part of the standard, went to ISO, and got it as a standard. But Sony turned around said,we aren't taking this standard...it sucks.

He compares this to OOXML. I don't understand the problem there. First time I heard about it was yesterday.

Microphone Comments (I didn't get them all):
David Lee - How does one get involved with a standards body?
Response: Jon Bosak and Lauren Wood - $$$ is an issue. But figure out what you are interested in, take a look and see if they will let you in, and if you really want to be in.

Simon St. Laurent -
He states that no one seems to be able to enforce their standards or even their own process or procedures? Is there any cause for hope in standards organizations. Simon seems very disturbed by the entire standards process. I wonder if he's written about it anywhere?
Response: Patrick says there have been informal conversation about reform but that's been the way it has been before.

Kurt Cagle - ISO has taken a significant hit in credibility due to the OOXML standard problem (what the hell happened?) How do you reestablish authority again after that kind of failing?
Response:
Jon Bosak - JTC1 is actually responsible but everyone seems to be blaming ISO. JTC1 is a combination of ISO and IEEC (?). So it is hard to get credibiilty when the actual culprit isn't identified correctly?
Response:
James Mason - ISO does a lot of standards that have nothing to do with software or protocalls or anything like that. So the processes and procedures may not always line up and they might be spread a bit thin at times.

Murray Altheim-
The motivation involved in becoming a part of a standard committee. Altruism is often a part of being a member. If you feel as a person that your skills can contribute to a standard that impacts millions of people then you have reached the highest part of charity.

Response: Jon Bosak- "Those of us who weren't put there as attack dogs for our company are there for the glory. Pure and simple." (I sure hope I got that out of context cause that doesn't seem to be the point Murray was making) - He clarified that by glory he means "between you and your maker", not between you and the rest of humanity.

Steve DeRose -
There is a human cost to doing what they are doing. I think that eludes back to what James was talking about earlier. Part of it is human nature (people get in a bad mood,just happens). Almost everyone on the original XML committee threatened to walk out, but no body did. Somehow they got past that and made it work. The way the committees are setup makes it difficult.

1. There is a cost in $$ and time
2. Experts may want to serve...but you serve at the will of the corporate interests
3. Lack of respect from co-workers and academic institutions.

(Seriously? The people working on these things are geniuses. How can they be considered less than by others for taking on the challenge and REALLY understanding how this all works? I don't get it.)

How might we be able to change that bias?

State of the Art of Streaming: Why W3C Xproc, W3C XSLT WGs and ISO SC34 WG 1 are looking closely at streaming

Mohamed Zergaoui - Xproc & XSLT 2.0 WG - Paper

Define Streaming:

• related to memory usage
• related to input size
• related to latency of process

It appears that the document size and the requirement to keep it in memory greatly increases the time of processing and thus the difficulty of constant streaming.

Murray Altheim: How to deal with petabytes of data in streaming? Isn't the stream being returned in an event model? Why couldn't there be a model where you take every single hit on the query and not wait until the end?
Response: Not in scope of the XML but having an entry should trigger a usage...but might be a problem of defining what WG should be handling this.

Kurt Cagle: What's happening with DSDL? The site seems out of date on a lot of points.
Response: ODF and OOXML is taking up most of the time and perhaps they can get back to work now that 2 years have been lost. He seemed tentative about saying that.

Alex Milowski: Has anybody done the work to test XPath 2 within the concept of streaming?
Response: XPath has many different subsets of stream and it is going to be a problem. But there are papers out there talking about that very subject.

Xmlsh - a command language (shell) based on the philospohy of Unix Shells designed for XML

David A. Lee - Epocrates - Paper

Why? - Unix was a radical paradigm shift and 40 years of 'progress' have eroded the core design fundamentals. Data types are not byte/line streams,tools have not evolved with the data (XML), and working with XML is WAY too complicated.

Open Source/closed development - test it out, he needs the feedback but no coding assistance.
www.xmlsh.org

Pure Java, Saxon 9, Log4J, Optional external OS commands (not reinventing the wheel)

Why not write new commands? - The shell is aging....focus is on streams and byte streams...he wants to use XML natively.

He does a bunch of live demo...can't show that, but the link above does take you to the code. If you were here you could have gotten it on a tiny 512 USB stick.

Some annoying things but the one that seems to upset him the most is that console IO is limited in Java...no pure java implementation of clear screen...

Really what this is a personal project that everyone can try out. He has it working..but he'd love to see how it works for other people. Check it out.

Questions/Comments:
Murry Altheim: Has he looked into Groovey Scripting?
Response: more interested in a scripting environment...implementing a Groovy interface seems trivial. Something missing right now is the lack of documentation for calling it from another environment...but that is something for later.

It isn't ready to scale...and David is aware of that. He's hoping to learn something from other speakers about non-serialized XML.

Alex Milowski: Have you looked at XProc?
Response: He is looking for command line interface and XProc seems to be something that requires building entire documents for the XProc. But XProc to xmlsh seems doable.

Kurt Cagle:
Have you thought of integrating with eXist?
Response: I hadn't even heard of eXist until yesterday. but I definitively am going to look into it.

Another Attendee: Unicode support?
Response: It is a goal...since it is java it is native, but the parser is utf-8 but testing hasn't occurred on the extended sets. No fundelmental reason why it can't be end-to-end and I want it to be.

Same Attendee: Memory management?
Response: Xquery serialized to a native XML dbase. eXist might assist this problem.

Topic Maps In Near Real-Time

Sam Hunting - Universal Pantograph - Paper

Update: Simon St. Laurent has more well thought out discussions of the proceedings.
Rejects the search paradigm...he only finds 'finding' interesting.

States that his blog is part of the top 5k in the world. I must figure out if I already have that blog in my reader.

He is presenting with a working Drupal application not a set of slides.

Just in case you ever meet Sam, be aware that his presentation here was a topic map of 'The Criminal Bush Administration'. So, I doubt he's much of a Republican.

I apologize in advance....I just don't get this Topic Map stuff. But he is discussing some 'markup' related things so I'll focus on that.

Looks like Sam also attended another conference this year. Eschacon happened March 28-30, 2008...and this appears to have been the official T-Shirt.

Sam was wearing it during his presentation. You'd think DFH means something in XML land where everything has an abbreviation...but you'd be wrong. DFH is Dirty F-ing Hippy

His Drupal application has validation of the wiki markup (his assertions)...which it seems that wiki-media does not have. Which seems like a serious deficiency if true.

Also something to know about Sam...he just said "I LOVE administering Drupal sites". I think that is a pretty useful thing to know about a programmer of any stripe.

He shows how much of the pieces of his Drupal application function. Basically it is a smooth application for creating a real-time (just hit submit) website.

He had a couple hours of material but Steve Newcomb keeps badgering him about wrapping up...and now we are overtime.

12 minutes ago Steve asked for 10 minutes for questions...this is funny the old guard is haranguing him into submission...I think they are going to pull the video feed any moment.

XQuery Tips and Tricks - Kurt Cagle - Nocturne

Discussion on oXygen vs. Stylus Studio. Appears to come down to price and responsiveness. Kurt also mentioned that if you are comfortable with the Stylus Studio interface then you are fine, but he likes the oXygen interface.

He's using eXist as the database. Reminds people that images and binaries are accessible from the database as well.

<Pizza Break - Thank You Anonymous Donor>

One of the coolest features: "You can store the XQueries themselves within the database."

GEEK OUT! He is using creation of characters for a D&D like game as the example....Chaotic Neutral Cleric for me!

Tip: declare function local:randchoose($ctermset as node()) as xs:string{
Defining the function as local allows you to keep that namespace private and have another namespace for the public output.

Tip: {$gender}
Embed Xquery in XML by using the {} operator.

Tip: string{$character/gender}
Remember that XPath returns element() and attribute() (and others) not text, unless specifically requested. Could easily cause problems that aren't immediately clear in a browser.

conf.xml in eXist:
setup - serielizer, scheduler, automatic XSL generation, indentation, XSLT transformer, validation mode, Xquery modules section...optional modules (turn them on -uncomment them)
then run the build routine.

Tip: in eXist -

• util, and request modules are good to get to know.
• transform allows the use of XSLT on XML in the XQuery. So use XQuery to collect the data and XSLT to transform it...easier recursion.
• sql module allows you to get connections from just about anywhere with a URI

I'm not sure there was a lot of information for people using a dbase other than eXist or people who are pretty knowledgeable about XQuery. But it was pretty cool to watch him code in front of us. I like seeing people who do their jobs well, do their jobs well.

Evening Break

Well that day at Balisage went by quickly. I wonder why? The speakers were interesting and the topics were intriguing...perhaps time really does fly when one is having fun.

I had the chance to sit at lunch with several people who had bad trips to Montreal and several who had uneventful ones...but it appears that there is always someone with a worse flying story than I have...I guess I need to travel more.

I've just finished a Skype call with my wife and daughters and I will head out in search of a meal soon. It'll have to be a quick one because 7pm is both Kurt Cagle presenting on XQuery tips and tricks and the W3Québec user group. I was looking forward to the user group discussion but I will have to choose..and since XQuery is more likely to help pay the bills...

But I do wish I could hear about using XML in cool web sites. Hopefully someone else blogs about that session.

BTW, I noticed Simon St. Laurent taking pictures today. I hope he posts them soon to flickr.

Office Suite Markup: Is It Worthwhile

Patrick Durusau - Paper

He is the Editor of ODF....so I imagine he thinks it is worthwhile.

Open Office Allows:

• We are interested in explicit semantics. But users just want to get things done.
• Interchange
• Application Independence
• Long term preservation of content

Server pack 2 for Office 2007 will have native ODF support

Lots of argument with his conclusions that everyone making their own XML is a good thing. My computer crashed (damn Windows) so I am behind on the conversation. Liam and Tommie had some great back-n-forth with Patrick.

ODF tags are presentational? Patrick says they aren't all presentational but isn't specific about it. Sounds like most of the tags are presentational...which is not how I recall XML being used to its full potential.

Wendell: "What you left out is that these formats are REALLY ugly."

This talk brought up a lot of discussion but I think it can be mostly narrowed down to:

1. How does this help me as an XML developer?
2. Microsoft Office is pervasive...it is going to be difficult to get through that?

Optimized Cartesian product: A Hybrid approach to derivation-chain checking in XSD 1.1

Maurizio Casimirri - University of Bologna - Paper

Here comes the technical stuff. I'll try to keep up but check the paper if you are really interested.

XSD has no support for co-constraints (if some type of something then this situation)

XSD 1.1 has an xs:alternative element that allows for a test of the a node.

Basically he is describing how XSD 1.1 will determine the co-constraints based on the new elements.

Now he is describing how to used both the static and dynamic methods of detecting schema errors to form a hybrid method which might expose some errors that might not appear in the document instance.

Questions:
Murray Altheim - there's no way to go up the tree?
Response: Nope, it is specified to stay within parent.<at least I hope that is what they meant>

Note: multiple xs:alternative elements are processed in order. Thus an xs:alternative element that is blank (thereby default) would confirm the test and not allow any further elements to be processed.

I think this one stumped much of the audience. Wow, that makes me feel better.