Comments on: LMX 1.0 released

By: Jason

Jason — Thu, 29 Mar 2007 00:33:11 +0000

Wow, everyone but Simon seems unnecessarily negative about this. Anyways, thank you for making this available, it's a very useful idea.

By: Peter Hosey

Peter Hosey — Thu, 15 Mar 2007 06:06:51 +0000

Yeah, I've seen a couple of other LMXen on Google since releasing 1.0 (I didn't look previously—oops). Here's hoping that nobody makes a beef about it.

By: Alastair

Alastair — Thu, 15 Mar 2007 00:38:24 +0000

Interesting choice for name. Did you know that there is already another application called LMX? It is used for code generation and reading/writing in an XML Beans fashion. Was somewhat confusing for me as we actually use LMX in our development (not this particular LMX that is).

By: Dethe Elza

Dethe Elza — Wed, 14 Mar 2007 22:04:07 +0000

What I've done in similar circumstance is to put each XML snippet on its own line and use standard line-oriented tools (tail in the shell, for instance) to trim off the ones I'm interested in. For parsing snippets of XML you can use two approaches. The first is to wrap them in a single tag before passing them to your parser. The second is useful when you want to get the whole list but don't want to have an end-tag (so you can keep appending to the file): add the starting tag, omit the end tag, use the SAX api for parsing and simply catch the error about the missing end tag. Either way its pretty easy to get last-N messages.

By: Peter Hosey

Peter Hosey — Mon, 12 Mar 2007 20:09:44 +0000

Neither POSIX nor File Manager nor NSFileHandle has an insert mode, regardless of underlying file-system. I don't know about any other operating systems' file I/O APIs.

By: Håkan W

Håkan W — Mon, 12 Mar 2007 19:29:02 +0000

Oh, that's interesting. Offtopic random question: do you know if this is something HFS+-specific, or is the problem a general disk one? Are there file systems that do have a "prepend" mode? I admit my knowledge in that kind of territory is not the strongest.

By: Peter Hosey

Peter Hosey — Mon, 12 Mar 2007 05:18:58 +0000

No, because then we would need to rewrite all the previously-written files every time (file I/O does not have an insert mode, only overwrite). This would get very expensive for very long logs, and some people leave the chat open all the time.

By: Håkan W

Håkan W — Sun, 11 Mar 2007 16:35:45 +0000

Can't you just add every new message to the beginning of the file, putting them in newest-first order?

By: Peter Hosey

Peter Hosey — Sat, 10 Mar 2007 17:24:03 +0000

Because the XML format satisfies all the requirements on our LogFormatIdeas page.

By: Eugene Morozov

Eugene Morozov — Sat, 10 Mar 2007 14:49:41 +0000

Why use XML at all in this case? I'd say that using XML for logs and then writing another parser for it, is like creating problem out of nothing and spending significant effort to solve it.

There're plenty ways to store logs that wouldn't require backwards XML parser or XML parser at all.

By: Peter Hosey

Peter Hosey — Tue, 06 Mar 2007 06:12:01 +0000

Ah—the email just came in. Guess it got held up for some reason.

By: Peter Hosey

Peter Hosey — Mon, 05 Mar 2007 22:00:43 +0000

Matt Thomas: (building on Joshua Gooden's comment) It's theoretically possible that an implementation of XSL could do the job lazily (as lazily as it can, after analyzing the style-sheet to find out how lazy that is), but I don't know if any such implementations exist. Besides, I don't think any of us thought of using XSL for this. :) (Also, interestingly, your comment didn't get emailed to me. I wonder how that happened.) Joshua Gooden: Yes, LMX is lazy. It reports start and end tags and character-runs as it finds them, and the parser allows any such callback to pause the parse and resume it later. Adium takes advantage of this by pausing the parse after n messages, and then releasing the parser (never resuming it). This is subtly guaranteed by the API (see comment for -pause in LMXParser.h). Simon Rolfe: Well, LMX is a library, not a tool unto itself, but yes, LMX could definitely be used to create such a tool. :)

By: Simon Rolfe

Simon Rolfe — Mon, 05 Mar 2007 21:27:34 +0000

This could be hugely useful as a tail replacement for XML-formatted log files of any sort - seeing as more and more services are logging to XMLish structures, I can see LMX becoming a really handy tool.

By: Joshua Gooden

Joshua Gooden — Mon, 05 Mar 2007 20:39:23 +0000

That solution still requires that a DOM tree be built and parsed in order to re-order the elements. DOM trees are hugely memory hungry beasts, and though I haven't looked at the LMX source, I would imagine that it lazy parses the XML doc, saving hugely on memory and CPU requirements.

By: Matt Thomas

Matt Thomas — Mon, 05 Mar 2007 20:35:51 +0000

Another solution could be to run it through xslt. The syntax would go something like this:

(hopefully wordpress will render this correctly)