The Ligature Dev Diary
(archived)
This is the development diary of my now-discontinued auto-linking wiki software, ‘Ligature’. These posts were collapsed into this single page for permanent archiving, and are ordered from oldest to newest.
2009-01-16 12:51:00
Development of Ligature has begun. But what is Ligature?
I write a story and post it on my website. It's simply text on a screen, very Web 1.0. Ligature takes my story and turns every noun into a link automatically. Each link leads to another story, relevant to the linked noun itself, with yet more links. In this way what we end up with is a web of interlinked stories, like a wiki except there is no need to manually specify where a link goes.
Why is it called Ligature? The best-known ligature is perhaps the A-E ligature (Æ). With the letters so close together, they make a different sound than if they were apart. I am wondering if, by interlinking individual, dissonant stories, if each story changes the meaning and perception of the whole.
Tonight I finished the first task of production, which was to compile a list of as many nouns as I could. The list is here, for unlimited download and distribution by anyone who needs it. As far as I could find on the internet, this is the first such list of its kind, though it has been apparently much requested before. If only it has been as simple as telling dictionary.com to dump every noun it knew into a file, instead of me having to trawl ESL websites to compile the most common nouns.
This noun list is the hingepin of the automatic linking script, but before any programming is done I will need to write a few design documents to focus on the details of implementation.
=======
2009-01-18 11:54:00
I completed the public areas and administration frontend of Ligature in one day, over maybe eight hours of coding. I had forgotten a lot of functions, even simple operators, and had to look them up at every step. Luckily, I had refactored the code for Writer’s Block two or three times previously which had made the code somewhat modular, and I was thus able to lift many elements such as basically the entire backend, RSS script, installation script and many others with only a few changes.
As it stands:
The major hurdle now is the autolinking handler. Normally it is simple to scan through a body of text to find something. Writer's Block does it every time you comment. Three times, in fact. What complicates this, then, is the sheer number of search queries the software will have to make.
Writer's Block scans a comment three times for suspicious material that indicates a spam comment. To do this is uses a simple list of blacklisted phrases that are individually looked for in the comment. The lists total up to, let's say, 200 entries. That means that 200 individual searches, one after the other, are performed on every comment before it is posted. The same system will be used in Ligature to find nouns.
The thing is, the search list for nouns is already 2,332 entries long. That is 2,332 individual, one-after-the-other searches that must be performed every time a story is created or edited, almost twelve times as many searches as for a comment. And a comment is so short, while a story can be so long.
The delay created by the scan can't be helped, so I will need to make its use more efficient. By performing the scan as few times as possible, its delay will be bearable.
I also need to program a handler that removes autolinks so that you can edit plain text, instead of working around <a> tags at every fourth word.
======= ## Design time ## **2009-01-19 08:43:00** After making a host of silly accidents and a bit of forgetfulness (I changed a variable name and why my code wouldn't publish correctly stumped me for about an hour, for example) I think it's time to stop pushing hard at the programming. Tomorrow I will get more done, but for now I'll just kick back and design a little.Like in Paper Tiger, I'm going for an olde-worlde papery feel. I'll probably use a few more images this time; I found a nice paper texture, you see.
I must say though, using Writer's Block and having it simply work without having to worry about it, it's amazing. Only by going through development hell all over again can you appreciate a finished piece of software. I don't know how I even got through programming Writer's Block. The thing went through eight versions, two of them were large-scale code refactorings. Writer's Block has more than 5,050 lines of code, and counting. Ligature is about 80% finished, and hasn't reached 1,000 lines yet.
======= ## All programming finished ## **2009-01-24 12:30:00** Today all production programming has well and truly finished. No more bug testing, no more fiddling with regex. What's left is to add features and functions where I need them.It is, I think, as it always will be.
======= ## Combinatorial explosion ## **2009-02-01 10:01:00** There's a term I use in game design and programming, it's a cooler-sounding version of exponential growth. It's called combinatorial explosion, which does a much better job, I think, of explaining itself.Let's say you have a note in your game, cool. You can read it, you can drop it, you can give it to people. Now add a lighter. The note burns, right? Does the player keep the ashes? Now add a bottle of water. Can you soak the paper in water? Will it light if it's wet? What if you light it, and then douse the flames? Can the bottle be melted? Where does the water go?
Ad infinitum. The combination of each object (or feature) creates an explosion of consequences that must be addressed.
In terms of Ligature, the explosion occurs not only when a reader clicks on links to read new stories, which may lead them to reconsider or reorder what has already been read, but on the author's side too, every time a new story is written. The nouns from that new story are added to the Pending list, and the list quickly explodes into a giant wall of words.
To date I have written twelve stories. To date I have 111 Pending nouns ready for use. This means just less than ten nouns per story, and I haven't created any custom nouns yet.
The explosion would quickly make the Pending list unmanageable. It would be difficult to create a consistent interlinking of stories when there are so many words that need to be addressed.
So tonight I'll need to start making the list sort in order of importance. In this case, the importance of a noun increases the more times it is used in stories. A name, say, would be more important than a toothbrush, because presumably you'd be using the name in more stories than you would mention a toothbrush. By sorting in order of importance, the author is able to create as many cross-links as possible with every story.
Later that night! Wow, that was too easy. I'm waiting to see what horrible crippling bug must have been introduced.
======= ## RSS and refinements ## **2009-02-05 10:16:00** RSS feed was fixed. There was a WHERE condition in the SQL that shouldn't have been there.I've also allowed Ligature to recognise standard extensions of words aside from regular plurals: ing, d, ed. Now you can make gerunds and Ligature will know what to link! Breaded to bread, front to fronting, etc.
It's nice that I've been able to identify and rectify these and other issues as I find them, instead of when it's already out in the hands of end-users. Writer's Block was definitely a study in embarassment, with I think the first three releases being plagued by bugs and shortcomings because WB had such a giant scope, and I was a young kid who was impatient to start writing already.
======= ## Manual is up ## **2009-02-09 11:18:00** The Ligature Online Manual is now up.I've also redesigned the Ligature Project's page. It's a man lying on concrete; even more story-related than the last redesign.
======= ## One month after ## **2009-03-31 11:44:00** It's been a month now since I launched Ligature for public use. I am happy that I haven't thought of anything else I need to add ever since, and that is the way it should be.The Ligature Project (the writing project) is on blocks, and instead I am now working on One-Sentence Stories.
======= ## 1.5 released ## **2009-04-26 13:01:00** In preparation for the launch of Gradually, But With Love (domain is still clearing), I've released Ligature 1.5 which fixes some small bugs and adds some new features, the most awesome being that you can hide text from Ligature's autolink parser now!If you want to add some other content like a blog post or something under a story, but don't want Ligature to search it for nouns, or to autolink it, you wrap it in an Exclusion tag like so:
This is visible (!))) but this is hidden! (!)))
Ligature will leave the text that's between those tags completely alone.
======= ## Wildcards ## **2009-04-26 23:54:00** I don't know why I didn't wildcard the noun search before. I long suspected that there was a bug in the autogenerate links function, and it finally surfaced last night when a word that I knew had a story wouldn't autolink to it. The problem was created by the regex I used: noun\|? would not find |noun|. Easily fixed with \|?noun\|?. ======= ## Next and Previous links ## **2009-04-30 13:01:00** I've added Next and Previous links to the story.htm template. I just stuck it into the Ligature 1.5 download, so if you've already upgraded, just download it again and replace your index.php and admin/control.php files. In admin/control.php you'll find four new options for controlling the way your Next and Previous links will appear. Use {URL} in place of an actual URL in your link code when editing these options:
My Crafts
My Software
My Reading
My 3D Printer
About
Contact
Sitemap<a href="{URL}">
To actually make the links show up, just add {PREV} or {NEXT} to your story.htm template where you want the links to appear.
======= ## Another damn hotfix ## **2009-05-03 02:50:00** I just found a bug that only surfaced when you didn't specify a story to show whenever someone hadn't visited Ligature before. Of course the way these things work is that I never had the option disabled, and Ligature shipped with the option disabled by default.Just download the whole ligature-current.zip archive again and replace only your index.php (that's the root index file, not the one in your admin/ folder). I've also coded it so that Ligature won't write any cookies to your users' browsers unless you specify that new users should see a particular page.
======= ## RSS bugs fixed ## **2009-05-15 06:17:00** I fixed an RSS bug across both Writer's Block and Ligature that made feed links faulty if the installation was in a subdirectory.Just download the installation .zip again and replace you existing rss.php with the new copy.
=======