The Wongery

D minus four days and counting...

Holy cow. Hoooooly cow.

I have been saying as recently as yesterday that I was really going to make an effort to get my custom namespaces, and subspaces, implemented before the hard launch on January 1. But I'm going to be completely honest here... I wasn't at all confident that was actually going to happen. Even after I successfully created the RandomRules extension, creating an extension to implement the subspaces the way I wanted them seemed like a Herculean task (if Hercules was known for computer programming) well beyond my meager programming abilities. I was definitely going to try, but I was far from sure I'd be able to do it.

I don't want to get cocky, but it's looking like I may actually be able to do it.

Before I expand on that thought, though, let me first go back and address a matter left unresolved in the previous post. (This wouldn't be a blog post by me if I didn't digress for several paragraphs on an unrelated matter. Well, yes it would, just not a typical one.) At the time, I said I hadn't got my Wikitext extension for phpBB working yet, but that I was "cautiously optimistic that I'll figure out how to get [it] working by tomorrow". So let me relate how that went.

Oh, yes, sorry; this is going to be another blog post largely about computer programming. If all goes well, though, it'll be... well, not the last, but the second to last, at least for now. Anyway, as usual, of course you can skip this if you're not interested (though of course as usual I couldn't stop you from doing that anyway; you don't need my permission). I'm not entirely sure why I feel the need to go into such detail about my programming travails; to experienced programmers, all of this is probably boring because it's trivial and obvious, and to those who know nothing about programming it's probably boring because it doesn't mean anything to them, so the only people who might possibly derive a morsel of interest from any of this are those near my exact level of novice programming skills. And even they probably won't get much out of it, because I'm explaining it all badly. But I feel inexplicably compelled to continue anyway, so here we go.

Anyway, getting a BBCode tag working in phpBB involves creating a filter that replaces the content of the tag with other text. But after numerous attempts, I couldn't figure out a way to replace said content with dynamically generated HTML code, which is what I needed to do to parse it as wikitext. If I tried to put HTML in the text replacing the tag's content, phpBB automatically escaped that HTML, turning all the angle brackets into >s and <s and rendering it nonfunctional. And while there were ways to replace the tags themselves with other text, I couldn't figure out a way to do it with content generated by a custom function.

It took far later than it should have (or than it no doubt would have for a more experienced programmer, anyway) for the inspiration to strike me that maybe I could use another route altogether. Just as MediaWiki facilitates extensions by allowing them to be called by particular "hooks", phpBB does the same thing with "events", which are... as far as I can tell basically functional equivalent to MediaWiki's hooks but by a different name. Maybe there's a subtle distinction I'm missing between events and hooks (or even a not at all subtle but blatantly obvious distinction) and there's a concrete reason why MediaWiki calls its entry points hooks and phpBB calls them events, but if so I failed to entirely grasp that distinction but seem to have been able to make some use of them anyway. Regardless, I'd been using a hook—I mean event—called core.text_formatter_s9e_configure_after that comes into play, in the words of one tutorial, "every time a text is rendered, after the HTML is produced". This seemed like the appropriate place to deal with the bbCodes, and indeed applying filters worked just fine there—as long as I didn't try to include dynamically created HTML. But if the phpBB code was later undesiredly escaping my HTML, then maybe there was a later hook I could use, after that took place. I mean event. Instead of doing everything all at once, maybe I could use the first hook—I mean event—to convert the BBCode into some HTML tag, and then have some other event that triggered after the page was otherwise fully parsed to look for that tag and replace its content with a parsed version. (The repeated "hook"/"event" mixups here aren't a bit; I am honestly kind of reflexively writing "hook" each time, and then later looking back and realizing I should have written "event". Okay, maybe they're sort of a bit, in that I could of course have just replaced the word instead of leaving "hook" in and writing "I mean event" afterward to call attention to my confusion.)

The event I found that seemed best suited to the task was one called core.text_formatter_s9e_render_after, which, per the online event list, can be used to "[m]odify a rendered text", which seemed like exactly what I wanted to do. Now, having replaced the [wiki][/wiki] tags with <span class="wikitext"><span>, I just had to get the text of the post, search through it for those tags, and parse anything in between. There were at least two ways I could think of to do this. My initial thought was to do this by just manipulating the text directly with string functions, chopping it at the position of the tag, parsing the pieces that needed parsing, and assembling the results. But then it occurred to me that this might give me trouble with other <span> tags. So then I thought maybe I could take advantage of PHP's abilities to deal with the DOM Document Model (with my lack of PHP knowledge, I hadn't learned yet that it had such abilities, but I suspected it probably did and with a web search was able to find information on the relevant classes), and feed the string into a DOMDocument and let that take care of any nested HTML tags. Except that maybe the DOMDocument constructor would try parsing the wikitext as HTML, and I didn't think the conversion from string to DOMDocument was losslessly reversible; the wikitext might get messed up and lose formatting information. I went back and forth partially implementing my function one way before switching to the other, but in the end went with the former approach; the risk of losing information in the transformation from string to DOM was too significant. My function has its flaws; in particular, while it should be able to handle a single span tag within the wikitext, nested span tags might be an issue. But I don't think it's too likely I'm going to need nested span tags in the wikitext anyway; I'm going to go ahead and call it good enough for now.

In retrospect, I probably shouldn't have replaced the [wiki][/wiki] text with a span tag with a specific class; I should have used a custom HTML tag that wouldn't be used anywhere else. Or, for that matter, maybe I didn't need the first event at all; maybe I could have just left the [wiki][/wiki] tags as they were until the final parsing. Anyway, those are all things to explore later; I've got enough else I want to get done before the hard launch that as long as this extension is basically doing what I need to do, I'll leave it at that and move on to other things. Oh... if you want to see or download the extension yourself, it's at https://github.com/ClaySalvage/Wikitext. But you probably shouldn't want to see or download the extension yourself, because it's very poorly documented and haphazardly implemented and hardcodes some things that shouldn't be hardcoded because I couldn't figure out how else to do it. (In particular, the endpoint of the MediaWiki API is currently hardcoded in the hook handler; I tried putting it in the configuration file, but couldn't figure out a way to read it from there—the online documentation says to use "$extra = $event->getComposer()->getPackage()->getExtra();", but when I tried that I just got an error "Call to undefined method phpbbeventdata::getComposer()", so either the phpBB code has changed and that documentation hasn't been updated or (more likely) I'm doing something wrong. (I did notice that the error message seems to indicate the variable I'm using is an instance of event data rather than an actual event, but I couldn't figure out a way to get to the event itself.)) All of those things I do intend to work on and improve later, but not till after the hard launch. (Probably long after the hard launch. It's something I intend to do, but it's not a high priority.)

Unfortunately, when I went to test my new extension, I was in for an unpleasant surprise, which shouldn't have been a surprise if I'd thought about it for half a minute and weren't a complete idiot. For simple wikitext markup like '''this''', the parsing worked just fine. But of course the number one thing I had wanted this extension for was to be able to embed my blog posts in forum posts through interwiki transclusion, like so: {{:Blog:20231208}}. And I did get this working... except of course that that interwiki link doesn't just lead to the blog text; it leads to the entire blog page, with all the surrounding content. And so when I went to include the blog post, what I got was an entire webpage slapped into my forum post.

Well, I could see only one simple way around this. I'd have to write a second extension specifically to include blog posts in forum posts. And so I did. Fortunately, by this point most of the code I needed I'd already written in other extensions, so putting this extension together went relatively quickly, and in less than an hour I had another extension. This one is very site-specific and probably useless to anyone else, but I figured I'd put it up publicly on Github anyway in case anyone wanted to look at it or cannibalize it for some reason, so, eh, here it is: https://github.com/ClaySalvage/BlogEmbed.

So, yeah. This will be the first blog post for which I won't have to go through all the rigmarole of copying the HTML from the post page, pasting it into an online HTML-to-bbCode converter (I've been using this one, for what it's worth), copying and pasting the bbCode from there into Notepad to clean it up and fix some special characters, and then copying and pasting the final bbCode from there into the forum post. Now after making the post all I have to do is type the post key within my new custom bbCode tag, and the forum post should be transcluded automatically. So much faster and easier. Hooray!

Okay, so that's the phpBB extensions. Now let's get back to what I was writing about before, namely the implementation of custom namespaces, and why I'm somewhat more sanguine about my chances of getting it done by the hard launch.

There's one thing about the implementation of the subspaces that until yesterday I hadn't really fully firmed up, which is where to put the navigation tabs for them. I'd tentatively settled on having another row of tabs underneath the top row for pages in spaces with subspaces, and for subsubspaces (like Game:RPG:EABA) yet another row under that. But while that might look better than some other possibilities I'd considered, I still wasn't sure I was completely happy with it—plus I had no idea how I'd implement that, though I thought I may be able to figure it out.

Well, yesterday I finally had a better idea, one that would not only be much more æsthetically pleasing, but also seemed like it would be much simpler to actually implement. An idea so absolutely, thuddingly obvious that I felt like an utter doofus for not thinking of it before. (To be fair, I generally constantly feel like an utter doofus anyway for other reasons.) I'd just leave all the tabs in the top bar. As they are now, the tab for the namespace of the current page would be on the far left, and to its immediate right would be the tab for the corresponding Talk page. If the current namespace is a subspace of another space, then the next tab over would be the tab for its parent namespace. (Should there also be a tab for its "grandparent", if any, and so on up the hierarchy? Or at least a tab for the Mainspace? Eh... still undecided on that.) Then there would be tabs for other namespaces with the same parent, or other top-level namespaces if the current namespace has no parent—provided that the corresponding pages exist for those namespaces; if not, there'd be no tab for a nonexistent page. And after that, there would be tabs for any immediate subspaces of the current namespace—again, provided that such pages exist.

That's probably more comprehensible with specific examples, although I'm not sure how important it is to give an example given that if all goes well this will all be implemented anyway and the explanation here will be unnecessary. Still, what the hey, I'll give a few examples anyway, just in case. Suppose the page you're currently viewing has corresponding pages in the Gamespace, the Buildspace, the Build:LEGO subspace, the Game:RPG subspace, the Game:CCG subspace, the Game:RPG:GURPS subspace, the Game:RPG:Zero subspace, and the Game:RPG:Savage subspace. If you're on the main page, you'd see tabs for Main, Talk, Game, and Build, in that order. If you're on the corresponding Game page, you'd see tabs for Game, Talk, Main, RPG, and CCG. If you go to the corresponding page in the Game:RPG namespace, you'd see tabs for RPG, Talk, Game, (maybe Main?) GURPS, Zero, and Savage. And if you're in the Game:RPG:Zero subspace, you'd see tabs for Zero, Talk, RPG, (maybe Game?, maybe Main?), GURPS, and Savage. Okay, I'm not sure even with the explanation this is necessarily totally clear, but hopefully once it's implemented it will work smoothly in practice.

So, is that the only reason why I'm more bullish about my prospects of getting all this implemented before the hard launch? Just because I've decided where I want to put the navigation tabs? No, of course not; that would be stupid. I mean, I'm not necessarily saying I'm not stupid, but I'm not that stupid, or at least I'm not stupid in that particular way. No; I think this decision about the navigation tabs is an important and necessary step, but it's actually writing the extension that's the biggest hurdle. So what's the progress with that?

Well, first of all, I gave a bit of thought as to just how to implement the subspaces. I eventually decided it made the most sense to implement each subspace as its own full namespace, but then make the distinction in code. As for what to name them, well, I'd just name each subspace by its full "path": the Game:CCG:Strike namespace, for instance, would actually be titled "Game:CCG:Strike" in the code. That way any code I wrote could just parse the namespace itself to figure out its parents. And I realized it made more sense to split the namespace implementation into multiple extensions. For one thing, that way it's more modular, so if someone else wants to use a part of my custom namespace/subspace system on their own wiki but not all of it, they can just download the specific extension or extensions they need. And for another thing, well, splitting a difficult task into smaller tasks makes it seems a bit more attainable, since I can focus on one of the smaller, simpler tasks at a time. So instead of one extension, I was going to write three: One to implement the new namespaces, one to allow the wiki to parse article titles correctly according to the subspace system, and one to deal with displaying the tabs.

The first extension, the one to implement the custom namespaces, was the simplest—simple enough, in fact, that it didn't have to be an extension at all; you can implement new namespaces by just adding a few lines to the LocalSettings.php file. Still, while I could have implemented the new namespaces with just LocalSettings.php, I figured it would be better to go ahead and create an extension for it, both to avoid making LocalSettings.php too long and unwieldy (there were a lot of new namespaces, and would be more in the future), and to make it easier to share the implementation with others who might want to implement these namespaces on wikis of their own. So, anyway, this extension is at https://github.com/ClaySalvage/WongeryNamespaces if you want to see it, which you probably don't. (There are some files there that really don't need to be there, left over from the RandomRules extension when I copied it to make this new extension, and at some point I should clean up the repository and remove the superfluous files... but again, that can wait until after the hard launch.)

And then I started on the second extension, to parse the subspaces. I wasn't necessarily planning on getting all this done in one day, but I guess once I started I kind of got on a roll. Anyway, it didn't take me too long to figure out which hook I needed to use, ArticleFromTitle (I think it's possible in the future to get everything working correctly I may also need to make a handler for the TitleExists hook, but I'll deal with that if it becomes an issue.) And I wrote code to split up the article's full title (including namespace) by colons to match it to known namespaces—basically, the way MediaWiki parses article titles by default it only looks at everything up to the first colon as the namespace; I wanted to use a greedy algorithm that would match the longest existing namespace. This by itself wasn't too hard to do, and I successfully created a Title object with the correct namespace, but when I tried to convert that to an Article object for the hook to return, things broke down. Depending on which function and settings I used, either the full title got reparsed nongreedily into the wrong namespace, or the code threw an error: WikiPage constructed on a Title that cannot exist as a page. Apparently before it does anything with a new article, the code validates it with a function called canExist, and that function, or rather a second function it calls called isValid, decrees that articles with namespaces with colons cannot, in fact, exist.

I briefly planned to look for hooks I could use to maybe change the behavior of isValid to get it to accept articles with colons in the namespace before realizing that there was another way to address this issue. I could just... have the namespace not have colons in them, and have some other character in the namespace name that would be converted to a colon when the namespace name was matched against the full article title. So what character would be unproblematic in a namespace name but would I not be likely to want to use for anything else? I settled on the underscore character, which after all was already in use in the names of talk namespaces, but I didn't think that would cause any conflicts with what I was doing. Anyway, I modified the namespaces accordingly in my WongeryNamespaces extension (fortunately I'd written a helper function to generate the relevant part of the configuration file from a list of namespaces, so all I had to do was add a bit of string replacement code to this helper function and reëxecute it rather than changing all the namespaces manually); I made the necessary modifications to my new extension; and voilà! Everything worked! So here's the (poorly documented, no doubt inefficiently implemented) subspace extension in case you want to see it for yourself, though, again, you probably shouldn't: https://github.com/ClaySalvage/Subspace.

But while all those namespaces and subspaces now exist, there's no way to get to them rather than typing in the full article name directly, or I guess through links in other articles. Anyway, there are no tabs for them at the top of the page. That would be the job of the third extension. This one I haven't finished yet, but I've found which hook I'm pretty sure I need to use—SkinTemplateNavigation::Universal—and I think I have a pretty good idea of what I need to do. I haven't finished it yet not because I got stuck but just because I ran out of time and wanted to write this daily blog post. Heck, it's possible if I'd spent the time working on the extension that I spent writing this blog post, I would have finished it by now (although maybe not; it may end up taking me longer than that), and it seems very possible, maybe even likely, that I'll have it done tomorrow. Wow. Getting those namespaces and subspaces implemented will feel really good; it's a major part of my plans for the Wongery that's been looming over me for a long time, and it'll be great to finally have it in place.

That's not to say I'll have everything implemented that I intend for the subspaces. Eventually some namespaces are likely to have too many child subspaces for tabs for them all to fit in the top bar (the Game:RPG space is likely to have this issue soonest, but others eventually may as well), and when that happens I want to just put tabs some number of subspaces in the top bar and have the others reachable through a dropdown menu. That's going to be trickier to implement, and may involve some tinkering with the MediaWiki Skins, and that's something I've never done before. (Then again, writing an extension was something I'd never done until last Sunday, and now I've written five of them.) Also, I wanted the tabs for visited subspaces to appear first in the order the user has most recently visited them. And right now the All pages Special page on the wiki just lists all the namespaces and subspaces together in one big dropdown; I'd rather have only the top-level spaces in the initial Namespace dropdown, and have other dropdowns appear when you select a namespace with subspaces. All of those tasks involve things that I'm not currently sure how to do, and while I think I can figure it out, it'll take some time and I'm not going to tackle them before the hard launch. Still, I definitely do want to implement all that eventually.

Oh... since tabs wouldn't show up for namespaces (aside from Talk) with no pages corresponding to the current page's title, I was going to have an extra tab with maybe a plus sign in it that would maybe bring up a dropdown of such namespaces so an editor could easily add one. But, eh, since the only ones who'll be adding pages to the Central Wongery are the Grandmaster Wongers, this isn't a priority; we can deal with the slight inconvenience of having to type the page's full title in manually. Oh, wait... except that eventually we want to possibly have Master Wongeries, and we want their editors to be able to add pages there. Oh, and also the Public Wongery; we want anyone to be able to add pages there, and we'd like it to be as user-friendly as possible. (Speaking of which, so far I've only installed my new MediaWiki extensions on the Central Wongery; I need to install them on the Public Wongery too.) So yeah, okay, this is something I should eventually do. But again, not before the hard launch.

So, hey. If I do get that SubspaceTabs extension done by tomorrow... then that means I'll be basically done with all the programming tasks I wanted to get done before the hard launch, and I can spend the rest of the time doing more creative things, like writing and editing articles. Oh, yeah, and improving the page layout. And okay, there is one minor programming task I want to get done, which is enabling blog posts to be accessible through their more descriptive short titles rather than the date-based numerical keys I've been using, but that should be quick and easy to do. In any case, though, this feels like a big hurdle I'm on the verge of finally overcoming. Right now I think I'm feeling better about my progress on the Wongery than I have for a long time. I'm still not feeling good about it, mind you; I still wish I had a lot more articles written than I have, and there are still many features I wish I'd implemented. But better than I have for a long time, anyway.

D minus four days and counting...