UESPWiki:Administrator Noticeboard/Archives/Code Updates

This is an archive of past UESPWiki:Administrator Noticeboard discussions. Do not edit the contents of this page, except for maintenance such as updating links.

Code Updates

I'm in the final stages of testing a significant set of updates for the site's PHP code. Before passing the code to Daveh for installation, I wanted to doublecheck whether there are any updates that I've overlooked.

The primary purpose of this update is really just improved code maintenance for the site (read: it should hopefully be much easier for Daveh to upgrade our mediawiki code, so we're no longer a year behind in mediawiki updates and forced to use out-of-date versions of extensions). This became an even greater issue on the site when Daveh expanded to multiple content servers, since any upgrades need to be applied on each server separately.

However, amidst all of the code reorganization I've also tried to add in any code requests of which I was aware. That includes:

• Adding Shadowkey to Special:Search
• Removing captcha checks for patrollers
• Tweaking indentation on the site's sidebar, to make it look like this. I am not using the tweak suggested by Ratwar -- in other words, this will not force the entire sidebar to be written in wiki code (which would make us lose some sidebar features, in particular some that appear in upcoming mediawiki versions and that I've retrofitted into our current code). Rather, it will simply make it possible to have extra indents added for list entries such as Shivering Isles. Based on my memory, and from reviewing the failed tests on the Sidebar page, I think that's the only feature we need added to the sidebar -- but if there are reasons why we need more extensive customizations of the sidebar, let me know.
• Performance improvements for the site's search engine. The biggest improvement is completely invisible to readers (my previous sloppy coding meant that the "Go" button was probably requiring 250-odd separate database calls; I've consolidated that all into a single database call). In addition, though, I have also opted to implement a suggestion I made here (under "Tweaking the wiki search engine") a long time ago: using the sidebar search button will result in a title-only search for the requested term. To do a full-text search, you'll have to use the search button on the search page. My feeling is that eliminating a significant chunk of CPU processing every time a reader uses the search box is worth the minor inconvenience on those few occasions when someone actually wants a full-text search. But, again, if others disagree, speak up now (or forever hold your peace ;) ).
• Making our site's logo and skin more "official" -- so that every time the site hiccups, pages don't revert to the site's old logo and standard mediawiki color scheme.

I'm also adding a few new features in this round of updates -- for now, just sticking to some easy-to-implement features:

• Making it so that Orphaned pages automatically skips any disambiguation pages. In other words, this entire page can be blanked -- or converted to a better purpose, such as describing when disambiguations should be used, and how they should be used.
• Adding some new UESP-specific magic words and parser functions:
  • {{#sortable}} that provides the sortable version of any text (e.g., "{{#sortable:A page title}}" would print "page title, A"). Putting this into PHP code means that it can be done much more efficiently than is possible with templates -- particularly important for a function that is likely to end up on every page on the site.
  • {{CORENAME}} that always returns the snippet of {{PAGENAME}} that is relevant for category sorting. In other words, CORENAME would print "Port Telvannis" if it's called from any of Lore:Port_Telvannis, Lore:Port_Telvannis/Description, Tes3Mod:Tamriel_Rebuilt/Port_Telvannis, or Tes3Mod:Tamriel_Rebuilt/Port_Telvannis/Description.
  • {{SORTABLECORENAME}}: same as {{#sortable:{{CORENAME}}}}. I put this into its own magic word because I'm hoping it will be possible to use SORTABLECORENAME universally on the site (in every trail template, summary template, description page, etc.), in place of our current medley of PAGENAME, SUBPAGENAME, SORTABLEPAGENAME, SORTABLESUBPAGENAME, etc.
  • An assortment of improved namespace-related magic words. These should fix the shenanigans we need to go through for Shivering Isles on every summary template. And even more dramatically fix the shenanigans we're going through with adding Tamriel Rebuilt into templates. I'll provide full details once the new extension is in place, making it much easier to show how it works.

The upshot of this whole post is: are there are other needed fixes that I've forgotten? Or implications of these changes that I've overlooked? In general, are there any last-minute changes that I should make to this chunk of code before asking Daveh to install it? I'm hoping to have the code ready for Daveh within the next day -- although tweaks will still be possible after that. --Nephele^Talk 14:32, 15 March 2009 (EDT)

Wow. While you're fixing everything that's wrong with the site, could you take a look at my sink? It's been draining a bit slowly of late and I think the U-bend might need clearing. :-p

That looks like a pretty comprehensive list of things I can think of that could be improved at the moment (and it included several I didn't realize could be improved). In particular, the thought that the {{SORTABLE*}} templates will be implemented on the server rather than in wiki code is a welcome one. I always considered them a necessary evil that would - with any luck - become obsolete with time. I can imagine several pages being much faster with that one change.

From a reliability point of view, I imagine there's a relatively easy rollback if things don't work well in prime-time? I'm not casting aspersions on your code, but as somebody who has had their fair share of "Well it worked on my computer" moments, I've learned to be safe rather than sorry.

Lastly, thank you for spending the time on fixing these irritating little - and not so little - problems. The site should be much faster and much easier to maintain with these in place. –Rpeh^{•T•C•E•} 15:53, 15 March 2009 (EDT)

The kitchen sink will be part of my next overly-grandiose extension-that-fixes-everything-on-the-wiki :P

One of the (many) motivations behind this list of changes was my sneaking suspicion for why the site keeps getting slower even though page hits are decreasing: our pages keep getting more complex. In other words, each page request probably requires more CPU than it did a year ago. The SORTABLE templates are probably one culprit. Not that there's any flaw in the templates (in fact it's a pretty remarkable piece of template-writing), but there's a reason why regexp functions are so popular. A single PHP regexp function can replace three layers of templates. The downside is that any future changes need to go through Daveh -- for example, if we realize there's another pronoun that needs to be skipped.

The changes are being implemented as a mediawiki extension, which means that the vast majority of the code is turned on/off by a single line in the site's LocalSettings.php file. Most of the individual changes are turned on/off by one or two lines of code in the extension's main file. Basically, rollbacks with this code will be much easier than any previous code changes we've made to the site. (I've also tested the code far more thoroughly than any of the other code changes I've forwarded to Daveh -- most of my past changes were basically PHP code that hadn't even been compiled yet; now I've got a test wiki running where I can get as close as possible to checking what will happen when Daveh implements the code).

The most significant source of code changes with this update will actually be reverting a lot of our current customized php files to the vanilla mediawiki versions, or very-near-vanilla versions. (After this is all done, I'm hoping to have only about 20 lines of hard-wired custom code changes to a total of 4 standard mediawiki files, whereas right now we have at least 1000 lines of custom code modifying some 25 standard mediawiki files). Rolling back those code changes would be less trivial (although never difficult: I've added all the old code to my SVN server, and I'm sure Daveh has his own backup systems, too). But really all that means is that the worst case scenario is that if the new UespCustomCode extension was completely disabled, the site would revert to a vanilla mediawiki site. Fewer conveniences, but safe and non-buggy (or at least, any bugs wouldn't be my fault!)

In any case, I'm also putting together a fairly detailed set of step-by-step instructions for Daveh (that's the main thing I need to work on today) so that he knows what he's doing as he installs the extension and can test each change as it takes effect. In other words, yes, I've had my share of oops! moments, too, so I've tried to anticipate the unexpected ;) --Nephele^Talk 16:46, 15 March 2009 (EDT)

Site traffic has been roughly steady over the past year. Its been a little hard to tell since we've moved to using the Squid cache and now the two content servers. Just looking at the Google Adsense numbers, for example, gives the same average the past two weeks as last year (around 440,000 ads served/day). The recent performance bottleneck was definitely the content server where the MediaWiki pages are generated from scratch. Its possible that all the various templates used contributed to slowing the site down, although exactly how much I don't know (we could profile it but at this point its more effort than its worth).

These new changes sound good and will hopefully make it much easier to upgrade MediaWiki. I noticed recently its up to v1.14 while we're still using v1.10. While we don't necessarily have to always have the latest version its good not to get too far behind. -- Daveh 20:09, 15 March 2009 (EDT)

One potential reason to upgrade sooner rather than later is that extension-related features were majorly revamped in v1.11. The upshot is that nearly all extensions now require MediaWiki 1.11 (including, for example, the requested update to the Cite extension).

On the other hand, there's also one reason to wait until 1.15: I'm working on getting a set of our code modifications added into the vanilla MediaWiki code (since the code is fixing a bug that MediaWiki's been aware of for three years now). But we can work around that issue if you want to upgrade to v1.14 --Nephele^Talk 23:29, 15 March 2009 (EDT)

Will my newly-created custom sidebar still work after the sidebar indentation tweak? --Gez 17:01, 15 March 2009 (EDT)

Mostly :/ An advantage of the above-mentioned decision to stick with a minimal tweak to the sidebar (instead of changing to a completely wiki-code version) is that the sidebar will still be pretty similar to what wikimedia expects. The only problem is that your existing .js code won't remove any of the new, indented entries in the sidebar. But it's possible to tweak your .js so that it will be able to handle those entries, too -- I'll work out the necessary tweaks when I get a chance. --Nephele^Talk 18:18, 15 March 2009 (EDT)

I have mixed feelings about the change to the search functionality because I think the title-only functionality will be quite helpful in its own right, never mind the reduced number of database results that that'll cause (and as a DBA, I'm all for smaller result-sets and fewer hits <grin>). It'll just be annoying when I really do want to do a full-text search. I was going to suggest a toggleable option for it near the search box somewhere, but I'm concerned that it might make things too cluttered, so I'll let you decide on that one. :) --Robin Hood (Talk • E-mail • Contribs) 18:25, 15 March 2009 (EDT)

Hmm... how about replacing the current "?" button with a "More options..." button that simply takes you to Search? It requires an extra click, plus it's really just the same as clicking "Search" without entering anything in the box, but it's more obvious and covers more possibilities. As for the "?" button, I'm not sure that readers know what it does, plus there's a pretty obvious link to the Help page at the top of the Search page. --Nephele^Talk 18:50, 15 March 2009 (EDT)

That'd work for me. --Robin Hood (Talk • E-mail • Contribs) 22:07, 15 March 2009 (EDT)

OK, that's how I've now set it up. --Nephele^Talk 23:29, 15 March 2009 (EDT)

I'm not sure if it would have anything to do with what's being discussed here, but I've been considering requesting a feature I use often on another wiki. On the UESP, currently the only way to get to an article diff with a "mark as patrolled" link is to follow the diff link on Recent Changes. On the other wiki I use, any diff viewed by a Patroller/Admin automatically has the "mark as patrolled" link if it is not patrolled. This would be helpful for me because I use my watchlist e-mails rather frequently, to monitor pages so that I check the pages that are most important (to me) first, before checking Recent Changes. Just a thought... ;) --GuildKnight^Talk2me 12:56, 18 March 2009 (EDT)

Actually that reminds me. There's an old, old wish for a "Mark as unpatrolled" option when you suddenly realise you shouldn't have patrolled an edit after all. Sorry I didn't mention that one earlier but it completely slipped my mind. –Rpeh^{•T•C•E•} 13:34, 18 March 2009 (EDT)

True. I'm sysop on another wiki and sometimes, when looking at recent changes, I see an edit that I know will need to be rephrased heavily but I don't have the time to do it right now, so it's better left marked as unpatrolled for the time being. --Gez 13:47, 18 March 2009 (EDT)

Nothing new to add here, other than to pipe up that I too would appreciate both these options if they're doable. --Robin Hood (Talk • E-mail • Contribs) 14:06, 18 March 2009 (EDT)

The wider implementation of "mark as patrolled" links appears to be a feature that will automatically be added when the site upgrades to a more up-to-date version of mediawiki. On my test wiki, I'm running 1.10 and 1.15alpha side-by-side, and when I look at the exact same diff (starting from the history page instead of recent changes) in 1.10 and 1.15alpha, the "mark as patrolled" link appears in 1.15alpha, but not 1.10. FYI, 1.15alpha is the development version of mediawiki, which will morph into 1.15 when 1.15 is due for official release. I'll poke around a bit on the unpatrol option, too, but I'm guessing it will probably make more sense to implement any changes after UESP has upgraded, given that it's clear that particular section of code is going to be changed after the next upgrade. --Nephele^Talk 15:31, 18 March 2009 (EDT)

←

Speaking of the code update, I thought about something else that'd be on my wishlist: the ability for category pages to not display namespaces, and to not display parent pages (probably with magic words like __NOTOC__, maybe __NONS__ and __NOPARENT__). I think it'd look a lot neater for pages such as Category:Morrowind-Places-Tombs or Category:Tes3Mod-Tamriel Rebuilt-Scout, for example. --Gez 06:37, 20 March 2009 (EDT)