The state of The Official Site and why we’ve created reddwarf.info Features Posted by Jonathan Capps on 14th September 2023, 09:30 On 25 February 2021 it was announced that reddwarf.co.uk would be moving hosting providers and this would involve “a short break in service”. Four months later it returned, with the revelation that after 22 years the traditional weekly updates would be stopping. Also mentioned was that “it turns out that rebuilding a site that can comfortably re-house over two decades’ worth of content was actually a much bigger job than we thought it would be” and “At the time of writing, we’re still seeing a few 404 errors when it comes to some sections”. Well, it turns out that was a bit of an understatement as it doesn’t take long for anyone trying to use the site after the migration to notice that large amounts of content, images and even whole sections are either entirely gone or unnavigable through normal means. It’s all well and good saying that everything is pretty much back online, but what good is a 22 year archive of news updates when the archive navigation is entirely non-functional? Well, to cut to the chase before I go on more after the jump, here at Ganymede & Titan we’ve come to the conclusion that this isn’t getting fixed any time soon and so we’ve taken matters into our own hands. Using a cunning combination of the Wayback Machine, content that is still accessible on reddwarf.co.uk itself, and a great deal of finagling we’ve put together a totally fresh, and working, archive of The Official Site. Presenting… reddwarf.info It’s been a bit of a funny couple of years for Red Dwarf, as precisely no one reading this needs to be told. After an extended period of legal battles, the rights issues surrounding the show were settled and in the process Grant Naylor Productions was reduced to a grand total of two staff: Rob Grant and Doug Naylor. It was during the latter stages of this process that TOS was going through its migration. Although the significance of this timing and how much it was affected by other events is unknown, it’s pretty easy to draw conclusions. Unless Rob and Doug have been going to web development night classes in their free time between suing each other, GNP is no longer a company that can technically support an active website, nor it seems pay anyone to write any words to go on it. As such, this ‘migration’ was actually something far more complicated than moving files over to a new server and calling it a day; this was a ‘flattening’ of all content to divorce it from any dependency on a database or Content Management System and essentially produce a massive amount of flat HTML files that can be theoretically navigated as they could before, but without the overhead needed to host a more complex system. This is a common practice for sites going into archive mode, and it’s something I’ve done a lot when archiving a number of Seb’s old websites, as frankly the fewer WordPress installs you have to worry about the better. Unfortunately, this flattening process is the source of many of the problems facing the site now. To demonstrate the issue, take the ‘Space Corps Database’, one of the bigger and more complex areas of the site. The root URL of https://www.reddwarf.co.uk/database works great and is accessible via the main navigation. However, as soon as you use the select box navigation to, say, go to the ‘Mechanicals’ section you’re sent to https://www.reddwarf.co.uk/database/index.cfm which gives you a 404. However, if you decide to have a bit of a guess of what the URL might be and try https://www.reddwarf.co.uk/database/mechanicals then you get the page you’re looking for. And examples of this are absolutely everywhere. Basically if you end up somewhere with ‘index.cfm’ in the address – which are the dynamic files that the site used to run from – then you’ll be getting a 404, and guessing URLs doesn’t always work either. Anything with pages of archives such as the ‘Features’ section or ‘News’ are entirely inaccessible, although in the case of News you are able to navigate back in time by endless clicking the ‘Previous Story’ link, but it won’t be long until you hit a dead end with an article showing a 404 or other error. In short, it’s a total mess. It’s pretty clear to see how this has happened. Some of the flattening has been done, but the size of the job is undeniable and it seems they simply ran out of time and, most probably, money to allow the job to be finished. So it went out as is, with the vague hope it’ll be fixed eventually and, to be honest, when you’re dealing with a website like TOS, that has spanned the vast majority of many fans’ lives and been a constant weekly presence and provider of amazing work from some of the best people in the Dwarf world, being this shoddy with the archiving effort is completely unforgivable. It won’t take a genius to figure out the strong emotional connection we at G&T have with TOS, as we count all three previous webmasters as friends, which is also a pretty big fucking understatement in the case of Seb Patrick, and the prospect of their work being left in this sort of state is unpalatable. To be clear, I don’t blame anyone who was tasked with actually performing the migration task, but things like this very often get left by the wayside when business matters and issues take centre stage. Now, that’s not to say our archive is perfect either. There are some things that by chance were just never archived by the WBM (or if they were we’ve not yet managed to dig them out) such as a large number of images, the odd news post here and there and, more gallingly, the last few pages of Life During Dwarf Time and the majority of the DJXIX report. There are also parts we’re quite proud of, though, such as recreating the functionality of the webcomic viewer used for the original release of Prelude to Nanarchy and restoring functionality to the Random Article button. The restoration process is by no means over and we plan to recreate elements that we can and hopefully the means to restore some of the written content will present themselves, but for now the vast majority of the TOS you’ve known and loved for over 20 years is there and we really hope you enjoy digging through the archives with fresh eyes.
Well done guys. This is excellent, important archival work. Just a shame that the official channels are letting this stuff fall by the wayside.
AMAZING. It’s a sad state of affairs that it would come to this, but incredible that the 3 of you were willing and able to rise to the occasion, and not for the first time. It’s a really nice touch that TUA has a search function when TOS doesn’t. It’s also surprising just how many extra news links there are on the homepage of TUA compared to TOS. Is that because those links are outright missing on TOS, or just in a different order? (I did spot that “Downtime” and “Thirty-Four” are switched around, and “Thirty-Four” has “Downtime”‘s description.) Overall, … just take my advice and don’t try to play Geek Chase; that wasn’t a reasonable expectation.
It’s a really nice touch that TUA has a search function when TOS doesn’t. It’s also surprising just how many extra news links there are on the homepage of TUA compared to TOS. Is that because those links are outright missing on TOS, or just in a different order? (I did spot that “Downtime” and “Thirty-Four” are switched around, and “Thirty-Four” has “Downtime”‘s description.) Ah yeah, I should mention the search function is in a bit of a weird state. I’ve hooked it up to Google search but they’re still processing the sitemap. I think I’m going to build something bespoke, but this will do for now. Fixed the Thirty-Four description now, my little QA angel. The home page on TOS is a bit of a weird mishmash of stories. I think maybe the idea was to highlight the important ones and hide the time sensitive ones, but I’ve just put everything in their original chronological order and limited it to 20 so the archive navigation makes sense, as that’s 20 stories per page.
And I’m really pleased this has resonated with people. With any luck TOS will be restored at some point and I can just forward the whole domain back over there, but I feel better knowing we have something just in case.
Well done, guys, that looks great. Looking forward to having a proper investigation later, looking back at some of Andrew’s early stuff, from when I used to actually have TOS set as my browser’s home page.
Ah, makes sense. I guess that explains why Google thinks there are only 2 Talkie Toaster interviews total.
Cappsy, last week: “I expect that as soon as it’s launched Flap Jack will be in touch with a list of corrections.”
Bloomin’ ‘eck! Just when I thought you guys had run out of insanely time consuming projects to work on. This is remarkable and a real service to the fans. Top work.
Haha, well it’s just the one thing so far, so either you’re exceeding expectations or I’m falling short. Although you do seem to have accidentally added a bunch of parody articles from other websites into the news archive, about there being a Red Dwarf movie? Bit sloppy, that.
Bravo on this preservation work! I always find it upsetting when stuff vanishes from the internet (or even just partially breaks, like TOS has). I recently wanted to revisit someone’s blog post from a few years ago, and found that not only has the blog gone, but they had a robots.txt that stopped it being backed up by archive.org. Of course they’re entitled to do that, but it was still disappointing. So I welcome anything that people do to help preserve these bits of Internet history, no matter how unimportant or ephemeral it might seem.
Bravo on this preservation work! I always find it upsetting when stuff vanishes from the internet (or even just partially breaks, like TOS has). I recently wanted to revisit someone’s blog post from a few years ago, and found that not only has the blog gone, but they had a robots.txt that stopped it being backed up by archive.org. Of course they’re entitled to do that, but it was still disappointing. So I welcome anything that people do to help preserve these bits of Internet history, no matter how unimportant or ephemeral it might seem. I’m finding this with podcasts. There was a great Dave Gorman podcast on Absolute Radio that has just disappeared off the face of the earth.
Absolutely fantastic job! The services you guys deliver to the fandom is genuinely second to none! Agreed. Live G&T reaction:
I’m finding this with podcasts. There was a great Dave Gorman podcast on Absolute Radio that has just disappeared off the face of the earth. That’s a real issue with podcast being paid for and produced by production companies basically as commercial radio shows. Paywalling them to specific platforms is bad enough, but as soon as they decide it’s not worth the hosting cost any more they’re gone. So many podcasts don’t even have their entire categories available any more, others just vanish entirely.
Stupendous work. We all know what a crappy state of affairs has led to you feeling the need to do this so seriously good on you guys, much appreciated
You should resurrect the forum so we can all enjoy that really embarrassing, poorly informed rant I made at age 13 or so about Remastered that annoyed Andrew Ellard into correcting some of my blatantly incorrect statements.
We should install a like button so the meme threads can become an unhealthy competition that makes everyone participating miserable.
If there was a like button I would have pressed it. G&T should have a like/dislike system, but the like button should be labelled “Good” and the dislike button should be labelled “Shit”.
If there was a like button I would have pressed it. G&T should have a like/dislike system, but the like button should be labelled “Good” and the dislike button should be labelled “Shit”. G&T should have a like/dislike system, but the like button should be labelled “Good” and the dislike button should be labelled “Cunt”.
It’s on my to-do list, right after ‘Delete the idea for an episode thread’ and ‘make all Smegle screenshots greyscale’
What if the dislike button was a picture of Kill Crazy and the like button was a picture of Ackerman. That way people with huge numbers of likes will stay humble.
I appreciate the technical why, fascinating, and the archive is really appreciated. It’s been a hard few years in Red Dwarf land, this feels a bit like a monument to a better time. I agree with the “don’t blame anyone who’s job it was” statement but Rob and Doug can get to fuck for their parts in this sorry website episode. They know what’s there, and what it means to people. Also, would putting out a list of missing images be worthwhile? Do you have filenames? I know I have a load of random Red Dwarf images from all over the Internet that I’ve been saving and moving around for years, maybe there are others as sad as I.
Also, would putting out a list of missing images be worthwhile? Do you have filenames? I know I have a load of random Red Dwarf images from all over the Internet that I’ve been saving and moving around for years, maybe there are others as sad as I. Good point, reasonable chances some would be on Tumblr for a start.
Yep I plan to catalogue those as we have all the original file names. A lot of the merch section can probably be manually rebuilt as well.
The fact that the entire series coming to iPlayer or being repeated on BBC Two hasn’t warranted an update is… not good.
The fact that the entire series coming to iPlayer or being repeated on BBC Two hasn’t warranted an update is… not good. I assume from Doug’s recent comments that all the official channels have been effectively mothballed now.
Reposting, formerly retweeting, what Naylor said on X, formerly Twitter: As some of you may have noticed @RedDwarfHQ hasn’t been operational since March this year and sadly is unlikely to tweet Red Dwarf updates again. Not my choice! If you want the latest breaking news involving new Red Dwarf shows with the Posse follow me. Second paragraph under the web link pretty much covers it though, Shirley. And G&T has a like/dislike system. People say they like an exceptional post or will be sarcastic/indifferent until you storm off in a huff over your weird politics. Bad enough when the Smega Drive is used as a crutch. I like being able to find a post I liked in the Googles years later BECAUSE IT WAS TYPED.
And G&T has a like/dislike system. People say they like an exceptional post or will be sarcastic/indifferent until you storm off in a huff over your weird politics Corbyn wuz robbed (of a ‘like’ button)
