These are local mirrors of long-defunct historical websites, as taken from the Internet Archive Wayback Machine using ShiftaDeband's rate-limited fork of wayback-machine-downloader. I then use A1 Sitemap Generator to find dead links / missing files and try my best to reinstate them. I use Visual Studio Code for manual massaging like converting absolute links into relative ones etc, a variety of command line scripts and things that I've come up with along the way, and finally host them on GitHub Pages (for now). Every site is a git repo so all changes are tracked. The reason for this rather odd setup is that I'm aiming for this process to become increasingly automated as I work things out.
At the moment each site listed here is the result of days or even weeks of work on my part.
So why not just browse these sites on the Wayback Machine?
Well, don't get me wrong, The Internet Archive does absolutely amazing work, but the Wayback Machine is slow by its very nature and users often come across broken or missing things. They're now also heavily rate limiting historical proxies like WaybackProxy and various scraping / mirroring tools - an issue I've run up against time and again in the course of this work. So I've made it my mission to create these self-contained independent mirrors, track down every single missing element where possible and reinstate them, and package them up nicely here in their mostly original form for your perusal (some broken elements are removed at my discretion for a better user experience, but the substance and content is very much original).
These mirrors are also thoroughly sitemapped and submitted to search engines, which traditionally struggle to crawl the Wayback Machine, making them widely available to the world once more.
By the way, the intention is and always will be that this entire archive be browsable using actual historical browsers on real vintage hardware. If they also work in modern browsers, that's a bonus! :)
Will you be documenting your exact workflow in future?
Yes, this is part of my future plans.
Navigating the list is kind of a mess...
Not a question, but I hear ya. This will be turning into a proper directory as it grows, much like the old pre-search engine web directories of old. But I don't want to spend time on that until I have a load of sites to put in it.
What's the criteria for inclusion? Do you take requests?
I include anything that personally interests me and that I don't think will be too challenging to resurrect. I particularly like sites from around the "turn of the century" (wow that makes me feel old). Anything earlier is great, but Wayback Machine's coverage is incredibly patchy before around 2002ish. Of course, older 90s sites that were still online and being archived at this time are fantastic, but even by this point the link rot was starting to set in and sadly much of the 90s web is now forever lost to time. The more static the better - forms and other interactive elements don't mirror properly and won't work.
As for requests - I have quite a long list of sites "in progress" at the moment so I'm not looking for more work, but I may add a request form in due course.
Seems like you're just ripping off The Internet Archive's hard work?
My goals are very much aligned with theirs and I do support them financially, as well as giving full credit where due. This is a passion project and I never aim to monetise this collection. I don't add tracking or even indeed any kind of analytics. Is anyone even using this thing?
Can I mirror these sites myself?
I suppose there's nothing stopping you, but I think having them duplicated in full would probably be a net negative for both of us. Of course, feel free to grab your own copy for your own purposes - it's all static and a simple wget -m should work fine. I do aim to make the git repos public eventually so these mirrors can live on when I'm gone.
Do you want some help?
Not at the moment, but thanks for asking.
How do I report something broken / missing?
You don't - for now. But more often than not you'll find that it's an issue with the original mirror, which is unfortunately difficult for me to fix / work around. In some cases I just have to admit defeat, or make a note to come back later with fresh eyes.
Why are you doing this?
I'm a web developer in my day job and have been since the late 90s, and quite frankly, I pine for the good old days. This is probably the closest I'll ever get - plus it feels like a useful public service somehow.
Who are you?
That's not a secret but I also don't want to shout it from the rooftops until this project has a bit more substance to it. All will be revealed in due course. :)