Here is an app to bulk download URLs from the wayback machine.
It’s useful to check failed migrations, check historical redirect mapping, historical robots.txt changes as well as visuallually hinting at a replatforming.
Features
- Download URLs
- Folder Visualisation
- Status Code Visualisation
- Frequently Changed Pages
- Robots.txt Changes
Folder Structure Over Time
See the evolution of a Website’s structure over time. Useful to see when issues start creeping in or when a site was re-platformed.
Status Code Visualisation
View status code history over time, useful to see issues at a glance such as failed migrations or re-platforming.
Frequently Changed Pages
Spy on competitors or just gain insights to your most frequently changing pages.
Robots.txt Historical Changes
See how and which rules changed within robots.txt.
Download URLs
Simple URL downloader with options to filter out unrelated assets. Useful to check whether existing redirects have been properly implemented, or redirect additional 404 pages.
If you need help modifying or deploying this to your organisation, drop me a message via the contact form and I’ll get straight back to you.