Preston Maness ☭<p>I've mirrored a relatively simple website (redsails.org; it's mostly text, some images) for posterity via <a href="https://tenforward.social/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>wget</span></a>. However, I also wanted to grab snapshots of any outlinks (of which there are many, as citations/references). By default, I couldn't figure out a configuration where wget would do that out of the box, without endlessly, recursively spidering the whole internet. I ended up making a kind-of poor man's <a href="https://tenforward.social/tags/ArchiveBox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ArchiveBox</span></a> instead:</p><p>for i in $(cat others.txt) ; do dirname=$(echo "$i" | sha256sum | cut -d' ' -f 1) ; mkdir -p $dirname ; wget --span-hosts --page-requisites --convert-links --backup-converted --adjust-extension --tries=5 --warc-file="$dirname/$dirname" --execute robots=off --wait 1 --waitretry 5 --timeout 60 -o "$dirname/wget-$dirname.log" --directory-prefix="$dirname/" $i ; done</p><p>Basically, there's a list of bookmarks^W URLs in others.txt that I grabbed from the initial mirror of the website with some <a href="https://tenforward.social/tags/grep" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>grep</span></a> foo. I want to do as good of a mirror/snapshot of each specific URL as I can, without spidering/mirroring endlessly all over. So, I hash the URL, and kick off a specific wget job for it that will span hosts, but only for the purposes of making the specific URL as usable locally/offline as possible. I know from experience that this isn't perfect. But... it'll be good enough for my purposes. I'm also stashing a WARC file. Probably a bit overkill, but I figure it might be nice to have.</p><p><a href="https://tenforward.social/tags/RedSails" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>RedSails</span></a> <a href="https://tenforward.social/tags/archive" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archive</span></a> <a href="https://tenforward.social/tags/archival" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archival</span></a> <a href="https://tenforward.social/tags/archiving" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archiving</span></a> <a href="https://tenforward.social/tags/warc" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>warc</span></a></p>
Recent searches
No recent searches
Search options
Only available when logged in.
c.im is one of the many independent Mastodon servers you can use to participate in the fediverse.
C.IM is a general, mainly English-speaking Mastodon instance.
Administered by:
Server stats:
2.9Kactive users
c.im: About · Status · Profiles directory · Privacy policy
Mastodon: About · Get the app · Keyboard shortcuts · View source code · v4.3.6
#ArchiveBox
1 post · 1 participant · 0 posts today
Rad Web Hosting<p>How to Install and Run <a href="https://mastodon.social/tags/ArchiveBox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ArchiveBox</span></a> on <a href="https://mastodon.social/tags/Ubuntu" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Ubuntu</span></a> <a href="https://mastodon.social/tags/VPS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>VPS</span></a> Server </p><p>This article provides a guide for how to install and run ArchiveBox on Ubuntu VPS server.<br>What is ArchiveBox?<br>ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline. Without active preservation effort, everything on the internet eventually disappears or degrades.</p><p>In this blog post, ...<br>Keep reading 👉 <a href="https://blog.radwebhosting.com/how-to-install-and-run-archivebox-on-ubuntu-vps-server/?utm_source=mastodon&utm_medium=social&utm_campaign=ReviveOldPost" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">blog.radwebhosting.com/how-to-</span><span class="invisible">install-and-run-archivebox-on-ubuntu-vps-server/?utm_source=mastodon&utm_medium=social&utm_campaign=ReviveOldPost</span></a> <a href="https://mastodon.social/tags/vpsguide" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>vpsguide</span></a> <a href="https://mastodon.social/tags/selfhosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosting</span></a> <a href="https://mastodon.social/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>opensource</span></a> <a href="https://mastodon.social/tags/installguide" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>installguide</span></a></p>
Martín<p><a href="https://rebel.ar/tags/archivebox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archivebox</span></a> <a href="https://rebel.ar/tags/web" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>web</span></a> <a href="https://rebel.ar/tags/archivo" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archivo</span></a> <a href="https://rebel.ar/tags/archivado" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archivado</span></a> <a href="https://rebel.ar/tags/linkrot" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>linkrot</span></a></p>
Rad Web Hosting<p>How to Install and Run ArchiveBox on Ubuntu VPS Server<br><a href="https://mastodon.social/tags/archivebox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archivebox</span></a> <a href="https://mastodon.social/tags/installguide" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>installguide</span></a> <a href="https://mastodon.social/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>opensource</span></a> <a href="https://mastodon.social/tags/selfhosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosting</span></a> <a href="https://mastodon.social/tags/ubuntu" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ubuntu</span></a> <a href="https://mastodon.social/tags/vps" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>vps</span></a> <a href="https://mastodon.social/tags/vpsguide" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>vpsguide</span></a> <a href="https://mastodon.social/tags/Cloud" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Cloud</span></a> <a href="https://mastodon.social/tags/Guides" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Guides</span></a> <a href="https://mastodon.social/tags/VPS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>VPS</span></a> <br><a href="https://blog.radwebhosting.com/how-to-install-and-run-archivebox-on-ubuntu-vps-server" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">blog.radwebhosting.com/how-to-</span><span class="invisible">install-and-run-archivebox-on-ubuntu-vps-server</span></a></p>
Claudine C :antifa:<p><span class="h-card" translate="no"><a href="https://mastodon.online/@Carwil" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>Carwil</span></a></span> Hell yeah <a href="https://aus.social/tags/ArchiveBox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ArchiveBox</span></a> can turn everyone with a computer and spare storage into a home archivist.</p><p><a href="https://aus.social/tags/Alt4you" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Alt4you</span></a> two screenshots from an ArchiveBox web interface showing locally archived versions of pages from the USAID website</p>
Jessie Nabein :neofox_peek_owo:<p><a href="https://wetdry.world/tags/ArchiveBox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ArchiveBox</span></a> <br>13272 snapshots <br>running since 2023</p>
🇨🇦Samuel Proulx🇨🇦<p><span>I'm still on the hunt for a good self-hosted link manager. </span><a href="https://fed.interfree.ca/tags/Goodlinks" rel="nofollow noopener noreferrer" target="_blank">#Goodlinks</a><span> is okay, but only on IOS and mac, not windows. </span><a href="https://fed.interfree.ca/tags/linkwarden" rel="nofollow noopener noreferrer" target="_blank">#linkwarden</a><span> uses hover for, like, everything, making it almost completely inaccessible. </span><a href="https://fed.interfree.ca/tags/Hoarder" rel="nofollow noopener noreferrer" target="_blank">#Hoarder</a><span> has basically no labeled buttons or links. I want something that's </span><a href="https://fed.interfree.ca/tags/accessible" rel="nofollow noopener noreferrer" target="_blank">#accessible</a><span>, cross-platform, </span><a href="https://fed.interfree.ca/tags/selfhosted" rel="nofollow noopener noreferrer" target="_blank">#selfhosted</a><span>, will archive pages, and offers full-text search. Three quarters of the time when I use Google, I'm just searching for something I already know about. In an ideal world, a selfhosted link/bookmark manager would reduce my dependence on big-tech search engines. Maybe </span><a href="https://fed.interfree.ca/tags/archivebox" rel="nofollow noopener noreferrer" target="_blank">#archivebox</a><span>? But that looks like it's more about storing the data than actually doing anything useful with it like search or tagging or whatever. </span><a href="https://fed.interfree.ca/tags/a11y" rel="nofollow noopener noreferrer" target="_blank">#a11y</a></p>
Rad Web Hosting<p>How to Install and Run ArchiveBox on Ubuntu VPS Server<br><a href="https://mastodon.social/tags/archivebox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>archivebox</span></a> <a href="https://mastodon.social/tags/installguide" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>installguide</span></a> <a href="https://mastodon.social/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>opensource</span></a> <a href="https://mastodon.social/tags/selfhosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosting</span></a> <a href="https://mastodon.social/tags/ubuntu" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ubuntu</span></a> <a href="https://mastodon.social/tags/vps" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>vps</span></a> <a href="https://mastodon.social/tags/vpsguide" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>vpsguide</span></a> <a href="https://mastodon.social/tags/Cloud" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Cloud</span></a> <a href="https://mastodon.social/tags/Guides" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Guides</span></a> <a href="https://mastodon.social/tags/VPS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>VPS</span></a> <br><a href="https://blog.radwebhosting.com/how-to-install-and-run-archivebox-on-ubuntu-vps-server" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">blog.radwebhosting.com/how-to-</span><span class="invisible">install-and-run-archivebox-on-ubuntu-vps-server</span></a></p>
UbuntuHow to Install and Run ArchiveBox on Ubuntu VPS Server This article provides a guide for how to i...<br><br><a href="https://blog.radwebhosting.com/how-to-install-and-run-archivebox-on-ubuntu-vps-server/" rel="nofollow noopener noreferrer" target="_blank">https://blog.radwebhosting.com/how-to-install-and-run-archivebox-on-ubuntu-vps-server/</a><br><br><a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/Guides" target="_blank">#Guides</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/Cloud" target="_blank">#Cloud</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/VPS" target="_blank">#VPS</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/archivebox" target="_blank">#archivebox</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/install" target="_blank">#install</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/guide" target="_blank">#guide</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/open" target="_blank">#open</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/source" target="_blank">#source</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/self-hosting" target="_blank">#self-hosting</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/ubuntu" target="_blank">#ubuntu</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/vps" target="_blank">#vps</a><br><br><a href="https://awakari.com/pub-msg.html?id=HrFaXGnBdyz6Vup0qIJIEpotavQ" rel="nofollow noopener noreferrer" target="_blank">Event Attributes</a>
UbuntuHow to Install and Run ArchiveBox on Ubuntu VPS Server This article provides a guide for how to i...<br><br><a href="https://blog.radwebhosting.com/how-to-install-and-run-archivebox-on-ubuntu-vps-server/" rel="nofollow noopener noreferrer" target="_blank">https://blog.radwebhosting.com/how-to-install-and-run-archivebox-on-ubuntu-vps-server/</a><br><br><a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/Guides" target="_blank">#Guides</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/Cloud" target="_blank">#Cloud</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/VPS" target="_blank">#VPS</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/archivebox" target="_blank">#archivebox</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/install" target="_blank">#install</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/guide" target="_blank">#guide</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/open" target="_blank">#open</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/source" target="_blank">#source</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/self-hosting" target="_blank">#self-hosting</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/ubuntu" target="_blank">#ubuntu</a> <a rel="nofollow noopener noreferrer" class="mention hashtag" href="https://mastodon.social/tags/vps" target="_blank">#vps</a><br><br><a href="https://awakari.com/pub-msg.html?id=2qyRkt3TsA9mwztxIPHDZ0s4ggG" rel="nofollow noopener noreferrer" target="_blank">Event Attributes</a>
ExploreLive feeds
Mastodon is the best way to keep up with what's happening.
Follow anyone across the fediverse and see it all in chronological order. No algorithms, ads, or clickbait in sight.
Create accountLoginDrag & drop to upload