11 Commits

Author	SHA1	Message	Date
Lucas	147695edc4	fetch: change storage implementation ... Instead of partitioning over YYYY/mm/dd, take the SHA256 of the URL and partition over the first, second and tail bytes of it. It has the upside of being more rsync-friendly and there is no longer a need to keep a different file, "everything", to look up already retrieved URLs. Also replace the "url" file with a "meta" file, organized as key=value, currently holding URL and retrieval date.	2022-08-03 03:33:28 +00:00
Lucas	68e11fc972	fetch: explain why we call tty(1) ... While at it, also use safe test operands.	2022-08-03 03:27:43 +00:00
Lucas	09bfd184a2	fetch: use glob expansion instead of find, for speed	2021-08-02 03:12:01 +00:00
Lucas	141a56f51b	fetch: small fixes ... - Use `sha256 -s` instead of piping into `sha256` - Remove unused name output file	2021-04-11 14:32:18 +00:00
Lucas	0f6b0e5133	Shell cosmetic changes	2020-08-23 14:48:24 +00:00
Lucas	d5e71c46f7	Align shell style all over the repo	2020-05-03 14:47:27 +00:00
Lucas	17ef04b41a	fetch: use a slightly more correct check for TTY stdin	2020-05-03 14:00:41 +00:00
Lucas	649ead1257	fetch: fix showing already archived URLs	2020-03-21 11:54:45 +00:00
Lucas	c84c0e3a99	Unify shell script style	2019-12-08 18:39:16 +00:00
Lucas	a3ec887fb8	fetch: Don't mess stdout with progress bar ... Instead of using default ftp(1), if stdin is a tty, redirect ftp(1) stdout to the tty. This implies know you can safely do $ fetch ... | sxiv	2019-10-26 23:05:30 +00:00
Lucas	3779fdce2c	Initial import	2019-09-11 17:44:49 +00:00