1042 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
pictuga	c5a6386a53	actions: fix python use	2023-06-23 23:26:41 +02:00
pictuga	4340b678d0	actions: change image	2023-06-23 23:14:32 +02:00
pictuga	ff9503b0d0	Switch from Drone to Gitea Actions	2023-05-17 22:54:05 +02:00
Nesswit	8bdcd8f386	Add mode option	2023-05-04 16:01:52 +09:00
pictuga	ea2ebedfcb	Added systemd service file ... Fixing #94	2022-12-13 23:01:42 +01:00
pictuga	438c32a312	Remove sqlite & mysql cache backends ... Obsoleted since the introduction of diskcache & redis	2022-12-13 22:40:13 +01:00
pictuga	8b26797e93	README: add recommended install way ... Part of discussions on #94	2022-12-13 22:07:21 +01:00
pictuga	e1ed33f320	crawler: improve html iter code ... Ignores tags without attributes. Avoids bug with unclosed tags.	2022-02-09 15:57:12 +01:00
pictuga	b65272daab	crawler: accept more meta redirects	2022-02-01 23:32:49 +01:00
pictuga	4d64afe9cb	crawler: fix regression from d6b90448f3	2022-02-01 23:18:16 +01:00
pictuga	d3b623482d	pytest: crawler	2022-02-01 23:16:43 +01:00
pictuga	32645548c2	pytest: first batch with test_feeds ... And multiple related fixes	2022-01-31 08:32:34 +01:00
pictuga	d6b90448f3	crawler: improve handling of non-ascii urls	2022-01-30 23:27:49 +01:00
pictuga	da81edc651	log to stderr	2022-01-26 07:57:57 +01:00
pictuga	4f2895f931	cli: update --help	2022-01-25 22:36:57 +01:00
pictuga	b2b04691d6	Ability to pass custom data_files location	2022-01-25 22:36:34 +01:00
pictuga	bfaf7b0fac	feeds: clean up default item_link ... To be supported by feeds' `_rule_parse`	2022-01-24 16:16:14 +00:00
pictuga	32d9bc9d9d	feeds: proceed with conversion when rules do not match	2022-01-24 09:34:57 +00:00
pictuga	b138f11771	util: support more data_files location	2022-01-23 12:40:18 +01:00
pictuga	a01258700d	More ordering options	2022-01-23 12:27:07 +01:00
pictuga	4d6d3c9239	wsgi: limit supported mimetypes & return actual mimetype	2022-01-23 11:44:07 +01:00
pictuga	e81f6b173f	readabilite: remove code duplicate	2022-01-23 11:41:32 +01:00
pictuga	fe5dbf1ce0	wsgi: reuse mimetype table from crawler	2022-01-22 13:22:39 +01:00
pictuga	fdf9acd32b	helper: fix reload code	2022-01-19 13:44:15 +01:00
pictuga	d05706e056	crawler: fix typo	2022-01-19 13:41:12 +01:00
pictuga	e88a823ada	feeds: better handle rulesets without a 'mode' specified	2022-01-19 13:08:33 +01:00
pictuga	750850c162	crawler: avoid too many .append()	2022-01-19 13:04:33 +01:00
pictuga	c8669002e4	feeds: exotic xpath in html as well	2022-01-17 14:22:48 +00:00
pictuga	c524e54d2d	feeds: support some exotic xpath rules returning a single string	2022-01-17 13:59:58 +00:00
pictuga	ef14567d87	Handle morss-helper with setup.py	2022-01-08 16:10:51 +01:00
pictuga	fb643f5ef1	readabilite: remove unneeded reference to features (overriden by builder)	2022-01-03 18:01:12 +00:00
pictuga	dbdca910d8	readabilite: fix new parser code & drop PIs	2022-01-03 17:51:49 +00:00
pictuga	9eb19fac04	readabilite: use custom html parser within bs4's lxml parser ... Solves the following obscure error: ValueError: Invalid PI name 'b'xml''	2022-01-03 16:26:17 +00:00
pictuga	d424e394d1	readabilite: use lxml bs4 parser for speed	2022-01-01 14:52:48 +01:00
pictuga	3f92787b38	readabilite: limit html comments related issues	2022-01-01 13:58:42 +01:00
pictuga	afc31eb6e9	readabilite: avoid double parsing of html	2022-01-01 12:51:30 +01:00
pictuga	87d2fe772d	wsgi: fix py2 compatibility	2022-01-01 12:35:41 +01:00
pictuga	917aa0fbc5	crawler: do not re-save cached response ... Otherwise cache never gets invalidated!	2021-12-31 19:28:11 +01:00
pictuga	3e2b81286f	xsl: add link to favicon ... To limit error output when failing to fetch favicon.ico	2021-12-31 19:25:53 +01:00
pictuga	15430a2b83	helper: restore run if no param passed	2021-12-29 23:35:16 +01:00
pictuga	ecdb74812d	Make helper & main.py executable	2021-12-29 15:47:05 +01:00
pictuga	2c7844942c	drone: re order deploy commands	2021-12-29 15:41:29 +01:00
pictuga	e12cb4567a	helper: more debug options	2021-12-29 15:41:03 +01:00
pictuga	b74365b121	Make helper more posix compliant	2021-12-29 15:40:43 +01:00
pictuga	2020543469	Make morss-helper executable	2021-12-29 15:37:12 +01:00
pictuga	676be4a4fe	helper: work around for systems only having py3 binary	2021-12-29 14:07:12 +01:00
pictuga	8870400a6e	Clean up morss-helper	2021-12-28 16:30:20 +01:00
pictuga	8e9cc541b0	Turns out exec array is not supported in HEALTHCHECK	2021-12-28 15:23:40 +01:00
pictuga	2a7a1b83ec	Use alpine:edge to have up-to-date py packages	2021-12-28 13:41:42 +01:00
pictuga	106f59afa1	docker: shift HEALTHCHECK to helper	2021-12-27 16:08:55 +01:00