pictuga/morss
1
1
Fork 0
Code Issues Pull Requests Actions Packages Releases Activity
1,040 Commits 1 Branch 2 Tags
Commit Graph

1040 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
pictuga
539a48d401 Switch from Drone to Gitea Actions
Some checks failed
default / test-lint (push) Failing after 7m35s
Details
default / publish-deploy (push) Failing after 18s
Details
default / docker-publish (push) Failing after 6m16s
Details
2023-05-17 22:38:12 +02:00
Nesswit
8bdcd8f386 Add mode option 2023-05-04 16:01:52 +09:00
pictuga
ea2ebedfcb Added systemd service file
Some checks failed
continuous-integration/drone/push Build is failing
Details 
Fixing #94
 2022-12-13 23:01:42 +01:00
pictuga
438c32a312 Remove sqlite & mysql cache backends
Some checks failed
continuous-integration/drone/push Build is failing
Details 
Obsoleted since the introduction of diskcache & redis
 2022-12-13 22:40:13 +01:00
pictuga
8b26797e93 README: add recommended install way
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
continuous-integration/drone Build is passing
Details 
Part of discussions on #94
 2022-12-13 22:07:21 +01:00
pictuga
e1ed33f320 crawler: improve html iter code
All checks were successful
continuous-integration/drone/push Build is passing
Details 
Ignores tags without attributes. Avoids bug with unclosed tags.
 2022-02-09 15:57:12 +01:00
pictuga
b65272daab crawler: accept more meta redirects
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-02-01 23:32:49 +01:00
pictuga
4d64afe9cb crawler: fix regression from d6b90448f3dbfd07d8b6fdfb5bbb20f28b3a331f
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-02-01 23:18:16 +01:00
pictuga
d3b623482d pytest: crawler 2022-02-01 23:16:43 +01:00
pictuga
32645548c2 pytest: first batch with test_feeds
Some checks failed
continuous-integration/drone/push Build is failing
Details 
And multiple related fixes
 2022-01-31 08:32:34 +01:00
pictuga
d6b90448f3 crawler: improve handling of non-ascii urls 2022-01-30 23:27:49 +01:00
pictuga
da81edc651 log to stderr
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-01-26 07:57:57 +01:00
pictuga
4f2895f931 cli: update --help
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-01-25 22:36:57 +01:00
pictuga
b2b04691d6 Ability to pass custom data_files location 2022-01-25 22:36:34 +01:00
pictuga
bfaf7b0fac feeds: clean up default item_link
Some checks failed
continuous-integration/drone/push Build is failing
Details 
To be supported by feeds' `_rule_parse`
 2022-01-24 16:16:14 +00:00
pictuga
32d9bc9d9d feeds: proceed with conversion when rules do not match
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-01-24 09:34:57 +00:00
pictuga
b138f11771 util: support more data_files location
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-23 12:40:18 +01:00
pictuga
a01258700d More ordering options
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
2022-01-23 12:27:07 +01:00
pictuga
4d6d3c9239 wsgi: limit supported mimetypes & return actual mimetype
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-23 11:44:07 +01:00
pictuga
e81f6b173f readabilite: remove code duplicate 2022-01-23 11:41:32 +01:00
pictuga
fe5dbf1ce0 wsgi: reuse mimetype table from crawler 2022-01-22 13:22:39 +01:00
pictuga
fdf9acd32b helper: fix reload code
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-19 13:44:15 +01:00
pictuga
d05706e056 crawler: fix typo
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
2022-01-19 13:41:12 +01:00
pictuga
e88a823ada feeds: better handle rulesets without a 'mode' specified
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-01-19 13:08:33 +01:00
pictuga
750850c162 crawler: avoid too many .append() 2022-01-19 13:04:33 +01:00
pictuga
c8669002e4 feeds: exotic xpath in html as well
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-17 14:22:48 +00:00
pictuga
c524e54d2d feeds: support some exotic xpath rules returning a single string
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-17 13:59:58 +00:00
pictuga
ef14567d87 Handle morss-helper with setup.py
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-08 16:10:51 +01:00
pictuga
fb643f5ef1 readabilite: remove unneeded reference to features (overriden by builder)
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-03 18:01:12 +00:00
pictuga
dbdca910d8 readabilite: fix new parser code & drop PIs
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
2022-01-03 17:51:49 +00:00
pictuga
9eb19fac04 readabilite: use custom html parser within bs4's lxml parser
All checks were successful
continuous-integration/drone/push Build is passing
Details 
Solves the following obscure error:
ValueError: Invalid PI name 'b'xml''
 2022-01-03 16:26:17 +00:00
pictuga
d424e394d1 readabilite: use lxml bs4 parser for speed
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-01 14:52:48 +01:00
pictuga
3f92787b38 readabilite: limit html comments related issues
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-01 13:58:42 +01:00
pictuga
afc31eb6e9 readabilite: avoid double parsing of html
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-01 12:51:30 +01:00
pictuga
87d2fe772d wsgi: fix py2 compatibility 2022-01-01 12:35:41 +01:00
pictuga
917aa0fbc5 crawler: do not re-save cached response
All checks were successful
continuous-integration/drone/push Build is passing
Details 
Otherwise cache never gets invalidated!
 2021-12-31 19:28:11 +01:00
pictuga
3e2b81286f xsl: add link to favicon 
To limit error output when failing to fetch favicon.ico
 2021-12-31 19:25:53 +01:00
pictuga
15430a2b83 helper: restore run if no param passed
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-29 23:35:16 +01:00
pictuga
ecdb74812d Make helper & main.py executable
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-29 15:47:05 +01:00
pictuga
2c7844942c drone: re order deploy commands
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
2021-12-29 15:41:29 +01:00
pictuga
e12cb4567a helper: more debug options 2021-12-29 15:41:03 +01:00
pictuga
b74365b121 Make helper more posix compliant 2021-12-29 15:40:43 +01:00
pictuga
2020543469 Make morss-helper executable 2021-12-29 15:37:12 +01:00
pictuga
676be4a4fe helper: work around for systems only having py3 binary
Some checks are pending
continuous-integration/drone/push Build is running
Details
2021-12-29 14:07:12 +01:00
pictuga
8870400a6e Clean up morss-helper
Some checks failed
continuous-integration/drone/push Build is failing
Details
2021-12-28 16:30:20 +01:00
pictuga
8e9cc541b0 Turns out exec array is not supported in HEALTHCHECK
Some checks failed
continuous-integration/drone/push Build is failing
Details
2021-12-28 15:23:40 +01:00
pictuga
2a7a1b83ec Use alpine:edge to have up-to-date py packages
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-28 13:41:42 +01:00
pictuga
106f59afa1 docker: shift HEALTHCHECK to helper
Some checks failed
continuous-integration/drone/push Build is failing
Details
2021-12-27 16:08:55 +01:00
pictuga
ee514e2da3 helper: remove unneeded sudo
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-25 22:00:22 +00:00
pictuga
e7578e859a Clean up install/exec
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-25 18:21:55 +01:00