 pictuga
|
c952b85d92
|
crawler: cache 301 HTTP code, for a week
|
2017-03-09 09:37:05 -10:00 |
|
pictuga
|
e8023e4336
|
crawler: remove unused NotInCache error-class
|
2017-03-09 09:35:40 -10:00 |
|
pictuga
|
3385198f4f
|
README: remove removed :force
|
2017-03-08 19:20:43 -10:00 |
|
pictuga
|
385f9eb39a
|
morss: use crawler strict accept for feed
|
2017-03-08 19:05:48 -10:00 |
Florian Muenchbach
|
993ac638a3
|
Added override for auto-detected character encoding of parsed pages.
|
2017-03-08 18:45:20 -10:00 |
|
pictuga
|
627163abff
|
Make cache settings in morss nicer
|
2017-03-08 18:09:24 -10:00 |
|
pictuga
|
e5f8e43659
|
Shifted the <link rel='alternate'/> redirect to crawler
Now using MIMETYPE var from crawler within morss.py
|
2017-03-08 18:03:34 -10:00 |
|
pictuga
|
fb8825b410
|
crawler: parse html to get http-equiv
For sure slower, but way cleaner (and probably more stable)
|
2017-03-08 17:50:57 -10:00 |
|
pictuga
|
92b4a5c57c
|
README: remove :smart doc as it was dropped
|
2017-03-08 17:38:00 -10:00 |
|
Florian Muenchbach
|
543ae89e22
|
Added new 'encoding' option to README.
|
2017-03-08 17:38:00 -10:00 |
|
pictuga
|
f4f6a86147
|
feeds: make wheezy.template mandatory
Cleaner code. Less confusing.
|
2017-03-08 15:38:59 -10:00 |
|
pictuga
|
795a0b281f
|
README: add wheezy.template as dep
|
2017-03-08 15:25:45 -10:00 |
|
pictuga
|
af66ab5980
|
README: simplify the sample file tree for mod_cgi
So that it never gets outdated again...
|
2017-03-08 15:21:26 -10:00 |
|
pictuga
|
fe3f6b7e33
|
README: Fetch->FeedFetch
|
2017-03-08 15:17:40 -10:00 |
|
pictuga
|
ff9690666d
|
Add chardet to requirements.txt
|
2017-03-08 11:46:11 -10:00 |
|
pictuga
|
2b4269e1ac
|
README: add chardet
|
2017-03-08 11:45:13 -10:00 |
|
pictuga
|
ad9bf946ec
|
crawler: use chardet again
Always nice in case no encoding is specified. Somehow got dropped with commit 245ba99. Most probably by accident
|
2017-03-08 11:37:12 -10:00 |
|
pictuga
|
4b8e3d1b8b
|
Improve provided .htaccess
Should be less unsafe...
|
2017-03-04 19:30:30 -10:00 |
|
pictuga
|
3fc89d5359
|
readabilite: improve score for <p>
Helps a lot with bbc, le monde. Might backfire on other websites tho...
|
2017-03-01 18:02:45 -10:00 |
|
pictuga
|
a8ac2ed1ca
|
Turn FeedBefore/After into ItemBefore/After
To reduce the number of loops
|
2017-02-28 23:24:32 -10:00 |
|
pictuga
|
fcc5e8a076
|
Add "Feed/Item" in functions name
To make it instantly clearer what they work on
|
2017-02-28 23:23:15 -10:00 |
|
pictuga
|
60e3311e97
|
Use readabilite properly
Not thru some weird wrapper anymore
|
2017-02-28 22:45:26 -10:00 |
|
pictuga
|
52661985d1
|
Remove one more readability fork reference
|
2017-02-28 19:25:59 -10:00 |
|
pictuga
|
47d33099b2
|
Remove readability fork from deps
Since we now use our own, faster, implementation
|
2017-02-28 19:21:51 -10:00 |
|
pictuga
|
dc8423550f
|
Support xml starting with \s
|
2017-02-25 19:04:32 -10:00 |
|
pictuga
|
e0f533ca31
|
readabilite: test to replace <br/> with div
|
2017-02-25 18:16:15 -10:00 |
|
pictuga
|
c6c113b8a8
|
readabilite: function to clean up the html code
|
2017-02-25 18:15:33 -10:00 |
|
pictuga
|
58d9f65735
|
readabilite: explain the use of .tail
|
2017-02-25 18:14:13 -10:00 |
|
pictuga
|
a5aec8c7a6
|
readability: more keywords to the filter list
Also fixed indentation
|
2017-02-25 18:13:15 -10:00 |
|
pictuga
|
026903ce73
|
crawler: change http header after uncompressing
Change content-encoding to "identity"
|
2017-02-25 18:10:43 -10:00 |
|
pictuga
|
e71fc967ce
|
readabilite: shift "good" tags to a var (list)
So that this list can later be re-used
|
2017-02-25 18:07:28 -10:00 |
|
pictuga
|
b14381f575
|
Use internal readability fork
Much simpler, doesn't clean the html, probably less efficient, but much faster
|
2016-05-31 02:50:03 +02:00 |
|
pictuga
|
2b9bfb47e5
|
Remove :smart and etag headers
Dirty code, not very useful. Use simple cache-control instead.
|
2016-05-31 02:47:49 +02:00 |
|
pictuga
|
4ff80cec86
|
Check argv length before using it
|
2016-05-31 02:46:28 +02:00 |
|
pictuga
|
466d8e47d6
|
Also make buriy's readability port compatible
Should be faster, and it now supports py3
|
2015-08-29 18:33:12 +02:00 |
|
pictuga
|
95d9d847e9
|
:proxy implies :keep
|
2015-08-29 17:48:07 +02:00 |
|
pictuga
|
9b911213b6
|
Convenient README update
Break lines, update info, say something about uwsgi
|
2015-08-29 12:45:36 +02:00 |
|
pictuga
|
554bdb4650
|
Fix "typo" in README
|
2015-08-29 12:32:40 +02:00 |
|
pictuga
|
8a1c00abf0
|
Typo in python version check
|
2015-08-28 19:29:09 +02:00 |
|
pictuga
|
624fa47f4f
|
Allow CLI change of the www/ path
|
2015-08-28 19:22:55 +02:00 |
|
pictuga
|
31fc939d52
|
Allow CLI change of the http server port
|
2015-08-28 19:22:23 +02:00 |
|
pictuga
|
9c7660fd63
|
Make use thru uwsgi easier
Import the main cgi_wrapper as "application" in main.py
|
2015-08-28 19:19:09 +02:00 |
|
pictuga
|
4f9000beed
|
Comment code of launching modes
|
2015-08-28 19:18:09 +02:00 |
|
pictuga
|
5e87b56a03
|
Return error code in plain text in file server
|
2015-08-28 19:16:15 +02:00 |
|
pictuga
|
ffda3fac7e
|
Improve file detection in web server
|
2015-08-28 19:15:40 +02:00 |
|
pictuga
|
6741a408dd
|
Remove now-useless ca-cert file path
|
2015-08-28 19:13:54 +02:00 |
Massimo Vannucci
|
8656e53b84
|
Correct Python version check
|
2015-08-05 23:36:11 +02:00 |
|
Massimo Vannucci
|
098a306c91
|
Fixed typo
|
2015-08-05 23:24:44 +02:00 |
|
pictuga
|
5c2151ffd6
|
Improve widely feedsportal url decoder
|
2015-06-14 20:32:47 +08:00 |
|
pictuga
|
8418212475
|
Use good path for html template access
|
2015-05-04 22:26:31 +08:00 |