957 Commits

Author SHA1 Message Date
TinyToweringTree
b98d1c0d5a [archiveorg] Use and fix get_element_by_class()
Use get_element_by_class() from utils to get rid of yet another regex.
This function used to return only the content of the element, and not
the element itself, including its tag and attributes. The whole group
of get_element_by_X() functions are a bit of a misnomer, as they all
return the *content* of the element and not the element itself.

All these functions can now return the whole element when setting
their `include_tag` parameter to `True`. By default it is `False` so
no other code will be affected by this change. Tests have been added
to test/test_utils.py accordingly.

This uncovered a bug which prevented elements starting with a hyphen as
their class name from being found. This has been fixed by fixing the
regex used in get_elements_by_class().
2020-02-19 22:42:00 +01:00
Sergey M․
42db58ec73
[utils] Improve str_to_int 2019-12-15 23:15:24 +07:00
Remita Amine
348c6bf1c1 [utils] handle int values passed to str_to_int 2019-11-29 17:39:18 +01:00
Sergey M․
1ced222120
[utils] Add generic caesar cipher and rot47 2019-11-27 02:26:42 +07:00
InfernalUnderling
9d30c2132a [utils] Handle rd-suffixed day parts in unified_strdate (#23199) 2019-11-27 00:08:37 +07:00
Remita Amine
237513e801 [yahoo] restore support for cbs suffixed URLs 2019-10-31 07:38:53 +01:00
Sergey M․
824fa51165
[utils] Improve subtitles_filename (closes #22753) 2019-10-18 04:03:53 +07:00
Sergey M․
28cc2241e4
[utils] Restrict parse_codecs and add theora as known vcodec (#21381) 2019-06-14 01:56:17 +07:00
Sergey M․
53cd37bac5
[utils] Improve strip_or_none 2019-05-24 00:03:01 +07:00
Sergey M․
3089bc748c
Fix W504 and disable W503 (closes #20863) 2019-05-11 03:57:40 +07:00
Jakub Wilk
fd35d8cdfd [utils] Transliterate "þ" as "th" (#20897)
Despite visual similarity "þ" is unrelated to "p".
It is normally transliterated as "th":

    $ echo þ-Þ | iconv -t ASCII//TRANSLIT
    th-TH
2019-05-11 01:42:31 +07:00
Sergey M․
5e1271c56d
[utils] Improve int_or_none and float_or_none (#20403) 2019-03-23 01:08:54 +07:00
Sergey M․
d493f15c11
[extractor/common] Improve HTML5 entries extraction and add some realworld tests 2019-03-17 09:09:32 +07:00
Sergey M․
0dc41787af
[utils] Introduce parse_bitrate 2019-03-17 09:07:47 +07:00
Sergey M․
2e27421c70
[test_InfoExtractor] Add test for #20346 2019-03-15 01:20:24 +07:00
Sergey M․
067aa17edf
Start moving to ytdl-org 2019-03-11 04:00:54 +07:00
Sergey M․
fca9baf0da
[test] Fix test_compat_etree_Element 2019-03-06 02:46:26 +07:00
Sergey M․
399f76870d
[compat] Introduce compat_etree_Element 2019-03-06 01:18:52 +07:00
remitamine
e7e62441cd [utils] strip #HttpOnly_ prefix from cookies files (#20219) 2019-03-03 19:23:59 +07:00
Ales Jirasek
22f5f5c6fc
[malltv] Add extractor (closes #18058) 2019-02-08 00:43:26 +07:00
Sergey M․
e118a8794f
[YoutubeDL] Fix typo in string negation implementation and add more tests (closes #18961) 2019-01-24 01:39:39 +07:00
Sergey M․
fad4ceb534
[utils] Fix urljoin for paths with non-http(s) schemes 2019-01-20 20:22:19 +07:00
Remita Amine
fc746c3fdd [test/test_InfoExtractor] add test for #18923 2019-01-20 09:05:12 +01:00
Sergey M․
2cc779f497
[YoutubeDL] Add negation support for string comparisons in format selection expressions (closes #18600, closes #18805) 2019-01-20 13:48:49 +07:00
Sergey M․
a16c7c033a
[test/helper] Add support for maxcount and count collection len test checkers 2019-01-16 02:17:49 +07:00
Sergey M․
6e29458f24
[test/testdata/cookies/session_cookies.txt] Fix empty expires test data 2018-12-10 04:30:00 +07:00
Sergey M․
9e02c2c704
[YoutubeDLCookieJar] Add test for keeping session cookies 2018-12-09 22:57:00 +07:00
Sergey M․
6864855eb1
[tests] Fix invalid escape sequences 2018-11-23 00:43:42 +07:00
Xiao Di Guan
95e42d7336 [extractor/common] Ensure response handle is not prematurely closed before it can be read if it matches expected_status (resolves #17195, closes #17846, resolves #17447) 2018-11-03 01:18:20 +07:00
Sergey M․
25d110be30
[utils] Properly recognize AV1 codec (closes #17506) 2018-09-10 02:37:22 +07:00
Sergey M․
af03000ad5
[utils] Introduce url_or_none 2018-07-21 18:03:58 +07:00
Sergey M․
e9c671d5e8
[utils] Allow JSONP with empty func name (closes #17028) 2018-07-21 12:30:18 +07:00
Enes
85750f8972 [openload] Improve ext extraction 2018-06-02 00:16:22 +07:00
Remita Amine
3bb3ff38a1 [test_utils] add tests for b836118724122a639a1cb78d55d91724bf1e7251 2018-05-23 12:20:05 +01:00
Sergey M․
6cc622327f
[utils] Introduce merge_dicts 2018-04-28 02:47:17 +07:00
Sergey M․
1cc47c6674
[utils] Fix match_str for boolean meta fields 2018-04-24 23:54:49 +07:00
Dan Salmon
518d5ba519 Fix some tests 2018-04-18 00:10:02 +07:00
Sergey M․
47a5cb7734
Generalize XML manifest processing code and improve XSPF parsing (closes #15794) 2018-03-18 02:52:17 +07:00
Ricardo Constantino
96b8b9abae
[extractor/generic] Support relative URIs in _parse_xspf
<location> can have relative URIs, not just absolute.
2018-03-18 02:48:44 +07:00
Philipp Hagemeister
f226880c6d [tennistv] Add support for tennistv.com 2018-03-14 09:55:21 +01:00
Sergey M․
b871d7e954
[utils] Add parse_resolution 2018-03-02 23:39:04 +07:00
Sergey M․
befa4708fd
[utils] Fixup some common URL's typos in sanitize_url (closes #15649) 2018-02-19 22:50:23 +07:00
Sergey M․
6fcc053947
[test_downloader_http] Use try_rm 2018-02-03 23:24:27 +07:00
Sergey M․
f19eae429a
[test_http] Use 127.0.0.1 instead of localhost 2018-02-03 23:09:25 +07:00
Sergey M․
ba515388b8
Introduce --http-chunk-size 2018-02-03 23:08:58 +07:00
squibbysquibby
655c410063 [test_download] Fix download tests for lazy extractors (closes #13554, closes #13757) 2018-01-21 18:15:11 +07:00
Sergey M․
c707b1d828
[test_utils] Add tests for malformed JSON handling in js_to_json 2018-01-20 23:00:09 +07:00
Mike Fährmann
c384d537f8 [util] Improve scientific notation handling in js_to_json (closes #14789) 2018-01-20 22:54:21 +07:00
Sergey M․
9d6ac71c27
[extractor/common] Fix extraction of DASH formats with the same representation id (closes #15111) 2017-12-29 23:14:56 +07:00
Sergey M․
b555ae9bf1
[utils] Add another date format pattern (#14999) 2017-12-16 21:56:16 +07:00