50 Commits

Author SHA1 Message Date
Kade
fe8fb82c1e
Revert "updating again" (#11)
* Revert "[francetv] Separate main extractor and rework others to delegate to it"

This reverts commit 24b97ef14a944edeb00bf81ed0ddc990459e3bc3.

* Revert "[francetv] Improve manifest URL signing (closes #15536)"

This reverts commit 6f54dce07d5ab9780e653193a9db53013712f19d.

* Revert "[francetv] Sign m3u8 manifest URLs (closes #15565)"

This reverts commit 4eb4ace765dae302460291a790c5e81f373ce5cc.

* Revert "[veoh] Add support for embed URLs"

This reverts commit 6adfc88ce6034563f6e275c72131cdf87e2d2ab6.

* Revert "[dvtv] Skip download on failing test"

This reverts commit ad4167ed33a00c70d411e68e5745482880ad2855.

* Revert "[afreecatv] Fix extraction (closes #15556)"

This reverts commit 8cc0cd0a2ff7bfe27d3571aa75cab841f6f96f91.

* Revert "[periscope] Use accessVideoPublic endpoint (closes #15554)"

This reverts commit b6aef8f69f16724ca8705823c86a349b97eceeff.

* Revert "[YoutubeDL] Add support for filesize_approx in format selector (closes #15550)"

This reverts commit eb048b366869f19e3138a76af7c31325c7a66b70.

* Revert "[discovery] Fix auth request (closes #15542)"

This reverts commit 14adea5bc7d0c0324716403717b3f234aec4d7bb.

* Revert "[6play] Extract subtitles (closes #15541)"

This reverts commit aee441166254bf3d6810ddaaca2ca6de7f327899.

* Revert "Credit @mweinelt for #15124"

This reverts commit 82eaab7f58f0a1a0ff85d2829ce48dd61bbbcefe.

* Revert "Credit @iamleot for internazionale (#14973)"

This reverts commit fdd87ce20957a903653336621797cea866db204f.

* Revert "Credit @che0 for seznamzpravy (#14616) and dvtv (#15442)"

This reverts commit 9b904aaeba37fa218ddad01bb04a5ea7c8d585aa.

* Revert "[newgrounds] Fix metadata extraction (closes #15531)"

This reverts commit 8be35d3cd121029b0c134cd3b5672c46add7b88d.

* Revert "[nbc] add support for NBC Olympics Streams(closes #10295)"

This reverts commit ba6e116f59c296f78668b1b665657b33e7c40410.

* Revert "[dvtv] Simplify (closes #15442)"

This reverts commit 2da0581a746fe6a98a331a646e64840d7ef78bc0.

* Revert "[dvtv] Fix live streams extraction"

This reverts commit 773234624f2a8e63a382cba91228a0445261d783.

* Revert "release 2018.02.08"

This reverts commit 0204b2a7b183a412c5a13bc306f8d77a0de6f9a8.

* Revert "[ChangeLog] Actualize"

This reverts commit 72aa0a29e54dee40f1b52da5c2e2aa46886dee1b.

* Revert "[extractors] Import for myvi:embed"

This reverts commit 38e519ff0327b5b4067d5c3b0c40d651c8a508f9.

* Revert "[pokemon] PEP 8"

This reverts commit 7b11f92ae02d791a1115537c6bd3c129198a7810.

* Revert "[gameinformer] PEP 8"

This reverts commit aa71431e28de547fb55ab55e47817769c8e91029.

* Revert "[myvi] Extend _VALID_URL"

This reverts commit 441ec65fd33e224bee2e49092a45af9b1c92fd90.

* Revert "[myvi:embed] Add extractor (closes #15521)"

This reverts commit e35a984bc7623226bfba13e6c94a2f3cf8fc76fe.

* Revert "[prosiebensat1] Extend _VALID_URL (closes #15520)"

This reverts commit 0e1a732e390e0e90b48fc54e3b6b61c9accb043a.

* Revert "[pokemon] Relax _VALID_URL and extend title extraction (closes #15518)"

This reverts commit 8a768fe854833d29eab2cb695ed2e154d2b2cf28.

* Revert "[gameinformer] Use geo verification headers"

This reverts commit 6cdc00e9989adf7d2e4a09e838791f39dce7c118.

* Revert "[la7] Fix extraction (closes #15501)"

This reverts commit 8d2fbb06a9fe5821d143039716b058cbd2ae0267.

* Revert "[gameinformer] Fix brightcove id extraction"

This reverts commit dc1171da811f5f055aab82bb34b068bd6496ca47.

* Revert "[afreecatv] Pass referrer to video info request (closes #15507)"

This reverts commit c16af2cb4ea0011b3103751574e0b2e2404b3783.

* Revert "[telebruxelles] Relax _VALID_URL and add support for live streams"

This reverts commit 1f916ea01b9fc6df933d7220100ef14a859fd63c.

* Revert "[telebruxelles] Fix extraction (closes #15504)"

This reverts commit ea7a8f529844d5ccc201d2d689c07bddce91f3c5.

* Revert "[extractor/common] Respect secure schemes in _extract_wowza_formats"

This reverts commit b44f7f4fc8d69ea8da69287c4c94f2dfe4c8aec0.

* Revert "release 2018.02.04"

This reverts commit ed4de10767c633eaa13c4415b4b6819e2cc358c7.

* Revert "[ChangeLog] Actualize"

This reverts commit f277eb6c9d5d995697fc8f37cb8bcb0780be19b8.

* Revert "[brightcove] Pass embed page URL as referrer (closes #15486)"

This reverts commit a690cb67422b4d1b1a4976fa4e51444467ac6656.

* Revert "[downloader/http] Randomize HTTP chunk size"

This reverts commit c315342edb0caa310a184e7c6863cee38abca916.

* Revert "[youtube] Enforce using chunked HTTP downloading for DASH formats"

This reverts commit 7412fe8d5e8f6229135f3f9d2ca70c91cc5a9e11.

* Revert "[downloader/http] Add ability to pass downloader options via info dict"

This reverts commit 6cfd25a8e0bd236e1d276f44430e9a6d2545237c.

* Revert "[downloader/http] Fix 302 infinite loops by not reusing requests"

This reverts commit 57c3eea4e579a0a2e259c409d2139577e382e8f8.

* Revert "Document http_chunk_size"

This reverts commit ff3f520741a721b2022baa99fedbcb94b1c37632.
2018-02-11 09:35:28 -05:00
Sergey M․
c315342edb [downloader/http] Randomize HTTP chunk size 2018-02-11 09:34:38 -05:00
Sergey M․
6cfd25a8e0 [downloader/http] Add ability to pass downloader options via info dict 2018-02-11 09:34:38 -05:00
Sergey M․
57c3eea4e5 [downloader/http] Fix 302 infinite loops by not reusing requests 2018-02-11 09:34:38 -05:00
Sergey M․
ba515388b8
Introduce --http-chunk-size 2018-02-03 23:08:58 +07:00
Sergey M․
bec49996c6
[downloader/http] Return actual download result (closes #14971) 2017-12-13 23:49:05 +07:00
Sergey M․
a3c3a1e128
[http] Rework HTTP downloader
* Simplify code and split into separate routines to facilitate maintaining
* Make retry mechanism work on errors during actual download not only during connection establishment phase
* Retry on ECONNRESET and ETIMEDOUT during reading data from network
* Retry on content too short and various timeout errors
* Show error description on retry
* Closes #506, closes #809, closes #2849, closes #4240, closes #6023, closes #8625, closes #9483
2017-08-27 02:22:30 +07:00
Sergey M․
d3f0687cf7
[downloader/fragment] Use temp file for current fragment 2017-04-24 02:54:17 +07:00
Remita Amine
75a2485407 [fragment,hls,f4m,dash,ism] improve fragment downloading
- resume immediately
- no need to concatenate segments and decrypt them on every resume
- no need to save temp files for segments

and for hls downloader:
- no need to download keys for segments that already downloaded
2017-04-19 11:46:07 +01:00
Yen Chi Hsuan
d54739a2e6
[downloader/http] xattr values should be bytes 2016-10-01 19:58:13 +08:00
Yen Chi Hsuan
d7753d1948
[downloader/http] Use write_xattr function for --xattr-set-filesize 2016-10-01 14:47:20 +08:00
Sergey M․
611c1dd96e [refactor] Single quotes consistency 2016-02-14 15:37:17 +06:00
Sergey M․
d800609c62 [refactor] Do not specify redundant None as second argument in dict.get() 2016-02-14 14:25:04 +06:00
Sergey M?
5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
2015-11-23 21:56:23 +06:00
Sergey M․
84bc4dcb0f [downloader/http] Clarify rationale for Content-Range check (#6426) 2015-08-03 02:27:47 +06:00
Sergey M․
10eaa8ef1d [downloader/http] Report unable to resume 2015-08-03 02:25:40 +06:00
Sergey M․
c3124c3085 [downloader/http] Simplify 2015-08-03 02:25:08 +06:00
remitamine
8d5b8b477e [utils] import re 2015-08-03 02:23:35 +06:00
remitamine
d7d2a9a3db [utils] restart download if server does not support byte ranges 2015-08-03 02:23:31 +06:00
Jaime Marquínez Ferrándiz
2ddcd88129 Remove code that was only used by the Grooveshark extractor 2015-05-02 17:29:56 +02:00
Philipp Hagemeister
c59e701e35 Default to continuedl=True
We already do this in the CLI interface, so it should be just fine.
2015-03-28 08:11:39 +01:00
Jaime Marquínez Ferrándiz
f28fe66970 [downloader/http] Add missing fields for _hook_progress call
It would fail if you run 'youtube-dl --no-part URL' a second time when the file has already been downloaded.

(Reported in Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1195779)
2015-03-04 12:14:38 +01:00
Jaime Marquínez Ferrándiz
123397317c [downloader/http] Remove wrong '_hook_progress' call (fixes #5117) 2015-03-03 18:45:56 +01:00
Philipp Hagemeister
09ab40b7d1 Merge branch 'progress-as-hook2' 2015-02-17 23:41:48 +01:00
Philipp Hagemeister
a91a2c1a83 [downloader] Remove various unneeded assignments and imports 2015-02-17 21:44:41 +01:00
Philipp Hagemeister
16e7711e22 [downloader/http] Remove gruesome import 2015-02-17 21:42:31 +01:00
Philipp Hagemeister
5cda4eda72 [YoutubeDL] Use a progress hook for progress reporting
Instead of every downloader calling two helper functions, let our progress report be an ordinary progress hook like everyone else's.
Closes #4875.
2015-02-17 21:40:35 +01:00
Philipp Hagemeister
1283204917 [http] PEP8 (#4831) 2015-02-02 12:05:39 +01:00
light94
e77d2975af Handling Connection Reset by Peer Error 2015-02-01 00:10:58 +05:30
Philipp Hagemeister
881e6a1f5c Add --xattr-set-filesize option (Fixes #1348) 2015-01-25 04:49:44 +01:00
Jaime Marquínez Ferrándiz
e1554a407d [extractors] Use http_headers for setting the User-Agent and the Referer 2015-01-24 18:23:53 +01:00
Jaime Marquínez Ferrándiz
3fcfb8e9fa [utils] YoutubeDLHandler: don't use 'Youtubedl-user-agent' for overriding the default user agent
Setting the 'User-Agent' header is enough
2015-01-24 18:07:21 +01:00
Philipp Hagemeister
1cc79574fc Fix imports and general cleanup
· Import from compat what comes from compat. Yes, some names are available in utils too, but that's an implementation detail.
· Use _match_id consistently whenever possible
· Fix some outdated tests
· Use consistent valid URL (always match the whole protocol, no ^ at start required)
· Use modern test definitions
2014-12-13 12:35:45 +01:00
Philipp Hagemeister
cf372f0778 Merge remote-tracking branch 'SyxbEaEQ2/rate-limit' 2014-12-12 17:16:13 +01:00
Philipp Hagemeister
4d2d638df4 [http] Modernize 2014-11-26 12:27:36 +01:00
Philipp Hagemeister
b686fc18da [hlsnative] Support test parameter 2014-09-24 14:38:40 +02:00
Sergey M․
88a23aef5a [http] Avoid closing stdout (Fixes #3686) 2014-09-05 22:05:36 +07:00
Philipp Hagemeister
d769be6c96 [grooveshark,http] Make HTTP POST downloads work 2014-08-24 01:31:35 +02:00
SyxbEaEQ2
c7667c2d7f [downloader/(common/http)] Changes calculation of the rate-limit. (Fix #2297, fix #2140, fix #595, fix #2370) 2014-07-31 03:08:24 +02:00
Sergey M․
32fd27ec98 [http] Fix string/None comparison with int while in test 2014-04-30 20:02:17 +07:00
Sergey M․
df297c8794 [http] Improve download mechanism when Range HTTP header is ignored 2014-04-27 09:32:01 +07:00
Sergey M․
a59e40a1ea Replace 'referer' with 'http_referer' 2014-03-25 21:53:26 +07:00
Sergey M․
1d0e49e1c7 Use explicitly set Referer header for downloading 2014-03-25 21:22:27 +07:00
Jaime Marquínez Ferrándiz
22ff5d2105 [http] Use the YoutubeDL.urlopen method 2014-03-07 16:41:42 +01:00
rzhxeo
60daf7f0bb Set resume_len to 0 if download is restarted 2014-02-26 02:47:27 +01:00
Jaime Marquínez Ferrándiz
f89197d73e Some pep8 style fixes 2014-01-25 15:33:23 +01:00
Jaime Marquínez Ferrándiz
7976fcac55 [http] Fix ‘err’ variable not being assigned in an except block (#2045) 2013-12-31 13:44:57 +01:00
Philipp Hagemeister
dabc127362 Remove dead code 2013-12-23 16:03:06 +01:00
Philipp Hagemeister
1538eff6d8 [bliptv] Remove support for direct downloads
This is now handled by the generic IE
2013-12-23 15:49:21 +01:00
Jaime Marquínez Ferrándiz
3bc2ddccc8 Move FileDownloader to its own module and create a new class for each download process
A suitable downloader can be found using the 'get_suitable_downloader' function.

Each subclass implements 'real_download', for downloading an info dict you call the 'download' method, which first checks if the video has already been downloaded
2013-12-11 16:18:48 +01:00