47 Commits

Author SHA1 Message Date
Kade
fe8fb82c1e
Revert "updating again" (#11)
* Revert "[francetv] Separate main extractor and rework others to delegate to it"

This reverts commit 24b97ef14a944edeb00bf81ed0ddc990459e3bc3.

* Revert "[francetv] Improve manifest URL signing (closes #15536)"

This reverts commit 6f54dce07d5ab9780e653193a9db53013712f19d.

* Revert "[francetv] Sign m3u8 manifest URLs (closes #15565)"

This reverts commit 4eb4ace765dae302460291a790c5e81f373ce5cc.

* Revert "[veoh] Add support for embed URLs"

This reverts commit 6adfc88ce6034563f6e275c72131cdf87e2d2ab6.

* Revert "[dvtv] Skip download on failing test"

This reverts commit ad4167ed33a00c70d411e68e5745482880ad2855.

* Revert "[afreecatv] Fix extraction (closes #15556)"

This reverts commit 8cc0cd0a2ff7bfe27d3571aa75cab841f6f96f91.

* Revert "[periscope] Use accessVideoPublic endpoint (closes #15554)"

This reverts commit b6aef8f69f16724ca8705823c86a349b97eceeff.

* Revert "[YoutubeDL] Add support for filesize_approx in format selector (closes #15550)"

This reverts commit eb048b366869f19e3138a76af7c31325c7a66b70.

* Revert "[discovery] Fix auth request (closes #15542)"

This reverts commit 14adea5bc7d0c0324716403717b3f234aec4d7bb.

* Revert "[6play] Extract subtitles (closes #15541)"

This reverts commit aee441166254bf3d6810ddaaca2ca6de7f327899.

* Revert "Credit @mweinelt for #15124"

This reverts commit 82eaab7f58f0a1a0ff85d2829ce48dd61bbbcefe.

* Revert "Credit @iamleot for internazionale (#14973)"

This reverts commit fdd87ce20957a903653336621797cea866db204f.

* Revert "Credit @che0 for seznamzpravy (#14616) and dvtv (#15442)"

This reverts commit 9b904aaeba37fa218ddad01bb04a5ea7c8d585aa.

* Revert "[newgrounds] Fix metadata extraction (closes #15531)"

This reverts commit 8be35d3cd121029b0c134cd3b5672c46add7b88d.

* Revert "[nbc] add support for NBC Olympics Streams(closes #10295)"

This reverts commit ba6e116f59c296f78668b1b665657b33e7c40410.

* Revert "[dvtv] Simplify (closes #15442)"

This reverts commit 2da0581a746fe6a98a331a646e64840d7ef78bc0.

* Revert "[dvtv] Fix live streams extraction"

This reverts commit 773234624f2a8e63a382cba91228a0445261d783.

* Revert "release 2018.02.08"

This reverts commit 0204b2a7b183a412c5a13bc306f8d77a0de6f9a8.

* Revert "[ChangeLog] Actualize"

This reverts commit 72aa0a29e54dee40f1b52da5c2e2aa46886dee1b.

* Revert "[extractors] Import for myvi:embed"

This reverts commit 38e519ff0327b5b4067d5c3b0c40d651c8a508f9.

* Revert "[pokemon] PEP 8"

This reverts commit 7b11f92ae02d791a1115537c6bd3c129198a7810.

* Revert "[gameinformer] PEP 8"

This reverts commit aa71431e28de547fb55ab55e47817769c8e91029.

* Revert "[myvi] Extend _VALID_URL"

This reverts commit 441ec65fd33e224bee2e49092a45af9b1c92fd90.

* Revert "[myvi:embed] Add extractor (closes #15521)"

This reverts commit e35a984bc7623226bfba13e6c94a2f3cf8fc76fe.

* Revert "[prosiebensat1] Extend _VALID_URL (closes #15520)"

This reverts commit 0e1a732e390e0e90b48fc54e3b6b61c9accb043a.

* Revert "[pokemon] Relax _VALID_URL and extend title extraction (closes #15518)"

This reverts commit 8a768fe854833d29eab2cb695ed2e154d2b2cf28.

* Revert "[gameinformer] Use geo verification headers"

This reverts commit 6cdc00e9989adf7d2e4a09e838791f39dce7c118.

* Revert "[la7] Fix extraction (closes #15501)"

This reverts commit 8d2fbb06a9fe5821d143039716b058cbd2ae0267.

* Revert "[gameinformer] Fix brightcove id extraction"

This reverts commit dc1171da811f5f055aab82bb34b068bd6496ca47.

* Revert "[afreecatv] Pass referrer to video info request (closes #15507)"

This reverts commit c16af2cb4ea0011b3103751574e0b2e2404b3783.

* Revert "[telebruxelles] Relax _VALID_URL and add support for live streams"

This reverts commit 1f916ea01b9fc6df933d7220100ef14a859fd63c.

* Revert "[telebruxelles] Fix extraction (closes #15504)"

This reverts commit ea7a8f529844d5ccc201d2d689c07bddce91f3c5.

* Revert "[extractor/common] Respect secure schemes in _extract_wowza_formats"

This reverts commit b44f7f4fc8d69ea8da69287c4c94f2dfe4c8aec0.

* Revert "release 2018.02.04"

This reverts commit ed4de10767c633eaa13c4415b4b6819e2cc358c7.

* Revert "[ChangeLog] Actualize"

This reverts commit f277eb6c9d5d995697fc8f37cb8bcb0780be19b8.

* Revert "[brightcove] Pass embed page URL as referrer (closes #15486)"

This reverts commit a690cb67422b4d1b1a4976fa4e51444467ac6656.

* Revert "[downloader/http] Randomize HTTP chunk size"

This reverts commit c315342edb0caa310a184e7c6863cee38abca916.

* Revert "[youtube] Enforce using chunked HTTP downloading for DASH formats"

This reverts commit 7412fe8d5e8f6229135f3f9d2ca70c91cc5a9e11.

* Revert "[downloader/http] Add ability to pass downloader options via info dict"

This reverts commit 6cfd25a8e0bd236e1d276f44430e9a6d2545237c.

* Revert "[downloader/http] Fix 302 infinite loops by not reusing requests"

This reverts commit 57c3eea4e579a0a2e259c409d2139577e382e8f8.

* Revert "Document http_chunk_size"

This reverts commit ff3f520741a721b2022baa99fedbcb94b1c37632.
2018-02-11 09:35:28 -05:00
Sergey M․
ff3f520741 Document http_chunk_size 2018-02-11 09:34:38 -05:00
Sergey M․
a3c3a1e128
[http] Rework HTTP downloader
* Simplify code and split into separate routines to facilitate maintaining
* Make retry mechanism work on errors during actual download not only during connection establishment phase
* Retry on ECONNRESET and ETIMEDOUT during reading data from network
* Retry on content too short and various timeout errors
* Show error description on retry
* Closes #506, closes #809, closes #2849, closes #4240, closes #6023, closes #8625, closes #9483
2017-08-27 02:22:30 +07:00
Sergey M․
1433734c35
[downloader/common] Use utils.shell_quote for debug command line 2017-06-17 23:50:21 +07:00
Remita Amine
ea0c2f219c [downloader/fragment] use a general file to store fragment download context 2017-04-19 18:53:15 +01:00
Remita Amine
75a2485407 [fragment,hls,f4m,dash,ism] improve fragment downloading
- resume immediately
- no need to concatenate segments and decrypt them on every resume
- no need to save temp files for segments

and for hls downloader:
- no need to download keys for segments that already downloaded
2017-04-19 11:46:07 +01:00
Pratyush Singh
d374d943f3
[downloader/common] Limit displaying 2 digits after decimal point in sleep interval message 2017-02-25 20:59:04 +07:00
Sergey M․
bcd6276520
[downloader/common] Remove debug output 2016-10-12 21:22:33 +07:00
Sergey M․
1ad6b891b2
Add more checks for --min/max-sleep-interval arguments and use more idiomatic naming 2016-08-09 03:47:56 +07:00
singh-pratyush96
065bc35489
Add --max-sleep-interval (Closes #9930) 2016-08-09 03:32:42 +07:00
Sergey M․
617e58d850 [downloader/{common,fragment}] Fix total retries reporting on python 2.6 2016-03-19 20:51:30 +06:00
Yen Chi Hsuan
e9c0cdd389 [jython] Introduce compat_os_name
os.name is always 'java' on Jython
2016-03-03 19:24:24 +08:00
Sergey M․
d800609c62 [refactor] Do not specify redundant None as second argument in dict.get() 2016-02-14 14:25:04 +06:00
Jaime Marquínez Ferrándiz
7d106a65ca Add --hls-use-mpegts option
When using the mpegts container hls vidoes can be played while being downloaded (useful if you are recording a live stream).
VLC and mpv play them file, but QuickTime doesn't.
2016-01-30 12:26:40 +01:00
Jaime Marquínez Ferrándiz
32d77eeb04 [downloader/common] report_retry: Don't crash when retries is infinite (fixes #8299) 2016-01-22 14:49:17 +01:00
Sergey M․
9b9c5355e4 Rename error_to_str to error_to_compat_str 2015-12-20 07:00:39 +06:00
Sergey M․
7f8b271465 Properly convert errors to strings 2015-12-20 05:27:38 +06:00
ashutosh-mishra
17cc153435 Typo fix, found while going through the code. 2015-11-20 22:51:46 +05:30
Sergey M․
964e7b2dd0 [downloader/common] Always skip "already downloaded" check when outputting to stdout 2015-10-12 00:43:54 +06:00
Sergey M․
cd8a07a764 [downloader/common] Use decodeArgument 2015-04-26 04:30:45 +06:00
Philipp Hagemeister
c59e701e35 Default to continuedl=True
We already do this in the CLI interface, so it should be just fine.
2015-03-28 08:11:39 +01:00
Jaime Marquínez Ferrándiz
70a1165b32 Don't use bare 'except:'
They catch any exception, including KeyboardInterrupt, we don't want to catch it.
2015-03-27 13:02:20 +01:00
Philipp Hagemeister
c75f0b361a [downloader/external] Add support for custom options (Fixes #4885, closes #5098) 2015-03-02 18:21:31 +01:00
Philipp Hagemeister
8fb3ac3649 PEP8: W503 2015-02-21 14:55:13 +01:00
Philipp Hagemeister
5cda4eda72 [YoutubeDL] Use a progress hook for progress reporting
Instead of every downloader calling two helper functions, let our progress report be an ordinary progress hook like everyone else's.
Closes #4875.
2015-02-17 21:40:35 +01:00
Philipp Hagemeister
881e6a1f5c Add --xattr-set-filesize option (Fixes #1348) 2015-01-25 04:49:44 +01:00
Philipp Hagemeister
222516d97d [downloader] Lay groundwork for external downloaders.
This comes with a very simply implementation for wget; the real work is in setting up the infrastructure.
2015-01-24 01:38:48 +01:00
Philipp Hagemeister
5f0d813d93 Merge remote-tracking branch 'rupertbaxter2/master'
Conflicts:
	youtube_dl/__init__.py
	youtube_dl/downloader/common.py
2015-01-23 12:05:01 +01:00
Pierre
4340decad2 check for overwriting files in the downloader (fixes #3916, closes #3829) 2015-01-11 12:02:27 +01:00
Philipp Hagemeister
fdc8000810 [downloader] Handle a file ./- (Fixes #4498) 2014-12-17 11:39:06 +01:00
Philipp Hagemeister
71b640cc5b [YoutubeDL] Add declarative version of progress hooks 2014-12-15 01:26:20 +01:00
Philipp Hagemeister
1cc79574fc Fix imports and general cleanup
· Import from compat what comes from compat. Yes, some names are available in utils too, but that's an implementation detail.
· Use _match_id consistently whenever possible
· Fix some outdated tests
· Use consistent valid URL (always match the whole protocol, no ^ at start required)
· Use modern test definitions
2014-12-13 12:35:45 +01:00
Philipp Hagemeister
cc8c9281e6 [downloader/common] Do not use classic int division 2014-12-12 17:17:09 +01:00
Philipp Hagemeister
cf372f0778 Merge remote-tracking branch 'SyxbEaEQ2/rate-limit' 2014-12-12 17:16:13 +01:00
Jouke Waleson
5f6a1245ff PEP8 applied 2014-11-23 20:41:03 +01:00
Philipp Hagemeister
b6b70730bf [downloader/common] Modernize 2014-11-16 15:06:59 +01:00
Philipp Hagemeister
b686fc18da [hlsnative] Support test parameter 2014-09-24 14:38:40 +02:00
rupertbaxter2
a42c921598 Removed sleep and sleep output when interval is zero 2014-08-13 04:38:40 -07:00
rupertbaxter2
f96252b913 Merge remote-tracking branch 'upstream/master' 2014-08-13 04:22:45 -07:00
Sergey M․
c767dc74b8 [downloader/common] Fix typo 2014-08-10 01:41:01 +07:00
SyxbEaEQ2
00cf122d7a [downloader/common] Fix possible negative sleep time in slow_down() 2014-08-06 20:53:04 +02:00
rupertbaxter2
03359e9864 Added --sleep-interval option 2014-08-03 07:34:04 -07:00
SyxbEaEQ2
c7667c2d7f [downloader/(common/http)] Changes calculation of the rate-limit. (Fix #2297, fix #2140, fix #595, fix #2370) 2014-07-31 03:08:24 +02:00
Jaime Marquínez Ferrándiz
e3ced9ed61 [downloader/common] Use compat_str with the error in try_rename (appeared in #2389)
Otherwise on python 2.x we get `UnicodeDecodeError` because it may contain non ascii characters.
2014-04-04 14:59:11 +02:00
Jaime Marquínez Ferrándiz
f89197d73e Some pep8 style fixes 2014-01-25 15:33:23 +01:00
Philipp Hagemeister
dabc127362 Remove dead code 2013-12-23 16:03:06 +01:00
Jaime Marquínez Ferrándiz
3bc2ddccc8 Move FileDownloader to its own module and create a new class for each download process
A suitable downloader can be found using the 'get_suitable_downloader' function.

Each subclass implements 'real_download', for downloading an info dict you call the 'download' method, which first checks if the video has already been downloaded
2013-12-11 16:18:48 +01:00