31242 Commits

Author SHA1 Message Date
Thomas Tsiakalakis
193c93cf39 Merge branch 'ytdl-org-master' 2020-09-11 22:07:25 +02:00
Thomas Tsiakalakis
a5828ee1ff Merge branch 'master' of https://github.com/ytdl-org/youtube-dl into ytdl-org-master 2020-09-11 22:06:59 +02:00
Vilim Lendvaj
44a5a32080
Update to new Niconico playlist format 2020-09-08 13:57:38 +02:00
siikamiika
942fffbfc1 remove debug code 2020-09-08 00:31:08 +03:00
siikamiika
74e939dae2 handle errors 2020-09-08 00:31:08 +03:00
siikamiika
b8d4a87560 harden regex with lookbehind 2020-09-08 00:31:08 +03:00
siikamiika
13114df673 fix ytInitialData parsing 2020-09-08 00:31:08 +03:00
siikamiika
17d14e05f7 fix premiere live chat
They have isLiveContent = false so just check if the live chat renderer
continuation exists
2020-09-08 00:31:08 +03:00
siikamiika
b8eff9125a flake8 2020-09-08 00:31:08 +03:00
siikamiika
8ff9f315b0 attempt to fix syntax error on older python 2020-09-08 00:31:08 +03:00
siikamiika
b50548d893 rename variable 2020-09-08 00:31:08 +03:00
siikamiika
d3fd1d8b93 run flake8 2020-09-08 00:31:08 +03:00
siikamiika
f671e41ffe check live chat replay existence 2020-09-08 00:31:08 +03:00
siikamiika
2c4a7e0f68 support youtube live chat replay 2020-09-08 00:31:08 +03:00
siikamiika
a72a13dc4a use dl function for subtitles 2020-09-08 00:31:08 +03:00
tsukumijima
dfff793dae [niconico] fix "E225 missing whitespace around operator" error in flake8 2020-09-08 03:44:35 +09:00
tsukumijima
697fd6905d Merge remote-tracking branch 'upstream/master' into fix/niconico 2020-09-07 21:19:29 +09:00
Julian Kornberger
16c4f31c14 [youtube] Move comments into note field
This way they can be accessed externally.
2020-09-06 20:53:16 +02:00
oleksis
6a7dee2451 Merge branch 'master' of https://github.com/ytdl-org/youtube-dl into PictaExtractor 2020-09-06 11:02:31 -04:00
Mattias Wadman
ca058603e5 [svt] Add timestamp and thumbnail in more cases
Add timestamp, set to "valid from" which i think could been seen as publish time.
Add thumbnail in more cases, seems to was only done in the embedded data case for some reason.
Update API URL (current one redirects to it).
Switch svtplay test url to an existing video and also one with no expire date.
Also add an additional thumbnail url test regex.
2020-09-06 11:15:17 +02:00
xarantolus
c4a1d0eef8 Merge branch 'fixYTSearch' of https://github.com/xarantolus/youtube-dl into fixYTSearch 2020-09-06 09:25:02 +02:00
xarantolus
bea9b00588 Fix regex for other variable declaration type
This now supports declarations like `window["ytInitialData"] = ...` and `var ytInitialData = ...`
2020-09-06 09:22:49 +02:00
xarantolus
f442082a50 [youtube] More general title extraction
Seems like this attribute is moved every few weeks, so we just extract both and use the one that is present.
2020-09-06 09:22:48 +02:00
xarantolus
1f93faf60b [youtube] Make ytcfg.set config extraction non-fatal
If the markup of the page changes in the future, it might be possible that _FEED_DATA still works, but the other regex does not. SInce it is not necessary for the first page of videos, we make sure the program doesn't exit before extracting them.

TL;DR: Extract the first video page even if there are problems
2020-09-06 09:22:47 +02:00
xarantolus
299056ad52 Fix python2 compatibility and title extraction 2020-09-06 09:22:47 +02:00
xarantolus
f536080701 Run formatter 2020-09-06 09:22:46 +02:00
xarantolus
5c430b67bd [youtube] Fix feed extraction
In order to extract videos from further pages, we need to get various variables that are in an argument to the `ytcfg.set` call in a script on the feed page.
2020-09-06 09:22:46 +02:00
xarantolus
e03b4f3e05 [youtube] Make search extraction less dependent on json schema.
If an object looks like a video (it has a `videoId` key), assume that it is.
2020-09-06 09:22:30 +02:00
xarantolus
19f671f88b [youtube] Move search URL extraction to appropriate extractor 2020-09-06 09:21:26 +02:00
xarantolus
b948643f9c [youtube] Fix extraction of search urls (closes ytdl-org/youtube-dl#25696) 2020-09-06 09:21:25 +02:00
Sergey M․
d51e23d9fc
release 2020.09.06 2020.09.06 2020-09-06 13:00:41 +07:00
Sergey M․
6cd452acff
[ChangeLog] Actualize
[ci skip]
2020-09-06 12:57:56 +07:00
Sergey M․
50e9fcc1fd
[nrktv:episode] Improve video id extraction (closes #25594, closes #26369, closes #26409) 2020-09-06 12:43:50 +07:00
random-nick
16ee69c1b7
[youtube] Fix age gate content detection (#26100) (closes #26152, closes #26311, closes #26384) 2020-09-06 11:44:53 +07:00
Sergey M
82128d2332
Update youtube.py 2020-09-06 11:39:33 +07:00
Sergey M․
67171ed7e9
[youtube:user] Extend _VALID_URL (closes #26443) 2020-09-06 11:31:28 +07:00
Sergey M․
1d9bf655e6
[utils] Recognize wav mimetype (closes #26463) 2020-09-06 11:19:53 +07:00
TheRealDude2
62ae19ff76
[xhamster] Improve initials regex (#26526) (closes #26353) 2020-09-06 11:10:27 +07:00
Sergey M․
5ed05f26ad
[svtplay] Fix svt id extraction (closes #26425, closes #26428, closes #26438) 2020-09-06 10:45:57 +07:00
Sergey M․
841b683804
[twitch] Rework extractors (closes #12297, closes #20414, closes #20604, closes #21811, closes #21812, closes #22979, closes #24263, closes #25010, closes #25553, closes #25606)
* Switch to GraphQL.
+ Add support for collections.
+ Add support for clips and collections playlists.
2020-09-06 10:45:34 +07:00
xarantolus
4c47858c05 Fix regex for other variable declaration type
This now supports declarations like `window["ytInitialData"] = ...` and `var ytInitialData = ...`
2020-09-03 20:41:45 +02:00
zwb
b734153d66 [Bilibili] add video title as part of file name 2020-09-02 18:34:25 +08:00
oleksis
7da6facfe2 Add movie URL path 2020-08-31 01:18:02 -04:00
J. Randall Owens
b6ed608604 Merge branch 'master' of https://github.com/rg3/youtube-dl 2020-08-28 20:47:05 +01:00
Leonardo Taccari
8067a2b8b4 [la7] Fix extraction (closes #23323)
Parsing `videoParams' or `videoLa7' JavaScript snippets as JSON is getting too
hard for js_to_json.  Just extract the `vid' from there and use _search_og_*
for all other data.

Remove the 2nd test: unfortunately the page are removed after a couple of
days/weeks.

Thanks to Elia Geretto for spotting and suggesting several problems with the
tests!

Closes #23323.
2020-08-28 12:09:29 +02:00
Leonardo Taccari
abea91456e [rai] Fix extraction for recent raiplay.it updates
- Introduce _BASE_URL in RaiBaseIE class so it could be reused as base for the
  several subextractors.
- Remove first test of RaiPlayIE, it is no longer available
- Adjust RaiPlayIE to recent raiplay.it updates, make it extension agnostic
  (passing possible `.json' URLs is now supported too) and update test
  info_dict.
- Adjust RaiPlayLiveIE to recent raiplay.it updates.  Passing it as
  `url_transparent' is no longer supported (there is no longer an accessible
  ContentItem)
- Adjust RaiPlayPlaylistIE to recent raiplay.it updates and instruct it about
  ContentSet-s.
- Update a RaiIE test and remove two tests that are no longer availables

This fix issue #22923, #22906 and supersedes #23006.
2020-08-28 12:07:31 +02:00
Remita Amine
f5863a3ea0 [biqle] improve video_ext extraction 2020-08-27 19:20:41 +01:00
rigstot
9b60500b16 implement ThisVid extractor
deobfuscates the video URL using a reverse engineered version of KVS
player's algorithm. This was tested against version 4.0.4, 5.0.1 and
5.1.1.4 of the player and a warning will be issued if the major version
changes.
2020-08-26 20:28:09 +02:00
Aakash Gajjar
19a107f21c
Revert "pull changes from remote master (#190)" (#193)
This reverts commit b827ee921fe510a8730a9fab070148ed2b8279b5.
2020-08-26 20:22:32 +05:30
Aakash Gajjar
7f7edf837c
Revert "pull changes from remote master (#191)" (#192)
This reverts commit 63cdbe6ae88fbb3b53bcc17c86140fc7f9595fd8.
2020-08-26 20:21:29 +05:30