592 Commits

Author SHA1 Message Date
Cédric Luthi
bdbf6155eb Catch all errors when trying to run external commands
Trying to run a non-installed command (such as avprobe) through Popen may throw other errors than `OSError`. For example, a `ValueError` may happen on macOS 10.12.6:

```
$ ./youtube-dl -U -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-U', u'-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.06.25
[debug] Python version 2.7.10 (CPython) - Darwin-16.7.0-x86_64-i386-64bit
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "./youtube-dl/__main__.py", line 19, in <module>
  File "./youtube-dl/youtube_dl/__init__.py", line 472, in main
  File "./youtube-dl/youtube_dl/__init__.py", line 439, in _real_main
  File "./youtube-dl/youtube_dl/YoutubeDL.py", line 417, in __init__
  File "./youtube-dl/youtube_dl/YoutubeDL.py", line 2260, in print_debug_header
  File "./youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 76, in get_versions
  File "./youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 60, in __init__
  File "./youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 117, in _determine_executables
  File "./youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 117, in <genexpr>
  File "./youtube-dl/youtube_dl/utils.py", line 1962, in get_exe_version
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
    child_exception = pickle.loads(data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle
```

Catching all errors instead of only `OSError` should be safer.
2018-06-25 21:27:32 +02:00
Enes
85750f8972 [openload] Improve ext extraction 2018-06-02 00:16:22 +07:00
Remita Amine
261f47306c [utils] fix style id extraction for namespaced id attribute(closes #16551) 2018-05-26 14:38:24 +01:00
Remita Amine
5a16c9d9d3 [utils] keep the original TV_PARENTAL_GUIDELINES dict 2018-05-25 23:12:50 +01:00
Remita Amine
b836118724 [utils] Relax TV Parental Guidelines matching 2018-05-23 12:12:20 +01:00
Sergey M․
5f95927a62
Improve geo bypass mechanism
* Introduce geo bypass context
* Add ability to bypass based on IP blocks in CIDR notation
* Introduce --geo-bypass-ip-block
2018-05-02 07:20:59 +07:00
Sergey M․
6cc622327f
[utils] Introduce merge_dicts 2018-04-28 02:47:17 +07:00
Sergey M․
1cc47c6674
[utils] Fix match_str for boolean meta fields 2018-04-24 23:54:49 +07:00
Philipp Hagemeister
f226880c6d [tennistv] Add support for tennistv.com 2018-03-14 09:55:21 +01:00
Sergey M․
b871d7e954
[utils] Add parse_resolution 2018-03-02 23:39:04 +07:00
Sergey M․
befa4708fd
[utils] Fixup some common URL's typos in sanitize_url (closes #15649) 2018-02-19 22:50:23 +07:00
Remita Amine
b12cf31bb1 [cbc] add new extractor for olympics.cbc.ca(closes #15535) 2018-02-19 09:02:23 +01:00
Sergey M․
65220c3bd6
Add support for IronPython 2018-01-28 05:48:18 +07:00
Mike Fährmann
c384d537f8 [util] Improve scientific notation handling in js_to_json (closes #14789) 2018-01-20 22:54:21 +07:00
Philipp Hagemeister
d7cd9a9e84
[utils] Fix youtube-dl under PyPy3 on Windows 2018-01-01 22:48:27 +07:00
Sergey M․
b555ae9bf1
[utils] Add another date format pattern (#14999) 2017-12-16 21:56:16 +07:00
Remita Amine
d4f05d4731 [utils] add sami mimetype to mimetype2ext 2017-12-03 00:04:43 +01:00
Remita Amine
ffe6979ef9 [utils] add hvc1 codec code to parse_codecs 2017-11-27 16:55:27 +01:00
Sergey M․
056653bbb1
[utils] Add support for zero years and months in parse_duration 2017-10-29 07:04:48 +07:00
Sergey M․
6be08ce602
[utils] Use in OnDemandPagedList by default
Not using cache results in redundant network I/O due to downloading the same pages while using --playlist-items n-m
2017-10-06 23:13:53 +07:00
Sergey M․
4c54b89e03
Hide experimental phantomjs wrapper 2017-09-24 00:08:27 +07:00
Yen Chi Hsuan
4ed2d7b7d1 Fix flake8 issues after #14225 2017-09-17 13:53:04 +08:00
Yen Chi Hsuan
3869028ffb [utils] Use bytes-like objects in dfxp2srt
This fixes handling of non-UTF8 TTML subtitles

Closes #14191
2017-09-16 12:18:38 +08:00
Yen Chi Hsuan
a88d461dff Merge pull request #14225 from Tithen-Firion/openload-phantomjs-method
Openload phantomjs method
2017-09-16 02:28:28 +08:00
Sergey M․
c7e327c4d4
[utils] Introduce bool_or_none 2017-09-10 19:08:39 +07:00
Yen Chi Hsuan
95f3f7c20a
[utils] Fix unescapeHTML for misformed string like "&a&quot;" (#13935) 2017-08-19 21:40:53 +08:00
Sergey M․
5b232f46dc
[utils] Skip missing params in cli_bool_option (closes #13865) 2017-08-09 22:28:19 +07:00
Tithen-Firion
feee8d32e4 [phantomjs] add exe version to debug info 2017-08-03 14:17:25 +02:00
Yen Chi Hsuan
609ff8ca19 [utils] Support attributes with no values in get_elements_by_attribute() 2017-07-05 23:27:12 +08:00
Sergey M․
aefce8e6dc
[utils] Use compat_shlex_quote in shell_quote 2017-06-17 23:48:58 +07:00
Sergey M․
b4a3d461e4
[utils] Handle HTMLParseError in extract_attributes (closes #13349) 2017-06-12 01:52:24 +07:00
Sergey M․
2ae2ffda5e
[utils] Improve unified_timestamp 2017-06-11 21:27:22 +07:00
Yen Chi Hsuan
b407d8533d
[utils] Drop an compatibility wrapper for Python < 2.6
addinfourl.getcode is added since Python 2.6a1. As youtube-dl now
requires 2.6+, this is no longer necessary.

See 9b0d46db11
2017-05-27 23:05:02 +08:00
Yen Chi Hsuan
5552c9eb0f
[utils] Recognize more patterns in strip_jsonp()
Used in Youku Show pages
2017-05-26 21:58:18 +08:00
Sergey M․
60f5c9fb19
[utils] Recognize more audio codecs (#13081) 2017-05-14 12:33:33 +07:00
Yen Chi Hsuan
10c87c151b
[utils] Rename try_multipart_encode to _multipart_encode_impl
To state that this is an internal function and people should be careful
when using it outside youtube-dl.
2017-05-06 19:06:18 +08:00
Yen Chi Hsuan
b2ad479d17
[utils] Fix multipart_encode for Python < 3.5 2017-05-05 20:51:59 +08:00
Tithen-Firion
c89267d31a Merge branch 'master' into openload-phantomjs-method 2017-05-04 11:00:06 +02:00
Yen Chi Hsuan
0c26548601
[cda] Implement birthday verification (closes #12789) 2017-05-04 16:26:17 +08:00
Sergey M․
deef31955b
[utils] Improve unified_timestamp
Seen at http://zaq1.pl/video/xev0e
2017-04-30 21:45:53 +07:00
Remita Amine
98ce1a3fd3 [utils] add video/mp2t to mimetype2ext 2017-04-30 09:03:10 +01:00
Tithen-Firion
98f9d87381 [phantomjs] Add required version checking 2017-04-29 12:41:42 +02:00
Tithen-Firion
edd9221cd2 [utils] Fix inconsistent output of clean_html
`\s` in Python 2.x doesn't match unicode whitespace characters by
default
2017-04-28 17:34:27 +02:00
Tithen-Firion
40e41780f1 [phantomjs] add cookie support 2017-04-25 15:12:54 +02:00
Tithen-Firion
da57ebaf84 [openload] separate PhantomJS code from extractor 2017-04-25 01:06:14 +02:00
Remita Amine
5b995f713b [utils] add support for ttml styles 2017-04-19 14:38:40 +01:00
Sergey M․
a32a9a7ef5
[extractor/common] Add support multiple getters in try_get 2017-04-18 22:39:58 +07:00
Sergey M․
51098426b8
[utils] Introduce expand_path 2017-03-26 02:30:10 +07:00
Sergey M․
4b5de77bdb
[utils] Process bytestrings in urljoin (closes #12369) 2017-03-06 03:57:46 +07:00
Sergey M․
158af5242e
[utils] Carry long doc string 2017-03-01 23:04:02 +07:00