How to download PyCon US 2013 videos for offline viewing using youtube-dl

By Vasudev Ram


As Pythonistas would know, PyCon US 2013 happened recently.

While reading Python news on the Net, I saw that the videos of the conference were being uploaded to pyvideo.org.

I browsed the list of videos there. Since I had recently come across youtube-dl, a YouTube downloader tool, I thought of trying to download the PyCon videos using it.

When I went to the pyvideo site to do that, and played one or two videos available there, I was not sure at first whether they were from YouYube or not, so was not sure whether I could download them using youtube-dl.

A little experimentation showed that they were in fact hosted on YouTube, and I was able to find out how to download them, and then downloaded a few of them.

So I'm giving the steps for doing that below:

(Note: these steps worked for me - I could download a few videos using the steps, and none have given an error so far, but YMMV, depending on your OS, browser, Flash Player version, or other factors.)

Update: before (or after) reading these steps, you may wish to read the comments on this post. Some readers commented, and gave simpler methods of doing the same thing, in particular, some said that there is a direct link to download the video on some or all of the pyvideo video pages, and some said that the URL-fixing step is not needed since you can just right-click on the video to get the right URL without the extraneous stuff. Also, renaming the video to remove the extraneous stuff in the name can be avoided by using some command-line options of youtube-dl.

- Go to pyvideo.org in your browser.

- Select the category called PyCon US 2013 videos.

- Select one of the videos from that category and open the link to the video in another browser tab (or in the same tab).

For example, try Guido van Rossum's keynote at PyCon US 2013.

- Click the share video icon at the top right of the video screen; it looks like a V (with small knobs at its vertices), but rotated to the right by 90 degrees.

- The top left of the video screen will now show the URL for the video (along with buttons to share it on Facebook, Twitter, etc.)

- Hover your mouse over that URL and then right click it. You may have to right-click twice, or first click on the URL and then right-click on it.

- From the Adobe Flash Player pop-up menu that appears, select "Copy video URL".

- Now, open a DOS command prompt. I am using Windows in this example; adjust the steps for Linux or Mac OS X, i.e. use a bash or other shell and modify the steps for copy and paste below, to suit your OS.

- At the prompt (Windows), type the partial command:

youtube-dl -t (followed by a space)

- Then type Alt-Space, then E, then P.

- That brings up the Windows system menu for the DOS window, and then selects the Edit menu in it, and then the Paste option from that menu.

- So the video URL will get pasted to the end of youtube-dl command you typed earlier.

- Now, if you just try to run the command as it is, it may not work (for downloading by youtube-dl), because of some extra parameters in the URL, that youtube-dl cannot handle.

- For example, here is the copied video URL for the GvR keynote:

http://www.youtube.com/watch?feature=player_embedded&v=n6dUWXRdt48

- Edit the URL using arrow keys, backspace, delete key, etc., to remove the part from "feature" upto "embedded&"; in other words (using HTTP request terms), remove all the key-value pairs except for the "v=something" pair.

- Using the above GvR keynote example, the command-line should now look like this:

youtube-dl -t http://www.youtube.com/watch?v=n6dUWXRdt48

- Now hit Enter, and wait while the video downloads. The -t option to youtube-dl will add the title of the video as a prefix to the download filename. I typically rename the file to remove the cryptic characters after the title, like this:

c:\downloads> ren Keynote-n6dUWXRdt48.flv GvR-Keynote.flv

- Now you can play the video in VLC, MPlayer or any other suitable player.

Enjoy.

- Vasudev Ram - Dancing Bison Enterprises

Share |

youtube-dl, a YouTube downloader in Python

I came across youtube-dl recently. As the title says, it is a YouTube downloader tool, written in Python.

(Update: I made a typo in the title of this post - wrote YouRube. Unintended :-), and fixed now. The post URL can't be fixed, though, except by deleting and re-entering the post.)

I downloaded the tool (it comes both as a binary and as source) and ran the tool on Windows, at the command line, to download a favorite video of mine from YouTube. It worked fine.

youtube-dl has various command-line options that you can set to configure its behavior.

Here is the documentation for youtube-dl.

Then I took a look at the source code. The code for the tool is in the public domain.

It is a few thousand lines long, and divided into 5 or 6 Python source files. There is one file called FileDownloader, another called InfoExtractor, and a few other supporting files.

I've just had a short look at the code so far, but could figure out a few things about the design, and am giving an overview of that below. Going to check out the code in more detail later. It is a non-trivial app and can serve as a good codebase to study.

Instances of the FileDownloader class and InfoExtractor classes register themselves with each other. Actually, a number of InfoExtractors can be registered with a FileDownloader instance. The FileDownloader then passes the input video URL (which the user wants to download) to the chain of InfoExtractors, and uses the first one which reports that it can handle the request, to extract some required info about the video. The FileDownloader itself then does the actual downloading of the file (if requested by the user; it can do other things than download the file, too). Some of its features are:

- Supports multiple YouTube formats - they link to this list of YouTube formats on Wikipedia, so I guess they must support at least some of them; I haven't tried out formats other than the default, but I may later. Some of those YouTube formats include HD video (higher resolution) and 3GP (for mobile). If youtube-dl supports mobile, it could be used to download videos for mobile to your PC, and then transfer it to your mobile, for offline viewing, if you can find an offline video player app for your mobile. Going to try that out for my Android phone.

- Supports multiple video sites, not just YouTube. Some of them are vimeo, Google Video (not sure if that is still operational), blip.tv, SoundCloud, and infoq.

- The tool youtube-dl can be used to update itself, and they recommend you do it often, with the command:

youtube-dl -U

Since YouTube videos are usually in Flash video format, you can play them on your Windows or Linux PC using VLC or MPlayer.

The video I downloaded was in MP4 format, and I was able to play it in Windows Media Player (didn't have VLC just then on that machine), and it did play, though the audio was low and the picture quailty was not that clear, as compared to playing the same video in YouTube via the web client. Windows Media Player gave a message that the MP4 format was unknown, but was able to play it anyway, though with the stated limitations.

youtube-dl can use either urllib or urllib2 from the Python standard library, can do download rate-limiting if asked to, and many other things, as part of its features.

An interesting Python tool, overall.

On a related note, a lot of YouTube itself is written in Python. I had seen a video about that by one of the YouTube engineers, some time ago.

Here's the summary of a talk at PyCon 2012 on Scaling YouTube - it mentions that most of YouTube is written in Python, and that YouTube is one of the largest web sites in the world, that uses Python. I just saw on Wikipedia in the entry about YouTube that it may be the third-most viewed site in the world.