jugad2 - Vasudev Ram on software innovation

Email marketing for professional bloggers

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Saturday, March 31, 2018

Checking if web sites are online with Python

By Vasudev Ram

Hi readers,

Recently, I thought of writing a small program to check if one or more web sites are online or not. I used the requests Python library with the HTTP HEAD method. I also checked out PycURL for this. It is a thin wrapper over libcurl, the library that powers the well-known and widely used curl command line tool. While PycURL looks powerful and fast (since it is a thin wrapper that exposes most or all of the functionality of libcurl), I decided to use requests for this version of the program. The code for the program is straightforward, but I found a few interesting things while running it with a few different sites as arguments. I mention those points below.

Here is the tool: I named it is_site_online.py:

"""
is_site_online.py
Purpose: A Python program to check if a site is online or not.
Uses the requests library and the HTTP HEAD method.
Tries both with and without HTTP redirects.
Author: Vasudev Ram
Copyright 2018 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
"""

from __future__ import print_function
import sys
import requests
import time

if len(sys.argv) < 2:
    sys.stderr.write("Usage: {} site ...".format(sys.argv[0]))
    sys.stderr.write("Checks if the given site(s) are online or not.")
    sys.exit(0)

print("Checking if these sites are online or not:")
print("   ".join(sys.argv[1:]))

print("-" * 60)
try:
    for site in sys.argv[1:]:
        for allow_redirects in (False, True):
            tc1 = time.clock()
            r = requests.head(site, allow_redirects=allow_redirects)
            tc2 = time.clock()
            print("Site:", site)
            print("Check with allow_redirects =", allow_redirects)
            print("Results:")
            print("r.ok:", r.ok)
            print("r.status_code:", r.status_code)
            print("request time:", round(tc2 - tc1, 3), "secs")
            print("-" * 60)
except requests.ConnectionError as ce:
    print("Error: ConnectionError: {}".format(ce))
    sys.exit(1)
except requests.exceptions.MissingSchema as ms:
    print("Error: MissingSchema: {}".format(ms))
    sys.exit(1)
except Exception as e:
    print("Error: Exception: {}".format(e))
    sys.exit(1)

The results of some runs of the program:

Check for Google and Yahoo!:

$ python is_site_online.py http://google.com http://yahoo.com
Checking if these sites are online or not:
http://google.com   http://yahoo.com
-----------------------------------------------------------
Site: http://google.com
Check with allow_redirects = False
Results:
r.ok: True
r.status_code: 302
request time: 0.217 secs
------------------------------------------------------------
Site: http://google.com
Check with allow_redirects = True
Results:
r.ok: True
r.status_code: 200
request time: 0.36 secs
------------------------------------------------------------
Site: http://yahoo.com
Check with allow_redirects = False
Results:
r.ok: True
r.status_code: 301
request time: 2.837 secs
------------------------------------------------------------
Site: http://yahoo.com
Check with allow_redirects = True
Results:
r.ok: True
r.status_code: 200
request time: 1.852 secs
------------------------------------------------------------

In the cases where allow_redirects is False, google.com gives a status code of 302 and yahoo.com gives a status code of 301. The 3xx series of codes are related to HTTP redirection.

After seeing this, I looked up HTTP status code information in a few sites such as Wikipedia and the official site www.w3.org (the World Wide Web Consortium), and found a point worth noting. See the part in the Related links section at the end of this post about "302 Found", where it says: "This is an example of industry practice contradicting the standard.".

Now let's check for some error cases:

One error case: we do not give an http:// prefix (assume some novice user who is mixed up about schemes and paths), so they type a garbled site name, say http.om:

$ python is_site_online.py http.om
Checking if these sites are online or not:
http.om
------------------------------------------------------------
Traceback (most recent call last):
  File "is_site_online.py", line 32, in 
    r = requests.head(site, allow_redirects=allow_redirects)
[snip long traceback]
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL 'http.om':
No schema supplied. Perhaps you meant http://http.om?

This traceback tells us that when no HTTP 'scheme' [1][2] is given, requests raises a MissingSchema exception. So we now know that we need to catch that exception in our code, by adding another except clause to the try statement, which I later did, in the program you see in this post. In general, this technique can be useful when using a new Python library for the first time: just don't handle any exceptions in the beginning, use it a few times with variations in input or modes of use, and see what sorts of exceptions it throws. Then add code to handle them.

[1] The components of a URL

[2] Parts of URL

Another error case - a made-up site name that does not exist:

$ python is_site_online.py http://abcd.efg
Checking if these sites are online or not:
http://abcd.efg
------------------------------------------------------------
Caught ConnectionError: HTTPConnectionPool(host='abcd.efg',
port=80): Max retries exceeded with url: / (Caused
by NewConnectionError(': Failed
to establish a new connection: [Errno 11004] getaddrinfo
failed',))

From the above error we can see or figure out a few things:

- the requests library defines a ConnectionError exception. I first ran the above command without catching ConnectionError in the program; it gave that error, then I added the handler for it.

- requests uses an HTTP connection pool

- requests does some retries when you try to get() or head() a URL (a site name)

- requests uses urllib3 (from the Python standard library) under the hood

I had discovered that last point earlier too; see this post:

urllib3, the library used by the Python requests library

And as I mentioned in that post, urllib3 itself uses httplib.

Now let's check for some sites that are misspellings of the site google.com:

$ python is_site_online.py http://gogle.com Checking ... ------------------------------------------------------------ Site: http://gogle.com With allow_redirects: False Results: r.ok: True r.status_code: 301 request time: 3.377 ------------------------------------------------------------ Site: http://gogle.com With allow_redirects: True Results: r.ok: True r.status_code: 200 request time: 1.982 ------------------------------------------------------------

$ python is_site_online.py http://gooogle.com Checking ... ------------------------------------------------------------ Site: http://gooogle.com With allow_redirects: False Results: r.ok: True r.status_code: 301 request time: 0.425 ------------------------------------------------------------ Site: http://gooogle.com With allow_redirects: True Results: r.ok: True r.status_code: 200 request time: 1.216 ------------------------------------------------------------

Interestingly, the results show that that both those misspellings of google.com exist as sites.

It is known that some people register domains that are similar in spelling to well-known / popular / famous domain names, maybe hoping to capture some of the traffic resulting from users mistyping the famous ones. Although I did not plan it that way, I realized, from the above two results for gogle.com and gooogle.com, that this tool can be used to detect the existence of such sites (if they are online when you check, of course).

Wednesday, March 1, 2017

Show error numbers and codes from the os.errno module

By Vasudev Ram

While browsing the Python standard library docs, in particular the module os.errno, I got the idea of writing this small utility to display os.errno error codes and error names, which are stored in the dict os.errno.errorcode:

Here is the program, os_errno_info.py:

from __future__ import print_function
'''
os_errno_info.py
To show the error codes and 
names from the os.errno module.
Author: Vasudev Ram
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
'''

import sys
import os

def main():
    
    print("Showing error codes and names\nfrom the os.errno module:")
    print("Python sys.version:", sys.version[:6])
    print("Number of error codes:", len(os.errno.errorcode))
    print("{0:>4}{1:>8}   {2:<20}    {3:<}".format(\
        "Idx", "Code", "Name", "Message"))
    for idx, key in enumerate(sorted(os.errno.errorcode)):
        print("{0:>4}{1:>8}   {2:<20}    {3:<}".format(\
            idx, key, os.errno.errorcode[key], os.strerror(key)))

if __name__ == '__main__':
    main()

And here is the output on running it:

$ py -2 os_errno_info.py >out2 && gvim out2
Showing error codes and names
from the os.errno module:
Python sys.version: 2.7.12
Number of error codes: 86
 Idx    Code   Name                    Message
   0       1   EPERM                   Operation not permitted
   1       2   ENOENT                  No such file or directory
   2       3   ESRCH                   No such process
   3       4   EINTR                   Interrupted function call
   4       5   EIO                     Input/output error
   5       6   ENXIO                   No such device or address
   6       7   E2BIG                   Arg list too long
   7       8   ENOEXEC                 Exec format error
   8       9   EBADF                   Bad file descriptor
   9      10   ECHILD                  No child processes
  10      11   EAGAIN                  Resource temporarily unavailable
  11      12   ENOMEM                  Not enough space
  12      13   EACCES                  Permission denied
  13      14   EFAULT                  Bad address
  14      16   EBUSY                   Resource device
  15      17   EEXIST                  File exists
  16      18   EXDEV                   Improper link
  17      19   ENODEV                  No such device
  18      20   ENOTDIR                 Not a directory
  19      21   EISDIR                  Is a directory
  20      22   EINVAL                  Invalid argument
  21      23   ENFILE                  Too many open files in system
  22      24   EMFILE                  Too many open files
  23      25   ENOTTY                  Inappropriate I/O control operation
  24      27   EFBIG                   File too large
  25      28   ENOSPC                  No space left on device
  26      29   ESPIPE                  Invalid seek
  27      30   EROFS                   Read-only file system
  28      31   EMLINK                  Too many links
  29      32   EPIPE                   Broken pipe
  30      33   EDOM                    Domain error
  31      34   ERANGE                  Result too large
  32      36   EDEADLOCK               Resource deadlock avoided
  33      38   ENAMETOOLONG            Filename too long
  34      39   ENOLCK                  No locks available
  35      40   ENOSYS                  Function not implemented
  36      41   ENOTEMPTY               Directory not empty
  37      42   EILSEQ                  Illegal byte sequence
  38   10000   WSABASEERR              Unknown error
  39   10004   WSAEINTR                Unknown error
  40   10009   WSAEBADF                Unknown error
  41   10013   WSAEACCES               Unknown error
  42   10014   WSAEFAULT               Unknown error
  43   10022   WSAEINVAL               Unknown error
  44   10024   WSAEMFILE               Unknown error
  45   10035   WSAEWOULDBLOCK          Unknown error
  46   10036   WSAEINPROGRESS          Unknown error
  47   10037   WSAEALREADY             Unknown error
  48   10038   WSAENOTSOCK             Unknown error
  49   10039   WSAEDESTADDRREQ         Unknown error
  50   10040   WSAEMSGSIZE             Unknown error
  51   10041   WSAEPROTOTYPE           Unknown error
  52   10042   WSAENOPROTOOPT          Unknown error
  53   10043   WSAEPROTONOSUPPORT      Unknown error
  54   10044   WSAESOCKTNOSUPPORT      Unknown error
  55   10045   WSAEOPNOTSUPP           Unknown error
  56   10046   WSAEPFNOSUPPORT         Unknown error
  57   10047   WSAEAFNOSUPPORT         Unknown error
  58   10048   WSAEADDRINUSE           Unknown error
  59   10049   WSAEADDRNOTAVAIL        Unknown error
  60   10050   WSAENETDOWN             Unknown error
  61   10051   WSAENETUNREACH          Unknown error
  62   10052   WSAENETRESET            Unknown error
  63   10053   WSAECONNABORTED         Unknown error
  64   10054   WSAECONNRESET           Unknown error
  65   10055   WSAENOBUFS              Unknown error
  66   10056   WSAEISCONN              Unknown error
  67   10057   WSAENOTCONN             Unknown error
  68   10058   WSAESHUTDOWN            Unknown error
  69   10059   WSAETOOMANYREFS         Unknown error
  70   10060   WSAETIMEDOUT            Unknown error
  71   10061   WSAECONNREFUSED         Unknown error
  72   10062   WSAELOOP                Unknown error
  73   10063   WSAENAMETOOLONG         Unknown error
  74   10064   WSAEHOSTDOWN            Unknown error
  75   10065   WSAEHOSTUNREACH         Unknown error
  76   10066   WSAENOTEMPTY            Unknown error
  77   10067   WSAEPROCLIM             Unknown error
  78   10068   WSAEUSERS               Unknown error
  79   10069   WSAEDQUOT               Unknown error
  80   10070   WSAESTALE               Unknown error
  81   10071   WSAEREMOTE              Unknown error
  82   10091   WSASYSNOTREADY          Unknown error
  83   10092   WSAVERNOTSUPPORTED      Unknown error
  84   10093   WSANOTINITIALISED       Unknown error
  85   10101   WSAEDISCON              Unknown error

In the above Python command line, you can of course skip the "&& gvim out2" part. It is just there to automatically open the output file in gVim (text editor) after the utility runs.

The above output was from running it with Python 2.
The utility is written to also work with Python 3.
To change the command line to use Python 3, just change 2 to 3 everywhere in the above Python command :)
(You need to install or already have py, the Python Launcher for Windows, for the py command to work. If you don't have it, or are not on Windows, use python instead of py -2 or py -3 in the above python command line - after having set your OS PATH to point to Python 2 or Python 3 as wanted.)

The only differences in the output are the version message (2.x vs 3.x), and the number of error codes - 86 in Python 2 vs. 101 in Python 3.
Unix people will recognize many of the messages (EACCES, ENOENT, EBADF, etc.) as being familiar ones that you get while programming on Unix.
The error names starting with W are probably Windows-specific errors. Not sure how to get the messages for those, need to look it up. (It currently shows "Unknown error" for them.)

This above Python utility was inspired by an earlier auxiliary utility I wrote, called showsyserr.c, as part of my IBM developerWorks article, Developing a Linux command-line utility (not the main utility described in the article). Following (recursively) the link in the previous sentence will lead you to the code for both the auxiliary and the main utility, as well as the PDF version of the article.

Enjoy.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Follow me on: LinkedIn * Twitter

Share |

Monday, January 30, 2017

Finding the arity of a Python function

By Vasudev Ram

While browsing through some Python code, I had the idea of trying to find the arity of a Python function. The arity of a function is the number of arguments it takes.

Digression: I had first come across the concept of arity when reading some books on functional or logic programming, years ago. E.g. Prolog has the concept of arity, IIRC. Just did a quick search for the phrase prolog arity and found a few results confirming this. Here is one, from cse.unsw.edu.au:

arity (link 1)

But in Prolog, the word arity is related to a functor (link 2), not a function, according to the page at link 2.

Excerpt from link 1:

[ The arity of a functor is the number of arguments it takes. For example, the arity of likes/2, as in likes(jane, pizza), is 2, as it takes two arguments, jane and pizza. ]

In fact the use of likes/2 gives a clue to the fact that the Erlang language was influenced by Prolog (and I've read that the first version of Erlang was written in Prolog), because a function foo with arity 2 in Erlang is written as foo/2, as for the Prolog functor above.

[ Note: I'm not an expert in either Prolog or Erlang; these are just casual observations. Anyone who knows better, feel free to comment / correct. ]

Anyway, end of digression.

Here is the code I came up with to find the arity of a Python function. I've used the isfunction and getargspec functions of the inspect module.

'''
function_arity.py
Purpose: To find the arity of a Python function.
Author: Vasudev Ram
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
'''

import inspect

# Define a few functions with increasing arity:

def f0():
    pass

def f1(a1):
    pass

def f2(a1, a2):
    pass

def f3(a1, a2, a3):
    pass

def f4(a1, a2, a3, a4):
    pass

def main():

    # Define a few non-function objects:
    int1 = 0
    float1 = 0.0 
    str1 = ''
    tup1 = ()
    lis1 = []

    # Test the function arity-finding code with both the functions 
    # and the non-function objects:
    for o in (f0, f1, f2, f3, f4, int1, float1, str1, tup1, lis1):
        if not inspect.isfunction(o):
            print repr(o), 'is not a function'
            continue
        n_args = len(inspect.getargspec(o)[0])
        if n_args == 0:
            num_suffix = '(no) args'
        elif n_args == 1:
            num_suffix = 'arg'
        else:
            num_suffix = 'args'
        print o.__name__, 'is a function that takes', \
            n_args, num_suffix
    
if __name__ == '__main__':
    main()

And here is the output when I run the program:

$ python function_arity.py
f0 is a function that takes 0 (no) args
f1 is a function that takes 1 arg
f2 is a function that takes 2 args
f3 is a function that takes 3 args
f4 is a function that takes 4 args
0 is not a function
0.0 is not a function
'' is not a function
() is not a function
[] is not a function

I also did a web search:

https://www.google.com/search?q=find+the+arity+of+python+function

in which one hit showed another way of doing it (using gettattr). This StackOverflow question:

How to find out the arity of a method in Python

had some answers; the ones by Python guru Alex Martelli (the martellibot) and Anurag Uniyal were interesting.

Enjoy.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Follow me on: LinkedIn * Twitter

Share |

Thursday, January 5, 2017

Give your Python function a {web,CLI} hug!

By Vasudev Ram

I came across this interesting Python framework called hug recently:

www.hug.rest

Hug is interesting because it allows you to create a function in Python and then expose it via both the web and the command-line. It also does some data type validation using Python 3's annotations (not shown in my example, but see the hug quickstart below). Hug is for Python 3 only, and builds upon on the Falcon web framework (which is "a low-level high performance framework" for, among other things, "building other frameworks" :).

Here is the hug quickstart.

The hug site says it is "compiled with Cython" for better performance. It makes some claims about being one of the faster Python frameworks. Haven't checked that out.

Here is an HN thread about hug from about a year ago, with some interesting comments, in which the author of hug also participated, replying to questions by readers, explaining some of his claims, etc. Some benchmark results for hug vs. other tools are also linked to in that thread:

I tried out some of the features of hug, using Python 3.5.2, with a small program I wrote.

Below is the test program I wrote, hug_pdp.py. The pdp in the filename stands for psutil disk partitions, because it uses the psutil disk_partitions() function that I blogged about here recently:

Using psutil to get disk partition information with Python

Here is hug_pdp.py:

"""
hug_pdp.py
Use hug with psutil to show disk partition info 
via Python, CLI or Web interfaces.
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: http://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
"""

import sys
import psutil
import hug

def get_disk_partition_data():
    dps = psutil.disk_partitions()
    fmt_str = "{:<8} {:<7} {:<7}"

    result = {}
    result['header'] = fmt_str.format("Drive", "Type", "Opts")
    result['detail'] = {}
    for i in (0, 2):
        dp = dps[i]
        result['detail'][str(i)] = fmt_str.format(dp.device, dp.fstype, dp.opts)
    return result

@hug.cli()
@hug.get(examples='drives=0,1')
@hug.local()
def pdp():
    """Get disk partition data"""
    result = get_disk_partition_data()
    return result

@hug.cli()
@hug.get(examples='')
@hug.local()
def pyver():
    """Get Python version"""
    pyver = sys.version[:6]
    return pyver

if __name__ == '__main__':
    pdp.interface.cli()

Note the use of hug decorators in the code to enable different kinds of user interfaces and HTTP methods.

Here are some different ways of running this hug-enabled program, with their outputs:

As a regular Python command-line program, using the python command:

$ python  hug_pdp.py
{'detail': {'0': 'C:\\      NTFS    rw,fixed', '2': 'E:\\      CDFS    ro,cdrom'
}, 'header': 'Drive    Type    Opts   '}

As a command-line program, using the hug command:

$ hug -f hug_pdp.py -c pdp
{'detail': {'2': 'E:\\      CDFS    ro,cdrom', '0': 'C:\\      NTFS    rw,fixed'}, 
'header': 'Drive    Type    Opts   '}

You can see that this command gives the same output as the previous one.
But you can also run the above command with the "-c pyver" argument instead of "-c pdp", giving:

$ hug -f hug_pdp.py -c pyver
3.5.2

(I added the pyver() function to the program later, after the initial runs with just the pdp() function, to figure out how using the hug command to run the program was different from using the python command to run it. The answer can be seen from the above output, though there is another difference too, shown below (the web interface). Next, I ran it this way:

$ hug -f hug_pdp.py

which started a web server (running on port 8000), giving this output on the console:

/#######################################################################\
          `.----``..-------..``.----.
         :/:::::--:---------:--::::://.
        .+::::----##/-/oo+:-##----:::://
        `//::-------/oosoo-------::://.       ##    ##  ##    ##    #####
          .-:------./++o/o-.------::-`   ```  ##    ##  ##    ##  ##
             `----.-./+o+:..----.     `.:///. ########  ##    ## ##
   ```        `----.-::::::------  `.-:::://. ##    ##  ##    ## ##   ####
  ://::--.``` -:``...-----...` `:--::::::-.`  ##    ##  ##   ##   ##    ##
  :/:::::::::-:-     `````      .:::::-.`     ##    ##    ####     ######
   ``.--:::::::.                .:::.`
         ``..::.                .::         EMBRACE THE APIs OF THE FUTURE
             ::-                .:-
             -::`               ::-                   VERSION 2.2.0
             `::-              -::`
              -::-`           -::-
\########################################################################/

 Copyright (C) 2016 Timothy Edmund Crosley
 Under the MIT License


Serving on port 8000...

Then I went to this URL in my browser:

http://localhost:8000/pdp

which gave me this browser output:

{"detail": {"0": "C:\\      NTFS    rw,fixed", "2": "E:\\      CDFS    ro,cdrom"}, 
"header": "Drive    Type    Opts   "}

which is basically the same as the earlier command-line interface output I got.
Next I went to this URL:

http://localhost:8000/pyver

which gave me this:

"3.5.2 "

which again is the same as the earlier corresponding command-line output of the hug command.

Of course, the output from both the web and CLI interfaces is either JSON or a dict, so in a real life app, we would have to get that output and use it in some way, such as (process it further before we) format it better for human consumption. If using a JavaScript front-end, it can easily be done; if using the code as is with the command-line mode, we need to figure out a way to do it. The hug module may have some support for that.

What is also interesting is that when I run it this way:

http://localhost:8000/

I get this browser output:

{
    "404": "The API call you tried to make was not defined. Here's a definition 
of the API to help you get going :)",
    "documentation": {
        "overview": "\nhug_pdp.py\nUse hug with psutil to show disk partition 
info \nvia Python, CLI or Web interfaces.\nCopyright 2017 Vasudev Ram\nWeb site: 
https://vasudevram.github.io\nBlog: http://jugad2.blogspot.com\nProduct store: 
https://gumroad.com/vasudevram\n",
        "handlers": {
            "/pdp": {
                "GET": {
                    "usage": "Get disk partition data",
                    "examples": [
                        "http://localhost:8000/pdp?drives=0,1"
                    ],
                    "outputs": {
                        "format": "JSON (Javascript Serialized Object Notation)",
                        "content_type": "application/json"
                    }
                }
            },
            "/pyver": {
                "GET": {
                    "usage": "Get Python version",
                    "examples": [
                        "http://localhost:8000/pyver"
                    ],
                    "outputs": {
                        "format": "JSON (Javascript Serialized Object Notation)",
                        "content_type": "application/json"
                    }
                }
            }
        }
    }
}

which shows that trying to access an unsupported route, gives as output, this:

an overview, supported URLs/routes, HTTP methods, and documentation about how to use it and the output formats - almost none of which code was written for, mind.

Go give your Python code a hug!

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Follow me on: LinkedIn * Twitter

Share |

Friday, December 30, 2016

Jal-Tarang, and a musical alarm clock in Python

By Vasudev Ram

Hi readers,

Season's greetings!

After you check out this video (Ranjana Pradhan playing the Jal Tarang in Sydney, 2006, read the rest of the post below ...

Here is a program that acts as a musical command-line alarm clock. It is an adaptation of the one I created here a while ago:

A simple alarm clock in Python (command-line)

This one plays a musical sound (using the playsound Python library) when the alarm time is reached, instead of beeping like the above one does. I had recently come across and used the playsound library in a Python course I conducted, so I thought of enhancing the earlier alarm clock app to use playsound. Playsound is a nice simple Python module with just one function, also called playsound, which can play either WAV or MP3 audio files on Windows.

The playsound library is described as "a pure Python, cross platform, single function module with no dependencies for playing sounds."

Excerpt from its PyPI page:

[ On Windows, uses windll.winmm. WAVE and MP3 have been tested and are known to work. Other file formats may work as well.

On OS X, uses AppKit.NSSound. WAVE and MP3 have been tested and are known to work. In general, anything QuickTime can play, playsound should be able to play, for OS X.

On Linux, uses ossaudiodev. I don’t have a machine with Linux, so this hasn’t been tested at all. Theoretically, it plays WAVE files. ]

Here is the code for the program, musical_alarm_clock.py:

from __future__ import print_function

'''
musical_alarm_clock.py

Author: Vasudev Ram
Copyright 2016 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: http://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram

Description: A simple program to make the computer act like 
a musical alarm clock. Start it running from the command line 
with a command line argument specifying the number of minutes 
after which to give the alarm. It will wait for that long, and 
then play a musical sound a few times.
'''

import sys
import string
from playsound import playsound, PlaysoundException
from time import sleep, asctime

sa = sys.argv
lsa = len(sys.argv)
if lsa != 2:
    print("Usage: python {} duration_in_minutes.".format(sys.argv[0]))
    print("Example: python {} 10".format(sys.argv[0]))
    print("Use a value of 0 minutes for testing the alarm immediately.")
    print("The program plays a musical sound a few times after the duration is over.")
    sys.exit(1)

try:
    minutes = int(sa[1])
except ValueError:
    print("Invalid value {} for minutes.".format(sa[1]))
    print("Should be an integer >= 0.")
    sys.exit(1)

if minutes < 0:
    print("Invalid value {} for minutes.".format(minutes))
    print("Should be an integer >= 0.")
    sys.exit(1)

seconds = minutes * 60

if minutes == 1:
    unit_word = "minute"
else:
    unit_word = "minutes"

try:
    print("Current time is {}.".format(asctime()))
    if minutes > 0:
        print("Alarm set for {} {} later.".format(str(minutes), unit_word))
        sleep(seconds)
    else:
        print("Running in immediate test mode, with no delay.")
    print("Alarm time reached at {}.".format(asctime()))
    print("Wake up.")
    for i in range(5):
        playsound(r'c:\windows\media\chimes.wav')        
        #sleep(1.00)
        sleep(0.50)
        #sleep(0.25)
        #sleep(0.10)
except PlaysoundException as pe:
    print("Error: PlaysoundException: message: {}".format(pe))
    sys.exit(1)
except KeyboardInterrupt:
    print("Interrupted by user.")
    sys.exit(1)

Jal Tarang image attribution

The picture above is of a Jal Tarang.

The Jal Tarang is an ancient Indian melodic percussion instrument. Brief description adapted from the Wikipedia article: It consists of a set of bowls filled with water. The bowls can be of different sizes and contain different amounts of water. The instrument is tuned by adjusting the amount of water in each bowl. The music is played by striking the bowls with two sticks.

I've watched live jal-tarang performances only a few times as a kid (it was probably somewhat uncommon even then, and there are very few people who play it nowadays), so it was interesting to see this video and read the article.

Enjoy.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Follow me on: LinkedIn * Twitter

Share |

Friday, December 23, 2016

Using psutil to get disk partition information with Python

By Vasudev Ram

psutil is a Python library that enables you to get various kinds of operating system information, such as about CPUs, processes, memory and disks.

Here is a short program that shows how to get information about the disk partitions on your computer using psutil:

from __future__ import print_function
import psutil

dps = psutil.disk_partitions()
fmt_str = "{:<8} {:<7} {:<7}"
print(fmt_str.format("Drive", "Type", "Opts"))
# Only show a couple of devices.
for i in (0, 2):
    dp = dps[i]
    print(fmt_str.format(dp.device, dp.fstype, dp.opts))

Running the program gives this output:

$ py -3 psutil_disk_partitions.py
Drive    Type    Opts
C:\      NTFS    rw,fixed
E:\      CDFS    ro,cdrom

(Check out py, the Python launcher, if you don't already use it.)

Explanation of the above output:
The Drive column shows the drive letter.
The Type column shows the file system type of the partition.
The Opts column shows what options were used while mounting the device on the drive.

Mounting is the operation of logically associating a drive name or a path name with a file system (on a partition or physical drive), and making it accessible under that drive or path name.

For this run, it shows that:
C drive is an NTFS partition (Windows NT File System) and is mounted in read-write mode and is a fixed disk;
E drive is a CDFS partition (CD-ROM File System) and is mounted in read-only mode.

Here is a video about how a hard disk drive works:

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Follow me on: LinkedIn * Twitter

My Python posts Subscribe to my blog by email

Share |

Monday, August 8, 2016

How to kill yourself in Python

By Vasudev Ram

Here's a simple Python program that shows how the os.kill function from Python's standard library, along with the os.getpid function and the signal module [1], can be used to terminate the current program - the one it is called from:

'''
Program: test_self_kill.py
A program to show that the os.kill function 
can be used to terminate the current program.
Author: Vasudev Ram
Copyright 2016 Vasudev Ram
https://vasudevram.github.io
http://jugad2.blogspot.com
https://gumroad.com/vasudevram
'''

from __future__ import print_function
import sys, os, signal

print("Python version:", sys.version)
print("This line will be printed.")
os.kill(os.getpid(), signal.SIGTERM)
print("If os.kill works, this line will not be printed.")

Program output when run in Python 2.7:

$ python test_self_kill.py
Python version: 2.7.11 (v2.7.11:6d1b6a68f775, Dec  5 2015, 20:40:30) [MSC v.1500
 64 bit (AMD64)]
This line will be printed.

Program output when run in Python 3.6:

$ python test_self_kill.py
Python version: 3.6.0a2 (v3.6.0a2:378893423552, Jun 14 2016, 01:21:40) [MSC v.19
00 64 bit (AMD64)]
This line will be printed.

As you can see, the second call to the print function does not run, because the program terminates itself.
You can read about Unix signals here and here.

- Vasudev Ram - Online Python training and consulting

Share |

Wednesday, February 3, 2016

Using Python's trace module to understand the flow of programs (from many angles)

- Vasudev Ram - Online Python training and programming

Some time back, I had written this post:

Python's trace module and chained decorators

in which I had briefly described use of the Python standard library's trace module to help us understand and debug Python programs. In that post I showed a small program with 3 chained decorators. The trace module was used to trace the execution of the decorators and the functions that they decorated.

The trace output in that post also showed the difference between function definition and function execution, both of which occur at run time in Python, since it is a dynamic language.

In this post, I'm going to use the trace module in a few other ways.

Here is a small program in which we will use the trace module.

(Note that I said "in which", not "on which", because I am going to invoke the trace module as a library from within this program, and tell it to start tracing from a specific function call, unlike in my previous post (linked above), in which I used the trace module as though it was a program itself, by using the "python -m" option, to trace my own program containing those decorators.)

# This is a program to show some basic usage of the trace module 
# from the Python standard library.
# Author: Vasudev Ram - 
# http://jugad2.blogspot.in/p/about-vasudev-ram.html
# Copyright 2016 Vasudev Ram

def fa():
    fb()

def fb():
    fc()

def fc():
    fd(5)

def fd(n):
    if n <= 1:
        return
    else:
        fd(n - 1)

import trace

tracer = trace.Trace(
    count=0, trace=0, countfuncs=0, countcallers=1, 
)

tracer.run('fa()')
r = tracer.results()
r.write_results(show_missing=True, coverdir=".")

In this program, I have a function fa calling fb which calls fc which calls fd. Function fd also calls itself recursively, with a termination condition so the recursion is not infinite.

Note that the program contains all the three main programming constructs: sequential execution of statements, conditional execution and iteration (via the recursive call in function fd).

From the Python documentation, the trace.Trace() constructor has the following (partial) signature:

class trace.Trace(count=1, trace=1, countfuncs=0, countcallers=0, ...)

where I have used ellipsis (...) to represent the remaining arguments, which I do not pass. (See the docs (linked above, near top of post) for the full signature and the meaning of all the arguments. The ones I use are explained below.)

As for what those arguments do, again, from the docs:

Create an object to trace execution of a single statement or expression. All parameters are optional. count enables counting of line numbers. trace enables line execution tracing. countfuncs enables listing of the functions called during the run. countcallers enables call relationship tracking.

I ran the program simple_prog.py 4 times. Each time I passed a different combination of argument values to trace.Trace() - with only one argument set to 1 each time, and all the others set to 0 in that run. And each time I redirected the resulting trace output to an output file, except for the first run, in which Python created the output in the file simple_prog.cover (since the program being traced is named simple_prog.py.

The lines below show the sets of arguments passed, and the corresponding output file names for the trace output. (I will show the contents of each output file later, below.)

arg set 1: count=1, trace=0, countfuncs=0, countcallers=0, output: simple_prog.cover

arg set 2: count=0, trace=1, countfuncs=0, countcallers=0, output: run_2.txt

arg set 3: count=0, trace=0, countfuncs=1, countcallers=0, output: run_3.txt

arg set 4: count=0, trace=0, countfuncs=0, countcallers=1, output: run_4.txt

Notice that in each of the arg sets, only one flag is set.

I ran the program this time with just:

python simple_prog.py

since the tracing is started from within the program, unlike in my previous post (linked above) in which I started the tracing like this:

python -m trace -t test_chained_decorators.py

Here is the output for the 1st run, simple_prog.cover:

# This is a program to show some basic usage of the trace module 
       # from the Python standard library.
       # Author: Vasudev Ram - 
       # http://jugad2.blogspot.in/p/about-vasudev-ram.html
       # Copyright 2016 Vasudev Ram
       
>>>>>> def fa():
    1:     fb()
       
>>>>>> def fb():
    1:     fc()
       
>>>>>> def fc():
    1:     fd(5)
       
>>>>>> def fd(n):
    5:     if n <= 1:
    1:         return
           else:
    4:         fd(n - 1)
       
>>>>>> import trace
       
>>>>>> tracer = trace.Trace(
>>>>>>     count=1, trace=0, countfuncs=0, countcallers=0, 
       )
       
>>>>>> tracer.run('fa()') # This line starts the tracing process.
>>>>>> r = tracer.results()
>>>>>> r.write_results(show_missing=True, coverdir=".")

You can see that the output includes the counts of the number of times lines of code are called. Notice that there are no line counts for the def lines, presumably because function definitions are only supposed to be done once (by definition, ha ha), so it would not make sense to show it, and might even be confusing.

Here is the output for the 2nd run, run_2.txt:

--- modulename: simple_prog, funcname: <module>
<string>(1):   --- modulename: simple_prog, funcname: fa
simple_prog.py(6):     fb()
 --- modulename: simple_prog, funcname: fb
simple_prog.py(9):     fc()
 --- modulename: simple_prog, funcname: fc
simple_prog.py(12):     fd(5)
 --- modulename: simple_prog, funcname: fd
simple_prog.py(15):     if n <= 1:
simple_prog.py(18):         fd(n - 1)
 --- modulename: simple_prog, funcname: fd
simple_prog.py(15):     if n <= 1:
simple_prog.py(18):         fd(n - 1)
 --- modulename: simple_prog, funcname: fd
simple_prog.py(15):     if n <= 1:
simple_prog.py(18):         fd(n - 1)
 --- modulename: simple_prog, funcname: fd
simple_prog.py(15):     if n <= 1:
simple_prog.py(18):         fd(n - 1)
 --- modulename: simple_prog, funcname: fd
simple_prog.py(15):     if n <= 1:
simple_prog.py(16):         return
 --- modulename: trace, funcname: _unsettrace
trace.py(80):         sys.settrace(None)

Since the trace flag is set, we get line execution tracing. And due to that, there are 5 entries for function fd, since there are 5 calls to it (of which 4 are recursive).

Here is the output for the 3rd run, run_3.txt:

functions called:
filename: <string>, modulename: <string>, funcname: <module>
filename: D:\Anaconda-2.1.0-64\lib\trace.py, modulename: trace, funcname: _unsettrace
filename: simple_prog.py, modulename: simple_prog, funcname: fa
filename: simple_prog.py, modulename: simple_prog, funcname: fb
filename: simple_prog.py, modulename: simple_prog, funcname: fc
filename: simple_prog.py, modulename: simple_prog, funcname: fd

Since the countfuncs flag is set, it shows the functions called during the run of the program.

Here is the output for the 4th run, run_4.txt:

calling relationships:

*** <string> ***
  --> simple_prog.py
    <string>.<module> -> simple_prog.fa

*** D:\Anaconda-2.1.0-64\lib\trace.py ***
  --> <string>
    trace.Trace.runctx -> <string>.<module>
    trace.Trace.runctx -> trace._unsettrace

*** simple_prog.py ***
    simple_prog.fa -> simple_prog.fb
    simple_prog.fb -> simple_prog.fc
    simple_prog.fc -> simple_prog.fd
    simple_prog.fd -> simple_prog.fd

Since the countcallers flag is set, it shows the function call tree during the run of the program, i.e. what function called what other function(s). The last line shows the recursive call to fd.

So we can see, overall, that those four flags to trace.Trace(), allowed us to get insight into the behaviour of the program, from various angles or perspectives. This makes the trace module a useful tool for debugging and for understanding code that we have to work on.

- Enjoy.

Signup to hear about new products and services I create.

Share |

Tuesday, January 19, 2016

Simple drawing program with Python turtle graphics

- Vasudev Ram - Online Python training and programming

Image attribution: Claudio Giovenzana www.longwalk.it

Some time back I had written a post with a simple turtle graphics program in Python:

(The history of turtle graphics is of interest, including the early years, when they used actual robots for the turtles, which were controlled by programs). The Wikipedia page in the link above has the details.)

Here is the post I wrote earlier:

Python, meet Turtle

Python has support for turtle graphics in the standard library.

Today I wrote a different kind of turtle graphics program: a simple drawing program that allows you to interactively draw figures using a few keys on the keyboard.

The program is a first version, so is not polished. But it works, as far as I tested it.

You can use the following keys to control the turtle and draw:

- up arrow to move forward
- down arrow to move backward
- left arrow to turn left 45 degrees
- right arrow to turn right 45 degrees
- h or H key to move the turtle to its home position (center of screen, heading east)
- c or C key to clear the screen
- q or Q to quit the program

Note: If you do Clear and then Home, and if your turtle had been away from the home position when you did the Clear, the screen will get cleared, but then the Home action will draw a line from where the turtle just was to its (new) home position. If that happens, just press C again to clear the screen of that line.
The better way is to do Home first and then Clear.

Here is the code for the program:

# Program to do drawing using Python turtle graphics.
# turtle_drawing.py v0.1
# Author: Vasudev Ram
# http://jugad2.blogspot.in/p/about-vasudev-ram.html
# Copyright (C) 2016 Vasudev Ram.

import turtle

# Create and set up screen and turtle.

t = turtle
# May need to tweak dimensions below for your screen.
t.setup(600, 600)
t.Screen()
t.title("Turtle Drawing Program - by Vasudev Ram")
t.showturtle()

# Set movement step and turning angle.
step = 160
angle = 45

def forward():
    '''Move forward step positions.'''
    print "forward", step
    t.forward(step)

def back():
    '''Move back step positions.'''
    print "back", step
    t.back(step)

def left():
    '''Turn left by angle degrees.'''
    print "left", angle
    t.left(angle)

def right():
    '''Turn right by angle degrees.'''
    print "right", angle
    t.right(angle)

def home():
    '''Go to turtle home.'''
    print "home"
    t.home()

def clear():
    '''Clear drawing.'''
    print "clear"
    t.clear()

def quit():
    print "quit"
    t.bye()

t.onkey(forward, "Up")
t.onkey(left, "Left")
t.onkey(right, "Right")
t.onkey(back, "Down")
t.onkey(home, "h")
t.onkey(home, "H")
t.onkey(clear, "c")
t.onkey(clear, "C")
t.onkey(quit, "q")
t.onkey(quit, "Q")

t.listen()
t.mainloop()

Support for more turtle actions (see the Python turtle module's docs linked above) can be added, for many of them on the same pattern as the existing functions, without changing the overall structure of the program. Some others may require changes to the design, which is very simple as of now. (I applied the principle Do The Simplest Thing That Could Possibly Work.)

Also, a point to note: many of the capabilities of the turtle module are available in both procedural and object-oriented versions. Here, to keep it simple, I've used the procedural style.

Here are a few screenshots of drawings I created by running and using this program:

A diamond (just a square tilted):

A square:

A figure made out of squares:

The image at the top of the post is of Olive ridley sea turtles.

Signup to hear about new products and services I create.

Share |

Monday, December 7, 2015

Using JSON Schema with Python to validate JSON data

- Vasudev Ram - Online Python training and programming

I got to know about JSON Schema and the jsonschema Python library recently.

JSON Schema is a scheme (pun not intended) or method for checking that input JSON data adheres to a specified schema, roughly similar to what can done for XML data using an XML Schema.

So I thought of writing a small program to try out the jsonschema library. Here it is:

# test_jsonschema_unix.py
# A program to try the jsonschema Python library.
# Uses it to validate some JSON data.
# Follows the Unix convention of writing normal output to the standard 
# output (stdout), and errors to the standard error output (stderr).
# Author: Vasudev Ram
# Copyright 2015 Vasudev Ram

from __future__ import print_function
import sys
import json
import jsonschema
from jsonschema import validate

# Create the schema, as a nested Python dict, 
# specifying the data elements, their names and their types.
schema = {
    "type" : "object",
    "properties" : {
        "price" : {"type" : "number"},
        "name" : {"type" : "string"},
    },
}

print("Testing use of jsonschema for data validation.")
print("Using the following schema:")
print(schema)
print("Pretty-printed schema:")
print(json.dumps(schema, indent=4))

# The data to be validated:
# Two records OK, three records in ERROR.
data = \
[
    { "name": "Apples", "price": 10},
    { "name": "Bananas", "price": 20},
    { "name": "Cherries", "price": "thirty"},
    { "name": 40, "price": 40},
    { "name": 50, "price": "fifty"}
]

print("Raw input data:")
print(data)
print("Pretty-printed input data:")
print(json.dumps(data, indent=4))

print("Validating the input data using jsonschema:")
for idx, item in enumerate(data):
    try:
        validate(item, schema)
        sys.stdout.write("Record #{}: OK\n".format(idx))
    except jsonschema.exceptions.ValidationError as ve:
        sys.stderr.write("Record #{}: ERROR\n".format(idx))
        sys.stderr.write(str(ve) + "\n")

The name of the program is test_jsonschema_unix.py, because, as you can see in the source code, the normal output is sent to sys.stdout (standard output) and the errors are sent to sys.stderr (standard error output), as Unix tools often do. So, to run this with the stdout and stderr redirected to separate files, we can do this:

$ python test_jsonschema_unix.py >out 2>err

(where the filename out is for output and err is for error)

which gives us this for out:

Testing use of jsonschema for data validation.
Using the following schema:
{'type': 'object', 'properties': {'price': {'type': 'number'}, 'name': {'type': 'string'}}}
Pretty-printed schema:
{
    "type": "object", 
    "properties": {
        "price": {
            "type": "number"
        }, 
        "name": {
            "type": "string"
        }
    }
}
Raw input data:
[{'price': 10, 'name': 'Apples'}, {'price': 20, 'name': 'Bananas'}, {'price': 'thirty', 'name': 'Cherries'}, {'price': 40, 'name': 40}, {'price': 'fifty', 'name': 50}]
Pretty-printed input data:
[
    {
        "price": 10, 
        "name": "Apples"
    }, 
    {
        "price": 20, 
        "name": "Bananas"
    }, 
    {
        "price": "thirty", 
        "name": "Cherries"
    }, 
    {
        "price": 40, 
        "name": 40
    }, 
    {
        "price": "fifty", 
        "name": 50
    }
]
Validating the input data using jsonschema:
Record #0: OK
Record #1: OK

and this for err:

Record #2: ERROR
'thirty' is not of type 'number'

Failed validating 'type' in schema['properties']['price']:
    {'type': 'number'}

On instance['price']:
    'thirty'
Record #3: ERROR
40 is not of type 'string'

Failed validating 'type' in schema['properties']['name']:
    {'type': 'string'}

On instance['name']:
    40
Record #4: ERROR
'fifty' is not of type 'number'

Failed validating 'type' in schema['properties']['price']:
    {'type': 'number'}

On instance['price']:
    'fifty'

So we can see that the good records went to out and the bad ones went to err, which means that jsonschema could validate our data.

Signup to hear about new products and services I create.

- Vasudev Ram - Online Python training and programming

Share |

Monday, November 23, 2015

Convert XLSX to PDF with Python and xtopdf

By Vasudev Ram

XLSX => PDF

This is a simple application of my xtopdf toolkit, showing how to use it to convert XLSX data, i.e. Microsoft Excel data, to PDF (Portable Document Format). It only converts text data, not the formatting, colors, fonts, etc., that may be present in the Excel file.

For the input, I will use this small Excel file, fruits2.xlsx, which I created. A screenshot of it is below (click to enlarge):

Here is the code for XLSXtoPDF.py:

# XLSXtoPDF.py

# Program to convert the data from an XLSX file to PDF.
# Uses the openpyxl library and xtopdf.

# Author: Vasudev Ram - http://jugad2.blogspot.com
# Copyright 2015 Vasudev Ram.

from openpyxl import load_workbook
from PDFWriter import PDFWriter

workbook = load_workbook('fruits2.xlsx', guess_types=True, data_only=True)
worksheet = workbook.active

pw = PDFWriter('fruits2.pdf')
pw.setFont('Courier', 12)
pw.setHeader('XLSXtoPDF.py - convert XLSX data to PDF')
pw.setFooter('Generated using openpyxl and xtopdf')

ws_range = worksheet.iter_rows('A1:H13')
for row in ws_range:
    s = ''
    for cell in row:
        if cell.value is None:
            s += ' ' * 11
        else:
            s += str(cell.value).rjust(10) + ' '
    pw.writeLine(s)
pw.savePage()
pw.close()

And here is a screenshot of the PDF output in fruits2.pdf:

There are some points worth mentioning in connection with conversion of data to and from PDF. I will discuss them in a follow-up post.

Signup to hear about new products and services I create.

Share |

Tuesday, November 3, 2015

Using the wikipedia Python library (to search for oranges :)