Showing posts with label supercomputing. Show all posts
Showing posts with label supercomputing. Show all posts

Tuesday, November 13, 2018

Quick-and-dirty IPC with Python, JSON and pyperclip

By Vasudev Ram



Blue Gene image attribution

Hi, readers,

Some time ago I had written this post.

pyperclip, a cool Python clipboard module


The pyperclip module allows you to programmatically copy/paste text to/from the system clipboard.

Recently, I realized that pyperclip's copy and paste functionality could be used to create a sort of rudimentary IPC (Inter Process Communication) between two Python programs running on the same machine.

So I whipped up a couple of small programs, a sender and a receiver, as a proof of concept of this idea.

Here is the sender, pyperclip_ipc_sender.py:
'''
pyperclip_ipc_sender.py
Purpose: To send JSON data to the clipboard from  
a Python object.
Author: Vasudev Ram
Copyright 2018 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Training: https://jugad2.blogspot.com/p/training.html
Product store: https://gumroad.com/vasudevram
'''

from __future__ import print_function
import pyperclip as ppc
import json
import pprint

def generate_data():
    d = {"North": 1000, "South": 2000, "East": 3000, "West": 4000}
    return d

def send_data(d):
    ppc.copy(json.dumps(d))

def main():
    print("In pyperclip_ipc_sender.py")
    print("Generating data")
    d = generate_data()
    print("data is:")
    pprint.pprint(d)
    print("Copying data to clipboard as JSON")
    send_data(d)

main()
And here is the receiver, pyperclip_ipc_receiver.py:
'''
pyperclip_ipc_receiver.py
Purpose: To receive JSON data from the clipboard into 
a Python object and print it.
Author: Vasudev Ram
Copyright 2018 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Training: https://jugad2.blogspot.com/p/training.html
Product store: https://gumroad.com/vasudevram
'''

from __future__ import print_function
import pyperclip as ppc
import json
import pprint

def receive_data():
    d = json.loads(ppc.paste())
    return d

def main():
    print("In pyperclip_ipc_receiver.py")
    print("Pasting data from clipboard to Python object")
    data = receive_data()
    print("data is:")
    pprint.pprint(data)

main()
First I ran the sender in one command window:
$ python pyperclip_ipc_sender.py
In pyperclip_ipc_sender.py
data is:
{'East': 3000, 'North': 1000, 'South': 2000, 'West': 4000}
Copying data to clipboard as JSON
Then I ran the receiver in another command window:
$ python pyperclip_ipc_receiver.py
In pyperclip_rpc_receiver.py
Pasting data from clipboard to Python object
data is:
{u'East': 3000, u'North': 1000, u'South': 2000, u'West': 4000}
You can see that the receiver has received the same data that was sent by the sender - via the clipboard.

A few points about this technique:

- If you run the receiver without running the sender, or even before running the sender, the receiver will pick up whatever data was last put into the clipboard, either by some other program, or manually by you. For example, if you selected some text in an editor and then pressed Ctrl-C (to copy the selected text to the clipboard), the receiver would get that text (if it was JSON text - see two points below). However, that is not a bug, but a feature :)

- Obviously, this is not meant for production use, due to potential security issues. It's just a toy application as a proof of concept of this idea.

- Since I convert the Python object data to JSON in the sender before copying it to the clipboard with pyperclip, the receiver also expects the data it pastes from the clipboard into a Python object to be of type JSON. So if you instead copy some non-JSON data to the clipboard and then run the receiver, you will get an error. I tried this, and got:

ValueError: No JSON object could be decoded

To handle this gracefully, you can trap the ValueError (and maybe other kinds of exceptions that Python's json library may raise), with a try-except block around the code that pastes the data from the clipboard. You can then either tell the user to try again, or print/log the error and exit, depending on whether the receiver was an interactive or a non-interactive program.

The image at the top of the post is of an IBM Blue Gene supercomputer.

From the Wikipedia article about it:

[ The project created three generations of supercomputers, Blue Gene/L, Blue Gene/P, and Blue Gene/Q. Blue Gene systems have often led the TOP500[1] and Green500[2] rankings of the most powerful and most power efficient supercomputers, respectively. Blue Gene systems have also consistently scored top positions in the Graph500 list.[3] The project was awarded the 2009 National Medal of Technology and Innovation.[4] ]

- Enjoy.



- Vasudev Ram - Online Python training and consulting

I conduct online courses on Python programming, Unix / Linux commands and shell scripting and SQL programming and database design, with course material and personal coaching sessions.

The course details and testimonials are here.

Contact me for details of course content, terms and schedule.

Getting a new web site or blog, and want to help preserve the environment at the same time? Check out GreenGeeks.com web hosting.

DPD: Digital Publishing for Ebooks and Downloads.

Learning Linux? Hit the ground running with my vi quickstart tutorial. I wrote it at the request of two Windows system administrator friends who were given additional charge of some Unix systems. They later told me that it helped them to quickly start using vi to edit text files on Unix. Of course, vi/vim is one of the most ubiquitous text editors around, and works on most other common operating systems and on some uncommon ones too, so the knowledge of how to use it will carry over to those systems too.

Check out WP Engine, powerful WordPress hosting.

Sell More Digital Products With SendOwl.

Get a fast web site with A2 Hosting.

Creating or want to create online products for sale? Check out ConvertKit, email marketing for online creators.

Own a piece of history: Legendary American Cookware

Teachable: feature-packed course creation platform, with unlimited video, courses and students.

Managed WordPress hosting with Flywheel.

Posts about: Python * DLang * xtopdf

My ActiveState Code recipes

Follow me on:


Wednesday, December 11, 2013

Parallela low cost desktop supercomputer coming, w/ Python support


By Vasudev Ram

Parallella | Supercomputing for Everyone

In January 2013 I had blogged about The Parallella project - a low cost Linux supercomputer, and mentioned that Python and Erlang were two languages that would be supported by the project. According to the article linked in that post, Parallela units were supposed to start being delivered around May.

Today I saw in an article on an O'Reilly Media site, that the Parallela team seems to have hit some issues and got delayed, but are now supposed to be releasing units of Parallela this mid-December. Here is that article, by Federico Lucifredi - it also has a lot of technical details about the Parallela supercomputer, both the hardware and the software:

Supercomputing on the cheap with Parallella

Some excerpts from the article that I found interesting:

[ - supercomputing power inside a small credit card-sized board running Ubuntu ...

- powered by just a few Watts of juice, delivering 90 GFLOPS of number crunching ...

- a host side powered by a 667MHz Zynq 7020 ARM A9 System-on-Chip manufactured by Xilinx ...

- number-crunching side is powered by a 600MHz, 16-core Adapteva Epiphany-III numerical accelerator

- Out of the box, this build of the board came with the Linaro 12.10 Ubuntu derivative, so it is extremely standard as Linux ARM distributions go ...

- apt-get has access to the full Linaro 12.10 repositories, so the choice of software is pretty much infinite ...

- the key take-away is that this board promises to blow open the doors to low-power, on-demand supercomputing, putting on the table a real, low-cost alternative to current number crunching powerhouses costing thousands of dollars ...

- While much less brawny a number cruncher, the Raspberry PI remains better suited to multimedia applications (than the Parallela) where its on-board GPU can offload rendering tasks ]

Parallela FAQs

The FAQ has a question, What programming languages will be supported?, to which the answer is:

"Any programming language that is actively maintained and supports free and open source development tools is a candidate. C/C++, OpenCL, and MPI will be supported from day one.",

but since the article also says that any Ubuntu package is just an apt-get away, I guess Python can be installed on it, if not there by default, and Ubuntu (on x86 at least) does have Python pre-installed, IIRC.

And a final excerpt from the Linux.com article linked to in my January blog post about Parallela:

[ The uses to which backers have said that they will put their Parallella computers to use include sound processing, video encoding, 3D scanning, computer visioning, neural networks, physical simulation and, importantly, learning parallel programming!

Software-defined radio is an application that frequently comes up and Parallella is particularly well suited to this since the programmable logic it provides is situated between the ARM host, Epiphany accelerator and GPIO, allowing for digital radio hardware to be more easily integrated. ]

It was interesting to see software-defined radio (SDR) as one of the uses that customers have in mind for Parallela. I had blogged about software-defined radio here a while ago:

Software radio gaining ground?

[ Update: Just noticed that the Parallela blog has this post:

Porting GNU Radio Blocks to The Epiphany ]

GNU Radio is a Software Defined Radio (SDR) framework.

[ Updated the post with the paragraphs below, about the Wikipedia article on SDR, and also for a few typos. Apologies to readers seeing it twice as a result, via feed readers. ]

The Wikipedia article on software-defined radio is interesting. It says that SDR is expected to become the dominant form of radio after some time (this is said by the SDR Forum, though, may be somewhat biased).

Could this become another example of Marc Andreessen's oft-quoted 'Software is eating the world' thesis?

- Vasudev Ram - Dancing Bison Enterprises

Contact Page