deltildefiles: D language utility to recursively delete vim backup files

By Vasudev Ram

~

Here's a small utility I wrote in D (the D programming language). The utility is called deltildefiles (for "delete tilde files"). It lets you delete all files whose names end in "~", the tilde character (Wikipedia), under a specified directory, recursively, i.e. including such files found in sub-directories.

[ BTW, check out the Wikipedia article about tilde, linked above. It has many interesting uses in various domains, not just computer programming, for example, in physics, mathematics, economics, electronics and even juggling :) ]

The vim editor (if the option :se backup is set) names its backups of the files you edit, with the same file name as the original file, plus a tilde at the end. Of course, you can do the same task as this utility deltildefiles, with just a batch file (or two) which uses DEL (with or without /S) on Windows), so I just wrote this for fun and learning. However, if you want to enhance the task with other conditions, etc., then a program (or at least a script, in Python, Perl or other language) may be the better way to go, rather than writing an awkward batch file (even if that is possible) or using PowerShell (haven't used the latter). Another alternative is to use a shell script on Linux, or if you use Cygwin or other Unix emulation on Windows, to use the find and rm commands, that come with it, like:

$ find $dirName -print -name '*~' -exec rm {} \; # or some variation

Be careful with the above command. If you make a typing mistake, like omitting the ~, it could end up deleting the wrong files, and potentially many of them. This is also one reason to make the command into a binary (like I have done) or a shell script, test it thoroughly with dummy / safe data, and from then onward, only use that, instead of typing the find command each time - at least for commands that can delete data or cause other damage.

Anyway, here is the code for deltildefiles.d:

/****************************************************************
File: deltildefiles.d
Purpose: To delete vim backup files, i.e. files ending with ~.
Compile with:
$ dmd deltildefiles.d

Author: Vasudev Ram
Copyright 2016 Vasudev Ram
Web site: https://vasudevram.github.io
Products: https://gumroad.com/vasudevram

Description: To delete all files whose names end with 
the tilde character (~) in the directory subtree 
specified as the command-line argument. 

When you edit a file abc.txt with the vim editor, it will 
first make a backup in the file abc.txt~ (note ~ character
at end of file name). Over time, these files can accumulate.
This utility helps you to delete all such vim backup files 
in a specified directory and its subdirectories.

Use with caution and at your own risk!!!
On most operating systems, once a file is deleted this way
(versus sending to the Recycle Bin on Windows), it is not 
recoverable, unless you have installed some undelete utility.
****************************************************************/

import std.stdio;
import std.file;

void usage(string[] args) {
    stderr.writeln("Usage: ", args[0], " dirName");
    stderr.writeln(
        "Recursively delete files whose names end with ~ under dirName.");
}

int main(string[] args) {
    if (args.length != 2) {
        usage(args);
        return 1;
    }
    string dirName = args[1];
    // Check if dirName exists.
    if (!exists(dirName)) {
        stderr.writeln("Error: ", dirName, " not found. Exiting.");
        return 1;
    }
    // Check if dirName is not the NUL device and is actually a directory.
    if (dirName == "NUL" || !DirEntry(dirName).isDir()) {
        stderr.writeln("Error: ", dirName, " is not a directory. Exiting.");
        return 1;
    }
    try {
        foreach(DirEntry de; dirEntries(args[1], "*~", SpanMode.breadth)) {
            // The isFile() check may be enough, also need to check for
            // Windows vs POSIX behavior.
            if (de.isFile() && !de.isDir()) {
                writeln("Deleting ", de.name());
                remove(de.name());
            }
        }
    } catch (FileException) {
        stderr.writeln("Caught a FileException. Exiting.");
        return 1;
    } catch (Exception) {
        stderr.writeln("Caught an Exception. Exiting.");
        return 1;
    }
    return 0;
}

Compile it with:

$ dmd deltildefiles.d

And you can run it like this:

$ deltildefiles .

which will delete all files ending in ~, in and under the current directory. If you give a file name instead of a directory name, or if you give a non-existent directory name, it gives an error message and exits.

It seems to work as of now, based on some testing I did. May have a few bugs since I haven't tested it exhaustively. May make a few improvements to it over time, such as generalizing from ~ to filename wildcards, more error handling, verbose / quiet flags, etc.

Translate this post into another language with Google Translate
(and, just for kicks, click the speaker icon below the right hand side text box at the above Translate page :).

- Vasudev Ram - Online Python training and consulting

Follow me on Gumroad to get email updates about my products.


My Python posts     Subscribe to my blog by email

My ActiveState recipes

Share |

A simple text file indexing program in Python

By Vasudev Ram



Recently, something that I was working on made me think of creating a program to index text files, that is, to create an index file for a text file, something like the index of a book (*), in which, for words in the book, there is a list of page numbers where that word occurs. The difference here is that this program will create, for each word, a list of line numbers where the word occurs in the text file being processed.

(*) To be more specific, what I created was something like a back-of-the-book index, but for text files. I mention that because there are many types of index (Wikipedia), and not just for books. In fact, I was surprised to see the number of meanings or uses of the word index :-) Check the Wikipedia link in the previous sentence to see them. One type of index familiar to programmers, of course, is an array index (or list index, for Python).

Here is the program, called text_file_indexer.py, with a sample input, run and output shown below it. Comments in the code explain the key parts of the logic. Some improvements to the program are possible, of course. I may work on some of them over time. You can already customize the delimiter characters string that is used to remove those characters from around words.

"""
text_file_indexer.py
A program to index a text file.
Author: Vasudev Ram - www.dancingbison.com
Copyright 2014 Vasudev Ram
Given a text file somefile.txt, the program will read it completely, 
and while doing so, record the occurrences of each unique word, 
and the line numbers on which they occur. This information is 
then written to an index file somefile.idx, which is also a text 
file.
"""

import sys
import os
import string
from debug1 import debug1

def index_text_file(txt_filename, idx_filename, 
    delimiter_chars=",.;:!?"):
    """
    Function to read txt_file name and create an index of the 
    occurrences of words in it. The index is written to idx_filename.
    There is one index entry per line in the index file. An index entry 
    is of the form: word line_num line_num line_num ...
    where "word" is a word occurring in the text file, and the instances 
    of "line_num" are the line numbers on which that word occurs in the 
    text file. The lines in the index file are sorted by the leading word 
    on the line. The line numbers in an index entry are sorted in 
    ascending order. The argument delimiter_chars is a string of one or 
    more characters that may adjoin words and the input and are not 
    wanted to be considered as part of the word. The function will remove 
    those delimiter characters from the edges of the words before the rest 
    of the processing.
    """
    try:
        txt_fil = open(txt_filename, "r")
        """
        Dictionary to hold words and the line numbers on which 
        they occur. Each key in the dictionary is a word and the 
        value corresponding to that key is a list of line numbers 
        on which that word occurs in txt_filename.
        """

        word_occurrences = {}
        line_num = 0

        for lin in txt_fil:
            line_num += 1
            debug1("line_num", line_num)
            # Split the line into words delimited by whitespace.
            words = lin.split()
            debug1("words", words)
            # Remove unwanted delimiter characters adjoining words.
            words2 = [ word.strip(delimiter_chars) for word in words ]
            debug1("words2", words2)
            # Find and save the occurrences of each word in the line.
            for word in words2:
                if word_occurrences.has_key(word):
                    word_occurrences[word].append(line_num)
                else:
                    word_occurrences[word] = [ line_num ]

        debug1("Processed {} lines".format(line_num))

        if line_num < 1:
            print "No lines found in text file, no index file created."
            txt_fil.close()
            sys.exit(0)

        # Display results.
        word_keys = word_occurrences.keys()
        print "{} unique words found.".format(len(word_keys))
        debug1("Word_occurrences", word_occurrences)
        word_keys = word_occurrences.keys()
        debug1("word_keys", word_keys)

        # Sort the words in the word_keys list.
        word_keys.sort()
        debug1("after sort, word_keys", word_keys)

        # Create the index file.
        idx_fil = open(idx_filename, "w")

        # Write the words and their line numbers to the index file.
        # Since we read the text file sequentially, there is no need 
        # to sort the line numbers associated with each word; they are 
        # already in sorted order.
        for word in word_keys:
            line_nums = word_occurrences[word]
            idx_fil.write(word + " ")
            for line_num in line_nums:
                idx_fil.write(str(line_num) + " ")
            idx_fil.write("\n")

        txt_fil.close()
        idx_fil.close()
    except IOError as ioe:
        sys.stderr.write("Caught IOError: " + repr(ioe) + "\n")
        sys.exit(1)
    except Exception as e:
        sys.stderr.write("Caught Exception: " + repr(e) + "\n")
        sys.exit(1)

def usage(sys_argv):
    sys.stderr.write("Usage: {} text_file.txt index_file.txt\n".format(
        sys_argv[0]))

def main():
    if len(sys.argv) != 3:
        usage(sys.argv)
        sys.exit(1)
    index_text_file(sys.argv[1], sys.argv[2])

if __name__ == "__main__":
    main()

# EOF

Here is a sample input text file, file01.txt, that I tested the program with:

This file is a test of the text_file_indexer.py program.
The program indexes a text file.
The output of the program is another file called an index file.
The index file is like the index of a book.
For each word that occurs in the text file, there will be a line 
in the index file, starting with that word, and followed by all 
the line numbers in the text file on which that word occurs.

I ran the text file indexer program with the command:

python text_file_indexer.py file01.txt file01.idx

And here is the output of running the program on that text file, that is, the contents of the file file01.idx:

For 5 
The 2 3 4 
This 1 
a 1 2 4 5 
all 6 
an 3 
and 6 
another 3 
be 5 
book 4 
by 6 
called 3 
each 5 
file 1 2 3 3 4 5 6 7 
followed 6 
in 5 6 7 
index 3 4 4 6 
indexes 2 
is 1 3 4 
like 4 
line 5 7 
numbers 7 
occurs 5 7 
of 1 3 4 
on 7 
output 3 
program 1 2 3 
starting 6 
test 1 
text 2 5 7 
text_file_indexer.py 1 
that 5 6 7 
the 1 3 4 5 6 7 7 
there 5 
which 7 
will 5 
with 6 
word 5 6 7 

- Vasudev Ram - Python training and consulting

Share |

Glances CLI monitoring tool uses psutil

By Vasudev Ram

Interesting to see that the Glances system monitoring tool uses psutil. IIRC, I had blogged or tweeted about Glances some time ago.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Pyline, a grep-like, sed-like Python command-line tool

Pyline: a grep-like, sed-like command-line tool. « Python recipes « ActiveState Code

Glances, CLI/curses Python tool to monitor UNIX systems

By Vasudev Ram


Glances is "a CLI curses based monitoring tool for GNU/Linux and BSD OS". It uses Python and PsUtil.

Hacker News thread about it.

The thread has some positive comments about Glances.

EDIT: Sorry, readers, about the temporary multiple posts on the same topic. Due to getting some Blogger error, I clicked Submit a few times, so multiple posts resulted. I've deleted the duplicate posts now. The duplicate posts only existed for a few minutes before I deleted them, but mentioning it because I don't anyone to think I'm spamming them.

- Vasudev Ram - Dancing Bison Enterprises

Share |

TTYtter, new Twitter command-line client

http://www.floodgap.com/software/ttytter/

Seen on HN.

- VR
www.dancingbison.com

Some ways of doing UNIX-style pipes in Python

By Vasudev Ram - dancingbison.com | @vasudevram | jugad2.blogspot.com

For a project I'm working on, I was recently thinking about how to implement something similar to UNIX-style pipes in Python; not necessarily exactly the same, but conceptually similar.

I had deliberately decided *not* to search for this on the Net, so that I could first think about it myself, and figure something out.
But coincidentally today, while browsing the Usenet group comp.lang,python, I came across this post mentioning issues with doing one-liners in Python.

One of the answers given was to check out PyP, a tool for Python that lets you do pipes (in a sense) and data munging like the powerful UNIX tools sed and awk. PyP stands for "Python Power at the Prompt, meaning the UNIX shell prompt, of course. It has an interesting and unusual approach. It is open source, hosted on Google Code, and was apparently initially created by a division of Sony Pictures called ImageWorks, "to facilitate the construction of complex image manipulation unix commands during visual effects work on Alice in Wonderland, Green Lantern, and the upcoming The Amazing Spiderman". Good performance was mentioned as one of it's plus points, apart from the pipe facility itself.

So I checked PyP out a bit and it seems like a nice tool. It has a fairly intuitive syntax for at least basic operations, and is also extensible in at least couple or so ways for more advanced users

I also did a Google query or two with appropriate keywords to find other such tools. Here are some of them, including PyP again:

PyP: http://code.google.com/p/pyp

Osh: http://geophile.com/osh/

Pipe module for Python by Julien Palard:
http://dev-tricks.net/pipe-infix-syntax-for-python

Piping support in the standard Python library:

http://docs.python.org/library/pipes.html

Will update this post later after checking these tools out some more.

Posted via email

- Vasudev Ram @ Dancing Bison