Blogspot - we need to talk

Dear blogspot,

We need to talk. I have been meaning to tell you this for quite some time. Everytime I typed in this WYSIWYG word editor and accidentally did an esc+j followed by a Homer Simpson "doh" reminding me that it isn't vim, my heart shed a tear. I tried using addons, plugins but you didn't budge. I even did a one-night stand with wordpress - but she didn't even let me muck around with her plugins, the way you did - I'll give you that. But I had to wait an eternity to get code syntax highlighting enabled with you.

Before you think that this is just my usual rant and I will soon be drafting another post through this rich-text-editor of yours, you are mistaken. I've... I've met someone. And she/he/it/they let me type in markdown. You can't hear me but I say this with a lump in my throat while I weep tears of joy. I get to type my posts in vim (don't cringe - you know I've loved vim right from the start). And I get to check them in via git.

MARKDOWN!! VIM!! GIT CHECKINS!!! CAN YOU EVEN COMPREHEND THE EASE OF USE??

I am leaving you. I can't stand this white box peppered with small-t-big-T, font-size, bold, italics buttons anymore. I am moving on.

I am going to github pages. Now, now, don't play that single-sign-on card with me. git global credential cache is good enough for everyone.

As far as the custody of our existing posts go - I think it would be better for both of us if I slowly migrate each of them to their new home. You can come over whenever you want. Which I know you never will.

This is where I live now - http://saurabh-hirani.github.io/writing/

This is what I am leaving you for - a repo of my works -  https://github.com/saurabh-hirani/saurabh-hirani.github.io/

We had a good run. We really did. But I don't belong here.

So long and thanks for everything.

--
Saurabh

P.S: I will still host http://crashpoint-zeros-brain.blogspot.in/ on you - take care of that for me.

Lessons learnt while automating docstring checking

This blog post serves as a comment area for this ipython notebook

Using Python decorators for registering callbacks

In my previous post we talked about Python decorators and an intuitive way of remembering how decorators with arguments work.

The following code snippet had triggered the whole chain of thought:


Now that we know what decorators with arguments do, which is essentially - calling the decorator factory with the argument, using the argument to make logical branching in the decorator wrapper and returning the wrapper, we can now try to understand the above code. We first move on to the decorator route's source code found here to see it's implementation. Keep the source code opened in a different tab, as we will refer to it in the later sections.



This code deviates a bit from what we think about how decorators are used - decorators wrap the functionality of a target function with certain pre and post actions, like so:



Output:

 Decorator calling func
 In the function
 Decorator called func

But there isn't any call to the target function index in the route function definition. Instead, this code snippet throws light on another functionality of decorators - registering callbacks.

Callbacks are registered functions which are stored in some container data structure (mostly hashes with key as function name and value as function references). For example,


Output:

  calling func1 with arg 1
  calling func2 with arg 2

They are useful in building a map of functions and depending on specific user input, the hashes are looked up and the corresponding key's value - which is a function reference is called.

Thinking out aloud: Python decorators

This is not yet another explanation of decorators. There are a lot of people who have done a great job of explaining it. What this post is about is the way I understood it and how I wrap my head around decorators (pun intended).

Pre-requisite: this awesome stackoverflow answer on decorators.

I had been trying to learn Flask and came across this nice post. It contains a good introduction to flask in the form of a mini project. While going through the post, I came across this snippet:


@app.route('/') is a decorator which registers the index function to be called when a user makes GET requests to the root page.

I always get the part where we say that a decorator is a function which modifies another function - wraps it with a sort of pre and post functionality. The part of where decorators are passed arguments is what used to confuse me and led me to revisit the afforementioned stackoverflow post

For example, it is easy to understand this:



Output:

 Decorator calling func
 In the function
 Decorator called func

which means that when Python encountered the @ symbol it did an internal equivalent of

func = decorator(func)

which in turn means that a decorator is a function which takes in a function and returns a wrapper over that function and reassigns that wrapper to the original function variable.

This has the side effect of redefining the function name also (func.__name__), to be that of the wrapper function, but as the stackoverflow answer mentions functools.wraps comes to the rescue.

What used to stall me were these kind of examples:



Output:

  func1  args - [this, that]
  func2  called_from_line:31  args - [who, what]

For func1 our logging is not as verobse as func2 which has 'debug' log level. This is possible because the decorator creating function takes an argument which decides logging behavior (whether to print line no. of caller or not).

The magic of closures is also involved because the decorator and the wrapped function remember the original environment that they were bound with (for func1 the variable loglevel is set to 'info' in wrapped_around_func and for func2, it is set to 'debug')

But what is the deal with the nesting of functions?

googlecl = blog in vim

This post was uploaded from my Ubuntu box by running the command:

$ google blogger --blog "Curiosity heals the cat" post --title "googlecl = blog in vim" --src googlecl.txt

where googlecl.txt contains the content you are reading. Talk about making it easier to post! :)

Now I can type out on vim snuggled in the dark, warm comfort of the terminal without resorting to browser plugins supporting vim.

A small note on the setup - googlecl works with gdata library. Using the latest versions of googlecl and gdata caused problems for me.

But googlecl worked like a charm when I used the following packages:

googlecl-0.9.13.tar.gz + gdata-2.0.14.tar.gz= vimvana!

Be sure to set/unset http_proxy and https_proxy environment variables before starting to use googlecl. I had moved to a different network without proxy and googlecl gave me a stack trace, instead of a meaningful error message to disable my proxy.

When you run blogger operations for the first time, googlecl will check if you have obtained an access token to perform this action. If you haven't, it will try to obtain one (by opening a browser window and asking you to sign in)

Once you are signed in googlecl will put the authentication token in /home/your_user/.local/share/googlecl with the name as access_tok_your.google.user.name

I find it useful to delete this file once I am done with my work so that my acccess is revoked once I am done with my work. You can also invalidate the access token by going to Google settings => Security => Connected application and sites  and removing the access token manually.

googlecl supports editing of google-docs but not of blog posts. So you have to do a:

$ google blogger --blog "Curiosity heals the cat" delete --title "googlecl = blog in vim"
$ google blogger --blog "Curiosity heals the cat" post --title "googlecl = blog in vim" --src googlecl.txt

May be a deal breaker for some people.

More info about googlecl at googlecl manual

Using mdadm to learn RAID concepts

I've been using mdadm to learn about software RAID configurations on Linux and I find it to be a great resource to play around with and do some hands on work. How I wish we were taught using these tools in college, rather than just being bombarded by all possible combinations of RAID.

Having the ability to setup and tear down VMs at the snap of a finger makes me think how fortunate CS students are nowadays. We had it tough. At the risk of digression, I sound like one of the Yorkshiremen from the classic Monty Python sketch - The Four Yorkshiremen

Statistics have shown that 85% of the audience, after clicking the above link have spent at least 45 minutes visiting the various sketches of Monty Python, including, but not limited to - Woody and tinny words, Argument Clinic and others.

And.... we are back. :)

This tutorial gives a gentle introduction to RAID and then builds up on the concepts and walks the reader through the mdadm tool. As that HOWTO is already delivering, I will not go into the details of it. The aim of this post is to inform about the pre-requisite setup for building RAIDs.

I am using Virtualbox with a VM running Ubuntu 10.10. For mdadm I have installed the mdadm tool (installation steps mentioned in the tutorial page).

As you are going to play with different RAID configurations, you will need to add multiple disks. For VirtualBox, go to the main menu, right click on the VM, go to settings and then to storage. Out there add 2 more SATA disks each of 1 GB - I used this value because I am going to partition each disk into 2 partitions of 512 MB each. If there are no new disks added to your system, then these disks will show up as /dev/sdb and /dev/sdc on your Linux box.

Boot up your VM. Become root. Execute the following command:

# fdisk -l

You will see a partition table for /dev/sda - containing - /dev/sda1, /dev/sda2, etc. - this is the first disk in which your OS is installed. We are not going to touch this disk.

We are going to work with /dev/sdb and /dev/sdc - the 2 new disks that have been added. If you already had added other disks the device names may be different in your case - but one thing is for sure - for the new disks, the partition table will not exist. Hence for the new ones fdisk output will be something like this:

putty + screen = meant to be

I am getting used to the screen command for my Linux development environment and I find it to be a great tool for detaching and reattaching sessions - but that's just scratching the surface from what I see on the net. As there are many useful posts out there explaining why screen is something that you've been missing if you are working via putty, I will let them do the talking.

Introduction

A redhat magazine post

I am a newbie to screen and this is all that I have been doing:

1. Open multiple putty sessions via Putty connection manager
2. Foreach tabbed session I do a: $ screen -S session_name
3. Code
4. While leaving for the day, disconnect all sessions - I don't "exit"
5. When I want to come back again - I open the tabs once again and reattach to previous session - I normally name my sessions "tab1", "tab2", etc. so reattaching is equivalent to: $screen -d -r tab1
   

All this is good enough. I had just one rant while using screen - I did not know how to scroll back. Say I run a find command and the output spews out to more than 80 lines - the size of my terminal window. If I am running in screen mode - by default I can scroll back only upto 100 lines. And then, this post came to my rescue:

Stray thought

The difference between an average programmer and a good one is the same as the difference between an optometrist and an ophthalmologist.

The easiest way to recover admin password of BSNL ZTE DSL modem

In my previous post, I had described how to recover the admin password of your BSNL ZTE DSL modem, provided you have the modem configuration backed up.

But if you don't have the config file backed up - recovering the admin password is all the more easier. Read on to know more.

Model: BSNL Dataone ATE ZXDSL 531B modem.

This post shows how you can get the admin password of the modem without any backed up configuration. You just need to know the modem IP. Let me know if the link is broken - I will write a new post describing those steps.

If your modem/router's basic security is flawed to this extent, then deploying its in-built security measures is like trusting a visually impaired lame duck, one suffering from suicidal tendencies, arming it with banana skins, and telling yourself: "That ought to keep me safe from fire-breathing dragons and the 15 feet tall goblins who ride them!" (a justified exaggeration the "sitting duck" analogy).

Recovering admin password of your modem from backed up configuration file

Model: BSNL Dataone ATE ZXDSL 531B modem. (Below points worth a shot for any other modem also)

Assumption: You have saved your modem settings locally on your machine. If not, don't be disheartened (Do slap yourself thrice though), this post will help you out.

If you don't know how to upload/save your modem/router's configuration file, read this.

I had forgotten the admin password and I didn't want to reset my modem. Just wanted to see if I could get any info from the configuration that I backed up.

I opened up the file and searched for the keyword "admin" and I came across this block:

<sysusername value="admin">
<syspassword value="YWRtaW5fcGFzc3dk">

Took a blind shot to see if it is a base64 encoding and <insert your jumping-with-surprise word here> it was base64 encoded!

Securing my DSL modem

I had a BSNL modem installed at my home quite some time ago. The guy who installed it did the minimal configuration - just enough to connect to the internet. He did nothing on the front of basic security - hiding the wireless network, changing the default password, etc.

As with all standard modems, this one had a web interface at http://192.168.1.1 The interface was easy to navigate and through it I hardened a few of the settings. I am going to share them in this post.

The security measures that I describe will be available on any standard DSL/ADSL modem. The UI navigation will vary across different models. But the ideas - like changing your modem's IP from 192.168.1.1 to some non-standard value e.g. 10.2.3.44 will stay the same.

The changes are limited by my own experience and I'd appreciate if you could share any other security measures that you took. If you are happy with the way things are going on - your wireless network is not hidden, your modem's IP is 192.168.1.1, default password is "admin" - stay happy and go back to watching that skating cat video on youtube, while I download the entire LOTR trilogy on your connection :). If you want to get your hands dirty and learn a few things in the process, read on.

The author is not responsible for temporary/permanent loss of internet connectivity or inadvertent establishment of communication with alien life forms resulting from the changes described in this post.

Modem: BSNL Dataone - ZTE - ZXDSL 531B

Before moving on to the changes, I'd recommend that if you are going to do them as you read this post, please read the whole post first. And the changes don't come into effect until you save them or in some other cases restart the modem (changing the modem IP for instance). So if you want to experiment with the changes, do them one at a time - change the config, save it, check it and move on. Doing multiple changes and not knowing what went wrong is not worth the debugging time. You can also keep on saving changes as you move through the post, creating snapshots of the configuration file.

The first thing to do before messing around with any settings is to insure yourself against the destructive streak of the geek in you - the one which prods you with the "Go on, try it out, let's see if it blows anything up" thought. Hence I backed up the minimal configuration that was done by the BSNL guy.

Alpha(bet) Geek

#/usr/bin/perl
use strict;
use warnings;
my $str = $ARGV[0];
my @alpha_arr = ('a' .. 'z');
my %str_tok_hash = map { lc($_) => undef } 
                   grep { /[a-z]/i } 
                   split(//, $str);
my @diff = grep { not exists $str_tok_hash{$_} } @alpha_arr;
$str = "'The string ($str) contains all the letters of the alphabet";
$str .= " except (".join(' ', sort @diff).").' - contains all the letters"; 
$str .= " of the alphabet. :)";
print "$str\n";

The "Read More" plugin and how blogger beat me to it

For those who are searching for a way to add a "Read More" link to their blogspot blogs - click here

The rest of the post discusses the other "Read More" plugin that I found on the net, its limitations and how I wrote one myself (similar in its functionality to one in the above link) and its source code. Its a good study on how to preprocess your post after you click "PUBLISH POST" button but before blogger actually publishes it. The geeks may continue reading.

Quite some time ago, I was writing a humor article on my blog - Barberic Creatures It badly needed a "Read More" link because as per the the setting I had, I display 10 posts on a page and if one of them is too long (this one was) it dissuades the reader from scrolling all the way down to see the other ones. I couldn't find any plugins at that time (2008/11) and left it at that.

But it came back to haunt me when I wrote Schwartzian Transform Explained. This time I got a hit - Snaphow read more plugin

The instructions were clear and it works. But it didn't address my needs because:

Schwartzian transform used in a C Program

If you are looking for an in depth explanation of the Schwartzian transform with an example, please refer to my earlier post - Schwartzian Transform Explained

I was getting bored and asked a friend to give me something interesting to code in C. He being who he is, told me to write an implementation of the Schwartzian Transform (henceforth referred to as ST) in C. During the process of writing that program, I realized how Perl makes us focus on the problem at hand rather than the nuances of the language. This post aims provide the source and explain it.

The source is uploaded here - schwartz.c

Now that the (fo|sou)rce is with you, I shall explain the main() function because all the other functions are called from main() and their body is easy to understand once you know their purpose in the main(). Or rather, their main() purpose. :)

Schwartzian transform explained

Did you google for "explain schwartzian transform"?

If yes, then you've come to the right place. If no, then you've come to the right place (I wouldn't want to loose out on a prospective reader now, would I?). Please read on - you will find this worth your time.

This post aims to dissect the Schwartzian Transform, and explains it through an example. And while doing this, it shows how cool Perl is.

But what is the Schwartzian Transform? - A way to efficiently sort a list of items.

Why the name? - Legend has it that Randal "Merlyn" Schwartz (wiki page) demonstrated a Perlish version of a Lisp idiom to speed up sorting. And since that day, every time you use the word "Schwartzian Transform", somewhere far beyond the distant seas, the Randal smiles.

How does it speed up sorting? - Patience my child. The blooming of a flower is an analog process, revealing each petal gracefully unlike the open-throw-catch-roll-close process of a frog's tongue. (Yummy?)

Why do I need to know it? - Because it's fun.

Will each line be preceded by a question in the bold font? - Not from now on.

Suppose you have an array of salesman objects - @salesmen, each of which consists of the following attributes:

1. name           # salesman name
2. base_salary    # base salary
3. n_cust_conned  # number of customers conned
4. comm_per_sale  # commission per sale

And the sub routine which calculates the total salary of the salesman is:

sub get_total_sal {
    my ($obj) = @_;
    my $total_salary = $obj->{base_salary} + 
                       $obj->{n_cust_conned} * $obj->{comm_per_sale};
    return $total_salary;
}

If you were told to sort this array based on the total salary, what would be your code snippet which does the sort?

Without prior knowledge of the transform, maybe something like this:

my @sorted_salesmen = sort { $a->get_total_sal() 
                             <=>
                             $b->get_total_sal() } @salesmen;

This would do the job. But what limitation does it induce in your code?

Think about it.

Perl program to simulate tail -f

Created a github account to share my bin with the rest of the world.

Had written a perl program to simulate tail -f quite some time ago. Put it up there. Enjoyed writing it. Hope you feel the same while playing with it.

Here it is: simtail.pl

Bounced back to blogspot

#!/usr/bin/perl
use strict;
use warnings;
print "Because you cannot install plugins on wordpress.com blogs";
print "Which means I cannot install this cool plugin which";
print "lets me indent and syntax highlight my Perl code\n";
exit(0);

Wordpress.com has the <code> tag - which for the average programmer means that I can use this tag to copy paste my code on the blog and it will show up all nice and indented. But from the usage, code tag just "encodes" HTML i.e. you don't need to escape < tag in #include <stdio.h>

The code tag doesn't indent. It doesn't highlight. It just converts the text to code font. The forums tell you to post your code in pre tags. Pre tags work but not always. As per Shuvam's comment below - the quirk might be with the specific template I am working with. So I created a test page on wordpress -

Test page to check code layout on wordpress.com

and tried it out with a 10-15 initial templates. None of them did the trick. I am not going to try out the 100+ templates that they offer and neither would I compromise on the rest of the layout just because some theme shows the pre tags cleanly.

Moved to wordpress

Wanted to play with wordpress for quite some time and also wanted to see how difficult/easy it is to migrate posts from blogspot to wordpress.

Verdict: It don't get any simpler than this. Wordpress has an inbuilt import feature. All you need to do is to point it to the blogspot blog url and it asks whether you are allowing this or not. If you give it a nod, it does what's needed. Clean.

Point and click - Here

A very good link to know more about ELF files

Was searching the net to learn about the structure of ELF files. This was a godsend:

Linux Forums - understanding ELF

Cygwin + Poderosa = bye bye cmd.exe

Does your job require you to boot up your box in Windows instead of Ubuntu? Do you miss the vi-mode on the command line? Do you become nostalgic at the mention of multiple shell tab sessions? Do you feel like bashing your boss's head if he says he won't shell out any extra money to buy you a shiny new Windows power shell (I think it should be worth a try)

Fear not, for the open source world has made efforts to bring you closer to your $HOME.
 
Cygwin - a Linux like environment for Windows gives you all the tools that you missed on cmd.exe - find, grep, ls and many more. You can download Cygwin from Cygwin home

Just remember one thing - when you install it for the first time - do a basic installation - i.e. do not select too many packages. Let the default installation do its job. You can always add new packages by running the setup again (no - it won't erase the previous setup - that's just cygwin's way of adding new packages).