Progress in Python

Programming has always been a weak spot for me. Things that are seemingly obvious for others have not always been for me. Yet, I always do try my best to get my work done.

This class not an exception. For each homework assignment, I have a difficulty 1. understanding where to starts and 2. how to go about implementing parts of the project/homework. And now, after finished homework 2 and 3 with fair amount of difficulty, I am faced with homework 4, and I feel like I have hit a wall.

One thing I do have to say though is that I think python is very useful if I can really get around to learning it. It is very different from C/C++. Also, it does give me a chance to really learn about another language and hopefully be more proficient in programming, not just in python, but in other languages as well. But for now, I have to go back to learning and working on homework 4…

Past and Current Python Homeworks

These past two weeks have mostly been looking at our last and next python assignments. The last homework looked like it was going to be quite difficult. However, after starting the homework, I realized that pretty much everything I needed to know was in the assignment description. The links to other websites with examples helped out a tremendous amount when it came to argparse. If I was not told about argparse, I am sure the homework would have taken twice as long. The next homework that is due seems even more complex. I am just glad for the high level of detail in the description as well as a blog post by the instructor on how to achieve certain actions that the program must do. The main issue I have seen so far in class is the lack of time. It seems like once you just start understanding something, class is over and the next class is on a completely different topic.

What’s New With Ubuntu

I haven’t posted in a while because I have been completely bombed with 4 tests, homework, and a project in the past week. So here’s my post about my dealings with Ubuntu. I have still been trying to get used to the syntax differences from Unix and C++ and C. It has been rather troubling, but I’m still working on it. The basic commands have become rather simple for me, but as we progress through programming, the harder concepts take a little bit more time to comprehend. I admit that what we are learning now is very interesting and I’m looking forward to gaining a better understanding of how to program with these concepts and using them in a wider variety of functions as opposed to just these few assignments.

Posted in Uncategorized

I’m back

I realized today that it's been a little while since I posted here so I decided to correct that.  I've been pretty swamped with tests and projects and finally seem to have a few day break here.

Anyway,
I have continued to use Linux increasingly over the past few weeks and I enjoy it quite a bit.  I am going to my first Linux Users Group meeting tonight, though I already know most of the members form CSec. 

I also heard today that Steam is doing a closed beta for its Linux version next month, it would be good to have it multi platform in the future as long as it works pretty well. 

Well I've gotta work but I'll try to get back to this more often.

Posted in Uncategorized

I’m back

I realized today that it's been a little while since I posted here so I decided to correct that.  I've been pretty swamped with tests and projects and finally seem to have a few day break here.

Anyway,
I have continued to use Linux increasingly over the past few weeks and I enjoy it quite a bit.  I am going to my first Linux Users Group meeting tonight, though I already know most of the members form CSec. 

I also heard today that Steam is doing a closed beta for its Linux version next month, it would be good to have it multi platform in the future as long as it works pretty well. 

Well I've gotta work but I'll try to get back to this more often.

Posted in Uncategorized

Python

I’ve always heard of python but have never actually used it until this semester.  It seems like a good program to introduce students to because it seems more user friendly then most programs.  As i’m using python, I realize how it could have helped students if this was their introductory program to learn instead of C++.  It is a little bit easier to use then C++ and reading python syntax and code seems more human nature then reading regular code (because of python’s convenient variable names and syntax).

Posted in Uncategorized

Post #3

Hmm i swear i had posted my #3 a few days ago, but after looking back today i don’t see it.. hmm thats annoying. Mainly what i talked about in my last post was how i had been behind on my post. A lot of things have happened since my last post, with several homeworks being done. The one thing i found difficult to use at first was simply submitting my homework through github. I felt the directions from github weren’t exactly clear but i was able to get through it eventually. Another thing in this class is that i still have to adapt to the linux environment. Its just very foreign to me, and having to do lots of things through the terminal, im just kinda slow at it. So im going to make it a goal from now on to start using the virtual machine when i can, as well as keep up to date with blogging. So until then..

But What Does It All Mean?

We are at that point in the semester where we are all feeling a bit overwhelmed.  We’re past the introductory material, covered a lot of new concepts and ideas, there’s the threat of a midterm on the horizon, and to make matters worse the same thing is happening in every other class as well. It is not surprising that this is also the part of the semester in which I get questions like “why are we learning this?”, “how do these assignments teach me about Unix?”, or “we’ve covered a lot of commands, a lot of new ideas, done several assignments, what should I focus on?” or “how does it all fit together?” These are good questions to ask, and a good exercise to answer, both for me, so that I can better shape the course to meet its goals, and for the participants in the course to solidify the connections between different material and concepts.  I will attempt to start that process by providing my own thoughts addressing these questions: Learning Unix is as much (if not more) about learning a different way of thinking and problem solving with a computer as it is about learning how to use the terminal and acclimating yourself to the different GUI environments.  And while we’re on the topic of user interfaces, there are several.

Rule of Diversity: Distrust all claims for “one true way”

Unlike Windows or OS X which provides only a single graphical environment to its users and a single graphical toolkit to its developers, Unix systems have multiple options for both environments (e.g. Ubuntu’s Unity, GNOME Shell, KDE) and toolkits (GTK, Qt are the most common).  This confusing jumble isn’t just to make it needlessly annoying for users, in fact it is the result of one of Unixes core philosophies: the users has a better idea of what he/she wants to do than the programmer writing an application.  As a result, many decisions are pushed closer to the user level.  This is sometimes listed as a downside to Unix/Linux, since it increases the perceived complexity of the system to the user, but luckily many distributions have been set up to select a sensible default choice for most people who would rather jump right into using their computer, rather than configuring it.  Ubuntu, with its Unity interface, is a good example of this. But it’s empowering to be aware that you aren’t stuck with Unity, you can install and use any of the other graphical environments as well, such as GNOME, KDE, LXDE, and more. Moving on, but keeping The Rule of Diversity in mind, let’s revisit the examples we looked at in class.  The first was related to the homework in which we were asked to read in lines in a white-space delimited record format containing fields for “First Name”, “Last Name”, “Amount Owed”, and “Phone Number”.  A short example of such a file is

Bill Bradley 25.20 Blacksburg 951-1001
Charles Cassidy 14.52 Radford 261-1002
David Dell 35.00 Blacksburg 231-1003

We were then asked to write a program that would print out information for people in ‘Blacksburg’ in the order of “Phone Number”, “Last Name”, “First Name”, “Amount Owed”. A straight forward way to solve this using Python is with the following code snippet

for line in f:
    fields = line.split()
    if fields[3] == 'Blacksburg':
        record = [fields[4], fields[1], fields[0], fields[2]]
        print ', '.join(record)

In class we looked at an alternative solution using list comprehension:

for fields in (r for r in imap(str.split, f) if r[3] == 'Blacksburg'):
        print ', '.join(fields[i] for i in [4,1,0,2])

Both of these examples can be found on github.  They both do the same thing.   The first takes 5 lines, the second 2.  I made use of a few convenience features to make this happen, the first is the imap function, the iterator version of the map function.  The map function is common under many functional programming languages and implements a common task of applying a function (in this case str.split) to every element in a list (in this case f, the file object). This is an extremely common task in programming, but there is no analog in C but luckily the STL Algorithms library gives us std::transform for C+, though the syntax isn’t nearly as clean as Python’s. So the big question is “If I’ve been implementing this idiom all along, without ‘map’, why change now?”  The answer is that implementing it without map will be guaranteed to use more lines of code, with which we know there is a statistically higher chance of making a mistake.  In addition, the implementation would look a lot like any of the other loops that you have written in the same program and you will find yourself pausing at it to ask yourself “What am I trying to do here?”.  Once you learn the concept of ‘map’, using it is much more concise.  Looking at the call to “map” you know exactly what is going on without having to mentally process a “for” or “while” loop.  This idea is generalized to the concept of list comprehension, which is what we’re doing with the rest of that line.  Working with a list of things is really common in programming, and one of the common things we do with lists is to generate new lists that are some filtered and transformed version of the original.  List comprehension provides a cleaner syntax (similar to the set comprehension that you may be familiar with in mathematics) to transforming lists than the traditional “for” or “while” loop would yield.  And more importantly, once you get familiar with the syntax, it lets you more quickly recognize what is going on.  For example, let’s look at two ways of computing a list of the Pythagorean triples for values 1 through 10

triples1 = []
for x in xrange(1,11):
    for y in xrange(1,11):
        for z in xrange(1,11):
            if x**2 + y**2 == z**2:
                triples1.append((x,y,z))
print triples1

and now, using list comprehension:

triples2 = [ (x,y,z) for x in xrange(1,11)
                     for y in xrange(1,11)
                     for z in xrange(1,11)
                     if x**2 + y**2 == z**2 ]
print triples2

I’ve broken the second example across several lines so that it will all fit on the screen, but it could be left on a single line (see the full, working example) and still be just as readable.  Right off the bat we can look at the second version and tell that `triples2` will be a list of tuples containing three values (x,y,z).  We had to work our way down to five levels of nested blocks to figure that out in the first example.  And while you may not realize it because you’re so used to doing it, our brains have a much more difficult time following what is going on in a nested loop, it implies a specific hierarchy that is misleading for this problem. Let’s shift gears just a bit and look at some of the commands I ran at the end of class.  I first wanted to count all the lines of code in all of the *.py files in my current directory:

cat *.py | wc -l

Then I wanted to revise that and filter out any blank lines:

cat *.py | sed '/^$/d' | wc -l

And let’s also filter out any lines that only contain a comment

cat *.py | sed '/^$/d' | sec '/^#.*$/d' | wc -l

(note, we could have combined the two `sed` commands into one, I separated them to emphasize the idea of using a pipeline to filter data) Next I wanted to know what modules I was importing.

cat *.py | grep '^import'

Say I wanted to isolate just the names, I could use the `cut` command

cat *.py | grep '^import' | cut -d ' ' -f 2

If you didn’t know about the `cut` command you could use sed’s `s` command to do a substitution using regular expressions. I will leave the implementation of this as an exercise for the reader.

We notice that there are few duplicates, let’s only print out unique names

cat *.py | grep '^import' | cut -d ' ' -f 2 | sort | uniq

Exercise for the reader: why is the `sort` necessary?

And finally, let’s count the number of uniq modules I’m using

cat *.py | grep '^import' | cut -d ' ' -f 2 | sort | uniq | wc -l

I could have just shown you the final command and said “this prints the number of modules I’m using” but I wanted to demonstrate the thought process to get there. We started with just a two command pipeline, and then started building up the command one piece at a time. This is a great example of another core Unix philosophy: write simple programs that do one thing and one thing well, and write them with a consistent interface so that they can easy be used together. Now I admit, counting the number of modules uses this way required us to start up 6 processes. Luckily process creation on Unix systems is relatively cheap by design. This had the intended consequence of creating an operating environment in which it made sense to build up complex commands from simpler ones and thereby encouraged the design of simple programs that do one thing and one thing well. We could write a much more efficient program to do this task in C or another compiled language, but the point is, we didn’t have to. As you get more familiar with the simple commands you’ll find that there are many tasks like this you want to do that occur too infrequently for writing a dedicated program, but can be pieced together quickly with a pipeline.

So what the heck do these to different topics: list comprehension and command pipelines, have in common?  And why are we using Python at all? Well, Unix’s strength is that it provides a huge wealth of excellent tools and supports a large number of programming languages.  It does everything an operating system can do to allow you, the developer, to pick the best tool for the job.  As we mentioned before, when we’re developing a program the “best tool” usually means the one that will allow us to solve the problem in the fewest lines possible.  Python’s syntax is much cleaner than that of C or C++, and its support of convenience features like list comprehension allow us to implement algorithms that might normally take several loops in a less expressive language in one, easy to understand line.

This has been a rather long post, I hope you’re still with me.  To summarize, don’t worry too much about memorizing every single command right away, that will come naturally as you use them more often (and a refresher is always just a quick call to `man`).  Instead shift your thinking to a higher level of abstraction and always ask yourself “what tools do I have available to solve this problem” and try to pick the “best” one, whatever “best” means in the context you are in.  Unix/Linux puts you, the user and you, the developer in the drivers seat, it provides you with a wealth of knobs and buttons to press, but does little to tell you which ones it thinks you *should* press.  This can be intimidating, especially coming from a Windows or OS X environment which tends to make most of the choices for the user.  That’s ok, and to be expected.  With practice, you will learn to appreciate your newly discovered flexibility and will start having fun!

I want to know what you think! Share your thoughts on what we’ve gone over  in class, the assignments we’ve done, and the reading we’ve discussed.  How do you see it all fitting together?

Week 4/5

Last week was an interesting week.  I was actually looking forward to class.  So, as usual, I packed my uniform and brought it to PT so that I could shower in McComas Hall and get to class on time.  However, disaster struck when I got out of the shower, and realized I had forgotten 2 critical pieces of my uniform in my room, my brown undershirt and my cover (wearing BDUs).  So I weighed my options:  break the regs and go in sweaty pt gear, or follow the regs, and end up missing class.  For better or worse, I chose to follow the regs which had the unfortunate result of skipping class.

Despite this,  I have done my best to keep up with the readings while fighting with the band for time to do homework, and my body which randomly decided that this would be an excellent week to get sick (I have 3 midterms, 2 Wednesday and 1 Friday).  On top of this, I ended up missing all of my classes today (I was supposed to only miss Air Force PT) due to an incentive flight in a C-130.  The original aircraft had an engine failure on landing, so we had to wait for another to come from west virginia.  It was an amazing experience despite the wait.

On the Unix/Python side:  Python seems to be making pretty good sense to me.  I am not using my Debian partition of my harddrive enough though.  Its hard to switch over just because of how much stuff I always keep open (right now, 2 web browsers, email, skype, itunes, 6 word documents, and an excel spreadsheet).  This is after I closed everything that ceased to be relevant…  It also doesn’t help that I just use cygwin and packages from cygwin in order to do all my python and unix homework.  I need to start forcing myself to switch over in order to do class work.

 

Ubuntu Ubuntu Ubuntu

Its been awhile since I have blogged about my Unix experience so this is going to be somewhat of a lengthy post.

So far, I’ve learned a lot about Ubuntu and it’s terminal commands.
To open up the terminal, Ctrl + Alt + t
To list what is under the directory, within the terminal type: ls
To change directory within the terminal: cd filename
To make a new directory: mkdir filename
also
To make a new file: touch filename

For some reason, I’ve found coding on Ubuntu to be more convenient compared to coding on the Windows platform.
Since most of my classes require us to use Ubuntu, I’m slowly adapting to it.

I’ve been going to the Virginia Tech Unix/Linux User Groups meetings (VTLuug)!!! >_< er-mah-gawd those dudes are sooo smart!!!! Most if not all the things they discuss in the meeting is over my head. I realize that there is soo much more to learn about Unix. Hopefully someday, I'll become a guru of Unix.

As for the Intro the Unix class, we started coding using Python. Python is scripting language that is very bleh.
I personally don't like it because it feels soo linear. I feel like you can't do as much with it compared to C++ or Java. >_< The most recent homework, I was forced to hard code everything mostly because I did not understand the full usage of “argparse”.

I guess that wraps up my blog for now.
Peace out!
-Michael

Posted in Uncategorized