Make Comments

Even though I used make in ECE3574, I never really knew what was going inside of it. We compiled the programming in QT by typing

1) qmake -project

2) qmake

3) make

The three above commands compiled all the C++ and header files, and created an application which was run by ./(application name). After doing the homework for Unix, I finally learned what goes inside of a make file. Make might not be beneficial when compiling one or two files, but it is very helpful in real world problems where it is required to compile 100′s of files. It saves time and time is a big factor in large companies. I initially struggled doing the make file, but the exercise on make assisted a lot and made the whole concept look fairly easy.

I also finished my Tic-Tac-Toe game for ECE 3574. It took a considerable amount of time, but the functionality of the program was working fine. The project required us to save passwords as Sha1 encryption. I initially did not how to encrypt this message, but upon reading QT’s documents, it made the projects seem easy. Regardless, it took a considerable amount of time. In addition to that, we are also learning about process and threads. These concepts are fairly “alien” to me, since I have never used them.

Also, I believe that the format of this class is perfect. The fact that its a point based system  enables us to not work on all the projects, yet still get a good grade. I am just worried, because I haven’t even reached 300 points. These are my updates for this week.

Intro to Unix ECE 2524 Entry #7

It’s been a while since my last post so need to get back on it to be sure I get 10 in before the end of the semester. Haven’t had any assignments in Unix lately which has been a nice break.  I ended up getting the futures paper done to get me some added points to my total grade. I’ve also gotten a fair amount of work done on our battleship final project.  I’ve gotten most of the text based part of the game done and implemented in Qt; this should give us a good base should we have time and choose to implement a GUI.  Error checking is always one of the things I hate most about programming because there’s just so many stupid things that a user could input that you have to check for and handle.  I think I’ve gotten everything covered in terms of inputting bad coordinates and so on. Currently the ships are only one size big, and they need to be re-implemented to the correct size, and error checking needs to be done in terms of placement, i.e. checking for overlap or run off over the screen.

As for 3574, we had our first test this past Thursday which seemed to go pretty well to me.  Most of the information was easily found in the book or online slides from the book, and  I can’t imagine getting a poor grade on this test.  The past homework has been quite difficult, and it took me awhile to get the thing of moving back and forth between different UI forms via signals/slots, and hiding/showing different widgets.  It has however been a lot more fun to write than previous assignments.  Getting used to QtDesigner took me a few minutes, but it is very intuitive, and makes creating GUI’s so much easier than coding them by hand.  As part of the assignment we had to implement tic-tac-toe and write our own algorithm for the computer. I think I have gotten pretty close to if not an unbeatable computer player.  Hopefully the homeworks will remain interesting, but now that I believe we are done with GUI’s, I’m afraid I will lose a lot of my interest.

Third Look at Ubuntu – reposted

I posted this one on the wrong blog too, but I have figured it out for future posts.

Having been using Ubuntu through the first half of the semester, I thought I would revisit my feelings about Linux in general and about Ubuntu specifically. When not doing any assignments I find myself using only Windows 7. I know some people who after this class have adopted Ubuntu as their primary operating system, but I won’t be one of those people. I know the ins and outs of windows and I just feel there is so much in Ubuntu that I don’t know and I won’t ever be able to know it as well as windows.

The idea of an open source “do anything you want/need to do” is great and that is why Linux is so popular. It can adapt to your needs and let you accomplish everything you need to. When using ubuntu for class or homework, I find myself dreading using it. It may just be my computer but it feels extremely slow and hard to use. Some of the basic settings have default settings that I don’t like but don’t know how to change.them.

I started off on the wrong foot with ubuntu trying to run off a flashdrive. I completed our first homework assignments on the multiple commands we had to learn. I didn’t upload it because I was going to check it later before it was due. A while later I booted to Ubuntu again and the files were gone because I had set up the flashdrive wrong. I ended up going with VirtualBox. If I had started learning Ubuntu first instead of Windows I’m sure I would like it more, but I am resistant to change.

Post Midterms Updates

Even though we haven’t had our class for two weeks, the midterm and other projects helped me a lot to learn Ubuntu Operating System, Ubuntu terminal, Python etc. There is still a long way to go, but I am in the process of getting there. In addition, we got our grades for the first exam. I got a better grade that what I anticipated. Looking at the schedule, I see that we are behind in schedule, mostly in projects. I have been experimenting with some new  Unix commands in the terminal. Data/ files manipulation in Ubuntu is so much easier than windows in Unix terminal. Shortcut keys such as cp and mv enables you to make copies and pasted them in directories which can also be created by typing mkdir. I am looking forward to do more assignments in C++.

For ECE 3574, I completed my first GUI project. It was a simple QWidget which had four buttons. Each button had a signal which connected to a corresponding slot and performed associated action. In addition to that, our 2nd GUI project was assigned as well. HW5 demands us to implement a log in page where the information is stored in password.dat file. If the user enters the password correctly, the user is then directed to play  a tic-tac-toe game. Meanwhile, there are other frames which could register a new user, change password, display welcome screen, etc. This project might be tough, but using Unix Philosophies of modularity might simplify this project. These are my updates for this week. Check back next week for more updates.

Midterm Week Updtes

We finally had our first midterm for ECE 2524. Before the day of the exam, I went over the lecture notes and sample exercises for the exam. I felt pretty confident, but the exam did not end up what I was expecting. Downloading the file, working on the file i.e. solving all the questions, and uploading the file back up on scholar took too long. In addition, the instructions/questions or the guidelines for the exam were so long that it took about 20-25 minutes to read and comprehend the questions. I did not think that the exam was tough, but the time spent on others things did not allow me to work on the actual test. As a result, I had to rush through the exam and make guesses through out the exam. There was also a part where the function to record script wasn’t right. It took me three tries to figure out that the syntax was wrong, so I was not able to show my work for question 4. Since this class is a point based system, I believe that I could still get a decent grade if possible an A, if I do well on the other projects.

As far as ECE 3574, we got our first GUI assignment. The project doesn’t seem that difficult, if we use the QT creator, but the instructions specify us not to use it. In addition, we have to use signal and slots to map the corresponds parent elements of the Qwidget to its child elements. I think this might be tricky. Well, these are my updates for this week. Please check back next week for more updates.

Week4/5

The first few Python projects weren’t as difficult as the latest homework(homework 4). Despite having a decent knowledge in Python syntax, homework 4 is taking a fair bit of time. We have to create a mini language in Python. The basic functionality of the program involves getting keywords from the user commands and executing them such as adding  files, removing, sort, search, etc. So far I have implemented two of the functions and it was very tough doing that. Initially, I created a list to store the data. I later realized that this could improved by having a dictionary which stores two values for each node of the list instead of one. This enables us to have the data variable name in one field and the content in the other. After making slight modifications to the existing code, I was able to implement it using both list and a Dictionary. I am planning on finishing this project today or tomorrow based on my workload.

For the first time, I have projects due for Software Engineering and Unix within two days. The project for Unix is due on Wednesday and the project for Software engineering is due on Friday. For the software engineering project, I have to loop across Directories and print out files and associated memory of each files and directories. The Qt libraries have several built in functions that we can use to iterate through the directories and files.

Previously, the sound was not working while I was on the Unix system. After doing some research and installing some drives, it was finally working. I later realized that the sound was not coming from the head phone jack. Presently, I am working on that issues. These are my updates for week 3/4> Please check back next week for a new post.

But What Does It All Mean?

We are at that point in the semester where we are all feeling a bit overwhelmed.  We’re past the introductory material, covered a lot of new concepts and ideas, there’s the threat of a midterm on the horizon, and to make matters worse the same thing is happening in every other class as well. It is not surprising that this is also the part of the semester in which I get questions like “why are we learning this?”, “how do these assignments teach me about Unix?”, or “we’ve covered a lot of commands, a lot of new ideas, done several assignments, what should I focus on?” or “how does it all fit together?” These are good questions to ask, and a good exercise to answer, both for me, so that I can better shape the course to meet its goals, and for the participants in the course to solidify the connections between different material and concepts.  I will attempt to start that process by providing my own thoughts addressing these questions: Learning Unix is as much (if not more) about learning a different way of thinking and problem solving with a computer as it is about learning how to use the terminal and acclimating yourself to the different GUI environments.  And while we’re on the topic of user interfaces, there are several.

Rule of Diversity: Distrust all claims for “one true way”

Unlike Windows or OS X which provides only a single graphical environment to its users and a single graphical toolkit to its developers, Unix systems have multiple options for both environments (e.g. Ubuntu’s Unity, GNOME Shell, KDE) and toolkits (GTK, Qt are the most common).  This confusing jumble isn’t just to make it needlessly annoying for users, in fact it is the result of one of Unixes core philosophies: the users has a better idea of what he/she wants to do than the programmer writing an application.  As a result, many decisions are pushed closer to the user level.  This is sometimes listed as a downside to Unix/Linux, since it increases the perceived complexity of the system to the user, but luckily many distributions have been set up to select a sensible default choice for most people who would rather jump right into using their computer, rather than configuring it.  Ubuntu, with its Unity interface, is a good example of this. But it’s empowering to be aware that you aren’t stuck with Unity, you can install and use any of the other graphical environments as well, such as GNOME, KDE, LXDE, and more. Moving on, but keeping The Rule of Diversity in mind, let’s revisit the examples we looked at in class.  The first was related to the homework in which we were asked to read in lines in a white-space delimited record format containing fields for “First Name”, “Last Name”, “Amount Owed”, and “Phone Number”.  A short example of such a file is

Bill Bradley 25.20 Blacksburg 951-1001
Charles Cassidy 14.52 Radford 261-1002
David Dell 35.00 Blacksburg 231-1003

We were then asked to write a program that would print out information for people in ‘Blacksburg’ in the order of “Phone Number”, “Last Name”, “First Name”, “Amount Owed”. A straight forward way to solve this using Python is with the following code snippet

for line in f:
    fields = line.split()
    if fields[3] == 'Blacksburg':
        record = [fields[4], fields[1], fields[0], fields[2]]
        print ', '.join(record)

In class we looked at an alternative solution using list comprehension:

for fields in (r for r in imap(str.split, f) if r[3] == 'Blacksburg'):
        print ', '.join(fields[i] for i in [4,1,0,2])

Both of these examples can be found on github.  They both do the same thing.   The first takes 5 lines, the second 2.  I made use of a few convenience features to make this happen, the first is the imap function, the iterator version of the map function.  The map function is common under many functional programming languages and implements a common task of applying a function (in this case str.split) to every element in a list (in this case f, the file object). This is an extremely common task in programming, but there is no analog in C but luckily the STL Algorithms library gives us std::transform for C+, though the syntax isn’t nearly as clean as Python’s. So the big question is “If I’ve been implementing this idiom all along, without ‘map’, why change now?”  The answer is that implementing it without map will be guaranteed to use more lines of code, with which we know there is a statistically higher chance of making a mistake.  In addition, the implementation would look a lot like any of the other loops that you have written in the same program and you will find yourself pausing at it to ask yourself “What am I trying to do here?”.  Once you learn the concept of ‘map’, using it is much more concise.  Looking at the call to “map” you know exactly what is going on without having to mentally process a “for” or “while” loop.  This idea is generalized to the concept of list comprehension, which is what we’re doing with the rest of that line.  Working with a list of things is really common in programming, and one of the common things we do with lists is to generate new lists that are some filtered and transformed version of the original.  List comprehension provides a cleaner syntax (similar to the set comprehension that you may be familiar with in mathematics) to transforming lists than the traditional “for” or “while” loop would yield.  And more importantly, once you get familiar with the syntax, it lets you more quickly recognize what is going on.  For example, let’s look at two ways of computing a list of the Pythagorean triples for values 1 through 10

triples1 = []
for x in xrange(1,11):
    for y in xrange(1,11):
        for z in xrange(1,11):
            if x**2 + y**2 == z**2:
                triples1.append((x,y,z))
print triples1

and now, using list comprehension:

triples2 = [ (x,y,z) for x in xrange(1,11)
                     for y in xrange(1,11)
                     for z in xrange(1,11)
                     if x**2 + y**2 == z**2 ]
print triples2

I’ve broken the second example across several lines so that it will all fit on the screen, but it could be left on a single line (see the full, working example) and still be just as readable.  Right off the bat we can look at the second version and tell that `triples2` will be a list of tuples containing three values (x,y,z).  We had to work our way down to five levels of nested blocks to figure that out in the first example.  And while you may not realize it because you’re so used to doing it, our brains have a much more difficult time following what is going on in a nested loop, it implies a specific hierarchy that is misleading for this problem. Let’s shift gears just a bit and look at some of the commands I ran at the end of class.  I first wanted to count all the lines of code in all of the *.py files in my current directory:

cat *.py | wc -l

Then I wanted to revise that and filter out any blank lines:

cat *.py | sed '/^$/d' | wc -l

And let’s also filter out any lines that only contain a comment

cat *.py | sed '/^$/d' | sec '/^#.*$/d' | wc -l

(note, we could have combined the two `sed` commands into one, I separated them to emphasize the idea of using a pipeline to filter data) Next I wanted to know what modules I was importing.

cat *.py | grep '^import'

Say I wanted to isolate just the names, I could use the `cut` command

cat *.py | grep '^import' | cut -d ' ' -f 2

If you didn’t know about the `cut` command you could use sed’s `s` command to do a substitution using regular expressions. I will leave the implementation of this as an exercise for the reader.

We notice that there are few duplicates, let’s only print out unique names

cat *.py | grep '^import' | cut -d ' ' -f 2 | sort | uniq

Exercise for the reader: why is the `sort` necessary?

And finally, let’s count the number of uniq modules I’m using

cat *.py | grep '^import' | cut -d ' ' -f 2 | sort | uniq | wc -l

I could have just shown you the final command and said “this prints the number of modules I’m using” but I wanted to demonstrate the thought process to get there. We started with just a two command pipeline, and then started building up the command one piece at a time. This is a great example of another core Unix philosophy: write simple programs that do one thing and one thing well, and write them with a consistent interface so that they can easy be used together. Now I admit, counting the number of modules uses this way required us to start up 6 processes. Luckily process creation on Unix systems is relatively cheap by design. This had the intended consequence of creating an operating environment in which it made sense to build up complex commands from simpler ones and thereby encouraged the design of simple programs that do one thing and one thing well. We could write a much more efficient program to do this task in C or another compiled language, but the point is, we didn’t have to. As you get more familiar with the simple commands you’ll find that there are many tasks like this you want to do that occur too infrequently for writing a dedicated program, but can be pieced together quickly with a pipeline.

So what the heck do these to different topics: list comprehension and command pipelines, have in common?  And why are we using Python at all? Well, Unix’s strength is that it provides a huge wealth of excellent tools and supports a large number of programming languages.  It does everything an operating system can do to allow you, the developer, to pick the best tool for the job.  As we mentioned before, when we’re developing a program the “best tool” usually means the one that will allow us to solve the problem in the fewest lines possible.  Python’s syntax is much cleaner than that of C or C++, and its support of convenience features like list comprehension allow us to implement algorithms that might normally take several loops in a less expressive language in one, easy to understand line.

This has been a rather long post, I hope you’re still with me.  To summarize, don’t worry too much about memorizing every single command right away, that will come naturally as you use them more often (and a refresher is always just a quick call to `man`).  Instead shift your thinking to a higher level of abstraction and always ask yourself “what tools do I have available to solve this problem” and try to pick the “best” one, whatever “best” means in the context you are in.  Unix/Linux puts you, the user and you, the developer in the drivers seat, it provides you with a wealth of knobs and buttons to press, but does little to tell you which ones it thinks you *should* press.  This can be intimidating, especially coming from a Windows or OS X environment which tends to make most of the choices for the user.  That’s ok, and to be expected.  With practice, you will learn to appreciate your newly discovered flexibility and will start having fun!

I want to know what you think! Share your thoughts on what we’ve gone over  in class, the assignments we’ve done, and the reading we’ve discussed.  How do you see it all fitting together?

Intro to Unix ECE 2524 Entry #4

Not too much happening with Unix this week. No python assignment this week and finished most of project two for ECE 3574 last weekend.  That project was quite annoying seeing that book didn’t make much sense with most of what it wanted you to do. After seeing the TA, I realized how many flaws there were in the books spec. Constructors missing members that they needed, parameters that aren’t specified as to what they are to do, destructors with nothing to destruct, are just a few things among many of the problems with the spec. I guess the only good thing about it is that the program can’t be graded as harshly because the spec is so ambiguous.  Glad to be done with that one though, hopefully the next one is better.

As for ECE 2524, not sure what the next assignment will entail but I’m not looking forward to it after the exercise on Wednesday. I don’t really understand the protocols at all, and I have no idea how to implement them.  I have no idea how to go about creating the project like what was mentioned in the exercise. Hopefully that assignment will make more sense when the spec comes out.

Assignment 3 and so far…

Through the Assignment 3, I learned a few more things about Python including how to use ‘Try and Exception’ and how to make a function. ‘Try and Exception’ is very similar concept of ‘Try and Catch’ in C++ so it was easy to understand the usage of it. For me, using ‘argument parse’ library was hardest part. In fact, I still don’t know how to use it exactly. I think I need more practice to use it.

Now, I might know just few things about Python but I want to say Python is interesting and easy language ( Even I had a conflict over finishing Assignment 3) especially, when a variable is declared, it is interesting that I do not need to select a type of data such as int, double, float and char. (actually, user can declare a type of data after declare a variable.) Of course, there is a thing that I do not like it. That is indentation. I think it is very strict about indentation since there is no use of brackets in Python. Sometimes, it makes me hard to keep tracking my code.